Try   HackMD

Fatiando Development Calls 2024

📱 Join the video call: https://meet.jit.si/fatiando-a-terra

📅 Shared calendar: https://www.fatiando.org/calendar

IMPORTANT: Everyone is required to follow our Code of Conduct when participating in the Fatiando community. Please review it carefully.

Development calls are open to everyone! Here we discuss project development, socialize, and do some live coding from time to time.

Everyone is encouraged to participate and edit the notes below.

2024-12-10

Time: 17:00 UTC

Participants:

  • Santi
  • Matt
  • Arthur

Notes

  • Issue that Matt opened Verde:
    • https://github.com/fatiando/verde/issues/474
    • Would be nice to include such class
    • No need to add optuna as dependency, the class doesn't need it.
    • For testing we could either use optuna to define a Trial (to test the call method), or mock our own Trial class that returns a single value of damping.
    • Would be nice to allow users to pass the gridder they want to use, instead of hard coding the Spline in the call method.
  • EquivalentSourcesCV:
    • https://github.com/fatiando/harmonica/issues/543
    • Would be nice to have this as well!
    • Santi doesn't quite like having to code a new CV class for every gridder (even in Verde), but generalizing CVs for eqs seems hard (due to the different interfaces that Eqs classes have). So let's start simple with EquivalentSourcesCV.
  • Security breach in ultralytics (not affecting any Fatiando package)
  • Equivalent sources, depth_ attribute. We could extend the description to specify what happens when we pass a float to the depth argument.
  • Set next meeting for Jan 7, since we have holidays in two weeks

2024-11-26

Time: 17:00 UTC

Participants:

  • Leo
  • Matt
  • Gelson
  • Santi

Notes

Pooch

  • We have a few PRs that add support for other sources:
    • NIST data base (doi-based)
    • Network shared file systems
      • Seems ok as long as there's a test. It's basically just file copying with our cache system added.
  • Broad decisions on supporting new sources?
    • minimum requirement: documented API
    • Would be nice to move these doi downloaders to plugin libraries.
    • Need to open an issue to discuss the plug in part.
  • Ditch pylint, maybe replace it with ruff. The lazy solution is copying flake8 config from our other projects.

Harmonica

  • Reader for gxf files
    • Should it always return an xarray?
    • Do we need more than one public function for them?
    • Single function that returns an dataarray should be ok.
    • Use x and y for rotated grids, but easting and northing for non-rotated ones.
  • Using Harmonica in class at USP:
    • Some annoying bits while computing the derivatives.
      • Having to add and subtract padding. https://github.com/fatiando/harmonica/issues/390
        • Would be nice to include padding in the functions.
        • By default would be nice to use linear ramp to the mean value of the grid.
      • Having to remove the vertical coordinates (or any non-dimensional coordinates), and then add them back.
        • It would make sense if the functions take non-dimensional coordinates out and restore them back before returning the inverted-FFT grid.

Verde

Choclo v0.3.1 release:

  • Make 2nd order and 3rd order kernels to return nan
    • This is the proper behaviour, returning zero is not right. So a patch release is ok.
  • Make docstrings smaller by collapsing similar arguments into the same line (easting, northing, upward).

Improve docstrings

Calls

  • Calls have been low traffic and the earlier time slot doesnt seem to work for most people. How about we keep only this one and do these every 2 weeks instead of weekly?
    • Ask what people think in the chat?

2024-11-12

Time: 17:00 UTC

Participants:

  • Santi
  • Thomas
  • Gelson

Notes

  • PR opened by Thomas to add a reader function for gfx files (Geosoft): https://github.com/fatiando/harmonica/pull/539
    • Make sure the files we upload for testing has a compatible license with BSD 3-Clause.
    • Add attribution to the gist.

2024-10-29

Time: 17:00 UTC

Participants:

  • Santi
  • Gelson
  • Leo
  • India

Notes

2024-10-08

Time: 14:00 UTC

Participants:

  • Santi
  • Mariana
  • Gelson

Notes

  • Mariana was required to make a JupyterBook in her current job, and her experience with the fatiando tutorials helped her a lot!
    • The job: analyzing data of how the water permits are distributed across the country between companies and how those change in different locations.
    • The position is at Universidad Iberoamericana.
  • Choclo updates:
  • We merged a few PRs in Boule and Mark suggested us to make a release soon so he can use the latest changes in SHTOOLS.

2024-10-01

Time: 17:00 UTC

Participants:

  • Santi
  • Gelson
  • Leo
  • India

Notes

  • Santi's draft PR for magnetic gradiometry kernels in Choclo: https://github.com/fatiando/choclo/pull/97
    • Implementation of the kernels and forward functions are done.
    • Working on more tests (specially on singular points of the kernels).
  • India started working on magnetic equivalent sources!
    • Should we able able to fit the norm of B?
      • It would be nice to have if it's not too much work.
      • But we could add it later once the first draft is finished.
    • Stack the jacobian matrices if we want to use the least squares function in verde
    • We could sum the jacobians and the hessians instead (that's how we were doing it in the old fatiando). But this won't allow us to use the least squares function in verde.
    • The predict method should not return tfa (we have a function for it). Return the three components for now. If we want to have some fields argument, we could add it later.
  • Leo used the Legendre functions in class and they worked great
  • Inverse problems solvers in SimPEG and Fatiando

2024-09-24

Time: 14:00 UTC

Participants:

  • Santi
  • Gelson
  • India

Notes

  • Santi started working on forward functions and kernels for magnetic gradiometry components due to prisms (in Choclo).

    • These forward functions would compute the spatial derivatives of the magnetic field components due to rectangular prisms

  • Takahashi's repository for fast equivalent sources using Toeplitz matrices: https://github.com/pinga-lab/eq_layer_mag_toeplitz/

2024-09-17

Time: 17:00 UTC

Participants:

  • Gelson
  • Santi

Notes

  • Idea to implement equivalent sources in grid that can predict on a co-registered grid of observation points
    • We can take advantage of the symmetry of the sources and observation points locations:
      • This would allow us to speed up the computations and significantly reduce the memory needed to fit the sources' coefficients.
    • Possible applications:
      • Process magnetic microscopy data.
      • Alternatives to the FFT processing methods on grids (RTP, etc) with comparable computation times and memory requirements.
    • We can apply Takahashi's method (doi: 10.1190/geo2019-0826.1:
      • The jacobian matrices are Toeplitz matrices (elements in the diagonals are all equal), so we can avoid building the whole matrix (only first row and firt column are enough).
      • The matrix vector dot product can be computed using FFTs.
    • Considerations:
      • The data used to fit the sources should be in a regular grid
      • The sources should be in a regular grid (co-registered with the observation points) at constant depth.
      • Once the sources are fit, we can used them to predict on the same observation points (we can reuse the Toeplitz jacobian matrix), or on any observation point (through direct forwards, which might take time, but don't require huge memory).
    • Disadvantages:
      • This only works if the data is already in a regular grid. This is not a replacement of the gradient-boosted equivalent sources.
    • Other relevant papers:
    • Design ideas:
      • The new class should not be a child of the equivalent sources one we already have
      • The only parameters the class should take are depth, damping, inclination and declination of the sources (no points!)
        • All sources will have the same inclination and declination
        • The coefficients theclass will fit are going to be the magnitude of the magnetic moment
          • do we need to use a positivity constrain here? Or shall we allow the moments to be in opposite direction?
      • The depth of the sources will be constant, so depth will be a float
      • The fit method should take an xarray.DataArray with the easting and northing coordinates and the data we want to fit in a grid.
        • We'll have to sort out how to specify which fields they can predict (the whole B, the bz only, the tfa, etc)
        • For now, we can aim to fit the bz field
      • For the predict method:
        • Simple: make a direct forward
        • Optionally: we can store the jacobian matrix (it's only two vectors), but we would need to make sure that the data grid is the same as the one where we want to predict.

2024-09-03

Time: 14:00 UTC

Participants:

  • Santi
  • Gelson
  • Leo
  • Yago

Notes:

2024-08-27

Time: 14:00 UTC

Participants:

  • Santi

Notes:

  • Noticed that the package build available in conda-forge is outdated, python-build should be used instead.
    • The latest version of build is not compatible with Python 3.12 and is causing problems while trying to create environments.

2024-08-20

Time: 17:00 UTC

Participants:

  • India
  • Gelson
  • Mariana
  • Agustina
  • Santi

Notes:

  • Harmonica v0.7.0 has been released!
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
  • LATINMAG 2025 in Mexico
    • Gelson and Leo will attend
  • Agustina's adventure above the Artic circle

2024-07-16

Time: 14:00 UTC

Participants:

  • Santi
  • Leo

Notes:

  • Santi at SciPy2024
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
    • Awesome conference! Totally recommended!
    • Scientific Python: https://www.scientific-python.org
      • Developer guide: https://learn.scientific-python.org/development
        • Recommending using hatchery as the build engine (instead of setuptools)
      • SPECs: https://scientific-python.org/specs/
      • There are discussions on including tips to enhance the security of scientific packages. Santi proposed to create a thread model and then list tasks that would mitigate those risks: from direct attack by state/independent actors to bus factor (only one person having access to the credentials for services like PyPI or ownership of the GitHub community, etc.)
    • Ruff included rules to warn ppl about Numpy deprecated features that were removed in Numpy 2.0
      • These rules were created in collaboration between Ruff devs and Numpy community
      • A reason to consider moving away from flake8 into ruff
    • ipython sphinx directive being under development
      • Might be an alternative to the jupyter-execute one
    • Numba devs are working on PIXIE: a package to compile numba code ahead of time.
    • Xarray will have a new class: DataTree
    • Curious things:
      • Using mpi4py to create non-shared-memory parallelizations. Parallelizing Numba code (with parallel=False) across cluster nodes.
      • Idea to optimize our Numba codes. We can loop over threads in parallel instead of over the whole range of number of observation points. That way, we can assign "chunks" of the observation points to each thread, so they can read and write arrays sequentially, which might speed up the computations.
  • Gelson is implementig magnetic forward modelling of tetrahedrons. Coming soon to Choclo
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →

2024-07-09

Time: 17:00 UTC

Participants:

No meeting

  • Holiday in São Paulo
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
  • Santi in Scipy
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
  • Agus in Groelandia!
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →

2024-07-02

Time: 14:00 UTC

Participants:

  • India
  • Lu
  • Santi
  • Mariana
  • Leo

Notes:

2024-06-25

Time: 17:00 UTC

Participants:

  • India
  • Mariana
  • Santi
  • Leo

Notes:

  • Santi started working on moving Pooch to src layout: https://github.com/fatiando/pooch/pull/427
  • Pooch talk in SciPy
    • Split the talk for different audiences.
    • Record the live demos just in case wifi doesn't work.
    • Main features
      • Unpack
      • Download from DOI
      • Don't focus on custom downloaders and postprocessors, but maybe mention them.
    • History of Pooch
      • Ryan May, John Leeman.
      • Scipy 2018?
      • Merging MetPy and Verde downloaders into a new package.
      • Started to being used by other packages.
    • Future roadmap
      • Improve registry file: just use a JSON file.
      • Merge the two dictionaries with filename, url and filename, hash
      • Improve logging.
      • Plugin system, so custom downloaders are easy to develop and share without the need to push them upstream.
    • fspec and intake
      • Expect questions on "what are the differences, why not use them instead". Pooch is just a simple tool.

2024-06-18

Time: 14:00 UTC

Participants:

  • Santi
  • Leo
  • Mariana

Notes:

2024-06-11

Time: 17:00 UTC

Participants:

  • Santi
  • India
  • Jose Maria

Notes:

2024-06-04

Time: 14:00 UTC

Participants:

  • Leo
  • India
  • Lu
  • Yago
  • Gelson
  • Arthur

Notes:

  • India finish the default window size PR!
  • Idea: the whole CI and development workflow is hard to document and hard to get started. Maybe it would be worth running a yearly development bootcamp where we train people on this. Could work in break out rooms individually. 2h would be enough: 1h for intro and 1h to work on things in breakout rooms. Topics: github workflow, running checks, CI, tests, building docs, etc.
  • Idea from TRANSFORM:
    • People like video tutorials
    • Do a joint tutorial with SimPEG
    • Use Harmonica et al to process gravity or mag data
    • Hand the grid to simpeg for a simple inversion
    • Gelson: shared a tutorial that had very short (a few mins) videos for each section of the notebook. That makes it easy to update the videos. Example notebook: https://github.com/FORCaist/forcsensei/blob/master/FORCsensei.ipynb
    • This format would be great with the tutorials that Mariana is working on.
    • So format doesn't really fit Transform but it's better in the long term
    • Think about what we'd want from an offline event

2024-05-28

Time: 17:00 UTC

Participants:

  • Mariana

Posted Santi's video about gravity disturbance

Leo: Sorry for leaving you alone Mariana!

2024-05-21

Time: 14:00 UTC

Participants:

  • Santi
  • Mariana
  • Leo
  • Lu
  • India

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Lu's amazing talk!
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →

Link to the recorded talk. Mariana edited her unnecessary mumbling in the beginning and the end :P : https://www.youtube.com/live/XNJ1WRfaSkk

2024-05-14

Time: 17:00 UTC

Participants:

  • Santi
  • Mariana
  • Leo

Quick updates:

  • Next meeting we'll have Lu's talk about his adventures in Antarctica
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
    !
    • We should promote it on Mattermost, social media, etc
    • Live streaming? Or just record it and then upload?
    • Ask Lu if we can live stream it
    • We can live stream it directly from Jitsi
      • Make a practice run before the talk
      • Need Youtube API and start the stream on Youtube!
    • We can schedule the live stream ahead of time. This way we can share the link in social media, add it to the calendar, etc
  • Santi's maintenance works:
    • Replace Sphinx napoleon for numpydoc in multiple libraries: https://github.com/fatiando/community/issues/149
      • These changes would improve how we show the API docs of classes (solving issues with attributes that have a trailing underscode, like coefs_).
      • Check if we can also list methods as a toc on the right menu.
    • Solved issues with tests failing in MacOS runners using the oldest dependencies: https://github.com/fatiando/harmonica/pull/499
      • New macos GitHub runners make use of Apple silicon (M1, M2 chips). Numpy for Python 3.8 is not available for those chips.
      • Solved the issue by running the Python 3.8 tests on the latest x86 (intel chips) macos runners.
    • Codecov failing when trying to upload multiple files (specially when PR is opened from a fork)
  • Leo's nerdy week: 
class IGRF(date, ellipsoid=boule.WGS84):
    ...

IGRF(datetime(1990, 3, 14)).predict((lon, lat, height), field="b")
IGRF(datetime(1990, 3, 14)).grid(grid_coords, field="b")

sh = MagneticSphericalHarmonics(g, h, ref_radius=..., normalization="schmidt", ellipsoid=None)
sh.predict(..., field="b")
sh.grid(..., field="b")

sh = MagneticSphericalHarmonics(max_degree, ref_radius=..., normalization="schmidt", ellipsoid=None)
sh.fit(
    be=((lon, lat, height), be), 
    bn=((lon, lat, height), bn), 
    bu=((lon, lat, height), bu), 
)

sh = GravitySphericalHarmonics(max_degree, ref_radius=..., normalization="full", ellipsoid=None)
sh.fit(
    gz=((lon, lat, height), gz), 
    gxx=((lon, lat, height), gxx), 
)
sh.predict((lon, lat, height), field="gz")

2024-05-07

Time: 14:00 UTC

Participants:

  • India
  • Santi

Quick updates:

  • Harmonica CI's failing on MacOS with Python 3.8
    • Not being able to install Numpy with pip
    • Trying to fix it: #496
  • Pair programming on India's PR for changing default window_size in gradient-boosted eqs:
    • https://github.com/fatiando/harmonica/pull/487
    • If the passed coordinates have less than 5e3 points, then a warning is raised and a single window will be used.
    • The self.window_size_ is set to None in that case, since it's not being used for defining the window.
    • Few ideas to improve this: use slice instead of np.arange in the intermediate returns.

2024-04-30

Time: 17:00 UTC

Participants:

  • Mariana
  • Santi (only during the end)

Quick updates:

  • Santi talk is available in Zoom, can I download it and publishing in Fatiando Youtube Channel? Slides and notebooks: https://github.com/santisoler/2024-uh
    • Yes! I got permission to publish it. Maybe check if there are bits that should be removed?
  • Today's is Fatiando birthday. Santi check the git log to understand better the transition. Before April 30th 2010, Fatiando was written in cpp (Pregnancy stage)
    • Post on LinkedIn
      Image Not Showing Possible Reasons
      • The image file may be corrupted
      • The server hosting the image is unavailable
      • The image path is incorrect
      • The image format is not supported
      Learn More →
    • Post on Mattermost
      Image Not Showing Possible Reasons
      • The image file may be corrupted
      • The server hosting the image is unavailable
      • The image path is incorrect
      • The image format is not supported
      Learn More →
    • Post on Mastodon
    • Post on Twitter

2024-04-23

Time: 14:00 UTC

Participants:

  • Lu
  • Leo
  • India
  • Santi
  • Mariana

Quick updates:

2024-04-16

Time: 17:00 UTC

Participants:

  • Santi
  • Leo Miquelutti
  • Mariana
  • India
  • Martin

Quick updates:

  • Codedov push failing in https://github.com/fatiando/harmonica/pull/486
    • Reran the test to see if it pass nows
  • Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
    Choclo v0.2.0 has been released!
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
  • Roadmap to Harmonica v0.7.0:
  • Martin
    • Progress on the xml reader grd files
    • Able to read the wellknown_epsg attribute on the projection tag
    • Add a private function to read the xml file
    • We talked about the signature of that function: it's better if it returns a dictionary witht the "EPSG" value inside. Then the attrs dictionary can be extended before passing it to the xarray object.
      • This would allow us to extend the reader of xml without having to change other code.
  • Chat on #487
    • We need to add tests checking if the window size is set properly
    • Maybe defining a grid with N times 5_000 points and comparing the value of window size it gets assigned against the expected one. We should use numpy.testing.assert_allclose() for that since we are comparing floats.
  • Plan for Lu's talk on one of the 14:00 GMT meetings!
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →

2024-04-09

Time: 14:00 UTC

Participants:

  • Lu
  • Mariana
  • Santi
  • Leo

Quick updates:

  • Mariana Questions
  • Many new PRs opened in Boule by Mark (SHTOOLS)
  • New contributor in Verde working on grid with nans
  • Question regarding converting mineralogy to physical properties.
  • Verde:
    • Issue with pkg_resources, it doesn't work in Python 3.12
    • Warning coming from Pandas: https://github.com/fatiando/harmonica/actions/runs/8607607182/job/23588411532#step:12:1135 
      ​​​​​​​​tests/test_gradient_boosted_eqs.py: 8 warnings
​​​​​​​​  /opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/verde/blockreduce.py:179: FutureWarning: The provided callable <function median at 0x7fb9141e1b20> is currently using DataFrameGroupBy.median. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "median" instead.
​​​​​​​​    blocked = pd.DataFrame(columns).groupby("block").aggregate(reduction)

2024-04-02

Time: 17:00 UTC

Participants:

  • Santi
  • India
  • Mariana
  • Leo
  • Martin

Quick updates:

  • India's workflow:
    • JupyterHub (because here she has configured ssh key with Github) → terminal → git clone repo → mamba create env → conda activate harmonica → git grep EquivalentSourcesGB (to search for which file needs to change) → edit file gradient_boosted.py →
    • To get a nice bash prompt with little effort: https://starship.rs/
  • Mariana's idea: use breakout rooms for tackling multiple issues at the same time

Todo:

  • Update meeting times:
    • Update fatiando calendar (santi)
    • Post on Mattermost (santi)
    • Post on forum (mariana)
    • Post on linkedin (mariana)

2024-03-26

Time: 17:00 UTC

Participants:

  • Mariana
  • India
  • Lu
  • Santi

Quick updates:

  • Question regarding Zenodo citations: fatiando/community@145
  • Done some maintenance work on Choclo and Harmonica
  • I think Leo Miquellutti's PR on total gradient amplitude might be ready to be merged: https://github.com/fatiando/harmonica/pull/478
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
  • Lu came back from Antarctica!
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
    • Ice drilling and sampling every 5 cm (on 200m hole!)
    • Grav, mag, MT surveys
    • He's willing to give a presentation about his work there
  • We should have another meeting time that's more suitable for Lu
  • Discussion around some Harmonica issues regarding equivalent sources
    • Better default for the depth: we could make it clever enought to set the depth argument as 4.5 times the average distance between the data points. Shall we still allow users to set the depth manually? Should we include a depth_factor argument that defaults to 4.5? Should depth=None by default and in that case the class should compute it with that formula?
    • Better default for window size: set it allowing to have in average 5k data points per window. Maybe we should include the 5_000 as an argument of the class instead of hard coding it.

2024-03-19

Time: 17:00 UTC

Participants:

  • Mariana
  • India
  • Santi
  • Leo

Quick updates:

  • Mariana. New post in LinkedIn promoting community meetings

    WARNING: No ICDs were found. Either, Install a conda package providing a OpenCL implementation (pocl, oclgrind, intel-compute-runtime, beignet) or Make your system-wide implementation visible by installing ocl-icd-system conda package.

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Group programming
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →

  • We'll continue our work on Harmonica ssue #423
  • Any volunteers to pick up where Santi left off?
  • For Mariana to pull changes from Santi's fork:
    • git remote add santisoler git@github.com:santisoler/harmonica.git
    • git fetch santisoler
    • git switch remove-depth-type
    • git grep depth_type

2024-03-12

Time: 17:00 UTC

Participants:

  • Santiago
  • Mariana
  • Martin
  • India
  • Leo

Quick updates:

  • Mariana
    • Update svg figure on Inkscape :). Approved by Leo https://github.com/fatiando/tutorials/issues/14
    • Question. Ideas to fix my communist gravity figure?
      • Minor thing: pass projection=projection to eqs.grid
    • Gave a talk at the Uni in Bristol about her PhD and at the end invited people to Fatiando
      • We should add it to the Fatiando news in the website!
  • Santi submitted a talk about Pooch to Scipy. Waiting to see if they'll accept.

2024-03-05

Time: 17:00 UTC

Participants:

  • Mariana
  • India
  • Gelson
  • Santi
  • Leo
  • Agustina

Quick announcements

  • Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
    We are changing time of the meetings to every Tuesday 17:00 GMT

Agenda

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Group programming
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →

  • The goal is to tackle one of the Harmonica Issues (#423 was chosen last week) in group so we can all learn from the process and the train of thought involved in contributing to Fatiando.
  • One could take the lead and type, while the rest chime in with ideas and questions.
  • The idea is not to make it a live stream where the protagonist is the typer, but to make it as interactive as possible. Other people can take the lead and start typing themselves.
  • Ultimate goals:
    • Allow new contributors to get more familiar with the process
    • Learn about best practices for software development
    • Learn about why we make certain decisions (backward compatibility, raise warnings on breaking changes, adding tests and documentation, etc)
    • Encourage new contributors to open their own PRs

What we did

  • Started working on solving #423
  • Opened a PR: 468
  • We still need to apply changes to gradient boosted equivalent sources

2024-02-29

Time: 17:00 UTC

Participants:

  • Mariana
  • Santi
  • India

Quick updates:

  • Mariana: Working in issue https://github.com/fatiando/tutorials/issues/12

    • How can I get rid of this warning? It means something needs to be changed in Verde?
      image
    ​​​​/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:177: FutureWarning: Series.ravel is deprecated. The underlying array is already 1D, so ravel is not necessary.  Use `to_numpy()` for conversion to a numpy array instead.
​​​​  columns = {"data{}".format(i): comp.ravel() for i, comp in enumerate(data)}
​​​​/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:179: FutureWarning: The provided callable <function mean at 0x7f47517da8e0> is currently using DataFrameGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
​​​​  blocked = pd.DataFrame(columns).groupby("block").aggregate(reduction)
​​​​/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:236: FutureWarning: The provided callable <function mean at 0x7f47517da8e0> is currently using DataFrameGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
​​​​  grouped = table.groupby("block").aggregate(self.reduction)
​​​​/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:177: FutureWarning: Series.ravel is deprecated. The underlying array is already 1D, so ravel is not necessary.  Use `to_numpy()` for conversion to a numpy array instead.
​​​​  columns = {"data{}".format(i): comp.ravel() for i, comp in enumerate(data)}
​​​​/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:179: FutureWarning: The provided callable <function mean at 0x7f47517da8e0> is currently using DataFrameGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
​​​​  blocked = pd.DataFrame(columns).groupby("block").aggregate(reduction)
​​​​/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:236: FutureWarning: The provided callable <function mean at 0x7f47517da8e0> is currently using DataFrameGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
​​​​  grouped = table.groupby("block").aggregate(self.reduction)

    • Santi: these warnings are comming from Pandas (Mariana was using pandas==2.2.1 here). We should check if these are reproduced in Verde and fix them upstream. In summary: it's great the tutorials are raising these warnings, but we should not tackle them in the tutorials repo.

    • For this warning:

      ​​​​​​​​NumbaWarning: The TBB threading layer requires TBB version 2021 update 6 or later i.e., TBB_INTERFACE_VERSION >= 12060. Found TBB_INTERFACE_VERSION = 12050. The TBB threading layer is disabled.
​​​​​​warnings.warn(problem)

  • Changing meeting times:

    • Currently the only ones that can make it to the meeting of 00:00 GMT are the ones in PT (Pacific Time). Until Lu returns to his routine, we could move it to a time that's better for everyone else.
    • Leo cannot attend to the Thursdays 17:00 GMT meetings from now until August. We could move these meetings to same time on Mon, Tue, or Fri.
    • Santi prefers Tue or Fri. One Tue per month it will clash with Swung meeting.
    • TODO:
      • Move all meetings to 17:00 GMT
      • Decide which day might be better for everyone: Tuesday
      • Update Google calendar
      • Write a message in Mattermost letting people know. Let ppl know we are planning to do group programming
      • Post in the forum
      • Check if we need to update the time in fatiando/community

  • Harmonica issues to tackle in the near future:

    • We can go and change the default value of window_size directly without going through the warning-release process: https://github.com/fatiando/harmonica/issues/425
    • For #424 we would make one PR for raising the warnings, and then another one for actually making the change (that will be merged for 0.8)

  • Ideas for future meetings:

    • Do some pair programming or demos/intros on how to code and contribute
    • We can start with #423 next week!

2024-02-15

Time: 17:00 UTC

Participants:

  • Mariana
  • Leo
  • India

Quick updates:

2024-02-01

Time: 17:00 UTC

Participants:

  • Leo
  • Mariana
  • India

Quick updates:

  • Leo: Found a bug in the vertical derivative when implementing Euler deconvolution for a class: https://github.com/fatiando/harmonica/issues/454 Good issue for beginners
  • Leo: Trying to update versions of packages in Verde but running into a few issues. I think it's something on my computer with GMT and sphinx. I keep getting crashes that don't happen on CI. Will try changing OS and see if it keeps happening.
  • India: Do one of the calls as a demo of porting a function or notebook code from a paper/research repo into a package.

2024-01-26

Time: 00:00 UTC

Participants:

  • Santi :-(

Quick updates:

2024-01-18

Time: 17:00 UTC

Participants:

  • Santi
  • India
  • Mariana
  • Leo
  • Agustina

Quick updates:

  • Merged the forward functions for magnetic prisms: https://github.com/fatiando/harmonica/pull/448
    • Also changed the expected shape of the magnetization vector to better integrate with hm.magnetic_angles_to_vec
    • Plan to start working on the same changes for the forward fucntions for dipoles
  • Merged the update for the forward of point sources using Choclo: https://github.com/fatiando/harmonica/pull/422
    • Thanks to Lu for the work!
    • It might be the first Fatiando PR merged from Antarctica!
      Image Not Showing Possible Reasons
      • The image file may be corrupted
      • The server hosting the image is unavailable
      • The image path is incorrect
      • The image format is not supported
      Learn More →
      • Would make a nice post on social media!
  • Opened issue to improve error message while reading a corrupted Oasis Montaj grd file: https://github.com/fatiando/harmonica/issues/451
  • Example running Fatiando and PyGMT on Colab with conda: https://colab.research.google.com/drive/1rfXOt4NSgmnU5XknLoq0nQNqM5j0FzY7?usp=sharing
  • New package: Magali! For processing and inversion of magnetic microscopy.
    • Currently on my lab org (https://github.com/compgeolab/magali) but want to move to Fatiando. Objections? We talked about this before but just confirming.
    • MSc student Yago will arrive in São Paulo in the next month to work on this as his dissertation.
    • Will generate PRs in Harmonica as well (like Euler Deconvolution).
  • New article citing Fatiando: https://jurnal.stageofsleman.id/index.php/jsgs/article/download/11/12
    • Mariana sent an invitation to the corresponding author to join us in the community calls
  • Promoting Dominic Kempf to Developer on Pooch:
    • Has been very active and enthusiastic.
    • We need more hands in the project.
    • Leo will email and CC council to invite.
  • Add MAINTAINERS.md file to repos listing maintainers
    • That file should live in the repo for each package
    • Leo will update the GitHub teams
  • India shared and idea for generating regular grids at angles, useful for "diagonal" flight lines
    • It would be nice to see an issue with more details about it
    • One possible implementation would be to generate the grid and then rotate the coordinate system

2024-01-04

Time: 17:00 UTC

Participants:

  • Santi
  • India
  • Mariana
  • Leo

Quick updates:

  • Santi went to AGU

    • Met Matt Tankersely in person! Matt had a poster, and a lightning talk in the same session as me. Was great to talk about Fatiando and also see his presentation mentioning the same tools.
    • Met Blazej Bucha. He contributed to Boule. Great work on spherical harmonics: https://github.com/blazej-bucha.
    • Lunch with Fernando Perez, Jim (2i2c) and Rowan and Steve (Curvenote)
    • Geolatinas gatherings!
    • Met Yago and Suzana (Yago will start as Master Student with Leo)
    • Met Carlos Alberto Moreno Chaves, from USP.
    • Ines Sanchez Rodriguez, Hydrology. opencollaborationhydrology.com. Got invited to participate in a podcast about open-source. It'll probably be in Spanish.
    • Hélène Le Mével. Awesome inversions of gravity satelite data before, during and after the eruption of Hunga Tonga.
    • Tasha Snow. CryoCloud: https://www.cryocloud.io/. Great talk about why open science!
      • Intelectual generosity
      • Intelectual humility
      • Right to participate in science (democratizing it)
      • Everyone deserves dignity and respect
      • Open Science is a process, not a product!
    • Project Pythia.
    • Center of Open Science: https://www.cos.io/
    • Folks from NumFOCUS were there: Xarray, Dask, etc. Ryan Abernathey, Deepak Cherian.
    • Nigel Rees. Haven't got the chance to meet him, but he emailed me about the Fatiando poster. Matt had a talk with him.
    • Jiajia Sun. Interested in having magnetic equivalent sources in Harmonica.
    • I gave away a lot of Fatiando stickers. Even in the plane: the student that was sitting next to me knew about it
      Image Not Showing Possible Reasons
      • The image file may be corrupted
      • The server hosting the image is unavailable
      • The image path is incorrect
      • The image format is not supported
      Learn More →
      • Sarai Montañez Muñoz

  • Faster terrain correction by block averaging.

  • Plan to start adding kernels in Choclo for magnetic gradiometry

  • Discussion on Harmonica Issues for better defaults for equivalent sources:

    • Santi will tackle the removal of the depth_type
    • India will tackle the better defaults for window_size and depth

  • Leo: Started playing with spherical harmonics. SHTools is a bit large and overkill when we just want simple synthesis of a low degree model (like a lot of magnetic models). Plus, there is no package I found that can do the inversion part for scattered data (SHTools only does it for grids). I'll need this for class and for some projects so will start coding something and maybe get a student involved. Should this be part of Harmonica or its own package? The surface harmonics can be used for other things and the associated Legendre function calculation as well (scipy has it but it wraps an ugly fortran code and the normalization isn't very clear). But otherwise, this fits with the equivalent source stuff in Harmonica.

    • It could live in Harmonica!

  • Improving tests for Pooch downloaders:

    • Right now Pooch's tests rely on third-party servers being available when testing
    • If those servers go down, Pooch tests start failing
    • Idea: mock httpservers with pytest-httpserver for testing purposes.
    • Keep the current tests, but maybe make them optional
    • We should open an issue for creating an sftp server using pytest just to test it

  • Design choices for forwards and physical equivalent sources:

    • Merge magnetic forward functions in Harmonica to a single one, that either returns a single array, or a tuple of arrays
      • Those should take a field argument that could be b_e, b_n, b_u or b for the full vector, required argument.
    • Constructing the equivalent sources takes regular parameters plus the inc,dec of the sources
    • Fitting would be nice as ``
    • Design draft for the fit and predict methods for the eq mag sources 
      ​​​​​​​​class EquivalentMagSources:
​​​​​​​​    def __init__(self, ...):
​​​​​​​​        ...
​​​​​​​​        
​​​​​​​​    def fit(self, tfa=None, b_ee=None, ...):
​​​​​​​​        ...

​​​​​​​​    def predict(self, coordinates, field):
​​​​​​​​        ...
​​​​​​​​    
​​​​​​​​eqs = hm.EquivalentMagSources(
​​​​​​​​    depth=..., 
​​​​​​​​    inc_sources: float =..., 
​​​​​​​​    dec_sources: float =...,
​​​​​​​​)
​​​​​​​​eqs.fit(
​​​​​​​​    tfa=(coordinates, data, weights, inclination, declination),
​​​​​​​​    b_ee=(coordinates, data, weights),
​​​​​​​​    b_nn=(coordinates, data, weights),
​​​​​​​​)
​​​​​​​​b_ee = eqs.predict(coordiantes, data, field="b_ee")
​​​​​​​​(b_e, b_n, b_u) = eqs.predict(coordiantes, data, field="b")
​​​​​​​​tfa = hm.total_field_anomaly((b_e, b_n, b_u), inc, dec)

  • We might want to check if hm.magnetic_angles_to_vec work well with inc and dec as floats and amplitude as array.

    • Write tests!

  • The mag forward modelling functions take magnetizations as 2d array with each magnetic moment per row. We should change this to accept a tuple with the three components:

    ​​​​dipole_magnetic(
​​​​    coordinates,
​​​​    dipoles,
​​​​    magnetic_moments=(magnetization_e, magnetization_n, magnetization_u),
​​​​    parallel=True,
​​​​    dtype="float64",
​​​​    disable_checks=False,
​​​​):
    • This makes it consistent with hm.magnetic_angles_to_vec and with how coordinates are handled.

  • Leo: Trying to find a good magnetic microscopy dataset to add to Ensaio. Will clean up the one we used in this paper because it's relatively good with a bunch of well behaved dipoles. Great for running grid-related examples in Harmonica since the data is on a regular grid at constant height from the very start. Would also be good for an Euler Deconvolution example since they're all very close to dipoles.

Last changed by 
 
Fatiando a Terra
Open-source tools for geophysics
0
1308

Read more

Fatiando Development Calls 2025

📱 Join the video call: https://meet.jit.si/fatiando-a-terra

May 27, 2025
Fatiando Development Calls 2023

📱 Join the video call: https://meet.jit.si/fatiando-a-terra

Dec 19, 2023
Fatiando Development Calls 2022

:::info Link to the meeting notes for 2023: https://hackmd.io/@fatiando/development-calls-2023 ::: 📱 Join the video call: https://meet.jit.si/fatiando-a-terra 📅 Shared calendar: https://www.fatiando.org/calendar IMPORTANT: Everyone is required to follow our Code of Conduct when participating in the Fatiando community. Please review it carefully.

Mar 29, 2023
Fatiando Development Calls 2021

&#x1F4F1; Join the video call: https://swu.ng/chateau (Gather Town) &#x1F4C5; Shared calendar: https://www.fatiando.org/calendar IMPORTANT: Everyone is required to follow our Code of Conduct when participating in the Fatiando community. Please review it carefully. Development calls are open to everyone! Here we discuss routine project development: implementation details, code refactoring, maintenance tasks, documentation, etc. Discussions tend to be more technical than our Community Calls.

Dec 23, 2021
Read more from Fatiando a Terra
Published on HackMD