owned this note
owned this note
Published
Linked with GitHub
# Fatiando Development Calls 2024
📱 **Join the video call:** https://meet.jit.si/fatiando-a-terra
📅 **Shared calendar:** https://www.fatiando.org/calendar
> **IMPORTANT**: Everyone is required to follow our
> [Code of Conduct](https://github.com/fatiando/community/blob/main/CODE_OF_CONDUCT.md)
> when participating in the Fatiando community. Please review it carefully.
Development calls are **open to everyone**! Here we discuss project development, socialize, and do some live coding from time to time.
**Everyone is encouraged to participate and edit the notes below.**
--------------------------------------------------------------
## 2024-09-03
**Time:** 14:00 UTC
**Participants:**
* Santi
* Gelson
* Leo
* Yago
**Notes:**
* Plans for bringing Moulder to Python3
* Checking if the polygon is clockwise https://github.com/santisoler/fatiando/blob/be2bc3001849279f8e8ccbb67d934f6b684220a9/fatiando/mesher/geometry.py#L93-L213
* Implement the code directly in Harmonica the same as it is in the old Fatiando
* We can then start a kernel in Choclo and swap out the numpy code once it's done
* Moulder didn't have a license (too bad!). Now it has: https://github.com/fatiando/moulder/blob/master/LICENSE.txt
* Pooch talk at SciPy2024
* We should add the video to the docs
## 2024-08-27
**Time:** 14:00 UTC
**Participants:**
* Santi
**Notes:**
* Noticed that the package `build` available in conda-forge is outdated, `python-build` should be used instead.
* The latest version of `build` is not compatible with Python 3.12 and is causing problems while trying to create environments.
## 2024-08-20
**Time:** 17:00 UTC
**Participants:**
* India
* Gelson
* Mariana
* Agustina
* Santi
**Notes:**
* Harmonica v0.7.0 has been released! :tada:
* Team effort in São Paulo with India, Gelson and Yago!
* It was a release with a lot of changes. For the future, would be better to release more often than waiting for a huge release.
* Things we would like to have in the next release:
* Magnetic equivalent sources: https://github.com/fatiando/harmonica/issues/426
* Allow to forward magnetic fields using the `prism_layer` accessor: https://github.com/fatiando/harmonica/issues/399
* Checkout milestones for v0.8.0: https://github.com/fatiando/harmonica/milestone/9
* LATINMAG 2025 in Mexico
* Gelson and Leo will attend
* Agustina's adventure above the Artic circle
## 2024-07-16
**Time:** 14:00 UTC
**Participants:**
* Santi
* Leo
**Notes:**
* Santi at SciPy2024 :snake: :snow_capped_mountain: :bird:
* Awesome conference! Totally recommended!
* Scientific Python: https://www.scientific-python.org
* Developer guide: https://learn.scientific-python.org/development
* Recommending using `hatchery` as the build engine (instead of `setuptools`)
* SPECs: https://scientific-python.org/specs/
* Like PEPs and NEPs, but for the whole scientific community
* Still under construction/discussion. Join the conversation: https://discuss.scientific-python.org/c/specs/6
* There are discussions on including tips to enhance the security of scientific packages. Santi proposed to create a thread model and then list tasks that would mitigate those risks: from direct attack by state/independent actors to bus factor (only one person having access to the credentials for services like PyPI or ownership of the GitHub community, etc.)
* [Ruff](https://docs.astral.sh/ruff/) included rules to warn ppl about Numpy deprecated features that were removed in Numpy 2.0
* These rules were created in collaboration between Ruff devs and Numpy community
* A reason to consider moving away from `flake8` into `ruff`
* [`ipython` sphinx directive](https://ipython.readthedocs.io/en/stable/sphinxext.html) being under development
* Might be an alternative to the `jupyter-execute` one
* Numba devs are working on PIXIE: a package to compile numba code ahead of time.
* Still in beta
* https://numba-pixie.readthedocs.io/en/latest/
* Xarray will have a new class: `DataTree`
* Allows to fully support the netCDF standard: nested datasets.
* Original experimental repo: https://github.com/xarray-contrib/datatree
* Current implementation in `xarray` repo: https://github.com/pydata/xarray/blob/main/xarray/core/datatree.py
* Still under development
* Coordinate inheritance: https://github.com/pydata/xarray/issues/9077
* Santi worked on it during the sprints
* Curious things:
* Using `mpi4py` to create non-shared-memory parallelizations. Parallelizing Numba code (with `parallel=False`) across cluster nodes.
* Idea to optimize our Numba codes. We can loop over threads in parallel instead of over the whole range of number of observation points. That way, we can assign "chunks" of the observation points to each thread, so they can read and write arrays sequentially, which might speed up the computations.
* Gelson is implementig magnetic forward modelling of tetrahedrons. Coming soon to Choclo :slightly_smiling_face:
## 2024-07-09
**Time:** 17:00 UTC
**Participants:**
No meeting
* Holiday in São Paulo :tada:
* Santi in Scipy :snake:
* Agus in Groelandia! :snowflake:
## 2024-07-02
**Time:** 14:00 UTC
**Participants:**
* India
* Lu
* Santi
* Mariana
* Leo
**Notes:**
* Santi giving Verde some love
* Numpy 2.0 now plots the floats different now (Numpy>=2.0.0 prints `np.float64(3.0)` instead of `3.0` (to avoid confusion with Python floats). That change is breaking several tests, see 2024-06-18
* Problem with `verde.rolling_window` docstring: https://github.com/orgs/fatiando/discussions/155 and https://github.com/fatiando/verde/pull/458
* Pooch talk:
* HTTP dataset: https://scrippsco2.ucsd.edu/data/atmospheric_co2/primary_mlo_co2_record.html
* Mark is wanting a release of Boule. Will be breaking some compatibility. TODO:
* Check with Numpy 2.0
* Merge these: https://github.com/fatiando/boule/pull/192 and https://github.com/fatiando/boule/pull/187
* Social Media Communication:
* Santi, can you share the details of your Scipy talk https://www.scipy2024.scipy.org/ (day, time, title?) in the notes so I can make a quick flyer inviting people to come and listen to you if they are there?
* India, do you have a picture to announce your research stay In Brazil? :flag-br:
* We will :wink:
* Lu: Field school in Australia
* https://www.aseg.org.au/news/cage-2024-applications-open
* Opportunity to use Fatiando for processing the field data
* Run inversions with SimPEG
* For ppl with low coding experience:
* Jupyter notebook that imports functions from a file
* use ipywidgets: https://ipywidgets.readthedocs.io/en/latest/
## 2024-06-25
**Time:** 17:00 UTC
**Participants:**
* India
* Mariana
* Santi
* Leo
**Notes:**
* Santi started working on moving Pooch to src layout: https://github.com/fatiando/pooch/pull/427
* Pooch talk in SciPy
* Split the talk for different audiences.
* Record the live demos just in case wifi doesn't work.
* Main features
* Unpack
* Download from DOI
* Don't focus on custom downloaders and postprocessors, but maybe mention them.
* History of Pooch
* Ryan May, John Leeman.
* Scipy 2018?
* Merging MetPy and Verde downloaders into a new package.
* Started to being used by other packages.
* Future roadmap
* Improve registry file: just use a JSON file.
* Merge the two dictionaries with filename, url and filename, hash
* Improve logging.
* Plugin system, so custom downloaders are easy to develop and share without the need to push them upstream.
* fspec and intake
* Expect questions on "what are the differences, why not use them instead". Pooch is just a simple tool.
## 2024-06-18
**Time:** 14:00 UTC
**Participants:**
* Santi
* Leo
* Mariana
**Notes:**
* Harmonica v0.7:
* Maybe we can get this in as well: https://github.com/fatiando/harmonica/pull/493
* Numpy 2.0.0 is out there!
* Rerunning some tests on CI to see if there are any issues with our packages
* Harmonica works fine! Some doctests are failing because Numpy>=2.0.0 prints `np.float64(3.0)` instead of `3.0` (to avoid confusion with Python floats).
* Choclo and numba work great!
* Verde tests are broken because of a badly formatted exception: https://github.com/fatiando/verde/actions/runs/9555832761/job/26339861103
* Harmonica:
* After dropped Python 3.9, we don't need the thing with macos x86
* src vs flat layout
* https://www.pyopensci.org/python-package-guide/package-structure-code/python-package-structure.html
* https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/
* https://hynek.me/articles/testing-packaging/
* We could start implementing a src layout in Pooch and see how it goes
## 2024-06-11
**Time:** 17:00 UTC
**Participants:**
* Santi
* India
* Jose Maria
**Notes:**
* Shall we release Verde v1.8.1 (https://github.com/fatiando/verde/issues/446)?
* Verde 1.8.0 doesn't work in Python 3.12 because of the `pkg_resources` that Leo already fixed in https://github.com/fatiando/verde/pull/448
* This blocks extending support for Python 3.12 on all packages that depend on Verde (Harmonica for example: https://github.com/fatiando/harmonica/pull/484/)
* pypozo: https://github.com/JoseMariaGarciaMarquez/pypozo
* Conversations about moving tests outside the package folders: https://github.com/fatiando/community/issues/154
* Roadmap for Harmonica v0.7.0:
* We would like to have the total field anomaly function
* Support for Py 3.12
* Change default value for depth in eqs
## 2024-06-04
**Time:** 14:00 UTC
**Participants:**
* Leo
* India
* Lu
* Yago
* Gelson
* Arthur
**Notes:**
* India finish the default window size PR!
* Idea: the whole CI and development workflow is hard to document and hard to get started. Maybe it would be worth running a yearly development bootcamp where we train people on this. Could work in break out rooms individually. 2h would be enough: 1h for intro and 1h to work on things in breakout rooms. Topics: github workflow, running checks, CI, tests, building docs, etc.
* Idea from TRANSFORM:
* People like video tutorials
* Do a joint tutorial with SimPEG
* Use Harmonica et al to process gravity or mag data
* Hand the grid to simpeg for a simple inversion
* Gelson: shared a tutorial that had very short (a few mins) videos for each section of the notebook. That makes it easy to update the videos. Example notebook: https://github.com/FORCaist/forcsensei/blob/master/FORCsensei.ipynb
* This format would be great with the tutorials that Mariana is working on.
* So format doesn't really fit Transform but it's better in the long term
* Think about what we'd want from an offline event
## 2024-05-28
**Time:** 17:00 UTC
**Participants:**
* Mariana
Posted Santi's video about gravity disturbance
Leo: Sorry for leaving you alone Mariana!
## 2024-05-21
**Time:** 14:00 UTC
**Participants:**
* Santi
* Mariana
* Leo
* Lu
* India
:penguin: :penguin::penguin: :penguin: Lu's amazing talk! :penguin: :penguin: :penguin: :penguin:
Link to the recorded talk. Mariana edited her unnecessary mumbling in the beginning and the end :P :
https://www.youtube.com/live/XNJ1WRfaSkk
## 2024-05-14
**Time:** 17:00 UTC
**Participants:**
* Santi
* Mariana
* Leo
**Quick updates:**
* Next meeting we'll have Lu's talk about his adventures in Antarctica :penguin:!
* We should promote it on Mattermost, social media, etc
* Live streaming? Or just record it and then upload?
* Ask Lu if we can live stream it
* We can live stream it directly from Jitsi
* Make a practice run before the talk
* Need Youtube API and **start the stream** on Youtube!
* We can schedule the live stream ahead of time. This way we can share the link in social media, add it to the calendar, etc
* Santi's maintenance works:
* Replace Sphinx napoleon for numpydoc in multiple libraries: https://github.com/fatiando/community/issues/149
* These changes would improve how we show the API docs of classes (solving issues with attributes that have a trailing underscode, like `coefs_`).
* Check if we can also list methods as a toc on the right menu.
* Solved issues with tests failing in MacOS runners using the oldest dependencies: https://github.com/fatiando/harmonica/pull/499
* New macos GitHub runners make use of Apple silicon (M1, M2 chips). Numpy for Python 3.8 is not available for those chips.
* Solved the issue by running the Python 3.8 tests on the latest x86 (intel chips) macos runners.
* Codecov failing when trying to upload multiple files (specially when PR is opened from a fork)
* Working on solving it by uploading coverage reports as artifacts and uploading them all on a single run
* https://github.com/fatiando/harmonica/pull/502
* Leo's nerdy week:
* Associated Legendre functions! Did a deep dive and started implementing (almost finished, just speeding them up now) https://github.com/fatiando/harmonica/pull/505
* IGRF in Harmonica: https://github.com/fatiando/harmonica/issues/504
* It's fine to add Boule as a dependency
```python
class IGRF(date, ellipsoid=boule.WGS84):
...
IGRF(datetime(1990, 3, 14)).predict((lon, lat, height), field="b")
IGRF(datetime(1990, 3, 14)).grid(grid_coords, field="b")
sh = MagneticSphericalHarmonics(g, h, ref_radius=..., normalization="schmidt", ellipsoid=None)
sh.predict(..., field="b")
sh.grid(..., field="b")
sh = MagneticSphericalHarmonics(max_degree, ref_radius=..., normalization="schmidt", ellipsoid=None)
sh.fit(
be=((lon, lat, height), be),
bn=((lon, lat, height), bn),
bu=((lon, lat, height), bu),
)
sh = GravitySphericalHarmonics(max_degree, ref_radius=..., normalization="full", ellipsoid=None)
sh.fit(
gz=((lon, lat, height), gz),
gxx=((lon, lat, height), gxx),
)
sh.predict((lon, lat, height), field="gz")
```
## 2024-05-07
**Time:** 14:00 UTC
**Participants:**
* India
* Santi
**Quick updates:**
* Harmonica CI's failing on MacOS with Python 3.8
* Not being able to install Numpy with pip
* Trying to fix it: [#496](https://github.com/fatiando/harmonica/pull/496)
* Pair programming on India's PR for changing default `window_size` in gradient-boosted eqs:
* https://github.com/fatiando/harmonica/pull/487
* If the passed `coordinates` have less than 5e3 points, then a warning is raised and a single window will be used.
* The `self.window_size_` is set to None in that case, since it's not being used for defining the window.
* Few ideas to improve this: use `slice` instead of `np.arange` in the intermediate returns.
## 2024-04-30
**Time:** 17:00 UTC
**Participants:**
* Mariana
* Santi (only during the end)
**Quick updates:**
* Santi talk is available in Zoom, can I download it and publishing in Fatiando Youtube Channel? Slides and notebooks: https://github.com/santisoler/2024-uh
* Yes! I got permission to publish it. Maybe check if there are bits that should be removed?
* Today's is Fatiando birthday. Santi check the git log to understand better the transition. Before April 30th 2010, Fatiando was written in cpp (Pregnancy stage)
* Post on LinkedIn :heavy_check_mark:
* Post on Mattermost :heavy_check_mark:
* Post on Mastodon
* Post on Twitter
## 2024-04-23
**Time:** 14:00 UTC
**Participants:**
* Lu
* Leo
* India
* Santi
* Mariana
**Quick updates:**
* Many Boule PRs merged!
* So many new ellipsoids: https://www.fatiando.org/boule/dev/ellipsoids.html
* New mean_radius: https://github.com/fatiando/boule/blob/main/boule/_ellipsoid.py#L233
* mean_radius for TriAxialEllipsoid: https://github.com/fatiando/boule/blob/main/boule/_triaxialellipsoid.py#L190
* Adds the potential for the ellipsoid: https://github.com/fatiando/boule/pull/187
* NumFOCUS trouble as something for us to keep an eye on for Fatiando
* https://pirsquared.org/blog/numfocus-concerns.html#NF-europe
* Santi will give a talk at Jiajia Sun's undergrad course today about gravity processing and why we should use gravity disturbance
* Santi's Pooch talk at SciPy got accepted :tada:
* Share reviews? Post on mattermost without reviewer names
* Issue with attributes of classes in Verde and Harmonica
* Switch back to numpydoc instead of sphinx napoleon
## 2024-04-16
**Time:** 17:00 UTC
**Participants:**
* Santi
* Leo Miquelutti
* Mariana
* India
* Martin
**Quick updates:**
- Codedov push failing in https://github.com/fatiando/harmonica/pull/486
- Reran the test to see if it pass nows
- :rocket::tada: Choclo v0.2.0 has been released! :tada::rocket:
- Changelog: https://www.fatiando.org/choclo/latest/changes.html#version-0-2-0
- Highlights:
- Improved docs with dark theme :sunglasses:
- Restored kernel functions for point sources (needed for micropaleomag applications)
- Roadmap to Harmonica v0.7.0:
- Milestones: https://github.com/fatiando/harmonica/milestone/8
- What about Python 3.12 support? Blocked by issue with `pkg_resources` in Verde. (see https://github.com/fatiando/harmonica/pull/484)
- Fixed in Verde! https://github.com/fatiando/verde/pull/448 Will be available in v1.8.1 coming soon.
- I think we can make a release after finishing [#487](https://github.com/fatiando/harmonica/pull/487) and tackling [#424](https://github.com/fatiando/harmonica/issues/424) (Santi can work on that one)
- Martin
- Progress on the xml reader grd files
- Able to read the `wellknown_epsg` attribute on the `projection` tag
- Add a private function to read the xml file
- We talked about the signature of that function: it's better if it returns a dictionary witht the "EPSG" value inside. Then the `attrs` dictionary can be extended before passing it to the xarray object.
- This would allow us to extend the reader of xml without having to change other code.
- Chat on [#487](https://github.com/fatiando/harmonica/pull/487)
- We need to add tests checking if the window size is set properly
- Maybe defining a grid with N times 5_000 points and comparing the value of window size it gets assigned against the expected one. We should use `numpy.testing.assert_allclose()` for that since we are comparing floats.
- Plan for Lu's talk on one of the 14:00 GMT meetings! :penguin:
## 2024-04-09
**Time:** 14:00 UTC
**Participants:**
* Lu
* Mariana
* Santi
* Leo
**Quick updates:**
- Mariana Questions
- Discussions Forum
- Wonder if it is ok to close discussions outdated? Like BIRS, Fatiando Community Survey. SO we can keep just the relevant posts with easy access?
- R= Ok. Go ahead :tada:
- How to subscribe discussions? I'm not sure I get notifications
https://github.com/orgs/fatiando/discussions/categories/announcements
- R= Go to [Community repo](https://github.com/fatiando/community) → "Watch" → "All activity" or "custom" → "Discussions"
- To do
- Add this to https://github.com/orgs/fatiando/discussions/56 Done! :heavy_check_mark:
- Opensource software Wikipedia page
- Should Fatiando be included on https://en.wikipedia.org/wiki/List_of_free_geology_software? This one was created by Matt in 2009! We also have this repo https://github.com/softwareunderground/awesome-open-geoscience
- R= :cricket: :cricket: (Totally)
- Many new PRs opened in Boule by Mark (SHTOOLS)
- New contributor in Verde working on grid with nans
- Question regarding converting mineralogy to physical properties.
- Verde:
- Issue with pkg_resources, it doesn't work in Python 3.12
- Warning coming from Pandas:
https://github.com/fatiando/harmonica/actions/runs/8607607182/job/23588411532#step:12:1135
```
tests/test_gradient_boosted_eqs.py: 8 warnings
/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/verde/blockreduce.py:179: FutureWarning: The provided callable <function median at 0x7fb9141e1b20> is currently using DataFrameGroupBy.median. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "median" instead.
blocked = pd.DataFrame(columns).groupby("block").aggregate(reduction)
```
## 2024-04-02
**Time:** 17:00 UTC
**Participants:**
* Santi
* India
* Mariana
* Leo
* Martin
**Quick updates:**
* Update meeting times
* Go back to the two times (one every two weeks)
* 14:00 GMT sounds like the best option
* Discussions around arguments for equivalent sources:
* https://github.com/fatiando/harmonica/issues/425
* https://github.com/fatiando/harmonica/issues/424
* Use `"default"` as default value instead of `None`
- India's workflow:
- JupyterHub (because here she has configured ssh key with Github) → terminal → `git clone repo` → `mamba create env` → `conda activate harmonica` → `git grep EquivalentSourcesGB` (to search for which file needs to change) → edit file gradient_boosted.py →
- To get a nice bash prompt with little effort: https://starship.rs/
- Mariana's idea: use breakout rooms for tackling multiple issues at the same time
**Todo:**
- [x] Update meeting times:
- [x] Update fatiando calendar (santi)
- [x] Post on Mattermost (santi)
- [x] Post on forum (mariana)
- [x] Post on linkedin (mariana)
## 2024-03-26
**Time:** 17:00 UTC
**Participants:**
* Mariana
* India
* Lu
* Santi
**Quick updates:**
* Question regarding Zenodo citations: [fatiando/community@145](https://github.com/fatiando/community/issues/145)
* Done some maintenance work on Choclo and Harmonica
* I think Leo Miquellutti's PR on total gradient amplitude might be ready to be merged: https://github.com/fatiando/harmonica/pull/478 :tada:
* Lu came back from Antarctica! :penguin:
* Ice drilling and sampling every 5 cm (on 200m hole!)
* Grav, mag, MT surveys
* He's willing to give a presentation about his work there
* We should have another meeting time that's more suitable for Lu
* [timeanddate meeting planner](https://www.timeanddate.com/worldclock/meetingtime.html?iso=20240402&p1=256&p2=136&p3=196&p4=233&p5=247)
* Discussion around some Harmonica issues regarding equivalent sources
* Better default for the `depth`: we could make it clever enought to set the depth argument as 4.5 times the average distance between the data points. Shall we still allow users to set the `depth` manually? Should we include a `depth_factor` argument that defaults to 4.5? Should `depth=None` by default and in that case the class should compute it with that formula?
* Better default for window size: set it allowing to have in average 5k data points per window. Maybe we should include the `5_000` as an argument of the class instead of hard coding it.
## 2024-03-19
**Time:** 17:00 UTC
**Participants:**
* Mariana
* India
* Santi
* Leo
**Quick updates:**
- Mariana. New post in LinkedIn promoting community meetings
>WARNING: No ICDs were found. Either,
>Install a conda package providing a OpenCL implementation (pocl, oclgrind, intel-compute-runtime, beignet) or Make your system-wide implementation visible by installing ocl-icd-system conda package.
:rotating_light: **Group programming** :rotating_light:
* We'll continue our work on Harmonica ssue [#423](https://github.com/fatiando/harmonica/issues/423)
* Any volunteers to pick up where Santi left off?
* For Mariana to pull changes from Santi's fork:
* `git remote add santisoler git@github.com:santisoler/harmonica.git`
* `git fetch santisoler`
* `git switch remove-depth-type`
* `git grep depth_type`
## 2024-03-12
**Time:** 17:00 UTC
**Participants:**
* Santiago
* Mariana
* Martin
* India
* Leo
**Quick updates:**
* Maintenance:
* New tool: [Burocrata](https://github.com/fatiando/burocrata) :rocket:
* Automatically add the copyright notice on every source file
* The old [`license_notice.py`](https://github.com/fatiando/harmonica/blob/15119051a81e0a360d15a2ce2d4b55f220ad651c/tools/license_notice.py) with a CLI and available in PyPI
* Using Dependabot for automatically updating the GitHub Actions: https://github.com/fatiando/harmonica/pull/455 (not only in harmonica, but everywhere else as well)
* Replace PyPI API tokens with Trusted Publisher configuration: https://github.com/fatiando/community/issues/141
* Started building out Magali: https://www.fatiando.org/magali/dev/overview.html
* Need a release of Choclo and fix to bug in vertical derivative in Harmonica (see https://github.com/fatiando/harmonica/issues/454)
* Martin is planning on taking on the Euler Deconvolution issue
* New filter that we could add: Calculate anomalous field vector B from the total field anomaly. This can be done with FFT. Sort of based on this paper https://doi.org/10.1029/2008JB006006
* Leo will open an issue
- Mariana
- Update svg figure on Inkscape :). Approved by Leo https://github.com/fatiando/tutorials/issues/14
- Question. Ideas to fix my communist gravity figure?
- Minor thing: pass `projection=projection` to `eqs.grid`
- Gave a talk at the Uni in Bristol about her PhD and at the end invited people to Fatiando
- We should add it to the Fatiando news in the website!
- Santi submitted a talk about Pooch to Scipy. Waiting to see if they'll accept.
## 2024-03-05
**Time:** 17:00 UTC
**Participants:**
* Mariana
* India
* Gelson
* Santi
* Leo
* Agustina
**Quick announcements**
* :mega: We are changing time of the meetings to every **Tuesday 17:00 GMT**
**Agenda**
:rotating_light: **Group programming** :rotating_light:
- The goal is to tackle one of the Harmonica Issues ([#423](https://github.com/fatiando/harmonica/issues/423) was chosen last week) in group so we can all learn from the process and the train of thought involved in contributing to Fatiando.
- One could take the lead and type, while the rest chime in with ideas and questions.
- The idea is not to make it a _live stream_ where the protagonist is the typer, but to make it as interactive as possible. Other people can take the lead and start typing themselves.
- Ultimate goals:
- Allow new contributors to get more familiar with the process
- Learn about best practices for software development
- Learn about why we make certain decisions (backward compatibility, raise warnings on breaking changes, adding tests and documentation, etc)
- Encourage new contributors to open their own PRs
**What we did**
- Started working on solving [#423](https://github.com/fatiando/harmonica/issues/423)
- Opened a PR: [468](https://github.com/fatiando/harmonica/pull/468)
- We still need to apply changes to gradient boosted equivalent sources
## 2024-02-29
**Time:** 17:00 UTC
**Participants:**
- Mariana
- Santi
- India
**Quick updates:**
* Mariana: Working in issue https://github.com/fatiando/tutorials/issues/12
* How can I get rid of this warning? It means something needs to be changed in Verde?
![image](https://hackmd.io/_uploads/BkrhF3O2p.png)
```
/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:177: FutureWarning: Series.ravel is deprecated. The underlying array is already 1D, so ravel is not necessary. Use `to_numpy()` for conversion to a numpy array instead.
columns = {"data{}".format(i): comp.ravel() for i, comp in enumerate(data)}
/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:179: FutureWarning: The provided callable <function mean at 0x7f47517da8e0> is currently using DataFrameGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
blocked = pd.DataFrame(columns).groupby("block").aggregate(reduction)
/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:236: FutureWarning: The provided callable <function mean at 0x7f47517da8e0> is currently using DataFrameGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
grouped = table.groupby("block").aggregate(self.reduction)
/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:177: FutureWarning: Series.ravel is deprecated. The underlying array is already 1D, so ravel is not necessary. Use `to_numpy()` for conversion to a numpy array instead.
columns = {"data{}".format(i): comp.ravel() for i, comp in enumerate(data)}
/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:179: FutureWarning: The provided callable <function mean at 0x7f47517da8e0> is currently using DataFrameGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
blocked = pd.DataFrame(columns).groupby("block").aggregate(reduction)
/home/m/anaconda3/envs/fatiando-tutorials/lib/python3.11/site-packages/verde/blockreduce.py:236: FutureWarning: The provided callable <function mean at 0x7f47517da8e0> is currently using DataFrameGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
grouped = table.groupby("block").aggregate(self.reduction)
```
* Santi: these warnings are comming from Pandas (Mariana was using `pandas==2.2.1` here). We should check if these are reproduced in Verde and fix them upstream. In summary: it's great the tutorials are raising these warnings, but we should not tackle them in the tutorials repo.
* For this warning:
```
NumbaWarning: The TBB threading layer requires TBB version 2021 update 6 or later i.e., TBB_INTERFACE_VERSION >= 12060. Found TBB_INTERFACE_VERSION = 12050. The TBB threading layer is disabled.
warnings.warn(problem)
```
* Possible solution? https://github.com/numba/numba/issues/7531#issuecomment-1793233020
* Santi: With the latest version of `tbb` it should be fine. Although I'm curious why conda installed an old version of it...
* Mariana pushed the new environment to a new branch so we reproduce this: https://github.com/fatiando/tutorials/tree/update_environment
* Changing meeting times:
* Currently the only ones that can make it to the meeting of 00:00 GMT are the ones in PT (Pacific Time). Until Lu returns to his routine, we could move it to a time that's better for everyone else.
* Leo cannot attend to the Thursdays 17:00 GMT meetings from now until August. We could move these meetings to same time on Mon, Tue, or Fri.
* Santi prefers Tue or Fri. One Tue per month it will clash with Swung meeting.
* TODO:
* [x] Move all meetings to 17:00 GMT
* [x] Decide which day might be better for everyone: **Tuesday**
* [x] Update Google calendar
* [x] Write a message in Mattermost letting people know. Let ppl know we are planning to do _group_ programming
* [x] Post in the forum
* [x] Check if we need to update the time in `fatiando/community`
* Harmonica issues to tackle in the near future:
* We can go and change the default value of `window_size` directly without going through the warning-release process: https://github.com/fatiando/harmonica/issues/425
* For [#424](https://github.com/fatiando/harmonica/issues/424) we would make one PR for raising the warnings, and then another one for actually making the change (that will be merged for 0.8)
* Ideas for future meetings:
* Do some pair programming or demos/intros on how to code and contribute
* We can start with [#423](https://github.com/fatiando/harmonica/issues/423) next week!
## 2024-02-15
**Time:** 17:00 UTC
**Participants:**
* Mariana
* Leo
* India
**Quick updates:**
* Mariana: Working in issue https://github.com/fatiando/tutorials/issues/12
* Verde env has the latest compatible versions of most things: https://github.com/fatiando/verde/blob/main/environment.yml - Generates these pages: https://www.fatiando.org/verde/dev/
* Mariana: Is this image correct? https://www.fatiando.org/tutorials/_images/topographic-correction.svg
* India and Marian's meet up in the UK
* Leo did some work on Verde
* Leo tried and failed to get the Spline fit working with a conjugate-gradient method using Scipy's `LinearOperator`. For some reason, the method doesn't want to converge with the type of matrix we have in the Spline. Tested it on a simple matrix that looks like our and it just doesn't converge (but it does for a random matrix).
## 2024-02-01
**Time:** 17:00 UTC
**Participants:**
* Leo
* Mariana
* India
**Quick updates:**
* Leo: Found a bug in the vertical derivative when implementing Euler deconvolution for a class: https://github.com/fatiando/harmonica/issues/454 **Good issue for beginners**
* Leo: Trying to update versions of packages in Verde but running into a few issues. I think it's something on my computer with GMT and sphinx. I keep getting crashes that don't happen on CI. Will try changing OS and see if it keeps happening.
* India: Do one of the calls as a demo of porting a function or notebook code from a paper/research repo into a package.
## 2024-01-26
**Time:** 00:00 UTC
**Participants:**
* Santi :-(
**Quick updates:**
* Santi won't attend to the next few meetings. See you all back after Feb 15.
* Harmonica
* I think we should really tackle a few last PRs before the end of Feb so we can make a new release of Harmonica.
* Milestone for v0.7.0: https://github.com/fatiando/harmonica/milestone/8
* I decided to move [#352](https://github.com/fatiando/harmonica/issues/352) and [#353](https://github.com/fatiando/harmonica/issues/353) to v0.8.0.
* Leo, would be nice to have https://github.com/fatiando/harmonica/pull/441 as well. No pressure, but I think it just need some little work to be ready.
## 2024-01-18
**Time:** 17:00 UTC
**Participants:**
* Santi
* India
* Mariana
* Leo
* Agustina
**Quick updates:**
* Merged the forward functions for magnetic prisms: https://github.com/fatiando/harmonica/pull/448
* Also changed the expected shape of the `magnetization` vector to better integrate with `hm.magnetic_angles_to_vec`
* Plan to start working on the same changes for the forward fucntions for dipoles
* Merged the update for the forward of point sources using Choclo: https://github.com/fatiando/harmonica/pull/422
* Thanks to Lu for the work!
* It might be the first Fatiando PR merged from Antarctica! :penguin:
* Would make a nice post on social media!
* Opened issue to improve error message while reading a corrupted Oasis Montaj grd file: https://github.com/fatiando/harmonica/issues/451
* Example running Fatiando and PyGMT on Colab with conda: https://colab.research.google.com/drive/1rfXOt4NSgmnU5XknLoq0nQNqM5j0FzY7?usp=sharing
* New package: Magali! For processing and inversion of magnetic microscopy.
* Currently on my lab org (https://github.com/compgeolab/magali) but want to move to Fatiando. Objections? We talked about this before but just confirming.
* MSc student Yago will arrive in São Paulo in the next month to work on this as his dissertation.
* Will generate PRs in Harmonica as well (like Euler Deconvolution).
* New article citing Fatiando: https://jurnal.stageofsleman.id/index.php/jsgs/article/download/11/12
* Mariana sent an invitation to the corresponding author to join us in the community calls
* Promoting [Dominic Kempf](https://github.com/dokempf) to Developer on Pooch:
* Has been very active and enthusiastic.
* We need more hands in the project.
* Leo will email and CC council to invite.
* Add `MAINTAINERS.md` file to repos listing maintainers
* That file should live in the repo for each package
* Leo will update the GitHub teams
* India shared and idea for generating regular grids at angles, useful for "diagonal" flight lines
* It would be nice to see an issue with more details about it
* One possible implementation would be to generate the grid and then rotate the coordinate system
## 2024-01-04
**Time:** 17:00 UTC
**Participants:**
* Santi
* India
* Mariana
* Leo
**Quick updates:**
* Santi went to AGU
* Met Matt Tankersely in person! Matt had a poster, and a lightning talk in the same session as me. Was great to talk about Fatiando and also see his presentation mentioning the same tools.
* Met Blazej Bucha. He contributed to Boule. Great work on spherical harmonics: https://github.com/blazej-bucha.
* Lunch with Fernando Perez, Jim (2i2c) and Rowan and Steve (Curvenote)
* Geolatinas gatherings!
* Met Yago and Suzana (Yago will start as Master Student with Leo)
* Met Carlos Alberto Moreno Chaves, from USP.
* Ines Sanchez Rodriguez, Hydrology. opencollaborationhydrology.com. Got invited to participate in a podcast about open-source. It'll probably be in Spanish.
* Hélène Le Mével. Awesome inversions of gravity satelite data before, during and after the eruption of Hunga Tonga.
* Paper is out: https://www.science.org/doi/10.1126/sciadv.adh3156
* Tasha Snow. CryoCloud: https://www.cryocloud.io/. Great talk about why open science!
* Intelectual generosity
* Intelectual humility
* Right to participate in science (democratizing it)
* Everyone deserves dignity and respect
* Open Science is a process, not a product!
* Project Pythia.
* Center of Open Science: https://www.cos.io/
* Folks from NumFOCUS were there: Xarray, Dask, etc. Ryan Abernathey, Deepak Cherian.
* Nigel Rees. Haven't got the chance to meet him, but he emailed me about the Fatiando poster. Matt had a talk with him.
* Jiajia Sun. Interested in having magnetic equivalent sources in Harmonica.
* I gave away a lot of Fatiando stickers. Even in the plane: the student that was sitting next to me knew about it :rolling_on_the_floor_laughing:
* Sarai Montañez Muñoz
* Faster terrain correction by block averaging.
* Santi discussed some implementation with Craig before Banff.
* We kept talking about it with Matt.
* After AGU I started drafting an implementation: https://github.com/santisoler/harmonica/blob/prism-layer-fast-forward/harmonica/_forward/prism_layer.py
* Plan to start adding kernels in Choclo for magnetic gradiometry
* Discussion on Harmonica Issues for better defaults for equivalent sources:
* Santi will tackle the removal of the `depth_type`
* India will tackle the better defaults for `window_size` and `depth`
* Leo: Started playing with spherical harmonics. SHTools is a bit large and overkill when we just want simple synthesis of a low degree model (like a lot of magnetic models). Plus, there is no package I found that can do the inversion part for scattered data (SHTools only does it for grids). I'll need this for class and for some projects so will start coding something and maybe get a student involved. **Should this be part of Harmonica or its own package?** The surface harmonics can be used for other things and the associated Legendre function calculation as well (scipy has it but it wraps an ugly fortran code and the normalization isn't very clear). But otherwise, this fits with the equivalent source stuff in Harmonica.
* It could live in Harmonica!
* Improving tests for Pooch downloaders:
* Right now Pooch's tests rely on third-party servers being available when testing
* If those servers go down, Pooch tests start failing
* Idea: mock httpservers with `pytest-httpserver` for testing purposes.
* Keep the current tests, but maybe make them optional
* We should **open an issue** for creating an sftp server using pytest just to test it
* Design choices for forwards and physical equivalent sources:
* Merge magnetic forward functions in Harmonica to a single one, that either returns a single array, or a tuple of arrays
* Those should take a `field` argument that could be `b_e`, `b_n`, `b_u` or `b` for the full vector, required argument.
* Constructing the equivalent sources takes regular parameters plus the inc,dec of the sources
* Fitting would be nice as ``
* Design draft for the `fit` and `predict` methods for the eq mag sources
```
class EquivalentMagSources:
def __init__(self, ...):
...
def fit(self, tfa=None, b_ee=None, ...):
...
def predict(self, coordinates, field):
...
eqs = hm.EquivalentMagSources(
depth=...,
inc_sources: float =...,
dec_sources: float =...,
)
eqs.fit(
tfa=(coordinates, data, weights, inclination, declination),
b_ee=(coordinates, data, weights),
b_nn=(coordinates, data, weights),
)
b_ee = eqs.predict(coordiantes, data, field="b_ee")
(b_e, b_n, b_u) = eqs.predict(coordiantes, data, field="b")
tfa = hm.total_field_anomaly((b_e, b_n, b_u), inc, dec)
```
* We might want to check if `hm.magnetic_angles_to_vec` work well with inc and dec as floats and amplitude as array.
* Write tests!
* The mag forward modelling functions take magnetizations as 2d array with each magnetic moment per row. We should change this to accept a tuple with the three components:
```
dipole_magnetic(
coordinates,
dipoles,
magnetic_moments=(magnetization_e, magnetization_n, magnetization_u),
parallel=True,
dtype="float64",
disable_checks=False,
):
```
* This makes it consistent with `hm.magnetic_angles_to_vec` and with how coordinates are handled.
* Leo: Trying to find a good magnetic microscopy dataset to add to Ensaio. Will clean up the one we used [in this paper](https://doi.org/10.31223/X5QD5Z) because it's relatively good with a bunch of well behaved dipoles. Great for running grid-related examples in Harmonica since the data is on a regular grid at constant height from the very start. Would also be good for an Euler Deconvolution example since they're all very close to dipoles.