# Fatiando Development Calls 2021
📱 **Join the video call:** https://swu.ng/chateau (Gather Town)
📅 **Shared calendar:** https://www.fatiando.org/calendar
> **IMPORTANT**: Everyone is required to follow our
> [Code of Conduct](https://github.com/fatiando/community/blob/main/CODE_OF_CONDUCT.md)
> when participating in the Fatiando community. Please review it carefully.
Development calls are **open to everyone**! Here we discuss routine project development: implementation details, code refactoring, maintenance tasks, documentation, etc. Discussions tend to be **more technical** than our Community Calls.
**Everyone is encouraged to edit the notes below.**
--------------------------------------------------------------
## 2021-12-17
**Time:** 14:00 (UTC)
**Discussion points:**
* **Meeting got cancelled**. Some of the notes were moved to the 2022 document.
* Last meeting of the year! :tada: :champagne:
* Started a new document for 2022: https://hackmd.io/@fatiando/development-calls-2022
## 2021-12-10
**Time:** 14:00 (UTC)
**Participants**
* Leo
* Lu
* Santi
**Discussion points:**
* Rename *master* to *main* in Verde: https://github.com/fatiando/verde/pull/335
* Dependency version support plan: https://github.com/fatiando/community/issues/40
* AGU poster finished and published! https://github.com/fatiando/agu2021
* Gravity in Osborne Mine, Australia:
Only have grid data (50 meters raster)
https://geoscience.data.qld.gov.au/de/gravity-gradiometry/gg001178
* Some test run for FFT-Based Filter:
* Need to drop 'upward' information for xrft.fft
* The input array contains coordinate variable(s) (['upward']) whose dims include the transform dimension(s) `northing`. Please drop these coordinates (`.drop(['upward']`) before *invoking xrft.
* https://github.com/xgcm/xrft/issues/166
* Pad function : Deal with dimension with odd number
* Original data: xarray.DataArray'magnetic_anomaly'northing: 131 easting: 80
* Pad xrft.pad(Input,easting=24,northing=24) :heavy_check_mark: xarray.DataArray'magnetic_anomaly'northing: 179 easting: 128
* Pad xrft.pad(Input,easting=24,northing=**21-23**)
--> 276 vector[-n_end:] = np.arange(vmax + spacing, vmax + spacing * (n_end + 1), spacing)
ValueError: could not broadcast input array from shape (24,) into shape (23,)
* Lets open an issue to fix this bug https://github.com/xgcm/xrft/issues/172
* Invite the xrft devs for a community call next year to talk about our needs and things like non-dimensional coordinates.
## 2021-12-03
**Time:** 14:00 (UTC)
**Participants**
* Agustina
* Lu
* Santi
**Discussion points:**
* Harmonica release :tada: :confetti_ball:
* Future releases: more work on the docs, tutorials.
* Add tutorial for gradient-boosted eqs using a mid-size dataset (a few hundred thousand data points).
* Prism layer to pyvista
* https://github.com/fatiando/harmonica/pull/291
* Shout out to the pyvista forum for feedback
* New magnetic data from the Osborne Mine, Australia:
* 
* [Notebook with data preparation](https://nbviewer.org/github/fatiando/data/blob/main/osborne-magnetic/prepare.ipynb)
* Sadly, the ground gravity data is way too sparse to be useful in this region.
* New ProtonMail account so we can sign up for services without depending on Leo
* fatiandoaterra@protonmail.com
* Controlled by Leo and Santiago at the moment
* Includes 2FA and a strong password
* Credentials shared between admins by GPG encrypted messages and stored in secure password managers
* Used for our Twitter account
* Can give tweeting permissions through TweetDeck without sharing the credentials. If anyone wants to tweet from @fatiandoaterra let us know!
* Admin by Leo and Santiago right now
* Will make an account on https://www.hover.com and transfer the fatiando.org domain there once it comes up for renewal next year (and share credentials with Santiago as a backup)
* Right now under Leo's personal account
* LinkedIn page has admin features and doesn't need to share credentials
* Current admins are Leo and Santiago
* Let us know if you'd like to post from this account (glad to add more managers)
* Started drafting tensor components for point sources
* Add support for `g_en` and `g_ne`, they should return the same value.
* Testing with precoumputed values
* We could use another package to compute them.
* Escript
* Test linearity of the models
## 2021-11-26
**Time:** 14:00 (UTC)
**Participants**
* Andy
* Leo
* Santiago
**Discussion points:**
* Harmonica:
* Renamed `point_mass_gravity` for `point_gravity` :heavy_check_mark:
* Split `requirements.txt` into `env/requirements-*.txt` files :heavy_check_mark:
* Replacing pylint for flake8 extensions.
* GitHub throws me an error 500 after trying to change branch protection rules :man-shrugging:
* Renaming default branch to `main`
* Started drafting gravity tensor components for point sources in Cartesian coordinates
* More ideas for testing are welcome!
* Symmetry tests
* Dig the old Python 2.7 code for inspiration.
* Plans for 3D visualization of Harmonica models (prisms, tesseroids)
* `pyvista`?
* It's the best tool we have so far
* We would probably need some plotting functions: it's not straight-forward to plot our geometries in `pyvista`.
* Worth starting an issue and pinging Bane or one of the other devs
* Santi first try on pyvista: https://github.com/compgeolab/eql-gradient-boosted/blob/pyvista/eql_source_layouts/pyvista.py
* Start with a method to export the prism_layer to a vtk mesh: `layer.prism_layer.to_pyvista()`
* New data collection released: https://github.com/fatiando/data/releases/tag/v1.0.0
* Archived on Zenodo: https://doi.org/10.5281/zenodo.5167357
* Follow semantic versioning
* Renamed repository to `fatiando/data`
* Created a contributing guide: https://github.com/fatiando/data/pull/13
* Now need to start working on `ensaio`
* Some tips from Andy:
* Denmark gravity salt domes
* Yellowstone lake bathymetry
* Reviving the `fatiando/tutorials` repository:
* Place for notebook tutorials that use a combination of packages to achieve something
* Tutorials using (mostly) a single package should go in the package docs instead
* Exception is material for workshops
* Should still contribute the material back to the docs if possible
* Future focus should be in problem-based workshops instead of package-based workshops
* These will be used for videos and workshops.
* Adding a tutorial for gravity processing based on the excellent notebook by Santiago, Agustina, and Andrea for Transform2021
* Comments welcome: https://github.com/fatiando/tutorials/pull/1
* The tutorial uses all of our packages and the core functionality of each that is currently available:
* `pooch.retrieve`
* `boule.WGS84.normal_gravity`
* Interpolation, grid projection, and slicing from Verde
* Prism layer and equivalent sources from Harmonica
* Will use for AGU2021
* Pooch:
* Can now enable progress bars from `Pooch.fetch` and `pooch.retrieve`: https://github.com/fatiando/pooch/pull/277
* Renamed `master` to `main`. All seems OK after some fixes: https://github.com/fatiando/pooch/pull/278
## 2021-11-19
**Time:** 14:00 (UTC)
**Participants**
* Federico
* Leo
* Lu
* Santi
**Discussion points:**
* Ditch `pylint` and use `flake8` extensions: [harmonica#284](https://github.com/fatiando/harmonica/issues/284)
* Santi was experimenting with some extensions
* Advantages:
* easier to configure: just install them
* all configs inside `.flake8`, very simple
* not so strict
* Disadvantages:
* depend on a lot of small plugins that might get unsopported in the future
* some plugins are not available through conda
* won't use them
* scattered documentation, but not so bad...
* No need. Just do what Ryan does: https://github.com/Unidata/MetPy
* AGU 2021 presentation:
* Got some instructions from AGU and conveners: https://github.com/fatiando/agu2021/issues/5
* Will be an online poster session
* Includes a Q&A and a 2-3 min presentation (which I imagine we could record ahead of time, but not sure).
* Content for the digital poster can be text, images, videos, etc.
* How about:
* Add a bit of text about fatiando and each package (harvest from the website)
* Return to `fatiando/tutorials` and add the transform tutorial there since it covers all our packages.
* Update the tutorial a bit and make it runnable in Binder
* Use the newly packaged datasets in `rockhound-data` (published to Zenodo?)
* Add links, screenshots, and YouTube video of the tutorial in the poster
* Goal is to get people to our website
* Include code snipets as well (take from the tutorial)
* Add 1-2 paragraphs about development process, challenges, goals
* Big spike in views (100s) of the Verde docs coming from El Salvador. Wonder what happened there. Geolatinas?
* Move analytics to [plausible](https://plausible.io/)?
* Good to topic for next SwUng Dev Summit
* With a single account we can serve all projects under the 2nd level tier
## 2021-11-12
**Time:** 14:00 (UTC)
**Participants**
* Federico
* India
* Leo
* Santi
**Discussion points:**
* Verde seems to be having issues to build docs on CIs
* https://github.com/fatiando/verde/pull/328
* Probably related to some update on Sphinx and its plugins?
* Might be a good opportunity to switch to the new sphinx book theme
* This is broken everywhere because something updated in docutils and the old sphinx version we use is broken.
* Solution is to upgrade to the new theme and unpin sphinx.
* Gridders in Harmonica
* Equivalent source gridders in Harmonica are childs of the `verde.base.BaseGridder`. This inheritance has raised a few issues in the past: some of them were in benefit of improving Verde code, but some of them are trickier (e.g. [verde#291](https://github.com/fatiando/verde/issues/291))
* Ideas:
* Implement `BaseGridder` and `BaseGridderSph` in Harmonica: two base clases similar to `verde.base.BaseGridder` but with their particular characteristics.
* The new classes could be childs of `sklearn.base.BaseEstimator`.
* Since most of the `verde.base.BaseGridder` methods are already being overwritten in `EquivalentSources` and `EquivalentSourcesSph`, only the `filter` and the `score` methods will be repeated (in fact, for spherical gridders they should also be overwritten because of docstrings).
* We can support `profile` and `grid` methods for now (not `scatter`):
* `grid` could take only the coordinates of the grid points (as Verde will on the next major release).
* `profile` could take the `point1` and `point2` as three components tuple, with interpolation between heights as well. Easier to have `point1`, `point2` and `upward` arguments.
* Drawbacks:
* the `grid` and `profile` methods in `verde.base.BaseGridder` are quite large, we would need to repeat a bunch of code.
* Santi will prototype a base class for Harmonica.
* Don't worry about Verde too much here, feel free to experiment
* Bonus if it works with `vd.Chain`.
* Do Cartesian only first?
* Ideas for gravity tensor components:
* Make the `kernel_*_cartesian` and `kernel_*_spherical` functions general purpose: they are being used by point masses and tesseroids, but they are the integration kernels for any potential field that behaves as 1/r.
* Don't do this. The tesseroid use of this functions is just because we are numerically integrating.
* Maybe change the scope of `points_masses` to `points` or `point sources`.
* Ideas for computing several fields at a time:
* It's tricky because we would need to move the loop over the fields/kernels right to the bottom
* The `results` array should now be a 2D array: `(n_fields, n_coords)`
* This would change the whole definition of our kernel functions.
* E.g.:
```python
for l in coordinates:
for m in body:
distance = distance_cartesian(...)
for k, kernel in enumerate(kernels):
results[k, l] += masses[m] * kernel(..., distance)
```
* Implement magnetics first and then work on this.
* Magnetics use:
```python
import harmonica as hm
import verde as vd
coordinates = vd.grid_coordinates(...)
points = [easting, northing, upward]
magnetizations = [mag_east, mag_north, mag_up]
magnetic_field = hm.dipole_magnetic(coordinates, points, magnetizations)
total_field_anomaly = hm.total_field_anomaly(
magnetic_field, inclination=45, declination=30,
)
```
```python
for l in coordinates:
for m in dipole:
distance = distance_cartesian(...)
results[0, l] += magnetization[m] * kernel_mag_easting(..., distance)
results[1, l] += magnetization[m] * kernel_mag_northign(..., distance)
results[2, l] += magnetization[m] * kernel_mag_upward(..., distance)
```
* Possible improvement to gravity: compute the entire gradient
```python
g_easting, g_northing, g_z = hm.point_mass_gravity(
coordinates, points, masses, field="gradient"
)
```
Quick updates:
* `xrft` now has `pad` and `unpad` functions :tada:
* Trying out Olympic Dam magnetic:
## 2021-11-05
**Time:** 14:00 (UTC)
**Participants**
* India
* Leo
* Lu
* Santi
**Discussion points:**
* Magnetic forward modelling in Harmonica:
* Start thinking of the API. Good first go would be dipoles.
* The total field anomaly is the calculated by projecting the source magnetic field (all 3 components) in the direction of the Earth's field.
* This is the same for all sources
* And it's annoying because it requires passing the Earth's inc/dec to every function as well as the source magnetization
* Proposal:
* Don't implement total field anomaly directly.
* Instead, forward modelling functions only caculate the 3-component magnetic field.
* The total field anomaly is computed by a separate function that takes the source field and the Earth's field parameters.
* Start with gravity gradient kernels since they are needed
* This is the old implementation: https://github.com/fatiando/fatiando/blob/master/fatiando/gravmag/_prism.pyx#L94
* Can test divergence, curl, and laplace equation
* Using finite-difference derivatives
* Open issues for the different steps so that people can choose
1. Imploment the gravity gradient kernels
2. Implement the magnetic field forward modelling (single function that can return multiple components)
3. Seeing if we need to do an internal refactoring of the kernel calculations
Quick updates:
* Leo added a bunch of new data to the `rockhound-data` repository (for future sample data):
* LiDAR: https://github.com/fatiando/rockhound-data/pull/7
* GPS: https://github.com/fatiando/rockhound-data/pull/8
* Magnetic: https://github.com/fatiando/rockhound-data/pull/10
* Gravity: https://github.com/fatiando/rockhound-data/pull/9
* Working on a public domain Afghanistan magnetic dataset from https://ngdc.noaa.gov/geomag/aromag.shtml (looks promising)
* Santi has been working on the `xrft.pad` and `xrft.unpad` features: [xrft#158](https://github.com/xgcm/xrft/pull/158)
## 2021-10-29
**Time:** 14:00 (UTC)
**Participants**
* Leo
* Santiago
* Lu
* Federico
**Discussion points:**
* Should we really unify the Boule ellipsoid classes?
* Maybe a generator function instead/also?
* Simplest solution:
* Forget unification
* Stop inheriting from Ellipsoid
* Each class defines:
* basic properties (no eccentricity)
* normal gravity (requires lon, lat, h)
* surface radius (requires lon, lat)
* Coordinate conversions are only on oblate ellipsoids
* Leo will try ti implement a unified interface
* If it works we can kill the `dev` branch and current PRs
* If we really want a generator function, that is better because decisions about number of axis happen only there, not in all methods.
* We could potentially offload the coordinate conversions to https://github.com/geospace-code/pymap3d (their ellipsoid and ours are compatible)
* New pages seem to be doing well. https://www.fatiando.org/learn/ and https://www.fatiando.org/install/ are the most visited after the home page.
* Variable density tesseroids
* Lesson: never jit (decorate) inside another function
* Performance is now same as constant density and parallel works well
* Gradient boosted equivalent sources
* For some reason giving long wavelength residuals on the magnetic data
* Maybe can't resolve regionals. Try remove-compute-restore with a deep layer.
* Or maybe it's a bug.
## 2021-10-22
**Time:** 14:00 (UTC)
**Participants**
* Leo
* Santiago
* Lu
* Federico
**Discussion points:**
* Boule and Harmonica releases! (several due to slight bugs 🥲)
* Just spotted a new bug: no `EquivalentSources` class in the docs :crying_cat_face:
* New release checklist item to remember to build and check the changelog: https://github.com/fatiando/.github/pull/9
* New organization profile banner (defined here: https://github.com/fatiando/.github/blob/main/profile/README.md)
* Verbosity in Pooch: https://github.com/fatiando/pooch/pull/232
* Not everyone wants messages printed out.
* The current code uses `logging` to print messages but the way we do it is not recommended
* Open an issue with a proposal:
* Survey packages that depend on us (ping devs) to see if we break anyone's code
* Do simpler instead of fancier
* Roadmap for Harmonica v0.4
* Block-averaged sources: [#260](https://github.com/fatiando/harmonica/pull/260)
* Implement variable density tesseroids
* The first implementation might have some repeated code (cannot handle density types within Numba jitted functions)
* Implement gradient-boosted equivalent sources
* Only in Cartesian for now
* Name idea: `EquivalentSourcesGB`
* New arguments:
* `window_size`: Size of the overlapping windows
* `overlapping`: 0.5 by default, do we really need it?
* I would add it later if needed than inlcude it from the start (experience: tesseroids)
* Classmethod for estimating memory usage beforehand?
* `EquivalentSourcesGB.memory_needed(coordinates, block_size=..., window_size=...)` or
* `EquivalentSourcesGB.memory_needed(coordinates, points=points)`
* Roadmap for Harmonica v0.5
* Docs, docs, docs (and tutorials!)
* Add papers in citations instructions (the `CITATION.rst` file)
* Include references for methods implemented in particular functions
* An easy way for people to cite the original methods as well as the software
* Add math to Boule attribute documentation
## 2021-10-15
**Time:** 14:00 (UTC)
**Participants**
* Agustina
* Leo
* Lu
* Santi
**Discussion points:**
* Repo reorganization done!
* [fatiando/community](https://github.com/fatiando/community): has the meeting notes, Contributing + other guides, CoC, and tracking of org-wide maintenace tasks.
* Need to update links in packages
* [fatiando/.github](https://github.com/fatiando/community): standard issue and PR templates
* Synch the issue/PR labels across projects:
* We can make an organization default for new repos
* Edited list implemented in Pooch: https://github.com/fatiando/pooch/labels
* Migrate to other repos? There is a tool but probably easier to do it by hand since new repos will have it automatically
* Remove `duplicate`
* Rename `help wanted` to `I need help` (Use if you need help from other developers on this). Remove from current issues because it's used to mean "please code this for us" instead, which applies to basically every issue.
* Actully, just remove this since there isn't much need for it.
* There is an issue label notifier: https://github.com/marketplace/actions/issue-label-notifier
* Leo will synch this across repos
* Boule revival!
* Lu and Mariana had their first PRs merged! :tada:
* Merged the unifinished PRs by Chris into the `dev` branch
* Leo will make a push to finish the refactoring into a single `Ellipsoid` class in the next few weeks
* Dissertation student at U. Liverpool will be working on implementing gravitation of ellipsoids (inside and outside) this year :muscle:
* Reverted back to our Makefile + conda setup instead of nox
* Too much work to set up and created more confusion than help
* Coordinates on methods:
* All methods take `coordinates` as input to be uniform with Harmonica and Verde.
* Document which ones are not used and let users pass None
* Update the docs to the new theme
* Then make a release before messing with the Ellipsoid class
* From the documentation visits, it seems that rockhound is actually being used for some things.
* Completely changing it to our sample data package would cause these existing users' codes to break all of a sudden
* Maybe we should use another name and archive RockHound.
* In favour?
* Yes!
* If so, name suggestions:
* `fatiando_data`: boring and informative
* `carpincho`: sounds nice, South American, says absolutely nothing about the package
* More! Search https://pypi.org/ to see if taken
* New place for hosting large data: https://plus.figshare.com/
## 2021-10-01
**Time:** 14:00 (UTC)
**Participants**
* Leo
* Lu
* Federico
* Santiago
**Discussion points:**
* Boule:
* Leo may have a student working on implementing ellisoid gravitation equations (inside and outside the ellipsoid)
* Will try to make a push on this to get it ready for a new release
* Things to do:
* Revert back to our Makefile/conda/etc workflow instead of nox. Turns out it complicates things more than needed and creates a bit of a disconnect with other projects.
* Triage the current PRs and see what can be merged and what needs some work. There are some that may be outdated and need some re-thinking.
* Finish looking at Lu's coordiante system tutorial
* Community:
* Add another point of contact to the CoC: https://github.com/fatiando/contributing/issues/29
* Too many repositories:
* https://github.com/fatiando/meeting-notes with info on call times and archived notes
* https://github.com/fatiando/maintenance with maintenance tasks but not the actual maintainers guide
* https://github.com/fatiando/.github/ Issue and PR templates for the entire organization.
* https://github.com/fatiando/contributing with our contributing, authorship, and maintainer guides + CoC
* **Could we merge these into just 1 or 2?**
* Proposal:
* Keep `.github` with issue and PR templates (also where the front page readme needs to live)
* Merge `meeting-notes`, `maintenance`, `contributing` into a single `community` repo. There may be some broken links but GitHub redirects when renaming repos.
* Accepted. Leo will implement these changes later today.
* Pooch:
* Release v1.5.2 once https://github.com/fatiando/pooch/pull/266 is merged
* Publication of Analytics on Santi's Thesis
* Will use the analytics data for the website about number of visitors and location (all anonymous)
* Website analytics:
* Leo is looking at Plausible
* Would be great to self-host on [software underground](https://softwareunderground.org/)
* Can be shared by other OSS projects
* Leo will post about it in the `#swung_org` channel to see if they bring this up in the next board meeting
* Self-hosting docs: https://plausible.io/docs/self-hosting
## 2021-09-24
**Time:** 14:00 (UTC)
**Participants** (please add your name)
* Leo
* Santi
* Federico
**Discussion points:**
* Santi added cards for the authors of each project: https://www.fatiando.org/about/
* Start with Harmonica
* Download images and convert them to .jpg? Check if they are really that big.
* Enable `pooch.retrieve` to download to the default Downloads folder
* It's not worth doing
* Might be headache to test
* Very likely to be source of future problems
* Best compromise:
* Write a `get_default_downloads_dir()` or something that returns the path to the default Downloads folder.
* Bug report on Pooch: https://github.com/fatiando/pooch/issues/264
* Add support for LZW decompression: https://github.com/fatiando/pooch/issues/263
* Variable density tesseroids:
* Repeat code it's ok for a first implementation that doesn't hit efficiency
* We can iterate in the future.
* Gridding bathymetry data
* Finding another set of data (easier to get or with more coverage) that could correlate with bathymetry.
## 2021-09-17
**Time:** 14:00 (UTC)
**Participants** (please add your name)
* Leo
* Lu
* Federico
**Discussion points:**
* New sample datasets in https://github.com/fatiando/rockhound-data:
* We have:
* Bathymetry https://github.com/fatiando/rockhound-data/pull/2
* Global gravity, topography, geoid: https://github.com/fatiando/rockhound-data/pull/4
* Ground gravity (Souther Africa + Australia)
* LIDAR (not incorporated but Leo found a nice one from Canada)
* Need:
* Airborne gravity (BAS and GRAV-D)
* Magnetics (BAS - ADMAP, GA) https://doi.pangaea.de/10.1594/PANGAEA.892724 ; GA: https://portal.ga.gov.au/persona/gadds
* GPS (Leo write UNAVCO about license) https://doi.pangaea.de/10.1594/PANGAEA.889923 (This one with vertical velocity only)
* Radar IceBridge https://nsidc.org/data/icebridge/data_summaries.html
* Organization profile READMEs: https://twitter.com/github/status/1438842052117204993?s=19
## 2021-09-10
**Time:** 14:00 (UTC)
**Participants** (please add your name)
* Leo
* Santi
* Agustina
**Discussion points:**
* Harmonica housekeeping
* Add option to set EQLHarmonic points to constant depth [#236](https://github.com/fatiando/harmonica/pull/236)
* Generate version string on `_version.py` on build [#237](https://github.com/fatiando/harmonica/pull/237)
* Allow prism_layer to take xarray objects as arguments [#243](https://github.com/fatiando/harmonica/pull/243) < **Merge first**
* Transform require_numba decorator into a global variable [#245](https://github.com/fatiando/harmonica/pull/245)
* Leo will review all of these after the meeting
* Refactoring Tesseroids [#244](https://github.com/fatiando/harmonica/pull/244)
* Small fixes on docstrings
* Refactor code to follow pylint guidelines
* Enable parallelization through Numba: running 4-5 faster on my 6 cores CPU :tada:
* Parallelization can be turned off through `parallel=False`, following the same interface as the other forward models
* Working on the implementation of variable density tesseroids in Harmonica
* Run density-based discretization as a preprocessing step at Python level
* Use `scipy.optimize.minimize_scalar` to maximize the absolute max difference between normalized density and straight line, instead of defining an array of radii to explore (see [Soler et al., 2019](https://doi.org/10.31223/osf.io/3548g))
**Quick updates:**
* New fatiando.org design: https://github.com/fatiando/website/pull/28 (comments and suggestions welcome)
* Agustina showed how to use the Firefox developer tools to find accessibility issues
* There are many :white_frowning_face:
* But at least it's not worse than the current website
* Will merge the PR as is
* Agustina will open a few issues about accessibility so we can keep track
* Follow up PRs will fix these
## 2021-09-03
**Time:** 14:00 (UTC)
**Participants** (please add your name)
* Leo
* Santiago
* Lu
**Discussion points:**
* Assign more formal roles within Fatiando:
* Helps clarify who is who and who is responsible for what. Right now, it's not really clear unless you look through discussions and try to infer who makes decisions.
* We could really use a "social media manager" (handling twitter/linkedin/SwUng/etc) and a "community manager" (help setup forum, welcome people to slack, lead community calls) for example.
* What roles do we need initially?
* Social media
* Releases
* New features
* Patches
* Talks
* Events
* Examples
* Engage in conversations
* Highlight publications using the project
* Promote projects in the ecosystem
* Website
* Maintain up dated
* Add/remove people from lists
* Redesign (one time thing)
* Community lead
* Answer issues and prs quickly
* Probalby overlap with maintainers but not all maintainers should be community leads
* Compile list of new issues, stale issues, PRs, etc for the weekly meetings
* Maintainers (for each project)
* Reviewing and merging PRs
* Will have full admin rights to the repo
* Make releases (compile changelog, Zenodo, etc)
* Contributor (for each project)
* Code
* Docs
* Answer questions
* Largely a list of contributors from GitHub (AUTHORS.md?)
* What is the process for gaining responsibility?
* Good amount of engagement and contributions is a requirement
* Need somewhat regular contributions
* Maintainer initiates discussion with rest of mainteners to invite someone for promotion
* Promote early rather than late
* What is the process for stepping down from roles?
* Notify the team
* Find a replacement (if possible)
* Go be happy
* Include time off in the document:
* Periods where nothing happens for a little while (e.g. end of the year)
* Draft a governance document.
* Use SimPEG for inspiration: https://github.com/simpeg/community/blob/main/governance.md
* Specify expectations for each role
* Santi can reach out Geolatinas for input
* Where?
* Website: page saying who is who and linking to the governance document
* https://github.com/fatiando/contributing: document with what the roles and processes are but not who is occupying them
* Would allow us to have a team/gallery to showcase who works on fatiando.
* Like this maybe: https://docs.generic-mapping-tools.org/dev/devdocs/team.html
* Source: https://github.com/GenericMappingTools/gmt/pull/5589/files#diff-0048a43cdc8224c75c38cce087aca2e653311c65140402916c132ae83ba1f817R26
* Functions that take "arrays" should also support `DataArray`, pandas `Series`, etc? Should we make it explicit only when we truly support them?
* See https://github.com/fatiando/verde/issues/330 and https://github.com/fatiando/harmonica/pull/242
* Use `np.asarray` and add tests for this with `pytest.parametrize` to check that it works
**Quick updates:**
* Old documentation pages now all redirect to https://legacy.fatiando.org (v0.5 docs)
* Mailing list is deactivated and there is a message telling people to go to Slack
* `fatiando/website` now uses `main` as the default branch. Update local clones with:
```
git branch -m master main
git fetch origin
git branch -u origin/main main
git remote set-head origin -a
```
## 2021-08-27
**Time:** 14:00 (UTC)
**Participants** (please add your name)
* Leo
* Agustina
* Lu
* Santiago
**Discussion points:**
* Keep guides (contributor, maintenance, etc) only in the [`fatiando/contributing`](https://github.com/fatiando/contributing) repository.
* Right now, we have a complicated setup that pushes changes to the files in this repo to the other repositories.
* This relies on TravisCI, which we largely don't use anymore.
* No one had the time to go in and update to GitHub Actions.
* It doesn't allow projects to include custom changes to the files (since it would cause conflicts).
* Proposal:
* The files on repositories will remain
* They will have a small message linking to the original files on `contributing`
* They can include extra instructions that are only for that particular project
* Removes the need for the CI setup in `contributing`
* Having a lot of stuff on the project creates barriers to entry.
* Authorship guidelines should be easily findable
* System of layers (from mainteiners to new users):
* Maintenance
* Contributing + authorship
* Documentation, tutorials, how to contact devs and other users
* Do we want:
* 1 central document with links to it on all repos
* no central document and individual guides on all repos
* Proposal 2.0:
* Kill the central document and it's TravisCI
* This will break the Welcome Bot
* TODO:
* Open a issue on `fatiando/contributing`
* May need to copy the welcome bot script from `fatiando/.github` to individual repos
* People asking help through mailing list while trying to install `fatiando` on Python2.7
* Probably they are getting instructions for installation and how to contact us from fatiando.org/v0.5
* Should we add a big disclaimer on the fatiando.org/v0.5 website about it being deprecated?
* TODO:
* Stop serving everything below v0.5 and also dev/
* Serve 0.5 under legacy.fatiando.org
* Add a banner to the 0.5 pages point to the main Fatiando website
* Future: Add banners to old versions of the docs in all libraries. See how numpy and matplotlib do this.
* Create an issue in `fatiando/maintenance` for this
* Disable posting to the mailing list. Maybe automated reply might be worth the effort. https://support.google.com/groups/answer/9792589?hl=en
**Quick updates:**
* Pooch v1.5.0 released! https://www.fatiando.org/pooch/latest/changes.html#version-1-5-0
* New LinkedIn page: https://www.linkedin.com/company/fatiando/
* Be more careful when moving internal functions. They can break a lot of code: https://github.com/fatiando/pooch/issues/252
## 2021-08-06
**Time:** 14:00 (UTC)
**Participants** (please add your name)
* Leo Uieda
* Santi Soler
* Lu Li
**Discussion points:**
* Fast inverse square root: https://www.lomont.org/papers/2003/InvSqrt.pdf
* We calculate `1 / sqrt(x)` quite often in gravmag modelling and equivalent sources.
* This is usually the largest bottle neck for point sources
* This method is supposed to be 4-5x faster than doing `sqrt` and then division
* Has some crazy hacks and may be hard to generalize (implementation depends on number of bits and float format)
* Could be a very impactful addition to [Bottleneck](https://bottleneck.readthedocs.io/en/latest/index.html) or [numbagg](https://github.com/numbagg/numbagg)
* Probably not for the faint of heart :slightly_smiling_face:
* Redesign of the `grid` method on Verde gridders (PR [#326](https://github.com/fatiando/verde/pull/326))
* Let `grid` to accept the `coordinates` of a preexisting grid (2d meshgrids or 1d arrays)
* Benefits:
* BaseGridder can be easily generalized to any number of coordinates. EQLs for example, with `easting`, `northing`, `upwards`
* Proposal: deprecate `spacing`, `shape`, `region`: don't create the coordinates inside `grid`
* Benefits:
* Users can inspect the coordinates before running the predictions
* Simplifies the arguments of `grid`
* _Do a single task and do it well_
* Name of the `coordinates` argument: `coordinates` vs `grid_coords`
* Argument should be `coordinates`
* But variables in examples should be `grid_coords` since there is likely already a `coordinates` variable
* TODO:
* Issue 1:
* Add the coordinates argument to `grid` but make it optional for now (Santi has a PR for this already)
* Include a `FututeWarning` that things will change in 2.0
* Issue 2:
* Update the examples to use `grid_coordinates` and pass them in to `grid` with a warning that this will be the only way in the future
* Issue 3:
* Remove the arguments of `grid_coordinates` from `grid`
* Make `coordinates` a required argument
* Open issues for these things
* Verde
* Start a new `dev` branch for deprecations and backward-breaking changes
* Name: `dev-2.0`
* Make PRs that cause breaking changes against this branch.
* https://github.com/fatiando/verde/tree/dev-2.0
* Which git workflow should we use for it?
* Should `dev` be always updated with `master`? (would create a lot of 'Merge...' commits)
* or we should merge `master` on `dev` right before the next major release? (we might face conflicts and some bugs right before the release)
* `git rebase` instead of `git merge` to avoid merge commits.
* Probably do it all when we will release to avoid prolonging our suffering
* TODO:
* Create a milestone for 2.0
* Create issue for removing FutureWarnings form the code
* Assign existing issues to the milestone
* :white_frowning_face: https://github.com/fatiando/verde/milestone/5
* Is `conda install -c conda-forge verde harmonica boule pooch rockhound` working fine?
* Deactivate the mailing list once and for all
* Create CI job (weekly?) that setups miniconda and Anaconda, installs everything at the latest version, and run tests.
* Could be a new repo just for this or https://github.com/fatiando/maintenance/
* Documentation https://hackmd.io/wPxyX6c2ROGjcMvxDL0TmQ?view
* See https://documentation.divio.com/
* Documentation sources are here: https://github.com/fatiando/harmonica/tree/master/doc
* To add a new page:
1. Create a `.rst` file in that folder.
2. Add the new file to the `toctree` entry in `index.rst`
* Focus on theory side of the gravity disturbance
* Open the pull request as soon as you have something (anything) which helps when asking for help or feedback
**Quick updates:**
* Abstract submitted for our AGU2021 talk: https://github.com/fatiando/agu2021
* Leo will be touch later in the year when the program is finalized and we know exactly what to present.
* Likely to record a video to acompany the live talk. Contributions from the community would be very welcome (video, audio, images, etc)
* kdenlive can apparently make subtitltes from a script. Can embed into mp4 and youtube/figshare pick it up.
* https://mastodon.uy/web/statuses/106704380721771095
## 2021-07-23
**Time:** 14:00 (UTC)
**Participants** (please add your name)
* Leo Uieda
* Santi Soler
* Lu Li
**Discussion points:**
* AGU talk
* Leo was invited to give a talk at a session
* Asked to talk about the project and also touch on the maintenance/sustainability
* Outline our process for adding new features:
* Docs first > internface > tests > implementation
* Tie into Unix philosophy. Focus on outcome, not implementation. ([Brandon Rhodes talk about Unix and Python](https://www.youtube.com/watch?v=zFMdhXYlFfY))
* Can submit up to 3 abstracts so if there is anything else we want to submit, we can.
* Parker-Oldenburg inversion:
* Not on the immediate roadmap but welcome if anyone wants to code it
* Can suffer from instability and may require filtering
* Check if coordinates are latitude, longitude prior to FFT and yell at the user if they are (can be done across all FFT-based functions right now).
* Should be straight forward to implement.
* Next meeting discuss roadmap and wishlist for Verde 2.0
## 2021-07-16
**Time:** 14:00 (UTC)
**Participants** (please add your name)
* Leo Uieda
* Santi Soler
**Discussion points:**
* Leo was invited to write a short news article for the Brazilian Geophysical Society about Fatiando (in Portuguese). Have to do this by August and will send around for some feedback.
* Split up the `verde.ScipyGridder` into 3:
* `vd.NearestNeighbors`: NN interpolation using `cKDTree` or `pykdtree`. Option to average K nearest neighbors. Include a search radius as well (`max_distance` in the `query` method).
* `vd.LinearGridder` and `vd.CubicGridder`: Wrap the scipy methods and add the option `extrapolate` (default to `True`) to fill NaNs with nearest neighbors (default to 1 neighbor).
* Can help move the focus away from Green's functions.
* Add these classes then deprecate `ScipyGridder` to remove in 2.0.0.
* Roadmap for Harmonica
* Focus for v0.3:
* MVP: Processing raw gravity -> Residual grid -> transformations (FFT or EQL)
* Restrict to Cartesian
* Missing components:
* EQLHarmonicBoost
* Finish FFT filters (explain using gridded data):
* Upward derivative (horizontal uses xarray)
* Upward continuation (optional)
* Total gradient amplitude (optional)
* Total horizontal gradient (optional)
* Theory:
* Coordinate systems
* Gravity disturbance and corrections explained
* Tutorials:
* Basically what we did for Transform (but maybe split it up)
* Tutorial on transformations (with EQLs and FFT and when to use which)
* Documentation!
* Good docstrings but no tutorials
* Santi's thesis will be a good basis for the theory
* What is the workflow and the theory
* Wishlist:
* Tesseroids with variable density
* Gradient boosted EQLs
* FFT solutions: derivatives, TGA, reduction to the pole, upward continuation
* IGRF:
* Coeffs are here: https://www.ngdc.noaa.gov/IAGA/vmod/igrf.html
* Calculation is actually really easy
* No point in wrapping a fortran 77 code
* Can use outputs from the fortran to test
* Need to derive the equations for the x, y, z components of the SH model
## 2021-07-02
**Time:** 14:00 (UTC)
**Participants** (please add your name)
* Santi Soler
* Lu Li
**Discussion points:**
* Some test for `xrft`: https://nbviewer.jupyter.org/github/LL-Geo/PFToolbox/blob/master/xrft_note.ipynb
* Looks like we can direct use `fft` and ignore `dft`
* We can save the coordinate information from original grid, and pass it back for the `ifft`
* Santi opened a PR on `xrft` for padding grid and coordinates: https://github.com/xgcm/xrft/pull/158
* They asked us to upstream the pad_coordinates to xarray
* Start with supporting only padding through number of points (we can work on lenght of coordinates in the future)
* Set up a Bot for reminders on Fatiando calls
* One day before
* One hour before
* Tweet reminders
## 2021-06-25
**Time:** 14:00 (UTC)
**Participants** (please add your name)
* Leo
* Santiago
* Agustina
* Lu
**Discussion points:**
* Interaction with `xrft`:
* Santi has opened an Issue to start engaging the `xrft` community: [xrft #154](https://github.com/xgcm/xrft/issues/154)
* Naming of the default function for applying FFT: `fft` vs `dft`
* Devs agree and seem to favour `fft`
* Requires improvement of docs around true_phase and true_amplitude
* Coordinates shifts: `lag` argument in `fft` and including `lag` in the Fourier transform
* Padding implementation:
* They asked Santi to open a PR in `xrft` with a padding implementation
* Wouldn't be better to enhance the `xarray.DataArray` pad for also pad the coordinates if they are evenly spaced?
* Limitation of current padding in xarray:
* By default, pads the coordinates with nans
* Make sense because there is no way to know what the coordinates are if the spacing is irregular
* Could do it if the coordinates are regular
* Pitch:
* Implement the padding coordinates in xrft for now
* Basically check for nans and then add coordinates as needed
* When it's implemented in xarray, we can shift to using that instead
**Quick updates:**
* Leo started preparing some data for RockHound to use as new sample data in our docs:
* [Bathymetry from the Caribbean](https://nbviewer.jupyter.org/github/fatiando/rockhound-data/blob/bathymetry/bathymetry-caribbean/prepare.ipynb)
* [Global gravity and topography](https://nbviewer.jupyter.org/github/fatiando/rockhound-data/blob/global_grids/gravity-topography-global/prepare.ipynb)
## 2021-06-18
**Time:** 14:00 (UTC)
**Participants** (please add your name)
* Santi Soler
* Agustina Pesce
* Leo Uieda
**Discussion points:**
* Reconcile the Harmonica and Verde gridding API?
* The EQLHarmonic and Spline APIs are slightly different:
* `.grid` takes the extra `upward` coordinate in EQLHarmonic
* Done for a more intuitive and descriptive interface
* But it means we can't use anything that relies on the Verde API, like `vd.Chain`, `vd.Vector`, and the proposed `vd.Derivative`.
* Actually, we can but the `Chain.grid` and `EQLHarmonica.grid` methods are different, making it a bit awkward to use.
* Is there a way to keep the `upward` argument and make the APIs compatible again (even if it means changing the Verde API)?
* PROPOSAL:
* Deprecate `region`, `shape`, `shaping` from the `grid` method, make it only take the actual grid coordinates, either from `vd.grid_coordinates` or from an `xarray` object (as 1d arrays).
* Add a new property to gridders that documents the number of dimensions (`n_dims`?)
* Make the `Chain` to inherit the maximum `n_dim` from the gridders in the chain
* Add a `grid` method to `Chain` that complains if the passed coordinates are less than `n_dim`
* Implement this in a `dev-2.0` branch that will be merged before major release
* Invite people from `xrft` to next meeting
## 2021-06-11
**Time:** 14:00 (UTC)
**Participants** (please add your name)
* Leo
* Santi Soler
* Lu
* Andrea
* Agustina
**Quick updates**
* Pooch is using the `sphinx-book-theme` and trying out a new docs layout: [#237](https://github.com/fatiando/pooch/pull/237)
* Pooch PR to download from Zenodo and figshare: https://github.com/fatiando/pooch/pull/241
* Will be useful for refactoring RockHound into our sample data repository
* Can put every dataset in the same Zenodo/figshare archive and download individual files
**Longer discussion**
* Frequency-domain filters in Harmonica: [#238](https://github.com/fatiando/harmonica/pull/238)
* Two type of functions:
* Filters: work only in frequency domain. I would like to create a `filter` module where they should live.
* Takes FFT from xrft and outputs still in frequency domain
* Mostly for internal use but can be used by others as well
* Filters will be public but not the wrapped FFT functions
* Transform and processing functions (derivatives, upward continuation, RTP, etc).
* Their arguments and outputs are always in spatial domain.
* They call xrft, use the filters, and then inverse the transform
* We should engage with `xrft` and discuss some things regarding their implementation.
* What `true_amplitude` and `true_phase` mean?
* Why `lag` is not working properly?
* Inverse transform removes the absolute value of the coordinates so they are centered at zero
* Need to shift the coordinates in our own code
* The `lag` seems like it does this but doesn't work for some reason.
* Check if the coordinates of input and output match to see if we need to shift
* Why having `dft` and `fft`? What are the differences between them?
* Santi may have found some bugs in the phase
* Engaging with them instead of wrapping everything locally is better for the community as a whole
* Open an issue to start the conversation
* Derivatives:
* Should we add the option to choose between finite difference or FFT derivative for easting and northing?
* The finite difference can be achieved by `xr.differentiate`.
* For now, remove the horizontal ones from the PR and leave just the vertical
* In the docs, use xarray for the horizontal derivatives
* How about using an argument to specify the direction?
* Better to have 1 function in the docs
* But can't auto-complete the direction names in notebooks
* Directional derivatives could be a usecase for FFT based horizontal derivatives
* Lu will provide a reference/use case for this that we could base the implementation on
* Could also be done with finite-differences
* The best solution might be: ? $$ F(DD)=F(g)*(u*i*sin(\theta)+v*i*cos(\theta))$$ So that we could combine the horizontal derivatives with directional derivatives together by different angle
* Ref 1: Hang, N. T. T. (2017). Application of directional derivative method to determine boundary of magnetic sources by total magnetic anomalies. VIETNAM JOURNAL OF EARTH SCIENCES, 39(4), 360-375.
* Ref 2: Stewart, I. C., & Miller, D. T. (2018). Directional tilt derivatives to enhance structural trends in aeromagnetic grids. Journal of Applied Geophysics, 159, 553-563.
* Need to engage with xarray on their padding function.
* It doesn't set coordinates (just NaNs) for the padded region
* This makes xrft angry
* Not trivial to implement in xarray since their Datasets aren't necessarily regular grids
Sign in with Wallet
Connect another wallet