Try   HackMD

Fatiando Development Calls 2021

📱 Join the video call: https://swu.ng/chateau (Gather Town)

📅 Shared calendar: https://www.fatiando.org/calendar

IMPORTANT: Everyone is required to follow our Code of Conduct when participating in the Fatiando community. Please review it carefully.

Development calls are open to everyone! Here we discuss routine project development: implementation details, code refactoring, maintenance tasks, documentation, etc. Discussions tend to be more technical than our Community Calls.

Everyone is encouraged to edit the notes below.

2021-12-17

Time: 14:00 (UTC)

Discussion points:

2021-12-10

Time: 14:00 (UTC)

Participants

  • Leo
  • Lu
  • Santi

Discussion points:

2021-12-03

Time: 14:00 (UTC)

Participants

  • Agustina
  • Lu
  • Santi

Discussion points:

  • Harmonica release :tada: :confetti_ball:
    • Future releases: more work on the docs, tutorials.
    • Add tutorial for gradient-boosted eqs using a mid-size dataset (a few hundred thousand data points).
  • Prism layer to pyvista
  • New magnetic data from the Osborne Mine, Australia:
  • New ProtonMail account so we can sign up for services without depending on Leo
    • fatiandoaterra@protonmail.com
      • Controlled by Leo and Santiago at the moment
      • Includes 2FA and a strong password
      • Credentials shared between admins by GPG encrypted messages and stored in secure password managers
    • Used for our Twitter account
      • Can give tweeting permissions through TweetDeck without sharing the credentials. If anyone wants to tweet from @fatiandoaterra let us know!
      • Admin by Leo and Santiago right now
    • Will make an account on https://www.hover.com and transfer the fatiando.org domain there once it comes up for renewal next year (and share credentials with Santiago as a backup)
      • Right now under Leo's personal account
    • LinkedIn page has admin features and doesn't need to share credentials
      • Current admins are Leo and Santiago
      • Let us know if you'd like to post from this account (glad to add more managers)
  • Started drafting tensor components for point sources
    • Add support for g_en and g_ne, they should return the same value.
    • Testing with precoumputed values
      • We could use another package to compute them.
        • Escript
    • Test linearity of the models

2021-11-26

Time: 14:00 (UTC)

Participants

  • Andy
  • Leo
  • Santiago

Discussion points:

  • Harmonica:
    • Renamed point_mass_gravity for point_gravity :heavy_check_mark:
    • Split requirements.txt into env/requirements-*.txt files :heavy_check_mark:
    • Replacing pylint for flake8 extensions.
      • GitHub throws me an error 500 after trying to change branch protection rules :man-shrugging:
    • Renaming default branch to main
    • Started drafting gravity tensor components for point sources in Cartesian coordinates
      • More ideas for testing are welcome!
      • Symmetry tests
      • Dig the old Python 2.7 code for inspiration.
  • Plans for 3D visualization of Harmonica models (prisms, tesseroids)
  • New data collection released: https://github.com/fatiando/data/releases/tag/v1.0.0
  • Reviving the fatiando/tutorials repository:
    • Place for notebook tutorials that use a combination of packages to achieve something
    • Tutorials using (mostly) a single package should go in the package docs instead
      • Exception is material for workshops
      • Should still contribute the material back to the docs if possible
      • Future focus should be in problem-based workshops instead of package-based workshops
    • These will be used for videos and workshops.
    • Adding a tutorial for gravity processing based on the excellent notebook by Santiago, Agustina, and Andrea for Transform2021
    • Comments welcome: https://github.com/fatiando/tutorials/pull/1
    • The tutorial uses all of our packages and the core functionality of each that is currently available:
      • pooch.retrieve
      • boule.WGS84.normal_gravity
      • Interpolation, grid projection, and slicing from Verde
      • Prism layer and equivalent sources from Harmonica
    • Will use for AGU2021
  • Pooch:

2021-11-19

Time: 14:00 (UTC)

Participants

  • Federico
  • Leo
  • Lu
  • Santi

Discussion points:

  • Ditch pylint and use flake8 extensions: harmonica#284
    • Santi was experimenting with some extensions
    • Advantages:
      • easier to configure: just install them
      • all configs inside .flake8, very simple
      • not so strict
    • Disadvantages:
      • depend on a lot of small plugins that might get unsopported in the future
      • some plugins are not available through conda
        • won't use them
      • scattered documentation, but not so bad…
  • AGU 2021 presentation:
    • Got some instructions from AGU and conveners: https://github.com/fatiando/agu2021/issues/5
    • Will be an online poster session
    • Includes a Q&A and a 2-3 min presentation (which I imagine we could record ahead of time, but not sure).
    • Content for the digital poster can be text, images, videos, etc.
    • How about:
      • Add a bit of text about fatiando and each package (harvest from the website)
      • Return to fatiando/tutorials and add the transform tutorial there since it covers all our packages.
      • Update the tutorial a bit and make it runnable in Binder
      • Use the newly packaged datasets in rockhound-data (published to Zenodo?)
      • Add links, screenshots, and YouTube video of the tutorial in the poster
      • Goal is to get people to our website
      • Include code snipets as well (take from the tutorial)
      • Add 1-2 paragraphs about development process, challenges, goals
  • Big spike in views (100s) of the Verde docs coming from El Salvador. Wonder what happened there. Geolatinas?
  • Move analytics to plausible?
    • Good to topic for next SwUng Dev Summit
      • With a single account we can serve all projects under the 2nd level tier

2021-11-12

Time: 14:00 (UTC)

Participants

  • Federico
  • India
  • Leo
  • Santi

Discussion points:

  • Verde seems to be having issues to build docs on CIs
    • https://github.com/fatiando/verde/pull/328
    • Probably related to some update on Sphinx and its plugins?
    • Might be a good opportunity to switch to the new sphinx book theme
    • This is broken everywhere because something updated in docutils and the old sphinx version we use is broken.
    • Solution is to upgrade to the new theme and unpin sphinx.
  • Gridders in Harmonica
    • Equivalent source gridders in Harmonica are childs of the verde.base.BaseGridder. This inheritance has raised a few issues in the past: some of them were in benefit of improving Verde code, but some of them are trickier (e.g. verde#291)
    • Ideas:
      • Implement BaseGridder and BaseGridderSph in Harmonica: two base clases similar to verde.base.BaseGridder but with their particular characteristics.
        • The new classes could be childs of sklearn.base.BaseEstimator.
        • Since most of the verde.base.BaseGridder methods are already being overwritten in EquivalentSources and EquivalentSourcesSph, only the filter and the score methods will be repeated (in fact, for spherical gridders they should also be overwritten because of docstrings).
        • We can support profile and grid methods for now (not scatter):
          • grid could take only the coordinates of the grid points (as Verde will on the next major release).
          • profile could take the point1 and point2 as three components tuple, with interpolation between heights as well. Easier to have point1, point2 and upward arguments.
        • Drawbacks:
          • the grid and profile methods in verde.base.BaseGridder are quite large, we would need to repeat a bunch of code.
    • Santi will prototype a base class for Harmonica.
      • Don't worry about Verde too much here, feel free to experiment
      • Bonus if it works with vd.Chain.
      • Do Cartesian only first?
  • Ideas for gravity tensor components:
    • Make the kernel_*_cartesian and kernel_*_spherical functions general purpose: they are being used by point masses and tesseroids, but they are the integration kernels for any potential field that behaves as 1/r.
    • Don't do this. The tesseroid use of this functions is just because we are numerically integrating.
    • Maybe change the scope of points_masses to points or point sources.
  • Ideas for computing several fields at a time:
    • It's tricky because we would need to move the loop over the fields/kernels right to the bottom
    • The results array should now be a 2D array: (n_fields, n_coords)
    • This would change the whole definition of our kernel functions.
    • E.g.: 
      ​​​​​​​​for l in coordinates:
​​​​​​​​    for m in body:
​​​​​​​​        distance = distance_cartesian(...)
​​​​​​​​        for k, kernel in enumerate(kernels):
​​​​​​​​            results[k, l] += masses[m] * kernel(..., distance)
    • Implement magnetics first and then work on this.
  • Magnetics use: 
    ​​​​import harmonica as hm
​​​​import verde as vd
​​​​
​​​​coordinates = vd.grid_coordinates(...)
​​​​
​​​​points = [easting, northing, upward]
​​​​magnetizations = [mag_east, mag_north, mag_up]
​​​​
​​​​magnetic_field = hm.dipole_magnetic(coordinates, points, magnetizations)
​​​​total_field_anomaly = hm.total_field_anomaly(
​​​​    magnetic_field, inclination=45, declination=30,
​​​​)

    ​​​​for l in coordinates:
​​​​    for m in dipole:
​​​​        distance = distance_cartesian(...)
​​​​        results[0, l] += magnetization[m] * kernel_mag_easting(..., distance)
​​​​        results[1, l] += magnetization[m] * kernel_mag_northign(..., distance)
​​​​        results[2, l] += magnetization[m] * kernel_mag_upward(..., distance)
  • Possible improvement to gravity: compute the entire gradient
g_easting, g_northing, g_z = hm.point_mass_gravity(
    coordinates, points, masses, field="gradient"
)

Quick updates:

  • xrft now has pad and unpad functions :tada:
  • Trying out Olympic Dam magnetic:

2021-11-05

Time: 14:00 (UTC)

Participants

  • India
  • Leo
  • Lu
  • Santi

Discussion points:

  • Magnetic forward modelling in Harmonica:
    • Start thinking of the API. Good first go would be dipoles.
    • The total field anomaly is the calculated by projecting the source magnetic field (all 3 components) in the direction of the Earth's field.
    • This is the same for all sources
    • And it's annoying because it requires passing the Earth's inc/dec to every function as well as the source magnetization
    • Proposal:
      • Don't implement total field anomaly directly.
      • Instead, forward modelling functions only caculate the 3-component magnetic field.
      • The total field anomaly is computed by a separate function that takes the source field and the Earth's field parameters.
    • Start with gravity gradient kernels since they are needed
    • This is the old implementation: https://github.com/fatiando/fatiando/blob/master/fatiando/gravmag/_prism.pyx#L94
    • Can test divergence, curl, and laplace equation
      • Using finite-difference derivatives
    • Open issues for the different steps so that people can choose
      1. Imploment the gravity gradient kernels
      2. Implement the magnetic field forward modelling (single function that can return multiple components)
      3. Seeing if we need to do an internal refactoring of the kernel calculations

Quick updates:

2021-10-29

Time: 14:00 (UTC)

Participants

  • Leo
  • Santiago
  • Lu
  • Federico

Discussion points:

  • Should we really unify the Boule ellipsoid classes?
    • Maybe a generator function instead/also?
    • Simplest solution:
      • Forget unification
      • Stop inheriting from Ellipsoid
      • Each class defines:
        • basic properties (no eccentricity)
        • normal gravity (requires lon, lat, h)
        • surface radius (requires lon, lat)
      • Coordinate conversions are only on oblate ellipsoids
    • Leo will try ti implement a unified interface
    • If it works we can kill the dev branch and current PRs
    • If we really want a generator function, that is better because decisions about number of axis happen only there, not in all methods.
    • We could potentially offload the coordinate conversions to https://github.com/geospace-code/pymap3d (their ellipsoid and ours are compatible)
  • New pages seem to be doing well. https://www.fatiando.org/learn/ and https://www.fatiando.org/install/ are the most visited after the home page.
  • Variable density tesseroids
    • Lesson: never jit (decorate) inside another function
    • Performance is now same as constant density and parallel works well
  • Gradient boosted equivalent sources
    • For some reason giving long wavelength residuals on the magnetic data
    • Maybe can't resolve regionals. Try remove-compute-restore with a deep layer.
    • Or maybe it's a bug.

2021-10-22

Time: 14:00 (UTC)

Participants

  • Leo
  • Santiago
  • Lu
  • Federico

Discussion points:

  • Boule and Harmonica releases! (several due to slight bugs 🥲)
  • New organization profile banner (defined here: https://github.com/fatiando/.github/blob/main/profile/README.md)
  • Verbosity in Pooch: https://github.com/fatiando/pooch/pull/232
    • Not everyone wants messages printed out.
    • The current code uses logging to print messages but the way we do it is not recommended
    • Open an issue with a proposal:
      • Survey packages that depend on us (ping devs) to see if we break anyone's code
      • Do simpler instead of fancier
  • Roadmap for Harmonica v0.4
    • Block-averaged sources: #260
    • Implement variable density tesseroids
      • The first implementation might have some repeated code (cannot handle density types within Numba jitted functions)
    • Implement gradient-boosted equivalent sources
      • Only in Cartesian for now
      • Name idea: EquivalentSourcesGB
      • New arguments:
        • window_size: Size of the overlapping windows
        • overlapping: 0.5 by default, do we really need it?
          • I would add it later if needed than inlcude it from the start (experience: tesseroids)
      • Classmethod for estimating memory usage beforehand?
        • EquivalentSourcesGB.memory_needed(coordinates, block_size=..., window_size=...) or
        • EquivalentSourcesGB.memory_needed(coordinates, points=points)
  • Roadmap for Harmonica v0.5
    • Docs, docs, docs (and tutorials!)
  • Add papers in citations instructions (the CITATION.rst file)
    • Include references for methods implemented in particular functions
    • An easy way for people to cite the original methods as well as the software
  • Add math to Boule attribute documentation

2021-10-15

Time: 14:00 (UTC)

Participants

  • Agustina
  • Leo
  • Lu
  • Santi

Discussion points:

  • Repo reorganization done!
    • fatiando/community: has the meeting notes, Contributing + other guides, CoC, and tracking of org-wide maintenace tasks.
      • Need to update links in packages
    • fatiando/.github: standard issue and PR templates
  • Synch the issue/PR labels across projects:
    • We can make an organization default for new repos
    • Edited list implemented in Pooch: https://github.com/fatiando/pooch/labels
    • Migrate to other repos? There is a tool but probably easier to do it by hand since new repos will have it automatically
      • Remove duplicate
      • Rename help wanted to I need help (Use if you need help from other developers on this). Remove from current issues because it's used to mean "please code this for us" instead, which applies to basically every issue.
        • Actully, just remove this since there isn't much need for it.
      • There is an issue label notifier: https://github.com/marketplace/actions/issue-label-notifier
    • Leo will synch this across repos
  • Boule revival!
    • Lu and Mariana had their first PRs merged! :tada:
    • Merged the unifinished PRs by Chris into the dev branch
    • Leo will make a push to finish the refactoring into a single Ellipsoid class in the next few weeks
    • Dissertation student at U. Liverpool will be working on implementing gravitation of ellipsoids (inside and outside) this year :muscle:
    • Reverted back to our Makefile + conda setup instead of nox
      • Too much work to set up and created more confusion than help
    • Coordinates on methods:
      • All methods take coordinates as input to be uniform with Harmonica and Verde.
      • Document which ones are not used and let users pass None
    • Update the docs to the new theme
    • Then make a release before messing with the Ellipsoid class
  • From the documentation visits, it seems that rockhound is actually being used for some things.
    • Completely changing it to our sample data package would cause these existing users' codes to break all of a sudden
    • Maybe we should use another name and archive RockHound.
    • In favour?
      • Yes!
    • If so, name suggestions:
      • fatiando_data: boring and informative
      • carpincho: sounds nice, South American, says absolutely nothing about the package
      • More! Search https://pypi.org/ to see if taken
    • New place for hosting large data: https://plus.figshare.com/

2021-10-01

Time: 14:00 (UTC)

Participants

  • Leo
  • Lu
  • Federico
  • Santiago

Discussion points:

  • Boule:
    • Leo may have a student working on implementing ellisoid gravitation equations (inside and outside the ellipsoid)
    • Will try to make a push on this to get it ready for a new release
    • Things to do:
      • Revert back to our Makefile/conda/etc workflow instead of nox. Turns out it complicates things more than needed and creates a bit of a disconnect with other projects.
      • Triage the current PRs and see what can be merged and what needs some work. There are some that may be outdated and need some re-thinking.
      • Finish looking at Lu's coordiante system tutorial
  • Community:
  • Pooch:
  • Publication of Analytics on Santi's Thesis
    • Will use the analytics data for the website about number of visitors and location (all anonymous)
  • Website analytics:

2021-09-24

Time: 14:00 (UTC)

Participants (please add your name)

  • Leo
  • Santi
  • Federico

Discussion points:

  • Santi added cards for the authors of each project: https://www.fatiando.org/about/
    • Start with Harmonica
    • Download images and convert them to .jpg? Check if they are really that big.
  • Enable pooch.retrieve to download to the default Downloads folder
    • It's not worth doing
    • Might be headache to test
    • Very likely to be source of future problems
    • Best compromise:
      • Write a get_default_downloads_dir() or something that returns the path to the default Downloads folder.
  • Bug report on Pooch: https://github.com/fatiando/pooch/issues/264
  • Add support for LZW decompression: https://github.com/fatiando/pooch/issues/263
  • Variable density tesseroids:
    • Repeat code it's ok for a first implementation that doesn't hit efficiency
    • We can iterate in the future.
  • Gridding bathymetry data
    • Finding another set of data (easier to get or with more coverage) that could correlate with bathymetry.

2021-09-17

Time: 14:00 (UTC)

Participants (please add your name)

  • Leo
  • Lu
  • Federico

Discussion points:

2021-09-10

Time: 14:00 (UTC)

Participants (please add your name)

  • Leo
  • Santi
  • Agustina

Discussion points:

  • Harmonica housekeeping

    • Add option to set EQLHarmonic points to constant depth #236
    • Generate version string on _version.py on build #237
    • Allow prism_layer to take xarray objects as arguments #243 < Merge first
    • Transform require_numba decorator into a global variable #245
    • Leo will review all of these after the meeting

  • Refactoring Tesseroids #244

    • Small fixes on docstrings
    • Refactor code to follow pylint guidelines
    • Enable parallelization through Numba: running 4-5 faster on my 6 cores CPU :tada:
    • Parallelization can be turned off through parallel=False, following the same interface as the other forward models

  • Working on the implementation of variable density tesseroids in Harmonica

    • Run density-based discretization as a preprocessing step at Python level
    • Use scipy.optimize.minimize_scalar to maximize the absolute max difference between normalized density and straight line, instead of defining an array of radii to explore (see Soler et al., 2019)

Quick updates:

  • New fatiando.org design: https://github.com/fatiando/website/pull/28 (comments and suggestions welcome)
    • Agustina showed how to use the Firefox developer tools to find accessibility issues
    • There are many :white_frowning_face:
    • But at least it's not worse than the current website
    • Will merge the PR as is
    • Agustina will open a few issues about accessibility so we can keep track
    • Follow up PRs will fix these

2021-09-03

Time: 14:00 (UTC)

Participants (please add your name)

  • Leo
  • Santiago
  • Lu

Discussion points:

  • Assign more formal roles within Fatiando:
    • Helps clarify who is who and who is responsible for what. Right now, it's not really clear unless you look through discussions and try to infer who makes decisions.
    • We could really use a "social media manager" (handling twitter/linkedin/SwUng/etc) and a "community manager" (help setup forum, welcome people to slack, lead community calls) for example.
    • What roles do we need initially?
      • Social media
        • Releases
        • New features
        • Patches
        • Talks
        • Events
        • Examples
        • Engage in conversations
        • Highlight publications using the project
        • Promote projects in the ecosystem
      • Website
        • Maintain up dated
        • Add/remove people from lists
        • Redesign (one time thing)
      • Community lead
        • Answer issues and prs quickly
        • Probalby overlap with maintainers but not all maintainers should be community leads
        • Compile list of new issues, stale issues, PRs, etc for the weekly meetings
      • Maintainers (for each project)
        • Reviewing and merging PRs
        • Will have full admin rights to the repo
        • Make releases (compile changelog, Zenodo, etc)
      • Contributor (for each project)
        • Code
        • Docs
        • Answer questions
        • Largely a list of contributors from GitHub (AUTHORS.md?)
    • What is the process for gaining responsibility?
      • Good amount of engagement and contributions is a requirement
      • Need somewhat regular contributions
      • Maintainer initiates discussion with rest of mainteners to invite someone for promotion
      • Promote early rather than late
    • What is the process for stepping down from roles?
      • Notify the team
      • Find a replacement (if possible)
      • Go be happy
    • Include time off in the document:
      • Periods where nothing happens for a little while (e.g. end of the year)
    • Draft a governance document.
    • Would allow us to have a team/gallery to showcase who works on fatiando.
  • Functions that take "arrays" should also support DataArray, pandas Series, etc? Should we make it explicit only when we truly support them?

Quick updates:

  • Old documentation pages now all redirect to https://legacy.fatiando.org (v0.5 docs)
  • Mailing list is deactivated and there is a message telling people to go to Slack
  • fatiando/website now uses main as the default branch. Update local clones with: 
    ​​git branch -m master main
​​git fetch origin
​​git branch -u origin/main main
​​git remote set-head origin -a

2021-08-27

Time: 14:00 (UTC)

Participants (please add your name)

  • Leo
  • Agustina
  • Lu
  • Santiago

Discussion points:

  • Keep guides (contributor, maintenance, etc) only in the fatiando/contributing repository.
    • Right now, we have a complicated setup that pushes changes to the files in this repo to the other repositories.
    • This relies on TravisCI, which we largely don't use anymore.
    • No one had the time to go in and update to GitHub Actions.
    • It doesn't allow projects to include custom changes to the files (since it would cause conflicts).
    • Proposal:
      • The files on repositories will remain
      • They will have a small message linking to the original files on contributing
      • They can include extra instructions that are only for that particular project
      • Removes the need for the CI setup in contributing
    • Having a lot of stuff on the project creates barriers to entry.
    • Authorship guidelines should be easily findable
    • System of layers (from mainteiners to new users):
      • Maintenance
      • Contributing + authorship
      • Documentation, tutorials, how to contact devs and other users
    • Do we want:
      • 1 central document with links to it on all repos
      • no central document and individual guides on all repos
    • Proposal 2.0:
      • Kill the central document and it's TravisCI
      • This will break the Welcome Bot
    • TODO:
      • Open a issue on fatiando/contributing
      • May need to copy the welcome bot script from fatiando/.github to individual repos
  • People asking help through mailing list while trying to install fatiando on Python2.7
    • Probably they are getting instructions for installation and how to contact us from fatiando.org/v0.5
    • Should we add a big disclaimer on the fatiando.org/v0.5 website about it being deprecated?
    • TODO:
      • Stop serving everything below v0.5 and also dev/
      • Serve 0.5 under legacy.fatiando.org
      • Add a banner to the 0.5 pages point to the main Fatiando website
      • Future: Add banners to old versions of the docs in all libraries. See how numpy and matplotlib do this.
        • Create an issue in fatiando/maintenance for this
      • Disable posting to the mailing list. Maybe automated reply might be worth the effort. https://support.google.com/groups/answer/9792589?hl=en

Quick updates:

2021-08-06

Time: 14:00 (UTC)

Participants (please add your name)

  • Leo Uieda
  • Santi Soler
  • Lu Li

Discussion points:

  • Fast inverse square root: https://www.lomont.org/papers/2003/InvSqrt.pdf
    • We calculate 1 / sqrt(x) quite often in gravmag modelling and equivalent sources.
    • This is usually the largest bottle neck for point sources
    • This method is supposed to be 4-5x faster than doing sqrt and then division
    • Has some crazy hacks and may be hard to generalize (implementation depends on number of bits and float format)
    • Could be a very impactful addition to Bottleneck or numbagg
    • Probably not for the faint of heart :slightly_smiling_face:
  • Redesign of the grid method on Verde gridders (PR #326)
    • Let grid to accept the coordinates of a preexisting grid (2d meshgrids or 1d arrays)
      • Benefits:
        • BaseGridder can be easily generalized to any number of coordinates. EQLs for example, with easting, northing, upwards
    • Proposal: deprecate spacing, shape, region: don't create the coordinates inside grid
      • Benefits:
        • Users can inspect the coordinates before running the predictions
        • Simplifies the arguments of grid
        • Do a single task and do it well
    • Name of the coordinates argument: coordinates vs grid_coords
      • Argument should be coordinates
      • But variables in examples should be grid_coords since there is likely already a coordinates variable
    • TODO:
      • Issue 1:
        • Add the coordinates argument to grid but make it optional for now (Santi has a PR for this already)
        • Include a FututeWarning that things will change in 2.0
      • Issue 2:
        • Update the examples to use grid_coordinates and pass them in to grid with a warning that this will be the only way in the future
      • Issue 3:
        • Remove the arguments of grid_coordinates from grid
        • Make coordinates a required argument
      • Open issues for these things
  • Verde
    • Start a new dev branch for deprecations and backward-breaking changes
    • Which git workflow should we use for it?
      • Should dev be always updated with master? (would create a lot of 'Merge…' commits)
      • or we should merge master on dev right before the next major release? (we might face conflicts and some bugs right before the release)
      • git rebase instead of git merge to avoid merge commits.
      • Probably do it all when we will release to avoid prolonging our suffering
    • TODO:
  • Is conda install -c conda-forge verde harmonica boule pooch rockhound working fine?
    • Deactivate the mailing list once and for all
    • Create CI job (weekly?) that setups miniconda and Anaconda, installs everything at the latest version, and run tests.
    • Could be a new repo just for this or https://github.com/fatiando/maintenance/
  • Documentation https://hackmd.io/wPxyX6c2ROGjcMvxDL0TmQ?view

Quick updates:

  • Abstract submitted for our AGU2021 talk: https://github.com/fatiando/agu2021
    • Leo will be touch later in the year when the program is finalized and we know exactly what to present.
    • Likely to record a video to acompany the live talk. Contributions from the community would be very welcome (video, audio, images, etc)
    • kdenlive can apparently make subtitltes from a script. Can embed into mp4 and youtube/figshare pick it up.

2021-07-23

Time: 14:00 (UTC)

Participants (please add your name)

  • Leo Uieda
  • Santi Soler
  • Lu Li

Discussion points:

  • AGU talk
    • Leo was invited to give a talk at a session
    • Asked to talk about the project and also touch on the maintenance/sustainability
    • Outline our process for adding new features:
    • Can submit up to 3 abstracts so if there is anything else we want to submit, we can.
  • Parker-Oldenburg inversion:
    • Not on the immediate roadmap but welcome if anyone wants to code it
    • Can suffer from instability and may require filtering
    • Check if coordinates are latitude, longitude prior to FFT and yell at the user if they are (can be done across all FFT-based functions right now).
    • Should be straight forward to implement.
  • Next meeting discuss roadmap and wishlist for Verde 2.0

2021-07-16

Time: 14:00 (UTC)

Participants (please add your name)

  • Leo Uieda
  • Santi Soler

Discussion points:

  • Leo was invited to write a short news article for the Brazilian Geophysical Society about Fatiando (in Portuguese). Have to do this by August and will send around for some feedback.
  • Split up the verde.ScipyGridder into 3:
    • vd.NearestNeighbors: NN interpolation using cKDTree or pykdtree. Option to average K nearest neighbors. Include a search radius as well (max_distance in the query method).
    • vd.LinearGridder and vd.CubicGridder: Wrap the scipy methods and add the option extrapolate (default to True) to fill NaNs with nearest neighbors (default to 1 neighbor).
    • Can help move the focus away from Green's functions.
    • Add these classes then deprecate ScipyGridder to remove in 2.0.0.
  • Roadmap for Harmonica
    • Focus for v0.3:
      • MVP: Processing raw gravity -> Residual grid -> transformations (FFT or EQL)
        • Restrict to Cartesian
      • Missing components:
        • EQLHarmonicBoost
        • Finish FFT filters (explain using gridded data):
          • Upward derivative (horizontal uses xarray)
          • Upward continuation (optional)
          • Total gradient amplitude (optional)
          • Total horizontal gradient (optional)
        • Theory:
          • Coordinate systems
          • Gravity disturbance and corrections explained
        • Tutorials:
          • Basically what we did for Transform (but maybe split it up)
          • Tutorial on transformations (with EQLs and FFT and when to use which)
    • Documentation!
      • Good docstrings but no tutorials
      • Santi's thesis will be a good basis for the theory
      • What is the workflow and the theory
    • Wishlist:
      • Tesseroids with variable density
      • Gradient boosted EQLs
      • FFT solutions: derivatives, TGA, reduction to the pole, upward continuation
      • IGRF:
        • Coeffs are here: https://www.ngdc.noaa.gov/IAGA/vmod/igrf.html
        • Calculation is actually really easy
        • No point in wrapping a fortran 77 code
        • Can use outputs from the fortran to test
        • Need to derive the equations for the x, y, z components of the SH model

2021-07-02

Time: 14:00 (UTC)

Participants (please add your name)

  • Santi Soler
  • Lu Li

Discussion points:

  • Some test for xrft: https://nbviewer.jupyter.org/github/LL-Geo/PFToolbox/blob/master/xrft_note.ipynb
    • Looks like we can direct use fft and ignore dft
    • We can save the coordinate information from original grid, and pass it back for the ifft
  • Santi opened a PR on xrft for padding grid and coordinates: https://github.com/xgcm/xrft/pull/158
    • They asked us to upstream the pad_coordinates to xarray
    • Start with supporting only padding through number of points (we can work on lenght of coordinates in the future)
  • Set up a Bot for reminders on Fatiando calls
    • One day before
    • One hour before
    • Tweet reminders

2021-06-25

Time: 14:00 (UTC)

Participants (please add your name)

  • Leo
  • Santiago
  • Agustina
  • Lu

Discussion points:

  • Interaction with xrft:
    • Santi has opened an Issue to start engaging the xrft community: xrft #154
      • Naming of the default function for applying FFT: fft vs dft
        • Devs agree and seem to favour fft
        • Requires improvement of docs around true_phase and true_amplitude
      • Coordinates shifts: lag argument in fft and including lag in the Fourier transform
      • Padding implementation:
        • They asked Santi to open a PR in xrft with a padding implementation
        • Wouldn't be better to enhance the xarray.DataArray pad for also pad the coordinates if they are evenly spaced?
        • Limitation of current padding in xarray:
          • By default, pads the coordinates with nans
          • Make sense because there is no way to know what the coordinates are if the spacing is irregular
          • Could do it if the coordinates are regular
        • Pitch:
          • Implement the padding coordinates in xrft for now
          • Basically check for nans and then add coordinates as needed
          • When it's implemented in xarray, we can shift to using that instead

Quick updates:

2021-06-18

Time: 14:00 (UTC)

Participants (please add your name)

  • Santi Soler
  • Agustina Pesce
  • Leo Uieda

Discussion points:

  • Reconcile the Harmonica and Verde gridding API?
    • The EQLHarmonic and Spline APIs are slightly different:
      • .grid takes the extra upward coordinate in EQLHarmonic
    • Done for a more intuitive and descriptive interface
    • But it means we can't use anything that relies on the Verde API, like vd.Chain, vd.Vector, and the proposed vd.Derivative.
      • Actually, we can but the Chain.grid and EQLHarmonica.grid methods are different, making it a bit awkward to use.
    • Is there a way to keep the upward argument and make the APIs compatible again (even if it means changing the Verde API)?
    • PROPOSAL:
      • Deprecate region, shape, shaping from the grid method, make it only take the actual grid coordinates, either from vd.grid_coordinates or from an xarray object (as 1d arrays).
      • Add a new property to gridders that documents the number of dimensions (n_dims?)
      • Make the Chain to inherit the maximum n_dim from the gridders in the chain
      • Add a grid method to Chain that complains if the passed coordinates are less than n_dim
      • Implement this in a dev-2.0 branch that will be merged before major release
  • Invite people from xrft to next meeting

2021-06-11

Time: 14:00 (UTC)

Participants (please add your name)

  • Leo
  • Santi Soler
  • Lu
  • Andrea
  • Agustina

Quick updates

  • Pooch is using the sphinx-book-theme and trying out a new docs layout: #237
  • Pooch PR to download from Zenodo and figshare: https://github.com/fatiando/pooch/pull/241
    • Will be useful for refactoring RockHound into our sample data repository
    • Can put every dataset in the same Zenodo/figshare archive and download individual files

Longer discussion

  • Frequency-domain filters in Harmonica: #238
    • Two type of functions:
      • Filters: work only in frequency domain. I would like to create a filter module where they should live.
        • Takes FFT from xrft and outputs still in frequency domain
        • Mostly for internal use but can be used by others as well
      • Filters will be public but not the wrapped FFT functions
      • Transform and processing functions (derivatives, upward continuation, RTP, etc).
        • Their arguments and outputs are always in spatial domain.
        • They call xrft, use the filters, and then inverse the transform
    • We should engage with xrft and discuss some things regarding their implementation.
      • What true_amplitude and true_phase mean?
      • Why lag is not working properly?
        • Inverse transform removes the absolute value of the coordinates so they are centered at zero
        • Need to shift the coordinates in our own code
        • The lag seems like it does this but doesn't work for some reason.
        • Check if the coordinates of input and output match to see if we need to shift
      • Why having dft and fft? What are the differences between them?
      • Santi may have found some bugs in the phase
      • Engaging with them instead of wrapping everything locally is better for the community as a whole
      • Open an issue to start the conversation
    • Derivatives:
      • Should we add the option to choose between finite difference or FFT derivative for easting and northing?
        • The finite difference can be achieved by xr.differentiate.
        • For now, remove the horizontal ones from the PR and leave just the vertical
        • In the docs, use xarray for the horizontal derivatives
      • How about using an argument to specify the direction?
        • Better to have 1 function in the docs
        • But can't auto-complete the direction names in notebooks
      • Directional derivatives could be a usecase for FFT based horizontal derivatives
        • Lu will provide a reference/use case for this that we could base the implementation on
        • Could also be done with finite-differences
        • The best solution might be: ?
          F(DD)=F(g)∗(u∗i∗sin(θ)+v∗i∗cos(θ))           So that we could combine the horizontal derivatives with directional derivatives together by different angle
        • Ref 1: Hang, N. T. T. (2017). Application of directional derivative method to determine boundary of magnetic sources by total magnetic anomalies. VIETNAM JOURNAL OF EARTH SCIENCES, 39(4), 360-375.
        • Ref 2: Stewart, I. C., & Miller, D. T. (2018). Directional tilt derivatives to enhance structural trends in aeromagnetic grids. Journal of Applied Geophysics, 159, 553-563.
      • Need to engage with xarray on their padding function.
        • It doesn't set coordinates (just NaNs) for the padded region
        • This makes xrft angry
        • Not trivial to implement in xarray since their Datasets aren't necessarily regular grids
Last changed by 
 
Fatiando a Terra
Open-source tools for geophysics
0
1
1192

Read more

Fatiando Development Calls 2025

📱 Join the video call: https://meet.jit.si/fatiando-a-terra

May 27, 2025
Fatiando Development Calls 2024

📱 Join the video call: https://meet.jit.si/fatiando-a-terra

Dec 10, 2024
Fatiando Development Calls 2023

📱 Join the video call: https://meet.jit.si/fatiando-a-terra

Dec 19, 2023
Fatiando Development Calls 2022

:::info Link to the meeting notes for 2023: https://hackmd.io/@fatiando/development-calls-2023 ::: 📱 Join the video call: https://meet.jit.si/fatiando-a-terra 📅 Shared calendar: https://www.fatiando.org/calendar IMPORTANT: Everyone is required to follow our Code of Conduct when participating in the Fatiando community. Please review it carefully.

Mar 29, 2023
Read more from Fatiando a Terra
Published on HackMD