yt team meeting notes

Attendees

• Nathan Goldbaum
• Bili Dong
• Abhishek
• Cameron Hummels
• Desika Narayanan
• Madicken Munk
• Mike Zingale

Agenda

• GSOC update
• yt 4.0 / demeshening update
• Jenkins downtime
• yt 3.5 release
  • Release manager?
  • What to do about open bugs marked for 3.5?
  • Dropping Python3.4 support
  • Python3.7 support?
• Postdoc position at NCSA
• Development priorities
  • optimization (grid_visitors)
  • paper
  • volume rendering
• How we can help with the paper?

Notes

• GSOC update
  • Abhishek has been developing test coverage improvement and reducing overall dependencies on large datasets.
  • Ashley Kelly also a student at Durham, starting PhD in the Fall, working on GSoC for demeshening project
  • Projects going great!
• yt 4.0 / demeshening
  • Major, backward incompatible release with 4.0. Big updates:
  • Domain context system that is functioning and working. Likely astronomy and nuclear engineering, maybe one or two others.
  • DC system is designed to set up jargon and fields and whatnot that are better representative of the domain/field-of-study the data comes from.
  • demeshening! This removes the global octree mesh. Discussed in YTEP-0032.
  • Over the summer, Ash is working to add SPH smoothing to uniform/arbitrary grids.
  • Adding support for setting gather/scatter styles for collecting particles.
  • Demeshening should eventually support octree collections.
  • grid_visitors – method of removing a bunch of python objects, pushing lots more operations for selection into cython. Too much RAM is currently used; this can reduce that a fair bit. Hopefully can speed up ghost zones. Big holdup right now is getting the grid visitors working without python objects except on-demand.
  • Q: "Under-the-hood changes for particle-based datasets, specifically projections?" A: "Idea for how to do that. Got sick of it, put it away for a while. But, NG thinks he knows how to address it now." Issue is project-on-demand versus project-ahead-of-time – right now, projections of demeshened datasets are always on demand, rather than ahead of time. NG is working to make it work with ahead-of-time, so that panning/zooming/etc are much faster. Currently, it's O(Npart), so project-once-generate-many is lost. Work is ongoing to recover that ability with kD-tree in a dimensionally reduced way. Now has a kD-tree projecting at power-of-two boundaries. That should work.
  • 4.0 isn't going to come out for another year. (During which time presumably we could rename it to yt NT, now that we are on Microsoft's GitHub)
  • Q: "Gather method – [note taker didn't quite understand so this got dropped]"
• Jenkins downtime: nebula has been exceptionally unreliable. This week, the downtime is supposedly because of a thunderstorm (which was real and did exist) that did not result in power outage. Not totally sure how that worked. Long term, Abhishek is working to reduce our reliance on real test datasets. Nathan suggested we think carefully about reliance on answer testing.
• yt 3.5 release
  • Correntin (not on call) has been working super duper hard on improving the RAMSES frontend.
  • There have been enough improvements that it's an OOM better in every way to use RAMSES in yt 3.5.
  • We need a new release manager. "What do we do about open bugs in yt 3.5?"
  • Do bugs that are marked as blockers need to be fixed, or can we push them to a later release?
  • What's our punting strategy?
  • Comment: "Perfect is the enemy of the good." Voiced in favor of improving things where possible, and deciding when it's enough based on our availability.
  • Maybe transparency, with sprints, asking for help, would be best.
  • Suggestion: hangout for triaging existing bugs. Interest expressed by: Cameron, Nathan, Matt, Madicken.
  • Q: Can we drop Python 3.4? A: No one objects.
  • Q: Python 3.7? For yt 3.5, we can support it. Maybe even building wheels!
• Hiring a postdoc at NCSA, paid to work on yt. In support of NSF grant.
  • Pass it around! (Please?)
• What's everybody workin' on?
  • Matt: paper, grid_visitors, working with Chuck on volume rendering (aiming for 3.5)
  • Nathan: neat particle rotation stuff with student at NCSA (Bili and Cameron are super excited about that.)
  • Mike: wants to add support for higher-order finite volume. And gamma stuff.
  • Nathan brought up the new field system.
  • Madicken, Nathanael and Matt have been doing wasm work.
  • Cameron: sparsely packed time outputs. Wants to look at interpolation between timesteps. How would one do this with grids?
  • Cameron was at the astroviz conference last few days and presented about yt stuff. Firefly being developed at Northwestern. (Example) Everyone agrees this is pretty rad. Possible to have lots of really cool synergy.
• How to help with the paper? Matt says look at issues and contribute words.
• Bili brings up wanting to HEALpix slice or projection of SPH data. Bili has volunteered to lead the HEALpix project.
• Nathan ends by saying we should do this more often.