Meeting Minutes
===

###### tags: `salting` `Meeting` `AI` `Salting` `AI/ML` `ML` `CDMS` `ML/AI`

:::info
- **Location:** https://ucdenver.zoom.us/j/97994358165
- **Date:** July 6, 2023, 10 AM Mountain
- **Participants:**
- Amy Roberts (AR)
-
- **Meeting Goal(s)**

:::

## Summary
<!-- Please fill me in after the meeting -->

## Notes
<!-- Other important details discussed during the meeting can be entered here. -->
- Sukee is running into issues loading her boolean data - or perhaps has created it incorrectly? Could RDataFrame help? https://root.cern/doc/master/classROOT_1_1RDataFrame.html
- Aditi will present on the physics overview next week
- We're pretty sure we understand the shape of the data that TimeGAN expects. But now the training is taking a very long time. Farnoush: now we're loading a lot of data into memory, if it starts thrashing it can take very long. This is something mini-batch training can help - training on smaller batches and then moving on to the next one.
- Mithun is getting out-of-memory errors on Compute Canada. Maybe want to request some high-memory GPU resources from PSC Bridges.
- Is there a way to get priority on Bridges? Amy: not so much, but if we make the jobs shorter they'll go through faster. Farnoush: tensorflow has built-in snapshotting capabilities that will allow you to save and re-start jobs. https://www.tensorflow.org/tutorials/keras/save_and_load
- Mithun: does tensorflow try to allocate all the memory available?