---
# System prepended metadata

title: Meeting Minutes
tags: [Meeting]

---

Meeting Minutes
===

###### tags: `salting` `Meeting` `AI` `Salting` `AI/ML` `ML` `CDMS` `ML/AI` 

:::info
- **Location:** https://ucdenver.zoom.us/j/97994358165
- **Date:** July 6, 2023, 10 AM Mountain
- **Participants:**
    - Amy Roberts (AR)
    - 
- **Meeting Goal(s)**

:::

## Summary
<!-- Please fill me in after the meeting -->



## Notes 
<!-- Other important details discussed during the meeting can be entered here. -->
- Sukee is running into issues loading her boolean data - or perhaps has created it incorrectly?  Could RDataFrame help?  https://root.cern/doc/master/classROOT_1_1RDataFrame.html
- Aditi will present on the physics overview next week
- We're pretty sure we understand the shape of the data that TimeGAN expects.  But now the training is taking a very long time.  Farnoush: now we're loading a lot of data into memory, if it starts thrashing it can take very long.  This is something mini-batch training can help - training on smaller batches and then moving on to the next one.  
- Mithun is getting out-of-memory errors on Compute Canada.  Maybe want to request some high-memory GPU resources from PSC Bridges.
- Is there a way to get priority on Bridges?  Amy: not so much, but if we make the jobs shorter they'll go through faster.  Farnoush: tensorflow has built-in snapshotting capabilities that will allow you to save and re-start jobs.  https://www.tensorflow.org/tutorials/keras/save_and_load
- Mithun: does tensorflow try to allocate all the memory available?