# Rust ML WG Meeting 00016

## Meeting Info

Date: 20201215

Start time: 1600 ET

Zoom link: https://1password.zoom.us/j/98269163669?pwd=U2E4TDltOHM5Vk00azJ4QnF4YVJaZz09
- Meeting ID: 982 6916 3669
- Passcode: 485260

## Participants

- chrism
- jeremy thompson
- william moses (Billy)
- https://github.com/wsmoses
- ivano donadi
- lorenz
- andraz tori
- manuel
-

## Agenda

- Short update on Fwumious Wabbit (by Andraz)

- chrism wrote a small prototype for itegrating a convolutional with fully-connected layer
- Still double-checking that the backwards pass between the two layers types are working (has been talking with Benard, who leads juice devel)
- Could use another set of eyes on the linear algebra if anyone's interested before moving

## Minutes

- just chatting about chrism's work and other people's interests as a pre-show :)

### Chrism small prototype for integrating fully-connected convolutional layer

- [tsuga](https://github.com/quietlychris/tsuga)
- generalizing away from fully-connected layers into convolution layers
- can do horizontal line detection
- with sliding window and matrix multiplation
- working on adding convolutonal layers
- next step
- move way from NIST into CIFAR10
- less confident about backwards passes
- needs some eyes on the linear alegabra for the backward passes from fully-connected into convolutional layer
- Could you take an input and put through edge detection
- the output layer should be the total number of pixels and then converge to 1
- single kernel and sincel convolutional layer

### Update on Fwumious Wabbit from Andraz

- continung development
- two new features
- from java to rust code directly JNI bindings
- [flapigen](https://crates.io/crates/flapigen)
- rework that will enable hogwild model loading
- hogwild loading of models
- training for deep models in parallel

### Enzyme automatic differenation from Billy

- https://github.com/wsmoses/Enzyme
- lorenz
- integrate that into the Rust ecosystem
- creating bindings to enzyme
- in a `build.rs` file
- still creating the bindings and reading the llvm code
- type trees problem
- special case of -1
- enzyme wants to know the types of data (e.g. float vs double)
- type trees are
- type prop rules, for any given instruction in llvm (adding to types together means the resulting type is of the same time)
- from the Rust types, what are the corresponding type tree equivilant
- this is a repeating type
- positive offset is a subtree is exactly what the types are
- -1 means for example double array and every x bytes is a double
- Billy
- working on parallelism
- GPU and CPU parallelism
- has working parallelism for CUDA mostly working

- Hijack the linker from LTO -> LLVM
- ideal
- dynamically load into `rustc` (or custom codegen pass)
- modify or fork the codegen pass that allows lowering of the automactially diff tree function
- if you dynmically load enzyme, it will autmocally insert enzyme into the pass pipeline
- co-author on eyzme is one of the main contributors of julia and julia supports the codegen pass

### linfa update from lorenz

- not a lot since last meeting
- updated decision tree
- moved to new CI infrastructure
- added code coverage tool

### Chris asked about how linfa datasets are handled with test data

- How do other ecosystems package their testing data?
- linfa is using tarballs
- use include macro compile who dataset into project
- then publish project
- git submodules possibly
- don't forget to do `git submodules --recursive`
- or else you'll clone the repo but not the included modules
- any interest in using a `pytorch` model where you download and cache the data locally?
- need hasn't really shown up to put together

### Billy asked if the WG is Looking into any Bayesian models at all?
- probabilitic programming 
- not really yet ecosystem is fairly young

### Chrism talking about St. Claire license
- https://github.com/quietlychris/st-claire-license
- looking for feedback
- any feedback :)
-

## Actions

- ~~Add https://github.com/wsmoses to tsuga~~
- feedback about the St. Clair license
- next meeting tenatively 20210105 possibly 20200112 due to Holidays