# brainstorming a 1-day workshop on SBI

## Venue & Connection Details

Friday, April 30, 10am

Topic: Brainstorming a one-day SBI workshop
Time: Apr 30, 2021 10:00 AM Amsterdam, Berlin, Rome, Stockholm, Vienna

Join Zoom Meeting
https://zoom.us/j/99276028397?pwd=WmZ3aEZobXB1TXpzUW0zcTZjaUtpQT09

Meeting ID: 992 7602 8397
Passcode: 156017
One tap mobile
[phone],,99276028397#,,,,*156017# Germany
[phone],,99276028397#,,,,*156017# Germany
Find your local number: https://zoom.us/u/asVi9HV5H

Participants:
David Greenberg, Michael Deistler, Daniela Huppenkothen, Jan Bolts, Alvaro Tejero-Cantero, Peter Steinbach, Sebastian Starke

## SBI users

- Daniela Huppenkothen
- bayesian inference of time series
- institute for astrononmy something in NL
- data from xray telescopes when observing very bright objects
- use SBI to assess systematic effects of instruments
- involved in proposals wrt to surrogate models in astronomy/cosmology
- example:
- https://arxiv.org/abs/2104.03278
- simulator can be coded easily (write the function in 30min), but inversion is hard if not impossible to assess
- simulated very short time series
- Q: in neuroscience models are typically mechanistic -> what is the inference goal
- phyiscal process generates time seris
- black hole and a star: gas falling into black hole
- induces quasi periodic process -> need to measure this
- non-linear effects scale with brightness
- generate time series -> goes through simulator -> list of observed photons
- 2 simulators: one for physical process and one for measure
- Jan@Daniela: for hierarchical inference have a look a
- https://arxiv.org/abs/2102.06477
- or in general, learn a synthetic likelihood and then you can arrange the likelihood terms as you need them and run MCMC.

- Sebastian (statistician by training with a blind eye on Bayesian theory)
- 2 interests:
- where does SBI internals fit into statistics?
- work with radiation therapy dept at local uni clinic
- working on risk models for patient induced treatment
- trying to predict how long a patient will survive given a CT image
- idea for SBI: feed patient data into SBI and retrieve CT image
- idea for SBI: provide CT and obtain expected survival time
- David: conditional density estimation (partially contained in SBI)
- used simple CNNs with mediocre results

- SBI designed for (conditional) density estimation
- maybe invest first half of a workshop day

- Peter Steinbach
- working on simulation of accelerator beamline (produces beam profile images)
- inverting the simulation is the goal
- image highly discontinuous
- also small-angle scattering: reconstruct from an image the shape that produced (?) the scattering image
- time series estimation (down the line, currently working on static)

## learner profiles

- Phd students from the Helmholtz AI network and affiliated groups
- Helmholtz AI consultants
- Scientists at Uni Tübingen
- MLColab consultants

- DH: mostly have 2 crowds
- beginners that like to learn SBI as a concept and `sbi` as a tool
- exchange with more advnaced users like DH ;-) in the line of "how to do hierarchical inference with SBI"

- what is the scope of this?
- DH: more didactical to learn the tool `sbi` or more a workshop to exchange wishes/experiences
- PS: not discussed yet -> my interest is learning the tool and get grasp of theory (depends on available time budget of all)
- DG: time budget is limit, would contribute a talk, supervise interactive session
- interested to see how SBI is used and applied on new domains, learn about strenghts and weaknesses of methods used
- workshop might open the room for collaboration, long run publications?
- ATC: long term effort to come up with tutorials more closer to domains
- should we aim to produce papers with this? publications form this workshop might happen naturally
- balance consulting and research
- joint event between Tubingen and Helmholtz?
- tool for (conditional) density estimation? (community would appreciate this)
- JB: pyknos?
- ATC: yes, but maybe not too amendable to users
- ATC: would appreciate feedbac

- FORMAT: at least one day. Maybe split in two half days
- split into "introduction" and "work on your data"
- we can help with infrastructure, getting software up and running, beta-testing, participant communication
- JB: good ideas so far -> 1-2 day workshop sounds fine
- MD: would be up for the introduction, have lecture ready for master students in Tuebingen

- need about 3 months for preparation -> target second half of September

- how large should audience be?
- PS: suggest 20 people in first session
- AT agrees to use small number as a trial run -> scaling up later not too much effort
- maybe do a video-recording for the introduction part
- flipped class-room: after watching recording, learners can get in touch with mentors to ask questions
- DH: maybe learners will not prepare properly if asked to do something in advance

- further communication among this group will be done by mail

## content wishes

- Introduction to the theory on how posteriors are estimated
- which role do the neural networks play here (what exactly is optimised given the simulation data)
- how are the networks used in posterior sampling
- which network architectures are used (what are MAFs, NSFs etc...)
- how do the concepts SNPE, SNLE, SNRE differ and when should we use which
- is MCMC involved?

- Introduction to the SBI terms
- what are "rounds of inference"
- what exactly is amortization
- simple interface vs flexible interface -> use cases
- leakage of the prior
- Evaluation - how do I know I am right
- introduce the sbi benchmark (for offline perusal)
- how to interpret a posterior (marginals, conditionals)
- model criticism
- model misspecification, model comparison (-> Jan? just describe the limitations of sbi at this point)
- Advanced topics (we should discuss if this is to be part of the WS, or part 2)
- Hierarchical inference https://arxiv.org/abs/2102.06477
- SBI on manifolds (cf. Cranmer group paper)
- Move 'advanced' topics to 2022 workshop, merge broader interest ones above.

-
## AOB

- targeted workshop for more advanced users and method developers -> targeted at 2022
- https://www.lorentzcenter.nl/lorentz-escience-competition.html