# DUNE-ND Reconstruction Milestones A live note for re-implementing _Supera_ software, a toolkit to produce labels to optimize (via supervised learning) ML based data reconstruction chain for DUNE-ND. Technical discussion takes place during a dedicated weekly meeting (Wednesday 3PM ET, noon PT). Other communication channels include a mailing list and slack channel. To join the mailing list, send an empty email to the SLAC listserv (listserv@listserv.slac.stanford.edu) without a title and with a body message: ``` SUBSCRIBE DUNEND-SIMRECO-TECHNICAL $FIRST_NAME $LAST_NAME ``` where you should replace the `$FIRST_NAME` and `$LAST_NAME` with yours. ## 1. Label making = Supera Overhaul Need three pieces of software ### SuperaAtomic [(link)](https://github.com/DeepLearnPhysics/SuperaAtomic) [x] 1.A ... A compilable software with Supera data types (`SuperaMCParticleClusterData`) **-Kazu** [x] 1.B ... Implement Python binding (no ROOT) for data types. [x] 1.C ... Add `SuperaBBoxInteraction` **-Kazu** [ ] 1.D ... Add `SuperaMCParticleCluster` **-Kazu** ### Edep2Supera [(link)](https://github.com/DeepLearnPhysics/edep2supera) [x] 1.E ... A compilable software with only a function that can access `EDepSim`'s `TG4Event` as a `PyROOT` object (handed through Python script) **-Kazu** [ ] 1.F ... Translate `EdepSim` objects into Supera objects defined in `SupraAtomic` **-Zach/Kazu** [ ] 1.G ... Write a script to run an event loop on `EDepSim` file and create `LArCV3` output file. **-Zach** ## 2. ~~Interfacing LArCV3~~ Implementation of LArCV3 is postponed. Items below will be pursued after we first implement LArCV2 interface. This is to accommodate time pressure to deliver a working reconstruction chain (i.e. making LArCV3 output requires the reconstruction chain software to also have an interface to take LArCV3 input files, which does not exist yet). ### LArCV3 file generation [x] 2.A ... use a converter code: LArCV2 => LArCV3 (contact Corey Adams) while waiting for a new Supera to work (which can generate LArCV3 output) **-Zach** ### LArCV3 interface [ ] 2.B ... implement torch DataSet inherited class to interface with LArCV3 inside [datasets.py](https://github.com/DeepLearnPhysics/lartpc_mlreco3d/blob/develop/mlreco/iotools/datasets.py) (currently you see `LArCVDataSet` for LArCV2). Ideally implement in a similar manner so that parsers can be re-used (understand this point before working on this project). [ ] 2.C ... implement/modify parsers by introducing necessary changes wherever appropriate in [parsers.py](https://github.com/DeepLearnPhysics/lartpc_mlreco3d/blob/develop/mlreco/iotools/parsers.py). Verify parsed tensors agree to FP32 precision between larcv2 and larcv3 (for the same source file).
Last changed by  
 
Drinking Kazu
216

Read more

ND Simulation How To

TLDR; prerequest /sdf/group/neutrino/images/larcv2_ub20.04-cuda11.1-cudnn8-pytorch1.9.0-larndsim.sif login to a GPU node (only required for det-sim stage w/ larnd-sim) source /sdf/group/neutrino/kvtsang/nd-sim/mc-prod/setup.sh gen-edepsim-g4mac mpvmpr_cfg.yaml g4.mac edepsim-nd g4.mac 10 edepsim-output detsim-nd -i edepsim-output.h5

9/30/2022
Untitled

The goal The goal of my ECA was to develop a scalable, high quality machine-learning-based data reconstruction method for neutrino experiments that utilize Liquid Argon Time Projection Chamber detectors, or LArTPCs in short, including the Short Baseline Neutrino program and Deep Underground Neutrino Experiment. ~~## Reco chain in a nutshel Steps of physics data reconstruction is naturally hierarchical: starting from a low-level signal processing, identification of indibidual particle trajectories, inference of particle-level information, then finally event-level information inferred from correlations between multiple particles. ~~ Progress in the past A full reconstruction chain has been completed by the end of the 3rd year. In this 4th year, we have introduced capabilities to quantify calibrated uncertainties for key physics outputs. Just like a traditional reconstruction method, the chain consists of multiple modules of ML algorithms that perform individual reconstruction tasks. The whole chain combines these algorithms in a hierarchical manner based on domain physics knowledge. The full chain was presented at the NeurIPS workshop for physical science in 2020, and individual algorithms have been published in multiple PRD papers. Some of them received editor's pick and DOE science highlights. Accessible research

8/25/2022
Yifan's intro prrojects

Run the full reconstruction chain. Modify the config to train U-ResNet + PPN only. Run as long as possible on a100 gpu. Plot the loss curve (while training is ongoing). Save weights every 1000 steps. Run the inference on 32000 samples per saved weights to make a validation accuracy plot. Show (train loss and validation loss) in an overlay. Learn how to run particle bomb event generator + DUNE-ND geant 4 (edep-sim) and detector simulation larnd-sim.

11/1/2021
Current To-Do's and Future Plans

Reconstruction related Software for production Scalable software for distributing the execution of a reconstruction pipeline to thousands of GPUs. Targetting by the end of 2021 (300,000 A100 GPU hours allocated at the Polaris HPC cluster). Optimization of the chain Benchmark the quality of labels at edepsim level + reproduce the performance in the paper Implement an event generator used to train the chain in edepsim (Andrew Morgan)

10/5/2021
Read more from Drinking Kazu
Published on HackMD