# Software WG taskforce: simulator selection resource

Meeting 1: Wed 2 Nov, 1500 UTC

- scope limited to biophysically plausible neuroscience simulation models
- https://github.com/OCNS/SoftwareWG/issues/61

# Attendees

- Ankur Sinha (UCL: computational modelling, Open Source Brain, NeuroML: `ankur.sinha AT ucl DOT ac DOT uk`)
- Brent Huisman (FZJ: product owner for the Arbor simulator (arbor-sim.org). Interested in improving end-user experience.)
- Reema Gupta (LMU Munich)
- Marcel Stimberg (Institut de la Vision/Sorbonne Université; Brian simulator)
- Dinara Issagaliyeva (Charite)
- Eliane Rodrigues (University of Campinas)

# Task force chairs

- Eliane
- Marcel

## Chair responsibilities:

- sending meeting reminders
- running meetings
- being general points of contact for the task force
- liasing with other related communities (such as the Software WG that we are part of)

# Communication

- Stay on the GitHub issue for the time being

# Notes

:::info
**Consolidated notes are now here: https://hackmd.io/@mstimberg/HJXqYPMPo**
:::
- Taskforce: https://github.com/OCNS/SoftwareWG/issues/117
- Initial discussion: https://github.com/OCNS/SoftwareWG/issues/61
- Curation is more work than listing

# Goals

- Listing or curation?
- What do I use wizard, or does that require curation?
- popularity contest

# idea

- Free form tagging?
- start with existing list? e.g.

# (minimum) criteria

- open dev?
- last commit/commit freq
- Ankur's proposal for minimum crit: https://github.com/OCNS/SoftwareWG/issues/61#issuecomment-1214758892
1. Must be under a Free/Open Source License (and so, the source code should be publicly visible): this one is important to us. I'd like to be more strict and say "must not require proprietary tools to work", but given that GPUs etc. are necessary for what we do, as long as the tool itself is FOSS, that should suffice.
2. Should be installable on current systems: This is sort of my proxy for "must work". Initially, I was going to say "must have code activity in the last 2 years", but there are will be tools that are considered stable, do install and work, and are really just being maintained with minor bug-fixes as required. So, I tweaked the requirement to make it even looser: the tool must just be installable and should work. To formalise this, we could pick a commonly used OS like the latest Ubuntu LTS that a lot of CI uses, and we could (if we wanted to), also have GitHub actions that periodically test that the tools on the list remain installable using CI.
3. Must have a point of contact: Again, I'd like to say "must have a mailing list/forum", but to keep it suitably loose, I'd just say that it needs to have a PoC (or a way of contacting a PoC). This could be anything GitHub issues, forums, mailing lists, a private e-mail. I don't think we can practically check if folks do receive a response, but we can check if a way of contacting them exists.
4. Must have user documentation: there should be something out there---tutorials, rtd, website, slides. Something to help users learn to use the tool.

We only want to exclude items from the list if adding them on the list would be worse than not having them

# Tasks

- Go over one of the lists (or not!), to find outliers to check our inclusion criteria: does it need to be trimmed down?
- List simulator features/properties that we think maybe useful
- ACTION: Ankur set up table for outliers: DONE
- ACTION: Ankur set up sections for task force members to list simulator features/properties: DONE (Someone (Brent?) did it already)

# Next meeting

Next meeting: 2PM UTC on 16 November.

## Existing lists

- https://github.com/asoplata/open-computational-neuroscience-resources#simulation-software
- https://compneuroweb.com/sftwr.html#modeling

# Our list (amend at will!)

- Arbor
- website: arbor-sim.org
- tags: multicompartment, hpc, gpu, python, c++, large-scale, neuroml, neurolucida, swc, nmodl
- floss license: BSD
- installable: lin/win/macos
- Point of contact: Gitter https://gitter.im/arbor-sim/community, Github https://github.com/arbor-sim/arbor/, email [e-mail]
- user docs: docs.arbor-sim.org
- status: maintained
- inclusion: yes :)
- Arbor GUI
- website: https://github.com/arbor-sim/gui/
- tags: multicompartment, hpc, gpu, python, c++, large-scale, gui,
- floss license: BSD
- installable: lin/win/macos
- Point of contact: Gitter https://gitter.im/arbor-sim/gui, Github https://github.com/arbor-sim/gui/issues
- user docs: https://github.com/arbor-sim/gui/
- status: maintained
- inclusion: yes
- Brian
- website: https://briansimulator.org/
- tags: python
- license: CeCILL 2.1 license
- installable:
- Point of contact: [discourse](https://brian.discourse.group/), github issues
- user docs: https://brian2.readthedocs.io/en/stable/
- status: maintained
- inclusion: yes
- BrainDynamics Tool box
- website: https://bdtoolbox.teachable.com/
- tags: dynamical systems, matlab, ode
- floss license: BSD but requires Matlab (Octave explicitly noted to not be supported)
- installable: lin/win/macos (Matlab!)
- PoC: e-mail
- user docs: a free beginner level course, and chapter 1 of handbook is free; paid course and full handbook needs to be purchased
- inclusion: yes
- BMTK
- website: https://alleninstitute.github.io/bmtk/ , https://alleninstitute.github.io/bmtk/index.html#base-installation
- tags: sonata
- floss license: BSD
- installable: Lin/win/macos (Python, but may require other tools: NEST/dipde etc. so may depend on individual tools in the stack)
- PoC: issues/PRs
- user docs: https://alleninstitute.github.io/bmtk/
- inclusion: yes
- Nest
- website: https://nest-simulator.org
- tags: python, c++, gui, large-scale
- license: GPLv2
- installable: lin/win/macos
- PoC: GitHub, mailing list, e-mail
- user docs: https://nest-simulator.org
- status: maintained
- inclusion: yes
- Neuron (and CoreNEURON)
- website: https://neuron.yale.edu/neuron/
- tags: python, gpu, c/c++, gui, compartment,
- license: BSD
- installable: lin/win/macos
- point of contact:
- user docs: [CoreNeuron](https://nrn.readthedocs.io/en/latest/coreneuron/index.html) [Neuron](https://nrn.readthedocs.io/en/8.2.1/)
- status: maintained
- inclusion: yes
- LFPy
- NetPyNE
- PyNN
- website: http://neuralensemble.org/PyNN/
- tags: python,
- license: CeCILL
- installable: lin/macos
- point of contact: GitHub, [NeuralEnsemble group](groups.google.com/group/neuralensemble)
- user docs: http://neuralensemble.org/docs/PyNN/
- inclusion: yes
- Genesis
- MOOSE
- website: https://moose.ncbs.res.in/
- tags: Python, GUI, C++
- license: GPLv3
- installable: Linux and MacOSX
- point of contact: mailing list,
- user docs: https://moose.ncbs.res.in/readthedocs/index.html
- tutorials and examples: https://github.com/BhallaLab/moose-examples
- status: maintaned (last commit 2020)
- inclusion:
- other: NeuroML reader available
- TheVirtualBrain (TVB)
- website: https://www.thevirtualbrain.org/
- tags: python, large-scale, LFP, fMRI, MEG, BOLD, gui, connectome, docker
- license: Baycrest
- installable: lin/win/macos
- point of contact: GitHub, email
- user docs: https://docs.thevirtualbrain.org/
- status: actively developed
- inclusion: hopefully yes!
- NeuroRD
- website: https://krasnow1.gmu.edu/CENlab/software.html
- status: maintained
- license: GPLv2
- tags: reaction-diffusion, java
- installable: platform independent (Java)
- PoC: GitHub
- user docs: tutorials, publications and published models (also on ModelDB)
- inclusion: yes
- Smoldyn
- website: https://www.smoldyn.org/
- status: maintained
- license: LGPL + GPLv3 + public domain
- tags: reaction-diffusion, biochemistry, Python
- installable: Linux/Mac/Windows
- PoC: e-mail
- user docs: on website
- inclusion: yes
- Steps
- website: https://steps.sourceforge.net/STEPS/default.php
- status: maintained
- license: GPLv2
- tags: reaction-diffusion, Python
- installable: Linux/Mac/Windows (but not sure if explicitly mentioned)
- PoC: GitHub discussions, e-mail
- user docs: [on website](https://steps.sourceforge.net/manual/manual_index.html)
- inclusion: yes
- MUSIC (MUlti-SImulation Coordinator)
- website: https://github.com/INCF/MUSIC
- tags: large-scale, c++, python, multi-simulation
- license: GPL-3.0
- installabel: lin/
- point of contact: GitHub
- user docs: [pdf file](https://raw.githubusercontent.com/wiki/INCF/MUSIC/files/music-manual.pdf)
- status: last update 11 months ago
- inclusion: ?
- NeuroML
- website: https://neuroml.org
- tags: standard, LEMS, multi-scale, ecosystem, multi-simulator
- license: GPL
- installable: linux/windows/mac (java + python, with C++ and matlab bindings also available)
- PoC: Github, mailing list, e-mail
- user docs: https://docs.neuroml.org
- status: maintained
- inclusion: ?
- Brain dynamics toolbox
- website: http://bdtoolbox.org/about/
- status: maintained
- license: BSD
- tags: MATLAB, non-linear dynamical systems, gui, large-scale, theoretical
- installable: win/macos
- PoC: not specified
- user docs: need to purchase the handbook, the first chapter is free [here](https://github.com/bdtoolbox-org/bdtoolbox/releases/download/2022b/HandbookSample2022.pdf)
- note: requires MATLAB + purchasing the handbook, not sure if needed to be excluded
- inclusion: ?
- c302
- website: https://www.opensourcebrain.org/projects/c302
- status: maintained (last commit 8 months ago)
- tags: multi-scale, single & multi-compartment, python, docker
- installable: Linux/Mac/Windows (but not sure if explicitly mentioned)
- PoC: not specified
- user docs: no docs, just the link to the website: https://www.opensourcebrain.org/projects/c302
- note: uses libNeuroML, jNeuroML & pyNeuroML
- inclusion: ?
- MiMiC
- website: https://mimic-project.org/en/latest/
- status: active development
- tags: multi-scale, python
- installable: lin/macos
- PoC: not specified
- user docs: https://mimic-project.org/en/latest/
- note: introduced in the Simulate with EBRAINS event
- inclusion: ?

- ReMoto- Simulation system of spinal network and the innervated muscles
- website: http://remoto.leb.usp.br
- tags: non-linear dynamical systems, gui, multi-scale, neuromuscular simulator
- license: website
- installable: linux/windows/mac (java)
- PoC: e-mail
- user docs: on website
- status: active development
- inclusion: ?

- Brain Scaffold Builder
- website: https://bsb.readthedocs.io
- tags: multiparadigm neural modelling, code-free configuration , parallelized workflow
- license: GPL3
- installable: linux/windows/macos
- PoC: https://github.com/dbbs-lab/bsb/issues/
- user docs: https://bsb.readthedocs.io
- status: active
- inclusion: ?

- BluePyOpt
- website: https://bluepyopt.readthedocs.io
- tags: multiparadigm neural modelling, code-free configuration , parallelized workflow
- license: Mixed GPL, LGPL, BSD
- installable: linux/windows/macos
- PoC: https://github.com/BlueBrain/BluePyOpt/issues
- user docs: https://bluepyopt.readthedocs.io
- status: active
- inclusion: ?

- nrn-patch
- website: https://github.com/helveg/patch
- tags:
- license: MIT
- installable: linux/windows/macos
- PoC: https://github.com/helveg/patch/issues
- user docs: -
- status: ?
- inclusion: ?

- DiPDE
- website: https://alleninstitute.github.io/dipde/index.html
- tags:
- license: GPL
- installable: build passing on linux, pip installable
- PoC:
- user docs:
- status:
- inclusion:

## General comments/questions regarding tags/metadata (Marcel)

* how do we make sure "beginners" understand tags correctly ("multi-scale", "multi-simulation", how large is "large-scale", etc.)
* we need a glossary, linking to further materials explaining these concepts.
* A modeling hierarchy like eg https://docs.arbor-sim.org/en/latest/ecosystem/index.html#simulators
* difference between "comes with three pre-defined neuron models" and "arbitrary neuron models"

# Ideas for criteria

- instead of is-installable (Ankur point 2), has-ci
- not enough projects have this, so manual installation testing is required.
- scope: must simulate neuronal mechanisims (even if it's sub-cellular, it should be somehow related to a neuron---this will include glia/astrocytes too)
- when is the simulator too chemical?
- frameworks? eg netpyne, music, patch
- Ankur point 1: addition: must be accessible to all: not require logging in/signing up to download. I don't think this violates FOSS licensing since it can still be FOSS but it sort of violates the ethics/principles of FOSS. e.g. EONS
- programming language in tags: for users, I think we should note what language they can use the software in. e.g., something may be developed in C/C++, but if it has Python bindings that users are meant to use, we use the "python" tag instead of "C/C++".
- prerequisites: an account? proprietary software (MATLAB)? pay for documentation?
- interop: file formats, cosim, APIs?

## General comments/questions regarding criteria (Marcel)
* we focus users, not developers (I guess?), so "good practice" criteria such as CI less relevant?
* For long term use, they may be very relevant.
* do we need to make a difference between "frontend tools" like PyNN, BMTK, NetPyNE, and simulators? What about things like MUSIC?
* how do we deal with "subpackages", e.g. would only be Brian listed, or also Brian2CUDA (would Brian be listed with a GPU tag or not?)
* What if a tool has no ongoing development, but is still "active" in the sense that it is installable and the authors replies to issues (e.g. Auryn: https://github.com/fzenke/auryn/)
* Unknown/Unmaintained/Archived, functional/maintained, active dev?
* active dev: regular releases or commits (at least one commit within last month, one release in last year).
* maintained: occasional bug fixes (at least one in past year)
* unknown: unknown ;)
* What's documentation? A handbook? Website? Are the docs 'free'?
* An unpaid and freely accessible resource that covers a significant portion of the package's functionality.

# Excluded tools table

| Tool excluded | Why excluded? | Comment |
| ------------- | ------------- | --------|
| Abstracted Protein Simulator | out of scope | protein simulator, not quite in our research domain |
| BioNetGen | out of scope | general purpose biochemical mechanism simulator |
| Copasi | out of scope | general purpose chemical pathway simulator |
| E-cell | out of scope | general purpose cell simulator |
| EONS | unmaintained?, free but requires login to download, unclear if FOSS | login link doesn't work for me |
| Meredys | link from Jim's page doesn't work | GitHub search didn't return anything either |
| Modigliani | unmaintained | a repo on [GitHub](https://github.com/rouzbeh/Modigliani) has the last commit from 2018, website link from Jim's page returns nginx test page |
| psics | unmaintained? | last changelog entry [from 2011](http://www.psics.org/about/log.html), link to forum also does not work |
| Brain Lab | limited scope, proprietary(?), unmaintained | teaching resource, not really a simulator ; limited to ipad/iphone ; website link from Jim's page doesn't open for me, so limited info available |
| [Conical](https://strout.net/conical/package/doc/note-release.html) | unmaintained | last release, page update from 1996 |

# Unsure tools table

| Tool| Comment |
|-----|---------|
| [CalC](https://web.njit.edu/~matveev/calc.html)| Intracellular Ca dynamics, but not necessarily neuronal specific |
| [Mcell](https://mcell.org/) | general cell micro-physiology simulator: in scope or not? |
| [VCell](https://vcell.org/why-vcell)| general cell simulator: in scope or not? |
| [c302](https://github.com/openworm/c302/) | isn't really a simulator, looks like scripts to generate NeuroML models for C. elegans (can double check with Padraig) |

### Shared sources to check models
- [ModelDB](https://senselab.med.yale.edu/modeldb/)
- [Fedora Project](https://docs.fedoraproject.org/en-US/neurofedora/related-links/#_neuroscience_databases)
- [Open source brain](https://www.opensourcebrain.org/projects)