# QDSim - Quantum Dot Simulator Package

## Project Details

**Applicant:** Valentina Gualtieri
**Faculty:** QuTech
**DCC members:** Manuel Garcia (RSE) and Yasel Quintero (lead RSE)
**Support period:** July 2024
**Repository/Archive:** [https://gitlab.com/QMAI/papers/qdsim](https://gitlab.com/QMAI/papers/qdsim)

## Research Background

QDSim stands for Quantum Dots simulator. It efficiently and quickly simulates charge stability diagrams for large quantum dot devices (even 100+ dots). It was developed to make the simulation process as quick and as user-friendly as possible.

## Project Description:

The project aimed to address several key challenges associated with QDSim:

- **Single Simulation Per Program Execution**
QDSim was limited to running only one simulation per program execution. To thoroughly analyze a single device, users had to repeatedly execute the software, adjusting configurations each time. This process was inefficient and time-consuming.

- **Complex Installation of Third-Party Dependencies**
QDSim relies on the MOSEK and SCIP mathematical solvers, which are not available as standard _conda_ or _pip_ packages. Installing these solvers required manual setup through traditional GUI-based installers, posing significant challenges, particularly when deploying QDSim on HPC clusters.

In addition, the project prioritized establishing QDSim as a fully functional open-source Python package by implementing:
- a pyproject.toml file for easy package installation
- testing
- a ReadtheDocs website to host code documentation and user tutorials

## Project goals

The DCC aimed to provide hands-on support in implementing solutions for the challenges described above. The support activities included:
- Designing and implementing a parallelization strategy for QDSim that allows users to execute multiple simulations at once
- Simplifying the installation process for third-part dependencies
- Developing tests for old and new functionality
- Setting up a ReadtheDocs project website

Additionally, the project aimed to facilitate knowledge transfer in the following areas:
- Git Flow development workflow
- programming, testing, and documentation best practices
- utilizing HPC clusters like DelftBlue

## Project Results

By the end of the support period, the project achieved the following outcomes:

- **Parallelization with MPI:** QDSim was enhanced to support parallel simulations using the Message Passing Interface (MPI) via mpi4py. This allowed multiple devices and configurations per device to be simulated simultaneously. The implementation included an input file to define the setup for all simulations.
- **Dependency management with Conda:** A Conda environment was introduced to simplify dependency management. MOSEK, SCIP, and other Python packages, were listed in a requirements.txt file, enabling seamless environment creation.
- **Documentation:** A ReadtheDocs website was deployed to host the application's docstrings and user tutorials, providing accessible and organized documentation for end-users.
- **Easy installation:** A pyproject.toml file was implemented to facilitate a standardized and straightforward installation process for the application.

## Lessons learned

### Collaboration
The collaboration setup for this project worked well as it allowed for balanced collaboration, feedback, and independent work. It consisted of:

- **RSE Roles:** The lead RSE worked 2.5 days/week (Monday, Wednesday, Friday), and the partnering RSE worked 1 day/week (Monday).
- **Researcher Contribution:** The researcher contributed 4 hours/week (Monday and Wednesday).
- **Weekly Workflow:**
- **Monday:** Morning stand-ups (30–60 mins) between the RSE's to plan weekly tasks and review prior progress. Afternoon meetings with the researcher for feedback, gathering requirements, and knowledge transfer.
- **Wednesday:** Morning collaborative sessions between the lead RSE and the researcher, recorded for future reference.
- **Friday:** Lead RSE focused on solo development in the afternoon.

### Technical lessons:
- The Message Passing Interface standard can be used in Python via the [mpi4py](https://github.com/mpi4py/mpi4py) package.
- Conda environments work well in HPC clusters! In DelftBlue, a special configuration can be used to store the conda envs in the /scratch partition to prevent the user's /home directory from running out of storage space. See the [DelftBlue documentation](https://doc.dhpc.tudelft.nl/delftblue/howtos/conda/) for more details.
- Deploying a [Ray](https://docs.ray.io/en/latest/index.html) application on Slurm (e.g. on an HPC cluster) is extremely complicated. See the [Ray documentation](https://docs.ray.io/en/latest/cluster/vms/user-guides/community/slurm.html) for details.