# DFT/FFT/G2G tutorial for beginner
This documentation gives a brief overview of how to use basic imaging pipelines (DFT/FFT/G2G) to input a nenuFAR-type measurementSet and output reconstructed images.

## Biblio
:page_facing_up: [G2G](https://arxiv.org/pdf/1712.02078) *N. Monnier, F. Orieux, N. Gac, C. Tasse, E. Raffin, D. Guibert «Fast Grid to Grid Interpolation for Radio Interferometric Imaging»*

:page_facing_up: [Generic imaging pipeline](https://hal.science/hal-04361151/file/paper_dasip24_5_wang_updated-2.pdf): *S. Wang, N. Gac, H. Miomandre, J.-F. Nezan, K. Desnos, F. Orieux « An Initial Framework for Prototyping Radio-Interferometric Imaging Pipelines »*.

:page_facing_up: [Pipeline sigularity image](soon_under_review_at_sips_2025): *O. Renaud, N. Gac, F. Orieux, C. Viou « SimSDP, a Rapid Prototyping tool for Radio Astronomy: From NenuFAR Experiments to SKAO-Scale Simulation »*

## Requirements
- Download pipelines singularity image: `sdp_pipeline.sif` [GitHub](https://github.com/Ophelie-Renaud/simsdp-generic-imaging-pipeline/tree/main/param_code)
- Having a measurementSet (i.e. **\*.ms** ==> set of folders)
- (option) Download the script that facilitates reading **\*.fits** (based on the ds9 tool): `dsm.py` [NAS - vaader](https://nasext-vaader.insa-rennes.fr/ietr-vaader/)

## Preliminary step

- Run the following commands to install dependencies and prepare your working environment:

# install singularity
sudo apt update
sudo apt install singularity-container -y

# Install DS9 viewer (for FITS files)
sudo apt install saods9

- Once everything is ready, you can start imaging pipelines inside the container:

# run the singularity environment
singularity run sdp_pipeline.sif <dft/fft/g2g> <NUM_VIS> <GRID_SIZE> <NUM_MINOR_CYCLE> <NUM_NODES> <MS_PATH>

## A step forward in research

The goal is to show:
1. The pipelines behavior at different scale through simulation (NenuFAR scale to validate simalution with true implementation, until SKAO scale)
2. The position of the pipelines in relation to the target real time

<iframe src="https://docs.google.com/spreadsheets/d/1TrOneD3_Yvld3oE4-2DfjHym0LoUFofHqwNGQ7sH8Eo/edit?usp=sharing" width="600" height="400"></iframe>

<p align="center">
<img src="https://hackmd.io/_uploads/S1kGRWOEll.png)" alt="Figure DDFacet" width="400"/>
<br>
_{The Figure presents in gray background, nenuFAR scale simulation varying the number of visibility from 10⁴ to 10⁷, with fixed minor cycle (1) and fixed grid size (2048) validated on SB155.rebin.MS; in red backgroud is the extrapolated simulation at SKAO scale varying the number of visibility from 10⁸ to 10⁺, with fixed minor cycle (1) and fixed grid size (32768).
<br>
Given that the general formula:
<br>
_{Data (bytes) = <span style="color:#e74c3c;">N_ANT</span> × (<span style="color:#e74c3c;">N_ANT</span> - 1) / 2 × <span style="color:#3498db;">N_CHANNEL</span> × <span style="color:#9b59b6;">N_POL</span>² × <span style="color:#f1c40f;">BANDWIDTH</span> (Hz) × 1/ <span style="color:#1abc9c;">ACCUMULATION_TIME</span> (s) ×<span style="color:#2ecc71;">OBS_TIME</span> (s) × <span style="color:#e67e22;">complex_value</span> × <span style="color:#34495e;">N_BYTE</span>}.
<br>
Hence regarding our experiments (ref. Table) SB155.rebin ≈ 0.01 Go and results in 1477782 visibilities.
<br>
The Real-time curve corresponds to a Latency = <span style="color:#2ecc71;">OBS_TIME</span> (s),
which implies a minimum processing rate of:
PROCESSING_RATE = <span style="color:#e74c3c;">1 477 782</span> / <span style="color:#2ecc71;">232</span> ≈ <strong>6,34E+03 visibilities/s</strong> at <strong>NenuFAR scale</strong>.
<br> Now we want to extrapolate to the <strong>SKAO scale</strong>. In the literature, targeted visibility data rates are reported around <strong>7.8 to 8.9 TB/s</strong>. Using the same formula structure and assuming the visibility rate scales proportionally with the number of antennas squared and the number of channels and polarizations.<br>
If we reverse-engineer the formula assuming: <span style="color:#f1c40f;">8.9 TB/s</span> / <span style="color:#34495e;">32 Byte</span> ≈ <strong>2,78E+8 visibilities/s</strong>. <br>
Or, if we extrapolate NenuFAR value to SKAO level.this correspond <strong>2,92E+10 visibilities/s</strong> for SKA-Mid and <strong>3,16E+12 visibilities/s</strong> for SKA-Low, i.e., a scale factor of over <strong>1E+04</strong> to <strong>1E+08</strong> according to the estimation method. <br> This justifies the need for massive parallelization and hardware acceleration, such as with GPUs or FPGAs, and explains why real-time performance becomes significantly more challenging at this scale.}
</p>

## Note
:warning: *This tutorial is in progress.*
Data are obtained fixing parameters:
- `NUM_MAJOR_CYCLES`: 5
- `OVERSAMPLING_FACTOR`: 16
- `NUM_KERNELS`: 17 , *i.e: totat number of w planes*
- `KERNEL_SUPPORT`: 4 , *pixels*

This is based on the experience of the developers and requires a real evaluation to obtain reliable values. Soluce consist in running DDFacet on the MS a computing parset value based on an obscur algo...

## Contact

For questions or feedback, please contact:

- This tutorial was written by [Ophélie Renaud](mailto:[e-mail]) from PREESM/SimSDP community (previously @INSA, @IETR, now @SATIE paris-saclay, @IRISA, @ENS Rennes :fr:).