# How to use Heimdall to search for FRBs
Feel free to ask me if you have any questions. My email: Simon.Ho @ anu.edu.au (remove the space for the correct email address)
### Things you'll need
* Hardware(s)
* GPU
* sinteractive --account=oz002 --cpus-per-task=2 --mem=20g -p ${SYS_ARCH} --time=1:00:00 --tmp=10G --x11 --gres=gpu
* Software(s)
* [PSRCHIVE](https://psrchive.sourceforge.net/index.shtml): for converting .fits to .fil
* [Heimdall](https://sourceforge.net/p/heimdall-astro/wiki/Use/): for FRB searching
* Python
* [Optional] Sigpyproc

### **What is .fits?**
Flexible Image Transport System (FITS) is an open standard defining a digital file format useful for storage, transmission and processing of data: formatted as multi-dimensional arrays (for example a 2D image), or tables. They are widely used in astronomy.

### **What is .fil?**
Filterbank files are a binary data storage format which we use to store time series of power spectra (time-frequency data cube). They are normallly be used in radio astronomy.

## **Convert .fits to .fil**
FAST data are in .fits format but Heimdall only reads .fil. Therefore, we need to convert .fits to .fil. There are several ways you can convert .fits to .fil. Here, I demonstrate using PSRCHIVE to do the conversion.

1. To use PSRCHIVE on NCHU server, run this command below. This will activate the singularity environment.

`singularity shell -B /run/media/hashmttt:/mnt --bind $PWD /home/radio/psrchive.img`

Your command bar will appear as `Singluarity>` if you have sucessfully set up the environment.

1. Covert .fits to fil using digifil
`digifil -d 4 -b 8 filename.fits`

Where `-d 4` refers to the 4 polarizations, `-b 8` refers to 8 bits.

## **Running Heimdall**

To use Heimdall on NCHU server, run this command below. This will activate the singularity environment. (You need to type `exit` first to exit the environment if you are already in a singularity environment e.g. PSRCHIVE)

`singularity shell --nv -B /run/media/hashmttt:/mnt /home/radio/radio_transients_latest.sif`

To run heimdall, simply type `heimdall` in your terminal.

There are many options you can apply to heimdall before running it. you can do `heimdall -h` to check these options or go to [Heimdall](https://sourceforge.net/p/heimdall-astro/wiki/Use/) to check.
If you don't apply any options, heimdall will be run with the default options.

An example command is shown below.

`heimdall -baseline_length 4 -zap_chans 0 100 -zap_chans 1350 1500 -zap_chans 1600 1750 -zap_chans 2150 2300 -zap_chans 3270 3445 -zap_chans 3800 4096 -output_dir ./test -f xxx.fil`

In this command, I apply the option `-baseline_length 4` which smooth the baseline of the data with a factor of 4 before searching. `-zap_chans` flags the frequecy channels (in bins) in the data so heimdall won't consider detection in these frequecy channels. `-output_dir` set the output directory of the .cand file.

After running heimdall, there will be a xxx.cand as the output file. The columns are.
1. Detection SNR
2. Candidate Sample Number (for highest frequency channel)
3. Candidate Time (s)
4. Boxcar Filter Number (e.g. 3 equates to 2^3 or 8 samples)
5. Dispersion Measure Trial Number
6. Dispersion Measure (in pc cm^-3)
7. Members (the number of individual boxcar/DM trials that were clustered into this event)
8. First sample that the candidate spans
9. Last sample that the candidate spans

See [Heimdall](https://sourceforge.net/p/heimdall-astro/wiki/Use/) for more informations.

With the candidate time, you can now check the FAST data to see if it is a real detection.
However, there would be many rows in the .cand file as some RFIs are not flagged and considered as detections by Heimdall.

**To remove these, we can set up criteria regarding the Detection SNR, Dispersion Measure, or Boxcar Filter Number so on.
For example, we can remove candidates with SNR<10 or with dispersion measure < 100**

## **Visualizing the data**
There are several ways you can do to visualize the FRB data. Python is an easy one to do the job. If you don't want to write the code, you can use the plotting code by Chenhui Niu in FAST_Miner.

* [FAST_Miner](https://github.com/peterniuzai/FAST_FRB)

You can also use Sigpyproc, which is a Python based package for checking time-domain radio data.

* [Sigpyproc](https://sigpyproc3.readthedocs.io/en/latest/)

Sigpyproc has a useful module that can remove RFI quickly.

`clean_rfi(method='mad', threshold=3, chanmask=None, custom_funcn=None, outfile_name=None, gulp=16384, start=0, nsamps=None, **plan_kwargs)`

sigpyproc manual
https://sigpyproc3.readthedocs.io/_/downloads/en/latest/pdf/

Here, I would just demonstrate how to use the existing code in FAST_Miner

1. Use the code in https://github.com/peterniuzai/FAST_FRB/blob/master/src/bursts_plot_single.py

1. Change the `fil_path =` to the path of your .fits file and `filename =` to the filename of your .fits file
2. Run the code
3. Then, you may see plotting like this