# Running autobids locally (LocalBids)

The two components of autobids are cfmm2tar and tar2bids:

- **cfmm2tar**: Downloads the dicom files directly from the CFMM dicom server into tar files.
- **tar2bids**: Convert those dicom-tar files to the BIDS format.

We have containers for both components available in the CBS server. This document assumes we are working from the CBS server and will use those containers.

In case of not having access to the CBS server you have these options:
- Create an account. Instructions [here](https://osf.io/k89fh/wiki/Computational%20Core%20Server/?view_only=8281493cc75d429285735c98c1267261).
- Install [apptainer/singularity](https://apptainer.org/) locally (requires linux). We can provide a copy of the singularity containers used in this document.
- Windows users:
- Install apptainer/singularity using the [WSL linux subsystem](https://learn.microsoft.com/en-us/windows/wsl/install). (Recommended)
- Create and use the containers using docker. Instruction to download a cfmm2tar docker container in its github [repository](https://github.com/khanlab/cfmm2tar). Tar2bids [repository](https://github.com/khanlab/tar2bids) contain a docker file to create the container image locally.

# cfmm2tar

cfmm2tar retrieves studies from dicom server based on Principal^Project, Date and/or PatientName. If no search strings provided, all studies are downloaded.

Running the container will prompt for UWO username/password. The UWO account has to have access to the project in the DICOM server to download the data. Optionally you can create the a with the credentials in ~/.uwo_credentials (line1: username, line2: password) so cfmm2tar read the credential from there.

## Usage
If we want to download all the data in the project *Principal^Project* into the folder *dataTar*, we do the following:

singularity run /srv/containers/cfmm2tar_v1.0.0.sif -p 'Principal^Project' dataTar`

To download the data of a project of a specific date:

singularity run /srv/containers/cfmm2tar_v1.0.0.sif -p 'Principal^Project' -d '20240530' dataTar

date format: year/month/day

To download the data of project within a range of days

singularity run /srv/containers/cfmm2tar_v1.0.0.sif -p 'Principal^Project' -d '20240530-20240827' dataTar

For the full list of options and more examples, you can access cfmm2tar help just running the container without options:
`singularity run /srv/containers/cfmm2tar_v1.0.0.sif`

The resulting files will have a long filename that container info of the PI, project, date of acquisition and patient ID.
# tar2bids
tar2bids converts the dicom-tar files into the BIDS standar. It assumes the tar files are the output of cfmm2tar.

tar2bids uses heudiconv as its engine for the bids conversion. Heudiconv internally uses dcm2niix to convert the dicom files into niftis and then organize them into the BIDS standard following the criteria in a python file called the "heuristic". If you already have an autobids projects, the heuristic file should be stored in https://github.com/khanlab/tar2bids-heuristics/tree/main/heuristics . Identify your file and download it to use it locally. If you are a new autobids user, there is more information at the end of this document to create a new heuristic file.

## Usage
For example, for the 'subject.tar' file in *dataTar* from the last step (the name of the files will be longer, this is just an example name), with the heuristic file *myheuristic.py*. If we want to convert the tar file into the folder *BIDS* we do the following:

singularity run /srv/containers/tar2bids_v0.2.5.sif -h myheuristic.py -o BIDS subject.tar

Giving the same output *BIDS* folder will ensure the BIDS standard is respected while running tar2bids for multiple subjects.

For the full list of options, you can access tar2bids help just running the container without options:
`singularity run /srv/containers/tar2bids_v0.2.5.sif`
For example, the option -D will deface T1w images.

### Renaming files
tar2bids uses a search pattern for file name to automatically detect the subject Id. If the files are coming from a cfmm2tar the conversion should work. I have encountered a couple of cases when tar2bids struggle to properly parse the subject ID. An easy solution is to rename the files from cfmm2tar into something easily readable, for example to *sub-001.tar* and the use the option *-P sub-'{subject}'* to help tar2bids to do the proper parsing. BUT it wont probably be necessary for the majority of cases.

## The heuristic file
The heuristic files contain the BIDS fields and criteria to organize the nifti files into the BIDS standard. Currently there is a default heuristic in tar2bids called *cfmm_base.py* that have a collection of standard BIDS criteria for multiple MRI acquisition for the 3T and 7T scanners. You can use this file if you don't use the *-h* option.

If you want to create your own heuristic file I recommend following heudiconv [tutorial](https://heudiconv.readthedocs.io/en/latest/custom-heuristic.html). After that, you can study [cfmm_base.py](https://github.com/khanlab/tar2bids/blob/master/heuristics/cfmm_base.py) for multiples use cases for the CFMM scanners.

In a nutshell, the heuristics file has three sections:

**Creating keys**
We define the keys of the multiple BIDS entries in our study. These keys have the path of the BIDS structure that we want to create, and they use predefined variables (in the example the ones in {}) that read directly from the metadata to fill some predefined variables (e.g. {bids_subject_session_dir} refer to the current subjec BIDS variable).

In this example we create a key that will write in the dwi folder for an

dwi_ufa = create_key('{bids_subject_session_dir}/dwi/{bids_subject_session_prefix}_dir-AP_run-{item:02d}_dwi')

If this was *sub-001*, the resulting BIDs for *run 01* file will be: `sub-001/dwi/sub-001_acq-mde_dir-AP_run-01_dwi.nii.gz`

**Listing the keys**
We fill a dictionary with the keys we created. In a way we just list the keys we want to use in the dictionary.

info[dwi_ufa]=[]

**The conversion criteria**
In this step we establish a criteria for heudiconv to process the multiple nifti files into the BIDS standard with the keys that we already defined. The nifti files are temporally converted from the DICOM files internally by heudiconv using dcm2niix. The criteria use string matching cases in the metadata fields to identify the needed files.

In this example, we search for the string "UFA_PA" in the protocol name metadata. If the string is there, we label this entry as part of the dwi_mde key that we defined in the first section. This nifti file will be moved to the BIDS entry defined there.

    if ('UFA_AP' in s.protocol_name):
	   info[dwi_mde].append({'item': s.series_id})

Importantly, its possible to import the definitions in cfmm_base.py and add new ones for your specific needs. I recently created an heuristic that does this, you can check it [here](https://github.com/khanlab/tar2bids-heuristics/blob/main/heuristics/hyades_heuristic.py) (The examples above come from this file).