---
# System prepended metadata

title: Synapse Seg Project

---

# Synapse Seg Project

## switch to HarrisLabGPU
```ssh khlab```
Only work in a tmux ```tmux```
detach wiht control + b then d
reattach with ```tmux a```
tmux ls to see active tmux
end tmux with ```exit```

### commands:
cds get into science directory
cdd get into data directory



## General Tips
1. compressing a zarr to send over email

```
tar -czvf compressed.tar.gz <location of zarr>
```

ssh: getting into dif machine terminal

## Installing miniconda3
For mac:
```
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh
```
```
chmod +x Miniconda3-latest-MacOSX-arm64.sh 
```
```
./Miniconda....
```
For khlabgpu:
```
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
```

## Uninstalling minconda3
```
pip install anaconda-clean
anaconda-clean
```
```
rm -r Miniconda3
```
## Downloading napari

1. Create + activate new environment
###### these can apply to creating any env, just replace napari with your env name
```
conda create -y -n napari -c conda-forge python=3.9
conda activate napari

conda create -n autoseg python=3.9
conda activate autoseg
conda install pytorch pytorch-cuda=11.7 boost jupyter -c pytorch -c nvidia
```
:::info
:bulb: **After creating + activating a new environment, download all modules in Notes w Vijay doc**
:::spoiler Installs
pip install cython zarr matplotlib mahotas

pip install git+https://github.com/funkelab/funlib.geometry.git

pip install git+https://github.com/funkelab/funlib.persistence.git

pip install git+https://github.com/funkelab/daisy.git

pip install git+https://github.com/funkey/gunpowder.git

pip install git+https://github.com/funkelab/funlib.math.git

pip install git+https://github.com/funkelab/funlib.evaluate.git

pip install git+https://github.com/funkelab/funlib.learn.torch.git

pip install git+https://github.com/htem/waterz@hms

pip install git+https://github.com/funkelab/funlib.segment.git

pip install git+https://github.com/funkelab/lsd.git

pip install neuroglancer cloud-volume

pip install tensorboard tensorboardx

pip install jsmin


#### Need to make some quick changes because there are some bugs in one of the packages you pip install: gunpowder.

Open in vim (or your preferred text editor) the following files and make the edits:
$CONDA_PREFIX/lib/python3.9/site-packages/gunpowder/nodes/add_affinities.py

Line 24 and line 187: change np.int32 to np.uint64

$CONDA_PREFIX/lib/python3.9/site-packages/gunpowder/nodes/random_location.py

Line 321: change size() to size

(ONLY FOR M1 MAC USERS:)
$CONDA_PREFIX/lib/python3.9/site-packages/gunpowder/torch/nodes/train.py (+predict.py)

Lines 164 and 165: change from this:
self.use_cuda = torch.cuda.is_available()
self.device = torch.device("cuda" if self.use_cuda else "cpu")
To:
self.use_mps = torch.backends.mps.is_available()
self.device = torch.device("mps" if self.use_mps else "cpu")

^^Basically change any instance of cuda to mps

:::
2. Download napari
If Qt being weird
run ```CONFIG+=sdk_no_version_check```
```
pip install "git+https://github.com/napari/napari.git#egg=napari[all]"
```
or
```
pip install "napari[all]""
```
or actually hopefuly
```
conda install -c conda-forge napari
```

3. Activate napari
```
napari
```
## Painting Synapses
:::info
[Click to familiarize yourself with neuron structures](https://synapseweb.clm.utexas.edu/atlas)
:::
1. Open raw images in napari
```
Click File -> Open Folder, navigate to raw in apical_crop.zarr, click Open
```
2. Add Labels layer, rename to labels
3. Draw on synapses
    a. Press 2 to activate brush setting
    b. click shuffle colors 
    c. draw on synapses
4. Create labels_mask layer and paint fill the biggest boundary around your painted synapses such that all synapses within this boundary are painted by you
5. Save your painted Labels layers individually into .tif files from napari as 'labels' and 'labels_mask' into your synapse project folder

:::info
Sections 87, 88, 89 are painted
:::

# ORRR
## Painting training data in PyReconstruct
Clone the pyrecontruct repository: ```git clone https://www.github.com/SynapseWeb/PyReconstruct PyReconstruct```

Switch to "autoseg-meeting" branch: ```git checkout autoseg-meeting```

Pull updates: ```git pull origin autoseg-meeting```

Hard reset, so you actually use this branch: ```git reset --hard origin/autoseg-meeting```

Activate environment:  ```source env/bin/activate```

Install required packages: ```pip install -r src/requirements.txt```

Run PyReconstruct: ```python3 src/PyReconstruct.py```

Open jser file
Trace
Save trace as zarr

## Preparing images to be trained on
1. Run the tif_labels_to_zarr.py script
```
python tif_labels_to_zarr.py ../labels.tif apical_crop.zarr 
python tif_labels_to_zarr.py ../labels_mask.tif apical_crop.zarr 
```

2. Check if you can view them in neuroglancer
```
python -i zarr_view.py apical_crop.zarr
```

3. Run the make_2d.py script
```
python make_2d.py apical_crop.zarr
```
View new 2d zarr
```
python -i view_stacked.py 2d_apical_crop.zarr
```

Convert datasets from uint8 to uint64 (jk maybe dont)
```
python convert_uint8_to_uint64.py 2d_apical_crop.zarr
```
## Create training environment
```
conda create -y -n train_synapse -c conda-forge python=3.9
```
Install modules from Notes w Vijay doc

## Train
```
PYTORCH_ENABLE_MPS_FALLBACK=1 python train.py 
```
To view loss curve during training:
```
tensorboard --logdir .
```
:::warning
tensorflow is weird on macs rn, create tf environment
```
conda create -n tf tensorflow
conda activate tf
```
https://docs.anaconda.com/anaconda/user-guide/tasks/tensorflow/
:::


View predicted lables:
```
python -i view.py snapshots/batch_1.zarr 
```
Predict on some untrained (83):
```
python predict.py <raw file> <raw ds> <section>
```

:::info

:::