---
# System prepended metadata

title: Setup Development Server for Jupyter

---

# Setup Development Server for Jupyter

**Basic Tools**
 - Install conda
      
          # Move to /opt directory and switch to su (adminisrative) user
          
              $ cd /opt 
              $ sudo su
          
          # Download and install anaconda
          
              $ wget https://repo.anaconda.com/archive/Anaconda3-2022.05-Linux-x86_64.sh
          
              $ bash Anaconda3-2022.05-Linux-x86_64.sh -b -p /opt/anaconda3
              # During installation choose /opt/anaconda3 as the install directory

\* You will need to type `exit` to logoff administrative user and disconnect/reconnect from the machine. When you reconnect your prompt should look like 

**Link conda tools to path-searchable folders**

    # make sure all conda packages will be in path by symbolic links to /bin
    # this step is a bit of a hack and you may get some warnings about
    # symbolic links that cannot be created - it's ok.
    
        $ ln -s /opt/anaconda3/bin/* /bin
        
        $ ln -s /opt/anaconda3/pkgs/*/lib/* /usr/lib

**Configure Bioconda**

This will help make installations smoother and the `-c bioconda` and smiliar commands won't be needed. 

    conda config --add channels defaults
    conda config --add channels bioconda
    conda config --add channels conda-forge
    conda config --set channel_priority strict


**Confgure and Run Jupyter**

Setup a password for the server

    jupyter notebook password

Intsall the bash kernel 

    pip install bash_kernel
    python3 -m bash_kernel.install

Start at tmux session so that this program can run in the background 

    tmux new-session -n jupyter

Run the command to start the Jupyter server

    jupyter-lab --no-browser --allow-root --ip=0.0.0.0 --port=8001 --notebook-dir='/home/nano_user'

To leave the tmux screen hit `CNTRL`+`b` and then `d` this key combination is the "detatch" command. See [tmux docs]().


**Install Jupyter Book**


**Install iCommands**

Ubuntu 20.04

```
wget -qO - https://packages.irods.org/irods-signing-key.asc \
  | sudo apt-key add -
echo "deb [arch=amd64] https://packages.irods.org/apt/ focal main" \
  | sudo tee /etc/apt/sources.list.d/renci-irods.list
sudo apt update
sudo apt install irods-icommands
```



## Configuring an instance to automaticaly run jupyterlab

### Script to start jupyter (exe)

```
# Created a script that starts a Jupyter hub
# This relies on jupyter-notebook password having been run 
# with a configuration json/hashed password in .jupyter/ in 
# the lab start folder
```
/home/exouser/jstart.sh 
```bash
#!/bin/bash

# Start a jupyter session
jupyter-lab --no-browser --allow-root --ip=0.0.0.0 --port=8001 --notebook-dir='>

```

### Script to start tmux (exe)

```
# Created a script that runs the above jupyter script
# in a detached tmux session
```
/home/exouser/tstart.sh 
```bash
#!/bin/bash

# Create a new session named "jupytertmux" and run a command inside the session
tmux new-session -d -s jupytertmux
tmux send-keys -t jupytertmux "bash /home/exouser/jstart.sh" Enter
```

Both scripts in `/home/exouser/` are made executable

```
chmod +x /home/exouser/tstart.sh 
chmod +x /home/exouser/jstart.sh 
```

### Entry to start at boot time (exe)

Edit the cronfile so that `tstart.sh` runs at boot

Open the crontab file
`crontab -e`

Add the line to the script
`@reboot bash /home/exouser/tstart.sh`



# Configuring Manilla Shares


*Create a share* 

- Following documentation here: https://docs.jetstream-cloud.org/ui/horizon/manila/
- Edit the share rule
    - Use Allocation name - i.e. 'BIO220126-nanopore-share'
        - This will create an access key


For this test in Dec 2022
- Key: AQDJOoljmdHMGRAAn/nIAC4aDVKoqINKcd37vQ== 
- Path: 149.165.158.38:6789,149.165.158.22:6789,149.165.158.54:6789,149.165.158.70:6789,149.165.158.86:6789:/volumes/_nogroup/e3390b33-2305-4713-b32e-46395eac9015/15d85743-1626-40e5-ade4-27fce24bfbf6


*Configure the VM to use the share*

https://docs.jetstream-cloud.org/general/manilaVM/

```
sudo su

mkdir /mnt/ceph

# Create a file and add the secret 

nano /etc/ceph.BIO220126-nanopore-share.secret

# add the secret in the above file

# Change permissions 

chmod 600 /etc/ceph.BIO220126-nanopore-share.secret

# Edit /etc/fstab to include the following 

nano /etc/fstab

149.165.158.38:6789,149.165.158.22:6789,149.165.158.54:6789,149.165.158.70:6789,149.165.158.86:6789:/volumes/_nogroup/e3390b33-2305-4713-b32e-46395eac9015/15d85743-1626-40e5-ade4-27fce24bfbf6 /mnt/ceph ceph name=BIO220126-nanopore-share,secretfile=/etc/ceph.BIO220126-nanopore-share.secret,x-systemd.device-timeout=30,x-systemd.mount-timeout=30,noatime,_netdev,rw 0   2

#Mount the share

mount -a

# confirm share is mounted

df -h|grep vol

```

I  also created a script if neded

```
#!/bin/bash

# This script will setup a VM to connect to a pre-configured share on JetStream2
# The script easily breaks if any of the values have changed so be careful

sharepath='149.165.158.38:6789,149.165.158.22:6789,149.165.158.54:6789,149.165.158.70:6789,149.165.158.86:6789:/volumes/_nogroup/e3390b33-2305-4713-b32e-46395eac9015/15d85743-1626-40e5-ade4-27fce24bfbf6'
sharekey='BIO220126-nanopore-share'
fstabline="$sharepath /mnt/ceph ceph name=${sharekey},secretfile=/etc/ceph.${sharekey}.secret,x-systemd.device-timeout=30,x-systemd.mount-timeout=30,noatime,_netdev,rw 0   2"


mkdir -p /mnt/ceph
touch /etc/ceph.${sharekey}.secret
sudo chmod 600 /etc/ceph.${sharekey}.secret


if grep "secretfile" /etc/fstab
then
    echo "Do nothing"
else
    echo $fstabline >> /etc/fstab
fi

```

**Setup files on the share**

Create directories for everyone. Technically, everyone is exouser so the names are really just a way of organizing. Trying to make the `chamecrista_fast5`, `notebooks`, and `tutorial_example` folder read only so that we reduce the chance of accidential writing over. 

```
sudo mkdir notebooks jagosto jlopez jwilliams afeitzinger candujar tutorial_example
sudo chmod 766 -R jagosto jlopez jwilliams afeitzinger candujar
sudo chmod 644 -R chamecrista_fast5 notebooks tutorial_example
```

# Setup a new running instance from the Jupyter base instance 

1. Access your server in a web browser: IP address:8001
2. Login with the password
3. Press the **Terminal** button to open a terminal
4. Paste in the following command to download the Setup notebook
    `wget https://github.com/JasonJWilliamsNY/genome_camp_2021/raw/main/genome_camp_2022/notebooks/setup_notebook.ipynb`
    
    You should get an output similar to this:
    ```
    --2022-12-05 05:27:27--  https://github.com/JasonJWilliamsNY/genome_camp_2021         /raw/main/genome_camp_2022/notebooks/setup_notebook.ipynb
    Resolving github.com (github.com)... 140.82.114.4
    Connecting to github.com (github.com)|140.82.114.4|:443... connected.
    HTTP request sent, awaiting response... 302 Found
    Location: https://raw.githubusercontent.com/JasonJWilliamsNY/genome_camp_2021        /main/genome_camp_2022/notebooks/Setup_notebook.ipynb [following]
    --2022-12-05 05:27:27--  https://raw.githubusercontent.com/JasonJWilliamsNY            /genome_camp_2021/main/genome_camp_2022/notebooks/setup_notebook.ipynb
    Resolving raw.githubusercontent.com (raw.githubusercontent.com)...                 
    185.199.111.133, 185.199.110.133, 185.199.108.133, ...
    Connecting to raw.githubusercontent.com     
    (raw.githubusercontent.com)|185.199.111.133|:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 13598 (13K) [text/plain]
    Saving to: ‘Setup_notebook.ipynb’

    setup_notebook.ipynb                       
    100%[===============================================================>]  13.28K      
    --.-KB/s    in 0s      

    2022-12-05 05:27:27 (60.4 MB/s) - ‘Setup_notebook.ipynb’ saved [13598/13598]
    
    ```
6. Look in the file navigation pane in your Jupyter lab window, you should see the Setup_notebook.ipynb. Double click to open this notebook and follow instructions to setup your computer's connection to the data share. 



