All I really need to know and didn't learn in kindergarden.

For things I did learn in kindergarden consult Robert Fulghum's book

Setting up

Local or remote

• Install conda for package management

# Download
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
# Run and follow instructions on screen
bash Miniconda3-latest-Linux-x86_64.sh

Important config files

• ~/.bashrc configuring your baseline shell (all commands here get execute when opening a new window or after typing bash)
• ~/.ssh/config configuring your ssh connection

Configuring remote work

• Two things you might want to do remotely
  • run jupyter notebooks and see them in your browser
  • edit files using your favourite editor like Sublime or vscode

Remote Editing

• This supposes you are using vscode, but will work with SublimeText and maybe others
• On your local machine install the rmate plugin for vscode
  • https://github.com/rafaelmaiolla/remote-vscode
  • In VScode
    • Press F1 and type "Remote: Start server"
    • You should see "Starting Server" in the bottom status bar
  • Modify your ~/.ssh/config to include (as example connection to lxplus)

    ​​​​​​Host lxplus*
    ​​​​​​  HostName lxplus7.cern.ch
    ​​​​​​  User <your-user-name>
    ​​​​​​  ForwardX11 yes
    ​​​​​​  ForwardAgent yes
    ​​​​​​  ForwardX11Trusted yes
    ​​​​​​Host *_f
    ​​​​​​  RemoteForward 52572 127.0.0.1:52572
    ​​​​​​  ExitOnForwardFailure yes
    

  • Now your regular connection is available as ssh lxplus
  • And to connect with enabled forwad simply run ssh lxplus_f
• On remote server
  • Install rmate while in your conda env -> pip install rmate
  • Creat a new file called ~/.rmate.rc and enter the following:

    ​​​​​​port: 52572 
    

  • Running rmate dummy_file.py will open a new file called "dummy_file.py" in your local editor window, which gets synced automatically to the remote. The connection will obviously get interrupted if your ssh connection gets interrupted

Remote jupyter

• On your local machine
  • Modify your ~/.ssh/config again to include one new line

    ​​​​​​Host lxplus*
    ​​​​​​  HostName lxplus7.cern.ch
    ​​​​​​  User <your-user-name>
    ​​​​​​  ForwardX11 yes
    ​​​​​​  ForwardAgent yes
    ​​​​​​  ForwardX11Trusted yes
    ​​​​​​Host *_f
    ​​​​​​  LocalForward localhost:8800 localhost:8800
    ​​​​​​  ExitOnForwardFailure yes
    

• Connect to remote as ssh lxplus_f which uses the above config
• On remote server
  • Install jupyter notebook in your conda environment -> pip install jupyter
  • Create an alias in your ~/.bashrc for convenience

    ​​​​​​alias jup="jupyter notebook --ip=127.0.0.1 --port 8800 --no-browser"
    

  • It's usually convenient to start your notebook within a screen such that it doesn't die if you get disconnected and you can still use the terminal
  • Running jup in your terminal will start a notebook print bunch of stuff including a link looking like this, which you can open in your local browser to view the remote notebook

  ​​​​     or http://127.0.0.1:8800/?token=e36d18232a77b59af020ea09c76f495bde
  

Connecting through more than one tunnel

• The above piping works just as well for chained connections, such as connecting to one cluster and connecting to another that's only accessible from there.
• There is no trick, simply modify the ~/.ssh/config on the first remote with the same forward option as your local machine.

Connecting to more than one remote

• You might want to work simultaneously on more remotes. To do this the forwards need to be unique, so you'll likely need a new entry for your ~/.ssh/config such as:

Host *_f2
        LocalForward localhost:8801 localhost:8801
        RemoteForward 52573 127.0.0.1:52573
        ExitOnForwardFailure yes

• Don't forget to modify your jup alias in ~/.bashrc and the port number in ~/rmate.rc

Setting up git and GitHub

• You can interact with GitHub using two protocols https or SSH.
• https
  • Doesn't require authentication
  • Useful for one-off checkouts
  • git push will require typing your username and passwoord
• It's recommended to use SSH
  • Requires setup (once per machine), but will save time in long run
  • Follow instructions here:
    • https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent
  • Add the last set of lines to your ~/.bashrc

    ​​​​​​eval $(ssh-agent -s)
    ​​​​​​ssh-add ~/.ssh/id_rsa   # or replace id_rsa with whatever you named your key
    

  • Add the key in your GitHub account
    • https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/adding-a-new-ssh-key-to-your-github-account

Using git

• Its recommended to start with this tutorial from within CMS (Kevin Pedro) that has all the basics for the minimum you need to setup and start working
  • https://twiki.cern.ch/twiki/bin/view/CMSPublic/CMSGitTutorialPublic
• A great visual tutorial to learn the how to work efficiently with git
  • https://learngitbranching.js.org/

All you don't need to know

Powerline is convenient

• There are several implementations, the one that worked the best for me is
  • https://github.com/justjanne/powerline-go
• It's super convenient to keep track of which branch you are on in git
  

Setting up your proxy and dasgoclient