# Live Debugging: SSH into a Running GitLab CI or GitHub Action Job

I've always like the idea of being able to ssh into a running job to debug CI-specific things like firewall rules.

## Live Debugging in GitHub Actions

In GitHub Actions, the solution is super simple. Just use the action [`owenthereal/action-upterm`](https://github.com/owenthereal/action-upterm):

name: CI
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: Setup upterm session
      uses: owenthereal/action-upterm@v1
      with:
        limit-access-to-actor: true

**Security implications:** `limit-access-to-actor: true` makes sure only the pub keys of the person who triggered the workflow can SSH into the Action. Note that the entire TCP traffic is going through [owenthereal/upterm](https://github.com/owenthereal/upterm)'s servers, but since it is being encrypted (SSH), so all good. We could improve the situation by running our own upterm server.

## Live Debugging in GitLab CI

With GitLab CI, Actions don't exist and you have to craft your own. To make things worse, you can't find examples of people using `upterm` in their `.gitlab-ci.yml` files since GitLab code search isn't available on the public gitlab.com instance... (one of the many reasons one should prefer GitHub over GitLab).

> Tested on 7 March 2025.

Here is the commands I use. You can also copy-paste these commands into your job's `script` block as long as it is an Alpine image. For Debian-based images, adjustments will need to be made.

# Alpine version.
test:
  image: alpine:3.19
  script:
    # Heavily inspired from the GitHub Action owenthereal/action-upterm, you can
    # look into its index.ts to understand the below tricks.
    - apk add --no-cache curl tmux openssh-client
    - curl -sSfL https://github.com/owenthereal/upterm/releases/download/v0.14.3/upterm_linux_amd64.tar.gz | tar xz -C /usr/local/bin
    - mkdir -p ~/.ssh
    - ssh-keyscan uptermd.upterm.dev 2>/dev/null | awk '{ print "@cert-authority * " $2 " " $3 }' | tee /dev/stderr >> ~/.ssh/known_hosts
    - ssh-keygen -t ed25519 -N "" -f ~/.ssh/id_ed25519
    - tmux new -d -s upterm-wrapper -x 132 -y 43 "upterm host --accept --force-command 'tmux attach -t upterm' -- tmux new -s upterm -x 132 -y 43"
    - tmux set -t upterm-wrapper window-size largest; tmux set -t upterm window-size largest
    - until upterm session current --admin-socket ~/.upterm/*.sock; do sleep 5; done
    - until ! ls ~/.upterm | grep -q '\.sock$'; do sleep 5; done

# Debian and Ubuntu version.
test:
  image: alpine:3.19
  script:
    # Heavily inspired from the GitHub Action owenthereal/action-upterm, you can
    # look into its index.ts to understand the below tricks.
    - apt update && apt install -y tmux openssh-client
    - curl -sSfL https://github.com/owenthereal/upterm/releases/download/v0.14.3/upterm_linux_amd64.tar.gz | tar xz -C /usr/local/bin
    - mkdir -p ~/.ssh
    - ssh-keyscan uptermd.upterm.dev 2>/dev/null | awk '{ print "@cert-authority * " $2 " " $3 }' | tee /dev/stderr >> ~/.ssh/known_hosts
    - ssh-keygen -t ed25519 -N "" -f ~/.ssh/id_ed25519
    - tmux new -d -s upterm-wrapper -x 132 -y 43 "upterm host --accept --force-command 'tmux attach -t upterm' -- tmux new -s upterm -x 132 -y 43"
    - tmux set -t upterm-wrapper window-size largest; tmux set -t upterm window-size largest
    - until upterm session current --admin-socket ~/.upterm/*.sock; do sleep 5; done
    - until ! ls ~/.upterm | grep -q '\.sock$'; do sleep 5; done

### Notes for Live Debugging in GitLab CI

Note that you can't just do:

test:
  image: alpine:3.19
  script:
    - upterm host --force-command 'tmux attach -t upterm' --accept -- tmux new-session -d upterm

Not sure why... but when trying to connect, you would get "Connection closed":

$ ssh psxScav2LZnOJugeSE8A:ZTc4NGUxMWRiMjI5MDgudm0udXB0ZXJtLmludGVybmFsOjIyMjI=@uptermd.upterm.dev
Connection closed by 37.16.25.99 port 22