Live Debugging: SSH into a Running GitLab CI Job

I've always like the idea of being able to ssh into a running job to debug CI-specific things like firewall rules.

In GitHub Actions, the solution is super simple. Just use the action owenthereal/action-upterm:

name: CI
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: Setup upterm session
      uses: owenthereal/action-upterm@v1

But with GitLab CI, Actions don't exist and you have to craft your own. To make things worse, you can't find examples of people using upterm in their .gitlab-ci.yml files since GitLab code search isn't available on the public gitlab.com instance… (one of the many reasons one should prefer GitHub over GitLab).

Setting up Upterm in your GitLab CI job

Tested on 7 March 2025.

Here is the commands I use. You can also copy-paste these commands into your job's script block as long as it is an Alpine image. For Debian-based images, adjustments will need to be made.

test:
  image: alpine:3.19
  script:
    # Heavily inspired from the GitHub Action owenthereal/action-upterm, you can
    # look into its index.ts to understand the below tricks.
    - apk add --no-cache curl tmux openssh-client
    - curl -sSfL https://github.com/owenthereal/upterm/releases/download/v0.14.3/upterm_linux_amd64.tar.gz | tar xz -C /usr/local/bin
    - mkdir -p ~/.ssh
    - ssh-keyscan uptermd.upterm.dev 2>/dev/null | awk '{ print "@cert-authority * " $2 " " $3 }' | tee /dev/stderr >> ~/.ssh/known_hosts
    - ssh-keygen -t ed25519 -N "" -f ~/.ssh/id_ed25519
    - tmux new -d -s upterm-wrapper -x 132 -y 43 "upterm host --accept --force-command 'tmux attach -t upterm' -- tmux new -s upterm -x 132 -y 43"
    - tmux set -t upterm-wrapper window-size largest; tmux set -t upterm window-size largest
    - until upterm session current --admin-socket ~/.upterm/*.sock; do sleep 5; done
    - until ! ls ~/.upterm | grep -q '\.sock$'; do sleep 5; done

Notes

Note that you can't just do:

test:
  image: alpine:3.19
  script:
    - upterm host --force-command 'tmux attach -t upterm' --accept -- tmux new-session -d upterm

Not sure why… but when trying to connect, you would get "Connection closed":

$ ssh psxScav2LZnOJugeSE8A:ZTc4NGUxMWRiMjI5MDgudm0udXB0ZXJtLmludGVybmFsOjIyMjI=@uptermd.upterm.dev
Connection closed by 37.16.25.99 port 22