# GitHub webhook and API server for CI jobs

## Create a GitHub webhook for a repository or an organization

Refer to [this official documentation](https://docs.github.com/en/webhooks/using-webhooks/creating-webhooks) for webhook creation.

* Create a token for the webhook, save it in a safe place. This token will not be shown anywhere on GitHub later;
* Payload URL: `https://ci.x2d2.net/github-webhook`;
* Content type: `application/json`;
* Enable SSL verification;
* Select invididual events: `Workflow jobs`.

## Create an API server for the webhook to `POST` messages

### K8s namespace preparation

* create a namespace on Spin, e.g. `ci`, on the development cluster;
* create a CNAME record pointing to `api.ci.development.svc.spin.nersc.org`;
* `api` is the ingress name we will be using in the next step;
* Follow Option 2 in [this instruction](https://github.com/dingp/spin-acme) to install `spin-acme` Helm Chart into the namespace.

### API server and backend script

* This is a pod wtih persistent storage where the API application will be stored;
* The pod runs a image with `python3` and `python3-pip` installed;
* The API server is provided by `litestar`;
* `litestar` is installed in a python virtual environment.

API server logic:

1. Verify the singature in the POST message;
2. Parsing the payload (json string), verify the GitHub user who triggered the workflow job event;
3. Call Superfacility API to run a job of creating ephemeral runner.

## Set up auto-scaling of self-hosted runners on Perlmutter with Superfacility API.

Refer to [GitHub documentation](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/autoscaling-with-self-hosted-runners) on creating ephemral runners.

1. Use GitHub CLI to request a token for creating a new runner ([Using GitHub CLI on Perlmutter](https://gist.github.com/dingp/e35f572ebc5b1e25834bca0019216149));
2. Go to the directory where the runner client is installed, create a new work directory for the runner;
3. Start the runner with the token, specify the work directory, and supply the `--ephemeral` option.

## Organization setup

* Create a robot account for the organization
* Generate a GitHub token, with proper permission scope to:
* manager runners for the org;
* publish packages (container images).

## Token/password involved:

* GitHub Personal Access Token, stored on Spin:
* used on API server for requesting a token for runner registration, valid for 1 hour;
* the requested token will be used by job script run on Perlmutter.
* SFAPI JWT stored on Spin:
* used on API server for requesting a token to dispatch job to start a GitHub runner on Perlmutter.

## Development environment setup

* Create directory on CFS, mount into Spin pod;
* SFAPI token etc...