# Guide: Setting up a private wally registry

## Author
This guide was written by Britton. If you notice any problems, or have any questions about the content of this guide, feel free to contact me.

## Contact Info
- Roblox: [Scarious](https://www.roblox.com/users/18982229/profile)
- Github: [nottirb](https://github.com/nottirb)
- Discord: Britton#5000
- Email: [[e-mail]](mailto:[e-mail])

# Foreword
Wally registries come in a few parts:

- The github wally package index, which stores version and package information, and allows local package lookup via CLI
- A cloud storage bucket to store the actual package files
- A backend API to connect to the storage bucket
- A frontend website

For this tutorial we will be focusing on the first 3 and ignoring the frontend website, as we do not care about the frontend website anyways.

**This tutorial will specifically be for Google Cloud Platform. Wally currently does not support any other cloud providers.**

# Step 0 \[Optional]: Create a Github Organization

To store the github package index, we can create a github organization. This could also be stored on your own github account, but if you're deciding to create your own package registry, you probably have some sort of organization setup already.

This step should be self-explanatory. Do this if you would like.

# Step 1: Create a 'wally-index' repository

Create a github repository named `wally-index`. You can now choose whether or not you want this to be public or private.

Public: No git authentication, but anyone on the internet can see package versions/metadata, along with your backend API's URL. They cannot access the actual package content/files/code without an API key though.

Private: Only people with access to your organization/repository can see package versions/metadata. Requires git authentication (i.e. you must be able to `git clone` the repository without entering a user/password combination).


Now we need to add a `config.json` file, which will store your api url, and fallback registries.

The config json should look as follows (we will change ``temp.com`` to something else after we have our API setup).

{
    "api": "https://temp.com",
    "fallback_registries": [
        "https://github.com/UpliftGames/wally-index"
    ]
}

Commit this to github.




# Step 2: Fork Wally from Github

Go to https://github.com/upliftgames/wally and fork wally into your organization.



# Step 3: Create a new gcp project

I named mine `test-wally-registry`, feel free to name yours whatever you'd like.




# Step 4: Create a new storage bucket




Name your bucket whatever you'd like, I will be naming mine `test-wally-registry-bucket`. This must be globally unique, you could also but random numbers/letters on the end if you'd like.


Choose whatever location type you would like, I want high availability across the largest area so I will be choosing Multi-region


Choose the `Standard`storage class


Enforce `public access prevention` and choose uniform access control.

I would recommend choosing object versioning for storage, with 2 max versions that expire after 7 days.

Now just make a note of your buckets name, we will be referring to it later. Mine is `test-wally-registry-bucket`, if you already forgot the name it is available under bucket details.

# Step 5: Setup a Cloud Build pipeline to deploy to Cloud Run

## Step 5.1: Create a cloud build yaml file
Go back to your wally fork, and add a `cloudbuild-backend.yaml` file. Make sure to replace `[YOUR-GCP-PROJECT-NAME]` with the name of the project you setup on Google Cloud Platform. You can also replace `wally-backend` with whatever name you would like.

We need a high cpu machine type or it will be impossible to compile the wally registry backend. This will cost you some money out-of-the-gate ([$0.016 / build-minute](https://cloud.google.com/build/pricing)), so just be aware of that.

steps:
  # Build the container image
  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t', 'gcr.io/[YOUR-GCP-PROJECT-NAME]/wally-backend:$COMMIT_SHA', '-f', 'wally-registry-backend/Dockerfile', '.']

  # Push the container image to Container Registry
  - name: 'gcr.io/cloud-builders/docker'
    args: ['push', 'gcr.io/[YOUR-GCP-PROJECT-NAME]/wally-backend:$COMMIT_SHA']

  # Deploy container image to Cloud Run
  - name: "gcr.io/cloud-builders/gcloud"
    args:
      [
        "run",
        "deploy",
        "wally-backend",
        "--image",
        "gcr.io/[YOUR-GCP-PROJECT-NAME]/wally-backend:$COMMIT_SHA",
        "--region",
        "us-central1",
        "--platform",
        "managed",
        "--command",
        "./launch",
        "--allow-unauthenticated",
      ]

timeout: 900s
options:
 machineType: 'E2_HIGHCPU_8'

Commit this to github:



## Step 5.2: Setup Cloud Run and our temp Cloud Build workflow

Go to `https://console.cloud.google.com/run?project=[YOUR-GCP-PROJECT-NAME]`

Create a new Cloud Run Service.

Choose `Continuously deploy new revisions from a source repository` and `SET UP WITH CLOUD BUILD`


Enable the `CLOUD BUILD API`

Select a Repository. You might need to `Manage connected repositories`.

Have it use a `Dockerfile`, we don't really care about the Source location since we're just having this connect temporarily. **We expect it to fail anyways.**

Choose a service name and region that fits your needs. Make sure that you service name matches the name you chose in step 5.1. The default is `wally-backend`, so if you didn't change that, use that as your service name.

We don't need to be processing requests 24/7 (in fact we don't expect to process them very often, except for large bursts in a short amount of time). So I will be choosing to only allocate CPU during request processing.

~~I will be selecting 1 minimum instance to always be running (in idle mode), with a maximum of 4 instances to ensure we don't have an excessive amount of instances running at any time. You organization (depening on its size), may need more but that's highly unlikely.~~

**Edit:** Wally does not support more than one instance, set the maximum number of instances to 1. You can set the minimum number of instances to 0 or 1 depending on your needs.

See:



Allow all traffic, and unauthenticated invocations. You can change these if you know what you're doing, but we will be doing API key validation anyways.


Open `Container, Connections, Security`, and set the Container port to `8000`, and the Container command to `./launch`.


You can also edit capacity here if you would like. We will be setting up environment variables, secrets, and security in a moment. But for just create the Service. Like I said earlier, we do not expect this to run properly anyways (yet).


# Step 6: Setup secrets and environment variables

## Step 6.1 Create a GCP service account to access the GCP bucket we setup earlier.
We need to setup some secrets and environment variables. First we will setup a GCP service account.

Go to `https://console.cloud.google.com/iam-admin/serviceaccounts?project=[YOUR-GCP-PROJECT-NAME]`

Now click `CREATE SERVICE ACCOUNT`. Give it the name `WALLY_SERVICE_ACCOUNT`


Now give it the following roles:
- Service Account Token Creator
- Storage Object Admin



Now we need to download a key to access the service account.







Keep this key in mind, we will need to store it as a secret later.

## Step 6.2: Create a Github Personal Access Token

I highly recommend creating a new github account for this, and just giving it access to the wally-index and wally repositories. Assuming you have done this, now you need to go to github and create a personal access token.





Set it to `No expiration`, and give it the `repo` scope. Also make sure to give it a note.



Copy the access token you generated. Store it in a text file temporarily.

## Step 6.3 Generate an API key

We need an API key to access wally via the CLI. Go to https://codepen.io/corenominal/pen/rxOmMJ and generate an API key. Copy this API key and store it, you will need to reference it later.


Mine is `8581b05f-997c-4db0-a1a2-a9910798ec34`

## Step 6.4 Enable GCP Secret Manager

We don't want to store secrets in environment variables.

Go to https://cloud.google.com/secret-manager and go to console. Enable the Secret Manager API.


## Step 6.5 Create some secrets

Now go to `https://console.cloud.google.com/security/secret-manager?project=[YOUR-GCP-PROJECT-NAME]`

Create a new secret named `WALLY_GITHUB_TOKEN`, give it the secret you generated in step 6.2 (the reference you see below is the same value generated in 6.3, but you should use the one in 6.2), surrounded by double quotation marks:


Create a new secret named `WALLY_ROCKET_AUTH`, give it the following value, replacing `value="..."` with your API key generated in step 6.3:

{type="api-key",value="8581b05f-997c-4db0-a1a2-a9910798ec34"}


Create a new secret named `WALLY_SERVICE_ACCOUNT`. Upload the file you downloaded in step 6.1.


# Step 7: Setup secrets and environment variables in Cloud Run

Go to `https://console.cloud.google.com/run?project=[YOUR-GCP-PROJECT-NAME]` and select your cloud run service. Edit and deploy a new revision.

## Step 7.1: Setup secrets



Scroll down to environment variables and secrets. We're going to reference the secrets we just made.


Select `WALLY_GITHUB_TOKEN`, and expose it as an environment variable with the name `WALLY_github_token`. Have it use the latest version. **Also grant permission to access the secret if necessary.**


Select `WALLY_ROCKET_AUTH`, and expose it as an environment variable with the name `WALLY_auth`. Have it use the latest version. **Also grant permission to access the secret if necessary.**


Select `WALLY_SERVICE_ACCOUNT`, and mount it as a volume with the mount path of `secrets`. Specify the path for secret versions as `SERVICE_ACCOUNT`. Have it use the latest version. **Also grant permission to access the secret if necessary.**


## Step 7.2: Setup environment variables

Add 3 environment variables.


For the first, give it the name `GOOGLE_APPLICATION_CREDENTIALS`, Google will give you a warning about this, but we are mounting it as a secret volume and specifying its path, not directly exposing it as an environment variables like Google thinks we are. Set its value to `/secrets/SERVICE_ACCOUNT`

For the second, give it the name `WALLY_storage`, and set its value to:

{type="gcs",bucket="[YOUR-BUCKET-NAME]"}

replacing `[YOUR-BUCKET-NAME]` with the name you set in step 4.

For the third, give it the name `WALLY_INDEX_URL`, and set its value to the wally-index github page, surrounded by double quotes.

Mine is: `"https://github.com/TestWallyCreation/wally-index"`

## Step 7.3: Redeploy

Deploy this revision. It should still fail since we haven't updated our cloud build workflow yet.


# Step 8: Configure our cloud build workflow

Edit the continuous deployment


Scroll down to configuration, and change the Location to `Repository`, specify the configuration file to whatever it is in your repository. Mine is `cloudbuild-backend.yaml`. Scroll down and hit "Save"


This should take you to your `Cloud build triggers` page. If not this can be accessed by going to `https://console.cloud.google.com/cloud-build/triggers?project=[YOUR-GCP-PROJECT-NAME]`


Optionally, you can rename your Trigger right now as follows:

Select the trigger, change its name at the top, I will choose `wally-registry-tutorial-trigger`, but you can choose whatever name you like. Scroll down and click save.



# Step 9: Run your cloud build workflow

Go to `https://console.cloud.google.com/cloud-build/triggers?project=[YOUR-GCP-PROJECT-NAME]`. You should already be here if continuing directly from the last stage.

Now run your trigger.



You can see it build if you click on `History` and select it. This will take a few minutes.




# Step 10: Have your wally-index target your newly-created API

After your build finishes, go back to Cloud Run at `https://console.cloud.google.com/run?project=[YOUR-GCR-PROJECT-NAME]`, and click on your service.


Copy the service URL to clipboard.

Set it as the URL in the wally-index github repository's `config.json` that you setup in step 1. Mine is now:

{
    "api": "https://tutorial-wally-registry-backend-mmr4ezmpha-uc.a.run.app",
    "fallback_registries": [
        "https://github.com/UpliftGames/wally-index"
    ]
}

Commit this change to github.

# Step 11: Connecting to your private wally registry

## Step 11.1: Make note of your API key and wally-index

Get your wally API key, which you setup in step 6.3. Mine is `8581b05f-997c-4db0-a1a2-a9910798ec34`
Get your wally-index github url, which you setup in step 1. Mine is `https://github.com/TestWallyCreation/wally-index`

You will use this to connect, login, and use to your private wally registry.

## Step 11.2 Login and connect to your private wally registry

We're going to create a new wally package. I assume you already have wally installed.

Create a new folder, and run `wally init` in your CLI (under the new folder)


Open the `wally.toml` and set the `registry` value to your private registry. Mine is `https://github.com/TestWallyCreation/wally-index`

Note that when using a private wally package

[package]
name = "testuser/testpackage"
version = "0.1.0"
registry = "https://github.com/TestWallyCreation/wally-index"
realm = "shared"

[dependencies]

Save the file and then run `wally login` in your CLI (under the same folder). It will ask for the API key. Paste that and hit enter. (This is the key you generated in step 6.3, mine is `8581b05f-997c-4db0-a1a2-a9910798ec34`)


You are now connected to, and logged in to your private wally package registry.

## Step 11.3 Publish and retrieve a package

We're going to create a very simple test package.

Create a `src` folder (under the same folder you made in step 11.2), and an `init.lua` file under the `src` folder. Put the following code into it:

local TestPackage = {}

function TestPackage.Print()
	print("Hello World!")
end

return TestPackage

Now create a `default.project.json` file, and put the following code into it:

{
	"name": "TestPackage",
	"tree": {
		"$path": "src"
	}
}

Now type `wally publish` in your CLI (under the same folder).

This package should now show up in your wally-index, and can be added as a dependency in future packages.