# Making a Catalog

## Creating the Directory Layout

First, we need to set up some directories for our catalog to live within.

## (optional) Establish a Root Workspace

For this walkthrough, we will be creating a Warpforge "root workspace" for our catalog, which
ensures that we have an isolated environment for catalog development.
(You don't always have to do this, but we'll do it here.)

First, make a directory:

mkdir my-workspace
cd my-workspace

Next, create the `.warpforge` directory for this workspace, and create the
`root` file to mark it as a root workspace.

mkdir .warpforge
touch .warpforge/root

## Ways of adding content to a catalog

There are lots of different ways to add content to a catalog.

- Adding existing tarballs (e.g. from http urls);
- Adding existing git repos (picking one tag, or, ingesting all tags!);
- Building new data in warpforge, and putting the results in a catalog!

## Adding Sources

In most cases, we want to add source code to our catalog so we can use it in
builds. In this example, we will add the source code for GNU Awk (gawk).

### Adding a TAR Archive

If our source code is available as a TAR, we can add the file to our catalog.
Here we will add the gawk source code to our archive using a TAR from the
`ftp.gnu.org` mirror.

warpforge catalog add tar my-catalog/gawk:v5.1.1:src https://ftp.gnu.org/gnu/gawk/gawk-5.1.1.tar.gz

We can now access the source within plots using the input `catalog:my-catalog/gawk:v5.1.1:src`

### Adding a Git Reference

Alternatively, we can add source from a Git repository. We can add a single reference,
or ingest all the available tags. We will use the offical gawk repository at
`https://git.savannah.gnu.org/git/gawk.git`.

We can ingest a single reference with:

warpforge catalog add git my-catalog/gawk:v5.1.1:src https://git.savannah.gnu.org/git/gawk.git gawk-5.1.1

Note, this will fail if a TAR was previously added.

### Ingesting All Git Tags

We can also ingest all the available tags using the `ingest-git-tags` subcommand:

warpforge catalog ingest-git-tags my-catalog/gawk https://git.savannah.gnu.org/git/gawk.git src

In this example, we provide the module name (`my-catalog/gawk`), the URL of the repository
(`https://git.savannah.gnu.org/git/gawk.git`), and the item name (`src`). The release names
are taken directly from the Git tag names.

## Creating a Build

Now that we have source, we need to define how to build it. First off,
we will create a directory and run the Warpforge quickstart to create a module.

mkdir gawk
cd gawk
warpforge quickstart my-catalog/gawk

Next, we need to edit the plot. The default rootfs is a very limited Alpine Linux
system, so we will use a debian rootfs instead. We will also provide instructions
to build gawk. We will also define the output as `amd64`, which will allow us to
access the output with `catalog:my-catalog/gawk:v5.1.1:amd64`.

The full `plot.wf` is provided here. Note, this example uses the source from the
"Adding a Git Reference" section above, and will not work with the TAR.

{
    "inputs": {
        "rootfs": "catalog:min.warpforge.io/debian/rootfs:bullseye-1646092800:amd64",
    },
    "steps": {
        "build": {
            "protoformula": {
                "inputs": {
                    "/": "pipe::rootfs",
                    "/src": "catalog:my-catalog/gawk:v5.1.1:src"
                },
                "action": {
                    "script": {
                        "interpreter": "/bin/sh",
                        "contents": [
                            "cd /src",
                            "./configure --prefix=/tmp/prefix"
                            "make LDFLAGS='-static'",
                            "make install",
                        ]
                    }
                },
                "outputs": {
                    "gawk-bin": {
                        "from": "/tmp/prefix/bin/",
                        "packtype": "tar"
                    }
                }
            }
        }
    },
    "outputs": {
        "amd64": "pipe:build:gawk-bin"
    }
}

## Running the Build

To run the build, we first need to update the catalog in this root workspace
to acquire the Debian rootfs:

warpforge catalog update

Now we can run the build with:

warpforge run

The first build in this workspace will take a while, since the rootfs must
be downloaded and unpacked. Subsequent builds will be much faster.

## Publishing a Release

Once our build is working, we can publish a release into the catalog:

warpforge catalog release v5.1.1

We have now created `catalog:my-catalog/gawk:v5.1.1:amd64`, which can be used
in other plots!

Note, we do not need to wait for the build to run again since
it was *memoized* in the previous run. Unless the formula changes,
the build will not be re-run. However, can be manually re-run
with `warpforge run -f`.

## Publishing our Catalog

The catalog files are created within the `.warpforge/catalog/default` directory.
To publish this, we can create a Git repository there and push to a remote.
This process is not yet integrated into Warpforge.