# Terraform ideas for conda-forge

## Idea

Use declarative infrastructure as code (IaC) approaches to both document and maintain conda-forge's infrastructure

Given its popularity, Terraform would be the chosen tool to implement IaC practices. Decision items that were considered:

- Terraform uses a DSL (`hcl`) for the configuration files, but it's really simple and human readable.
- Terraform providers include many of the services already in use in conda-forge (Github, Azure, Travis...).
- Anaconda.org's service could be implemented with an OpenAPI-driven meta-provider (like [this one](https://github.com/dikhan/terraform-provider-openapi))
- Terraform's state can be stored (and encrupted) in a Github repo with:
- [this community plugin](https://github.com/plumber-cd/terraform-backend-git)
- [or this other one](https://github.com/sturlabragason/terraform_state_artifact)

> Imagine saying "I want to archive this repo" and your PR just looks like changing `archived=false` to `archived=true`.
> In practice this might be unfeasible given conda-forge's scale, for performance reasons ([see this issue](https://github.com/integrations/terraform-provider-github/issues/567)), although there are mitigations (like [using workspaces](https://www.rea-group.com/about-us/news-and-insights/blog/so-your-terraform-is-taking-too-long-as-it-scales/) to fragment the state in manageable chunks.)

## Why?

- We don't know how to deploy conda-forge from scratch. We don't even know what parts are essential, which ones are optional, which ones are historical artifacts, duplicates, dead code...
- It doubles as documentation as code :) Infrastructure inventory that can be deployed with a couple actions sounds like a dream.
- We could have a testing conda-forge for free, instead of rolling changes in production.
- Current approaches in conda-forge are imperative (collection of actions to arrive to a given state). One clear example is `admin-requests`. It takes a lot of code to just change a label in Anaconda.org. That change of state could be expressed in the PR itself!
- It would be so cool to have a project named `terraforge`. It has a nice sound to it, doesn't it.

## How?

The idea is to document things first. Understand the connections. Then create a mini-mirror forge with the absolute essentials to run conda-forge. Then slowly add pieces of infra til it looks like the real conda-forge. Eventually, migrate conda-forge to the terraforge'd equivalent.

The plan might not succeed entirely, but each step in that direction will be a net positive.

## Inventory

The first task is to enumerate all pieces of infrastructure we can find on conda-forge.

The following blocks detail each element as part of a provider-resource abstraction, mimicking the Terraform object model. Take it as "Infrastructure as pseudo-code". It doesn't need to be runnable or actionable, but it does need to be human-readable and exhaustive.

- provider: GitHub
  resources:
    organizations: 
      - conda-forge:
        org-wide-state:
          metadata (name, description, archive status, repo labels)
          access
          automation (branch protection, etc)
          security (tokens, secrets)
          integrations (apps, webhooks)
        repositories:
          - *-feedstock:
              state:
                metadata (name, description, archive status, repo labels)
                access
                automation (branch protection, etc)
                security (tokens, secrets)
                integrations (apps, webhooks)
          - sources: # todo
          - data: # todo
          - tasks: # todo
        teams:
          - core
          - staged-recipes
          - help-*
          - *(-feedstock)
        users:
          - /see access/
        apps:
          - /see integrations/

      - regro:
        repositories:
          - tasks:
            - autotick-bot
          - sources:
            - cf-scripts
            - libcfgraph
          - data:
            - cf-graph-countyfair
        users:
          - /enumerate bots/
    bots:
      - /enumerate/
    apps:
      - /enumerate/

- provider: azure
  ...

- provider: travis
  ...
  
- provider: cirun
  ...

- provider: circleci
  ...
  
- provider: woodpecker
  ...

- provider: drone
  ...

- provider: heroku
  ...
  
- provider: anaconda.org
  channels:
    - conda-forge
    - cf-staging

- provider: apple
  team:

## References

- This [blog series](https://blog.gruntwork.io/a-comprehensive-guide-to-terraform-b3d32832baca), which was then turned into [this book](https://www.terraformupandrunning.com/). I am reading the 3rd edition (2022). Important bits:
- The preface is a good introduction to the motivation.
- Chapter 6: How to handle secrets
- [Performance tips blog post](https://www.rea-group.com/about-us/news-and-insights/blog/so-your-terraform-is-taking-too-long-as-it-scales/)
- State backends:
- https://github.com/plumber-cd/terraform-backend-git
- https://github.com/sturlabragason/terraform_state_artifact