---
# System prepended metadata

title: Pipeline Examples

---

## Buildkite Differentiators

* Scaling model
    * Jobs can be picked up by executors "anywhere"
* Flexible declarative model
* Anywhere the agent can run
    * multi-arch
    * multi-OS
* BYO workers first class

Unsure whether this is "the core"
* Plugin model
  * scripts get encapsulated into plugin
  * behavior gets copied


### Non Differentiators

* Always running in the same place


## What can we address easily in k8s

* Scaling is "easy"
* Plugins are harder to support, but necessary
    * Harder because it involves the environment variables/filesystem/requires dependencies
    * Different plugins come with different assumptions about what binaries are available on the PATH
    * Need to "share" information

## Requirements for scaling

* Need to support the basic operations of an agent 
    * log forwarding
    * artifacts
    * process management/status
    * retries
    * plugins
* Need to support executing a command in an arbitrary container
* Want to continue to support dind model and docker plugins
* Need to clone the code before doing work "generally"

## How do you do that

* One agent pod per queued job
    * Will have a "base" container image
    * This is assumed to contain the buildkite-agent image
* Need "sidecars"
    * dind
    * constructed container that a command is executed in
        * app containers should be small footprint
        * in order to access "results" from commands in this container on the filesystem, we need to be "in" the pod
    * Agent shells out to call plugins
* init container that does the "git" things


`/source` - git repo clone
`/buildkite` - agent "working stuff"
  * environment vars
  * files to pass around

need to specify entrypoint as command


# Pipeline Examples

## current agent model

* Support docker-compose plugin

init container w/ git that pulls the code
* git secret to authorize clone
* clone into empty dir (optionally PVC)
agent container
* mounts the code
* DOCKER_HOST environment variable pointing to dind
dind container
* run as privileged
* optionally has its own PVC to speed docker cache loads


agent would start and grab build step configuration from API
execute docker-compose "normally"
shut down once it completes
controller would delete the pod after some time

## Distroless agent model

### Design considerations
* docker & docker-compose are "special case" dependencies
* to put everything on equal footing, make nothing special


### Constructing a pod

* agent copies itself to emptydir volumes

Four containers
1. init container to pull git
2. init container to copy buildkite-agent distroless to emptydir volume
3. user container with startup "wrapped" by agent
4. dind sidecar run through agent
5. coordination sidecar(?)


```yaml=
steps:
  - label: ":docker: Build"
    plugins:
      - docker-compose#v4.0.0:
          build: app
          image-repository: index.docker.io/myorg/myrepo
```

```mermaid
sequenceDiagram
    participant agentSidecar
    participant gitSidecar
    participant dindSidecar
    participant userContainer
    participant emptyDir
    agentSidecar->>emptyDir: copy agent
    Note over agentSidecar: init container
    Note over gitSidecar: init container
    gitSidecar->>emptyDir: clone repo
    dindSidecar->>emptyDir: write DOCKER_HOST to environment file
    Note over userContainer: image: buildkite/docker-compose:v4.0.0
    userContainer->>emptyDir: source environment file
    userContainer->>emptyDir: read files on path
    userContainer->>userContainer: run docker-compose build & push
```

`/buildkite/workspace`


```mermaid
sequenceDiagram
    Participant agentSidecar
    Participant gitSidecar
    Note over controller: starts pod
    agentSidecar->>emptyDir: copy agent
    Note over agentSidecar: init container
    Note over gitSidecar: init container
    gitSidecar->>emptyDir: clone repo
    userContainer->>emptyDir: source environment
    userContainer->>emptyDir: read files on path
    userContainer->>emptyDir: run a command in user container
    controller->>userContainer: watch logs
    Note over userContainer: upload artifacts with agent
```

### Plugins
Is there a reason that something like [shellcheck-buildkite-plugin](https://github.com/buildkite-plugins/shellcheck-buildkite-plugin) exists, rather than just using docker directly to run the koalaman/shellcheck image?

## Milestones

### Initial MVP

In this first iteration, create a controller pod that will watch the buildkite API for new work, be configured by a `BuildkiteAgentQueue` CRD, and schedule pods using a configured image to execute commands. The code can be cloned using an init container from a public or private repository.

This MVP allows for quite a few use cases. By allowing execution in any Kubernetes image, we can make it easy to run language specific containers to build artifacts/output images. We can grab logs using the container logs. We could even advise folks that want to use docker in docker how to deploy a simple daemonset and make that available to the agent pod.

### Milestone 2: Process Coordination

In this milestone, we begin to split apart the execution component into the agent controlled pieces and the target image command. With this milestone, we allow the gathering of test results, multiple commands/containers in a single run, and multiple init containers that can write environment variables that will be sourced by the running processes.

This milestone will also complete the publishing of a number of use case documents that will enable specific workflow types. This may include:

* Buildpack builds
* Bazel workflows
* Caching via PVCs in the agent container

_NB_: After this point, it will be helpful to touch base with customers to determine which of the following milestones are most critical.

### Advanced Kubernetes Testing Strategies

In this milestone, we enhance support for using Kubernetes as the framework for testing. This will include highly parallel test execution using the test parameterization plugin and managing ephemeral Kubernetes resources for end to end application tests.

We will need to ensure appropriate log collection is handled across a set of pods that may be deployed for e2e tests.

### Debuggability

### Caching

### Scalability

### Priority Queues

* need to determine approach