# Analysis of the impact of Buildkit 0.10.0 in Buildpack

## Context

Recently I've been working on two RFCs for the Buildpack project.

* Deprecating the Daemon support [#201](https://github.com/buildpacks/rfcs/pull/201)
* Adding feature to export to OCI [#203](https://github.com/buildpacks/rfcs/pull/203)

The progress on these features have been a little slow, but a new variable was added into this ecuation. [Buildkit](https://github.com/moby/buildkit) project recently release a new version [0.10.0](https://github.com/moby/buildkit/releases/tag/v0.10.0) and the new features: MergeOp and DiffOp seems to be very helpful and could affect our next steps in the RFC mentioned above.

## Motivations behind these RFCs

I've been thinking about the impact of this new version and how it can affect our roadmap. The idea of this document is to express myself and the ideas regarding this topic.

Let me start on setting up some motivations behind the RFCs [#201](https://github.com/buildpacks/rfcs/pull/201) and [#203](https://github.com/buildpacks/rfcs/pull/203)

* Some developers may use Docker and they expect after building their code with [Pack](https://buildpacks.io/docs/tools/pack/) to check their image using `docker images` command
* Sometimes, developers want to do more than building a container image and saving it into the Daemon, they want to push the image into a registry using `docker push` command
* Some developers may want to add metadata (annotations) to their image or sign them and then push this image to a registry

### Use cases

Based on these assumptions, let me show the future use case matrix when we develop the missing capabilities on Buildpack

`As a Developer I want to build my source code using Pack`

| use dockerd | push to registry | add annotations | sign it | will I able to do it? | Notes |
| -------- | -------- | -------- | -------- | -------- | -------- |
| Yes | Yes | Yes | Yes | No | We loose data saving into the daemon |
| Yes | No | Yes | Yes | No | can't save annotations into the daemon |
| No | No | Yes | Yes | No | we don't export to disk or any other format |
| No | Yes | Yes | Yes | Yes |
| No | Yes | Yes | Yes | Yes |

As we can see, when the developer uses the daemon as the storage mechanism to save the output image from Buildpack we can't provide the whole set of capabilities because the format image are saved in the daemon is not 100% compatible with the OCI format.

Trying to support these cases is one of the motivations behind the RFCs I've been working on.

### How does this new Buildkit version affect us?

#### Analysis

After thinking about how to integrate [Buildkit](https://github.com/moby/buildkit) inside the Buildpack project I came up with the following idea where we can use it as a midle layer for interacting with the Daemon.

I hope the following diagram can be useful to make myself clear.


* **Translator** will be a new component with the responsability of building a new LLB graph based on the internal structures in the Lifecycle. The idea is make use of the new MergeOp and DiffOp features to solve some performance issues when we are interacting with the Daemon. If I understood correctly, the Buildkitd (the buildkit daemon) can expose the solution of the LLB graph through a Writer object.
* The output from Buildkit can be handle into a new **Exporter** component? and `load` into the docker daemon in the same way buildx [is doing it](https://github.com/docker/buildx/blob/1ca30a58c2f25a7f72a2fdb5d62e82bf1e2fa37c/build/build.go#L1357), using the docker client library
* The output to the registry is hanlded by the [GGCR library](https://github.com/google/go-containerregistry/#sources-1) (That's how it works right now)
* The export to OCI format in the filesystem, which I've been working in the RFC [#203](https://github.com/buildpacks/rfcs/pull/203) can be done in the following way:
* Throught GGCR when we are dealing with a registry
* Throught the *Exporter* handling the Writer from Buildkit when we are dealing with the Daemon

#### Let's update our previous use cases

| use dockerd | push to registry | add annotations | sign it | will I able to do it? | Notes |
| -------- | -------- | -------- | -------- | -------- | -------- |
| Yes | Yes | Yes | Yes | Yes | We will save the OCI image on disk and then offer a `pack push` |
| Yes | No | Yes | Yes | Yes | same as above |
| No | No | Yes | Yes | Yes | We will export to disk |
| No | Yes | Yes | Yes | Yes |
| No | Yes | Yes | Yes | Yes |

#### Conclusions

I can conclude that this Buildkit new features affects what I am working on my RFCs because the solution could be different if we decide to include Buildkit as part of the Lifecycle workflow

### Unresolved Questions

* In my use case matrix I didn't consider if the user is running the Buildkitd or not on their local machine, I think we can handle this problem in the same way we are doing right now with the Docker Daemon, but maybe we can .
* Lifecycle assumes **buildkitd** is running
* If the user is not running **buildkitd**, Platform, in this case, Pack can run it in a container with [rootless](https://github.com/rootless-containers/rootlesskit/)