# Motivation
The idea is have an proof of concept of the [multi-architecture RFC](https://github.com/buildpacks/rfcs/pull/295) to support the creation of buildpack packages and builders with multi-architecture support from the beginning
# Scenario
Let's use our samples repo to demonstrate the current situation
## Buildpacks
Today, our samples buildpacks:
- Need to keep a separate folder for each `operating system`
- Do not specify their supported `architecture`
- When publising to a registry, we need to use different names or tags to differentiate between platforms.
### Let's check the proposed improvements
Based on the [multi-architecture RFC](https://github.com/buildpacks/rfcs/pull/295) we proposed a new folder structure to organize the binaries by platform, also we will add a way to specify all the `os/architecture` we want to support and `pack` will take care creating each individual OCI image and the Image Index to combine them.
We organized our [hello-world](https://github.com/buildpacks/samples/tree/main/buildpacks/hello-world) buildpack folder structure to has different binaries for each platform and we also put together the `windows` binaries
```bash
➜ hello-world git:(main) ✗ tree .
.
├── CHANGELOG.md
├── LICENSE
├── README.md
├── buildpack.toml
├── linux
│ ├── amd64
│ │ ├── CHANGELOG.md
│ │ ├── LICENSE
│ │ ├── README.md
│ │ ├── bin
│ │ │ ├── build
│ │ │ └── detect
│ │ └── some-amd64-file
│ └── arm64
│ ├── CHANGELOG.md
│ ├── LICENSE
│ ├── README.md
│ ├── bin
│ │ ├── build
│ │ └── detect
│ └── some-arm64-file
└── windows
├── README.md
└── bin
├── build.bat
└── detect.bat
```
In this case `CHANGELOG.md`, `LICENSE` and `README.md` files on each linux folder are hardlinks to the one saved in the root directory.
We also updated the `buildapck.toml` to include `targets` according to the RFC
```toml!=
# Buildpack API version
api = "0.10"
# Buildpack ID and metadata
[buildpack]
id = "samples/hello-world"
version = "0.0.1"
name = "Hello World Buildpack"
homepage = "https://github.com/buildpacks/samples/tree/main/buildpacks/hello-world"
# Targets the buildpack will work with
[[targets]]
os = "linux"
arch = "amd64"
[[targets]]
os = "linux"
arch = "arm64"
[[targets]]
os = "windows"
arch = "amd64"
[[targets]]
os = "windows"
arch = "arm64"
# Stacks (deprecated) the buildpack will work with
[[stacks]]
id = "*"
```
Then we execute the following command
```bash!
> pack buildpack package jbustamantevmware/cnb-hello-world --verbose --publish
```
With the following output:
```bash!=
Downloading buildpack from URI: file:///Users/jbustamante/go/src/github.com/buildpacks/samples/buildpacks/hello-world/linux/amd64
Downloading buildpack from URI: file:///Users/jbustamante/go/src/github.com/buildpacks/samples/buildpacks/hello-world/linux/arm64
Downloading buildpack from URI: file:///Users/jbustamante/go/src/github.com/buildpacks/samples/buildpacks/hello-world/windows
Downloading buildpack from URI: file:///Users/jbustamante/go/src/github.com/buildpacks/samples/buildpacks/hello-world/windows
Successfully published package jbustamantevmware/cnb-hello-world and saved to registry
```
What just happened?
As we are creating a buildpack package with the `--publish` flag enabled, it means we want to push the images to a registry, `pack` read the `buildpack.toml` and find several `targets`, `pack` is going to try to create multiple OCI images, one for each target, and create an `Image Index` in front of them.
Notice, how `pack` will determine the **root platform folder** for each `target` and use it to create each buildpack package with the correct content.
Using a tool like [crane](https://github.com/google/go-containerregistry/tree/main/cmd/crane) we can explore the [Image Index](https://github.com/opencontainers/image-spec/blob/v1.1.0/image-index.md) created
```json
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
"manifests": [
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 425,
"digest": "sha256:2f0715137552e8e7444d421ae92944a0f0d2d8052ef926c7439cf296cef49928",
"platform": {
"architecture": "amd64",
"os": "linux"
}
},
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 425,
"digest": "sha256:45100d46883330a1e4485b6d3c248be3ada0407f6aaccf2bc60fdff5d8f8d22a",
"platform": {
"architecture": "arm64",
"os": "linux"
}
},
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 586,
"digest": "sha256:82d4996fd1f91d72074cb5f794f970530c9a6ef7b10edec0f85f2f9aaf3467a1",
"platform": {
"architecture": "amd64",
"os": "windows"
}
},
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 586,
"digest": "sha256:15ce49e82bd0588ffaa5b982d85216ee15d1904f3a2f9663bdf40e3872c602d9",
"platform": {
"architecture": "arm64",
"os": "windows"
}
}
]
}
```
And a tool like [dive](https://github.com/wagoodman/dive) will show us the content of the OCI image, for example, the one for `linux/arm64`
Let's consider another example, in this case let's take a look at our [hello-moon](https://github.com/buildpacks/samples/tree/main/buildpacks/hello-moon) sample buildpack
```bash
➜ hello-moon git:(main) ✗ tree .
.
├── buildpack.toml
├── linux
│ ├── README.md
│ └── bin
│ ├── build
│ └── detect
└── windows
├── README.md
└── bin
├── build.bat
└── detect.bat
```
Using a `buildapck.toml` similar to this one
```toml!=
# Buildpack API version
api = "0.10"
# Buildpack ID and metadata
[buildpack]
id = "samples/hello-moon"
version = "0.0.1"
name = "Hello Moon Buildpack"
homepage = "https://github.com/buildpacks/samples/tree/main/buildpacks/hello-moon"
sbom-formats = ["application/vnd.cyclonedx+json"]
# Targets the buildpack will work with
[[targets]]
os = "linux"
arch = "amd64"
[[targets]]
os = "linux"
arch = "arm64"
[[targets]]
os = "windows"
arch = "amd64"
[[targets]]
os = "windows"
arch = "arm64"
# Stacks (deprecated) the buildpack will work with
[[stacks]]
id = "*"
```
And running the following command
```bash
> pack buildpack package jbustamantevmware/cnb-hello-moon --verbose --publish
```
We will get an output similar to this one:
```bash!=
Downloading buildpack from URI: file:///Users/jbustamante/go/src/github.com/buildpacks/samples/buildpacks/hello-moon/linux
Downloading buildpack from URI: file:///Users/jbustamante/go/src/github.com/buildpacks/samples/buildpacks/hello-moon/linux
Downloading buildpack from URI: file:///Users/jbustamante/go/src/github.com/buildpacks/samples/buildpacks/hello-moon/windows
Downloading buildpack from URI: file:///Users/jbustamante/go/src/github.com/buildpacks/samples/buildpacks/hello-moon/windows
Successfully published package jbustamantevmware/cnb-hello-moon and saved to registry
```
Similar to our previous example, `pack` will determine the **platform root folder** for each `os/arch` and create and push the appropiate OCI image.
The last example we want to check is: **How do we create a multi-archicture composite buildpack?**, let's check our [hello-universe]() buildpack. In this case, there is not change to the current folder structure for a composite buildpack.
```bash
➜ hello-universe git:(main) ✗ tree .
.
├── README.md
├── buildpack.toml
└── package.toml
```
But we need to declare our `targets` in the `package.toml` file, similar to what we did before, something like:
```toml!=
[buildpack]
uri = "."
# Targets the buildpack will work with
[[targets]]
os = "linux"
arch = "amd64"
[[targets]]
os = "linux"
arch = "arm64"
[[targets]]
os = "windows"
arch = "arm64"
[[targets]]
os = "windows"
arch = "amd64"
[[dependencies]]
uri = "docker://jbustamantevmware/cnb-hello-world:latest"
[[dependencies]]
uri = "docker://jbustamantevmware/cnb-hello-moon:latest"
```
For this example, the `buildpack.toml` looks like
```toml!=
# Buildpack API version
api = "0.10"
# Buildpack ID and metadata
[buildpack]
id = "samples/hello-universe"
version = "0.0.1"
name = "Hello Universe Buildpack"
homepage = "https://github.com/buildpacks/samples/tree/main/buildpacks/hello-universe"
# Order used for detection
[[order]]
[[order.group]]
id = "samples/hello-world"
version = "0.0.1"
[[order.group]]
id = "samples/hello-moon"
version = "0.0.1"
```
When creating our `buildpack package` using the following command
```bash!
> pack buildpack package jbustamantevmware/cnb-hello-universe --verbose --publish --config ./package.toml
```
An output like this is shown
```bash!=
Warning: A new '--target' flag is available to set the platform for the buildpack package, using 'linux' as default
Downloading buildpack from URI: file:///Users/jbustamante/go/src/github.com/buildpacks/samples/buildpacks/hello-universe
Downloading buildpack dependency for platform linux/amd64
Downloading buildpack from image: jbustamantevmware/cnb-hello-world:latest
Downloading buildpack dependency for platform linux/amd64
Downloading buildpack from image: jbustamantevmware/cnb-hello-moon:latest
Downloading buildpack from URI: file:///Users/jbustamante/go/src/github.com/buildpacks/samples/buildpacks/hello-universe
Downloading buildpack dependency for platform linux/arm64
Downloading buildpack from image: jbustamantevmware/cnb-hello-world:latest
Downloading buildpack dependency for platform linux/arm64
Downloading buildpack from image: jbustamantevmware/cnb-hello-moon:latest
Downloading buildpack from URI: file:///Users/jbustamante/go/src/github.com/buildpacks/samples/buildpacks/hello-universe
Downloading buildpack dependency for platform windows/arm64
Downloading buildpack from image: jbustamantevmware/cnb-hello-world:latest
Downloading buildpack dependency for platform windows/arm64
Downloading buildpack from image: jbustamantevmware/cnb-hello-moon:latest
Downloading buildpack from URI: file:///Users/jbustamante/go/src/github.com/buildpacks/samples/buildpacks/hello-universe
Downloading buildpack dependency for platform windows/amd64
Downloading buildpack from image: jbustamantevmware/cnb-hello-world:latest
Downloading buildpack dependency for platform windows/amd64
Downloading buildpack from image: jbustamantevmware/cnb-hello-moon:latest
Successfully published package jbustamantevmware/cnb-hello-universe and saved to registry
```
We also get an [Image Index](https://github.com/opencontainers/image-spec/blob/v1.1.0/image-index.md) created for all of our `targets` architectures and if we check one of them using [dive](https://github.com/wagoodman/dive), for example `linux/amd64`, we get something like
Noticed that **hello-world** buildpack actually have the layer that corresponds to the `amd64` architecture
## Builders
Awesome, now we are able to package our buildpacks with multi-architecture in mind, `pack` creates a [Image Index](https://github.com/opencontainers/image-spec/blob/v1.1.0/image-index.md) for us and we have a new folder structure that can help us organizing our binaries depending on our particular use case.
All we need right now is to create a multi-architecture builder, **how do we do it?**
When creating a builder, we need a **builder** and **run** images, tipically these images are created using a `Dockerfile`. I took the alpine image from our [samples](https://github.com/buildpacks/samples/tree/main/base-images/jammy) repository and using [docker buildx](https://docs.docker.com/build/building/multi-platform/) and a few changes in our `Dockerfiles` I managed to create a multi-arch images
**Run image**
**Build image**
Once we have all our pieces in place, multi-architecture **buildpacks**, **build image** and **run image**, let's create our multi-architecture **builder**!
We need to also add `targets` into our `builder.toml`, for this particular case we only use `linux` with `amd64` and `arm64` architectures.
```toml!=
# Buildpacks to include in builder
[[buildpacks]]
uri = "docker://jbustamantevmware/cnb-hello-universe:latest"
[[order]]
[[order.group]]
id = "samples/hello-universe"
version = "0.0.1"
# Targets the buildpack will work with
[[targets]]
os = "linux"
arch = "amd64"
[[targets]]
os = "linux"
arch = "arm64"
# Base images used to create the builder
[build]
image = "jbustamantevmware/cnbs-build-alpine:latest"
[run]
[[run.images]]
image = "jbustamantevmware/cnbs-run-alpine:latest"
# Stack (deprecated) used to create the builder
[stack]
id = "io.buildpacks.samples.stacks.alpine"
build-image = "jbustamantevmware/cnbs-build-alpine:latest"
run-image = "jbustamantevmware/cnbs-run-alpine:latest"
```
Then, execute the following command to create a builder
```bash!=
> pack builder create jbustamantevmware/cnbs-builder-alpine --config builder.toml --publish --verbose
```
Hopefully, we will get an output similar to:
```bash!=
Creating builder jbustamantevmware/cnbs-builder-alpine from build-image jbustamantevmware/cnbs-build-alpine:latest
Using cached version of https://github.com/buildpacks/lifecycle/releases/download/v0.18.4/lifecycle-v0.18.4+linux.x86-64.tgz
Looking up buildpack docker://jbustamantevmware/cnb-hello-universe:latest
Downloading buildpack for platform: linux/amd64
Downloading buildpack from image: jbustamantevmware/cnb-hello-universe:latest
Creating builder with the following buildpacks:
-> samples/hello-universe@0.0.1
-> samples/hello-moon@0.0.1
-> samples/hello-world@0.0.1
Adding buildpack samples/hello-moon@0.0.1 (diffID=sha256:5a529d6800e6098922a05dfb2e1156bbca4a8a64b4cc4a302cc2d32b7bb8a9ab)
Adding buildpack samples/hello-universe@0.0.1 (diffID=sha256:720806c484f17fa1b546baab38b16888f5e749a267c7975c4611c2eca7bbc69a)
Adding buildpack samples/hello-world@0.0.1 (diffID=sha256:7ffc37a2c4151c83a0259390f4294751e8aca908503fcb0ad6e426da42da8b73)
Creating builder jbustamantevmware/cnbs-builder-alpine from build-image jbustamantevmware/cnbs-build-alpine:latest
Using cached version of https://github.com/buildpacks/lifecycle/releases/download/v0.18.4/lifecycle-v0.18.4+linux.arm64.tgz
Looking up buildpack docker://jbustamantevmware/cnb-hello-universe:latest
Downloading buildpack for platform: linux/arm64
Downloading buildpack from image: jbustamantevmware/cnb-hello-universe:latest
Creating builder with the following buildpacks:
-> samples/hello-universe@0.0.1
-> samples/hello-moon@0.0.1
-> samples/hello-world@0.0.1
Adding buildpack samples/hello-moon@0.0.1 (diffID=sha256:5a529d6800e6098922a05dfb2e1156bbca4a8a64b4cc4a302cc2d32b7bb8a9ab)
Adding buildpack samples/hello-universe@0.0.1 (diffID=sha256:720806c484f17fa1b546baab38b16888f5e749a267c7975c4611c2eca7bbc69a)
Adding buildpack samples/hello-world@0.0.1 (diffID=sha256:188afc1b5661bc89ef8a07b3e543f47d5d4eed515830334f7cec901d460c411b)
Successfully created builder image jbustamantevmware/cnbs-builder-alpine
Tip: Run pack build <image-name> --builder jbustamantevmware/cnbs-builder-alpine to use this builder
```
We have our builder published for the two targets we provided and we can check the [Image Index](https://github.com/opencontainers/image-spec/blob/v1.1.0/image-index.md) using [crane](https://github.com/google/go-containerregistry/tree/main/cmd/crane)
```json
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
"manifests": [
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"size": 2481,
"digest": "sha256:5454f39ed7608edcdafb3ceb54012d8c5e021675b0c7297bd13c81defa7acce7",
"platform": {
"architecture": "amd64",
"os": "linux"
}
},
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"size": 2481,
"digest": "sha256:f143eabb646acd5e71ef2b8f474f49262e7c1a2aa1b12f4b6f3fd8359ca8183b",
"platform": {
"architecture": "arm64",
"os": "linux"
}
}
]
}
```
Remember our [hello-world](https://github.com/buildpacks/samples/tree/main/buildpacks/hello-world) buildpack? we added a particular file on it to differentiate each artifact, in the previous log, we can see how the **diffID** is different for this buildapck.
When we inspect the layers using [dive](https://github.com/wagoodman/dive) we can verify the correct layers were included.
Sign in with Wallet
Connect another wallet