Managing Images and Builds

--- title: Managing Images and Builds owner: Build Service Team --- # <a id='images'></a> Images Images provide a configuration for build service to build and maintain a docker image utilizing Tanzu, Paketo, and custom [Cloud Native Buildpacks](https://buildpacks.io). Build Service will monitor the inputs to the image configuration to rebuild the image when the underlying source or buildpacks have changed. The following procedures describe how to create and manage images in Build Service with the `kp` cli. ## <a id='create-image'></a> Creating Images Prerequisites: * [Access to a cluster](accessing-tbs.html) running Build Service. * Configured write [secrets](managing-secrets.html) for your docker registry. #### Source Code The `kp` cli supports creating Images using source code from the following locations: * [Git based source](#create-image-with-git-repo) * [Blob store](#create-image-by-referencing-blob-store-source) * [Local machine](#create-image-with-local-source-code) You can specify only one location for app source code. #### Builders Users can select a Custom Builder or a Custom Cluster Builder to be used to create image builds. You can use any of the available Custom Builders or Custom Cluster Builders with any of the source types (git, blob, or local). If you do not use the `--custom-builder` or `--custom-cluster-builder` flags, the `default` Custom Cluster Builder will be used. For more information on Builders, see [Managing Builders](managing-builders.html). ### <a id='create-image-with-git-repo'></a> Creating an Image With Source Code in a Git Repository To create an image using source code from a git repository run: kp image create <name> <tag> \ [--custom-builder <custom-builder> or --custom-cluster-builder <custom-cluster-builder>] \ --namespace <namespace> \ --env <env> \ --git <git-repo> \ --git-revision <git-revision> Where: * `name`: The name of the image. * `tag`: The registry location where the image will be created. * `custom-builder`: (optional) Custom Builder name to be used in the image. * `custom-cluster-builder`: (optional) Custom Cluster Builder name to be used in the image. Defaults to `default` when `custom-builder` is not set. * `namespace`: (optional) The kubernetes namespace for the image. Defaults to the local kubernetes current-context namespace. * `env` (optional): Image environment variable configuration as key=val pairs (`env_var=env_val`). The `--env` flag can be specified multiple times. * `git-repo` Git repository url of the source code. * `git-revision` (optional) The Git revision of the code that the image is built against. Can be either a `branch`, `tag` or a commit `sha`. When you target the image against a branch, Build Service triggers a build for every new commit. Defaults to `master`. Note: If the `git-repo` is a private repository, you must configure the git credentials. For more info, see [Managing Git Repositories for a Project](managing-secrets.html#manage-git-repos) ### <a id='create-image-with-local-source-code'></a> Creating an Image With Local Source Code Users can apply local source code from a directory, compressed source code (`zip`, `tar.gz`, `.tar`), or a `.jar` file. To create an image using source code from a local machine run: kp image create <name> <tag> --custom-builder <custom-builder> --namespace <namespace> --env <env> --local-path <source-path> or kp image create <name> <tag> --custom-cluster-builder <custom-cluster-builder> --namespace <namespace> --env <env> --local-path <source-path> Where: * `name`: The name of the image. * `tag`: The registry location where the image will be created. * `custom-builder`: (optional) Custom Builder name to be used in the image. * `custom-cluster-builder`: (optional) Custom Cluster Builder name to be used in the image. Defaults to `default` when `custom-builder` is not set. * `namespace`: (optional) The kubernetes namespace for the image. Defaults to the local kubernetes current-context namespace. * `env` (optional): Image environment variable configuration as key=val pairs (`env_var=env_val`). The `--env` flag can be specified multiple times. * `source-path` Path to local source code. ### <a id='create-image-by-referencing-blob-store-source'></a> Create an Image With Source Code In A Blob Store Users can specify source code in a blob store saved as a compressed file (`zip`, `tar.gz`, `.tar`) or a `.jar` file. To create an image using source code from blob store: kp image create <name> <tag> --custom-builder <custom-builder> --namespace <namespace> --env <env> --blob <blob-url> or kp image create <name> <tag> --custom-cluster-builder <custom-cluster-builder> --namespace <namespace> --env <env> --blob <blob-url> Where: * `name`: The name of the image. * `tag`: The registry location where the image will be created. * `custom-builder`: (optional) Custom Builder name to be used in the image. * `custom-cluster-builder`: (optional) Custom Cluster Builder name to be used in the image. Defaults to `default` when `custom-builder` is not set. * `namespace`: (optional) The kubernetes namespace for the image. Defaults to the local kubernetes current-context namespace. * `env` (optional): Image environment variable configuration as key=val pairs (`env_var=env_val`). The `--env` flag can be specified multiple times. * `blob-url` URL of the source code blob file. Note: The source code file in the blobstore must be publically viewable. ## <a id='patch-images'></a> Patching Images Users can patch their existing images with the `kp` cli with the same arguments as (create)[#create-image]. Running a patch will trigger a new build of the image if any of the build inputs are changed. Patch images with the following commands: * With Source Code in a Git Repository ``` kp image patch <name> --namespace <namespace> --env <env> --git <git-repo> --git-revision <git-revision> ``` * With With Local Source Code ``` kp image patch <name> --namespace <namespace> --env <env> --local-path <source-path> ``` * With Source Code In A Blob Store ``` kp image patch <name> --namespace <namespace> --env <env> --blob <blob-url> ``` Where: * `name`: The name of the image to patch. * `tag`: The registry location where the image will be created. * `namespace`: (optional) The kubernetes namespace for the image. Defaults to the local kubernetes current-context namespace. * `env` (optional): Image environment variable configuration as key=val pairs (`env_var=env_val`). The `--env` flag can be specified multiple times. * `git-repo` Git repository url of the source code. Must select one of `git-repo`, `blob-url`, or `source-path` * `git-revision` (optional) The Git revision of the code that the image is built against. Can be either a `branch`, `tag` or a commit `sha`. When you target the image against a branch, Build Service triggers a build for every new commit. Defaults to `master`. * `blob-url` URL of the source code blob file. Must select one of `git-repo`, `blob-url`, or `source-path` * `source-path` Path to local source code. Must select one of `git-repo`, `blob-url`, or `source-path` Note: If the `git-repo` is a private repository, you must configure the git credentials. For more info, see [Managing Git Repositories for a Project](managing-secrets.html#manage-git-repos) Note: The `tag` (location in a registry) of an image cannot be modified. To push builds to a new `tag` you must create a new image. ## <a id='list-images'></a> Listing Images To list all the image configurations in a kubernetes namespace: kp image list --namespace <namespace> The following is an example of the output for this command: ``` NAME READY LATEST IMAGE first-image True first/image:tag second-image True second/image:tag first-image False third/image:tag ``` ## <a id='rebuild-image'></a> Image Rebuilds Rebuilds happen in three ways: 1. An imperative rebuild occurs when you patch an image with `kp image patch`. 1. An automatic rebuild occurs when build inputs change (source code, stack, or buildpacks). 1. A user can trigger a rebuild manually. An imperative rebuild will be initiated if any of the following changes are made to an image: * An update to the commit, branch, Git repository, or other arguments to `kp image patch`. * You upload a new copy of the local source code by running `kp image patch --local-path <source-path>`, where `<source-path>` is the source code path. For more information, see [Patching Images](#patch-images). Build Service auto-rebuilds images when one or more of the following build inputs change: * New buildpack versions are made available through an updated builder image. - The Build Service Bundle on [Tanzu Network](https://network.pivotal.io/products/build-service) includes the latest buildpacks contained in a builder image. - When upgrading TBS, you must relocate new buildpack versions over to your container registry. See [Updating Build Service Dependencies](updating-deps.html) for more details. * There is a new commit on a branch or tag Tanzu Build Service is tracking. * There is a new Stack (ie. base OS image) available, such as `full`, `tiny`, or `base`. ### <a id='trigger-rebuild-image'></a> Trigger an Image Rebuild You can initiate a manual rebuild using `kp`: kp image trigger <image-name> This is useful for debugging image builds. ## <a id='view-image-status'></a> Viewing the Status of an Image When a user creates an image using the above workflow, they are configuring Tanzu Build Service to start creating builds of the image which create container images to be pushed to a registry. If a particular build associated with an image fails, check the status of the image by running: kp image status <image-name> Where `image-name` is the name of the image. See [Listing Images](#list-images) to get image names. The following is an example output of this command: ``` Status: Not Ready Message: -- LatestImage: gcr.io/myapp@sha256:9d7b1fbf7f5cb0f8efe797f30e598b5e38bb1c08ada143d4c96e4f78111a9239 Last Successful Build Id: 1 Reason: CONFIG Last Failed Build Id: 2 Reason: COMMIT ``` ## <a id='delete-images'></a> Deleting an Image This procedure describes how to delete a Build Service image with the `kp` cli. Warning: Deleting an image also deletes all the builds that the image owns. It does not delete the images generated by those builds. To delete an image: kp image delete <image> Where `image` is the name of the image. When you successfully delete an image, you will see this message: ``` "<image>" deleted ``` # <a id='builds'></a> Builds The procedures in this section describe how to view information and logs for image builds using the `kp` cli. ## <a id='list-builds'></a> Listing Builds for an Image Build Service stores the ten most recent successful builds and the ten most recent failed builds. To see a the list of builds for an image run: kp build list <image-name> The following is an example of the output for this command: ``` BUILD STATUS IMAGE STARTED FINISHED REASON 1 SUCCESS gcr.io/myapp@sha256:some-sha1 2019-09-09 21:55:27 2019-07-08 21:55:54 CONFIG 2 SUCCESS gcr.io/myapp@sha256:some-sha2 2019-09-09 21:56:55 2019-07-08 21:57:40 COMMIT 3 SUCCESS gcr.io/myapp@sha256:some-sha3 2019-09-09 21:57:55 2019-07-08 21:58:40 STACK 4 FAILURE gcr.io/myapp@sha256:some-sha4 2019-09-09 21:58:55 2019-07-08 21:59:40 CONFIG+ 5 BUILDING gcr.io/myapp@sha256:some-sha5 2019-09-09 21:59:55 -- BUILDER ``` The following describes the fields in the example output: * `BUILD`: Describes the index of builds in the order that they were built. * `STATUS`: Describes the status of a previous build image. * `IMAGE`: The full image reference for the app image produced by the build. * `STARTED`: Describes when a build started. * `FINISHED`: Describes when a build finished. * `REASON`: Describes why an image rebuild occured. These reasons include: * `CONFIG`: Occurs when a change is made to commit, branch, Git repository, or build fields on the image's configuration file and you run `kp image apply`. * `COMMIT`: Occurs when new source code is committed to a branch or tag build service is monitoring for changes. * `BUILDER`: Occurs when new buildpack versions are made available through an updated builder image. Note: A rebuild can occur for more than one reason. When there are multiple reasons for a rebuild, the <code>kp</code> CLI output shows the primary <code>Reason</code> and appends a <code>+</code> sign to the <code>Reason</code> field. The priority rank for the <code>Reason</code>, from highest to lowest, is <code>CONFIG</code>, <code>COMMIT</code>, and <code>BUILDER</code>. * `STACK`: Occurs when a new base OS image, called a `run image`, is available. * `TRIGGER`: Occurs when a new build is manually triggered. ## <a id='build-status'></a> Viewing Build Details for an Image To display retrieve a detailed Bill of Materials for a particular build: kp build status <image> -b <build-number> Where: * `image-name` is the name of the image the build is associated with * `build-name` (optional) is the index of the build from [listing builds](#list-builds). Defaults to latest build. The following is an example of the output for this command: ``` Image: gcr.io/myapp@sha256:f87b614257af05c3301c1554c4f15131793caec3adf55e45d2c612e90445765a Status: SUCCESS Reasons: CONFIG Builder: gcr.io/my-builder:base@sha256:grtewwads0asdvf09asdf Run Image: gcr.io/base-image:run@sha256:asdas098asdas Source: Git Url: http://github.com/myapp Revision: ad123ad BUILDPACK ID BUILDPACK VERSION io.java.etc 123 io.kotlin.etc 321 ``` The following describes the fields in the example output: * `Image`: The full image reference for the app image produced by the build. * `Status`: Describes the status of a previous build image. * `Reason`: Describes why an image rebuild occured. These reasons include: * `CONFIG`: Occurs when a change is made to commit, branch, Git repository, or build fields on the image's configuration file and you run `kp image apply`. * `COMMIT`: Occurs when new source code is committed to a branch or tag build service is monitoring for changes. * `BUILDER`: Occurs when new buildpack versions are made available through an updated builder image. Note: A rebuild can occur for more than one reason. When there are multiple reasons for a rebuild, the <code>kp</code> CLI output shows the primary <code>Reason</code> and appends a <code>+</code> sign to the <code>Reason</code> field. The priority rank for the <code>Reason</code>, from highest to lowest, is <code>CONFIG</code>, <code>COMMIT</code>, and <code>BUILDER</code>. * `STACK`: Occurs when a new base OS image (called a `run image`) is available. * `TRIGGER`: Occurs when a new build is manually triggered. * `Builder`: The full image tag for the builder image used by the build. * `Run Image`: The full image tag for the run image used by the app. * `Source`: Describes where the source code used to build the image is coming from. Can be `Git`, `Blob`, or `Local Source`. * `Url`: The Git repository URL for `Git` source, the Blob file URL for `Blob` source. Unset for `Local Source`. * `Revision`: The Git commit sha of the source code used to create the build for `Git` source. * `BUILDPACK ID`: A list of buildpack ids the build used. * `BUILDPACK VERSION`: A list of buildpack versions the build used. ## <a id='build-logs'></a> Getting Build Logs An image that a user creates will cause builds to be initiated for that image. Builds are where Cloud Native Buildpacks are run and apps get built into images. Build logs are a good way to debug issues and to get information about how your app is being built. If you get logs of a build in progress, the logs will be tailed and will terminate when the build completes. To get logs from a build run: kp build logs <image> --build <build-number> Where: * `image-name` is the name of the image the build is associated with * `build-name` (optional) is the index of the build from [listing builds](#list-builds). Defaults to latest build. The following is an example of the output of the command: ``` ===> PREPARE Loading secret for "gcr.io" from secret "some-creds" at location "/var/build-secrets/some-creds" Successfully pulled gcr.io/test-app-source@sha256:d6ddb6d02e07b2a40022929ca83d73482ae1aff99edefc10c76f43ff57d3ef40 in path "/workspace" ===> DETECT 6 of 32 buildpacks participating paketo-buildpacks/bellsoft-liberica 2.7.3 paketo-buildpacks/maven 1.4.3 paketo-buildpacks/executable-jar 1.2.5 paketo-buildpacks/apache-tomcat 1.2.2 paketo-buildpacks/dist-zip 1.3.3 paketo-buildpacks/spring-boot 1.5.5 ===> ANALYZE Previous image with name "gcr.io/test-app" not found ===> RESTORE ===> BUILD Paketo BellSoft Liberica Buildpack 2.7.3 https://github.com/paketo-buildpacks/bellsoft-liberica Build Configuration: $BP_JVM_VERSION 11.* (default) the Java version Launch Configuration: $BPL_JVM_HEAD_ROOM 0 (default) the headroom in memory calculation $BPL_JVM_LOADED_CLASS_COUNT 35% of classes (default) the number of loaded classes in memory calculation $BPL_JVM_THREAD_COUNT 250 (default) the number of threads in memory calculation BellSoft Liberica JDK 11.0.7: Contributing to layer Downloading from https://github.com/bell-sw/Liberica/releases/download/11.0.7+10/bellsoft-jdk11.0.7+10-linux-amd64.tar.gz Verifying checksum Expanding to /layers/paketo-buildpacks_bellsoft-liberica/jdk Adding 134 container CA certificates to JVM truststore Writing env.build/JAVA_HOME.override Writing env.build/JDK_HOME.override BellSoft Liberica JRE 11.0.7: Contributing to layer Downloading from https://github.com/bell-sw/Liberica/releases/download/11.0.7+10/bellsoft-jre11.0.7+10-linux-amd64.tar.gz Verifying checksum Expanding to /layers/paketo-buildpacks_bellsoft-liberica/jre Writing env.launch/JAVA_HOME.override Writing env.launch/MALLOC_ARENA_MAX.override Writing profile.d/active-processor-count.sh Memory Calculator 4.0.0: Contributing to layer Downloading from https://github.com/cloudfoundry/java-buildpack-memory-calculator/releases/download/v4.0.0/memory-calculator-4.0.0.tgz Verifying checksum Expanding to /layers/paketo-buildpacks_bellsoft-liberica/memory-calculator Writing profile.d/memory-calculator.sh Class Counter: Contributing to layer Copying to /layers/paketo-buildpacks_bellsoft-liberica/class-counter JVMKill Agent 1.16.0: Contributing to layer Downloading from https://github.com/cloudfoundry/jvmkill/releases/download/v1.16.0.RELEASE/jvmkill-1.16.0-RELEASE.so Verifying checksum Copying to /layers/paketo-buildpacks_bellsoft-liberica/jvmkill Writing env.launch/JAVA_OPTS.append Link-Local DNS: Contributing to layer Copying to /layers/paketo-buildpacks_bellsoft-liberica/link-local-dns Writing profile.d/link-local-dns.sh Java Security Properties: Contributing to layer Writing env.launch/JAVA_OPTS.append Writing env.launch/JAVA_SECURITY_PROPERTIES.override Security Providers Configurer: Contributing to layer Copying to /layers/paketo-buildpacks_bellsoft-liberica/security-providers-configurer Writing profile.d/security-providers-classpath.sh Writing profile.d/security-providers-configurer.sh OpenSSL Certificate Loader: Contributing to layer Copying to /layers/paketo-buildpacks_bellsoft-liberica/openssl-security-provider Writing profile.d/openssl-certificate-loader.sh Paketo Maven Buildpack 1.4.3 https://github.com/paketo-buildpacks/maven Build Configuration: $BP_MAVEN_BUILD_ARGUMENTS -Dmaven.test.skip=true package (default) the arguments to pass to Maven $BP_MAVEN_BUILT_ARTIFACT target/*.[jw]ar (default) the built application artifact explicitly. Supersedes $BP_MAVEN_BUILT_MODULE $BP_MAVEN_BUILT_MODULE (default) the module to find application artifact in Creating cache directory /home/vcap/.m2 Compiled Application: Contributing to layer Executing mvnw -Dmaven.test.skip=true package [INFO] --- spring-boot-maven-plugin:2.3.0.RELEASE:repackage (repackage) @ spring-petclinic --- [INFO] Replacing main artifact with repackaged archive [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 03:10 min [INFO] Finished at: 2020-06-09T21:35:17Z [INFO] ------------------------------------------------------------------------ Removing source code Paketo Executable JAR Buildpack 1.2.5 Writing env.launch/CLASSPATH Process types: executable-jar: java -cp "${CLASSPATH}" ${JAVA_OPTS} org.springframework.boot.loader.JarLauncher task: java -cp "${CLASSPATH}" ${JAVA_OPTS} org.springframework.boot.loader.JarLauncher web: java -cp "${CLASSPATH}" ${JAVA_OPTS} org.springframework.boot.loader.JarLauncher Paketo Spring Boot Buildpack 1.5.5 Image labels: org.opencontainers.image.title org.opencontainers.image.version org.springframework.boot.spring-configuration-metadata.json org.springframework.boot.version ===> EXPORT Adding layer 'launcher' Adding layer 'paketo-buildpacks/bellsoft-liberica:class-counter' Adding layer 'paketo-buildpacks/bellsoft-liberica:java-security-properties' Adding layer 'paketo-buildpacks/bellsoft-liberica:jre' Adding layer 'paketo-buildpacks/bellsoft-liberica:jvmkill' Adding layer 'paketo-buildpacks/bellsoft-liberica:link-local-dns' Adding layer 'paketo-buildpacks/bellsoft-liberica:memory-calculator' Adding layer 'paketo-buildpacks/bellsoft-liberica:openssl-security-provider' Adding layer 'paketo-buildpacks/bellsoft-liberica:security-providers-configurer' Adding layer 'paketo-buildpacks/executable-jar:class-path' Adding 1/1 app layer(s) Adding layer 'config' *** Images (sha256:af858b3972b567fffa5792a9bed924c3638aba5eab4593c5a43364a67871efdc): gcr.io/test-app gcr.io/test-app:b1.20200609.213101 Adding cache layer 'paketo-buildpacks/bellsoft-liberica:jdk' Adding cache layer 'paketo-buildpacks/maven:application' Adding cache layer 'paketo-buildpacks/maven:cache' ===> COMPLETION Build successful ``` # <a id='image-with-kubectl'></a> Managing Images with kubectl Build Services images can be created by applying the [kpack image resources](https://github.com/pivotal/kpack/blob/master/docs/image.md) to cluster. The `default` service account should be used to utilize build service registry and git secrets. ### <a id='kubectl-builders'></a> Using a Build Service Builder with kubectl The `builder` field on the kpack Image spec describes the builder that will build the image. * Using a a Custom Builder ```yaml builder: name: project-builder-name kind: CustomBuilder ``` - `name`: The name of the Custom Builder. * Using a Custom Cluster Builder ```yaml builder: name: builder-name kind: ClusterClusterBuilder ``` - `name`: The name of the cluster builder resource. For more info on manging builders, see [Managing Builders](managing-builders.html). Warning: Unlike with <code>kp</code> you will need to explicitly set the builder in the image configuration and it will not default to the <code>default</code> builder. ### <a id='kubectl-secrets'></a> Using Secrets The `default` service account should be used to utilize build service registry and git secrets. kpack will default to the `default` service account if no service account is specified. ### <a id='image-status'></a> Image Status When an image has successfully built with its current configuration, its status will report the up to date fully qualified built image reference. This information is available with `kubectl describe image <image-name>`. ```yaml status: conditions: - lastTransitionTime: "2020-01-17T16:16:36Z" status: "True" type: Succeeded latestImage: index.docker.io/sample/image@sha256:d3eb15a6fd25cb79039594294419de2328f14b443fa0546fa9e16f5214d61686 ... ``` When a build fails its status will report the condition Succeeded=False. ```yaml status: conditions: - lastTransitionTime: "2020-01-17T16:13:48Z" status: "False" type: Succeeded ... ```