## CircleCI Optimization Tips --- ### Choose the right persisting data  --- ### Cache  ---- - Persists data between the same job in multiple workflow runs - Global within a project - Example: Package dependency --- ### Workspace  ---- - Moves data in-between sequential jobs in a workflow - No longer exists once a workflow is complete - Concurrent call issue - Example: Genreated version number, Metadata --- ### Artifact  ---- - Persist data after a workflow is completed and gone - Longer-term storage - Example: `.apk` `.jar` --- ### Custom storage usage Plan > Usage Controls  --- ### Partial cache restore https://circleci.com/docs/caching#restoring-cache ---- ```yaml - restore_cache: keys: # Find a cache corresponding to this specific checksum # when this file is changed, this key will fail - go-mod-{{ checksum "go.sum" }} # Find the most recently generated cache used from # any branch - go-mod- - save_cache: key: go-mod-{{ checksum "go.sum" }} paths: - /home/circleci/go/pkg/mod ``` --- ## Source caching https://circleci.com/docs/caching#source-caching ---- ```yaml steps: - restore_cache: keys: - source-v1-{{ .Branch }}-{{ .Revision }} - source-v1-{{ .Branch }}- - source-v1- - checkout - save_cache: key: source-v1-{{ .Branch }}-{{ .Revision }} paths: - ".git" ``` ---- - When the repository is too large - git clone v.s. restore_cache - Need run `git gc` manually --- ### Chooese the right executor Setup time: Docker (seconds) v.s. machine (minutes) ```yaml docker: - image: cimg/base:latest ``` --- ### Enable Docker layer cache https://circleci.com/docs/docker-layer-caching ```yaml steps: - setup_remote_docker: docker_layer_caching: true ``` --- ### Choose the right resource class for each job https://circleci.com/docs/configuration-reference#resourceclass ```yaml resource_class: large ``` --- ### Test splitting and parallelism https://circleci.com/docs/parallelism-faster-jobs --- ### Using BuildKit mount SSH key in Docker build ```yaml - add_ssh_keys: fingerprints: - "A:B:C:D:E:F:G:H" - setup_remote_docker: version: "20.10.14" # 18.09 or higher # specify circle ci ssh key path, the path is fixed - run: docker build --ssh main=/home/circleci/.ssh/id_rsa_abcdefgh . ``` ```dockerfile RUN --mount=type=ssh,id=main go mod download ``` --- ### Thanks for listening! --- ## References - https://circleci.com/blog/persisting-data-in-workflows-when-to-use-caching-artifacts-and-workspaces/ - https://docs.docker.com/develop/develop-images/build_enhancements/#using-ssh-to-access-private-data-in-builds