# Identus Platform CI

## Common definitions of the terms in the CI

### Build

A build refers to compiling source code into binary code that will be distributed and executed on a machine (local or cloud environment). It includes compiling, linking, and packaging the application. In the Continuous Integration (CI) context, a build is triggered automatically whenever changes are pushed to the code repository. The build process may include running automated tests to ensure the new code doesn't break existing functionality.

### Publishing

Publishing is the process of distributing compiled software artifacts (like libraries, packages, or applications) to a repository or registry where others can access and use them. For example, in a CI pipeline, publishing may involve uploading a package to a package manager (like npm, GitHub Packages, Maven Central, Helm Chart registry, or Docker registry). It makes the artifact available for downstream processes.

### Deployment

Deployment is moving the built and tested software to a live environment where end-users can access and use it. It typically involves copying and updating the Helm Chart of the corresponding component, setting up the necessary environment, and configuring the application. In a CI/CD pipeline, deployment is often automated and can be triggered by successful builds or passing tests. Two environments are used for the automated distribution of the Cloud-Agent and the Mediator services: DEV and SIT. Automated deployment of the latest component version to the DEV environment is essential to the CI process. It's performed by distributing the docker image of the service and updating the Helm chart. Argo CD is configured to track the correct version of the component according to the [Tracking Deployment Strategies](https://argo-cd.readthedocs.io/en/stable/user-guide/tracking_strategies/) configuration of the Argo CD state.

### Release

A release is the distribution of a specific version of software to end-users. It marks the point where the software is considered stable and ready for use. A release includes release notes, changelog, dependencies, the corresponding version of the Open API specification, and other important artifacts and is often tied to a milestone or feature set. In CI/CD, a release is a formal process that includes tagging the codebase, generating release artifacts, and possibly notifying users or stakeholders.
//TODO: add the reference to the release documentation that includes the list of the assets and common process

### Revision Build

Revision Build refers to a new software build version with minor changes or fixes, typically without significant new features. It often addresses bugs or applies minor updates to an existing release. In CI/CD, a revision build is triggered by small code changes or hotfixes and is versioned with an incremented revision number. For micro-services, the revision number is a current version of the project, the build number, and the short commit sha (for instance, 1.38.0-4-abcdefj). For the library, it's the current version of the library and short commit sha (for example, 1.2.0-abcdefj).

## Continuous Integration of the micro-services

Current flow is applied to the microservices: the Cloud Agent and the Mediator.
The sequence diagram below shows a more complex flow that is applied to the Cloud Agent.
For each commit to the tracked branches (`main` is used by default) the `build.yml` workflows is triggered and based on the current version of the project, build number and short commit sha, the next revision version is set:
- for the docker image the format is `version`-`build`-`sha`
- for the Open API speficiation the version is `version`-`sha`
- for the client liberaries the version is `version`-`sha`

**NOTE**:
- The unit, integration, e2e and performance tests are not run. It's assumed that everything that is committed to the tracked branch already passed these checks
- The build number in the docker tag is required to distinguish the version update based on the increment number of each build. Build number is not required for the liraries. (the packages with a changes in the Open API specification are marked with `sha` of the revision)

sequenceDiagram
    actor E as Engineer
    participant ICAR as identus-cloud-agent repository
    participant BWF as build-workflow
    participant RCWF as release-client-workflow
    participant GHCR as GitHub Packages
    participant APHC as atala-prism-helm-charts repository
    participant ArgoCD as ArgoCD
    participant DEV as DEV

E ->> ICAR: merge PR
Note over ICAR, APHC: Publish the docker image, clients and helm chart
ICAR ->> BWF: trigger build 
BWF -->> BWF: build a docker image of "cloud-agent-1.35.0-N-abcdefg"
BWF -) GHCR: publish a docker image
BWF -->> BWF: build a revision version of the OAS
BWF -->> ICAR: publish a revision version of the OAS
BWF --) +RCWF: trigger release-client workflow
ICAR -->> RCWF: download a published OAS
RCWF -->> RCWF: build clients (TS, Kotlin) for revision version
RCWF -) GHCR: publish Kotlin client
RCWF -) -GHCR: publish TypeScript client
BWF -) +APHC: trigger repo_dispatch "build-chart-package"
Note over GHCR, DEV: Deploy revision version of the cloud-agent to the DEV
APHC -->> APHC: build a new helm chart for revision version
APHC -->> -APHC: commit a new package to the helm chart repository
ArgoCD -->> APHC: sync the chart
ArgoCD -->> DEV: apply the latest revision version for the DEV
DEV -->> GHCR: fetch the image of the revision version
DEV ->> DEV: sync with the helm chart

## Release a new version of the Cloud-Agent or the Mediator

The following flow covers three cases when the new version is released:
- regular release
- RC release
- patch release

The configuration of the semantic-release plugin contains the branch configuration section that dictates the patters of the branches and what release is it possible to do from the branch.

By default, the `main` branch is used for a regular release, `beta` or `beta/*` for RC releases, and the branches with particular patters are used for the patch (`1.38.x` - for patches of 1.38.x versions, or `1.x.x` for patching all versions that match 1.x.x patternt)

The following flow should be executed manually by the engineer and can be targeted the `main` branch or any release branch.

Checks:
- PR title follows conventional-commits
- DCO
- commit is signed by the GPG key
- Megalinter
- unit tests
- e2e tests
- performance tests

Outputs of the flow:
- a new docker image of the cloud-agent is published to GHCR
- a new helm chart for the cloud-agent is published to the atala-prism-helm-charts
- a new version of clients for TS and Kotlin are published to GHCR
- a new version of the cloud-agent is deployed to the DEV environment
- a new tag with a release version is pushed to identus-cloud-agent repo
- a chore commit with the updates is pushed to the identus-cloud-agent repo:
- CHANGELOG.md
- OAS specification
- DEPENDENCIES.md
- infrastructure/local/.env
- Slack notification is sent about the successful or failed release

Any type of release must be manually triggered by the engineer.

The following flow shows the release process:

sequenceDiagram
    actor E as Engineer
    participant ICAR as identus-cloud-agent repository
    participant RWF as release-workflow
    participant RCWF as release-client-workflow
    participant GHCR as GitHub Packages
    participant NPMJS as NPMJS
    participant APHC as atala-prism-helm-charts repository
    participant ArgoCD as ArgoCD
    participant DEV as DEV

Note over ICAR, APHC: Release RC version of the cloud-agent
E ->> RWF: trigger release 
RWF -->> +RWF: define a new RC version "cloud-agent-v1.35.0-rc.1"
RWF -->> RWF: run unit tests
RWF -->> RWF: run e2e tests
RWF -->> RWF: run peformance tests
RWF -->> RWF: build a docker image of "cloud-agent-v1.35.0-rc.1"
RWF --) GHCR: publish a docker image "v1.35.0-rc.1"
RWF --) ICAR: commit changelog, dependencies, OAS
RWF -) APHC: trigger repo_dispatch "build-chart-package"
RWF -) Slack: notify about release "v1.35.0-rc.1"
RWF --) -ICAR: puch the RC tag
ICAR --) RCWF: trigger release-client workflow
RCWF -->> RCWF: build a version of the OAS for "v1.35.0-rc.1"
RCWF -->> RCWF: build clients (TS, Kotlin) for  "v1.35.0-rc.1"
RCWF -) GHCR: publish Kotlin client
RCWF -) NPMJS: publish TypeScript client
Note over GHCR, DEV: Deploy RC version of the cloud-agent to the DEV
APHC -->> APHC: build a new helm chart for "v1.35.0-rc.1"
APHC -->> APHC: commit a new package to the helm chart repository
ArgoCD -->> APHC: sync the chart "v1.35.0-rc.1"
ArgoCD -->> DEV: apply the latest version for DEV
DEV -->> GHCR: fetch the image for "v1.35.0-rc.1"
DEV ->> DEV: sync with the chart "v1.35.0-rc.1"

## Code Freeze of the Cloud-Agent or the Mediator

Code freeze is a phase in the Software Development Life Cycle (SDLC) where no new code changes are allowed, except for critical bug fixes or essential updates. This phase is typically implemented just before a software release to stabilize the codebase, ensuring that no new issues are introduced that could jeopardize the quality or functionality of the software.

### Purpose of Code Freeze

**Stabilization**: Ensures that the code is stable and all functionalities work as expected without introducing new features or changes that could potentially cause bugs or other issues.

**Quality Assurance**: Allows the QA team to focus on testing the existing codebase without worrying about changes that could invalidate their tests or introduce new bugs.

**Release Preparation**: Prepares the codebase for deployment by minimizing risks associated with last-minute changes. This is particularly important in ensuring that the final product meets the expected standards.

**Risk Mitigation**: Reduces the likelihood of last-minute changes that could introduce critical issues, thereby lowering the risk of a delayed or problematic release.

### Code-Freeze Process

**TL;DR**

The main purpose of the code freeze is to isolate a stable version of the components (services and libraries) for QA activities and stabilization in a dedicated environment. This process prevents the addition of new features planned for future releases.
The branch used for this isolation is beta, and the environment is SIT.

**Details**

The process consists of the following steps:
- The team announces the beginning of the code freeze stage.
- A new branch is created from the target revision (e.g., beta).
- The release workflow is triggered from the beta branch:
- Docker images are published.
- HTTP Clients are published.
- Helm Charts are published.
- The RC version is deployed to the SIT environment for stabilization.
- Critical fixes are applied to the beta branch.

During this stage, other engineers can continue merging PRs into the main branch, running CI in the DEV environment without affecting the SIT environment.

All commits to the beta branch must be merged back into the main branch. Engineers can choose to do this continuously (by creating two PRs for each change) or as a single PR at the end of the code-freeze stage.

Upon completion of the code-freeze stage, the engineering team has two options for proceeding with the release:
- If the main branch doesn't contain breaking changes, the changes from the beta branch are merged into the main branch, and the release workflow is triggered from the main branch.
- If the main branch contains breaking changes, a new branch for the LTS release (e.g., 1.40.x) is created, and the release flow is triggered from this branch.

**NOTE**:
Engineering team should undestand that in the case when the release is made from the release branch, the principle `Release From Trunk` is not used.

Since the builds and CI for the DEV environment are isolated from the SIT environment and the beta branch, engineers can continue working on feature branches and make builds from those branches. In this scenario, CI in the DEV environment will continue without any blockers.

## Versions

The following table shows the possible versions of the assets

| Build | Version | Example |
| -------- | -------- | -------|
| Revision | {component-version}-{build-number}-{commit-sha} | 1.39.0-45-abcd123 |
| RC | {component-version}-{rc}.{index} | 1.39.0-rc.1 |
| Release. | {component-version} | 1.39.0 |

According to the SemVer rules:

1.39.0-rc.1 < 1.39.0-45-abcd123 < 1.39.0

## Build Promotion Rules

**TODO**: These rules must be agreed with QA team. Additional rules to guarantee compabilitility with other components and quality will be defined later.

Rules:
- Revision Build is promoted to the DEV environment automatically
- RC Build is promoted to the DEV environment automatically
- RC Build is promoted to the SIT environment automatically
- Release Build is promoted to the SIT environment automatically