Custom Event Types

Contributors:

Andrea Frittoli

References:

Introduction

CDEvents brings standardization for events consumed and produced by tools in the SDLC. Some of the tools that are candidate for CDEvents adoption already produce events, in their own specific formats. Some of these events exist or can be mapped to events available in the CDEvents specification. Some events might be added to CDEvents, if they're considered for interoperability by the CDEvents community. Some events however, are very specific to a tool or are not relevant from an interoperability point of view.

This document proposes the introduction of custom event types as a mean to make it easier for tools to adopt CDEvents and provide event producers and consumer with a consistent way to produce and consume events aligned to the CDEvents specification.

Proposal

This proposal involves changes to the specification and accordingly to the SDKs.

Specification

The following changes are introduced to the specification:

The dev.cdeventsx event-type namespace: the dev.cdeventsx namespace for event types is introduced. This namespace is reserved for events that are compliant with the CDEvents specification, whose subject structure and semantics are defined outside of CDEvents.
The schemaURI property in the context: events may supply their schema URI via this new context property, as described in the corresponding issue. Events must always validate against the CDEvents official schema too.
The dev.cdeventsx jsonschema: any event of type dev.cdeventsx.* must respect this schema as well as any additional schema supplied via schemaURI
The subject format for dev.cdeventsx events: subjects must be in the format <tool-name>-<subject-name> to avoid event-type conflicts across tools

SDKs

When consuming (parsing) an event with context.schemaURI, SDKs will have to be able to fetch the schema defined in context.schemaURI and additionally validate the event against the supplied schema.

When consuming (parsing) an event with dev.cdeventsx type, the SDKs will return a object that is identical in structure for all events, and which include an unparsed blob for the subject.content part. SDKs may provide a way for users to register a function to be used to parse the subject.content of these messages.

When producing events, the SDK may provide a way for users to register additional functions to be used to render dev.cdeventsx events as JSON and CloudEvents.

Transitioning Custom CDEvents to Standard CDEvents

In certain cases, custom events may be implemented as a stopgap solution to allow for faster CDEvents adoption. The custom events may be eventually make its way into the CDEvents specification, with a structure that could be different from the original one defined in the custom events.

This situation is not dissimilar from the process of adopting a new, backwards incompatible version of an existing event. Once the new CDEvent is introduced in the spec, newer versions of the SDKs will be able to produce it and consume it.

Promoting a Custom CDEvent to Core CDEvent

To create a new CDEvent, start with the guide available in the CDEvents primer. Use cases and experiences from using the custom event would be a great addition to the proposal:

Start proposing the new CDEvent in a GitHub issue. Attach relevant use cases and context from the corresponding custom event. Highlight why the event would be beneficial from interoperability point of view
(Optional) Join one of the CDEvents working groups to present your proposal. This can also happen asynchronously if the working group schedule is not convenient
Create a PR that adds the new event. You may search in the recent PR history for other similar PRs to guide you in the process. Promote your PR at the working groups and/or on Slack to obtain feedback. If struggling with the CI jobs, ask on slack for help
Once the PR is merged, the new event will be available in the spec and in the SDKs with their next release. The release of the SDKs may happen some time after the release of the specification

Handling Heterogeneous Producers and Consumers

It is safe to assume that several producers and consumers of the event exist. At some point in time only a fraction of producers and consumers will have adopted the new SDK.

Consumers with the latest SDK will be able to parse the old custom event as well as the new CDEvent. This means that there is no pressure for producers to adopt the new version, even after all consumers have been updated.

Consumers with the old SDK however, won't be able to parse the new CDEvents. This means that producers that adopt the new SDK may have to consider producing both the old and the new events, until all consumers have update to the latest SDK. In this case, it is responsability of the system architect to ensure that old events are only sent to legacy consumers, to avoid the case of consumers receiving duplicate events.

CDEvents and Links

The CDEvents community is working on the introduction of links to the specification. Links will let CDEvents producer connect one event to others with specific semantics, and will help consumers trace through events to understand complete workflows.

Custom CDEvents can be linked like any other CDEvent, so links may exist between custom CDEvents as well as between a custom CDEVent and a normal CDEvent.

Case Study - Harbor Events

Harbor is a popular open source registry and a graduated project at the CNCF. The Harbor registry can produce CloudEvents today, and the project community is enthustiastic about adopting CDEvents. They do have one concern though, they would like Harbor users that adopt the CDEvents feature to have access to the same range of events they have through the existing CloudEvents feature.

Harbor Events

Harbor supports todays the following events (webhook notifications):

PUSH_ARTIFACT
PULL_ARTIFACT
DELETE_ARTIFACT
SCANNING_COMPLETED
SCANNING_STOPPED
SCANNING_FAILED
QUOTA_EXCEED
QUOTA_WARNING
REPLICATION
TAG_RETENTION

Mapping Harbor Events to CDEvents

According to the Harbor requirements, each of the ten Harbor events should be mapped to a CDEvent. The secionts below shows an analysis for each event of what the mapping could look like.

Artifact Events

Artifact events have a good matches the existing dev.cdevents.artifact.* CDEvents.

Harbor fields that do not exist on the corresponding CDEvent would go in customData and could be promoted over time into the CDEvents schema. Harbor could maintain the schema for the fields in customData.

The artifact events from Harbor may include an array of resources.
On CDEvents side, the first resource would be used as subject, and the whole list would be included in the customData.

PUSH_ARTIFACT

Harbor Event: PUSH_ARTIFACT
Area: Artifacts
CDEvent: dev.cdevents.artifact.published
Custom Harbor CDEvent: N/A
Extra Schema: Harbor extra fields in customData
Release: CDEvents v0.3.0

PULL_ARTIFACT

Harbor Event: PULL_ARTIFACT
Area: Artifacts
CDEvent: dev.cdevents.artifact.downloaded
Custom Harbor CDEvent: N/A
Extra Schema: Harbor extra fields in customData
Release: CDEvents v0.4.0 (upcoming)

DELETE_ARTIFACT

Harbor Event: DELETE_ARTIFACT
Area: Artifacts
CDEvent: dev.cdevents.artifact.deleted
Custom Harbor CDEvent: N/A
Extra Schema: Harbor extra fields in customData
Release: CDEvents v0.4.0 (upcoming)

Example Artifact Pushed

This example is taken from the Harbor docs:




























{
  "specversion": "1.0",
  // id is a generated UUID to make sure the unique
  "id": "66e18103-09c1-41f6-982f-37df223f3eeb",
  // requestid is the http request id for tracing the source request of this event
  "requestid": "51c0b694-0168-4f3c-b0db-282565455d7b",
  "source": "/projects/2/webhook/policies/15",
  "type": "harbor.artifact.pushed",
  "datacontenttype": "application/json",
  "time": "2023-04-03T06:04:46Z",
  "data": {
    "resources": [
      {
        "digest": "sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4",
        "tag": "sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4",
        "resource_url": "localhost/harbor/alpine@sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4"
      }
    ],
    "repository": {
      "date_created": 1680501893,
      "name": "alpine",
      "namespace": "harbor",
      "repo_full_name": "harbor/alpine",
      "repo_type": "private"
    }
  },
  "operator": "harbor-jobservice"
}

Corresponding CDEvents:






































{
  "context": {
    "version": "0.4.0-draft",
    "id": "271069a8-fc18-44f1-b38f-9d70a1695819",
    "source": "/harbor/alpine",
    "type": "dev.cdevents.artifact.published.0.2.0-draft",
    "timestamp": "2023-03-20T14:27:05.315384Z",
    "schemaURI": "https://goharbor.io/cdevents/schema/artifact/published/0_2_0"
  },
  "subject": {
    "id": "pkg:oci/localhost/harbor/alpine@sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4",
    "source": "/harbor/alpine",
    "type": "artifact",
    "content": {
      "sbom": {
        "uri": "https://sbom.repo/myorg/234fd47e07d1004f0aed9c.sbom"
      },
      "user": "harbor-jobservice"
    }
  },
  "customData": {
    "requestid": "98ecbced-4169-443b-8e19-459a8c81675d",
    "resources": [
      {
        "digest": "sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4",
        "tag": "sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4",
        "resource_url": "localhost/harbor/alpine@sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4"
      }
    ],
    "repository": {
      "date_created": 1680501893,
      "name": "alpine",
      "namespace": "harbor",
      "repo_full_name": "harbor/alpine",
      "repo_type": "private"
    }
  }
}

Artifact Scanning Events

Artifact scanning events do not have a direct match on CDEvents. There are a few options available:

Map scanning operations to the existing testing events
Introduce new artifact security/scanning events in CDEvents. See this thread for some example ideas in this area
Use dev.cdeventsx.harbor-scanning.* event types, with type, subject.type and subject.content maintained by Harbor

When using existing or new CDEvents, Harbor fields that do not exist on the corresponding CDEvent would go in customData and could be promoted over time into the CDEvents schema. Harbor could then maintain the schema for the fields in customData.

SCANNING_COMPLETED

Harbor Event: SCANNING_COMPLETED
Area: Security
CDEvent: Testing events or new artifact security event
Custom Habor CDEvent: fallback option, dev.cdeventsx.harbor-scan.completed
Extra Schema: Harbor extra fields in customData
Release: Testing events in v0.3. Dedicated events in >= v0.4

SCANNING_STOPPED

Harbor Event: SCANNING_STOPPED
Area: Security
CDEvent: Testing events or new artifact security event
Custom Habor CDEvent: fallback option, dev.cdeventsx.harbor-scan.stopped
Extra Schema: Harbor extra fields in customData
Release: Testing events in v0.3. Dedicated events in >= v0.4

SCANNING_FAILED

Harbor Event: SCANNING_FAILED
Area: Security
CDEvent: Testing events or new artifact security event
Custom Habor CDEvent: fallback option, dev.cdeventsx.harbor-scan.failed
Extra Schema: Harbor extra fields in customData
Release: Testing events in v0.3. Dedicated events in >= v0.4

Example Scan Complete

This example is taken from the Harbor docs:













































{
  "specversion": "1.0",
  "id": "64bce2fe-6159-454d-8389-852d01ef1e9d",
  "requestid": "98ecbced-4169-443b-8e19-459a8c81675d",
  "source": "/projects/2/webhook/policies/15",
  "type": "harbor.scan.completed",
  "datacontenttype": "application/json",
  "time": "2023-04-03T06:12:55Z",
  "data": {
    "resources": [
      {
        "digest": "sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4",
        "resource_url": "localhost/harbor/alpine@sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4",
        "scan_overview": {
          "application/vnd.security.vulnerability.report; version=1.1": {
            "report_id": "af0546c1-67dc-4e9d-927e-372900ead0df",
            "scan_status": "Success",
            "severity": "None",
            "duration": 8,
            "summary": {
              "total": 0,
              "fixable": 0,
              "summary": {}
            },
            "start_time": "2023-04-03T06:12:47Z",
            "end_time": "2023-04-03T06:12:55Z",
            "scanner": {
              "name": "Trivy",
              "vendor": "Aqua Security",
              "version": "v0.37.2"
            },
            "complete_percent": 100
          }
        }
      }
    ],
    "repository": {
      "name": "alpine",
      "namespace": "harbor",
      "repo_full_name": "harbor/alpine",
      "repo_type": "private"
    }
  },
  "operator": "auto"
}

Corresponding CDEvent using a testing event:

































































{
  "context": {
    "version": "0.4.0-draft",
    "id": "271069a8-fc18-44f1-b38f-9d70a1695819",
    "source": "/harbor/alpine",
    "type": "dev.cdevents.testcaserun.finished.0.1.0",
    "timestamp": "2023-03-20T14:27:05.315384Z",
    "schemaURI": "https://goharbor.io/cdevents/schema/testcaserun/finished/0_1_0"
  },
  "subject": {
    "id": "Trivy",
    "source": "/harbor/alpine",
    "type": "testCaseRun",
    "content": {
      "outcome" : "pass",
      "environment": {
        "id": "prod",
        "source": "harbor-prod"
      },
      "testCase": {
        "id": "0001",
        "version": "v0.37.2",
        "name": "Trivy Scan by Acqua Security",
        "type": "security"
      }
    }
  },
  "customData": {
    "requestid": "98ecbced-4169-443b-8e19-459a8c81675d",
    "resources": [
      {
        "digest": "sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4",
        "resource_url": "localhost/harbor/alpine@sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4",
        "scan_overview": {
          "application/vnd.security.vulnerability.report; version=1.1": {
            "report_id": "af0546c1-67dc-4e9d-927e-372900ead0df",
            "scan_status": "Success",
            "severity": "None",
            "duration": 8,
            "summary": {
              "total": 0,
              "fixable": 0,
              "summary": {}
            },
            "start_time": "2023-04-03T06:12:47Z",
            "end_time": "2023-04-03T06:12:55Z",
            "scanner": {
              "name": "Trivy",
              "vendor": "Aqua Security",
              "version": "v0.37.2"
            },
            "complete_percent": 100
          }
        }
      }
    ],
    "repository": {
      "name": "alpine",
      "namespace": "harbor",
      "repo_full_name": "harbor/alpine",
      "repo_type": "private"
    },
    "operator": "auto"
  }
}

Operational Events

Operational scanning events do not have a direct match on CDEvents and are not good matches for CDEvents use cases; the proposed route forward for these is to implement them as custom CDEvents owned by Harbor.

QUOTA_EXCEED

Harbor Event: QUOTA_EXCEED
Area: Operational
CDEvent: N/A
Custom Habor CDEvent: dev.cdeventsx.harbor-quota.exceeded
Extra Schema: Event type, subject.type and subject.content
Release: Custom events available in CDEvents >= v0.4

QUOTA_WARNING

Harbor Event: QUOTA_EXCEED
Area: Operational
CDEvent: N/A
Custom Habor CDEvent: dev.cdeventsx.harbor-quota.warning
Extra Schema: Event type, subject.type and subject.content
Release: Custom events available in CDEvents >= v0.4

QUOTA_WARNING

Harbor Event: REPLICATION
Area: Operational
CDEvent: N/A
Custom Habor CDEvent: dev.cdeventsx.harbor-replication-status.changed
Extra Schema: Event type, subject.type and subject.content
Release: Custom events available in CDEvents >= v0.4

QUOTA_WARNING

Harbor Event: TAG_RETENTION
Area: Operational
CDEvent: N/A
Custom Habor CDEvent: dev.cdeventsx.harbor-tag-retention.finished
Extra Schema: Event type, subject.type and subject.content
Release: Custom events available in CDEvents >= v0.4

Example Quota Exceeded

This example is taken from the Harbor docs:


























{
  "specversion": "1.0",
  "id": "81f243ce-699c-44d6-9dbe-b2ee5f10237a",
  "requestid": "4b9dcf9a-db23-460c-9b52-c9d994e362ee",
  "source": "/projects/2/webhook/policies/15",
  "type": "harbor.quota.exceeded",
  "datacontenttype": "application/json",
  "time": "2023-04-03T07:04:44Z",
  "data": {
    "resources": [
      {
        "digest": "sha256:402d21757a03a114d273bbe372fa4b9eca567e8b6c332fa7ebf982b902207242"
      }
    ],
    "repository": {
      "name": "alpine",
      "namespace": "harbor",
      "repo_full_name": "harbor/alpine",
      "repo_type": "private"
    },
    "custom_attributes": {
      "Details": "adding 2.1 MiB of storage resource, which when updated to current usage of 8.3 MiB will exceed the configured upper limit of 10.0 MiB."
    }
  },
  "operator": ""
}

Corresponding CDEvent using a dev.cdeventsx.* type:

































{
  "context": {
    "version": "0.1.0",
    "id": "271069a8-fc18-44f1-b38f-9d70a1695819",
    "source": "/harbor/alpine",
    "type": "dev.cdeventsx.harbor-quota.exceeded.0.1.0",
    "timestamp": "2023-03-20T14:27:05.315384Z",
    "schemaURI": "https://goharbor.io/cdeventsx/schema/harbor-quota/exceeded/0_1_0"
  },
  "subject": {
    "id": "/projects/2/webhook/policies/15",
    "source": "/harbor/alpine",
    "type": "quota",
    "content": {
      "operator": "",
      "resources": [
        {
          "digest": "sha256:402d21757a03a114d273bbe372fa4b9eca567e8b6c332fa7ebf982b902207242"
        }
      ],
      "repository": {
        "name": "alpine",
        "namespace": "harbor",
        "repo_full_name": "harbor/alpine",
        "repo_type": "private"
      },
    }
  },
  "customData": {
    "requestid": "4b9dcf9a-db23-460c-9b52-c9d994e362ee",
    "details": "adding 2.1 MiB of storage resource, which when updated to current usage of 8.3 MiB will exceed the configured upper limit of 10.0 MiB."
  }
}

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`	在筆記中貼入程式碼
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.