# Message Specification
## Project Creation Data

Properties on data service:

{
    project_id: str
    name: str
    slug: str
    repositories: list[str]
    visibility: “public”|”private”
    description: optional[str]
    created_by: user
    creation_date: datetime
    members:list[user]
}

## General Message Shell
(based on cloud events)

{
    id: redis id 
    headers:{
      source: “renku-data-service”
      type: “renku.project.created”
      datacontenttype: “application/avro+json”|”application/avro+binary”
      schemaversion: "v1"
      requestid: "p33imb6qfnr817p9ltl8nqn38btvbibz" //follows the X-REQUEST-ID header on gateway
      time: datetime
    }
    payload:{
      ...payload
    }
}

Not used:
- specversion: major.minor, required in cloud events, not sure we need it. * The version of the CloudEvents specification which the event uses. This enables the interpretation of the context. Compliant event producers MUST use a value of 1.0 when referring to this version of the specification.*
- subject: *This describes the subject of the event in the context of the event producer (identified by source). In publish-subscribe scenarios, a subscriber will typically subscribe to events emitted by a source, but the source identifier alone might not be sufficient as a qualifier for any specific event if the source context has internal sub-structure.*
- dataschema: *Identifies the schema that data adheres to. Incompatible changes to the schema SHOULD be reflected by a different URI*

## AsyncAPI
For channels, in rabbitmq we could have had routing keys like `project.<id>.created` and subscribers could listen for e.g. `project.1.*` or `project.*`. In Redis streams this isn't feasible, as there's no pattern matching. So we need generic streams and consumers need to potentially do pattern matching. We should also add consumer groups for search.
We could have streams per entity or per entity+event-type. For now I assume entity+event-type.

# in file asyncapi/v1/project.yaml
asyncapi: 3.0.0
info:
  title: Project Events
  version: 0.0.1
servers:
  redis:
    url: renku-redis
    protocol: redis
    description: Renku Redis Instance
channels:
  project.created:
    publish:
      messages:
        projectCreated:
          schemaFormat: "application/vnd.apache.avro;version=1.9.0"
          payload:
            type: object
            properties:
              payload:
              - $ref: '/project/v1/avro/created.avsc#/ProjectCreated'
          traits:
          - $ref: '#/components/messageTraits/headers'
        
components:
  messageTraits:
    headers:
      payload:
        type: object
        properties:
          id:
            type: string
          headers:
            - $ref: '/common/v1/avro/headers.avsc#/Header'
---
# on file common/v1/avro/headers.avsc
{
  "type":"record",
  "name":"Header",
  "fields":[
    {
      "name": "source",
      "type":"string"
    },
    {
      "name": "type",
      "type":"string"
    },
    {
      "name": "datacontenttype",
      "type":"enum",
      "symbols":["application/avro+json","application/avro+binary"]
    },
    {
      "name":"schemaversion",
      "type":"string"
    },
    {
      "name":"time",
      "type":"int",
      "logicalType":"timestamp-millis"
    },
    {
      "name":"requestid",
      "type":"string"
    }
  ]
}
---
# in file /project/v1/avro/created.avsc
{
   "type":"record",
   "name":"ProjectCreated",
   "fields":[
      {
         "name":"name",
         "type":"string"
      },
      {
         "name":"slug",
         "type":"string"
      },
      {
         "name":"repositories",
         "type":"array",
         "items": "string",
         "default":[]
      },
      {
         "name":"visibility",
         "type":"enum",
         "symbols":["public","private"]
      },
      {
         "name":"description",
         "type":["null", "string"]
      },
      {
         "name":"created_by",
         "type":"string"
      },
      {
         "name":"creation_date",
         "type":"int",
         "logicalType":"timestamp-millis"
      },
      {
        "name":"members",
        "type": "array",
        "items":"string",
        "default":[]
      }
   ]
}

- do we need servers section? It seems a bit useless to me. (we'll leave it in with just renku-redis)
- should only payload be avro or whole message (headers is avro-json, payload is avro json or binary)
- one downside of avro is that it doesn't allow merging schemas (to my knowledge), so we couldn't have the common properties in an avsc file and refer to them from other avsc. we could define an avro type like "Headers" and refer to it in other schemas, but unsure if common clients handle multiple schemas

## Folder Layout

there's two top level folders, one for AsyncAPI, one for Avro Schemas, with the former referencing the latter.

.

├── project/
│   └── v1/
│       ├── asyncapi.yaml
│       └── events/
│           ├── created.avsc
│           ├── updated.avsc
│           └── deleted.avsc
├── user/
│   └── v1/
│       ├── asyncapi.yaml
│       └── events/
│           ├── created.avsc
│           └── deleted.avsc
└── common/
    └── v1/
        └── headers.avsc

How to do versioning isn't really clear. I went with the simple option of only changing the version with breaking changes and otherwise just updating the existing file, as avro should handle non-breaking changes.
We should add some CI to check that no breaking change sneaks in.