---
# System prepended metadata

title: KG to use Core for triples generation

---

# KG to use Core for triples generation

### KG sending an event to Core

```mermaid
sequenceDiagram
    participant EventLog
    participant TriplesGenerator
    participant TokenRepository
    participant Core

    EventLog ->> TriplesGenerator: sends AWAITING_GENERATION (commit_id, project_id, project_path)
    activate TriplesGenerator
    TriplesGenerator ->> Core: check if there are not too many jobs in the waiting status
    Core ->> TriplesGenerator: 1 in waiting status
    TriplesGenerator ->> EventLog: sends the Accepted 202 status
    TriplesGenerator ->> TokenRepository: fetches access token
    activate Core
    TriplesGenerator ->> Core: generate graph (commit_id, project_id, project_path, callback_url = http://triples-generator:9002/events) (oauth token)
    Core ->> TriplesGenerator: 200 Accepted with jobId
    deactivate Core
    TriplesGenerator ->> EventLog: sends EVENTS_STATUS_CHANGE to: GENERATING_TRIPLES with jobId
    deactivate TriplesGenerator
```

### Core sending a callback with the json-ld payload (or failure)

```mermaid
sequenceDiagram
    participant EventLog
    participant TriplesGenerator
    participant Core

    Core ->> TriplesGenerator: sends json-ld of a project or a failure + info about eventId, projectId and some info about the generation: processingTime
    TriplesGenerator ->> EventLog: sends EVENTS_STATUS_CHANGE to: TRIPLES_GENERATED with payload or failure
```

### Too many events waiting on the Core side

```mermaid
sequenceDiagram
    participant EventLog
    participant TriplesGenerator
    participant Core

    EventLog ->> TriplesGenerator: sends AWAITING_GENERATION (commit_id, project_id, project_path)
    activate TriplesGenerator
    TriplesGenerator ->> Core: check if there are not too many jobs in the waiting status
    Core ->> TriplesGenerator: 3 in waiting status
    TriplesGenerator ->> EventLog: sends 429 Too Many Requests (they BUSY status)
    deactivate TriplesGenerator
```

### Questions?
* Does Core expose an endpoint to verify CLI and schema versions it's running?
* Can Core work with a bearer token (GitLab OAuth token)?
* Can Core send us processing time of the generation process on callback?
* What are the statuses of the graph generation job on the Core?
* Can we get status other than 200 while posting graph generation request?
* Do we have the same robustness around repo cloning and commit checkout as we have now on TG?
* Can Core send us back on callback exact Git failure messages when they happen on graph generation? (we use them to distinguish recoverable and non-recoverable failures).
* Can Core give us a list of graph generation jobs only?


### Things to do:
* TG to check on startup CLI and schema versions to decide about re-provisioning.
* on `EVENTS_STATUS_CHANGE` to: `GENERATING_TRIPLES` from TG to EL, EL would have to associate the `jobId` with the event in the `event_delivery` table (probably we'd need to add a new `core_job_id` column which can be used exchangably with `delivery_id` and `event_type_id`);
* the Zombie Chasing process would have to be able to take the `core_job_id` from the `event_delivery` and check if the event is still running by quering a relevant endpoint on the Core. Then, if
  * Core says it's running -> just wait;
  * Core says xxx -> ???
* a new endpoint on TG to receive the payload from Core (or reuse the existing `POST /events` on TG; the problem here is forcing Core to shape the payload the way it conforms with the KG event standards)
* TG upon `AWAITING_GENERATION` event arrival to verify if the number of jobs scheduled/running/waiting (to be decides which status) on Core is not higher than the max (e.g. there should not be more that 2 waiting in the queue otherwise the algorithm to select events by their 'interactiveness' or size of the project won't work)