owned this note
owned this note
Published
Linked with GitHub
# Gateway API Endpoints and Data Structures
### Content
* **[Service Discovery](#Service-Discovery1)**
* [Retrieve Service Catalogue List](#Retrieve-Service-Catalogue-List)
* [Query Service Catalogue List](#Query-Service-Catalogue-List)
* [Read Service Description](#Read-Service-Description)
* [Create Service Description](#Create-Service-Description)
* [Update Service Description](#Update-Service-Description)
* [Delete Service Description](#Delete-Service-Description)
* **[Organisation and User Accounts](#Organisation-and-User-Accounts1)**
* [Read Organisation](#Read-Organisation)
* [List all Organisations](#List-Organisations)
* [Create Organisation](#Create-Organisation)
* [Update Organisation](#Update-Organisation)
* [Delete Organisation](#Delete-Organisation)
* [Retrieve User List for Organisation](#Retrieve-User-List-for-Organisation)
* [Read User](#Read-User)
* [Read User Wallet](#Read-User-Wallet)
* [Create User](#Create-User)
* [Update User](#Update-User)
* [Delete User](#Delete-User)
* **[Storage Services](#Storage-Service1)**
* [Retrieve Collection List](#Retrieve-Collection-List)
* [Read Collection](#Read-Collection)
* [Create Collection](#Create-Collection)
* [Update Collection](#Update-Collection)
* [Delete Collection](#Delete-Collection)
* [Check Object Access](#Check-Object-Acces)
* [Read Object (meta data)](#Read-Object-meta-data)
* [Read Object (object itself)](#Read-Object-object-itself)
* [Create Object (meta data)](#Create-Object-meta-data)
* [Write Object (object itself)](#Write-Object-object-itself)
* [Update Object (meta data)](#Update-Object-meta-data)
* [Delete Object](#Delete-Object)
* **[Integration Helper](#Integration-Helper)**
* [Match Usage Policies](#Match-Usage-Policies)
* [Match D2A](#Match-D2A)
* [Create D2A](#Create-D2A)
* [Extract Usage Policy](#Extract-Usage-Policy)
* [Authentication](#Authentication)
* [Further Information](#Further-Information)
## Status
last update: June 13, 2023
#### [Service Discovery](#Service-Discovery1)
| tested | implemented | stub | draft |
| ------:| -----------:| ----:| -----:|
| 6 | 0 | 0 | 0 |
#### [Organisation and User Accounts](#Organisation-and-User-Accounts1)
| tested | implemented | stub | draft |
| ------:| -----------:| ----:| -----:|
| 11 | 0 | 0 | 0 |
#### [Storage Service](#Storage-Service1)
| tested | implemented | stub | draft |
| ------:| -----------:| ----:| -----:|
| 13 | 0 | 0 | 0 |
## Service Discovery
### Retrieve Service Catalogue List
Info:
* Requirement: *sd_1*
* API: `GET /list?filter=value`
available paging parameters are ([RFC-8288](https://tools.ietf.org/html/rfc8288) compliant http response headers)
* page: page number
* items: number of items per page
* sort: field name to sort results, either 'service-id' (default) or 'name'
* publicly available
* Status: *tested*
**Examples:**
`curl https://babelfish.data-container.net/list`
`curl https://babelfish.data-container.net/list?page=2&items=15&sort=name`
Response:
```json=
[
{"service-id": 1, "name": "DID Lint"},
{"service-id": 2, "name": "xyz"}
]
```
### Query Service Catalogue List
Info:
* Requirement: *sd_2*
* API: `GET /service/search?field1=value1&field2=value2`
* publicly available
* Status: *tested*
The field names can be set to any of the fields listed under service description:
* title
* url
* party
* data
**Example:**
`curl https://babelfish.data-container.net/service/search?title=did%20lint`
Response:
```json=
[
{"service-id": 1, "name": "DID Lint"}
]
```
### Read Service Description
Info:
* Requirement: *sd_3*
* API: `GET <base-uri>/service/SERVICE_ID`
* publicly available
* Status: *tested*
**Example:**
`curl https://babelfish.data-container.net/service/1`
Response:
```json=
{
"service-id": 1,
"interface": {
"info": { "title": "DID Lint" },
"servers": [{"url": "https://didlint.ownyourdata.eu"}],
"party": "data_consumer",
"paths": {
"/api/validate": {
"post": {
"requestBody": {
"content": {
"application/json": {
"schema": {}
}
}
}
}
}
}
},
"data": null,
"governance": {
"dpv:hasProcessing": ["dpv:Use"],
"dpv:hasPurpose": "dpv:Purpose",
"dpv:hasExpiryTime": "6 months"
}
}
```
### Create Service Description
Info:
* Requirement: *sd_4*
* API: `POST <base-uri>/service/`
* requires write permission
* Status: *tested*
**Example:**
<details><summary>Service Description <code>service.json</code></summary>
```json=
{
"interface": {
"info": { "title": "DID Lint" },
"servers": [{"url": "https://didlint.ownyourdata.eu"}],
"party": "data_consumer",
"paths": {
"/api/validate": {
"post": {
"requestBody": {
"content": {
"application/json": {
"schema": {}
}
}
}
}
}
}
},
"data": null,
"governance": {
"dpv:hasProcessing": ["dpv:Use"],
"dpv:hasPurpose": "dpv:Purpose",
"dpv:hasExpiryTime": "6 months"
}
}
```
Comments:
* `interface` follows [Open API Spec v3](https://spec.openapis.org/oas/v3.1.0)
* `data` uses [SOyA (Semantic Overlay Architecture)](https://ownyourdata.github.io/soya) structure names
* `governace` based on [Data Privacy Vocabulary v1](https://w3c.github.io/dpv/dpv/)
</details>
`cat service.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/service`
Response:
```json=
{"name": "DID Lint", "service-id": 1}
```
### Update Service Description
Info:
* Requirement: *sd_5*
* API: `PUT <base-uri>/service/SERVICE_ID`
* requires write permission for a user of the organisation that created the service
* Status: *tested*
**Example:** (`service.json` from above)
`cat service.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/service/1`
Response:
```json=
{"name": "DID Lint", "service-id": 1}
```
### Delete Service Description
Info:
* Requirement: *sd_6*
* API: `DELETE <base-uri>/service/SERVICE_ID`
* requires write permission for a user of the organisation that created the service
* Status: *tested*
**Example:**
`curl -H "Authorization: Bearer $TOKEN" -X DELETE https://babelfish.data-container.net/service/1`
Response:
```json=
{"name": "DID Lint", "service-id": 1}
```
## Organisation and User Accounts
### Read Organisation
Info:
* Requirement: *ou_3*
* API: `GET <base-uri>/organization/ORGANIZATION_ID`
* use `GET <base-uri>/organization/ORGANIZATION_ID/meta` for metadata access
* requires read permission
* Status: *tested*
**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/organization/1`
Response:
```json=
{
"organization-id": 1,
"name": "ACME Inc.",
"description": "information about organisation",
"address": "street number, zip code, city, country"
}
```
### List Organisations
Info:
* API: `GET <base-uri>/organization/`
* requires admin permission
* Status: *tested*
**Example:**
`curl -H "Authorization: Bearer $MASTER_TOKEN" https://babelfish.data-container.net/organization/`
Response:
```json=
[
{"organization-id": 1, "name": "ACME Inc."},
{...}
]
```
### Create Organisation
Info:
* Requirement: *ou_1*
* API: `POST <base-uri>/organization/`
* requires admin permission
* Status: *tested*
**Example:**
<details><summary>Organisation information <code>org.json</code></summary>
```json=
{
"name": "ACME Inc." ,
"description": "information about organisation",
"address": "street number, zip code, city, country"
}
```
</details>
`cat org.json | curl -H "Authorization: Bearer $MASTER_TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/organization/`
Response:
```json=
{"organization-id": 1, "name": "ACME Inc.","admin-user-id":2}
```
### Update Organisation
Info:
* Requirement: *ou_2*
* API: `PUT <base-uri>/organization/ORGANIZATION_ID`
* requires write permission from a user of this organisation
* Status: *tested*
**Example:** (`org.json` from above)
`cat org.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/organization/1`
Response:
```json=
{"organization-id": 1, "name": "ACME Inc."}
```
### Delete Organisation
Info:
* Requirement: *ou_5*
* API: `DELETE <base-uri>/organization/ORGANIZATION_ID`
* requires write permission from a user of this organisation
* Status: *tested*
**Example:**
`curl -H "Authorization: Bearer $TOKEN" -X DELETE https://babelfish.data-container.net/organization/1`
Response:
```json=
{"organization-id": 1, "name": "ACME Inc."}
```
### Retrieve User List for Organisation
Info:
* Requirement: *ou_4*
* API: `GET <base-uri>/organization/ORGANIZATION_ID/list`
* requires read permission from a user of this organisation
* Status: *tested*
**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/organization/1/list`
Response:
```json=
[
{"user-id": 2, "name": "John Doe", "organization-id": 1},
{"user-id": 3, "name": "Homer Simpson", "organization-id": 1}
]
```
### Read User
Info:
* Requirement: *ou_8*
* API: `GET <base-uri>/user/USER_ID`
* use `GET <base-uri>/user/USER_ID/meta` for metadata access
* requires read permission from a user of this organisation
* Status: *tested*
**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/user/2`
Response:
```json=
{
"user-id": 2,
"name": "Johne Doe",
"organization-id": 1
}
```
### Read User Wallet
Info:
* Requirement: *ou_10*
* API: `GET <base-uri>/user/USER_ID/wallet`
* requires read permission from this user
* Status: *tested*
**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/user/1/wallet`
Response:
```json=
{
"user-id": 1,
"oauth": {
"client-id": "123...",
"client-secret": "abc..."
}
"dlt": [
{
"type": "Convex",
"network": "testnet",
"address": 48,
"public-key": "0x82AbBf6EBb20cB21dB02375270b9C2078c2e09e9D7C492be6439c61F23917022",
"balance": 96816794
}
]
}
```
### Create User
Info:
* Requirement: *ou_6*
* API: `POST <base-uri>/user/`
* requires write permission from a user of this organisation
* Status: *tested*
**Example:**
<details><summary>User information <code>user.json</code></summary>
```json=
{
"name": "John Doe",
"email": "john@doe.com",
"phone": "(555) 123 4567",
"organization-id": 1,
}
```
</details>
`cat user.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/user/`
Response:
```json=
{
"user-id": 1,
"name": "John Doe",
"email": "john@doe.com",
"phone": "(555) 123 4567",
"organization-id": 1,
}
```
### Update User
Info:
* Requirement: *ou_7*
* API: `PUT <base-uri>/user/USER_ID`
* requires write permission from a user of this organisation
* Status: *tested*
**Example:** (`user.json` from above)
`cat user.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/user/1`
Response:
```json=
{"user-id": 1, "name": "John Doe"}
```
#### Provide DLT information for a User
Use the following structure to store DLT information for a user
```json
{
"name":"admin",
"meta": {
"dlt": [{
"type": "Convex",
"account":"123",
"seed":"123abc" }]
}
}
```
and use the Update User command (`PUT <base-uri>/user/USER_ID`) to store the data.
### Delete User
Info:
* Requirement: *ou_9*
* API: `DELETE <base-uri>/user/USER_ID`
* requires write permission from a user of this organisation
* Status: *tested*
**Example:**
`curl -H "Authorization: Bearer $TOKEN" -X DELETE https://babelfish.data-container.net/user/1`
Response:
```json=
{"user-id": 1, "name": "John Doe"}
```
## Storage Service
in this HackMD the various storage providers are demonstrated in a walkthrough: https://hackmd.io/l8aMfOrBRNq1qB6ZzQEFpA
### Retrieve Collection List
Info:
* Requirement: *ssr_4*
* API: `GET <base-uri>/collection/list`
* requires read permission from any user of the organisation that created the collection
* Status: *tested*
**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/collection/list`
Response:
```json=
[
{"collection-id": 1, "name": "My Repository"},
{"collection-id": 2, "name": "Repository 2"}
]
```
### Read Collection
Info:
* Requirement: *ssr_3*
* API: `GET <base-uri>/collection/COLLECTION-ID`
* requires read permission from any user of the organisation that created the collection
* Status: *tested*
**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/collection/1`
Response:
```json=
{
"collection-id": 1,
"name": "My Repository".
"storage": {
"type": "Semantic Container",
"url": "https://ontochain.data-container.net" },
"dlt": {
"type": "Convex",
"user-id": 1
}
}
```
### Create Collection
Info:
* Requirement: *ssr_1*
* API: `POST <base-uri>/collection/`
* requires write permission from a user
* Status: *tested*
**Example:**
<details><summary>Collection information <code>col.json</code></summary>
```json=
{
"name": "My Repository"
}
```
</details>
`cat col.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/collection/`
Response:
```json=
{
"collection-id": 1,
"name": "My Repository"
}
```
#### AWS S3 Storage Provider
To use AWS S3 (Amazon Web Services Simple Storage Service) as off-chain storage provider provide the following information when creating a collection:
```json=
{
"name": "My Semantic Container Repository",
"storage": {
"type": "AWS S3",
"service-id": 340 }
}
```
#### Semantic Container Storage Provider
To use Semantic Container as off-chain storage provider provide the following information when creating a collection:
```json=
{
"name": "My Semantic Container Repository",
"storage": {
"type": "Semantic Container",
"service-id": 339,
"url": "https://storage-oc3.data-container.net" }
}
```
#### Bellecour Storage Provider
To use Semantic Container as off-chain storage provider provide the following information when creating a collection:
```json=
{
"name": "My Bellecour Repository",
"dlt": {
"type": "Bellecour",
"service-id": 341 }
}
```
#### Convex Storage Provider
To use Semantic Container as off-chain storage provider provide the following information when creating a collection:
```json=
{
"name": "My Convex Repository",
"dlt": {
"type": "Convex",
"service-id": 342,
"url": "https://storage-oc3.data-container.net" }
}
```
### Update Collection
Info:
* Requirement: *ssr_2*
* API: `PUT <base-uri>/collection/COLLECTION-ID`
* requires write permission from any user of the organisation that created this collection
* Status: *tested*
**Example:** (`col.json` from above)
`cat col.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/collection/1`
Response:
```json=
{
"collection-id": 1,
"name": "My Repository"
}
```
### Delete Collection
Info:
* Requirement: *ssr_5*
* API: `DELETE <base-uri>/collection/COLLECTION-ID`
* requires write permission from any user of the organisation that created this collection
* Status: *tested*
**Example:**
`curl -H "Authorization: Bearer $TOKEN" -X DELETE https://babelfish.data-container.net/collection/1`
Response:
```json=
{
"collection-id": 1,
"name": "My Repository"
}
```
### Check Object Access
Info:
* Requirement: *ssr_12*
* API: `POST <base-uri>/object/OBJECT_ID/USER_ID`
* requires read permission from any user of the organisation that created this object
* Status: *tested*
**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/1/1`
Response:
```json=
{
"object-id": 1,
"collection-id": 1,
"name": "My Object",
"access": true
}
```
### List Objects
Info:
* Requirement: *none*
* API: `GET <base-uri>/collection/COLLECTION-ID/objects`
* requires read permission from any user of the organisation that created this collection
* Status: *implemented*
**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/collection/1/objects`
Response:
```json=
[
{"object-id": 2, "name": "My Object"},
{"object-id": 3, "name": "object2"}
]
```
### Read Object (meta data)
Info:
* Requirement: *ssr_8*
* API: `GET <base-uri>/object/OBJECT_ID`
* requires read permission from any user of the organisation that created this collection
* Status: *tested*
**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/1`
Response:
```json=
{
"object-id": 1,
"collection-id": 1,
"name": "My Object"
}
```
### Read Object (object itself)
Info:
* Requirement: *ssr_10*
* API: `GET <base-uri>/object/OBJECT_ID/read`
* requires read permission from any user of the organisation that created this collection
* Status: *tested*
**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/1/read`
Response:
```json=
["payload"]
```
### Create Object (metadata)
Info:
* Requirement: *ssr_6*
* API: `POST <base-uri>/object/`
* requires write permission from any user of the organisation that created this collection
* Status: *tested*
**Example:**
<details><summary>Object information <code>obj.json</code></summary>
```json=
{
"name": "My Object",
"collection-id": 1
}
```
</details>
`cat obj.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/object/`
Response:
```json=
{
"object-id": 1,
"collection-id": 1,
"name": "My Object"
}
```
### Write Object (object itself)
Info:
* Requirement: *ssr_11*
* API: `PUT <base-uri>/object/OBJECT_ID/write`
* requires write permission from any user of the organisation that created this collection
* Status: *tested*
**Example:**
<details><summary>Payload information <code>payload.json</code></summary>
```json=
["payload"]
```
</details>
`cat payload.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/object/1/write`
Response:
```json=
{
"object-id": 1,
"collection-id": 1
}
```
### Update Object (metadata)
Info:
* Requirement: *ssr_7*
* API: `PUT <base-uri>/object/OBJECT_ID`
* requires write permission from any user of the organisation that created this collection
* Status: *tested*
**Example:** (`obj.json` from above)
`cat obj.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/object/1`
Response:
```json=
{
"object-id": 1,
"collection-id": 1,
"name": "My Object"
}
```
### Delete Object
Info:
* Requirement: *ssr_9*
* API: `DELETE <base-uri>/object/OBJECT_ID`
* requires write permission from any user of the organisation that created this collection
* Status: *tested*
**Example:**
`curl -H "Authorization: Bearer $TOKEN" -X DELETE https://babelfish.data-container.net/object/1`
Response:
```json=
{
"object-id": 1,
"collection-id": 1,
"name": "My Object"
}
```
## Authentication
Most of the API endpoints require a *bearer token* in the HTTP request header. Retrieve this token using the client-id and client-secret for your user - example:
```bash=
export APP_KEY="w76IhupHXHEMk_2ueVwLrKL4Lcw0OKrBYoG9xq3cEFc"
export APP_SECRET="c6XNWeHwmDh8e7yl624Ss4QxrNz6Rxm97R3R3Y7iRqY"
export TOKEN=`curl -s -d grant_type=client_credentials -d client_id=$APP_KEY -d client_secret=$APP_SECRET -d scope=write -X POST http://localhost:3100/oauth/token | jq -r '.access_token'`
```
> Note: the example KEY and SECRET above don't actually yield a token!
Use `TOKEN` in the header:
```bash=
curl -H "Authorization: Bearer $TOKEN" \
https://babelfish.data-container.net/user/2
```
## Further Information
* Requirements document:
https://eurodyn.proofhub.com/link/goproof?8FqSDh-vUd6Si-I7qpxSb-l
* Status
possbile values are:
* *draft* - documented
* *stub* - endpoint accessible and returns example response
* *implemented* - endpoint implemented
* *tested* - automated tests covering all scenarios available
Sign in with Wallet
