# Gateway API Endpoints and Data Structures

### Content
* **[Service Discovery](#Service-Discovery1)**
* [Retrieve Service Catalogue List](#Retrieve-Service-Catalogue-List)
* [Query Service Catalogue List](#Query-Service-Catalogue-List)
* [Read Service Description](#Read-Service-Description)
* [Create Service Description](#Create-Service-Description)
* [Update Service Description](#Update-Service-Description)
* [Delete Service Description](#Delete-Service-Description)
* **[Organisation and User Accounts](#Organisation-and-User-Accounts1)**
* [Read Organisation](#Read-Organisation)
* [List all Organisations](#List-Organisations)
* [Create Organisation](#Create-Organisation)
* [Update Organisation](#Update-Organisation)
* [Delete Organisation](#Delete-Organisation)
* [Retrieve User List for Organisation](#Retrieve-User-List-for-Organisation)
* [Read User](#Read-User)
* [Read User Wallet](#Read-User-Wallet)
* [Create User](#Create-User)
* [Update User](#Update-User)
* [Delete User](#Delete-User)
* **[Storage Services](#Storage-Service1)**
* [Retrieve Collection List](#Retrieve-Collection-List)
* [Read Collection](#Read-Collection)
* [Create Collection](#Create-Collection)
* [Update Collection](#Update-Collection)
* [Delete Collection](#Delete-Collection)
* [Check Object Access](#Check-Object-Acces)
* [Read Object (meta data)](#Read-Object-meta-data)
* [Read Object (object itself)](#Read-Object-object-itself)
* [Create Object (meta data)](#Create-Object-meta-data)
* [Write Object (object itself)](#Write-Object-object-itself)
* [Update Object (meta data)](#Update-Object-meta-data)
* [Delete Object](#Delete-Object)
* **[Integration Helper](#Integration-Helper)**
* [Match Usage Policies](#Match-Usage-Policies)
* [Match D2A](#Match-D2A)
* [Create D2A](#Create-D2A)
* [Extract Usage Policy](#Extract-Usage-Policy)
* [Authentication](#Authentication)
* [Further Information](#Further-Information)

## Status

last update: June 13, 2023
#### [Service Discovery](#Service-Discovery1)

| tested | implemented | stub | draft |
| ------:| -----------:| ----:| -----:|
| 6 | 0 | 0 | 0 |

#### [Organisation and User Accounts](#Organisation-and-User-Accounts1)

| tested | implemented | stub | draft |
| ------:| -----------:| ----:| -----:|
| 11 | 0 | 0 | 0 |

#### [Storage Service](#Storage-Service1)

| tested | implemented | stub | draft |
| ------:| -----------:| ----:| -----:|
| 13 | 0 | 0 | 0 |

## Service Discovery

### Retrieve Service Catalogue List

Info:
* Requirement: *sd_1*
* API: `GET /list?filter=value`
available paging parameters are ([RFC-8288](https://tools.ietf.org/html/rfc8288) compliant http response headers)
* page: page number
* items: number of items per page
* sort: field name to sort results, either 'service-id' (default) or 'name'
* publicly available
* Status: *tested*

**Examples:**
`curl https://babelfish.data-container.net/list`
`curl https://babelfish.data-container.net/list?page=2&items=15&sort=name`

Response:

[
  {"service-id": 1, "name": "DID Lint"},
  {"service-id": 2, "name": "xyz"}
]

### Query Service Catalogue List
Info:
* Requirement: *sd_2*
* API: `GET /service/search?field1=value1&field2=value2`
* publicly available
* Status: *tested*

The field names can be set to any of the fields listed under service description:
* title
* url
* party
* data

**Example:**
`curl https://babelfish.data-container.net/service/search?title=did%20lint`

Response:

[
  {"service-id": 1, "name": "DID Lint"}
]

### Read Service Description
Info:
* Requirement: *sd_3*
* API: `GET <base-uri>/service/SERVICE_ID`
* publicly available
* Status: *tested*

**Example:**
`curl https://babelfish.data-container.net/service/1`

Response:

{
  "service-id": 1,
  "interface": {
    "info": { "title": "DID Lint" },
    "servers": [{"url": "https://didlint.ownyourdata.eu"}],
    "party": "data_consumer",
    "paths": {
      "/api/validate": {
        "post": {
          "requestBody": {
            "content": {
              "application/json": {
                "schema": {} 
              }
            }
          }
        }
      }
    }
  },
  "data": null,
  "governance": {
    "dpv:hasProcessing": ["dpv:Use"],
    "dpv:hasPurpose": "dpv:Purpose",
    "dpv:hasExpiryTime": "6 months"
  }
}

### Create Service Description
Info:
* Requirement: *sd_4*
* API: `POST <base-uri>/service/`
* requires write permission
* Status: *tested*

**Example:**
<details><summary>Service Description <code>service.json</code></summary>

{
  "interface": {
    "info": { "title": "DID Lint" },
    "servers": [{"url": "https://didlint.ownyourdata.eu"}],
    "party": "data_consumer",
    "paths": {
      "/api/validate": {
        "post": {
          "requestBody": {
            "content": {
              "application/json": {
                "schema": {} 
              }
            }
          }
        }
      }
    }
  },
  "data": null,
  "governance": {
    "dpv:hasProcessing": ["dpv:Use"],
    "dpv:hasPurpose": "dpv:Purpose",
    "dpv:hasExpiryTime": "6 months"
  }
}

Comments:
* `interface` follows [Open API Spec v3](https://spec.openapis.org/oas/v3.1.0)
* `data` uses [SOyA (Semantic Overlay Architecture)](https://ownyourdata.github.io/soya) structure names
* `governace` based on [Data Privacy Vocabulary v1](https://w3c.github.io/dpv/dpv/)

</details>

`cat service.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/service`

Response:

{"name": "DID Lint", "service-id": 1}

### Update Service Description
Info:
* Requirement: *sd_5*
* API: `PUT <base-uri>/service/SERVICE_ID`
* requires write permission for a user of the organisation that created the service
* Status: *tested*

**Example:** (`service.json` from above)
`cat service.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/service/1`

Response:

{"name": "DID Lint", "service-id": 1}

### Delete Service Description
Info:
* Requirement: *sd_6*
* API: `DELETE <base-uri>/service/SERVICE_ID`
* requires write permission for a user of the organisation that created the service
* Status: *tested*

**Example:**
`curl -H "Authorization: Bearer $TOKEN" -X DELETE https://babelfish.data-container.net/service/1`

Response:

{"name": "DID Lint", "service-id": 1}

## Organisation and User Accounts

### Read Organisation
Info:
* Requirement: *ou_3*
* API: `GET <base-uri>/organization/ORGANIZATION_ID`
* use `GET <base-uri>/organization/ORGANIZATION_ID/meta` for metadata access
* requires read permission
* Status: *tested*

**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/organization/1`

Response:

{
  "organization-id": 1,
  "name": "ACME Inc.",
  "description": "information about organisation",
  "address": "street number, zip code, city, country"
}

### List Organisations
Info:
* API: `GET <base-uri>/organization/`
* requires admin permission
* Status: *tested*

**Example:**
`curl -H "Authorization: Bearer $MASTER_TOKEN" https://babelfish.data-container.net/organization/`

Response:

[
  {"organization-id": 1, "name": "ACME Inc."},
  {...}
]

### Create Organisation
Info:
* Requirement: *ou_1*
* API: `POST <base-uri>/organization/`
* requires admin permission
* Status: *tested*

**Example:**
<details><summary>Organisation information <code>org.json</code></summary>

{
  "name": "ACME Inc." ,
  "description": "information about organisation",
  "address": "street number, zip code, city, country" 
}       

</details>

`cat org.json | curl -H "Authorization: Bearer $MASTER_TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/organization/`

Response:

{"organization-id": 1, "name": "ACME Inc.","admin-user-id":2}

### Update Organisation
Info:
* Requirement: *ou_2*
* API: `PUT <base-uri>/organization/ORGANIZATION_ID`
* requires write permission from a user of this organisation
* Status: *tested*

**Example:** (`org.json` from above)
`cat org.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/organization/1`

Response:

{"organization-id": 1, "name": "ACME Inc."}

### Delete Organisation
Info:
* Requirement: *ou_5*
* API: `DELETE <base-uri>/organization/ORGANIZATION_ID`
* requires write permission from a user of this organisation
* Status: *tested*

**Example:**
`curl -H "Authorization: Bearer $TOKEN" -X DELETE https://babelfish.data-container.net/organization/1`

Response:

{"organization-id": 1, "name": "ACME Inc."}

### Retrieve User List for Organisation
Info:
* Requirement: *ou_4*
* API: `GET <base-uri>/organization/ORGANIZATION_ID/list`
* requires read permission from a user of this organisation
* Status: *tested*

**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/organization/1/list`

Response:

[
  {"user-id": 2, "name": "John Doe", "organization-id": 1},
  {"user-id": 3, "name": "Homer Simpson", "organization-id": 1}
]

### Read User
Info:
* Requirement: *ou_8*
* API: `GET <base-uri>/user/USER_ID`
* use `GET <base-uri>/user/USER_ID/meta` for metadata access
* requires read permission from a user of this organisation
* Status: *tested*

**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/user/2`

Response:

{
  "user-id": 2,
  "name": "Johne Doe",
  "organization-id": 1
}

### Read User Wallet
Info:
* Requirement: *ou_10*
* API: `GET <base-uri>/user/USER_ID/wallet`
* requires read permission from this user
* Status: *tested*

**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/user/1/wallet`

Response:

{
  "user-id": 1,
  "oauth": {
    "client-id": "123...",
    "client-secret": "abc...",
    "scopes": "read write"
  }
  "dlt": [
    {
      "type": "Convex",
      "network": "testnet",
      "address": 48,
      "public-key": "0x82AbBf6EBb20cB21dB02375270b9C2078c2e09e9D7C492be6439c61F23917022",
      "balance": 96816794
    }
  ]
}

### Create User
Info:
* Requirement: *ou_6*
* API: `POST <base-uri>/user/`
* requires write permission from a user of this organisation
* Status: *tested*

**Example:**
<details><summary>User information <code>user.json</code></summary>

{
  "name": "John Doe",
  "email": "[e-mail]",
  "phone": "[phone]",
  "organization-id": 1,
  "meta": {
      "scope": "read"  
  }
}

</details>

`cat user.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/user/`

Response:

{
  "user-id": 1,
  "name": "John Doe", 
  "email": "[e-mail]",
  "phone": "[phone]",
  "organization-id": 1
}

### Update User
Info:
* Requirement: *ou_7*
* API: `PUT <base-uri>/user/USER_ID`
* requires write permission from a user of this organisation
* Status: *tested*

**Example:** (`user.json` from above)

`cat user.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/user/1`

Response:

{"user-id": 1, "name": "John Doe"}

#### Provide DLT information for a User

Use the following structure to store DLT information for a user

{
    "name":"admin", 
    "meta": {
        "dlt": [{ 
            "type": "Convex", 
            "account":"123",
            "seed":"123abc" }]
    }
}

and use the Update User command (`PUT <base-uri>/user/USER_ID`) to store the data.

### Delete User
Info:
* Requirement: *ou_9*
* API: `DELETE <base-uri>/user/USER_ID`
* requires write permission from a user of this organisation
* Status: *tested*

**Example:**
`curl -H "Authorization: Bearer $TOKEN" -X DELETE https://babelfish.data-container.net/user/1`

Response:

{"user-id": 1, "name": "John Doe"}

## Storage Service

in this HackMD the various storage providers are demonstrated in a walkthrough: https://hackmd.io/l8aMfOrBRNq1qB6ZzQEFpA

### Retrieve Collection List
Info:
* Requirement: *ssr_4*
* API: `GET <base-uri>/collection/list`
* requires read permission from any user of the organisation that created the collection
* Status: *tested*

**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/collection/list`

Response:

[
  {"collection-id": 1, "name": "My Repository"},
  {"collection-id": 2, "name": "Repository 2"}
]

### Read Collection
Info:
* Requirement: *ssr_3*
* API: `GET <base-uri>/collection/COLLECTION-ID`
* requires read permission from any user of the organisation that created the collection
* Status: *tested*

**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/collection/1`

Response:

{
  "collection-id": 1,
  "name": "My Repository",
  "storage": {
    "type": "Semantic Container",
    "url": "https://ontochain.data-container.net" },
  "dlt": {
    "type": "Convex",
    "user-id": 1
  }
}

### Create Collection
Info:
* Requirement: *ssr_1*
* API: `POST <base-uri>/collection/`
* requires write permission from a user
* Status: *tested*

**Example:**
<details><summary>Collection information <code>col.json</code></summary>

{
  "name": "My Repository"
}       

</details>

`cat col.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/collection/`

Response:

{
  "collection-id": 1,
  "name": "My Repository"
}

#### AWS S3 Storage Provider

To use AWS S3 (Amazon Web Services Simple Storage Service) as off-chain storage provider provide the following information when creating a collection:

{
  "name": "My Semantic Container Repository",
  "storage": {
    "type": "AWS S3",
    "service-id": 340 }
}       

#### Semantic Container Storage Provider

To use Semantic Container as off-chain storage provider provide the following information when creating a collection:

{
  "name": "My Semantic Container Repository",
  "storage": {
    "type": "Semantic Container",
    "service-id": 339,
    "url": "https://storage-oc3.data-container.net" }
}       

#### Bellecour Storage Provider

To use Semantic Container as off-chain storage provider provide the following information when creating a collection:

{
  "name": "My Bellecour Repository",
  "dlt": {
    "type": "Bellecour",
    "service-id": 341 }
}       

#### Convex Storage Provider

To use Semantic Container as off-chain storage provider provide the following information when creating a collection:

{
  "name": "My Convex Repository",
  "dlt": {
    "type": "Convex",
    "service-id": 342,
    "url": "https://storage-oc3.data-container.net" }
}       

### Update Collection
Info:
* Requirement: *ssr_2*
* API: `PUT <base-uri>/collection/COLLECTION-ID`
* requires write permission from any user of the organisation that created this collection
* Status: *tested*

**Example:** (`col.json` from above)

`cat col.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/collection/1`

Response:

{
  "collection-id": 1,
  "name": "My Repository"
}

### Delete Collection
Info:
* Requirement: *ssr_5*
* API: `DELETE <base-uri>/collection/COLLECTION-ID`
* requires write permission from any user of the organisation that created this collection
* Status: *tested*

**Example:**

`curl -H "Authorization: Bearer $TOKEN" -X DELETE https://babelfish.data-container.net/collection/1`

Response:

{
  "collection-id": 1,
  "name": "My Repository"
}

### Check Object Access
Info:
* Requirement: *ssr_12*
* API: `POST <base-uri>/object/OBJECT_ID/USER_ID`
* requires read permission from any user of the organisation that created this object
* Status: *tested*

**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/1/1`

Response:

{
  "object-id": 1,
  "collection-id": 1,
  "name": "My Object",
  "access": true
}

### List Objects
Info:
* Requirement: *none*
* API: `GET <base-uri>/collection/COLLECTION-ID/objects`
* requires read permission from any user of the organisation that created this collection
* Status: *implemented*

**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/collection/1/objects`

Response:

[
  {"object-id": 2, "name": "My Object"},
  {"object-id": 3, "name": "object2"}
]

### Read Object (meta data)
Info:
* Requirement: *ssr_8*
* API: `GET <base-uri>/object/OBJECT_ID`
* requires read permission from any user of the organisation that created this collection
* Status: *tested*

**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/1`

Response:

{
  "object-id": 1,
  "collection-id": 1,
  "name": "My Object"
}

### Read Object (object itself)
Info:
* Requirement: *ssr_10*
* API: `GET <base-uri>/object/OBJECT_ID/read`
* requires read permission from any user of the organisation that created this collection
* Status: *tested*

**Example:**
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/1/read`

Response:

["payload"]

### Create Object (metadata)
Info:
* Requirement: *ssr_6*
* API: `POST <base-uri>/object/`
* requires write permission from any user of the organisation that created this collection
* Status: *tested*

**Example:**
<details><summary>Object information <code>obj.json</code></summary>

{
  "name": "My Object",
  "collection-id": 1
}

</details>

`cat obj.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/object/`

Response:

{
  "object-id": 1,
  "collection-id": 1,
  "name": "My Object"
}

### Write Object (object itself)
Info:
* Requirement: *ssr_11*
* API: `PUT <base-uri>/object/OBJECT_ID/write`
* requires write permission from any user of the organisation that created this collection
* Status: *tested*

**Example:**
<details><summary>Payload information <code>payload.json</code></summary>

["payload"]      

</details>

`cat payload.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/object/1/write`

Response:

{
  "object-id": 1,
  "collection-id": 1
}

### Update Object (metadata)
Info:
* Requirement: *ssr_7*
* API: `PUT <base-uri>/object/OBJECT_ID`
* requires write permission from any user of the organisation that created this collection
* Status: *tested*

**Example:** (`obj.json` from above)

`cat obj.json | curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/object/1`

Response:

{
  "object-id": 1,
  "collection-id": 1,
  "name": "My Object"
}

### Delete Object
Info:
* Requirement: *ssr_9*
* API: `DELETE <base-uri>/object/OBJECT_ID`
* requires write permission from any user of the organisation that created this collection
* Status: *tested*

**Example:**

`curl -H "Authorization: Bearer $TOKEN" -X DELETE https://babelfish.data-container.net/object/1`

Response:

{
  "object-id": 1,
  "collection-id": 1,
  "name": "My Object"
}

## Authentication

Most of the API endpoints require a *bearer token* in the HTTP request header. Retrieve this token using the client-id and client-secret for your user - example:

export APP_KEY="w76IhupHXHEMk_2ueVwLrKL4Lcw0OKrBYoG9xq3cEFc"
export APP_SECRET="c6XNWeHwmDh8e7yl624Ss4QxrNz6Rxm97R3R3Y7iRqY"
export TOKEN=`curl -s -d grant_type=client_credentials -d client_id=$APP_KEY -d client_secret=$APP_SECRET -d scope=write -X POST http://localhost:3100/oauth/token | jq -r '.access_token'`

> Note: the example KEY and SECRET above don't actually yield a token!

Use `TOKEN` in the header:

curl -H "Authorization: Bearer $TOKEN" \
    https://babelfish.data-container.net/user/2

## Further Information
* Requirements document:
https://eurodyn.proofhub.com/link/goproof?8FqSDh-vUd6Si-I7qpxSb-l
* Status
possbile values are:
* *draft* - documented
* *stub* - endpoint accessible and returns example response
* *implemented* - endpoint implemented
* *tested* - automated tests covering all scenarios available