# D2A Process

## 0 - Prerequisites

### relevant SOyA Structures
all with Validation & Form Overlay
- [x] Organisation - [`SOyA 'Organisation' YAML`](https://soya.ownyourdata.eu/Organisation/yaml), [SOyA-Form](https://soya-form.ownyourdata.eu/?schemaDri=Organisation&data=%7B%7D)
- [x] Usage Policy - [`SOyA 'SCM_UsagePolicy' YAML`](https://soya.ownyourdata.eu/SCM_UsagePolicy/yaml), [SOyA-Form](https://soya-form.ownyourdata.eu/?schemaDri=SCM_UsagePolicy&data=%7B%7D)
- [x] D2A - [`SOyA 'D2A' YAML`](https://soya.ownyourdata.eu/D2A/yaml), [SOyA-Form](https://soya-form.ownyourdata.eu/?schemaDri=D2A&data=%7B%7D)
- [x] D2A-Template - [`SOyA 'D2A_Template' YAML`](https://soya.ownyourdata.eu/D2A_Template/yaml), [SOyA-Form](https://soya-form.ownyourdata.eu/?schemaDri=D2A_Template&data=%7B%7D)
- [x] HoneyBatch - [`SOyA 'HoneyBatch' YAML`](https://soya.ownyourdata.eu/HoneyBatch/yaml), [SOyA-Form](https://soya-form.ownyourdata.eu/?schemaDri=HoneyBatch&data=%7B%7D)

### Records

- [x] <details><summary>Honey Organisation (incl. Usage Policy for Honey Organisation)</summary>
- [x] input: `org_beekeeper.json`

    {
      "name": "Imkerei Hans Huber",
      "jurisdiction": "Vienna",
      "authority-party": "WKO",
      "location": {
        "address": "Street 1, 1010 Wien"
      },
      "usage-policy": {
        "name": "Beekeeper Usage Policy",
        "description": "default Usage Policy for beekeeper",
        "created_by": "Christoph",
        "created_at": "2023-03-27",
        "purpose": [
          "scdv:green_claim",
          "scdv:date_marking",
          "scdv:supply_chain_optimization"
        ],
        "processing": [
          "scdv:analyse",
          "scdv:query",
          "scdv:store"
        ],
        "recipient": [
          "scdv:authority",
          "scdv:data_space"
        ],
        "storage_location": [
          "scdv:EU"
        ],
        "retention_period": "6 months"
      }
    }

- [x] cmdline GatewayAPI:
- create record:
`cat org_beekeeper.json | curl -H "Authorization: Bearer $MASTER_TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/organization/`
- update record:
`cat org_beekeeper.json | curl -H "Authorization: Bearer $MASTER_TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/organization/60`
- read record:
`curl -H "Authorization: Bearer $MASTER_TOKEN" https://babelfish.data-container.net/organization/60`
- GatewayAPI reference:
`{"organization-id":60,"name":"Imkerei Hans Huber","admin-user-id":61}`
- [ ] Postman
- [x] cmdline to create DID:
- create DID:

      echo '{"service":[
          {"serviceEndpoint":"https://babelfish.data-container.net/api/data?dri=zQmNNauQjQkpxgZrCDH9tBdosiet83dg44f7uBbLupwdjdw"}
        ]}' | oydid create --doc-pwd honey_pwd --rev-pwd honey_pwd2 \
                -s -z 1 -l https://babelfish.data-container.net

- update DID:

      echo '{"service":[
          {"serviceEndpoint":"https://babelfish.data-container.net/api/data?dri=zQmNYQ6uiv19TxiR2KfVvAGqHDvnVFpxaJKyLyerTspbyiU"}
        ]}' | oydid update did:oyd:zQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%40babelfish.data-container.net --doc-pwd honey_pwd --rev-pwd honey_pwd2 --old-doc-pwd honey_pwd --old-rev-pwd honey_pwd2 \
                -s -z 1 -l https://babelfish.data-container.net

- DID: `did:oyd:zQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%40babelfish.data-container.net`

- [x] Postman

      curl -X 'POST' 'https://oydid.ownyourdata.eu/1.0/create?method=oyd' \
        -H 'Content-Type: application/json' \
        -d '{
        "options": {
          "location": "https://babelfish.data-container.net",
          "ts": 1
        },
        "secret": {
          "doc_pwd": "honey_pwd",
          "rev_pwd": "honey_pwd2"
        },
        "didDocument": {"service":[{"serviceEndpoint": "https://babelfish.data-container.net/api/data?dri=zQmNNauQjQkpxgZrCDH9tBdosiet83dg44f7uBbLupwdjdw"}]}
      }'

</details>

- [x] <details><summary>DISP Organisation (w/o Usage Policy)</summary>
- [X] input: `org_disp.json`

    {
      "name": "DISP",
      "jurisdiction": "Vienna",
      "authority-party": "WKO",
      "location": {
        "address": "Street 2, 1010 Wien"
      }
    }

- [x] cmdline GatewayAPI:
- create entry:
`cat org_disp.json | curl -H "Authorization: Bearer $MASTER_TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/organization/`
- update entry:
`cat org_disp.json | curl -H "Authorization: Bearer $MASTER_TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/organization/95`
- GatewayAPI reference: `{"organization-id":95,"name":"DISP","admin-user-id":96}`
- [ ] Postman
- [x] cmdline to create DID:
- create DID

      echo '{"service":[
          {"serviceEndpoint":"https://babelfish.data-container.net/api/data?dri=zQmQHmiJrCtvDsizy9SL92sqppXXuPNar6s8bDtprX4NgZq"}
        ]}' | oydid create --doc-pwd disp_pwd --rev-pwd disp_pwd2 \
                -s -z 1 -l https://babelfish.data-container.net

- update DID

      echo '{"service":[
          {"serviceEndpoint":"https://babelfish.data-container.net/api/data?dri=zQmP6NCKPVv7TQSofdSpNykY7hn9yr9C27f6zzthwWjpSzz"}
        ]}' | oydid update did:oyd:zQmbp9vm7VszVUj1gwGrsvZX5uHiXeFw2jsCbTRwDd5SYxp%40babelfish.data-container.net --doc-pwd disp_pwd --rev-pwd disp_pwd2 --old-doc-pwd disp_pwd --old-rev-pwd disp_pwd2 \
                -s -z 1 -l https://babelfish.data-container.net
  - [ ] Postman
  - DID: `did:oyd:zQmbp9vm7VszVUj1gwGrsvZX5uHiXeFw2jsCbTRwDd5SYxp%40babelfish.data-container.net` 
</details>

- [x] <details><summary>Market Maker (Shop)</summary>
    - create DID  

echo '' | oydid create --doc-pwd show_pwd --rev-pwd shop_pwd2 \
-s -z 1 -l https://babelfish.data-container.net

    - DID: `did:oyd:zQmZzphAYNtaatPHTSFM9TQgxGkUbdLuFNAcsWLYif2WSdH%40babelfish.data-container.net`
</details>

- [x] <details><summary>Honey Storage Service (incl. Usage Policy from DISP for beekeepers)</summary>
  - [x]  input: `service_write_honey.json`

{
"interface": {
"info": {
"title": "Write HoneyBatch",
"description": "this service is part of the Honey supply chain management use case to demonstrate various features of the Gateway API"
},
"servers": [
{
"url": "https://honey-oc3.data-container.net"
}
],
"paths": {
"/api/data/": {
"post": {
"requestBody": {
"content": {
"application/json": {
"schema": {}
}
}
}
}
}
}
},
"data": "HoneyBatch",
"governance": {
"name": "Intermediary Usage Policy for Agricultural Data",
"description": "example usage policy for a data intermediary",
"created_by": "Christoph",
"created_at": "2023-03-27",
"purpose": [
"scdv:green_claim"
],
"processing": [
"scdv:store"
],
"recipient": [
"scdv:authority",
"scdv:data_space"
],
"storage_location": [
"scdv:EU"
],
"retention_period": "6 months"
}
}

  - [x] cmdline GatewayAPI:
    - create entry:
    `cat service_write_honey.json | curl -H "Authorization: Bearer $DISP_TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/service/`
    - update entry:
    `cat service_write_honey.json | curl -H "Authorization: Bearer $DISP_TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/service/97`
    - GatewayAPI:  
      `{"service-id":97,"name":"Write HoneyBatch"}`
  - [ ] Postman
</details>

- [x] <details><summary>D2A Template</summary>
  - [x] Input: d2a_template.json

{
"purposes": [
{
"service": "Honey Production",
"data_owners": [
{
"name": "Imkerei Hans Huber",
"address": "Street 1, A-1010 Vienna",
"party_type": "Ministry of Agriculture",
"organization_id": "did:oyd:[e-mail]"
}
],
"jurisdiction": "Vienna",
"purpose_type": "Beekeeper Data Sharing",
"cii_information": [
{
"sensitivity": "none",
"attribute_id": "https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.beekeeper_did",
"data_optional": false,
"attribute_type": [
"scdv:identity"
]
},
{
"sensitivity": "csi",
"attribute_id": "https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.volume",
"attribute_type": [
"scdv:agricultural"
]
},
{
"sensitivity": "csi",
"attribute_id": "https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.wanderkarte",
"attribute_type": [
"scdv:agricultural"
]
}
],
"data_controllers": [
{
"name": "DISP",
"address": "Street 2, A-1010 Vienna",
"party_type": "Chamber of Commerce",
"organization_id": "did:oyd:[e-mail]"
}
],
"purpose_description": "default ",
"purpose": [
"scdv:green_claim"
],
"processing": [
"scdv:store",
"scdv:query"
],
"recipient": [
"scdv:authority",
"scdv:data_space"
],
"storage_location": [
"scdv:EU"
],
"retention_period": "6 months",
"geographic_restriction": "EEA",
"withdrawal_method": "contact via email",
"authority_party": "Austrian Chamber of Commerce"
}
]
}

  - [x] show & edit D2A Template: [SOyA Form link](https://soya-form.ownyourdata.eu/?schemaDri=D2A_Template&data=%7B%22purposes%22%3A%5B%7B%22service%22%3A%22Honey+Production%22%2C%22data_owners%22%3A%5B%7B%22name%22%3A%22Imkerei+Hans+Huber%22%2C%22address%22%3A%22Street+1%2C+A-1010+Vienna%22%2C%22party_type%22%3A%22Ministry+of+Agriculture%22%2C%22organization_id%22%3A%22did%3Aoyd%3AzQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%40babelfish.data-container.net%22%7D%5D%2C%22jurisdiction%22%3A%22Vienna%22%2C%22purpose_type%22%3A%22Beekeeper+Data+Sharing%22%2C%22cii_information%22%3A%5B%7B%22sensitivity%22%3A%22none%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.beekeeper_did%22%2C%22data_optional%22%3Afalse%2C%22attribute_type%22%3A%5B%22scdv%3Aidentity%22%5D%7D%2C%7B%22sensitivity%22%3A%22csi%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.volume%22%2C%22attribute_type%22%3A%5B%22scdv%3Aagricultural%22%5D%7D%2C%7B%22sensitivity%22%3A%22csi%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.wanderkarte%22%2C%22attribute_type%22%3A%5B%22scdv%3Aagricultural%22%5D%7D%5D%2C%22data_controllers%22%3A%5B%7B%22name%22%3A%22DISP%22%2C%22address%22%3A%22Street+2%2C+A-1010+Vienna%22%2C%22party_type%22%3A%22Chamber+of+Commerce%22%2C%22organization_id%22%3A%22did%3Aoyd%3AzQmbp9vm7VszVUj1gwGrsvZX5uHiXeFw2jsCbTRwDd5SYxp%40babelfish.data-container.net%22%7D%5D%2C%22purpose_description%22%3A%22default+%22%2C%22purpose%22%3A%5B%22scdv%3Agreen_claim%22%5D%2C%22processing%22%3A%5B%22scdv%3Astore%22%2C%22scdv%3Aquery%22%5D%2C%22recipient%22%3A%5B%22scdv%3Aauthority%22%2C%22scdv%3Adata_space%22%5D%2C%22storage_location%22%3A%5B%22scdv%3AEU%22%5D%2C%22retention_period%22%3A%226+months%22%2C%22geographic_restriction%22%3A%22EEA%22%2C%22withdrawal_method%22%3A%22contact+via+email%22%2C%22authority_party%22%3A%22Austrian+Chamber+of+Commerce%22%7D%5D%7D)
  
</details>

- [x] <details><summary>Collection for HoneyBatch data (incl. reference to D2A Template)</summary>
  - [X] input: `collection_honeybatch.json`
    (must include: D2A Template, SOyA Structure)

{
"name": "HoneyBatch",
"storage": {
"type": "Semantic Container",
"url": "https://honey-oc3.data-container.net" },
"schema": "zQmcfpdZVuLJcXUDtqoLnPAHnA4eQNKaX6AcYBttUPaLpC8",
"d2a-template": {
"purposes": [
{
"purpose_description": "this D2A focuses on the Green Claims purpose",
"purpose": [
"scdv:green_claim"
],
"processing": [
"scdv:store",
"scdv:query"
],
"recipient": [
"scdv:authority",
"scdv:data_space"
],
"storage_location": [
"scdv:EU"
],
"cii_information": [
{
"sensitivity": "none",
"attribute_id": "https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.beekeeper_did",
"data_optional": false,
"attribute_type": [
"scdv:identity"
]
},
{
"sensitivity": "csi",
"attribute_id": "https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.volume",
"attribute_type": [
"scdv:agricultural"
]
},
{
"sensitivity": "csi",
"attribute_id": "https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.wanderkarte",
"attribute_type": [
"scdv:agricultural"
]
}
],
"data_controllers": [
{
"name": "DISP",
"address": "Street 2, A-1010 Vienna",
"party_type": "Chamber of Commerce",
"organization_id": "did:oyd:[e-mail]"
}
],
"retention_period": "6 months",
"jurisdiction": "Vienna",
"geographic_restriction": "EEA",
"withdrawal_method": "contact via email",
"authority_party": "Austrian Chamber of Commerce",
"service": "Honey Production"
}
]
}
}

  - [x] cmdline GatewayAPI:
    - create entry:
    `cat collection_honeybatch.json | curl -H "Authorization: Bearer $DISP_TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/collection/`
    - update entry:
    `cat collection_honeybatch.json | curl -H "Authorization: Bearer $DISP_TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/service/98`
    - GatewayAPI:  
      `{"collection-id":98,"name":"HoneyBatch"}`
  - [ ] Postman
</details>

- [ ] <details><summary>Beekeeper User linked to Storage Service</summary>
  - [ ] input: Github
  - [ ] cmdline GatewayAPI:
  - [ ] Postman
  - GatewayAPI: link
  - [ ] cmdline to create DID:
  - [ ] Postman
  - DID:  
</details>

- [x] <details><summary>HoneyBatch dataset</summary>
  - [x] input: [Permalink](https://soya-form.ownyourdata.eu/?viewMode=embedded&schemaDri=zQmTvtiQd5w2BdYv8gncgiKq7cRFbvE29zS4UWqneXNyyt3&data=%7B%22unit%22%3A%22kg%22%2C%22volume%22%3A%225%22%2C%22produced_at%22%3A%222023-03-15%22%2C%22beekeeper_did%22%3A%22did%3Aoyd%3AzQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%2540babelfish.data-container.net%22%7D)
  - [x] Record:

{
"unit": "kg",
"volume": "5",
"produced_at": "2023-03-15",
"beekeeper_did": "did:oyd:zQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%40babelfish.data-container.net",
"meta": {"schema":"HoneyBatch"}
}

</details>


### Snippets for Command Line

- [x] <details><summary>Master Credentials to create Organisation: <code>MASTER_TOKEN</code></summary>

  * Key: `rkgYEN5QXmsFcwbH-awWoo9Q5BJd6vdGoyzRHI0lOZs`
  * Secret: `AoaoR9-Aj7woafUW7hnARWVbNqaBxINiqteyMDMuBoA`

export GW_HOST="https://babelfish.data-container.net"
export APP_KEY="rkgYEN5QXmsFcwbH-awWoo9Q5BJd6vdGoyzRHI0lOZs"
export APP_SECRET="AoaoR9-Aj7woafUW7hnARWVbNqaBxINiqteyMDMuBoA"
export MASTER_TOKEN=`curl -s -d grant_type=client_credentials -d client_id=$APP_KEY -d client_secret=$APP_SECRET -d scope=admin -X POST https://babelfish.data-container.net/oauth/token | jq -r '.access_token'`

</details>

- [x] <details><summary>Beekeeper Credentials: <code>BEE_TOKEN</code></summary>

export GW_HOST="https://babelfish.data-container.net"
export BEE_KEY="jo6Wf_LUYMwFm2Ai9H6exLqhUXU2xUrNh3KLN3H_unY"
export BEE_SECRET="vG0UfHw8tfIqU4cKKNDDs70qlvImSfGWEGJ4WHKASsM"
export BEE_TOKEN=`curl -s -d grant_type=client_credentials -d client_id=$BEE_KEY -d client_secret=$BEE_SECRET -d scope=write -X POST https://babelfish.data-container.net/oauth/token | jq -r '.access_token'`

</details>

- [x] <details><summary>DISP Credentials: <code>DISP_TOKEN</code></summary>

export GW_HOST="https://babelfish.data-container.net"
export DISP_KEY="des1QdHDgS4w9zGLacQs_fuAY_po8iSxVXfKQyvdpDI"
export DISP_SECRET="b2_iytO7JddGIMduDdlfiEgrUKZKEZQa_CntcvDsbeE"
export DISP_TOKEN=`curl -s -d grant_type=client_credentials -d client_id=$DISP_KEY -d client_secret=$DISP_SECRET -d scope=write -X POST https://babelfish.data-container.net/oauth/token | jq -r '.access_token'`

</details>

- [x] <details><summary>Dummy User Credentials: <code>TOKEN</code></summary>

export GW_HOST="https://babelfish.data-container.net"
export KEY="-6H7FHYo4aX5-dYMVF82x2_rzO1cXIB5URc24dPwMls"
export SECRET="cnhNgI77IMVeyenhZnPylcD_XKO72piWzzT68psVKJA"
export TOKEN=`curl -s -d grant_type=client_credentials -d client_id=$KEY -d client_secret=$SECRET -d scope=write -X POST https://babelfish.data-container.net/oauth/token | jq -r '.access_token'`

</details>

- [x] <details><summary>OwnYourData Credentials: <code>TOKEN</code></summary>

export GW_HOST="https://babelfish.data-container.net"
export KEY="YleLTVd6MksqdT5560x3iPYl0gEjnMJNeWtZOaVskE8"
export SECRET="PKzAtcFnETyw_ZPo4TcBLeshzAXj8uVXOhCWfbRSQ6s"
export TOKEN=`curl -s -d grant_type=client_credentials -d client_id=$KEY -d client_secret=$SECRET -d scope=write -X POST https://babelfish.data-container.net/oauth/token | jq -r '.access_token'`

</details>


### Open Tasks
- [x] CFA: complete D2A process documented
- [x] CFA: all SOyA structures available
- [x] CFA: create Usage Policy for Organisation
- [x] Seb: attribute values for SCM_UsagePolicy
- [ ] Seb: edit usage policies for Beekeeper & DISP
- [ ] Seb: edit organisation records of Beekeeper & DISP
- [x] Gabriel: multi-select drop-down for Usage Policies
- [ ] CFA: design & implement process for collections to require D2A and provide D2A Template in case of non-compliance
- [ ] CFA: implement GW_API/integration/match_up
- [ ] CFA: implement GW_API/integration/create_d2a_template
- [ ] CFA: implement GW_API/integration/create_d2a
- [ ] CFA: implement GW_API/integration/extract_up
- [ ] CFA: implement GW_API/integration/match_d2a
- [ ] CFA: implement validation service
- [ ] CFA: full metadata specification & implementation

### Open Questions
- [ ] Jan: should D2A include hash of dataset
    a malicious DISP could otherwise use D2A also for other datasets
- [ ] CFA: where store PKs for DID created by DISP?

### Process Overview

@startuml
participant Beekeeper as bk
participant DISP as disp
autonumber

bk<-disp: Retrieve DISP Context\n(get D2A Template)
bk->bk: check UP compliance
bk->bk: create D2A proposal
bk->disp: write data (incl. D2A proposal)
disp->disp: verify input material
disp->disp: store data + metadata
disp --> bk: response with reference to dataset

Relevant records (and link to SOyA-Form instance):

* [D2A Template](https://soya-form.ownyourdata.eu/?schemaDri=D2A_Template&data=%7B%22purposes%22%3A%5B%7B%22service%22%3A%22Honey+Production%22%2C%22data_owners%22%3A%5B%7B%22name%22%3A%22Imkerei+Hans+Huber%22%2C%22address%22%3A%22Street+1%2C+A-1010+Vienna%22%2C%22party_type%22%3A%22Ministry+of+Agriculture%22%2C%22organization_id%22%3A%22did%3Aoyd%3AzQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%40babelfish.data-container.net%22%7D%5D%2C%22jurisdiction%22%3A%22Vienna%22%2C%22purpose_type%22%3A%22Beekeeper+Data+Sharing%22%2C%22cii_information%22%3A%5B%7B%22sensitivity%22%3A%22none%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.beekeeper_did%22%2C%22data_optional%22%3Afalse%2C%22attribute_type%22%3A%5B%22scdv%3Aidentity%22%5D%7D%2C%7B%22sensitivity%22%3A%22csi%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.volume%22%2C%22attribute_type%22%3A%5B%22scdv%3Aagricultural%22%5D%7D%2C%7B%22sensitivity%22%3A%22csi%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.wanderkarte%22%2C%22attribute_type%22%3A%5B%22scdv%3Aagricultural%22%5D%7D%5D%2C%22data_controllers%22%3A%5B%7B%22name%22%3A%22DISP%22%2C%22address%22%3A%22Street+2%2C+A-1010+Vienna%22%2C%22party_type%22%3A%22Chamber+of+Commerce%22%2C%22organization_id%22%3A%22did%3Aoyd%3AzQmbp9vm7VszVUj1gwGrsvZX5uHiXeFw2jsCbTRwDd5SYxp%40babelfish.data-container.net%22%7D%5D%2C%22purpose_description%22%3A%22default+%22%2C%22purpose%22%3A%5B%22scdv%3Agreen_claim%22%5D%2C%22processing%22%3A%5B%22scdv%3Astore%22%2C%22scdv%3Aquery%22%5D%2C%22recipient%22%3A%5B%22scdv%3Aauthority%22%2C%22scdv%3Adata_space%22%5D%2C%22storage_location%22%3A%5B%22scdv%3AEU%22%5D%2C%22retention_period%22%3A%226+months%22%2C%22geographic_restriction%22%3A%22EEA%22%2C%22withdrawal_method%22%3A%22contact+via+email%22%2C%22authority_party%22%3A%22Austrian+Chamber+of+Commerce%22%7D%5D%7D)
* [D2A Proposal](https://soya-form.ownyourdata.eu/?schemaDri=D2A&data=%7B%22purposes%22%3A%5B%7B%22purpose_description%22%3A%22green+claim+for+honey+batch%22%2C%22purpose%22%3A%5B%22scdv%3Agreen_claim%22%5D%2C%22processing%22%3A%5B%22scdv%3Aquery%22%2C%22scdv%3Astore%22%5D%2C%22recipient%22%3A%5B%22scdv%3Adata_space%22%2C%22scdv%3Aauthority%22%5D%2C%22storage_location%22%3A%5B%22scdv%3AEU%22%5D%2C%22retention_period%22%3A%22P0Y6M0DT0H0M0S%22%2C%22service%22%3A%22Honey+Production%22%2C%22geographic_restriction%22%3A%22EEA%22%2C%22jurisdiction%22%3A%22Vienna%22%2C%22withdrawal_method%22%3A%22contact+via+email+office%40babelfish.org%22%2C%22authority_party%22%3A%22Austrian+Chamber+of+Commerce%22%2C%22cii_information%22%3A%5B%7B%22sensitivity%22%3A%22none%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.beekeeper_did%22%2C%22data_optional%22%3Afalse%2C%22attribute_type%22%3A%5B%22scdv%3Aidentity%22%5D%7D%2C%7B%22sensitivity%22%3A%22csi%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.volume%22%2C%22attribute_type%22%3A%5B%22scdv%3Aagricultural%22%5D%7D%2C%7B%22sensitivity%22%3A%22csi%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.wanderkarte%22%2C%22attribute_type%22%3A%5B%22scdv%3Aagricultural%22%5D%2C%22data_optional%22%3Atrue%7D%5D%2C%22data_controllers%22%3A%5B%7B%22name%22%3A%22DISP%22%2C%22address%22%3A%22Street+2%2C+A-1010+Vienna%22%2C%22party_type%22%3A%22Chamber+of+Commerce%22%2C%22organization_id%22%3A%22did%3Aoyd%3AzQmbp9vm7VszVUj1gwGrsvZX5uHiXeFw2jsCbTRwDd5SYxp%40babelfish.data-container.net%22%7D%5D%2C%22data_owners%22%3A%5B%7B%22name%22%3A%22Imkerei+Hans+Huber%22%2C%22address%22%3A%22Street+1%2C+A-1010+Vienna%22%2C%22party_type%22%3A%22WKO%22%2C%22organization_id%22%3A%22did%3Aoyd%3AzQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%40babelfish.data-container.net%22%7D%5D%7D%5D%7D)
* [Honey batch data set](https://soya-form.ownyourdata.eu/?viewMode=embedded&schemaDri=zQmTvtiQd5w2BdYv8gncgiKq7cRFbvE29zS4UWqneXNyyt3&data=%7B%22unit%22%3A%22kg%22%2C%22volume%22%3A%225%22%2C%22produced_at%22%3A%222023-03-15%22%2C%22beekeeper_did%22%3A%22did%3Aoyd%3AzQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%2540babelfish.data-container.net%22%7D)
* object-id at Babelfish: 106  
  `curl -s -H "Authorization: Bearer $DISP_TOKEN" https://babelfish.data-container.net/object/106/meta | jq`

![](https://i.imgur.com/Lfi2jBc.png)


## 1- Retrieve DISP Context

1. <details><summary>Query Service Catalogue</summary>

    - [x] cmdline GatewayAPI
          `curl "https://babelfish.data-container.net/service/search?name=honey"`  
          `curl https://babelfish.data-container.net/service/97`
    - [ ] Postman
</details>

  -> Usage Policy from DISP available
  -> API endpoint for Collection available

2. <details><summary>Write HoneyBatch to DISP</summary>

    - [ ] cmdline GatewayAPI
          `echo HoneyBatch.json | curl -H "Authorization: Bearer $BEE_TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/object/`
    - [ ] Postman
</details>

  -> D2A Template available

## 2 - Check compliance of Usage Policies

1. <details><summary>Show Beekeeper Usage Policy</summary>
  
    - [X] Link to SOyA Form: [Permalink](https://soya-form.ownyourdata.eu/?schemaDri=SCM_UsagePolicy&data=%7B%22name%22%3A%22Beekeeper+Usage+Policy%22%2C%22description%22%3A%22default+Usage+Policy+for+beekeeper%22%2C%22created_by%22%3A%22Christoph%22%2C%22created_at%22%3A%222023-03-27%22%2C%22purpose%22%3A%5B%22scdv%3Agreen_claim%22%2C%22scdv%3Adate_marking%22%2C%22scdv%3Asupply_chain_optimization%22%5D%2C%22processing%22%3A%5B%22scdv%3Aanalyse%22%2C%22scdv%3Aquery%22%2C%22scdv%3Astore%22%5D%2C%22recipient%22%3A%5B%22scdv%3Aauthority%22%2C%22scdv%3Adata_space%22%5D%2C%22storage_location%22%3A%5B%22scdv%3AEU%22%5D%2C%22retention_period%22%3A%226+months%22%7D)
</details>

2. <details><summary>Show DISP Usage Policy</summary>
  
    - [X] Link to SOyA Form: [Permalink](https://soya-form.ownyourdata.eu/?schemaDri=SCM_UsagePolicy&data=%7B%22name%22%3A%22Intermediary+Usage+Policy+for+Agricultural+Data%22%2C%22description%22%3A%22example+usage+policy+for+a+data+intermediary%22%2C%22created_by%22%3A%22Christoph%22%2C%22created_at%22%3A%222023-03-27%22%2C%22purpose%22%3A%5B%22scdv%3Agreen_claim%22%5D%2C%22processing%22%3A%5B%22scdv%3Astore%22%5D%2C%22recipient%22%3A%5B%22scdv%3Aauthority%22%2C%22scdv%3Adata_space%22%5D%2C%22storage_location%22%3A%5B%22scdv%3AEU%22%5D%2C%22retention_period%22%3A%226+months%22%7D)
</details>

3. <details><summary>Compare Usage Policies</summary>
  
    - [ ] cmdline for Usage Policy compliance service:
          `cat '{"provider":{["provider up"]}, "consumer":{["consumer up"]}}' | \`
`curl -H "Authorization: Bearer $BEE_TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/integration/match_up`
    - [ ] Postman
    - [ ] show result from Usage Policy comparison
    - if Usage Policies are not compliant: edit Beekeer Usage Policy and repeat step 3.
  
</details>

-> DISP compliant Usage Policy of Beekeeper available


## 3 - Create D2A Proposal

1. <details><summary>Use Beekeeper Usage Policy to create D2A Proposal</summary>
  
    - [ ] cmdline for Usage Policy compliance service:
          `cat '{"usage-policy":{["beekeeper usage policy"]}, "data-owner":{["beekeeper organisation record"]} "d2a-template":{["D2A template"]}}' | \`
`curl -H "Authorization: Bearer $BEE_TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/integration/create_d2a`
    - [ ] Postman
    - [ ] show D2A Proposal, [link to SOyA Form](https://soya-form.ownyourdata.eu/?schemaDri=D2A&data=%7B%22purposes%22%3A%5B%7B%22purpose_description%22%3A%22green+claim+for+honey+batch%22%2C%22purpose%22%3A%5B%22scdv%3Agreen_claim%22%5D%2C%22processing%22%3A%5B%22scdv%3Aquery%22%2C%22scdv%3Astore%22%5D%2C%22recipient%22%3A%5B%22scdv%3Adata_space%22%2C%22scdv%3Aauthority%22%5D%2C%22storage_location%22%3A%5B%22scdv%3AEU%22%5D%2C%22retention_period%22%3A%22P0Y6M0DT0H0M0S%22%2C%22service%22%3A%22Honey+Production%22%2C%22geographic_restriction%22%3A%22EEA%22%2C%22jurisdiction%22%3A%22Vienna%22%2C%22withdrawal_method%22%3A%22contact+via+email+office%40babelfish.org%22%2C%22authority_party%22%3A%22Austrian+Chamber+of+Commerce%22%2C%22cii_information%22%3A%5B%7B%22sensitivity%22%3A%22none%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.beekeeper_did%22%2C%22data_optional%22%3Afalse%2C%22attribute_type%22%3A%5B%22scdv%3Aidentity%22%5D%7D%2C%7B%22sensitivity%22%3A%22csi%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.volume%22%2C%22attribute_type%22%3A%5B%22scdv%3Aagricultural%22%5D%7D%2C%7B%22sensitivity%22%3A%22csi%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.wanderkarte%22%2C%22attribute_type%22%3A%5B%22scdv%3Aagricultural%22%5D%2C%22data_optional%22%3Atrue%7D%5D%2C%22data_controllers%22%3A%5B%7B%22name%22%3A%22DISP%22%2C%22address%22%3A%22Street+2%2C+A-1010+Vienna%22%2C%22party_type%22%3A%22Chamber+of+Commerce%22%2C%22organization_id%22%3A%22did%3Aoyd%3AzQmbp9vm7VszVUj1gwGrsvZX5uHiXeFw2jsCbTRwDd5SYxp%40babelfish.data-container.net%22%7D%5D%2C%22data_owners%22%3A%5B%7B%22name%22%3A%22Imkerei+Hans+Huber%22%2C%22address%22%3A%22Street+1%2C+A-1010+Vienna%22%2C%22party_type%22%3A%22WKO%22%2C%22organization_id%22%3A%22did%3Aoyd%3AzQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%40babelfish.data-container.net%22%7D%5D%7D%5D%7D)
</details>

-> D2A Proposal available

2. <details><summary>Create VC Signature for D2A Proposal</summary>
  
    - [ ] cmdline to create signature

TBD

    - [ ] Postman
</details>

-> D2A Proposal Signature available


## 4 - Send HoneyBatch data to DISP (incl. D2A Proposal)

1. <details><summary>create Object in Collection</summary>

      - [x] cmdline:

cat '{"collection-id": $HONEY_COLLECTION}' | \
curl -H "Authorization: Bearer $BEE_TOKEN" -H "Content-Type: application/json" -d @- \
-X POST $HONEY_COLLECTION_ENDPOINT

    Response: `OBJECT_ID`
      - [ ] Postman  
  
</details>

2. <details><summary>send data</summary>

      - [ ]  cmdline:

cat '{
"data": {["HoneyBatch data"]},
"meta": {
"d2a": {["D2A Proposal"]},
"d2a-signature": {["D2A Signature"]}
}
}' | \
curl -H "Authorization: Bearer $BEE_TOKEN" -H "Content-Type: application/json" -d @- \
-X POST https://babelfish.data-container.net/object/OBJECT_ID/write

      - [ ] Postman  
  
</details>


## 5 - DISP verifies input data

The following verification steps are performed:
1. <details><summary>Usage Policy from D2A proposal matches with DISP Usage Policy</summary>

    - [ ] extract Usage Policy from D2A proposal in metadata
      `POST /integration/extract_up`
    - [ ] match provided Usage Policy from Beekeeper with Usage Policy from DISP
      `POST GATEWAY_API/integration/match_up`
  
    Response in case of failed validation: "the Usage Policy provided is non-compliant" and compliance report from Usage Policy comparison
  
</details>

2. <details><summary>D2A proposal conforms to D2A Template in requested collection</summary>

    - [ ] check D2A proposal conforms to specified D2A Template
      `POST GATEWAY_API/integration/match_d2a`
  
    Response in case of failed validation: "the Data Agreement provided is non-compliant" and compliance report from D2A comparison
  
</details>

3. <details><summary>provided dataset conforms to specified SOyA structure</summary>

    - [ ] validate that provided data matches SOyA structure specified for collection
      `POST VALIDATION_SERVICE/validate/{DRI}`
  
    Response in case of failed validation: "the data provided is non-compliant with the expected structure" and compliance report from validation service
  
</details>

4. <details><summary>valid signature from Beekeeper to create VC</summary>

    - [ ] validate that provided signature can be verified with public key from Beekeeper
      `POST OYDID_REPOSITORY/check_signature`
  
    Response in case of failed validation: "the signature provided is non-compliant"
  
</details>



## 6 - DISP stores input data

1. <details><summary>Creat DISP VC Signature for D2A</summary>

    - [ ] create & sign Credential Subject object
  
</details>

2. <details><summary>Create Verifiable Credential</summary>

    - [ ] collect all pieces and compile VC
    - [ ] create VP
  
</details>

3. <details><summary>Create Metadata section</summary>

    - [ ] build metadata section with reference to VP of D2A, schema, and provenance record
  
</details>

5. <details><summary>Write data</summary>

    - [ ] store new object with data (HoneyBatch) and metadata
  
</details>

6. <details><summary>Create DID for HoneyBatch data</summary>

    - [ ] create DID Document for HoneyBatch data that provides service endpoints for data (HoneyBatch) and associated D2A as VP
  
</details>


## 7 - DISP responds with DID for HoneyBatch data

1. Responsed with DID for POST request in section 4 step 2