---
# System prepended metadata

title: D2A Process

---

# D2A Process

## 0 - Prerequisites

### relevant SOyA Structures
all with Validation & Form Overlay
- [x] Organisation - [`SOyA 'Organisation' YAML`](https://soya.ownyourdata.eu/Organisation/yaml), [SOyA-Form](https://soya-form.ownyourdata.eu/?schemaDri=Organisation&data=%7B%7D)
- [x] Usage Policy - [`SOyA 'SCM_UsagePolicy' YAML`](https://soya.ownyourdata.eu/SCM_UsagePolicy/yaml), [SOyA-Form](https://soya-form.ownyourdata.eu/?schemaDri=SCM_UsagePolicy&data=%7B%7D)
- [x] D2A - [`SOyA 'D2A' YAML`](https://soya.ownyourdata.eu/D2A/yaml), [SOyA-Form](https://soya-form.ownyourdata.eu/?schemaDri=D2A&data=%7B%7D)
- [x] D2A-Template - [`SOyA 'D2A_Template' YAML`](https://soya.ownyourdata.eu/D2A_Template/yaml), [SOyA-Form](https://soya-form.ownyourdata.eu/?schemaDri=D2A_Template&data=%7B%7D)
- [x] HoneyBatch - [`SOyA 'HoneyBatch' YAML`](https://soya.ownyourdata.eu/HoneyBatch/yaml), [SOyA-Form](https://soya-form.ownyourdata.eu/?schemaDri=HoneyBatch&data=%7B%7D)


### Records

- [x] <details><summary>Honey Organisation (incl. Usage Policy for Honey Organisation)</summary>
  - [x] input: `org_beekeeper.json`
    ```json
    {
      "name": "Imkerei Hans Huber",
      "jurisdiction": "Vienna",
      "authority-party": "WKO",
      "location": {
        "address": "Street 1, 1010 Wien"
      },
      "usage-policy": {
        "name": "Beekeeper Usage Policy",
        "description": "default Usage Policy for beekeeper",
        "created_by": "Christoph",
        "created_at": "2023-03-27",
        "purpose": [
          "scdv:green_claim",
          "scdv:date_marking",
          "scdv:supply_chain_optimization"
        ],
        "processing": [
          "scdv:analyse",
          "scdv:query",
          "scdv:store"
        ],
        "recipient": [
          "scdv:authority",
          "scdv:data_space"
        ],
        "storage_location": [
          "scdv:EU"
        ],
        "retention_period": "6 months"
      }
    }
    ```
  - [x] cmdline GatewayAPI:
    - create record:  
      `cat org_beekeeper.json | curl -H "Authorization: Bearer $MASTER_TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/organization/`
    - update record:  
      `cat org_beekeeper.json | curl -H "Authorization: Bearer $MASTER_TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/organization/60`
    - read record:  
      `curl -H "Authorization: Bearer $MASTER_TOKEN" https://babelfish.data-container.net/organization/60`
    - GatewayAPI reference:
    `{"organization-id":60,"name":"Imkerei Hans Huber","admin-user-id":61}`
  - [ ] Postman
  - [x] cmdline to create DID:
    - create DID:  
      ```bash=
      echo '{"service":[
          {"serviceEndpoint":"https://babelfish.data-container.net/api/data?dri=zQmNNauQjQkpxgZrCDH9tBdosiet83dg44f7uBbLupwdjdw"}
        ]}' | oydid create --doc-pwd honey_pwd --rev-pwd honey_pwd2 \
                -s -z 1 -l https://babelfish.data-container.net
      ```
    - update DID:  
      ```bash=
      echo '{"service":[
          {"serviceEndpoint":"https://babelfish.data-container.net/api/data?dri=zQmNYQ6uiv19TxiR2KfVvAGqHDvnVFpxaJKyLyerTspbyiU"}
        ]}' | oydid update did:oyd:zQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%40babelfish.data-container.net --doc-pwd honey_pwd --rev-pwd honey_pwd2 --old-doc-pwd honey_pwd --old-rev-pwd honey_pwd2 \
                -s -z 1 -l https://babelfish.data-container.net
      ```
   - DID: `did:oyd:zQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%40babelfish.data-container.net`

  - [x] Postman
      ```
      curl -X 'POST' 'https://oydid.ownyourdata.eu/1.0/create?method=oyd' \
        -H 'Content-Type: application/json' \
        -d '{
        "options": {
          "location": "https://babelfish.data-container.net",
          "ts": 1
        },
        "secret": {
          "doc_pwd": "honey_pwd",
          "rev_pwd": "honey_pwd2"
        },
        "didDocument": {"service":[{"serviceEndpoint": "https://babelfish.data-container.net/api/data?dri=zQmNNauQjQkpxgZrCDH9tBdosiet83dg44f7uBbLupwdjdw"}]}
      }'
      ```
</details>

- [x] <details><summary>DISP Organisation (w/o Usage Policy)</summary>
  - [X]  input: `org_disp.json`
    ```json
    {
      "name": "DISP",
      "jurisdiction": "Vienna",
      "authority-party": "WKO",
      "location": {
        "address": "Street 2, 1010 Wien"
      }
    }
    ```
  - [x] cmdline GatewayAPI:  
    - create entry:
    `cat org_disp.json | curl -H "Authorization: Bearer $MASTER_TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/organization/`
    - update entry:
    `cat org_disp.json | curl -H "Authorization: Bearer $MASTER_TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/organization/95`    
    - GatewayAPI reference: `{"organization-id":95,"name":"DISP","admin-user-id":96}`
  - [ ] Postman
  - [x] cmdline to create DID:
    - create DID  
      ```bash=
      echo '{"service":[
          {"serviceEndpoint":"https://babelfish.data-container.net/api/data?dri=zQmQHmiJrCtvDsizy9SL92sqppXXuPNar6s8bDtprX4NgZq"}
        ]}' | oydid create --doc-pwd disp_pwd --rev-pwd disp_pwd2 \
                -s -z 1 -l https://babelfish.data-container.net
      ```
    - update DID  
      ```bash=
      echo '{"service":[
          {"serviceEndpoint":"https://babelfish.data-container.net/api/data?dri=zQmP6NCKPVv7TQSofdSpNykY7hn9yr9C27f6zzthwWjpSzz"}
        ]}' | oydid update did:oyd:zQmbp9vm7VszVUj1gwGrsvZX5uHiXeFw2jsCbTRwDd5SYxp%40babelfish.data-container.net --doc-pwd disp_pwd --rev-pwd disp_pwd2 --old-doc-pwd disp_pwd --old-rev-pwd disp_pwd2 \
                -s -z 1 -l https://babelfish.data-container.net
  - [ ] Postman
  - DID: `did:oyd:zQmbp9vm7VszVUj1gwGrsvZX5uHiXeFw2jsCbTRwDd5SYxp%40babelfish.data-container.net` 
</details>

- [x] <details><summary>Market Maker (Shop)</summary>
    - create DID  
      ```bash=
      echo '' | oydid create --doc-pwd show_pwd --rev-pwd shop_pwd2 \
                -s -z 1 -l https://babelfish.data-container.net
      ```
    - DID: `did:oyd:zQmZzphAYNtaatPHTSFM9TQgxGkUbdLuFNAcsWLYif2WSdH%40babelfish.data-container.net`
</details>

- [x] <details><summary>Honey Storage Service (incl. Usage Policy from DISP for beekeepers)</summary>
  - [x]  input: `service_write_honey.json`
    ```json
    {
      "interface": {
        "info": {
          "title": "Write HoneyBatch",
          "description": "this service is part of the Honey supply chain management use case to demonstrate various features of the Gateway API"
        },
        "servers": [
          {
            "url": "https://honey-oc3.data-container.net"
          }
        ],
        "paths": {
          "/api/data/": {
            "post": {
              "requestBody": {
                "content": {
                  "application/json": {
                    "schema": {}
                  }
                }
              }
            }
          }
        }
      },
      "data": "HoneyBatch",
      "governance": {
        "name": "Intermediary Usage Policy for Agricultural Data",
        "description": "example usage policy for a data intermediary",
        "created_by": "Christoph",
        "created_at": "2023-03-27",
        "purpose": [
          "scdv:green_claim"
        ],
        "processing": [
          "scdv:store"
        ],
        "recipient": [
          "scdv:authority",
          "scdv:data_space"
        ],
        "storage_location": [
          "scdv:EU"
        ],
        "retention_period": "6 months"
      }
    }
    ```
  - [x] cmdline GatewayAPI:
    - create entry:
    `cat service_write_honey.json | curl -H "Authorization: Bearer $DISP_TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/service/`
    - update entry:
    `cat service_write_honey.json | curl -H "Authorization: Bearer $DISP_TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/service/97`
    - GatewayAPI:  
      `{"service-id":97,"name":"Write HoneyBatch"}`
  - [ ] Postman
</details>

- [x] <details><summary>D2A Template</summary>
  - [x] Input: d2a_template.json
    ```json=
    {
      "purposes": [
        {
          "service": "Honey Production",
          "data_owners": [
            {
              "name": "Imkerei Hans Huber",
              "address": "Street 1, A-1010 Vienna",
              "party_type": "Ministry of Agriculture",
              "organization_id": "did:oyd:zQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5@babelfish.data-container.net"
            }
          ],
          "jurisdiction": "Vienna",
          "purpose_type": "Beekeeper Data Sharing",
          "cii_information": [
            {
              "sensitivity": "none",
              "attribute_id": "https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.beekeeper_did",
              "data_optional": false,
              "attribute_type": [
                "scdv:identity"
              ]
            },
            {
              "sensitivity": "csi",
              "attribute_id": "https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.volume",
              "attribute_type": [
                "scdv:agricultural"
              ]
            },
            {
              "sensitivity": "csi",
              "attribute_id": "https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.wanderkarte",
              "attribute_type": [
                "scdv:agricultural"
              ]
            }
          ],
          "data_controllers": [
            {
              "name": "DISP",
              "address": "Street 2, A-1010 Vienna",
              "party_type": "Chamber of Commerce",
              "organization_id": "did:oyd:zQmbp9vm7VszVUj1gwGrsvZX5uHiXeFw2jsCbTRwDd5SYxp@babelfish.data-container.net"
            }
          ],
          "purpose_description": "default ",
          "purpose": [
            "scdv:green_claim"
          ],
          "processing": [
            "scdv:store",
            "scdv:query"
          ],
          "recipient": [
            "scdv:authority",
            "scdv:data_space"
          ],
          "storage_location": [
            "scdv:EU"
          ],
          "retention_period": "6 months",
          "geographic_restriction": "EEA",
          "withdrawal_method": "contact via email",
          "authority_party": "Austrian Chamber of Commerce"
        }
      ]
    }
    ```

  - [x] show & edit D2A Template: [SOyA Form link](https://soya-form.ownyourdata.eu/?schemaDri=D2A_Template&data=%7B%22purposes%22%3A%5B%7B%22service%22%3A%22Honey+Production%22%2C%22data_owners%22%3A%5B%7B%22name%22%3A%22Imkerei+Hans+Huber%22%2C%22address%22%3A%22Street+1%2C+A-1010+Vienna%22%2C%22party_type%22%3A%22Ministry+of+Agriculture%22%2C%22organization_id%22%3A%22did%3Aoyd%3AzQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%40babelfish.data-container.net%22%7D%5D%2C%22jurisdiction%22%3A%22Vienna%22%2C%22purpose_type%22%3A%22Beekeeper+Data+Sharing%22%2C%22cii_information%22%3A%5B%7B%22sensitivity%22%3A%22none%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.beekeeper_did%22%2C%22data_optional%22%3Afalse%2C%22attribute_type%22%3A%5B%22scdv%3Aidentity%22%5D%7D%2C%7B%22sensitivity%22%3A%22csi%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.volume%22%2C%22attribute_type%22%3A%5B%22scdv%3Aagricultural%22%5D%7D%2C%7B%22sensitivity%22%3A%22csi%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.wanderkarte%22%2C%22attribute_type%22%3A%5B%22scdv%3Aagricultural%22%5D%7D%5D%2C%22data_controllers%22%3A%5B%7B%22name%22%3A%22DISP%22%2C%22address%22%3A%22Street+2%2C+A-1010+Vienna%22%2C%22party_type%22%3A%22Chamber+of+Commerce%22%2C%22organization_id%22%3A%22did%3Aoyd%3AzQmbp9vm7VszVUj1gwGrsvZX5uHiXeFw2jsCbTRwDd5SYxp%40babelfish.data-container.net%22%7D%5D%2C%22purpose_description%22%3A%22default+%22%2C%22purpose%22%3A%5B%22scdv%3Agreen_claim%22%5D%2C%22processing%22%3A%5B%22scdv%3Astore%22%2C%22scdv%3Aquery%22%5D%2C%22recipient%22%3A%5B%22scdv%3Aauthority%22%2C%22scdv%3Adata_space%22%5D%2C%22storage_location%22%3A%5B%22scdv%3AEU%22%5D%2C%22retention_period%22%3A%226+months%22%2C%22geographic_restriction%22%3A%22EEA%22%2C%22withdrawal_method%22%3A%22contact+via+email%22%2C%22authority_party%22%3A%22Austrian+Chamber+of+Commerce%22%7D%5D%7D)
  
</details>

- [x] <details><summary>Collection for HoneyBatch data (incl. reference to D2A Template)</summary>
  - [X] input: `collection_honeybatch.json`
    (must include: D2A Template, SOyA Structure)
    ```json=
    {
      "name": "HoneyBatch",
      "storage": {
        "type": "Semantic Container",
        "url": "https://honey-oc3.data-container.net" },
      "schema": "zQmcfpdZVuLJcXUDtqoLnPAHnA4eQNKaX6AcYBttUPaLpC8",
      "d2a-template": {
        "purposes": [
          {
            "purpose_description": "this D2A focuses on the Green Claims purpose",
            "purpose": [
              "scdv:green_claim"
            ],
            "processing": [
              "scdv:store",
              "scdv:query"
            ],
            "recipient": [
              "scdv:authority",
              "scdv:data_space"
            ],
            "storage_location": [
              "scdv:EU"
            ],
            "cii_information": [
              {
                "sensitivity": "none",
                "attribute_id": "https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.beekeeper_did",
                "data_optional": false,
                "attribute_type": [
                  "scdv:identity"
                ]
              },
              {
                "sensitivity": "csi",
                "attribute_id": "https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.volume",
                "attribute_type": [
                  "scdv:agricultural"
                ]
              },
              {
                "sensitivity": "csi",
                "attribute_id": "https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.wanderkarte",
                "attribute_type": [
                  "scdv:agricultural"
                ]
              }
            ],
            "data_controllers": [
              {
                "name": "DISP",
                "address": "Street 2, A-1010 Vienna",
                "party_type": "Chamber of Commerce",
                "organization_id": "did:oyd:zQmbp9vm7VszVUj1gwGrsvZX5uHiXeFw2jsCbTRwDd5SYxp@babelfish.data-container.net"
              }
            ],
            "retention_period": "6 months",
            "jurisdiction": "Vienna",
            "geographic_restriction": "EEA",
            "withdrawal_method": "contact via email",
            "authority_party": "Austrian Chamber of Commerce",
            "service": "Honey Production"
          }
        ]
      }
    }
    ```
  - [x] cmdline GatewayAPI:
    - create entry:
    `cat collection_honeybatch.json | curl -H "Authorization: Bearer $DISP_TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/collection/`
    - update entry:
    `cat collection_honeybatch.json | curl -H "Authorization: Bearer $DISP_TOKEN" -H "Content-Type: application/json" -d @- -X PUT https://babelfish.data-container.net/service/98`
    - GatewayAPI:  
      `{"collection-id":98,"name":"HoneyBatch"}`
  - [ ] Postman
</details>

- [ ] <details><summary>Beekeeper User linked to Storage Service</summary>
  - [ ] input: Github
  - [ ] cmdline GatewayAPI:
  - [ ] Postman
  - GatewayAPI: link
  - [ ] cmdline to create DID:
  - [ ] Postman
  - DID:  
</details>

- [x] <details><summary>HoneyBatch dataset</summary>
  - [x] input: [Permalink](https://soya-form.ownyourdata.eu/?viewMode=embedded&schemaDri=zQmTvtiQd5w2BdYv8gncgiKq7cRFbvE29zS4UWqneXNyyt3&data=%7B%22unit%22%3A%22kg%22%2C%22volume%22%3A%225%22%2C%22produced_at%22%3A%222023-03-15%22%2C%22beekeeper_did%22%3A%22did%3Aoyd%3AzQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%2540babelfish.data-container.net%22%7D)
  - [x] Record:
    ```json=
    {
      "unit": "kg",
      "volume": "5",
      "produced_at": "2023-03-15",
      "beekeeper_did": "did:oyd:zQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%40babelfish.data-container.net",
      "meta": {"schema":"HoneyBatch"}
    }
    ```

</details>


### Snippets for Command Line

- [x] <details><summary>Master Credentials to create Organisation: <code>MASTER_TOKEN</code></summary>

  * Key: `rkgYEN5QXmsFcwbH-awWoo9Q5BJd6vdGoyzRHI0lOZs`
  * Secret: `AoaoR9-Aj7woafUW7hnARWVbNqaBxINiqteyMDMuBoA`
  ```bash=
  export GW_HOST="https://babelfish.data-container.net"
  export APP_KEY="rkgYEN5QXmsFcwbH-awWoo9Q5BJd6vdGoyzRHI0lOZs"
  export APP_SECRET="AoaoR9-Aj7woafUW7hnARWVbNqaBxINiqteyMDMuBoA"
  export MASTER_TOKEN=`curl -s -d grant_type=client_credentials -d client_id=$APP_KEY -d client_secret=$APP_SECRET -d scope=admin -X POST https://babelfish.data-container.net/oauth/token | jq -r '.access_token'`
  ```

</details>

- [x] <details><summary>Beekeeper Credentials: <code>BEE_TOKEN</code></summary>

  ```bash=
  export GW_HOST="https://babelfish.data-container.net"
  export BEE_KEY="jo6Wf_LUYMwFm2Ai9H6exLqhUXU2xUrNh3KLN3H_unY"
  export BEE_SECRET="vG0UfHw8tfIqU4cKKNDDs70qlvImSfGWEGJ4WHKASsM"
  export BEE_TOKEN=`curl -s -d grant_type=client_credentials -d client_id=$BEE_KEY -d client_secret=$BEE_SECRET -d scope=write -X POST https://babelfish.data-container.net/oauth/token | jq -r '.access_token'`
  ```

</details>

- [x] <details><summary>DISP Credentials: <code>DISP_TOKEN</code></summary>

  ```bash=
  export GW_HOST="https://babelfish.data-container.net"
  export DISP_KEY="des1QdHDgS4w9zGLacQs_fuAY_po8iSxVXfKQyvdpDI"
  export DISP_SECRET="b2_iytO7JddGIMduDdlfiEgrUKZKEZQa_CntcvDsbeE"
  export DISP_TOKEN=`curl -s -d grant_type=client_credentials -d client_id=$DISP_KEY -d client_secret=$DISP_SECRET -d scope=write -X POST https://babelfish.data-container.net/oauth/token | jq -r '.access_token'`
  ```

</details>

- [x] <details><summary>Dummy User Credentials: <code>TOKEN</code></summary>

  ```bash=
  export GW_HOST="https://babelfish.data-container.net"
  export KEY="-6H7FHYo4aX5-dYMVF82x2_rzO1cXIB5URc24dPwMls"
  export SECRET="cnhNgI77IMVeyenhZnPylcD_XKO72piWzzT68psVKJA"
  export TOKEN=`curl -s -d grant_type=client_credentials -d client_id=$KEY -d client_secret=$SECRET -d scope=write -X POST https://babelfish.data-container.net/oauth/token | jq -r '.access_token'`
  ```

</details>

- [x] <details><summary>OwnYourData Credentials: <code>TOKEN</code></summary>

  ```bash=
  export GW_HOST="https://babelfish.data-container.net"
  export KEY="YleLTVd6MksqdT5560x3iPYl0gEjnMJNeWtZOaVskE8"
  export SECRET="PKzAtcFnETyw_ZPo4TcBLeshzAXj8uVXOhCWfbRSQ6s"
  export TOKEN=`curl -s -d grant_type=client_credentials -d client_id=$KEY -d client_secret=$SECRET -d scope=write -X POST https://babelfish.data-container.net/oauth/token | jq -r '.access_token'`
  ```

</details>


### Open Tasks
- [x] CFA: complete D2A process documented
- [x] CFA: all SOyA structures available
- [x] CFA: create Usage Policy for Organisation
- [x] Seb: attribute values for SCM_UsagePolicy
- [ ] Seb: edit usage policies for Beekeeper & DISP
- [ ] Seb: edit organisation records of Beekeeper & DISP
- [x] Gabriel: multi-select drop-down for Usage Policies
- [ ] CFA: design & implement process for collections to require D2A and provide D2A Template in case of non-compliance
- [ ] CFA: implement GW_API/integration/match_up
- [ ] CFA: implement GW_API/integration/create_d2a_template
- [ ] CFA: implement GW_API/integration/create_d2a
- [ ] CFA: implement GW_API/integration/extract_up
- [ ] CFA: implement GW_API/integration/match_d2a
- [ ] CFA: implement validation service
- [ ] CFA: full metadata specification & implementation

### Open Questions
- [ ] Jan: should D2A include hash of dataset
    a malicious DISP could otherwise use D2A also for other datasets
- [ ] CFA: where store PKs for DID created by DISP?

### Process Overview
```plantuml
@startuml
participant Beekeeper as bk
participant DISP as disp
autonumber

bk<-disp: Retrieve DISP Context\n(get D2A Template)
bk->bk: check UP compliance
bk->bk: create D2A proposal
bk->disp: write data (incl. D2A proposal)
disp->disp: verify input material
disp->disp: store data + metadata
disp --> bk: response with reference to dataset
```

Relevant records (and link to SOyA-Form instance):

* [D2A Template](https://soya-form.ownyourdata.eu/?schemaDri=D2A_Template&data=%7B%22purposes%22%3A%5B%7B%22service%22%3A%22Honey+Production%22%2C%22data_owners%22%3A%5B%7B%22name%22%3A%22Imkerei+Hans+Huber%22%2C%22address%22%3A%22Street+1%2C+A-1010+Vienna%22%2C%22party_type%22%3A%22Ministry+of+Agriculture%22%2C%22organization_id%22%3A%22did%3Aoyd%3AzQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%40babelfish.data-container.net%22%7D%5D%2C%22jurisdiction%22%3A%22Vienna%22%2C%22purpose_type%22%3A%22Beekeeper+Data+Sharing%22%2C%22cii_information%22%3A%5B%7B%22sensitivity%22%3A%22none%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.beekeeper_did%22%2C%22data_optional%22%3Afalse%2C%22attribute_type%22%3A%5B%22scdv%3Aidentity%22%5D%7D%2C%7B%22sensitivity%22%3A%22csi%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.volume%22%2C%22attribute_type%22%3A%5B%22scdv%3Aagricultural%22%5D%7D%2C%7B%22sensitivity%22%3A%22csi%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.wanderkarte%22%2C%22attribute_type%22%3A%5B%22scdv%3Aagricultural%22%5D%7D%5D%2C%22data_controllers%22%3A%5B%7B%22name%22%3A%22DISP%22%2C%22address%22%3A%22Street+2%2C+A-1010+Vienna%22%2C%22party_type%22%3A%22Chamber+of+Commerce%22%2C%22organization_id%22%3A%22did%3Aoyd%3AzQmbp9vm7VszVUj1gwGrsvZX5uHiXeFw2jsCbTRwDd5SYxp%40babelfish.data-container.net%22%7D%5D%2C%22purpose_description%22%3A%22default+%22%2C%22purpose%22%3A%5B%22scdv%3Agreen_claim%22%5D%2C%22processing%22%3A%5B%22scdv%3Astore%22%2C%22scdv%3Aquery%22%5D%2C%22recipient%22%3A%5B%22scdv%3Aauthority%22%2C%22scdv%3Adata_space%22%5D%2C%22storage_location%22%3A%5B%22scdv%3AEU%22%5D%2C%22retention_period%22%3A%226+months%22%2C%22geographic_restriction%22%3A%22EEA%22%2C%22withdrawal_method%22%3A%22contact+via+email%22%2C%22authority_party%22%3A%22Austrian+Chamber+of+Commerce%22%7D%5D%7D)
* [D2A Proposal](https://soya-form.ownyourdata.eu/?schemaDri=D2A&data=%7B%22purposes%22%3A%5B%7B%22purpose_description%22%3A%22green+claim+for+honey+batch%22%2C%22purpose%22%3A%5B%22scdv%3Agreen_claim%22%5D%2C%22processing%22%3A%5B%22scdv%3Aquery%22%2C%22scdv%3Astore%22%5D%2C%22recipient%22%3A%5B%22scdv%3Adata_space%22%2C%22scdv%3Aauthority%22%5D%2C%22storage_location%22%3A%5B%22scdv%3AEU%22%5D%2C%22retention_period%22%3A%22P0Y6M0DT0H0M0S%22%2C%22service%22%3A%22Honey+Production%22%2C%22geographic_restriction%22%3A%22EEA%22%2C%22jurisdiction%22%3A%22Vienna%22%2C%22withdrawal_method%22%3A%22contact+via+email+office%40babelfish.org%22%2C%22authority_party%22%3A%22Austrian+Chamber+of+Commerce%22%2C%22cii_information%22%3A%5B%7B%22sensitivity%22%3A%22none%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.beekeeper_did%22%2C%22data_optional%22%3Afalse%2C%22attribute_type%22%3A%5B%22scdv%3Aidentity%22%5D%7D%2C%7B%22sensitivity%22%3A%22csi%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.volume%22%2C%22attribute_type%22%3A%5B%22scdv%3Aagricultural%22%5D%7D%2C%7B%22sensitivity%22%3A%22csi%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.wanderkarte%22%2C%22attribute_type%22%3A%5B%22scdv%3Aagricultural%22%5D%2C%22data_optional%22%3Atrue%7D%5D%2C%22data_controllers%22%3A%5B%7B%22name%22%3A%22DISP%22%2C%22address%22%3A%22Street+2%2C+A-1010+Vienna%22%2C%22party_type%22%3A%22Chamber+of+Commerce%22%2C%22organization_id%22%3A%22did%3Aoyd%3AzQmbp9vm7VszVUj1gwGrsvZX5uHiXeFw2jsCbTRwDd5SYxp%40babelfish.data-container.net%22%7D%5D%2C%22data_owners%22%3A%5B%7B%22name%22%3A%22Imkerei+Hans+Huber%22%2C%22address%22%3A%22Street+1%2C+A-1010+Vienna%22%2C%22party_type%22%3A%22WKO%22%2C%22organization_id%22%3A%22did%3Aoyd%3AzQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%40babelfish.data-container.net%22%7D%5D%7D%5D%7D)
* [Honey batch data set](https://soya-form.ownyourdata.eu/?viewMode=embedded&schemaDri=zQmTvtiQd5w2BdYv8gncgiKq7cRFbvE29zS4UWqneXNyyt3&data=%7B%22unit%22%3A%22kg%22%2C%22volume%22%3A%225%22%2C%22produced_at%22%3A%222023-03-15%22%2C%22beekeeper_did%22%3A%22did%3Aoyd%3AzQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%2540babelfish.data-container.net%22%7D)
* object-id at Babelfish: 106  
  `curl -s -H "Authorization: Bearer $DISP_TOKEN" https://babelfish.data-container.net/object/106/meta | jq`

![](https://i.imgur.com/Lfi2jBc.png)


## 1- Retrieve DISP Context

1. <details><summary>Query Service Catalogue</summary>

    - [x] cmdline GatewayAPI
          `curl "https://babelfish.data-container.net/service/search?name=honey"`  
          `curl https://babelfish.data-container.net/service/97`
    - [ ] Postman
</details>

  -> Usage Policy from DISP available
  -> API endpoint for Collection available

2. <details><summary>Write HoneyBatch to DISP</summary>

    - [ ] cmdline GatewayAPI
          `echo HoneyBatch.json | curl -H "Authorization: Bearer $BEE_TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/object/`
    - [ ] Postman
</details>

  -> D2A Template available

## 2 - Check compliance of Usage Policies

1. <details><summary>Show Beekeeper Usage Policy</summary>
  
    - [X] Link to SOyA Form: [Permalink](https://soya-form.ownyourdata.eu/?schemaDri=SCM_UsagePolicy&data=%7B%22name%22%3A%22Beekeeper+Usage+Policy%22%2C%22description%22%3A%22default+Usage+Policy+for+beekeeper%22%2C%22created_by%22%3A%22Christoph%22%2C%22created_at%22%3A%222023-03-27%22%2C%22purpose%22%3A%5B%22scdv%3Agreen_claim%22%2C%22scdv%3Adate_marking%22%2C%22scdv%3Asupply_chain_optimization%22%5D%2C%22processing%22%3A%5B%22scdv%3Aanalyse%22%2C%22scdv%3Aquery%22%2C%22scdv%3Astore%22%5D%2C%22recipient%22%3A%5B%22scdv%3Aauthority%22%2C%22scdv%3Adata_space%22%5D%2C%22storage_location%22%3A%5B%22scdv%3AEU%22%5D%2C%22retention_period%22%3A%226+months%22%7D)
</details>

2. <details><summary>Show DISP Usage Policy</summary>
  
    - [X] Link to SOyA Form: [Permalink](https://soya-form.ownyourdata.eu/?schemaDri=SCM_UsagePolicy&data=%7B%22name%22%3A%22Intermediary+Usage+Policy+for+Agricultural+Data%22%2C%22description%22%3A%22example+usage+policy+for+a+data+intermediary%22%2C%22created_by%22%3A%22Christoph%22%2C%22created_at%22%3A%222023-03-27%22%2C%22purpose%22%3A%5B%22scdv%3Agreen_claim%22%5D%2C%22processing%22%3A%5B%22scdv%3Astore%22%5D%2C%22recipient%22%3A%5B%22scdv%3Aauthority%22%2C%22scdv%3Adata_space%22%5D%2C%22storage_location%22%3A%5B%22scdv%3AEU%22%5D%2C%22retention_period%22%3A%226+months%22%7D)
</details>

3. <details><summary>Compare Usage Policies</summary>
  
    - [ ] cmdline for Usage Policy compliance service:
          `cat '{"provider":{["provider up"]}, "consumer":{["consumer up"]}}' | \`
`curl -H "Authorization: Bearer $BEE_TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/integration/match_up`
    - [ ] Postman
    - [ ] show result from Usage Policy comparison
    - if Usage Policies are not compliant: edit Beekeer Usage Policy and repeat step 3.
  
</details>

-> DISP compliant Usage Policy of Beekeeper available


## 3 - Create D2A Proposal

1. <details><summary>Use Beekeeper Usage Policy to create D2A Proposal</summary>
  
    - [ ] cmdline for Usage Policy compliance service:
          `cat '{"usage-policy":{["beekeeper usage policy"]}, "data-owner":{["beekeeper organisation record"]} "d2a-template":{["D2A template"]}}' | \`
`curl -H "Authorization: Bearer $BEE_TOKEN" -H "Content-Type: application/json" -d @- -X POST https://babelfish.data-container.net/integration/create_d2a`
    - [ ] Postman
    - [ ] show D2A Proposal, [link to SOyA Form](https://soya-form.ownyourdata.eu/?schemaDri=D2A&data=%7B%22purposes%22%3A%5B%7B%22purpose_description%22%3A%22green+claim+for+honey+batch%22%2C%22purpose%22%3A%5B%22scdv%3Agreen_claim%22%5D%2C%22processing%22%3A%5B%22scdv%3Aquery%22%2C%22scdv%3Astore%22%5D%2C%22recipient%22%3A%5B%22scdv%3Adata_space%22%2C%22scdv%3Aauthority%22%5D%2C%22storage_location%22%3A%5B%22scdv%3AEU%22%5D%2C%22retention_period%22%3A%22P0Y6M0DT0H0M0S%22%2C%22service%22%3A%22Honey+Production%22%2C%22geographic_restriction%22%3A%22EEA%22%2C%22jurisdiction%22%3A%22Vienna%22%2C%22withdrawal_method%22%3A%22contact+via+email+office%40babelfish.org%22%2C%22authority_party%22%3A%22Austrian+Chamber+of+Commerce%22%2C%22cii_information%22%3A%5B%7B%22sensitivity%22%3A%22none%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.beekeeper_did%22%2C%22data_optional%22%3Afalse%2C%22attribute_type%22%3A%5B%22scdv%3Aidentity%22%5D%7D%2C%7B%22sensitivity%22%3A%22csi%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.volume%22%2C%22attribute_type%22%3A%5B%22scdv%3Aagricultural%22%5D%7D%2C%7B%22sensitivity%22%3A%22csi%22%2C%22attribute_id%22%3A%22https%3A%2F%2Fsoya.ownyourdata.eu%2FHoneyBatch%3AHoneyBatch.wanderkarte%22%2C%22attribute_type%22%3A%5B%22scdv%3Aagricultural%22%5D%2C%22data_optional%22%3Atrue%7D%5D%2C%22data_controllers%22%3A%5B%7B%22name%22%3A%22DISP%22%2C%22address%22%3A%22Street+2%2C+A-1010+Vienna%22%2C%22party_type%22%3A%22Chamber+of+Commerce%22%2C%22organization_id%22%3A%22did%3Aoyd%3AzQmbp9vm7VszVUj1gwGrsvZX5uHiXeFw2jsCbTRwDd5SYxp%40babelfish.data-container.net%22%7D%5D%2C%22data_owners%22%3A%5B%7B%22name%22%3A%22Imkerei+Hans+Huber%22%2C%22address%22%3A%22Street+1%2C+A-1010+Vienna%22%2C%22party_type%22%3A%22WKO%22%2C%22organization_id%22%3A%22did%3Aoyd%3AzQmcHfufF5xtjhgqDbXmWeSSd2WKoncQC1S4dajk8eNnPM5%40babelfish.data-container.net%22%7D%5D%7D%5D%7D)
</details>

-> D2A Proposal available

2. <details><summary>Create VC Signature for D2A Proposal</summary>
  
    - [ ] cmdline to create signature
    ```bash=
    TBD
    ```
    - [ ] Postman
</details>

-> D2A Proposal Signature available


## 4 - Send HoneyBatch data to DISP (incl. D2A Proposal)

1. <details><summary>create Object in Collection</summary>

      - [x] cmdline:
    ```bash=
    cat '{"collection-id": $HONEY_COLLECTION}' | \
    curl -H "Authorization: Bearer $BEE_TOKEN" -H "Content-Type: application/json" -d @- \
      -X POST $HONEY_COLLECTION_ENDPOINT
    ```
    Response: `OBJECT_ID`
      - [ ] Postman  
  
</details>

2. <details><summary>send data</summary>

      - [ ]  cmdline:
    ```bash=
    cat '{
            "data": {["HoneyBatch data"]},
            "meta": {
                "d2a": {["D2A Proposal"]},
                "d2a-signature": {["D2A Signature"]}
            }
         }' | \
    curl -H "Authorization: Bearer $BEE_TOKEN" -H "Content-Type: application/json" -d @- \
      -X POST https://babelfish.data-container.net/object/OBJECT_ID/write
    ```
      - [ ] Postman  
  
</details>


## 5 - DISP verifies input data

The following verification steps are performed:
1. <details><summary>Usage Policy from D2A proposal matches with DISP Usage Policy</summary>

    - [ ] extract Usage Policy from D2A proposal in metadata
      `POST /integration/extract_up`
    - [ ] match provided Usage Policy from Beekeeper with Usage Policy from DISP
      `POST GATEWAY_API/integration/match_up`
  
    Response in case of failed validation: "the Usage Policy provided is non-compliant" and compliance report from Usage Policy comparison
  
</details>

2. <details><summary>D2A proposal conforms to D2A Template in requested collection</summary>

    - [ ] check D2A proposal conforms to specified D2A Template
      `POST GATEWAY_API/integration/match_d2a`
  
    Response in case of failed validation: "the Data Agreement provided is non-compliant" and compliance report from D2A comparison
  
</details>

3. <details><summary>provided dataset conforms to specified SOyA structure</summary>

    - [ ] validate that provided data matches SOyA structure specified for collection
      `POST VALIDATION_SERVICE/validate/{DRI}`
  
    Response in case of failed validation: "the data provided is non-compliant with the expected structure" and compliance report from validation service
  
</details>

4. <details><summary>valid signature from Beekeeper to create VC</summary>

    - [ ] validate that provided signature can be verified with public key from Beekeeper
      `POST OYDID_REPOSITORY/check_signature`
  
    Response in case of failed validation: "the signature provided is non-compliant"
  
</details>



## 6 - DISP stores input data

1. <details><summary>Creat DISP VC Signature for D2A</summary>

    - [ ] create & sign Credential Subject object
  
</details>

2. <details><summary>Create Verifiable Credential</summary>

    - [ ] collect all pieces and compile VC
    - [ ] create VP
  
</details>

3. <details><summary>Create Metadata section</summary>

    - [ ] build metadata section with reference to VP of D2A, schema, and provenance record
  
</details>

5. <details><summary>Write data</summary>

    - [ ] store new object with data (HoneyBatch) and metadata
  
</details>

6. <details><summary>Create DID for HoneyBatch data</summary>

    - [ ] create DID Document for HoneyBatch data that provides service endpoints for data (HoneyBatch) and associated D2A as VP
  
</details>


## 7 - DISP responds with DID for HoneyBatch data

1. Responsed with DID for POST request in section 4 step 2