# Data Agreements
Inputs:
* Honey Use Case Data: https://hackmd.io/hFhpiU6MQpKacHbijiWBFQ#Content
* VCs & VPs with OYDID: https://hackmd.io/q7vE3nuaSSyVDQkqccfFzA
Tasks
- Create google sheet with template of agreements
- Create example of D2A basic skeleton
- Create example of D3A basic skeleton
- Implement "honey origin" use case
- Implement "green transport" use case
# Concepts
## Usage policies
The usage policies are used to set conditions for data consumption. A data provider chooses which usages policies they would agree with while data consumer sets the usage policies.
Types of checks:
1. Geographic restrictions set by data provider or authority agency
2. Types of processing example copy, combine, access, profile ([DPV:Processing](https://w3c.github.io/dpv/dpv/#vocab-processing))
3. Purpose type restrictions example sell products, fulfillment of obligations ([DPV: Purpose](https://w3c.github.io/dpv/dpv/#vocab-purpose))
4. Set data retention limitations
5. What data are consumed example data category (soya)
Hierarchy of usage policies for matching in order to set data agreement (d2a or d3a):
1. intermediary
2. data provider (organisation, service)
3. data consumer
## Checks
There are different forms of checks performed aumatically in teh data exchange. These are the type of checks:
1. matching (relates to usage policy and d3a)
2. validation (data model validation)
Errors reported are bound to type of check.
AP: Fajar to set error descriptions
## Agreement
Sets the bounds of the processing of data similar to a contract.
## Roles
There are four main roles:
* Data provider (or source): originated the data (d2a only)
* Data consumer: consumes the data (d3a only)
* Intermediary (or storage provider): acts as proxy between data provider and consumer. (d2a and d3a)
* Authority agency: sets certain conditions for sharing of data (d2a and d3a)
In the relationship of a data agreement (d2a or d3a):
* Data Sender: Who sends data
* Data Receiver: Who receives data
# Data Agreements
Material for input to data agreements are in google document.
[D2: Domain specific Data Information](https://docs.google.com/document/d/1MlTVrH8ayzfrsIaXh9ZYe7Afhp9v9bpiSXuwlM845fM/edit?pli=1#heading=h.8xqzib96b64k)
[D2: Agreement schema](https://docs.google.com/document/d/1MlTVrH8ayzfrsIaXh9ZYe7Afhp9v9bpiSXuwlM845fM/edit?pli=1#heading=h.9pgu33jaduo0)
## Domain specific Data Agreement (D2A) Skeleton
```json=
{
"purposes":[{
"purpose_desription": "string",
"purpose_type": "string",
"service": "string",
"cii_information":[{
"attribute_id":"did",
"attribute_type": "string",
"sensitivity": "csi"
"data_optional": "true/false default not included false"
}],
"jurisdiction": "string",
"data_controllers": [{
"organization_id": "national id",
"name": "name",
"address": "string",
"party_type": "string"
}],
"data_owner": [{
"organization_id": "national id",
"name": "name",
"address": "string",
"party_type": "string"
}]
}],
}
```
attribute_type = ?
CSI values = ?
party_type = ?
## Domain specific Disclosure Data Agreement (D3A) Skeleton
Same structure as D2A. Exception:
data_controller = data_controller (same role "Intemediary")
data_owner = third_party ("data user")
```json=
{
"content": 1
}
```
# Use Cases
## Raw Data
**Stakeholder:** [`Organisation`](https://soya.ownyourdata.eu/Organisation/yaml)
* Honey Organisation in Babelfish account mgmt
* URL: https://babelfish.data-container.net/organization/1
```json
{"name":"Imkerei Hans Huber","address":"Hauptstraße 1, 1010 Wien","organization-id":1}
```
* DRI: `zQmZLBvttK9kK4yyr2JrpxEKJk2XSHsDjzVAR3M8LjrVbHw`
* DID: `did:oyd:zQmNoUmZbq17CZZpP4uGNNS9mQeodPKuJHhvezWSeNd9knt%40babelfish.data-container.net` (--doc-pwd orgpwd --rev-pwd orgpwd2)
* Beekeeper: [data.json](https://gist.github.com/fabianekc/345caad8e1776217c5b0e4de9af3198f)
* URL: https://babelfish.data-container.net/user/2
```json
{"name":"Hans Huber","email":"hans@huber.com","organization-id":1,"user-id":2}
```
* DRI: `zQmU3jzkjAyT8Uz9qCDB4hkG41vyy2fqraPVh4uSXhFxwuC`
* DID: `did:oyd:zQme7k5u6XryvmnUkS1bRBrbP6CrzHMcjKP3qduN9EkCFmp%40babelfish.data-container.net` (--doc-pwd beepwd --rev-pwd beepwd2)
* DISP
* URL: https://babelfish.data-container.net/user/3
```json
{"name":"Data Intermediary","email":"info@disp.com","organization-id":1,"user-id":3}
```
* DRI: `zQmc2FnJAeNEYdaF4V1MTy7VAWamhp388xuH6mTczWadoBK`
* DID: `did:oyd:zQmY75fpVSzXAe9VFNf6UxVW2t8DYqtmQbtBGmqcEytZW1z%40babelfish.data-container.net`(--doc-pwd dispwd --rev-pwd dispwd2)
* Transport Company: [data.json](https://gist.github.com/fabianekc/3ce126da1f01478684b5a21b2b3c48c9)
* [ ] Transport Company User in Babelfish account mgmt -> DRI
* [ ] Transport Company DID references data -> DID
* Shop: [data.json](https://gist.github.com/fabianekc/15365d7889e846b0be5028e510ab3a79)
* [ ] Shop User in Babelfish account mgmt -> DRI
* [ ] Shop DID references data -> DID
**Simple D2A Document**
```jsonld=
{
"purposes":[{
"purpose_desription": "I obey to D2A"
}]
}
```
#### Full Verifiable Credential
```jsonld=
{
"@context": ["https://www.w3.org/ns/credentials/v2"],
"type": ["VerifiableCredential"],
"identifier": "https://babelfish.data-container.net/credentials/zQmeCoAr67eoVdYpcvxe15djaDULQixJSdtRJhfEzkvq4H7",
"issuer": [
"did:oyd:zQme7k5u6XryvmnUkS1bRBrbP6CrzHMcjKP3qduN9EkCFmp%40babelfish.data-container.net",
"did:oyd:zQmY75fpVSzXAe9VFNf6UxVW2t8DYqtmQbtBGmqcEytZW1z%40babelfish.data-container.net"
],
"issuanceDate": "2023-02-13T00:36:33Z",
"credentialSubject": [
{
"id": "did:oyd:zQme7k5u6XryvmnUkS1bRBrbP6CrzHMcjKP3qduN9EkCFmp%40babelfish.data-container.net",
"type": "source",
"name": "Hans Huber"
},
{
"id": "did:oyd:zQmY75fpVSzXAe9VFNf6UxVW2t8DYqtmQbtBGmqcEytZW1z%40babelfish.data-container.net",
"type": "controller",
"name": "Data Intermediary"
},
{
"id": "did:oyd:zQmY75fpVSzXAe9VFNf6UxVW2t8DYqtmQbtBGmqcEytZW1z%40babelfish.data-container.net",
"dataAgreement": {
"purposes": [
{
"purpose_desription": "I obey to D2A"
}
]
}
}
],
"proof": [
{
"type": "Ed25519Signature2020",
"verificationMethod": "did:oyd:zQme7k5u6XryvmnUkS1bRBrbP6CrzHMcjKP3qduN9EkCFmp%40babelfish.data-container.net",
"proofPurpose": "assertionMethod",
"proofValue": "zqdKj8HrVeAqerwrVN3ed2pJbwWamRBNSPBLELbkxS8n9Z2RvDzxmyt1K9b94We75TJNrSC6ye4z7Kx9Z5ouf3Ho"
},
{
"type": "Ed25519Signature2020",
"verificationMethod": "did:oyd:zQmY75fpVSzXAe9VFNf6UxVW2t8DYqtmQbtBGmqcEytZW1z%40babelfish.data-container.net",
"proofPurpose": "assertionMethod",
"proofValue": "z2pU1pp8XrG3F1WM4PmMJosoSepZz5jpASLvGExFbbGPy4graJefZH86ZTWxw5v4BEGoYwdVA7mV3EMkZAtxRZYVt"
}
]
}
```
* [ ] Honey Collection in Storage Provider
Product ([`HoneyBatch`](https://soya.ownyourdata.eu/HoneyBatch/yaml)): [data.json](https://gist.github.com/fabianekc/1e6930333398d669dab72f9854f0b32d)
* [ ] Product Object -> DRI & DID
Transport ([`TransportEdge`](https://soya.ownyourdata.eu/TransportEdge/yaml)): [data.json](https://gist.github.com/fabianekc/80a42578033202052fccaca8d6523aaa)
* [ ] Transport Object -> DRI & DID
Sale ([`MarketMaker`](https://soya.ownyourdata.eu/MarketMaker/yaml)): [data.json](https://gist.github.com/fabianekc/f33b886fddab35d98c85a8619560d887)
* [ ] Sale Object -> DRI & DID
### Sequence Diagram
```plantuml
@startuml
participant Beekeeper as bk
participant Transport as tp
participant Shop as shop
actor Consumer as ind
collections storage as "Storage\nProvider"
== Data ==
bk->storage: HoneyBatch, D2A with all public
tp->storage: TransportEdge, D2A with\nsource, destination, mode
storage->shop: D3A read HoneyBatch
shop->storage: MarketMaker, D2A with\njar_id and weight
== DPP ==
storage->ind: D3A to\ngenerate DPP
```
### Prerequisites
1. establish Admin that can create Organisations
2. create Organisations (with optional Usage Policy)
* Honey Org
* Transport Org
* Shop Org
* Chamber of Commerce Org
* Intermediary Org
3. create users beyond automatically available admin-user
* bee-keeper can just use admin but creates DID
* Transport create user for biker and creates biker DID
* Shop creates user for staff person and creates staff DID
4. setup service for Storage Provider and includes in governance a Usage Policy
5. data consumer (e.g., Intermediary) creates collection that specifies D2A Template that must be used for all objects stored in this collection
**Functionality provided by Babelfish Integration Helper to manage Usage Policies and Data Agreements**
* check if 2 Usage Policies match
* POST /integration/usage_policy
* body: {"data_subject": {Usage Policy}, "data_controller": {Usage Policy}}
* response: {"match":true or false, "report":{if false list non-matching conditions}}
* create D2A from a Usage Policy
* POST /integration/create_d2a
* body: {Usage Policy}
* response: {D2A}
* extract Usage Policy from D2A
* POST /integration/extract_up
* body: {D2A}
* response: {Usage Policy}
**Assumption:** it is possible to switch between UP and D2A (bijective mapping between both)
this is important because there is an established process to compare (machine readable) UPs
#### Possible scenarios
4 configurations based on source/consumer with or without UP
* Option A: Data Consumer (Intermediary) has a UP and Data Provider has no UP
--> Data Provider accepts D2A
* Option B: Data Consumer (Intermediary) has no UP but Data consumer has UP
--> Data Consumer accepts D2A
* Option C: neither Data Consumer nor Data Provider have a UP
--> find a template and use it
* Option D: Data Consumer and Data Provider have each an own version of a UP
--> negotiation process
#### Negotiation Process to agree on D2A between 2 parties
possible input data (ordered by precedence - subsequent data overwrites previous data)
* for data source:
* organisation of the data provider has a Usage Policy specified (lowest priority)
* data providing service has a Usage Policy
* payload itself has a Usage Policy attached (highest priority)
* for data consumers:
* organisation of the data consumer has a Usage Policy (lowest priority)
* data consuming service has a Usage Policy (highest priority)
**Process:**
1. check if Usage Policies match (`POST /integration/usage_policy`)
2. if they match create D2A from Usage Policy of Data Provider
(`POST /integration/create_d2a`)
3. if no match: both make changes to UP until match
## 0. Step-by-step onboarding
This section describes the concrete steps between Beekeeper and DISP to store a recordset (HoneyBatch).
**Assumptions**
* dedicated Org records for Beekeeper and DISP, respective users and their DIDs
* service description for writing to the DISP
* storage provider for DISP configured with a collection that holds a D2A template
**Process**
0a. Beekeeper performs POST request with user linked to service description; the follwing information is then available
* Usage Policy from the Org the beekeeper belongs to
* Storage Provider through the service entry linked to the beekeeper user
* Usage Policy from the Org that provided the Storage Provider
* D2A template defined in the collection specified in the POST request (requires a `collection-id`)
0b. Storage Provider rejects POST request since it does not include a valid Verifiable Credential in the metadata section
1. Beekeeper compares own Usage Policy with Usage Policy from DISP <-- the process can actually start here and step 1) and 2) are just for demonstration
* when Usage Policies don't match, Beekeeper needs to adapt own Usage Policy until it is compliant with DISP Usage Policy
2. Beekeeper generates D2A proposal from own (adapted) Usage Policy and D2A Template provided in Collection + signed snippet for VC proof (with the crypto material from the Beekeeper organization)
3. Beekeeper sends D2A proposal + signed snipped (as metadata) with POST request to GatewayAPI
4. DISP verifies
* Usage Policy from D2A proposal matches with own Usage Policy
* D2A proposal conforms to D2A Template in requested collection
* Beekeeper signed D2A proposal
* if no problems occur, the DISP signs the D2A proposal (with the crypto material from the DISP organization) and returns this signature snippet in the response
5. DISP stores data
* create signature snippet of D2A (with the crypto material from the DISP organization)
* create Verifiable Credential of the D2A
* store data (HoneyBatch) + metadata (reference to D2A)
* create DID for dataset
6. respond with DID of dataset
```plantuml
@startuml
participant Beekeeper as bk
participant DISP as disp
bk<-disp: get Usage Policy + D2A Template (0)
bk->bk: check UP compliance (1)
bk->bk: create D2A proposal (2)
bk->disp: send D2A proposal (3)
disp->disp: verify input material (4)
disp->disp: store data + metadata (5)
disp --> bk: response with reference to dataset (6)
```
## 1. Honey Origin
D2A - Bee keeper (NEW)
```json=
{
"@context": [
{"scmo":"http://w3id.org/scmo/ns/v1"}
]
"purposes":[{
"purpose_desription": "Sharing of honey production with shops and consumer",
"purpose_type": "honeyProduction",
"service": "Honey Bee Inc produces highest quality honey in Austrian alps",
"cii_information":[{
"attribute_id":"https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.beekeeper_did",
"attribute_type": ["scmo:agricultureData", "scmo:identityOfActor"],
"sensitivity": true
},
{
"attribute_id":"https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.xxxx",
"attribute_type": ["scmo:agricultureData","scmo:productionBatchVolume"],
"sensitivity": true
},
{
"attribute_id":"https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.xxxx",
"attribute_type": ["scmo:agricultureData","scmo:productionBatchVolumeUnit"],
"sensitivity": true
},
{
"attribute_id":"https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.xxxx",
"attribute_type": ["scmo:agricultureData","scmo:productionBatchDate"],
"sensitivity": true
},
{
"attribute_id":"https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.region",
"attribute_type": ["agricultureData","beeCollectionRegion"],
"sensitivity": false
},
{
"attribute_id":"https://soya.ownyourdata.eu/HoneyBatch:HoneyBatch.wanderkarte",
"attribute_type": ["agricultureData","beeCollectionArea"],
"sensitivity": true
}],
"processing_method": "copy",
"retention_period": "P0003-00-00T00:00:00",
"storage_location": "EMA",
"geographic_restrictions": "EMA",
"jurisdiction": "AT",
"withdrawal_method": "Contact issuer of this agreement which is first organisation to sign agreement",
"authority_party": "Austrian government",
"data_controllers": [{
"organization_id": "national id",
"name": "name",
"address": "string",
"party_type": "string"
}],
"data_owner": [{
"organization_id": "national id",
"name": "name",
"address": "string",
"party_type": "string"
}]
}]
}
```
Questions:
1. Should attribute_id be a DID or a schema attribute reference? Address is too generic of a attribute to be concise.
2. Should controller and owner be outside and only part of the signing?
3. Which fields can be matched to governance: policy?
D2A - Bee keeper (OLD)
```json=
{
"purposes":[{
"purpose_desription": "The purpose is to share information on the honey production to be shared to third parties.",
"purpose_type": "sharing_production_information",
"service": "The service is farming.",
"soya_structure_dri": "DRI", # data_information is not needed
"cii_information":[{
"attribute_id":"organization_id",
"attribute_type": "identifier",
"sensitivity": ""
"data_optional": "false"
},
{
"attribute_id":"address",
"attribute_type": "identifier",
"sensitivity": ""
"data_optional": "false"
},
{
"attribute_id":"bee_keeper_id",
"attribute_type": "identifier",
"sensitivity": "csi"
"data_optional": "false"
},
{
"attribute_id":"produced_by",
"attribute_type": "identifier",
"sensitivity": ""
"data_optional": "false"
},
{
"attribute_id":"production_batch_volume",
"attribute_type": "characteristics",
"sensitivity": "csi"
"data_optional": "false"
},
{
"attribute_id":"production_batch_date",
"attribute_type": "characteristics",
"sensitivity": "csi"
"data_optional": "false"
},
{
"attribute_id":"bee_collection_area",
"attribute_type": "characteristics",
"sensitivity": "csi"
"data_optional": "false"
}],
"jurisdiction": "string",
"data_controllers": [{
"organization_id": "national id",
"name": "name",
"address": "string",
"party_type": "string"
}],
"data_owner": [{
"organization_id": "national id",
"name": "name",
"address": "string",
"party_type": "string"
}]
}],
}
```
## 2. Honey Shop
D3A - Honey Shop
What cii information to be retrieved from the Honey Bee Keeper.
```json=
{
"purposes":[{
"purpose_desription": "Provide information to the consumer when shopping for honey",
"purpose_type": "shopSellingOrganic",
"service": "Local Sustainable Products Inc sells locally produced products from the Austrian alps",
"cii_information":[
{
"attribute_id":"didxxxx",
"attribute_type": "producedBy",
"sensitivity": "no"
},
{
"attribute_id":"didxxxx",
"attribute_type": "productionBatchDate",
"sensitivity": "yes"
},
{
"attribute_id":"didxxxx",
"attribute_type": "beeCollectionArea",
"sensitivity": "no"
}],
"processing_method": "copy",
"retention_period": "P0003-00-00T00:00:00",
"geographic_restrictions": "EMA",
"jurisdiction": "AT",
"withdrawal_method": "Contact issuer of this agreement which is first organisation to sign agreement",
"authority_party": "Austrian government",
"data_controllers": [{
"organization_id": "national id",
"name": "name",
"address": "string",
"party_type": "string"
}],
"data_owner": [{
"organization_id": "national id",
"name": "name",
"address": "string",
"party_type": "string"
}]
}]
}
```
Questions:
1. Do we need a purpose specifically for shops selling honey or is it enough say shop selling products?
# Mapping between Usage Policy and D2A
| Field | Usage Policy (Honey) | D2A-P (Honey) | Usage Policy (DISP) | D2A-U (DISP) |
|-|-|-|-------|-------|
| purpose_desription | - | - | - | T:free text |
| purpose_type | (purpose)-prio | (x) | (purpose) | T / V min |
| processing_method | (processing)-prio | (x) | (processing) | T / V min |
| retention_period | (retentionPeriod) | (x) | (retentionPeriod) | T / V< |
| storage_location | (storageLocation) | (x) | (storageLocation) | T / V min |
| restrictions NOTE4 | (recipient) | (x) | (recipient) | T / V min |
| withdrawal_method FUTURE | - |- | - | T:free text |
| | | | |
| cii_information | - | - | - | T:NOTE1 |
| data_controller | - | - | NOTE2 | - |
| - jurisdiction | - | - | x | - |
| - authority_party | - | - | x | - |
| data_owner | NOTE3 | - | - |
| - jurisdiction | x | - | - | x |
| - authority_party | x | - | - | - | - |
NOTE1 - Data model set through SOyA, DRI is used as reference to governance in usage policy when creating service or organisation. DISP has sole responsibility of creating data model to be used in usage policy by bee keeper.
NOTE2 - Set by DISP. SOyA will set structure of organization information.
NOTE3 - Set by Honey Origin
NOTE4 - Restrictions can be geographic restrictions or other.
Questions:
1. Can there be a D2A template which Honey Origin chooses?
a. Governance could simply be a reference to template
2. Confirm who originates usage policy?
a. DIST gives templates
b. Honey Origin chooses from list when defining service
c. Honey Origin signs
d. DISP counter-signs
3. Can recipient be used for geographic restriction? Fashar
# References
## ISO consents
[ISO 27560 example](https://github.com/coolharsh55/dpv-consent-record/blob/master/record.json)
## DIF data agreements
[DIF Data Agreements schema](https://docs.google.com/spreadsheets/d/1GOFa8M-ioDCAMfRwe7D6sq_u0BZ0ww6C9UWQzmmtr9c/edit#gid=248840223) work under progress
## iGrant data agreements
[Data Agreement Specification original](https://github.com/decentralised-dataexchange/automated-data-agreements/blob/main/docs/data-agreement-specification.md)
[Data Disclosure Agreement github](https://github.com/decentralised-dataexchange/data-exchange-agreements/blob/main/docs/datadisclosure-agreement-specification.md#data-disclosure-agreement)
[Data Disclosure Agreement external](https://dda.igrant.io/)
Sign in with Wallet
Connect another wallet