EEG Use Case - HackMD

# EEG Use Case Version #1 #### Document Structure * [Description](#Description) * [Infrastructure](#Infrastructure) * [Identities](#Identities) * [Data Models](#Data-Models) * [Walk-Through](#Walk-Through) * [Sequence Diagram](#Sequence-Diagram) * [Step-by-Step Process](#Steps) * [Learnings & Planned Improvements](#Learnings-amp-Planned-Improvements) ## Description A research institute wants to access data on energy generated and consumed within an energy community. This access enables a more accurate assessment of the energy community’s role within a national energy framework, facilitating analyses of system efficiency, optimal system sizing, and the consumption patterns of community participants. A Data Intermediary will be used to facilitate secure and standardized data exchange, ensuring compliance with privacy and data protection regulations. **Challenges addressed:** * dynamic description of Data Agreements (data structure + form rendering) * flexible reasoner to check compliance between Data Agreements * ISO based contract management * infrastructure to manage digital artefacts (identities, data agreements, verifiable credentials, contracts, data itself) * Dashboard (UI framework) that integrates all components * identity management using government identity ### Infrastructure The following technical components are used: <details><summary>EEG Pod: Data Store for EEG data</summary> URL: https://eeg.go-data.at Github Repo: https://github.com/OwnYourData/dc-eeg Docker Image: https://hub.docker.com/r/oydeu/ </details> <details><summary>Dashboard: Intermediary Frontend</summary> URL: https://dashboard.go-data.at * Data Catalogue * My Contracts * Logs Github Repo: https://github.com/OwnYourData/dc-eeg Docker Image: https://hub.docker.com/r/oydeu/intermediary-frontend </details> <details><summary>Reasoner: Data Agreement Matching</summary> URL: https://reasoner.go-data.at/ * Swagger: https://reasoner.go-data.at/swagger-ui/index.html#/ Github Repo: https://github.com/OwnYourData/reasoner_service Docker Image: https://hub.docker.com/r/oydeu/reasoner </details>   ### Identities <details><summary>EEG User operates iGecos devices and provides data</summary> <ul> <li>uses Sphereon Mobile Wallet for signing data</li> <li>DID: <code>did:oyd:zQmeEPa5wZLD76WTp2CmV52Bu9UVZTVWBJJwkuaAR2yrUmK</code> (note: will differ based on DID in wallet)</li> <li>command: <code>echo '' | oydid create --doc-pwd eegpwd --rev-pwd eegrev -z 1</code></li> </ul> </details> <details><summary>Researcher scientist in research institute accessing EEG data</summary> <ul> <li>uses Sphereon Mobile Wallet for signing data</li> <li>DID: <code>did:oyd:zQmPp9uQx6dkAcFfoDCgbfzu9GPKwPt1fN8HvRxZoNHG8Gx</code> (note: will differ based on DID in wallet)</li> <li>command: <code>echo '' | oydid create --doc-pwd sripwd --rev-pwd srirev -z 1</code></li> </ul> </details> <details><summary>Data Intermediary public repository that can store data, DIDs, VCs, VPs</summary> <ul> <li>DID: <code>did:oyd:zQmSAxAeWkGDN1xm9eKWq547DTfdFYkgJGjpBs54Q5BqLx8</code></li> <li>command: <code>echo '' | oydid create --doc-pwd dipwd --rev-pwd direv -z 1</code></li> </ul> </details>   ### Data Models * [`EEGsmartMeterReading`](https://soya.ownyourdata.eu/EEGsmartMeterReading/yaml) - record provided via igecos ([edit](https://soya-form.ownyourdata.eu/?schemaDri=EEGsmartMeterReading&data=%7B%7D) | [example](https://soya-form.ownyourdata.eu/?schemaDri=EEGsmartMeterReading&data=%7B%22serial%22%3A%2230058326%22%2C%22timestamp%22%3A1708025835%2C%22powerIn%22%3A5%2C%22powerOut%22%3A0%2C%22energyIn%22%3A67252084%2C%22energyOut%22%3A0%2C%22pvPower%22%3A0%2C%22pvEnergy%22%3A0%2C%22voltageL1%22%3A2340%2C%22currentL1%22%3A6%2C%22voltageL2%22%3A2330%2C%22currentL2%22%3A0%2C%22voltageL3%22%3A2340%2C%22currentL3%22%3A0%7D)) * [`D2Aeeg`](https://soya.ownyourdata.eu/D2Aeeg/yaml) - Data Sharing Agreement for EEG ([edit](https://soya-form.ownyourdata.eu/?schemaDri=D2Aeeg)) * [`D3Aeeg`](https://soya.ownyourdata.eu/D3Aeeg/yaml) - Data Disclosure Agreement for recipient ([edit](https://soya-form.ownyourdata.eu/?schemaDri=D3Aeeg)) * [`OntologyEEG`](https://soya.ownyourdata.eu/OntologyEEG/yaml) - Ontology for D2A/D3A matching   ## Walk Through ### Sequence Diagram ```plantuml @startuml box "\nData Provider" #LightBlue actor "EEG\nContact" as provider participant "SSI\nWallet" as provider_wallet database "Data\nSource" as provider_store end box box "\nData Intermediary" #Yellow participant "Intermediary\nFrontend" as frontend database "Data\nStorage" as di_store end box box "\nData Consumer" #LightGreen database "Data\nSink" as consumer_store participant "SSI\nWallet" as consumer_wallet actor "Research\nInstitute" as consumer end box provider -> frontend: sign in with ID Austria provider -> frontend: create entry in "Data Catalog" frontend -> di_store: store D2A provider_wallet --> di_store: sign D2A consumer -> frontend: sign in with ID Austria consumer -> frontend: request access in "Data Catalog" frontend -> di_store: store D3A consumer_wallet --> di_store: sign D3A frontend -> frontend: validate\nD2A - D3A\ncompliance frontend --> consumer: give access to provider data source provider_store -> consumer_store: data exchange @enduml ``` ### Steps **1) Authentication** A user (EEG contact as data provider, or researcher as data consumer) logs in to the Data Intermediary using ID Austria: * Data Intermediary Frontend: https://dashboard.go-data.at (link for login: https://dashboard.go-data.at/api/login) * a login event is logged * after login the user can choose in the left panel to show the Data Catalogue (to perform steps 2 & 3), own contracts (step 4), and all logged events   **2) Data Source Setup by EEG** A data provider (EEG contact) can create a new data offering in the Data Catalogue by clicking "Add Data" * a dialog is shown with a form that describes the various aspects of the data offering: (a) description of the data provider, (b) the actual data source to share, and \(c) restriction for data consumers <details><summary>Example & curl</summary> ```json= {"data": { "subject": { "eegType": "Verein", "eegRegisteredNumber": "123", "contact": { "name": "John Doe", "addresse": "1234 Main Street, Anytown, CA 12345" } }, "consent": { "hasPurpose": [ "ResearchAndDevelopment", "PublicBenefit" ], "hasProcessing": [ "Disclose", "Store", "Use", "Transform" ], "hasExpiryTime": "6 Monate" }, "source": { "username": "intermediary", "password": "secret", "mqtt_host": "78.104.89.131", "mqtt_port": 1883 } } ``` </details> * when the user clicks "Submit" a new record is added to the Data Catalogue and the event is logged   **3) Data Request from Research Institute** A data consumer (Research Institute) selects a record from the Data Catalogue and requests "Access" * a dialog is shown with a form that describes the various aspects of the data request: (a) description of the data consumer, and (b) planned data usage <details><summary>Example & curl</summary> ```bash= echo '{"user_id": "bPK", "data": { "subject": { "eegType": "Körperschaft des öffentlichen Rechts", "eegRegisteredNumber": "456", "contact": { "name": "Jane Smith", "addresse": "5678 Oak Avenue, Sampletown, TX 67890" } }, "consent": { "hasPurpose": [ "ResearchAndDevelopment" ], "hasProcessing": [ "Store", "Use" ], "hasExpiryTime": "3 Monate" } }}' | curl -H "Content-Type: application/json" -d @- \ -X POST https://eeg.go-data.at/api/d3a_submit ``` </details> * when the user clicks "Submit" a number of steps are executed: a) store a copy of the D3A instance and documented as log entry b) compile the request for the reasoner with D2A and D3A * convert duration to timestamp * log reasoner request * log reasoner response <details><summary>Example for D2A/D3A matching</summary> ```bash= echo '{ "ontology": "https://soya.ownyourdata.eu/OntologyEEG", "d2aConsent": { "model": "D2Aeeg", "hasPurpose": [ "PublicBenefit", "ResearchAndDevelopment", "ServiceProvision" ], "hasProcessing": [ "Store", "Use" ], "hasExpiryTime": 15 }, "d3aConsent": { "model": "D3Aeeg", "hasPurpose": [ "ResearchAndDevelopment" ], "hasProcessing": [ "Store", "Organise", "Use" ], "hasExpiryTime": 123 } }' | curl -H "Content-Type: application/json" -d @- \ -X PUT https://reasoner.go-data.at/api/match ``` Response: ```json { "version": "1.0.0", "valid": false, "messages": [ "For the usage class hasProcessing, the value Organise is requested by the consumer but not covered by the provider", "The d2a value for hasExpiryTime must not be smaller than the one for d3a" ] } ``` </details>   **4) Contract Creation** If D2A/D3A match, a contract (based on [ISO 27560:2023](https://www.iso.org/standard/80392.html)) is compiled and the event is logged. (this step is automatically triggered in step #3 `POST /api/d3a_submit`) <details><summary>Example Contract</summary> ```json { "purpose": ["ResearchAndDevelopment"], "processing_method": ["Store", "Organise", "Use"], "lawful_basis": "consent", "content": "establish data exchange", "retention_period": "3 months", "storage_location": "EEC", "geographic_restriction": "EEC", "service": "data intermediation", "jurisdiction": "Vienna", "privacy_policy": "https://intermediary.at/datenschutz/", "withdrawal_method": "email to office@intermediary.at", "recipient_third_party": [{ "party_id": "did:oyd:zQm...", "credential": null, "party_role": "Data Intermediary", "party_type": "Company", "party_name": "DID Daten-Intermediär-Dienste FlexCo", "party_address": "Treustraße 52, 1200 Wien, Austria" },{ "party_id": "did:oyd:zQm...", "credential": null, "party_role": "Data Consumer", "party_type": "Körperschaft des öffentlichen Rechts", "party_name": "Forschung Burgenland", "party_address": "Campus 1, 7000 Eisenstadt, Austria" }], "data_controllers": [{ "party_id": "did:oyd:zQm...", "party_role": "Data Intermediary", "party_type": "Company", "party_name": "DID Daten-Intermediär-Dienste FlexCo", "party_address": "Treustraße 52, 1200 Wien, Austria" },{ "party_id": "did:oyd:zQm...", "party_role": "Data User", "party_type": "Körperschaft des öffentlichen Rechts", "party_name": "Forschung Burgenland", "party_address": "Campus 1, 7000 Eisenstadt, Austria" }], "event": { "time": "2024-10-30T12:00:00+02:00", "type": "implicit", "state": consent "entity_id": "did:oyd:zQm..DISP..", "reasoner": { "version": "1.0.0", "timestamp": 1725963104 }, "data_provider": [{ "party_id": "did:oyd:zQm..DataProvider..", "party_role": "Data Intermediary", "party_type": "Company", "party_name": "DID Daten-Intermediär-Dienste FlexCo", "party_address": "Treustraße 52, 1200 Wien, Austria" }], "data_intermediary": [{ "party_id": "did:oyd:zQm..DISP..", "party_role": "Data Intermediary", "party_type": "Company", "party_name": "DID Daten-Intermediär-Dienste FlexCo", "party_address": "Treustraße 52, 1200 Wien, Austria" }], "data_user": [{ "party_id": "did:oyd:zQm..DataUser..", "party_role": "Data User", "party_type": "Körperschaft des öffentlichen Rechts", "party_name": "Forschung Burgenland", "party_address": "Campus 1, 7000 Eisenstadt, Austria" }], "trigger": [] } } ``` </details>   **5) Data Exchange** The data consumer can access the credentials for data exchange in the last log message for creating the contract: ```json= { "mqtt_host": "78.104.89.131", "mqtt_port": 1883, "username": "intermediary", "password": "secret" } ```   ## Learnings & Planned Improvements **Domain-specific Findings** * Access data: provide in D3A only values defined by D2A * implement actual data exchange (not only credential sharing) through caching data in eeg.go-data.at * extend with data from EDA (compare live-data with data stored in EDA) - Scheiber Solutions data provider service * include forecast data - Scheiber Solutions data consumer service **Technical Findings** * consider [`https://adminlte.io/`](https://adminlte.io/) as GUI frontend * support 2 types of data (EEG + P2P) * update dc-intermediary to use JSON-B data type * trigger service for configurable actions after contract * EUDI wallet onboarding (DID for each user) * EUDI wallet signing * *DONE: Button to delete records* * Data Catalogue: Anzeige von Details zusätzlich zum Datum (Name, Beschränkungen) * Add-Data und Access (nicht im popup-öffnen sondern im frame) * contract: einzelne Vertragsdetails anzeigen im frame (vertragspartner)