---
# System prepended metadata

title: SOyA4DIDs Paper
tags: [soya, im4dec]

---

# SOyA for Validating DID Documents

Advisor: [Prof Raj Rajarajan](https://www.city.ac.uk/about/people/academics/muttukrishnan-rajarajan) ([Google Scholar](https://scholar.google.com/citations?hl=de&user=gJ5yqXwAAAAJ))

### Links
* ONTOCHAIN/TRUSTCHAIN Paper: [Semantic Web and blockchain technologies: Convergence, challenges and research trends](https://drive.google.com/file/d/1jqWBSCofU6IJ49d_d2AyrA4xdOl9sj2j/view?usp=drive_link)

## Basic information for Paper

Intro: 
* no new standard, but using W3C Standards

1) problem to solve / state of the art
2) what are we doing and how is it different to the state of the art
    (3-4 specific things that are results/contributions)
3) explain exactyl what it is and what the novelty is
4) demonstration / results / impact

Evaluation: add Semantic Container to show benefits of SOyA
* automatic annotation of data model
* automatic validation
* automatic transformation

### Problem
* large number of DID Methods: 183 (as of Oct 10, 2023)
    DID Spec Registry: https://w3c.github.io/did-spec-registries/#did-methods
* criticism of divergence rather convergence and missing interoperability
    https://lists.w3.org/Archives/Public/public-new-work/2021Sep/0000.html
    
### Solution
* allow to validate any DID Document against [DID Core Spec v1.0](https://www.w3.org/TR/did-core/)
* visualize results on uniresolver.io (de-facto standard) for DID resolution
* provide a human- and machine-readable version of the DID Core Spec
    (question: should it be SHACL or JSON Schema?)

### Detailed Explanation
* description about Semantic Overlay Architecture (SOyA) from NGI ONTOCHAIN Paper "The Semantic Overlay Architecture for Data Interoperability and Exchange"
* process of creating the Validation Overlay for DID Documents
  * useful templating feature from YAML
  * comparison to TTL from DID Core Spec
    https://w3c.github.io/did-spec-registries/vocabs/v1/vocab.html
* handling of sets/lists
    https://ontola.io/blog/ordered-data-in-rdf

## Introduction Section of Paper
In the ever-evolving landscape of data management, the need for robust, efficient, and user-friendly tools has become paramount. The Semantic Overlay Architecture (SOyA) proposes a comprehensive solution, addressing this need by harmonizing a collection of best-in-class tools designed for data management. Central to SOyA is the principle of building upon established standards rather than reinventing the wheel. By building on the foundation of existing protocols and methodologies, SOyA provides a familiar yet optimized environment for users and promotes broader acceptance of RDF and related semantic web technologies within the data management community.

SOyA's architecture is designed to encompass the entire data lifecycle.
(1) Data Model Authoring: SOyA uses a YAML-based data model for describing graph data and available tools convert this into RDF and make it available in a public or private repository.
(2) Data Capture and Extraction: allows to convert flat JSON to JSON-LD and align ontologies for automated transformation between different representations
(3) Data Validation: SOyA uses SHACL (Shapes Constraint Language) for data validation, a powerful language for validating RDF data against a set of conditions.
(4) Data Transformation: multiple engines (SPARQL, Jolt, and jq) provide a versatile platform for transforming data into the desired formats and structures
(5) Data Rendering: for visualizing datasets SOyA incorporates JSON-Forms, a modern framework for creating form-based interfaces to enhance accessibility and readability of the data.
(6) Data Storage: Semantic Containers facilitate the storage of data and the integration of SOyA allows a standards-base, scalable, and reproducible approach for data handling

The focus in the development of SOyA is its ability to lower the entry barrier and simplifying the adoption of RDF and semantic web technologies which is still often perceived as difficult and only a niche technology. By offering a user-friendly interface and streamlined processes, SOyA makes Semantic Web tools accessible to a broader audience, fostering greater understanding and use across diverse fields. The integration of a suite of tools spanning the entire data lifecycle, from capture to storage, ensures reproducibility and consistency in data handling. Notably, SOyA does not confine itself to a specific data model but rather focuses on fostering interoperability among various data structures. A crucial aspect in today's data-driven economy where versatility and adaptability in data management are key to accommodating the diverse and evolving data needs of different domains.

To demonstrate SOyA's practical utility, this research explores its application in validating Decentralized Identifier (DID) documents against the W3C DID-Core standard. DIDs, emerging as a novel identifier enabling verifiable, self-sovereign digital identities, have seen significant growth, with over 180 distinct DID method implementations as of October 2023. SOyA's architecture is particularly well-suited for ensuring standard compliance within this diverse and rapidly evolving field. Utilizing its RDF authoring tool combined with SHACL validation capabilities, SOyA efficiently streamlines the validation of DID Documents. This application has provided invaluable insights to the Decentralized Identity Foundation (DIF) community, fostering harmonization efforts and led developers to align their DID implementations with established standards.


## Results
Tool: DIDLint (https://didlint.ownyourdata.eu)
#### Testrun from Oct 10th, 2023
* source for DIDs: [Uniresolver Test DIDs](https://github.com/decentralized-identity/universal-resolver/blob/main/uni-resolver-web/src/main/resources/application.yml)
* DID methods tested: 62
* DID Documents validated: 112
* statistics on 62 DID methods overall:
  * no response: 18
  * non compliant: 25
  * partially compliant: 6
  * compliant: 13
* statistics on 112 DID Documents:
  * valid: 51
  * invalid: 61
* validation based on SHACL (SOyA) / JSON Schema / JSON-LD:
  * SHACL (SOyA): 51
  * JSON Schema: 70
    * interesting: 6 DID Documents with invalid JSON Schema but valid SHACL!
  * JSON-LD: 27 (using [JSON-LD Playground](https://json-ld.org/playground/))

Open Questions: What are contributions?
* enforce interoperability
* use RDF SHACL instead of JSON Schema

problem > create scenario > show 3 benefits from SOyA


## SOyA Validation Overlay
Source: https://soya.ownyourdata.eu/Did/yaml
<details><summary>show full SOyA Structure (YAML)</summary>

```yaml=
meta:
  name: Did
  namespace:
    did: "https://www.w3.org/ns/did#"

content:
  bases: 
    - name: Did #from: https://www.w3.org/TR/did-core/
      subClassOf: "did:DIDSubject"
      attributes: 
        id: String
        alsoKnownAs: String
        verificationMethod: set<VerificationMethod>
        assertionMethod: set<String>
        authentication: set<String>
        capabilityDelegation: set<String>
        capabilityInvocation: set<String>
        keyAgreement: set<String>
        controller: String
        service: set<Service>
    - name: VerificationMethod
      subClassOf: "did:VerificationMethod"
      attributes:
        id: String
        controller: String
        type: String
        publicKeyJwk: String
        publicKeyBase58: String
        publicKeyMultibase: String
        blockchainAccountId: String
        publicKeyPgp: String
        publicKeyHex: String
        ethereumAddress: String
    - name: Service
      subClassOf: "did:Service"
      attributes:
        id: String
        type: String
        serviceEndpoint: String

  overlays: 
    - type: OverlayValidation
      base: Did
      name: DidValidationOverlay
      attributes:
        id: 
          cardinality: "1..1"
          pattern: "^did:[a-z0-9]+:[a-zA-Z0-9.:\\-_%]+$"
          message: "there must be one valid 'id' in a DID document"
        verificationMethod: #from: https://www.w3.org/TR/did-spec-registries/#verification-method-types
          message: "invalid 'verificationMethod'"
          attributes:
            or: 
              - &id_jwk
                id: &cardinality1_pattern
                  cardinality: "1..1"
                  pattern: "^did:[a-z0-9]+:[a-zA-Z0-9.:\\-_%?=#]+$"
                type:
                  cardinality: "1..1"
                  valueOption:
                    - EcdsaSecp256k1RecoveryMethod2020
                    - EcdsaSecp256k1VerificationKey2019
                    - JsonWebKey2020
                    - JSONWebSignature2020
                    - RsaVerificationKey2018
                controller: &cardinality1
                  cardinality: "1..1"
                publicKeyJwk: *cardinality1
              - &id_base58
                id: *cardinality1_pattern
                type:
                  <<: *cardinality1
                  valueOption:
                    - Bls12381G1Key2020
                    - Bls12381G2Key2020
                    - EcdsaSecp256k1RecoveryMethod2020
                    - EcdsaSecp256k1VerificationKey2019
                    - Ed25519VerificationKey2018
                    - VerifiableCondition2021
                    - X25519KeyAgreementKey2019
                controller: *cardinality1
                publicKeyBase58: *cardinality1
              - &id_multibase
                id: *cardinality1_pattern
                type:
                  <<: *cardinality1
                  valueOption:
                    - EcdsaSecp256k1VerificationKey2019
                    - Ed25519VerificationKey2020
                controller: *cardinality1
                publicKeyMultibase:  *cardinality1
              - &id_blockchain_account_id
                id: *cardinality1_pattern
                type:
                  <<: *cardinality1
                  valueOption:
                    - EcdsaSecp256k1VerificationKey2019
                controller: *cardinality1
                blockchainAccountId: *cardinality1
              - &id_public_key_pgp
                id: *cardinality1_pattern
                type:
                  <<: *cardinality1
                  valueOption:
                    - PgpVerificationKey2021
                controller: *cardinality1
                publicKeyPgp: *cardinality1
              - &id_public_key_hex
                id: *cardinality1_pattern
                type:
                  <<: *cardinality1
                  valueOption:
                    - EcdsaSecp256k1RecoveryMethod2020
                controller: *cardinality1
                publicKeyHex: *cardinality1
              - &id_ethereum_address
                id: *cardinality1_pattern
                type:
                  <<: *cardinality1
                  valueOption:
                    - EcdsaSecp256k1RecoveryMethod2020
                controller: *cardinality1
                ethereumAddress: *cardinality1
        authentication:
          message: "invalid 'authentication'"
          attributes: &verification_method_attributes
            or:
              - datatype: String
              - *id_jwk
              - *id_base58
              - *id_multibase
              - *id_blockchain_account_id
              - *id_public_key_pgp
              - *id_public_key_hex
              - *id_ethereum_address
        assertionMethod:
          message: "invalid 'assertionMethod'"
          attributes: *verification_method_attributes
        keyAgreement:
          message: "invalid 'keyAgreement'"
          attributes: *verification_method_attributes
        capabilityInvocation:
          message: "invalid 'capabilityInvocation'"
          attributes: *verification_method_attributes
        capabilityDelegation:
          message: "invalid 'capabilityDelegation'"
          attributes: *verification_method_attributes
        or:
          - verificationMethod: &verification_method_id
              cardinality: "1..n"
              attributes:
                id: 
                  *cardinality1_pattern
          - authentication: *verification_method_id
          - assertionMethod: *verification_method_id
          - keyAgreement: *verification_method_id
          - capabilityInvocation: *verification_method_id
          - capabilityDelegation: *verification_method_id
        service:
          attributes:
            id:
              cardinality: "1..1"
              message: "there must be one 'id' in 'service'"
            type:
              cardinality: "1..1"
              message: "there must be one 'type' in 'service'"
            serviceEndpoint:
              cardinality: "1..1"
              message: "there must be one 'serviceEndpoint' in 'service'"

    - type: OverlayDidContextValidation
      base: Did
      name: DidContextValidationOverlay
      constraints:
        - id: "*"
          context: "https://www.w3.org/ns/did/v1"
        - type: JsonWebKey2020
          context: "https://w3id.org/security/suites/jws-2020/v1"
        - type: EcdsaSecp256k1VerificationKey2019
          context: "https://w3id.org/security/suites/secp256k1-2019"
        - type: Ed25519VerificationKey2018
          context: "https://w3id.org/security/suites/ed25519-2018/v1"
        - type: Ed25519VerificationKey2020
          context: "https://w3id.org/security/suites/ed25519-2020/v1"
        - type: Bls12381G1Key2020
          context: "https://w3id.org/security/suites/bls12381-2020/v1"
        - type: Bls12381G2Key2020
          context: "https://w3id.org/security/suites/bls12381-2020/v1"
        - type: PgpVerificationKey2021
          context: "https://w3id.org/security/suites/pgp-2021/v1"
        - type: RsaVerificationKey2018
          context: "https://www.w3.org/2018/credentials/v1"
        - type: X25519KeyAgreementKey2019
          context: "http://w3id.org/security/suites/x25519-2019/v1"
        - type: EcdsaSecp256k1RecoveryMethod2020
          context: "https://w3id.org/security/suites/secp256k1recovery-2020"
        - type: VerifiableCondition2021
          context: "https://w3c-ccg.github.io/verifiable-conditions/contexts/verifiable-conditions-2021-v1.json"
```
  
</details>

## JSON Schema Validation
Source: https://github.com/w3c/did-spec-registries/tree/main/json_schemas
<details><summary>show full JSON Schema</summary>
  
```json=
{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "$id": "https://w3c.github.io/did-spec-registries/schemas/core.schema.json",
    "title": "DID Core Vocabulary schema",
    "$comment": "The JWK.schema part is a copy of the JWK schema (https://preview.npmjs.com/package/jwk-schema), published by Jonathan Wilbur (https://github.com/JonathanWilbur)",
    "definitions": {
        "did.schema" : {
            "type" : "string",
            "pattern" : "^did:[a-z0-9]+:[a-zA-Z0-9.-_:]+"
        },

        "type.schema" : {
            "type" : [
                "string",
                "array"
            ],
            "uniqueItems": true,
            "items" : {
                "type" : "string"
            }
        },

        "JWK.schema" : {
            "type" : "object",
            "properties": {
                "kty": {
                    "type": "string"
                },
                "use": {
                    "type": "string"
                },
                "key_ops": {
                    "type": "array",
                    "items": {
                        "type": "string"
                    }
                },
                "alg": {
                    "type": "string"
                },
                "kid": {
                    "type": "string"
                },
                "x5u": {
                    "type": "string"
                },
                "x5c": {
                    "type": "array",
                    "items": {
                        "type": "string"
                    }
                },
                "x5t": {
                    "type": "string"
                },
                "x5t#S256": {
                    "type": "string"
                }
            },
            "required": ["kty"],
            "additionalProperties": true
        },

        "one_verification_method.schema" : {
            "type" : "object",
            "properties": {
                "publicKeyJwk" : {
                    "$ref" : "#/definitions/JWK.schema"
                },
                "publicKeyMultibase" : {
                    "type" : "string"
                },
                "controller" : {
                    "$ref" : "#/definitions/did.schema"
                },
                "id" : {
                    "type": "string",
                    "format" : "uri-reference"
                },
                "type" : {
                    "$ref" : "#/definitions/type.schema"
                }
            },
            "required" : ["id", "type", "controller"],
            "not": {
                "required" : ["publicKeyJwk","publicKeyMultibase"]
            },
            "additionalProperties" : true
        },

        "verification_method_strict.schema" : {
            "type" : "array",
            "uniqueItems": true,
            "items" : {
                "$ref" : "#/definitions/one_verification_method.schema"
            }
        },

        "verification_method_with_urls.schema" : {
            "type" : "array",
            "uniqueItems": true,
            "items" : {
                "oneOf" : [
                    {
                        "type": "string",
                        "format" : "uri-reference"
                    },
                    {
                        "$ref" : "#/definitions/one_verification_method.schema"
                    }
                ]
            }
        },

        "service.schema" : {
            "type" : "array",
            "uniqueItems": true,
            "items" : {
                "type" : "object",
                "properties" : {
                    "id" : {
                        "type" : "string",
                        "format" : "uri-reference"
                    },
                    "type" : {
                        "$ref" : "#/definitions/type.schema"
                    },
                    "serviceEndpoint" : {
                        "oneOf" : [
                            {
                                "type": "string",
                                "format" : "uri-reference"
                            },
                            {
                                "type" : "object"
                            },
                            {
                                "type" : "array",
                                "uniqueItems": true,
                                "items": {
                                    "oneOf" : [
                                        {
                                            "type": "string",
                                            "format" : "uri-reference"
                                        },
                                        {
                                            "type" : "object"
                                        }
                                    ]
                                }
                            }
                        ]
                    }
                },
                "required": ["id", "type", "serviceEndpoint"],
                "additionalProperties" : true
            }
        }
    },

    "type": "object",
    "properties": {
        "id" : {
            "$ref" : "#/definitions/did.schema"
        },
        "alsoKnownAs" : {
            "type" : "array",
            "uniqueItems": true,
            "items": {
                "type" : "string",
                "format" : "uri-reference"
            }
        },
        "controller" : {
            "$ref" : "#/definitions/did.schema"
        },
        "verificationMethod" : {
            "$ref" : "#/definitions/verification_method_strict.schema"
        },
        "authentication" : {
            "$ref" : "#/definitions/verification_method_with_urls.schema"
        },
        "assertionMethod" : {
            "$ref" : "#/definitions/verification_method_with_urls.schema"
        },
        "keyAgreement" : {
            "$ref" : "#/definitions/verification_method_with_urls.schema"
        },
        "capabilityDelegation" : {
            "$ref" : "#/definitions/verification_method_with_urls.schema"
        },
        "capabilityInvocation" : {
            "$ref" : "#/definitions/verification_method_with_urls.schema"
        },
        "service" : {
            "$ref" : "#/definitions/service.schema"
        }
    },
    "required" : ["id"],
    "additionalProperties" : true
}
```
  
</details>
  