W3C Web of Things - JSON Schema Case Study

After being approached by Juan Cruz Viotti and later by Ben Hutton from JSON Schema and Postman for writing a Case Study for JSON Schema, we as the WoT IG Marketing TF has started writing this. This is based on the template that is provided by them where we answer some questions first and then together with them turn it into a readable, flowing document.

Also see: https://github.com/w3c/wot-marketing/issues/329

Answers to Questions

Who is your organization, and where do you or your group/team sit within it?

Our organization is the World Wide Web Consortium (W3C), the standardization organization behind most of the standards used in the Web. We are the Web of Things Working and Interest Groups, who work on the standardization of Web of Things at the W3C.

What is the organization's mission? What is your group's/team's mission?

Create an interoperability layer so that different IoT platforms, protocols and standards can operate together.

The work has started as a discussion in the WoT Community Group in 2013-2014. It has proceed to an Interest Group in 2015 that has collected the use cases and defined the standards to be worked on. Since 2016, the working group is working on different standards on the Web of Things with the first publications of the Thing Description and Architecture recommendations in 2019.

What’s the organization's story that led to this work?

Ege: not sure about this. Let's solve https://github.com/w3c/wot-marketing/issues/18 at the same time.

How the challenge arose. What was the situation before the challenge presented?

There were, still are and will be proprieratary solutions for IoT problems. Without WoT, developers need to work with multiple protocols and API specification formats. This results in an integration work that is difficult to automate, resulting in repetitive tasks for the developers which result in non-scalable solutions as well.

Was there a solution which didn’t meet new requirements? (linked to the above question)

OpenAPI was considered but it was rather quickly dismissed given that :

It is focused on HTTP whereas IoT devices and platforms use other protocols as well or even multiple protocols.
It does not scale well when there are a variety of IoT devices with different APIs. This means that either the APIs have to aggregated into a single one (like a proxy) or an Open API definition with lots of different path elements will need to be written. This can be already seen in the Philips Hue HTTP API where even a somewhat small product portfolio of light and switch variations result in a difficult to understand API. Note: This is difficult to explain without examples.
It is not possible to semantically annotate affordances using already existing ontologies on the web via the use the semantic web technologies such as JSON-LD.
It is not possible group paths (affordances) by their physical meanings, like we have properties, actions and events in WoT. What this means is that API Specifications for Web APIs or even Event-driven APIs do not consider different modes of interaction based on their meaning in the physical world. Even when taking the same HTTP Method (let's say POST), it can be used for moving a robot arm (invoking an action in WoT) or changing the value of configuration parameter (writing to a property in WoT). We needed another abstraction for resources or chanels/topics other than paths or channels.

Considerations during research for a solution - What criteria did you use when deciding on potential solutions? What potential solutions did you decide were not as good a fit and why?

Ege: I think this overlaps with the above question

"A Ha!" moment - What was the moment or key feature of JSON Schema that meant it was the right solution for you to try?

It is difficult to name one, so here are a few:

The fact that it is JSON based, meaning it fits well with what we already began with, which is JSON-LD. JSON-LD was important for us because it builds upon JSON without losing its benefits but allows us to link other vocabularies and ontologies that already exist on the web (such as SAREF or Units of Measure or schema.org)
Even the first draft versions were quite enough to model data models of IoT devices. We are on board since draft 4. When we were in the feature freeze stage of our first publications in 2019, 2019-09 Draft was not out yet so we are currently on Draft 7. We will probably move to 2020-12 or a newer one if it is out when we start our new charter in February 2023.
Availability of tools, popularity (even then, it was popular for a just a schema language). This has two types of impacts:
- We do not need to specify a new schema language and then implement its parsers, validators, generators (Code, UI) in all possible programming languages that the community might be interested in. Community Adoption is not just something you can "engineer". All of this helps with the adoption of our standard.
- People who are familiar with JSON Schema have a smaller learning curve.
How do we use JSON Schema:
- We use it for validating WoT Thing Descriptions which are serialized in JSON-LD. Our official one is available at https://github.com/w3c/wot-thing-description/blob/main/validation/td-json-schema-validation.json .
- We use JSON Schema inside the Thing Description specification to model and describe the data sent by IoT devices. See examples at the end. This allows for:
  - Validating payloads
    - An IoT device can use its own schema to validate requests it receives
    - The Consumer (like an API Consumer, not the person who bought it) of an IoT device can use it to validate the payloads it receives from the device.
  - Building payloads
    - The TD Consumers need to send valid payloads to the device and for the person writing or choosing the payloads, JSON Schema is essential to correctly send the payloads.
    - One tool developed by the community uses JSON Schemas inside TDs to fake the payloads to be sent to the Things for stress and penetration testing purposes.
  - Building UIs: If we build a UI to interact with a Thing or a dashboard, the JSON Schemas in TDs help to generate basic template of a UI. E.g. generating a counter button instead of an empty field when we see "type":"number"

Wider influence? - Has your/your project's use of JSON Schema had influence in other areas of the organization?

Ege: we do not know. Need to ask W3C

A side question: Did the use of JSON Schema on WoT influence the use of JSON Schema in IoT in general?

Ege: difficult to comment on this.

What assurances do you now have? - Does JSON Schema provide you with any confidence looking forward to the future?

It is more and more stable with its core features not changing since a couple of years and more importantly, these features are more and more consistent accross implementations.

The vocabulary features are very promising to be used together with Semantic Web Technologies. We plan to explore this cooperation of technologies further.

Postman's support to the community gives more confidence that it will not die as a hobby project.

Additional meta-data if possible: Industry, location, Size… ?

Industry: Standardization, Internet of Things. In the end, any industry can profit from our work if that is the question's goal
Location: World
Size: Whole W3C: No idea, WoT WG: 96 representing 36 organizations, WoT IG: 131 participants representing 48 organizations. (This was the case in June 2022. In August 2022, the patent policy has changed and the people will probably need to rejoin, changing this number)

The questions below are added based on the comments of Juan Cruz Viotti

Challenges of using JSON Schema in the context of IoT

Even though that usual Web APIs are predominantly using JSON, IoT devices and platforms use other types of data as well, such as binary, CBOR, image types etc. However, we still want the users of W3C WoT to rely on JSON Schema as the primary way to model and describe their data before resorting to other ways, which decrease interoperability. Sometimes the data is modeled in a more complicated way, due to practices of the old times where saving every bit was important. E.g. a byte can contain the entire value for smaller values of a sensor but if a single bit is set to 1, the rest of the byte has another meaning, which is used for larger values.

Examples of the use of JSON Schema in WoT

Below are two Thing Descriptions. One has 3 affordances with HTTP and the other one has 1 affordance with MQTT.

Example 1































{
    "@context": "https://www.w3.org/2022/wot/td/v1.1",
    "id": "urn:uuid:0804d572-cce8-422a-bb7c-4412fcd56f06",
    "title": "MyLampThing",
    "securityDefinitions": {
        "basic_sc": {"scheme": "basic", "in": "header"}
    },
    "security": "basic_sc",
    "properties": {
        "status": {
            "type": "string",
            "enum":["on","off","error"],
            "readOnly":true,
            "forms": [{"href": "https://mylamp.example.com/status"}]
        }
    },
    "actions": {
        "toggle": {
            "forms": [{"href": "https://mylamp.example.com/toggle"}]
        }
    },
    "events":{
        "overheating":{
            "data": {"type": "string"},
            "forms": [{
                "href": "https://mylamp.example.com/oh",
                "subprotocol": "longpoll"
            }]
        }
    }
}

Explanation:

From this TD example, we know there exists one Property affordance with the title status. In addition, information is provided to indicate that this Property is accessible via (the secure form of) the HTTP protocol with a GET method at the URI https://mylamp.example.com/status (announced within the forms structure by the href member), and will return a string-based status value. The use of the GET method is not stated explicitly, but is one of the default assumptions defined by the TD specification

In a similar manner, an Action affordance is specified to toggle the switch status using the POST method on the https://mylamp.example.com/toggle resource, where POST is again a default assumption for invoking Actions.

The Event affordance enables a mechanism for asynchronous messages to be sent by a Thing. Here, a subscription to be notified upon a possible overheating event of the lamp can be obtained by using HTTP with its long polling subprotocol on https://mylamp.example.com/oh.

This example also specifies the basic security scheme, requiring a username and password for access. Note that a security scheme is first given a name in securityDefinitions and then activated by specifying that name in a security section. In combination with the use of the HTTP protocol this example demonstrates the use of HTTP Basic Authentication. Specification of at least one security scheme at the top level is mandatory, and gives the default access requirements for every resource. However, security schemes can also be specified per-form, with configurations given at the form level overriding configurations given at the Thing level, allowing for the specification of fine-grained access control.

Example 2


























{   
    "@context": "https://www.w3.org/2022/wot/td/v1.1",
    "title": "MyIlluminanceSensor",
    "id": "urn:uuid:9489991a-7622-45b6-8437-f858b59835d4",
    "securityDefinitions": {"nosec_sc": {"scheme": "nosec"}},
    "security": ["nosec_sc"],
    "events": {
        "illuminance": {
            "data":{
                "type": "integer",
                "minimum":0,
                "maximum":255,
                "multipleOf":5
            },
            "forms": [
                {
                    "href": "mqtt://192.168.1.187:1883",
                    "mqv:topic": "application/devices/thing1/illuminance",
                    "contentType": "application/json",
                    "mqv:controlPacket":"mqv:subscribe",
                    "op": "subscribeevent"
                }
            ]
        }
    } 
}

Explanation:

This is a simpler TD compared to the one above. It enables a consumer that is capable of MQTT to subscribe to the topic "application/devices/thing1/illuminance" at the MQTT broker with address mqtt://192.168.1.187:1883. The event data, as described by a JSON Schema, is an integer between 0 and 255, with multiple of 5 and is transmitted as JSON. Compared to the previous example, there is no need to provide any security credentials when connecting to the broker and executing the affordances.

How to get started with WoT?

There are many ways to get started. You might already have a WoT device next to you, just that maybe it did not come with a TD already. Some things you can do to get more familiar with WoT:

Try writing a TD for an IoT device that you know its API. Have a look at the examples provided in the TD specification. You can validate them at the TD Playground. You can then interact with it the WoT way by using node-wot or generate Node-RED nodes for it via the Node generator.
TODO: add another thing to do, like writing a server

JSON and the tooling around it was focusing on Web APIs. Since then, we see JSON being a first-class citizen on other industries such as IoT. What is your take on this? What is your vision of how you see IoT being JSON-first in the future?

Ege's comment: Difficult to comment here, I need to think. Also I am not sure if W3C would prefer taking a stance like this but why not. If we consider formats like CBOR or JSON BinPack which are very compatible with JSON, I don't see a limitation much. Maybe streaming related payloads will always exist.

Below is the template at https://github.com/json-schema-org/community/blob/main/docs/case-studies/template.md

Case Study Template

Preface

In looking to publish at least three case studies this year, we need a template of questions to ask organizations. Here are a list of questions which we should provide for each case study. The questions themselves should not be in the case study, but we should be able to put the resulting answers together should provide us with a mostly workable case study.

The list is mostly based on the case study format used at https://www.cncf.io/case-studies.

For each question or group of questions, before the dash is providing context, and would not be sent to the case study subject.

We should always seek final approval for the case study subject before publishing.

Questions

Challenge - What challenges were you encountering before you used JSON Schema?

Cristiano Aguzzi We can describe the IoT fragmentation problem regarding data payloads and data model. This should address the point the first bullet point in issue https://github.com/w3c/wot-marketing/issues/329#issuecomment-1174960085. Regarding the other points we can probably split the use case in at least two other (?): for example: Use case 1: Describing IoT payloads, Use case 2: validate IoT metadata descriptors (including also the issue of validating extensions and json-ld context), Use case 3 runtime interface typings generation (better name?).

Solution - In what way did JSON Schema solve this challenge?

Impact - What was the impact of the solution? What benefits have you seen?

Key impact results - Are there any metrics you used to evaluate the impact or success of the project using JSON Schema?

Cristiano Aguzzi I would put the answers to these question above in a preface or introduction. Not sure why they are here at the bottom.

Who is your organization, and where do you or your group/team sit within it?

What is the organization's mission? What is your group's/team's mission?

What’s the organization's story that led to this work?

How the challenge arose - What was the situation before the challenge presented? Was there a solution which didn’t meet new requirements?

Considerations during research for a solution - What criteria did you use when deciding on potential solutions? What potential solutions did you decide were not as good a fit and why?

"A Ha!" moment - What was the moment or key feature of JSON Schema that meant it was the right solution for you to try?

Wider influence? - Has your/your project's use of JSON Schema had influence in other areas of the organization?

What assurances do you now have? - Does JSON Schema provide you with any confidence looking forward to the future?

Additional meta-data if possible: Industry, location, Size… ?

Ege Korkan

2022/07/15 08:26:50

I would put the answers to these question above in a preface or introduction. Not sure why they are here at the bottom.

I think their main goal is to take these answers and rewrite it via their professional editors (Edited)

Juan Cruz Viotti

2022/07/18 22:08:26

I think this paragraph can be expanded a bit more to show people outside of IoT how painful making things communicate actually is. (Edited)

2022/07/18 22:08:35

Maybe worth even expanding on a small example? (Edited)

2022/07/18 22:09:26

It is focused on HTTP whereas IoT devices and platforms use other protocols as well or even multiple protocols. -

I think you would have to explain the fact that on IoT, a lot more protocols are popular. If somebody from the API world reads the case study, they might not know that that's the case, or why (Edited)

2022/07/18 22:10:12

It does not scale well when there are a variety of IoT devices with different APIs. This means that either the APIs have to aggregated into a single one (like a proxy) or an Open API definition with lots of different path elements will need to be written. This can be already seen in the Philips Hue HTTP API where even a somewhat small product portfolio of light and switch variations result in a difficult to understand API. Note: This is difficult to explain without examples. -

Why doesn't it scale well? Can you elaborate on that (Edited)

2022/07/18 22:11:10

It is not possible to semantically annotate affordances using already existing ontologies on the web via the use the semantic web technologies such as JSON-LD. -

I'm not sure I understand what this means (Edited)

2022/07/18 22:11:29

OpenAPI

Did you consider other OpenAPI alternatives like RAML? (Edited)

2022/07/18 22:12:02

It is not possible group paths (affordances) by their physical meanings, like we have properties, actions and events in WoT. What this means is that API Specifications for Web APIs or even Event-driven APIs do not consider different modes of interaction based on their meaning in the physical world. Even when taking the same HTTP Method (let's say POST), it can be used for moving a robot arm (invoking an action in WoT) or changing the value of configuration parameter (writing to a property in WoT). We needed another abstraction for resources or chanels/topics other than `paths` or `channels`. #

I think this one might not make much sense for people not familiar with IoT or WoT in general (Edited)

2022/07/18 22:12:57

The fact that it is JSON based, meaning it fits well with what we already began with, which is JSON-LD. JSON-LD was important for us because it builds upon JSON without losing its benefits but allows us to link other vocabularies and ontologies that already exist on the web (such as [SAREF](https://saref.etsi.org/) or [Units of Measure](http://www.ontology-of-units-of-measure.org/page/om-2) or [schema.org](https://schema.org/))

This one is a great point. Maybe you can include a paragraph about why and how you use JSON-LD, to then use as the basis to explain why JSON Schema made natural sense? (Edited)

2022/07/18 22:13:39

The fact that you have been using JSON Schema for a long time is a great story. Maybe you can comment on which year WoT was first published? (Edited)

2022/07/18 22:13:52

Also, can you explain what version of the JSON Schema draft WoT is currently using? (Edited)

2022/07/18 22:14:12

Create an interoperability layer so that different IoT platforms, protocols and standards can operate together. The work has started as a discussion in the WoT Community Group in

When did the work started? Is there a year you can mention? (Edited)

2022/07/18 22:14:42

I think a side question is: did the use of JSON Schema on WoT influence the use of JSON Schema in IoT in general? (Edited)

2022/07/18 22:14:55

It is more and more stable

In what way? (Edited)

2022/07/18 22:15:25

The vocabulary features are very promising

Are you planning to write a vocabulary at some point? If so, can you add some words about that? (Edited)

2022/07/18 22:16:03

Size: Whole W3C: No idea, WoT WG: 96 representing 36 organizations, WoT IG: 131 participants representing 48 organizations.

Interesting! I didn't know the WoT WG was so big! (Edited)

2022/07/18 22:16:51

Availability of tools

What type of JSON Schema do you use or are planning to use? That would be an interesting one (Edited)

2022/08/02 16:00:43

Not sure what you mean by type of JSON Schema. For validating TDs, we have one at https://github.com/w3c/wot-thing-description/blob/main/validation/td-json-schema-validation.json

2022/07/18 22:17:07

Also, do you use JSON Schema purely for validation purpose or also for annotation purposes? If so how? (Edited)

2022/07/18 22:17:33

I think it might also be worth adding a very small example of a WoT specification that shows some JSON Schema in use. I think people will be curious about how WoT actually looks like (Edited)

2022/07/18 22:18:15

Another one: I think you should include a section talking about how a reader finding WoT for the first time through the blog would get started using WoT, or learning more about it. Essentially, what are the next steps if somebody wants to know more or try it out? (Edited)

2022/07/18 22:19:45

Also, I think this touches on a very interesting topic, which is how a lot of the currently existent JSON-tooling historically focused very heavily on web APIs. Since then, we see JSON being a first-class citizen on other industries such as IoT. (Edited)

2022/07/18 22:20:03

Related to the last point, maybe worth sharing your vision of how you see IoT being JSON-first in the future? (Edited)

2022/07/18 22:21:08

I think you should probably also include a section on challenges using JSON Schema in the context of IoT. For example, how to validate binary data, etc. I think its also fine and interesting to let readers know about the limitations of JSON Schema that people would encounter in practice (Edited)

2022/08/02 09:47:23

I have added this as a new question and answered it (Edited)

2022/08/02 13:21:46

I have done the above points :) I have added the last two towards the end (Edited)

2022/08/02 13:30:09

Hmm I think RAML was known but not really considered. I am not sure (Edited)

2022/08/02 15:31:02

done :) (Edited)

2022/08/02 15:36:38

Regarding the last comment: That would be actually a pretty interesting point, a bold one too! I need to think for myself and I am not sure if W3C would like taking such a position. (Edited)

2022/08/02 15:43:16

I have extended this but not exactly satisfied, difficult to explain without examples. (Edited)

2022/08/09 11:36:39

JSON

We can say that anyone who uses WoT is using JSON Schema, at least for describing the payloads (Edited)

2022/08/09 11:38:50

According to https://www.w3.org/Consortium/Member/List , w3c has 464 members as of now (Edited)

2022/08/09 11:39:42

464 is the number of member companies, not individuals! (Edited)

2022/08/09 11:41:31

Also, thinking about that the whole world uses www in the end (Edited)