---
# System prepended metadata

title: Gateway API Storage Provider Walkthrough

---

# Gateway API Storage Provider Walkthrough

## 0 - Prerequisites

Use the credentials from a *dummy organisation* (ID: 77):

```bash=
export KEY="-6H7FHYo4aX5-dYMVF82x2_rzO1cXIB5URc24dPwMls"
export SECRET="cnhNgI77IMVeyenhZnPylcD_XKO72piWzzT68psVKJA"
export TOKEN=`curl -s -d grant_type=client_credentials -d client_id=$KEY -d client_secret=$SECRET -d scope=write -X POST https://babelfish.data-container.net/oauth/token | jq -r '.access_token'`
```

* current org: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/organization/current`
* current user: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/user/current`
* available collections: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/collection/list`

## 1 - Simple: store within Babelfish

*Note:* use the existing "Demo Collection" (ID: 99)
* `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/collection/99`

To write an object, the following steps must be performed:
1. create an object ([link](https://hackmd.io/faNBTCUcSRyQsLOf_Jhdag?view#Create-Object-metadata))
    ```bash=
    echo '{"name":"test - simple", "collection-id": 99}' | \
    curl -H "Authorization: Bearer $TOKEN" \
         -H "Content-Type: application/json" \
         -d @- -X POST https://babelfish.data-container.net/object/
    ```
    new object-id: 752
    * `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/752`
    
2. write payload ([link](https://hackmd.io/faNBTCUcSRyQsLOf_Jhdag?view#Write-Object-object-itself)):
    ```bash=
    echo '{"I":"am the payload"}' | \
    curl -H "Authorization: Bearer $TOKEN" \
         -H "Content-Type: application/json" \
         -d @- -X PUT https://babelfish.data-container.net/object/752/write
    ```
    read the data:
    * `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/752/read`

## 2 - AWS: store at Amazon

*Note:* this storage provider uses an Amazon account from OwnYourData to store data in the AWS cloud; configuration for the AWS storage provider backend is provided when starting the Gateway API in the following environment variables: `AWS_S3_KEY`, `AWS_S3_SECRET`

To write an object the following steps must be performed:
1. create a collection that configures the AWS storage provider
    ```bash=
    echo '{"name":"AWS Storage Provider", 
           "storage":{"type": "AWS", "service-id": 340}}' | \
    curl -H "Authorization: Bearer $TOKEN" \
         -H "Content-Type: application/json" \
         -d @- -X POST https://babelfish.data-container.net/collection
    ```

    new collection-id: 756
    * `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/collection/756`

2. create an object ([link](https://hackmd.io/faNBTCUcSRyQsLOf_Jhdag?view#Create-Object-metadata))
    ```bash=
    echo '{"name":"test - AWS", "collection-id": 756}' | \
    curl -H "Authorization: Bearer $TOKEN" \
         -H "Content-Type: application/json" \
         -d @- -X POST https://babelfish.data-container.net/object/
    ```
    new object-id: 757
    * `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/757`

3. write payload ([link](https://hackmd.io/faNBTCUcSRyQsLOf_Jhdag?view#Write-Object-object-itself)):
    ```bash=
    echo '{"payload":"stored on AWS"}' | \
    curl -H "Authorization: Bearer $TOKEN" \
         -H "Content-Type: application/json" \
         -d @- -X PUT https://babelfish.data-container.net/object/757/write
    ```
    inspect:
    * read data: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/757/read`
    * meta data on AWS: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/757/read/meta`
    * show in AWS: login to https://aws.amazon.com -> S3 -> Bucket "ontochain" -> check DRI matches meta data / show payload

## 3 - Bellecour: store at Amazon & Bellecour

*Note:* this section demonstrates writing data off-chain into the AWS cloud and additionally metadata on-chain on the Bellecour network

To write an object the following steps must be performed:
1. create a collection that configures the AWS and Bellecour storage provider
    ```bash=
    echo '{"name":"AWS & Bellecour Storage Providers", 
           "storage":{"type": "AWS", "service-id": 340},
           "dlt":{"type": "Bellecour", "service-id": 341}}' | \
    curl -H "Authorization: Bearer $TOKEN" \
         -H "Content-Type: application/json" \
         -d @- -X POST https://babelfish.data-container.net/collection
    ```

    new collection-id: 768
    * `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/collection/768`

2. create an object ([link](https://hackmd.io/faNBTCUcSRyQsLOf_Jhdag?view#Create-Object-metadata))
    ```bash=
    echo '{"name":"test - AWS+Bellecour", "collection-id": 768}' | \
    curl -H "Authorization: Bearer $TOKEN" \
         -H "Content-Type: application/json" \
         -d @- -X POST https://babelfish.data-container.net/object
    ```
    new object-id: 769
    * `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/769`

3. write payload ([link](https://hackmd.io/faNBTCUcSRyQsLOf_Jhdag?view#Write-Object-object-itself)): *this command takes a few seconds to execute because the response waits for the confirmation from the [smart contract executed on Bellecour](https://blockscout-bellecour.iex.ec/address/0xF31A125fb44E0c2dca45c2665F272e9fc09f92AE/transactions#address-tabs)*
    ```bash=
    echo '{"payload":"stored on AWS and Bellecour"}' | \
    curl -H "Authorization: Bearer $TOKEN" \
         -H "Content-Type: application/json" \
         -d @- -X PUT https://babelfish.data-container.net/object/769/write
    ```
    inspect:
    * read data: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/769/read`
    * meta data on AWS and Bellecour: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/769/read/meta`
    * show in AWS: login to https://aws.amazon.com -> S3 -> Bucket "ontochain" -> check DRI matches meta data / show payload
    * show in Bellecour: https://blockscout-bellecour.iex.ec/tx/0xa2f08541447fe11ec3519ede5ea7fe98826e786f4bfd1c2e3b8b1a202cc1a2ca
        * payload DRI must match *Raw Input* shown in UTF-8


## 4 - Convex: store in Semantic Container & Convex

*Note:* in the final storage provider demo data will be stored off-chain in a dedicated [Semantic Container](https://www.ownyourdata.eu/semcon) and on-chain on the [Convex network](https://convex.world/)

To write an object the following steps must be performed:

1. provide relevant credentials for the current user to access Convex:

    ```bash=
    echo '{"name":"admin", 
           "meta":{"dlt":[
               {"type": "Convex", 
                "account":"48",
                "seed":"7e4f674420a17067115cb26490351510cd25f0467795b725628e66d72d27ad4b"
               }
           ]}
          }' | \
    curl -H "Content-Type: application/json" \
         -H "Authorization: Bearer $TOKEN" \
         -d @- -X PUT https://babelfish.data-container.net/user/89
    ```
    inspect:
    * show user wallet: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/user/89/wallet`
    * show account on Convex: https://convex.world/testnet/account/48
    * attach to account: https://convex.world/testnet/wallet

2. create a collection that configures the location of the Semantic Container and the Convex storage provider:

    ```bash=
    echo '{"name":"SemCon and Convex Storage Providers", 
           "storage":{"type": "Semantic Container", 
                      "service-id": 339, 
                      "url": "https://storage-oc3.data-container.net"},
           "dlt":{"type": "Convex", "service-id": 342}}' | \
    curl -H "Authorization: Bearer $TOKEN" \
         -H "Content-Type: application/json" \
         -d @- -X POST https://babelfish.data-container.net/collection
    ```
    
    new collection-id: 771
    * `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/collection/771`

3. create an object ([link](https://hackmd.io/faNBTCUcSRyQsLOf_Jhdag?view#Create-Object-metadata))
    ```bash=
    echo '{"name":"test - SemCon+Convex", "collection-id": 771}' | \
    curl -H "Authorization: Bearer $TOKEN" \
         -H "Content-Type: application/json" \
         -d @- -X POST https://babelfish.data-container.net/object
    ```
    new object-id: 775
    * `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/775`

4. write payload ([link](https://hackmd.io/faNBTCUcSRyQsLOf_Jhdag?view#Write-Object-object-itself)):
    ```bash=
    echo '{"payload":"stored on SemCon and Convex"}' | \
    curl -H "Authorization: Bearer $TOKEN" \
         -H "Content-Type: application/json" \
         -d @- -X PUT https://babelfish.data-container.net/object/775/write
    ```
    inspect:
    * read data: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/775/read`
    * meta data: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/775/read/meta`
    * show in SemCon: https://storage-oc3.data-container.net/api/data?id=5
        * or with content-based address: https://storage-oc3.data-container.net/api/data?dri=zQmWFwzCtMTkk1tc3UtRbZ51qGLhB6JeKuJmuL8f7BjpD7w
    * validate hash value on Convex:
        * open Convex Sandbox: https://convex.world/sandbox  
        * be sure to attach to account: https://convex.world/testnet/wallet
            - account: 48
            - seed: `7e4f674420a17067115cb26490351510cd25f0467795b725628e66d72d27ad4b`
        *  run the following command in the sandbox
            `(get n/documents 32)`
            * the number after `documents` is the `convex-sequence` in the `dlt-repsonse`
            * the `storage-hash` in the `dlt-repsonse` needs to be equal to the hash stored in Convex
                * the storage-hash can be calculated manually by creating a string with payload-id and payload-hash from 
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/775`
                * use e.g., SHA256 Online: https://emn178.github.io/online-tools/sha256.html
                * calculate hash from `id:776,dri:zQmSaGcHgKoeJx678YUScjU1jAa43bXhHSgLtQoEzvuWWAY`
                
## Conclusion

In conclusion, selecting the appropriate storage provider for a Gateway API significantly influences the system's behaviour, reliability, and overall performance. This tutorial aimed to illuminate the available options, while also shedding light on various storage providers. Ultimately, a well-chosen storage provider will provide a solid foundation for your Gateway API, facilitating seamless data management and optimized operations.
