# Gateway API Storage Provider Walkthrough
## 0 - Prerequisites
Use the credentials from a *dummy organisation* (ID: 77):
```bash=
export KEY="-6H7FHYo4aX5-dYMVF82x2_rzO1cXIB5URc24dPwMls"
export SECRET="cnhNgI77IMVeyenhZnPylcD_XKO72piWzzT68psVKJA"
export TOKEN=`curl -s -d grant_type=client_credentials -d client_id=$KEY -d client_secret=$SECRET -d scope=write -X POST https://babelfish.data-container.net/oauth/token | jq -r '.access_token'`
```
* current org: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/organization/current`
* current user: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/user/current`
* available collections: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/collection/list`
## 1 - Simple: store within Babelfish
*Note:* use the existing "Demo Collection" (ID: 99)
* `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/collection/99`
To write an object, the following steps must be performed:
1. create an object ([link](https://hackmd.io/faNBTCUcSRyQsLOf_Jhdag?view#Create-Object-metadata))
```bash=
echo '{"name":"test - simple", "collection-id": 99}' | \
curl -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @- -X POST https://babelfish.data-container.net/object/
```
new object-id: 752
* `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/752`
2. write payload ([link](https://hackmd.io/faNBTCUcSRyQsLOf_Jhdag?view#Write-Object-object-itself)):
```bash=
echo '{"I":"am the payload"}' | \
curl -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @- -X PUT https://babelfish.data-container.net/object/752/write
```
read the data:
* `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/752/read`
## 2 - AWS: store at Amazon
*Note:* this storage provider uses an Amazon account from OwnYourData to store data in the AWS cloud; configuration for the AWS storage provider backend is provided when starting the Gateway API in the following environment variables: `AWS_S3_KEY`, `AWS_S3_SECRET`
To write an object the following steps must be performed:
1. create a collection that configures the AWS storage provider
```bash=
echo '{"name":"AWS Storage Provider",
"storage":{"type": "AWS", "service-id": 340}}' | \
curl -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @- -X POST https://babelfish.data-container.net/collection
```
new collection-id: 756
* `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/collection/756`
2. create an object ([link](https://hackmd.io/faNBTCUcSRyQsLOf_Jhdag?view#Create-Object-metadata))
```bash=
echo '{"name":"test - AWS", "collection-id": 756}' | \
curl -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @- -X POST https://babelfish.data-container.net/object/
```
new object-id: 757
* `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/757`
3. write payload ([link](https://hackmd.io/faNBTCUcSRyQsLOf_Jhdag?view#Write-Object-object-itself)):
```bash=
echo '{"payload":"stored on AWS"}' | \
curl -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @- -X PUT https://babelfish.data-container.net/object/757/write
```
inspect:
* read data: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/757/read`
* meta data on AWS: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/757/read/meta`
* show in AWS: login to https://aws.amazon.com -> S3 -> Bucket "ontochain" -> check DRI matches meta data / show payload
## 3 - Bellecour: store at Amazon & Bellecour
*Note:* this section demonstrates writing data off-chain into the AWS cloud and additionally metadata on-chain on the Bellecour network
To write an object the following steps must be performed:
1. create a collection that configures the AWS and Bellecour storage provider
```bash=
echo '{"name":"AWS & Bellecour Storage Providers",
"storage":{"type": "AWS", "service-id": 340},
"dlt":{"type": "Bellecour", "service-id": 341}}' | \
curl -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @- -X POST https://babelfish.data-container.net/collection
```
new collection-id: 768
* `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/collection/768`
2. create an object ([link](https://hackmd.io/faNBTCUcSRyQsLOf_Jhdag?view#Create-Object-metadata))
```bash=
echo '{"name":"test - AWS+Bellecour", "collection-id": 768}' | \
curl -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @- -X POST https://babelfish.data-container.net/object
```
new object-id: 769
* `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/769`
3. write payload ([link](https://hackmd.io/faNBTCUcSRyQsLOf_Jhdag?view#Write-Object-object-itself)): *this command takes a few seconds to execute because the response waits for the confirmation from the [smart contract executed on Bellecour](https://blockscout-bellecour.iex.ec/address/0xF31A125fb44E0c2dca45c2665F272e9fc09f92AE/transactions#address-tabs)*
```bash=
echo '{"payload":"stored on AWS and Bellecour"}' | \
curl -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @- -X PUT https://babelfish.data-container.net/object/769/write
```
inspect:
* read data: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/769/read`
* meta data on AWS and Bellecour: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/769/read/meta`
* show in AWS: login to https://aws.amazon.com -> S3 -> Bucket "ontochain" -> check DRI matches meta data / show payload
* show in Bellecour: https://blockscout-bellecour.iex.ec/tx/0xa2f08541447fe11ec3519ede5ea7fe98826e786f4bfd1c2e3b8b1a202cc1a2ca
* payload DRI must match *Raw Input* shown in UTF-8
## 4 - Convex: store in Semantic Container & Convex
*Note:* in the final storage provider demo data will be stored off-chain in a dedicated [Semantic Container](https://www.ownyourdata.eu/semcon) and on-chain on the [Convex network](https://convex.world/)
To write an object the following steps must be performed:
1. provide relevant credentials for the current user to access Convex:
```bash=
echo '{"name":"admin",
"meta":{"dlt":[
{"type": "Convex",
"account":"48",
"seed":"7e4f674420a17067115cb26490351510cd25f0467795b725628e66d72d27ad4b"
}
]}
}' | \
curl -H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d @- -X PUT https://babelfish.data-container.net/user/89
```
inspect:
* show user wallet: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/user/89/wallet`
* show account on Convex: https://convex.world/testnet/account/48
* attach to account: https://convex.world/testnet/wallet
2. create a collection that configures the location of the Semantic Container and the Convex storage provider:
```bash=
echo '{"name":"SemCon and Convex Storage Providers",
"storage":{"type": "Semantic Container",
"service-id": 339,
"url": "https://storage-oc3.data-container.net"},
"dlt":{"type": "Convex", "service-id": 342}}' | \
curl -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @- -X POST https://babelfish.data-container.net/collection
```
new collection-id: 771
* `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/collection/771`
3. create an object ([link](https://hackmd.io/faNBTCUcSRyQsLOf_Jhdag?view#Create-Object-metadata))
```bash=
echo '{"name":"test - SemCon+Convex", "collection-id": 771}' | \
curl -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @- -X POST https://babelfish.data-container.net/object
```
new object-id: 775
* `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/775`
4. write payload ([link](https://hackmd.io/faNBTCUcSRyQsLOf_Jhdag?view#Write-Object-object-itself)):
```bash=
echo '{"payload":"stored on SemCon and Convex"}' | \
curl -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d @- -X PUT https://babelfish.data-container.net/object/775/write
```
inspect:
* read data: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/775/read`
* meta data: `curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/775/read/meta`
* show in SemCon: https://storage-oc3.data-container.net/api/data?id=5
* or with content-based address: https://storage-oc3.data-container.net/api/data?dri=zQmWFwzCtMTkk1tc3UtRbZ51qGLhB6JeKuJmuL8f7BjpD7w
* validate hash value on Convex:
* open Convex Sandbox: https://convex.world/sandbox
* be sure to attach to account: https://convex.world/testnet/wallet
- account: 48
- seed: `7e4f674420a17067115cb26490351510cd25f0467795b725628e66d72d27ad4b`
* run the following command in the sandbox
`(get n/documents 32)`
* the number after `documents` is the `convex-sequence` in the `dlt-repsonse`
* the `storage-hash` in the `dlt-repsonse` needs to be equal to the hash stored in Convex
* the storage-hash can be calculated manually by creating a string with payload-id and payload-hash from
`curl -H "Authorization: Bearer $TOKEN" https://babelfish.data-container.net/object/775`
* use e.g., SHA256 Online: https://emn178.github.io/online-tools/sha256.html
* calculate hash from `id:776,dri:zQmSaGcHgKoeJx678YUScjU1jAa43bXhHSgLtQoEzvuWWAY`
## Conclusion
In conclusion, selecting the appropriate storage provider for a Gateway API significantly influences the system's behaviour, reliability, and overall performance. This tutorial aimed to illuminate the available options, while also shedding light on various storage providers. Ultimately, a well-chosen storage provider will provide a solid foundation for your Gateway API, facilitating seamless data management and optimized operations.
Sign in with Wallet
Connect another wallet