# 1. Log in to get the token from a REST endpoint
You need a user in the Pipeline `STG` environment:
*username: some-username
password: some-password
e-mail: email@gmail.com*
Login can be done via POST request to the server:
```bash
curl --request POST 'https://stg.jetstream.seeen.com/api/login/' \
--form 'username=some-username' \
--form 'password=some-password'
```
Output (example token):
`{"token":"1234567890abcdef"}`
You will use this token in the further requests to the GraphQL API.
# 2. Using the Pipeline GraphQL endpoint
`STG` environment GraphQL API endpoint: https://stg.jetstream.seeen.com/graphql/
## 0) What's available at the endpoint? (schema introspection - optional)
### List all available types
```graphql
{
__schema {
types {
name
}
}
}
```
Using cURL:
```bash
curl --request POST 'https://stg.jetstream.seeen.com/graphql/' \
--header 'Authorization: Token 1234567890abcdef' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"{\n __schema {\n types {\n name\n }\n }\n}"}'
```
### List fields in chosen type (e.g. PipelineExecutionType)
```graphql
{
__type(name: "PipelineExecutionType") {
name
fields {
name
type {
name
kind
}
}
}
}
```
Using cURL:
```bash
curl --request POST 'https://stg.jetstream.seeen.com/graphql/' \
--header 'Authorization: Token 1234567890abcdef' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"{\n __type(name: \"PipelineExecutionType\") {\n name\n fields {\n name\n type {\n name\n kind\n }\n }\n }\n}","variables":{}}'
```
More about introspection: https://graphql.org/learn/introspection/
## 1) Uploading video to the pipeline
This is the first needed step to have a video processed by the pipeline.
Example video URL: https://cs-backend-stg-storage.s3.amazonaws.com/public-media/videos/000042/video.mp4
### Upload the video
First, you need to send a direct video link to the Pipeline to start the download. Notice that the Authorization token needs to be placed in the headers. Query to the API (it should be in the request body):
```graphql
mutation VideoUploadMutation($input: VideoUploadMutationInput!) {
videoUpload(input: $input) {
video {
id
status
}
}
}
```
`variables = {"input": {"url": "https://cs-backend-stg-storage.s3.amazonaws.com/public-media/videos/000042/video.mp4"}}`
Using cURL:
```bash
curl --request POST 'https://stg.jetstream.seeen.com/graphql/'
--header 'Content-Type: application/json'
--header 'Authorization: Token 1234567890abcdef'
--data-raw '{"query":"mutation VideoUploadMutation($input: VideoUploadMutationInput!) {\n videoUpload(input: $input) {\n video {\n id\n status\n }\n }\n}\n","variables":{"input":{"url":"https://cs-backend-stg-storage.s3.amazonaws.com/public-media/videos/000042/video.mp4"}}}'
```
Example response:
```json
{
"data": {
"videoUpload": {
"video": {
"id": "VmlkZW99999999ZZZZZZZZZZZZ",
"status": "IDLE"
}
}
}
}
```
Save the **video id** returned by this request - it will be needed in the next steps.
### Check the video upload status
The video is being uploaded now. You need the following query for upload status checking. Once the upload is done, it will be `COMPLETED`. Use the **video id** from the previous step.
```graphql
query VideoDownload($id: ID!){
videoDownload(id: $id) {
status
}
}
```
`variables = {"id": "VmlkZW99999999ZZZZZZZZZZZZ"}`
Using cURL:
```bash
curl --request POST 'https://stg.jetstream.seeen.com/graphql/' \
--header 'Authorization: Token 1234567890abcdef' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"query VideoDownload($id: ID!){\n videoDownload(id: $id) {\n status\n }\n}\n","variables":{"id":"VmlkZW9Eb3dubG9hZFR5cGU6NTky"}}'
```
Example response:
```json
{
"data": {
"videoDownload": {
"status": "COMPLETED"
}
}
}
```
## 2) Submitting video for Pipeline Execution
Pipeline Execution means performing all the analysis of the video.
### Get the newest Pipeline Version
Pipelines may have different names - we are using the one called ***"Internal Pipeline"***.
The given Pipeline definition may change over time (for e.g. new tasks can be added), then its version is updated. We normally want to use the newest available version of the Pipeline.
Querying the newest Pipeline Version of the "Internal Pipeline":
```graphql
query LatestPipelineVersion($pipelineName: String!) {
pipelineVersions(orderBy: "-versionNumber", pipelineName: $pipelineName, first: 1) {
edges {
node {
id
}
}
}
}
```
`variables = {"pipelineName": "Internal Pipeline"}`
Using cURL:
```bash
curl --request POST 'https://stg.jetstream.seeen.com/graphql/' \
--header 'Authorization: Token 1234567890abcdef' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"query LatestPipelineVersion($pipelineName: String!) {\n pipelineVersions(orderBy: \"-versionNumber\", pipelineName: $pipelineName, first: 1) {\n edges {\n node {\n id\n }\n }\n }\n}\n","variables":{"pipelineName":"Internal Pipeline"}}'
```
Example response:
```json
{
"data": {
"pipelineVersions": {
"edges": [
{
"node": {
"id": "UGlwZWDwfxpbmVWZXuVHlwZToyOQ=="
}
}
]
}
}
}
```
Save the **pipeline version id** for further steps.
### Start Pipeline Execution
We will need the saved **pipeline version id** and **video id**.
```graphql
mutation Execute($input: PipelineExecutionMutationInput!){
pipelineExecute(input: $input) {
pipelineExecution {
id
state
}
errors {
field
code
message
}
}
}
```
`variables = {"input": {"pipelineVersionId": "UGlwZWDwfxpbmVWZXuVHlwZToyOQ==", "videoId": "VmlkZW99999999ZZZZZZZZZZZZ"}}`
Using cURL:
```bash
curl --request POST 'https://stg.jetstream.seeen.com/graphql/' \
--header 'Authorization: Token 1234567890abcdef' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"mutation Execute($input: PipelineExecutionMutationInput!){\n pipelineExecute(input: $input) {\n pipelineExecution {\n id\n state\n }\n errors {\n field\n code\n message\n }\n }\n}","variables":{"input":{"pipelineVersionId":"UGlwZWDwfxpbmVWZXuVHlwZToyOQ==","videoId":"VmlkZW99999999ZZZZZZZZZZZZ"}}}'
```
Example of a successfull response indicating start of the execution:
```json
{
"data": {
"pipelineExecute": {
"pipelineExecution": {
"id": "UGlwZWxpbmVFeGVjXXXXXXXXUeXBlOjYyNA==",
"state": "CREATED"
},
"errors": null
}
}
}
```
As you can see, we are querying the `errors` field also. Example of an unsuccessfull response:
```json
{
"data": {
"pipelineExecute": {
"pipelineExecution": null,
"errors": [
{
"field": "pipelineVersionId",
"code": "PIPELINE_VERSION_ID_DOES_NOT_EXIST",
"message": "Pipeline version with id XXXXXXXXXXXXX does not exist."
}
]
}
}
}
```
Pipeline Execution started. The processing will take some time. Save the **pipeline execution id** for further use.
### Checking Pipeline Execution state
Similarly as in the video upload you need to be checking for the Pipeline Execution state untill it's `COMPLETED`.
Query:
```graphql
query PipelineExecution($id: ID!){
pipelineExecution(id: $id) {
state
}
}
```
`variables = {"id": "UGlwZWxpbmVFeGVjXXXXXXXXUeXBlOjYyNA=="}`
Using cURL:
```bash
curl --request POST 'https://stg.jetstream.seeen.com/graphql/' \
--header 'Authorization: Token 1234567890abcdef' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"query PipelineExecution($id: ID!){\n pipelineExecution(id: $id) {\n state\n }\n}","variables":{"id":"UGlwZWxpbmVFeGVjXXXXXXXXUeXBlOjYyNA=="}}'
```
Response when still running:
```json
{
"data": {
"pipelineExecution": {
"state": "RUNNING"
}
}
}
```
Once the Pipeline Execution is `COMPLETED` you will be able to retrieve its results.
## 3) Retrieving Pipeline Execution results
### Annotations
Annotation is an object in the pipeline's database. It can come from different sources (ML classifiers output, Google Vision, Clarifai...) and it's a standard format for annotating videos in the pipeline.
It is possible to retrieve Annotations for the given Pipeline Execution - this way you will get the results of the given Pipeline Execution.
Basic Annotations query with all the available fields:
```graphql
query pipelineExecution($id: ID!) {
pipelineExecution(id: $id) {
annotations {
edges {
node {
name
timeOffset
score
boundingBox
annotationType
classType
trackId
}
}
}
}
}
```
`variables = {"id": "UGlwZWVjdXRpb25UeXBlOjYyNA=="}`
Using cURL:
```bash
curl --request POST 'https://stg-eai-pipeline.tg.10clouds.io/graphql/' \
--header 'Authorization: Token 1234567890abcdef' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"query pipelineExecution($id: ID!) {\n pipelineExecution(id: $id) {\n annotations {\n edges {\n node {\n name\n timeOffset\n score\n boundingBox\n annotationType\n classType\n trackId\n }\n }\n }\n }\n}\n# {\n# __type(name: \"PipelineExecutionType\") {\n# name\n# fields {\n# name\n# args {\n# name\n# }\n# }\n# }\n# }\n# {\n# __schema {\n# queryType {\n# fields {\n# name\n# description\n# }\n# }\n# }\n# }\n\n","variables":{"id":"UGlwZWVjdXRpb25UeXBlOjYyNA=="}}'
```
Part of an example response:
```json
{
"data": {
"pipelineExecution": {
"annotations": {
"edges": [
{
"node": {
"name": "Hispanic",
"timeOffset": 28695,
"score": 0.59122276,
"boundingBox": "BoundingBox(x=0.178125, y=0.13333333333333333, width=0.090625, height=0.19444444444444445)",
"annotationType": "OBJECT",
"classType": "FACE_ETHNICITY",
"trackId": 24
}
},
{
"node": {
"name": "technology",
"timeOffset": 28000,
"score": 0.83774865,
"boundingBox": null,
"annotationType": "TAG",
"classType": null,
"trackId": null
}
}
]
}
}
}
}
```
### a) Filtering by `timeOffsetRange`
You can filter Annotations by the `timeOffsetRange` (in miliseconds). It needs to be represented by a string with two integers separated by a comma, for e.g.: ``"0,1000"`` or ``"5000,10000"``.
Example query with `timeOffsetRange` filter:
```graphql
query pipelineExecution($id: ID!, $timeOffsetRange: String) {
pipelineExecution(id: $id) {
annotations(timeOffsetRange: $timeOffsetRange) {
edges {
node {
name
timeOffset
}
}
}
}
}
```
`variables = {"id": "UGlwZWVjdXRpb25UeXBlOjYyNA==", "timeOffsetRange": "0,200"}`
Using cURL:
```bash
curl --location --request POST 'https://stg.jetstream.seeen.com/graphql/' \
--header 'Authorization: Token 1234567890' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"query pipelineExecution($id: ID!, $timeOffsetRange: String) {\n pipelineExecution(id: $id) {\n annotations(timeOffsetRange: $timeOffsetRange) {\n edges {\n node {\n name\n timeOffset\n }\n }\n }\n }\n}","variables":{"id":"UGlwZWVjdXRpb25UeXBlOjYyNA==","timeOffsetRange":"0,200"}}'
```
Will return Annotations with timeOffset between 0 and 200 (inclusive!) miliseconds.
As we see above, the type of `$timeOffsetRange` variable is `String`.
### b) Filtering by `annotationType`
Available Annotation Types:
```python
OBJECT = "O"
TAG = "T"
FACE_PROPERTIES = "F"
FACE_NAME = "N"
ENTITY = "E"
SOUND = "S"
```
Example query with filters:
```graphql
query pipelineExecution($id: ID!, $annotationType: String!) {
pipelineExecution(id: $id) {
annotations(annotationType: $annotationType) {
edges {
node {
name
timeOffset
}
}
}
}
}
```
`variables = {"id": "pipeline-execution-id", "annotationType": "T"}`
Using cURL:
```bash
curl --location --request POST 'https://stg.jetstream.seeen.com/graphql/' \
--header 'Authorization: Token 1234567890' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"query pipelineExecution($id: ID!, $annotationType: String!) {\n pipelineExecution(id: $id) {\n annotations(annotationType: $annotationType) {\n edges {\n node {\n name\n timeOffset\n }\n }\n }\n }\n}","variables":{"id":"pipeline-execution-id","annotationType":"T"}}'
```
Will filter the "TAG" Annotations only.
### c) Filtering by `classType`
Available Class Types (only in Face Detection Annotations!):
```python
FACE_AGE = "A"
FACE_ETHNICITY = "E"
FACE_GENDER = "G"
FACE_EMOTION = "M"
```
Example query with filters:
```graphql
query pipelineExecution($id: ID!, $classType: String!) {
pipelineExecution(id: $id) {
annotations(classType: $classType) {
edges {
node {
name
timeOffset
}
}
}
}
}
```
`variables = {"id": "pipeline-execution-id", "classType": "G"}`
Using cURL:
```bash
curl --location --request POST 'https://stg.jetstream.seeen.com/graphql/' \
--header 'Authorization: Token 1234567890' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"query pipelineExecution($id: ID!, $classType: String!) {\n pipelineExecution(id: $id) {\n annotations(classType: $classType) {\n edges {\n node {\n name\n timeOffset\n }\n }\n }\n }\n}","variables":{"id":"pipeline-execution-id","classType":"G"}}'
```
Will filter the "GENDER" Class Type Annotations only.
### d) Pagination
By default, first 20 Annotations are returned when querying this field. You can adjust pagination yourself with the filters described here.
In order to know if you've reached the last page of the results, you need to check the `pageInfo` in your requests. Query with `pageInfo` and part of the response:
```graphql
query pipelineExecution($id: ID!) {
pipelineExecution(id: $id) {
annotations {
edges {
node {
name
}
}
pageInfo {
hasNextPage
hasPreviousPage
startCursor
endCursor
}
}
}
```
Using cURL:
```bash
curl --location --request POST 'https://stg.jetstream.seeen.com/graphql/' \
--header 'Authorization: Token 1234567890' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"query pipelineExecution($id: ID!) {\n pipelineExecution(id: $id) {\n annotations {\n edges {\n node {\n name\n }\n }\n pageInfo {\n hasNextPage\n hasPreviousPage\n startCursor\n endCursor\n }\n }\n}","variables":{"id":"your-pipeline-id"}}'
```
Part of the response - notice that the cursor starts at position 0 and ends at 20 (exclusive).
```json
{
"data": {
"pipelineExecution": {
"annotations": {
"edges": [
{
"node": {
"name": "monochrome photography",
}
},
// (...)
]
"pageInfo": {
"hasNextPage": true,
"hasPreviousPage": false,
"startCursor": "0",
"endCursor": "20"
}
}
}
}
}
```
If we wanted to get the next page of 20 results we would need to change the filter to: `annotations(first: 20, after: "19")`
**NOTE!** Indexing starts at 0, so Annotation with index 19 will be actually the 20th object in this dataset. To sum up, this filter will give you the *following* 20 Annotations *after* the first 20. Consider this as a second page of the results. For the *third* page you'd use: `annotations(first: 20, after: "39")`
---
Let's imagine that there are only 28 Annotations available. If we make a query asking for first 50 Annotations and check the `pageInfo` field, it will return us all the 28 Annotations and the `pageInfo` will look like this:
```json
"pageInfo": {
"hasNextPage": false,
"hasPreviousPage": false,
"startCursor": "0",
"endCursor": "28"
}
```
We can see that there is no next page available - this is the end of the results.
---
Further examples of queries with explanation:
- First 300 Annotations
```graphql
query pipelineExecution($id: ID!, $first: Int!) {
pipelineExecution(id: $id) {
annotations(first: $first) {
edges {
node {
name
}
}
}
}
}
```
`variables = {"id": "pipeline-execution-id", "first": 300}`
Using cURL:
```bash
curl --location --request POST 'https://stg.jetstream.seeen.com/graphql/' \
--header 'Authorization: Token 1234567890' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"query pipelineExecution($id: ID!, $first: Int!) {\n pipelineExecution(id: $id) {\n annotations(first: $first) {\n edges {\n node {\n name\n }\n }\n }\n }\n}","variables":{"id":"pipeline-execution-id","first":300}}'
```
---
- `first: 20, after: "39"` -> 20 Annotations that come after Annotation with index 39 (= after 40th object in the dataset).
```graphql
query pipelineExecution($id: ID!, $first: Int!, $after: String!) {
pipelineExecution(id: $id) {
annotations(first: $first, after: $after) {
edges {
node {
name
}
}
}
}
}
```
`variables = {"id": "pipeline-execution-id", "first": 20, "after": "39"}`
Using cURL:
```bash
curl --location --request POST 'https://stg.jetstream.seeen.com/graphql/' \
--header 'Authorization: Token 1234567890' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"query pipelineExecution($id: ID!, $first: Int!, $after: String!) {\n pipelineExecution(id: $id) {\n annotations(first: $first, after: $after) {\n edges {\n node {\n name\n }\n }\n }\n }\n}","variables":{"id":"pipeline-execution-id","first":20,"after":"39"}}'
```
### Transcription
Query for the video Transcription:
```graphql
query pipelineExecution($id: ID!) {
pipelineExecution(id: $id) {
transcriptions {
edges {
node {
text
}
}
}
}
}
```
`variables = {"id": "UGlwZWVjdXRpb25UeXBlOjYyNA=="}`
Using cURL:
```bash
curl --location --request POST 'https://stg.jetstream.seeen.com/graphql/' \
--header 'Authorization: Token 1234567890' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"query pipelineExecution($id: ID!) {\n pipelineExecution(id: $id) {\n transcriptions {\n edges {\n node {\n text\n }\n }\n }\n }\n}","variables":{"id":"UGlwZWVjdXRpb25UeXBlOjYyNA=="}}'
```
Example result:
```json
{
"data": {
"pipelineExecution": {
"transcriptions": {
"edges": [
{
"node": {
"text": "we're happy to announce the G Channel mobile app. This app was made by enthusiasts for enthusiasts. It's got a lot of unique features like contests and polls. You can even make memes and remix your videos. But the best part about it is that you can upload your videos straight onto the GP Channel network right from your phone. We're opening the door to collaborate with you, our audience to make this the automotive social app out there. So what you waiting for? Go download the free app, share with your friends and be a part of the GT Channel network."
}
}
]
}
}
}
}
```
### TranscriptionTokens
Query for the video TranscriptionTokens with all the available fields:
```graphql
query pipelineExecution($id: ID!) {
pipelineExecution(id: $id) {
transcriptionTokens {
edges {
node {
token
score
startTimestamp
endTimestamp
tokenType
}
}
}
}
}
```
`variables = {"id": "UGlwZWVjdXRpb25UeXBlOjYyNA=="}`
Using cURL:
```bash
curl --location --request POST 'https://stg.jetstream.seeen.com/graphql/' \
--header 'Authorization: Token 1234567890' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"query pipelineExecution($id: ID!) {\n pipelineExecution(id: $id) {\n transcriptionTokens {\n edges {\n node {\n token \n score \n startTimestamp\n endTimestamp\n tokenType\n }\n }\n }\n }\n}","variables":{"id":"UGlwZWVjdXRpb25UeXBlOjYyNA=="}}'
```
Example result:
```json
{
"data": {
"pipelineExecution": {
"transcriptionTokens": {
"edges": [
{
"node": {
"token": "we're",
"score": 0.9958,
"startTimestamp": 240,
"endTimestamp": 450,
"tokenType": "P"
}
},
{
"node": {
"token": "happy",
"score": 1.0,
"startTimestamp": 450,
"endTimestamp": 710,
"tokenType": "P"
}
},
{
"node": {
"token": "to",
"score": 1.0,
"startTimestamp": 710,
"endTimestamp": 810,
"tokenType": "P"
}
},
{
"node": {
"token": "announce",
"score": 1.0,
"startTimestamp": 810,
"endTimestamp": 1340,
"tokenType": "P"
}
},
// (...)
]
}
}
}
}
```
Available `tokenTypes`:
```python
PRONUNCIATION = "P"
PUNCTUATION = "U"
```
You can filter the TranscriptionTokens with `timeOffsetRange` similarly to how you filter Annotations. Check the Annotations section for more details. Here is a quick example of a query with variables:
```graphql
query pipelineExecution($id: ID!, $timeOffsetRange: String!) {
pipelineExecution(id: $id) {
transcriptionTokens(timeOffsetRange: $timeOffsetRange) {
edges {
node {
token
score
startTimestamp
endTimestamp
tokenType
}
}
}
}
}
```
`variables = {"id": "pipeline-execution-id", "timeOffsetRange": "0,900"}`
Using cURL:
```bash
curl --location --request POST 'https://stg.jetstream.seeen.com/graphql/' \
--header 'Authorization: Token 1234567890' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"query pipelineExecution($id: ID!, $timeOffsetRange: String!) {\n pipelineExecution(id: $id) {\n transcriptionTokens(timeOffsetRange: $timeOffsetRange) {\n edges {\n node {\n token \n score \n startTimestamp\n endTimestamp\n tokenType\n }\n }\n }\n }\n}","variables":{"id":"pipeline-execution-id","timeOffsetRange":"0,900"}}'
```
Sign in with Wallet
Connect another wallet