Try   HackMD

Skynet API Go Through

Overview

Use Cases:

  1. certik.org
  2. skynet quickscan report

Architecture

Component Types:

  1. API
  2. Indexer
  3. Worker
  4. Storage

API

All APIs are hosted as AWS Lambda functions.

Development

We use serverless to build all the APIs, typical workflow

sls create_domain
sls deploy

Permissions and Endpoints are described in serverless.yml.

Data Storage

Storage Types

  • Key Value - AWS DynamoDB
  • File - AWS S3
  • Queue - AWS SQS

Metrics Table:

Designed for storing data points that has continous value over time, e.g. totalTweetsCount,price,marketCap etc.

Sample Records:

name                     timestamp   value
price(projectId=CertiK)  100         1.95
price(projectId=CertiK)  200         2.55

Query:

# checking CertiK price when timestamp = 150

result = query({
  TableName: "skynet-prd-metrics",
  KeyConditionExpression: "#name = :name and #timestamp <= :timestamp",
  ExpressionAttributeNames: {
    "#timetstamp": "timestap",
    "#name": "name"
  },
  ExpressionAttributeValues: {
    ":timestamp": 150,
    ":name": "price(projectId=CertiK)"
  },
  Limit: 1,                  # fetch one record
  ScanIndexForward: false    # order by timestamp desc
})

result.Items[0] == { timestamp: 100, value: 1.95 }

Background Jobs

Indexer

Indexers are Batch jobs mostly doing ETL (extract, transform, load)

  • Nomad Platform (contract data indexer, twitter data indexer)
  • Serverless Platform (market data indexer)

Worker

Workers are long running jobs handling specific business logic

  • Bytecode
  • Sourcecode

Hands on Session

Tasks:

  1. Request QuickScan API to get result for project CertiK
  2. Request On-chain Monitoring Primitive API to get result for address bsc:0xe7f15597b7594e1516952001f57d022ba799b479
  3. Request Twitter Sentiment API to get result for twitterId dypfinance
  4. Check DynamoDB table skynet-prd-metrics for CertiK price history, hint: price name pattern is price(projectId=xxx)
  5. Check S3 for CertiK's tweets history

Roadmap

  1. Monitoring
  2. netlify function migrate to Security Rating API
  3. Bytecode/Sourcecode migrate to Nomad

Resources

https://github.com/CertiKProject/certik-skynet

https://nomad.certik-skynet.com/ui/jobs (Need IP whitelist)

https://app.serverless.com/certik (Need Authentication)