# IIBMP2022 Sapporo Demo

## Link

- Slide: https://docs.google.com/presentation/d/1M7EtFuSwC0p31pVHOxRzb2j854KZB0eCg1wMDCqDayI/edit?usp=sharing
- Repositories:
- [sapporo-wes/sapporo-service](https://github.com/sapporo-wes/sapporo-service)
- [sapporo-wes/sapporo-web](https://github.com/sapporo-wes/sapporo-web)
- Getting Started: https://github.com/sapporo-wes/sapporo/blob/main/docs/GettingStarted.md

## Demo

### Sapporo-service

WES API クライアントである `Sapporo-service` の起動

$ curl -O https://raw.githubusercontent.com/sapporo-wes/sapporo-service/main/docker-compose.yml
$ docker compose up -d
[+] Running 1/1
 ⠿ app Pulled                                                                                        0.9s
[+] Running 1/1
 ⠿ Container sapporo-service  Started                                                                1.0s
$ curl -s localhost:1122/service-info | jq .
{
  "auth_instructions_url": "https://github.com/sapporo-wes/sapporo-service",
  "contact_info_url": "https://github.com/sapporo-wes/sapporo-service",
  "default_workflow_engine_parameters": {
    "nextflow": [
      {
        "default_value": "",
        "name": "-dsl1",
        "type": "str"
      }
    ],
    "snakemake": [
      {
        "default_value": 1,
        "name": "--cores",
        "type": "int"
      },
      {
        "default_value": "",
        "name": "--use-conda",
        "type": "str"
      }
    ]
  },
  "supported_filesystem_protocols": [
    "http",
    "https",
    "file",
    "s3"
  ],
  "supported_wes_versions": [
    "sapporo-wes-1.0.1"
  ],
  "system_state_counts": {},
  "tags": {
    "get_runs": true,
    "news_content": "",
    "registered_only_mode": false,
    "sapporo-version": "1.4.4",
    "wes-name": "sapporo",
    "workflow_attachment": true
  },
  "workflow_engine_versions": {
    "cromwell": "80",
    "cwltool": "3.1.20220628170238",
    "ep3 (experimental)": "v1.7.0",
    "nextflow": "22.04.4",
    "snakemake": "v7.8.3",
    "streamflow": "0.1.3",
    "toil (experimental)": "4.1.0"
  },
  "workflow_type_versions": {
    "CWL": {
      "workflow_type_version": [
        "v1.0",
        "v1.1",
        "v1.2"
      ]
    },
    "NFL": {
      "workflow_type_version": [
        "1.0",
        "DSL2"
      ]
    },
    "SMK": {
      "workflow_type_version": [
        "1.0"
      ]
    },
    "StreamFlow": {
      "workflow_type_version": [
        "v1.0"
      ]
    },
    "WDL": {
      "workflow_type_version": [
        "1.0"
      ]
    }
  }
}

WES API エンドポイントの定義は https://app.swaggerhub.com/apis/suecharo/sapporo-wes/sapporo-wes-1.0.1-oas3 を参照


現状の状態で、API リクエストにより、workflow のジョブの実行が可能である

- Example of API request:
- https://github.com/sapporo-wes/sapporo-service/tree/main/tests/curl_example
- https://github.com/sapporo-wes/sapporo-service/blob/main/tests/curl_example/post_runs/cwltool/attach_all_files/post_runs.sh

$ curl -s localhost:1122/runs
{"next_page_token":"","runs":[]}

現状対応している Workflow Engine は:

- cwltool (CWL)
- cromwell (CWL, WDL)
- Nextflow
- Snakemake

実際に workflow 実行の処理の手順としては:

- 1. WES server (Sapporo-service) に Run Request が POST される
- 2. Request の validation や、添付されたファイルのダウンロードなどを行う (saved to [Run_Dir](https://github.com/sapporo-wes/sapporo-service#run-dir))
- 3. [run.sh](https://github.com/sapporo-wes/sapporo-service#runsh) が fork される
- 4. run.sh 内で、workflow engine の command が実行される

全ての component は Docker コンテナとして起動される:

- Sapporo-service
- Workflow Engine
- Bioinfomatics Tools

### Sapporo-web

`Sapporo-web` は、既に SPA として deploy されているため、起動の必要がない

https://sapporo-wes.github.io/sapporo-web/

WES の登録



- 一つの Sapporo-web にいくつもの Sapporo-service (WES) を登録できる
- 情報は全てブラウザの localStorage に保存される

実行したい Workflow の登録

- Example Workflow: https://github.com/sapporo-wes/sapporo-service/blob/main/tests/resources/cwltool/trimming_and_qc.cwl



Run Request の記述

- Worklfow Attachment: Workflow Engine の実行 directory に配置されるファイル群
- Remote URL と Local File Upload の両方に対応している
- fastq1: https://raw.githubusercontent.com/sapporo-wes/sapporo-service/main/tests/resources/cwltool/ERR034597_1.small.fq.gz
- fastq2: https://raw.githubusercontent.com/sapporo-wes/sapporo-service/main/tests/resources/cwltool/ERR034597_2.small.fq.gz

{
  "fastq_1": {
    "class": "File",
    "location": "ERR034597_1.small.fq.gz"
  },
  "fastq_2": {
    "class": "File",
    "location": "ERR034597_2.small.fq.gz"
  }
}

cwltool は、ジョブ定義ファイルに remote url を記述できるため、workflow attachment を使わなくても良い

{
  "fastq_1": {
    "class": "File",
    "location": "https://raw.githubusercontent.com/sapporo-wes/sapporo-service/main/tests/resources/cwltool/ERR034597_1.small.fq.gz"
  },
  "fastq_2": {
    "class": "File",
    "location": "https://raw.githubusercontent.com/sapporo-wes/sapporo-service/main/tests/resources/cwltool/ERR034597_2.small.fq.gz"
  }
}


## Public Instance

Deployed at DDBJ super-computer system

- sapporo-service: https://ddbj.nig.ac.jp/ga4gh/wes/v1/service-info
- sapporo-web: https://ddbj.nig.ac.jp/wes/