[Back to LOC CLI Guidebook](https://hackmd.io/4LIsNIuWQnuR70JMeuBAEA?view)
[Setup LOC CLI Environment](https://hackmd.io/igLh4azUT2aI8Fv-q-0e-g)
[Getting Started - JavaScript](https://hackmd.io/IiHvmAtjTTGFfakaH0dWuw?view)
[Getting Started - TypeScript](https://hackmd.io/kz93Th7vTCCbO3GFxp3r-A)
[LOC CLI Commands](https://hackmd.io/R4mrz2t1QSyTCHH73_2itg)

# Local Simple Runtime Execution

If you want to quickly test or debug your data process, there is a way to deploy and run it in a local LOC environment. The benefits of doing so are:

- No user login required
- No API routes nor HTTP clients needed
- Does not use LOC deployment environment resources
- See ```ctx.agents.logging``` logs in your console

Most of the [LOC agents](https://hackmd.io/Uj-tC7l9Q82VyGr8R5PL2Q?both) are supported *except* using SMB in file storage agent.

## Setup

### Install Docker and Docker Compose

The local runtime requires **Docker** and **Docker Compose**. You can simply install [**Docker Desktop**](https://docs.docker.com/desktop/) on your machine, which contains both and supports Windows/Mac/Linux.

- [Install on Windows](https://docs.docker.com/desktop/windows/install/)
- [Install on Mac](https://docs.docker.com/desktop/mac/install/)
- Install on Linux
- [Download Docker Desktop for Linux packages](https://docs.docker.com/desktop/linux/install/)
- [Set up the Docker repository](https://docs.docker.com/engine/install/ubuntu/#set-up-the-repository)
- [Install on Debian/Fedora/Ubuntu/Arch](https://docs.docker.com/desktop/linux/install/#generic-installation-steps)

:::info
### What is Docker Compose?

Docker Compose is a way to start up multiple Docker containers *at once*. The concept and effect are very similar to Kubernetes, which is the deployment environment that LOC runs on.
:::

### Create Local Profile

Create a new profile ```local.yaml``` under your LOC CLI workspace directory (for example, ```/FST```):

simpleRuntime:
  image: public.ecr.aws/m0s8j1u6/saffron/simple-runtime:release-0.5.1
  dockerNetwork: host
  eventstoreEndpoint: http://127.0.0.1:8087
  etcdEndpoints: http://127.0.0.1:2379

Set it as your default profile:

loc profile set -f local.yaml -p local
loc profile default local

### Create ```saffron-evenstore.yaml```

Create this file also in your LOC CLI workspace directory, which defines the containers we want to start up together:

version: "3.7"

networks:
  saffron:
    driver: bridge

volumes:
  es01:
    driver: local

services:
  saffron-evenstore:
    restart: "always"
    image: public.ecr.aws/m0s8j1u6/saffron/eventstore:release-0.5.1
    container_name: saffron-evenstore
    ports:
      - "8200:8200"
      - "8087:8087"
      - "8007:8007"
    environment:
      - SAFFRON_EVENTSTORE_API_ASSRESS=0.0.0.0
      - SAFFRON_EVENTSTORE_API_PORT=8007
      - SAFFRON_EVENTSTORE_API_AUTHORIZATION_SECRET=-----BEGIN PUBLIC KEY-----MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAmuzhqHVUKj0CVtnqCS1PbVOyq1TcyykKgXoBwQvPaF8N8zIHXUjCgwjmKbOK+SjLoLC7cgLWlwbV2u12J/9w2sTawbROOno7Td2OZte2qNLugsEf5RxGVKd4IZx5zshxTI/bwnYGvpyVWdbRGvtWMKpbkzkR7O8UumtUMdUl6h1iUKRXkAxoB2ge/CZUDCss643iS0KPO7gIowvcYFCKBX86G/+GBlIHcXWTIa2eEWuTx5T64lmSVJrlN1L6diZRlUIBq6JNuQv26ZJgNAO91DDcCLCxcQgLH5KqwzdXwz2jBOqL7ac1HAW/mov4oJrk7ytWVumoFJ+NlJlSZ+vd5wIDAQAB-----END PUBLIC KEY-----
      - SAFFRON_EVENTSTORE_GRPC_ADDRESS=0.0.0.0
      - SAFFRON_EVENTSTORE_GRPC_PORT=8087
      - SAFFRON_EVENTSTORE_METRICS_ADDRESS=0.0.0.0
      - SAFFRON_EVENTSTORE_METRICS_PORT=8200
      - SAFFRON_EVENTSTORE_ELASTICSEARCH_ENDPOINT=http://es01:9200
      - SAFFRON_EVENTSTORE_ELASTICSEARCH_USERNAME=
      - SAFFRON_EVENTSTORE_ELASTICSEARCH_PASSWORD=
    command:
      - run
    networks:
      - saffron
    depends_on:
      - etcd01
      - es01

  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.16.1
    restart: "always"
    container_name: es01
    environment:
      - xpack.security.enabled=false
      - bootstrap.memory_lock=true
      - discovery.type=single-node
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - es01:/usr/share/elasticsearch/data
    ports:
      - "9200:9200"
    networks:
      - saffron

  kibana:
    image: kibana:7.16.1
    restart: "always"
    container_name: kibana
    volumes:
      - ./kibana/kibana.yaml:/usr/share/kibana/config/kibana.yml
    ports:
      - "5601:5601"
    networks:
      - saffron

  etcd01:
    image: docker.io/bitnami/etcd:3.5.1-debian-10-r31
    restart: "always"
    container_name: etcd01
    environment:
      - ALLOW_NONE_AUTHENTICATION=yes
      - ETCD_NAME=etcd01
      - ETCD_INITIAL_ADVERTISE_PEER_URLS=http://etcd01:2380
      - ETCD_LISTEN_PEER_URLS=http://0.0.0.0:2380
      - ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379
      - ETCD_ADVERTISE_CLIENT_URLS=http://etcd01:2379
      - ETCD_INITIAL_CLUSTER_TOKEN=saffron-etcd-cluster
      - ETCD_INITIAL_CLUSTER_STATE=new
    ports:
      - "2379:2379"
    networks:
      - saffron

### Create ```/kibana/kibana.yaml```

Add a new dir ```kibana``` under CLI dir and create ```kibana.yaml``` in it:

server.name: "kibana"
server.host: "0.0.0.0"
elasticsearch.hosts: [http://es01:9200]
xpack.security.enabled: true
elasticsearch.username: "elastic"
elasticsearch.password: "aa123456"
monitoring.enabled: false

:::info
The new files would be structured like this (we will add ```payload.yaml``` later):

/FST
    /[your data process project]
    /kibana
        kibana.yaml
    loc
    local.yaml
    payload.yaml
    saffron-evenstore.yaml
    ...

:::

## Run Docker Containers

### Simple Runtime Image

Now go back to LOC CLI workspace dir and run the following command to download the "simple runtime" image:

docker pull public.ecr.aws/m0s8j1u6/saffron/simple-runtime:release-0.5.1

Whenever you (re)deploy new data process, a new container would be created from this image.

### Startup Local LOC Services

We also need to create a local runtime so that everything works, including a functional local event store.

Open a new terminal or console, switch to LOC CLI dir and enter the following command:

docker-compose -f saffron-eventstore.yaml up

This command will download several images and start them up in your Docker environment:

- saffron-evenstore
- etcd01
- es01
- kibana

:::info
Please be noted that the first time running this process will take a while, depending on your Internet speed.

When the containers start up, you may see ```saffron-evenstore exited with code 255``` appear a few times. Don't worry - this is simply because the local event store is waiting for other services.
:::

You can check if all of them are up and running in Docker Desktop:


Or you can use the console command ```docker ps -a```:

> docker ps -a

CONTAINER ID   IMAGE                                                      COMMAND                  CREATED       STATUS       PORTS                                                                    NAMES
56d44778ce08   public.ecr.aws/m0s8j1u6/saffron/eventstore:release-0.5.1   "/usr/bin/saffron-ev…"   2 hours ago   Up 2 hours   0.0.0.0:8007->8007/tcp, 0.0.0.0:8087->8087/tcp, 0.0.0.0:8200->8200/tcp   saffron-evenstore
cd621ca37438   kibana:7.16.1                                              "/bin/tini -- /usr/l…"   2 hours ago   Up 2 hours   0.0.0.0:5601->5601/tcp                                                   kibana
afa1658206b1   bitnami/etcd:3.5.1-debian-10-r31                           "/opt/bitnami/script…"   2 hours ago   Up 2 hours   0.0.0.0:2379->2379/tcp, 2380/tcp                                         etcd01
6568da78bdc9   docker.elastic.co/elasticsearch/elasticsearch:7.16.1       "/bin/tini -- /usr/l…"   2 hours ago   Up 2 hours   0.0.0.0:9200->9200/tcp, 9300/tcp                                         es01

You can press ```Ctrl+C``` to gracefully shut them down and run ```docker-compose -f saffron-evenstore.yaml up``` again to start them up.

:::info
You can start up all containers in the background with ```-d``` flag:

docker-compose -f saffron-evenstore.yaml up -d

Remember you'll have to wait a while for saffron-evenstore to be ready.

Then you can stop/delete all local services like this:

docker-compose -f saffron-evenstore.yaml down

:::

## Run Data Process in Simple Runtime

### Create Payload File

Since we are going to run a data process directly without an API route, we need a .yaml file to pass "payloads" to it.

Create a ```payload.yaml``` in the LOC CLI workspace dir:

tasks:
  - dataProcessIdentityContext:
      permanentIdentity: 00000000-0000-0000-0000-000000000000
      revision: 1
      name: ""
    taskId:
      executionId: AAAAAAAAAAAAAAAAAAAAAA
      id: AAAAAAAAAAAAAAAAAAAAAA
    payload:
      http:
        apiGatewayIdentityContext:
          id: 00000000-0000-0000-0000-000000000000
          name: ""
        apiIdentityContext:
          id: 00000000-0000-0000-0000-000000000000
          name: ""
        requestId: "4242"
        scheme: https
        method: POST
        path: /test/test
        query: "?name=Arthur&age=42"
        version: HTTP/1.1
        headers:
          Content-Type: "application/json"
        host: fst.network
        body:
          name: "Arthur"
          age: 42

:::info
DO NOT change the ```executionId``` or ```id``` under ```taskId```.
:::

You can see there are quite a few fields that you can customize. Most of the HTTP-related fields will directly pass to [```ctx.payload.http```](https://hackmd.io/Uj-tC7l9Q82VyGr8R5PL2Q?view#ctxpayloadhttp) as mock-up values.

body:
name: "Arthur"
age: 42
job:
- company: "FST Network"
- position: "Frontend Developer"

It would be transformed to a JSON like this in the data process:

{
"name": "Arthur",
"age": 42,
"job": {
"company": "FST Network",
"position": "Frontend Developer"
}
}

### Execute Data Process in Single Runtime

When the local LOC services are up and running, we can run any data processes in a simple runtime.

Here we use the example from [Getting Started - Hello World (JavaScript)](https://hackmd.io/IiHvmAtjTTGFfakaH0dWuw?view) (open a new terminal to enter this command):

loc dp run-local hello-world -f payload.yaml --docker

:::info
For executing **TypeScript** version data processes, add a ```-ts``` flag:

loc dp run-local hello-world-ts -f payload.yaml -ts --docker

:::

If the data process compiled successfully (if not, it will try to tell you where it went wrong), you should see something like this:

![](https://hackmd.io/_uploads/SkLT4_ndq.png)

Look at the outputs carefully, you can find the logs that you've sent in the data process:

2022-06-07T06:43:08.487874Z INFO saffron_process::agent::logger: [execution_id=AAAAAAAAAAAAAAAAAAAAAA,id=AAAAAAAAAAAAAAAAAAAAAA]@7c8c081e-0805-49cb-9ddd-6049b1a00562-0 {"payload":{"name":"Arthur","age":42}}
...
2022-06-07T06:43:08.544840Z INFO saffron_process::agent::logger: [execution_id=AAAAAAAAAAAAAAAAAAAAAA,id=AAAAAAAAAAAAAAAAAAAAAA]@7c8c081e-0805-49cb-9ddd-6049b1a00562-0 emit event: [{"sourceDID":"Hello_World","targetDID":"Arthur","labelName":"Hello, Arthur!","meta":"","type":"default"}]
...
2022-06-07T06:43:08.550495Z INFO saffron_process::agent::logger: [execution_id=AAAAAAAAAAAAAAAAAAAAAA,id=AAAAAAAAAAAAAAAAAAAAAA]@7c8c081e-0805-49cb-9ddd-6049b1a00562-0 {"response":{"message":"Hello, Arthur!"}}

### Cleanup Containers

Like we've mentioned above, every time you run a data process in simple runtime, a new container would be created. You can easily clean them up in Docker Desktop (clicking the trashcan icon on the right):

![](https://hackmd.io/_uploads/Hy7Crd3d9.png)

Or you can remove all inactive simple runtime containers with the following console command:

docker rm $(docker ps -a -q --filter ancestor=public.ecr.aws/m0s8j1u6/saffron/simple-runtime:release-0.5.1)

### Access ```localhost```

Any ```localhost``` services you have on your machine can be accessed with the host name **```host.docker.internal```**. For example, you can use the HTTP agent like this:

const response = await ctx.agents.http.post(
"http://host.docker.internal:8080/post", // = localhost:8080/post
{}, // headers
"Json",
new TextEncoder().encode(JSON.stringify(body)), // JSON body
);

---

:::info
### Simple Runtime Command Cheatsheet

# startup local LOC services
docker-compose -f saffron-evenstore.yaml up

# startup data process in local runtime
loc dp run-local <project name> -f payload.yaml --docker

# cleanup local runtime containers
docker rm $(docker ps -a -q --filter ancestor=public.ecr.aws/m0s8j1u6/saffron/simple-runtime:release-0.5.1)

### Additional Local Testing

- [View Local Event Logs in Kibana](https://hackmd.io/oE8gHd3hSLyl2XiAn66PGg?view)
- [Local Database Testing with Simple Runtime](https://hackmd.io/l2p0PF5yTC2UUnRtXnp5Qw?view)
- [Local FTP Testing with Simple Runtime](https://hackmd.io/QymXtDwOTcGNaym1YjRbkQ?view)

### Related Reading
- [Agent List](https://hackmd.io/Uj-tC7l9Q82VyGr8R5PL2Q?view#Agent-List)
- [Event Fields Comparison Table](https://hackmd.io/N2Km6-8kRI2tMR5X4tRHeQ?view)
- [Data Process Error Handling Tips](https://hackmd.io/Dfwa4fZkSdSAVzuq1Def2w?both)
- [HTTP Agent Request Examples](https://hackmd.io/xWdMYLD5RXiV8wt5cGVwJA)
:::

###### tags: `LOC CLI`