# CID Checker logs aggregation

Tools used:

Vector - tool for building observability pipelines https://vector.dev/
Loki - log aggregation system https://grafana.com/oss/loki/
Grafana - the open observability platform https://grafana.com/

To run Loki and Graphana use this docker-compose.yaml:

version: "3"

networks:
  loki:

services:
  loki:
    image: grafana/loki:1.5.0
    ports:
      - "3100:3100"
    command: -config.file=/etc/loki/local-config.yaml
    networks:
      - loki

  promtail:
    image: grafana/promtail:1.5.0
    volumes:
      - /var/log:/var/log
    command: -config.file=/etc/promtail/docker-config.yaml
    networks:
      - loki

  grafana:
    image: grafana/grafana:master
    ports:
      - "3000:3000"
    networks:
      - loki

Create vector configutaion file "vector.toml":

[sources.checker]
  type = "docker"
  include_images = ["cid-checker"]

[transforms.checker-remove]
  type = "remove_fields"
  inputs = ["checker"]
  fields = ["container_created_at", "container_id", "container_name", "label.com.docker.compose.config-hash", "label.com.docker.compose.container-number", "label.com.docker.compose.oneoff", "label.com.docker.compose.project.working_dir","label.com.docker.compose.service","label.com.docker.compose.version", "label.com.docker.compose.project.config_files", "label.com.docker.compose.project"]

[sinks.loki]
  # General
  type = "loki"
  inputs = ["checker-remove"]
  endpoint = "http://localhost:3100"
  healthcheck = true

  # Encoding
  encoding.codec = "text"

  # Labels
  labels.app = "checker"

With this config Vector would consume logs from “cid-checker” container, will remove unnecessary fields, and send it to the Loki endpoint.

For more info on configuration check Vector docs https://vector.dev/docs/reference/sources/

Run vector with nohup:

nohup vector --config ./vector.toml &

Grafana available at URL: http://filecoin.tools:3000/
(no HTTPS yet)