---
# System prepended metadata

title: Lorenzo - Centrifuge

---

# Lorenzo - Centrifuge
- nix & nixos 
- k8s operators implementation 
  - implementers on k8s api 
  - gke

## Centrifuge roadmap
### Setup Dev VPN
- In Centrifuge Dev project
- Admin Access to vpn

### Monitoring + Logging 
#### Monitoring
- Current grafana url: http://34.107.88.100/d/twYeaxRMz/my-polkadot-metrics?orgId=1&refresh=10s
    - u: admin
    - p: grafana
- Deploy Grafana + Prometheus cleanly
    - https://github.com/BenWhiteJam/node_deployer
- Ensure all pods/nodes are indexed 
- Use latest grafana dashboard for Polkadot
    - tweaking for network name
- Substrate visualize metrics: https://substrate.dev/docs/en/tutorials/visualize-node-metrics/
- Extra prometheus config substrate jobname/rule:
    - https://github.com/BenWhiteJam/node_deployer/blob/main/kube/monitoring/manifest/prometheus-configmap.yaml
```
    - job_name: "substrate_node"
      scrape_interval: 5s
      scheme: http
      kubernetes_sd_configs:
      - role: pod
      relabel_configs:
      - separator: ;
        regex: __meta_kubernetes_pod_label_(.+)
        replacement: $1
        action: labelmap
      - source_labels: [__address__]
        separator: ;
        regex: ([^:]+)(?::\d+)?
        target_label: __address__
        replacement: $1:9615
        action: replace
      - source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_pod_label_k8s_app]
        separator: ;
        regex: monitoring;prometheus
        replacement: $1
        action: keep
      - source_labels: [__meta_kubernetes_namespace]
        separator: ;
        regex: (.*)
        target_label: namespace
        replacement: $1
        action: replace
      - source_labels: [__meta_kubernetes_pod_name]
        separator: ;
        regex: (.*)
        target_label: pod
        replacement: $1
        action: replace
      - source_labels: [__meta_kubernetes_pod_name]
        separator: ;
        regex: (.*)
        target_label: instance
        replacement: $1
        action: replace
```

#### Logging
- Configure loki promtail to index from systemd:
    - Search journald level 
    - Add sample arbitrary label - Text: "Parachain"

#### Alerting
- Push alerts to some telegram channel
- Initial metrics and rules:
    - host mem
    - host cpu
    - host disk
    - validator (n_peers, block_number hasn't changed in n mintues)
    - error rate increased by n%

#### Documentation
https://docs.google.com/document/d/1kj9zx_vbtzJaUQqC29JZH37beFkjMKzHF1Nnuixk6uk/edit#heading=h.hfgs75y3mg9x

## Next
- infra more reboust + security 
- eng processes 
- access management