# Oakestra supported technologies/features

Here, we are keeping a list of all technologies that are used for building Oakestra and also frameworks that Oakestra supports.

## Root Orchestrator

| Technology/Feature | Usage | URL |
| -------- | -------- | -------- |
| Python | Programming Language for the services composing the Root Orch | [Text](https://www.python.org) |
| Angular | Dashboard | [Angular](https://angular.dev/) |
| Flask | API | [Flask](https://flask.palletsprojects.com/en/stable/) |
| OpenAPI | API | [OpenAPI](https://www.openapis.org/) |
| MongoDB | Database | [MongoDB](https://www.mongodb.com/) |
| Docker Compose | Spin-up the Root Orchestrator Services | [Docker Compose](https://docs.docker.com/reference/cli/docker/compose/) |
| Grafana | Clusters status and control plane alerting system | [Grafana](https://grafana.com)
| Loki | Log aggregation | [Loki](https://grafana.com/go/webinar/getting-started-with-logging-and-grafana-loki)
| Promtail| Log extraction | [Promtail](https://grafana.com/docs/loki/latest/send-data/promtail/)
| gRPC | Cluster Regsitration | [gRPC](grpc.io) |

## Cluster Orchestrator

| Technology/Feature | Usage | URL |
| -------- | -------- | -------- |
| Python | Programming Language for the services composing the Root Orch | [Python](https://www.python.org) |
| MongoDB | Database | [MongoDB](https://www.mongodb.com/) |
| Celery | Task queues | [Celery](https://docs.celeryq.dev/en/latest/index.html) |
| Eclipse Mosquitto | MQTT broker | [Mosquitto](https://mosquitto.org/) |
| Docker Compose | Spin-up the Cluster Orchestrator Services | [Docker Compose](https://docs.docker.com/reference/cli/docker/compose/) |
| Grafana | Clusters status and control plane alerting system | [Grafana](https://grafana.com)
| Loki | Log aggregation | [Loki](https://grafana.com/go/webinar/getting-started-with-logging-and-grafana-loki)
| Promtail| Log extraction | [Promtail](https://grafana.com/docs/loki/latest/send-data/promtail/)
| gRPC | Cluster Regsitration | [gRPC](grpc.io) |

## Worker

| Technology/Feature | Usage | URL |
| -------- | -------- | -------- |
| QEMU | Builidng multi-platform container images | [qemu](https://www.qemu.org/) |
| containerd | Container execution runtime | [containerd](https://containerd.io)|
| paho mqtt | MQTT client library for control plane communication | [paho mqtt](https://pypi.org/project/paho-mqtt/) |
| OCI Runtime Spec | Standardized container lifecycle | [OCI Spec](https://github.com/opencontainers/image-spec/blob/main/specs-go/v1/descriptor.go)|
| cgroups | Workload monitoring | [cgroups](https://man7.org/linux/man-pages/man7/cgroups.7.html) |

## Networking

| Technology/Feature | Usage | URL |
| -------- | -------- | -------- |
| Golang | Programming Language of Proxy and Load Balancer components | [Golang](golang.org) |
| iptables | Used to set the firewall rules for the containers | [iptables](https://linux.die.net/man/8/iptables)
| NAT | Used for the gateway component (implemented with iptables) | [NAT](https://en.wikipedia.org/wiki/Network_address_translation) |
| paho mqtt | MQTT client library for control plane communication | [paho mqtt](https://pypi.org/project/paho-mqtt/) |
| Mux | Local socket communication with NodeEngine | [Mux](https://gorilla.github.io) |
| Cobra | Command line utility | [Cobra](https://github.com/spf13/cobra)|
| Network namespaces | Service network isolation primitive | [netns](https://man7.org/linux/man-pages/man8/ip-netns.8.html) |
| Linux Virtual Network Interfaces | Bridges, Veths, vTAP for Service Networking | [ip-link](https://man7.org/linux/man-pages/man8/ip-link.8.html) |
| MongoDB | Network Routes and Service IPs | [MongoDB](https://www.mongodb.com/) |

## Runtime environments

| Technology/Feature | Usage | URL |
| -------- | -------- | -------- |
| Docker Containers | Container Execution Runtime | [Docker](https://www.docker.com/) |
| Unikernels | Unikernel Execution Runtime | [Unikraft](https://unikraft.org/) |

## CI/CD / Linting / Formatting
| Technology/Feature | Usage | URL |
| -------- | -------- | -------- |
| black | python code formatting | https://github.com/psf/black |
| flake8 | python style guide enforcer | https://flake8.pycqa.org/en/latest/ |
| isort | uniform and ordererd imports | https://pycqa.github.io/isort/ |
| docker buildx | build multi platform container images | https://docs.docker.com/reference/cli/docker/buildx/ |
| GitActions | Images build, release, tests | [GitHub Actions](https://docs.github.com/en/actions/about-github-actions/about-continuous-integration-with-github-actions)
| AWX | Automated pipeline for Oakestra Integration tests in TUM infrastructure | [AWX](https://github.com/ansible/awx)

## CLI

| Technology/Feature | Usage | URL |
| -------- | -------- | -------- |
| Pydantic | Managing / (de)serializing / parsing Python Classes/Objects | https://docs.pydantic.dev/latest/ |
| Poetry | Managing & Building Python packages / projects / dependencies | https://python-poetry.org/ |
| Typer | Underlying framework that powers the CLI (build ontop of Click & Rich) | https://typer.tiangolo.com/ |
| Rich | Styling and creating pretty outputs and CLI feel | https://github.com/Textualize/rich |
| Ansible (Python API Runner)| Potential to automate anything - here installation of dependencies | https://docs.ansible.com/ |
| icecream | superior python printing | https://github.com/gruns/icecream |

## Addons (already supported)
### FLOps

| Technology/Feature | Usage | URL |
| -------- | -------- | -------- |
| Flower | Power FL Components & Communication | https://flower.ai/ |
| mlflow | MLOps + GUI | https://mlflow.org/ |
| conda suite (anaconda, miniconda, libmamba)| Efficient Python Dependeny Management (For Container Image building) | https://www.anaconda.com/ https://docs.anaconda.com/miniconda/ https://www.anaconda.com/blog/a-faster-conda-for-a-growing-community |
| buildah | Building container images inside containers | https://buildah.io/ |
| Mongo DB | DB for FLOps internals/controlplane-objects | https://www.mongodb.com/ |
| Paho MQTT / eclipse-mosquitto | MQTT (lightweight event messaging) | https://pypi.org/project/paho-mqtt/ https://mosquitto.org/ |
| flask-openapi3 | REST API | https://luolingchun.github.io/flask-openapi3/v4.x/ |
| waitress | production grade WSGI Server | https://pypi.org/project/waitress/ |
| distribution.distribution | (Self hosted) Container Image Registry |https://github.com/distribution/distribution |
| docker-vsftpd | FTP server to store FL/mlfow artifacts | https://github.com/fauria/docker-vsftpd |
| mysql | DB to store FL/mlflow metrics | https://www.mysql.com/de/ |
| flower datasets | Source for test data (hugging face) and splitting it up for FL | https://flower.ai/docs/datasets/ |
| Apache Arrow Flight | gRPC based transport protocol for apache formatted data (ML Data Server communication) | https://arrow.apache.org/docs/format/Flight.html |
| Apache Parquet | Columnar Data Format for on disk-operations / storage | https://parquet.apache.org/ |
| Apache Arrow | Columnar Data Format for in-memory operations | https://arrow.apache.org/ |

---

## Near future plans

| Technology/Feature | Usage | URL |
| -------- | -------- | -------- |
| Wasmtime | WebAssembly Execution Environment | [Wasmtime](https://wasmtime.dev) |
| Multus CNI | For multi-container CNI, used for K8s integration | [Multus CNI](https://github.com/k8snetworkplumbingwg/multus-cni)|
| k8s CRD | Used for K8s cluster integration in Oakestra | [CRD](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/)|
| CNI | Used for standardized container networking in Oakestra-Net and K8s compatibility | [CNI](https://github.com/k8snetworkplumbingwg/multus-cni)|
| eBPF | Virtual Network Function orchestration and Load Balancing | [eBPF](https://ebpf.io) |