---
title: "Networking WG, 16th December 2022"
tags: networking, oakestra
datetime: 2022-12-16T{{time}}UTC
location: Online, Discord, the `#voice-meeting` voice channel
teams:
- networkingWG
participants:
- Giovanni (GB)
- Nitinder (NM)
- Simon (SZ)
---

## 📋 Agenda
- Add Simon to HackMd, other Oakestra repos(?)
- HPI infra additions
- Next steps
- Meeting time for networkingWG

## 📢 Discussions
SZ: Random behavior on `net-manager`

GB: When the pairing between cluster and root does not suceed, the cluster is not able to ask for a valid subnetwrok to the root orchestrator. Without a valid subnetwork the node cannot initialize and the startup fails with the error: "Invalid Subnetwork". This happens because the subnetwroks are assigned at Root level (To avoid clusters locally assigning the same subnetwork to two different worker nodes).

SZ: In last two weeks, set up cluster, docker-compose file configuration issues. Would have been issue with IPv6

SZ: Tries setting components with IPv6. Issue with it being set only on host.
Bridged solution: system-manager could connect to cluster orch but the subnet was not reachable.

NM: Why?

SZ: Despite the `system_manager_url` environment var being set in the docker-compose, it was not propagated to the docker container system_manager. This caused the NetManager to not be able to get a subnet, since the `system_manager_url` was set to some 3.120.x.x IP, making the NodeEngine unable to function. So the problem seems to be that the `root_service_manager_url` was set to `system_manager_url`, which has caused the problem described above.

GB: Maybe we should implement a check for the cluster not to start up at all, if those prerequisites are not met, such as the cluster_manager not being able to communicate with the system_manager correctly.

GB: We should keep the addresses seperate. Find a way to set these variables.
SZ: Should we do IPv6 only?

GB: For local scenario, it is what it is. Challenge would be networking between worker nodes.

SZ: After IPv6, we need to set up firewall?

NM: IPv6 only comm for workers as well?

SZ: After orchestrators, we can try that

GB: There are two layers in worker nodes. Inter and intra comm. IPv6 can give better load balancing capability. Raw communication over tunneling is another issue.

NM: We need to think about IPv6 only overheads in IPv4 subnnets. maybe we keep "legacy" support (esp how do you translate IPv6 load balanced addresses to IPv6 addresses)

SZ: When should we think about translations? Is cluster orch responsible for this?

NM: Yes. Lets try to understand the overheads of IPv6 only comms first. Maybe we don't need translation in the end.

GB: Potential issue with external comms over nodes that just support IPv4. Internally doesnt seem to be a apparent problem for now

SZ: Will keep root and cluster manager in `hybrid` mode. Also better to keep Cluster manager in `hybrid` mode for potential overhead issues

GB: Real bottleneck is intra cluster. Control path between cluster orch and workers is better case. Between nodes would also be issue for carrying data.

GB: Communication between workers require testing and measuring (for IPv6 only comms)

## 🛠️ TODOs
SZ: Send HackMd account to get added to team space.

SZ: transition cluster manager and root to IPv6 (Docker based)