## Icebreaker

**What does DevOps mean for you? Simply put, where and how do you run codes in 'production'?**

- Run locally and on HPC cluster. These are tools and softwares already deployed for specific NGS analysis, those are run on cluster and the downstream analysis locally.
- deploying the code and maintaining it as well
- To use best practices to deploy applications properly!
- For development environment in containers in my local machine, for production ECS, EKS in AWS, or Azure.
- faster iteration on dev, deployment, and infra. Less friction between writing code and having it run somewhere real.

## Repository

git clone https://github.com/mimer-ai/handson-k8s-workshop

## Essential commands

kubectl get pod -A
kubectl get nodes

## Questions, answers and information

- Is this how to ask a question?
- Yes, and an answer will appear like so!
- Click the pen or the open book button on top left of the page if you can't see the editor :)

- How does Kubernetes differ from Hadoop architecture? Could you share your opinion?
- Hadoop is a way more specialized solution. In fact, you could deploy Hadoop on top of Kubernetes, but not the other way around. Hadoop provides a specific way to design and run workloads. It provides a set of frameworks that your code is built on top of. On the other hand, Kubernetes effectively runs your code on barebones Linux, whatever your code is.
- The only common feature is that both have the ability to "parallelize" applications. Hadoop does it using MapReduce of "tasks", Kubernetes does it through running several containers. In every other sense, both are very different.

- When to use singulary vs Apptainer vs podman
- singularity is very often used in shared HPC environments (where multiple different projects co-exist, from different orgs), as it provides a security footprint that is better aligned with these shared systems. Apptainer is very similar to singularity (they are forks of each other).
- podman is often used when rootless containers are designed for local/cloud execution
- podman can use Docker images nearly as it is. Singularity / Apptainer needs a conversion if Docker is your base image.
- You can actually run podman in "Docker compatibility mode", which is basically aliasing every docker command into a podman one, and in most cases the containers can't tell the difference

- Does the procedural logic of kubernetes affect your running code? Are there object oriented containers for example?
- Not sure if I understand correctly, but here's a tentative answer: No, kubernetes does not affect the way you run your code, or the way you structure your code (i.e., whether you use object-oriented programming). Inside kubernetes, each container effectively runs on barebones Linux. If you want to write your code in a specific language or runtime, you simply need to embed the necessary tools in the image. Your code has no idea that it is running in Kubernetes instead of a "standard" Linux machine.
-
- Could you share the schedule of the workshop please?
- You can find it at the bottom of this page: https://mimer-ai.eu/event/hands-on-kubernetes-as-a-user/

- Talos Linux allows have multiple physical nodes?
- Talos Linux is design to be used as the OS running below Kubernetes. Kubernetes is not an hypervisor, it's an application itself that runs on a machine with a OS. Or, rather, in a Kubernetes cluster, every node needs to run an OS, and on top of it run the Kubernetes service (then all the nodes will talk to each other to form the cluster and distribute workloads). The OS does not know or care about the cluster. It simply runs Kubernetes. Kubernetes will take care of the logic necessary to have a cluster. Any Linux OS can be the host on which Kubernetes runs. Talos Linux is a Linux distro that was designed specifically to run Kubernetes, specifically to be used as the host OS of each node in the cluster. No special logic is needed for this, it's simply a matter of optimizing the built-in packages. And, in the case of Talos Linux, there is also a strong focus on security (if you plan to use the node only to run Kubernetes, as is often the case, there are a lot of things you don't need and can be removed to reduce the security footprint).

- Does k3s use Docker?
- k3s by default uses containerd, which is an alternative container engine. K3s can be configured to use Docker, if needed.

- Does exists several apiVersions? when use v1 or other?
- You should check the documentation for each individual resource type. While there is typically only one correct value for a specific resource type, different resource types use different values, as Kubernetes has been trying to modularize its API. For example, for a `Pod` you should use `apiVersion: v1`, while for a `ReplicaSet` you should use `apiVersion: apps/v1`

- how know kubectl that gpu is sopported?
- When you want your pod to use GPUs, you specify it in the requirements (`resources` part of the spec) within the pod definition. When you create a cluster and add nodes to it, you should make sure that the GPUs are enabled. Then, the Kubernetes control plane will know which types of GPUs and how many are available in the cluster. Kubectl does not directly do anything about GPUs. It submits your pod specification to the cluster control plane, which will be responsible for looking at the pod requirements and allocating resources on a suitable node. So if you specified you need 2 GPUs, the control plane will allocate a node with 2 unutilized GPUs.

- ==Daniel mentioned that he used ks3 for their studies, could you please share resources to check them by ourself?==
- TODO: raise this question to Daniel at the end of the presentation

- But will the kubernetes layer be hidden to users across AI factories? Otherwise, how would HPC systems enforce aspects such as job time limits? For instance, do you think it will it be necessary to have SLURM on top of kubernetes?
- This question is about how the AI factories will provide Kubernetes-based compute resources to users, so it's a bit separate from the main contents of the workshop, which is on kubernetes in general.
- I believe every AI factory will make its own decisions on how to offer their compute resources, and it's not a given that each of them will even use Kubernetes. So we cannot answer in general for all AI factories.

- Can init-setup have an external cloud-init file with bash instructions?
- you can use a script that was baked in the container image, or one that is in an external storage location that is mounted to the pod.

- When a claim has less space than the storage it refers to, like a pv of 1 GB and a claim of 500MB, does it mean I can only write up to 500 MB?
- IIRC, you will be allowed to write to the whole PV, even if the PVC is smaller.
- ==so what is the point of using PVCs?==
- Let's wait for Daniel for a better answer.
- Daniel showed the docs for k3s to build a cluster: https://docs.k3s.io/quick-start
- Once you have a cluster up and running, it works the same as k3d with kubectl and yaml resource definitions

- what is the difference between doing this:
`export KUBECONFIG=$(k3d kubeconfig write my-cluster)` vs

    sudo cp -R /root/.kube . 
    sudo chown -R daniel:daniel .kube/

because for me after creating the cluster it works with `export KUBECONFIG=$(k3d kubeconfig write my-cluster)` ?
- the export approach is temporary. You are effectively making a temporary config with write access to the cluster and storing it in an env variable. As long as that env variable is visible, you will be able to use the cluster. If you open a new terminal, it will not work there unless you repeat the command.
- running with user in the `docker` group, solves the issues with sudo directly - no need of any fiddling with ~/.kube or `KUBECONFIG`

- volume `/pv-data` became `/data` inside the pod. What does `pv` stand for? And what does @all do?
- `pv-data` is the name used on the host (where the data actually resides permanently), while `data` is the name used within the container. Both names are fully configurable by you, as one is listed in the persistent volume definition, the other in the pod spec.
- PV stands for Persistent Volume.

- Can nodes be created on different VMs in k3d?
- It *might* be possible with some wizardry, but k3d is mostly designed for local development and testing. K3s has more robust support for connecting multiple VMs into a multi-node cluster.

- In what situation would NoExecute taint be useful? (I.e. allow to be scheduled on the node but cannot execute)
- NoExecute will also prevent scheduling. The difference between NoSchedule and NoExecute is that NoExecute will also evict currently-running pods, while NoSchedule will only prevent additional pods from being scheduled while allowing already-running pods to continue.

- It seems that you're using nodeselector&labels to achieve the same effect as taints&tolerations, i.e. preventing some pods to run on some nodes. When would I choose one over the other?
- They work in opposite ways, so are complementary to each other. Taints are negative constraints (they avoid specific nodes), while node selectors are positive constraints (they actively select specific nodes). A node selector says "I need this feature". A toleration says "I don't mind this constraint". Typically you use node selectors for specific useful features (e.g. specific GPU types), to ensure that you will get a node with that. Those nodes will still be usable for pods that don't explicitly require the feature. Taints are typically used to "fence-off" some nodes and make them unavailable to all pods by default, and then specific pods can be allowed to operate on those nodes with tolerations. A typical example are the nodes in the control plane, which are used for cluster management. Those are typically tainted to prevent normal pods to run on them, so that they are fully-reserved for management pods.
- Thanks!

:::success
## Break until 11:00 (CEST)
:::

- What is the difference between changing the namespace of the current context with:
`kubectl config set-context --current --namespace=<namespace>`
and switching context with:
`kubectl config use-context <context-name>`
Also what is context?
- "context" refers to the combination of cluster, user and namespace. The first command changes part of the context while keeping the rest fixed; in this example, we chose to change the namespace while keeping the same cluster and same user. The second command loads a new context, which might different in part or completely. So for example you could have separate contexts for your dev cluster and your prod cluster, or different contexts for different projects you are working in.

- What is the difference between Calico and LinkerD?
- they work on different levels. Calico is a lower-level solution for routing network communications and policing them. LinkerD is about building a service mesh on top of the network.

- what is the default service type if not specified?
- IIRC, it's ClusterIP

- HorizontalPOdAutoscaler only works for virtual nodes? or also for physical nodes?
- HorizontalPodAutoscaler will change the number of replicas in a deployment based on usage metrics. It works on the level of pods, not nodes, so it's not affected by how your nodes are set up.
- To achieve the goal of also scaling out the number of nodes, you need to combine the HorizontalPodAutoscaler (HPA) with a Cluster Autoscaler, so when the HPA schedules more (or less) pods, if needed, more nodes can be added (or removed).

:::success
## Lunch until 13:15
:::

- Could you briefly explain what an Ingress Controller is, and in what situations we as developers need to know about it or configure it ourselves?
- An Ingress exposes your pods to external networks. For example if you have a domain name and you want to reach the web application running in a Kubernetes cluster from the internet. You can also add other kinds of routing and load balancing along with an ingress.
- Whether you as a developer need to know about it depends on how your organization structures its infrastructure. If you deploy your applications on Kubernetes clusters managed by another team (such as a central infrastructure team), then you most likely don't need to know the details of the Ingress Controller. You just need to know how to set an Ingress rule to reach your service. If, on the other hand, your team is responsible for managing the infrastructure directly, then you may need to make decisions about what Ingress Controller to use and how to configure it.

- What vllm stand for?
- https://vllm.ai/
- It is an inference software, very similar to Ollama. It is used for serving LLMs over REST API.

- How does `terminationGracePeriodSeconds` work, and how is it related to graceful shutdown during rolling updates?
- When kubernetes needs to shutdown a pod, it first sends a SIGTERM signal, then waits for the grace period, then sends a SIGKILL signal. In general, an application that needs to take special steps on shutdown should listen for the SIGTERM signal and when it receives it, it should initiate the shutdown. The SIGKILL signal on the other hand kills any application instantly without a chance to take any extra time to shutdown. Rolling updates are just one case in which Kubernetes needs to shutdown a pod.

- ==What is your opinion on devspace? https://www.devspace.sh/ ?==

- ...