# Autosécurité

Via Dell. Large VMware/Dell ELA
Already bought licenses, Tanzu Basic, Avi Enterprise

## People

Fredeeique Pongie?

Laurent Liesse

Florian Poussol
Sysadmin (monitoring)

Pierre Willot
Sysadmin - Does the Docker/Ansible stuff

Pierre Wlomainck (deputy chief)
Sysadmin

Laurent
Developers

Fabien

## Key Questions

1. What is the current situation? In more detail?
a. Standalone unmanaged Docker Hosts etc.? Docker Swarm?
Standalone Docker hosts in Linux VMs
Problem:
- No standard way to host and support the containerized applications
- Standalone Docker Hosts, hard to manage etc.
- Apps provided by single third party Company (no internal developers) that develops their main Production Applications, based on Java, MongoDB, ...
1 VM (docker host) per application?
Currently multiple Apps per Docker VM
b. Ansible does what in the environment?
c. Persistant Data Needs? Include Database, fiel shares, etc ?
Yes, mixed environment: DB & Persistant Storage mounted from Linux VM
2. Do they have vRA running (possible Self Service capabilities)?
3. Licenses for Tanzu & NSX-T ALB, correct?
So vSphere with Tanzu & NSX ALB Enterprise License (SE Core Count: 4)
4. Total number of Docker VMs for the moment?
Around 20
5. How many Applications are currently running?
1200 Containers for all Environments. Apps are Micro Serviced.
6. Environments?
Prod, QA,
7. Docker Compose?
Yes
8. How are Apps Delivered?
Raw .jar files and then they build their own Containers for it.
9. How is Jenkins used?
As a Scheduler
10. Container Registry? Nexus for saving the Artifacts, Docker Registry for the Images
11. K8s Experience?
Not a lot, not played around with it a lot. Never used K8s.
Pierre W already tried to install Tanzu in one of the datacenter once. Together with Dell, not used the Tanzu environment yet.
12. Backups on Containers?
Idea: LVM snapshot and just backup data of Docker Volume(path mounted to Docker Container and this Data is backed up). VM is backed up by VEEAM
13. Tanzu Platform Expectations / Outcomes?
Avoid deploying applications they don't know themselves as SysAdmin team. Providing the Developers with the tools to deploy the Apps themselves is the Goal. Move the Responsibility of the Application's Deployment Architecture to the Developer Team (goal of Sysadmin team).
14. Valid Expectation / Outcome from Tanzu?
Well the steps might initially remain (somebody has to create the YAML files etc.). Currently the Sysadmin team is doing the DevOps / Platform Role but this role does not go away with K8s. Developers will take over the Devops / Platform Role and deploy on the Tanzu K8s Clusters.
15. How would Autosecurité give access to the Devs to their Tanzu K8s Clusters?
Third party have VPN and Management VM inside
16. What do they currently have as Monitoring?
Currently global view of the CPU & Memory Usage per Container + Per Host via ELK.
17. What do they expect to see in Monitoring K8s?
Same functionality as they currently have (see questions 16. - ELK).
18. Will Autosecurité do the K8s Cluster Sizing?
Not known for the moment.
19. How will the Environment Separation work (E.g.: Prod, Dev, Test)?
Physical Clusters vs Namespaces (not clear yet). Allow Developers to create Physical Clusters?
20. Automate Day 0 & Day 2 Tasks on Clusters? Devs not clear on what they want.
Proposal: Working with T-Shirt Sizes.
21. Possible to involve Devs in Design Workshop?
Yes this is possible to invite them the Devs in the Design Workshop (or before the beginnign of the project). Goal: check Dev Requirements
22. Third party expected to backup own data, to a target VM with a share or S3 object store

## What we need to discuss in Workshop / Essential Design Decisions

AVI SE Core Count vs Datacenter Topo, means 1 SE pair per datacenter.
Each SE can only have max 1 core. Is this enough for the L4 LB Workload (ingress to the apps)

What does current App ingress path look like?
What is the current Docker network/nat setup
How do users get at the containers
What is the ratio between containers, and ingress points (per docker host)
How many apps??
How many frontends??

### Network Topo
Is there a tenanting requirement at all? If not, is 1 flat VS network ok? (per datacenter)
Where do they see AVI controller existing (in what network?)? (1 per datacenter)
Where do they see AVI SE Management network existing?
Do they want a separate network for the VS's?
Where do they want the TKGs Control plain VMs to live? What network?
All same management network??
How many TKGs workload networks do we need?

If assumption is.. devs just want k8s api

vsphere SSO rights / k8s clusters right. (Ad accounts? Local vsphere sso accounts?)
What sis the namespace/cluster sizing that is inititally needed?
What growth is expected?
What is the application divide between datacenters?

Responsibility Divide

Second day operations

## Workshop dag indeling

Hour 1/2 , Infra

### ~ Infra Hour 1

Introductie
- What are we doing here
- What have you asked of us?
- What is the plan for the day

Basic k8s / TKGs Concepts
- Whats a k8s cluster
- Docker vs k8s
- Container opslag centraal in Harbor
- How does storage work in k8s?
- How does networking work in k8s?
- Whats the kind of second-day ops you can expect around k8s and TKGs?
- Responsibility matrix

### ~ Infra Hour 2

Current Docker Build / Deploy Process
- Show us how it goes
- How do you build the containers.. show example dockerfiles
- Show example docker compose files
- How much do you keep needing to change?

### Devs Hour

- How do you build apps?
- DO you build containers yourselves at all?
- What is your k8s experience?
- Di you have build pipelines?
- Will these k8s clusters for hte targets for such pipelines
- Are you automating container build at all, maybe with kpack or similair?
- Are you going to be the ones making the k8s deployment yaml?
- What is min version of k8s you expect?
- What are your persistant storage requirements?
- Who is resposible for persistant data backup and restore?
- How do you expect to access these k8s clusters?
- How do you expect to get your containers over there
- Do you expect to be able to pull any image from the internet?

### Infra hour 3-4

- Zoom in on network topo and design
- Avi placement
- Use of SE's
- Ingress Route for apps
- LB Requirements

- Network access to
- k8s
- Harbor
- jumpbox?

- What will the rest of the engagement look like?

## License Stuff

Tanzu Basic
LB Avi requirements:
- IP X-Forwarded-For required (client IP passthrough)
- What license they have (all through Dell). What tier, how many SE cores??
- L7 Ingress?? - Yes, supported cause of AVI Enterprise License

https://docs.vmware.com/en/VMware-NSX-Advanced-Load-Balancer/20.1.4/Administration_Guide/GUID-B5EC8F3B-A75E-4809-A653-6EBE08CFED81.html

## Current Setup

- 2 Data Centers (proposal: We do 1 DC and do the other 1 together with them)
- 2 DCs interconnected fibre channel
- Each DC has 1 vCenter and several ESXi Servers
- DC's do not talk to each other using VMware
- Separate Storage in each DC's
- Storage Compellent PowerStore(?) - iSCSI
- VEEAM is used to replciate between the 2 DCs
- External Partners provide the Application at the Container Level
- Only Using VMware as a Hypervisor for now

--------------

From: Robert Kloosterhuis <[e-mail]>
Date: Wednesday, 22 December 2021 at 11:03
To: Thomas Bernard <[e-mail]>, Michael van de Gaer <[e-mail]>, José Cavalheri <[e-mail]>
Cc: Tom Vallons <[e-mail]>
Subject: RE: Tanzu - Training & Deployment opportunity - Autosécurité
> Any rough estimation of how many days we might need ?

Hard to say at the moment without more information, but especially need to know how much they.. already know 😊

For the first three topic areas:

- Design consideration (to deploy on top of their existing hosts)
- Installation
- Daily operations & maintenance

This is pretty standard stuff. Same as in the pocs we have done. Of course, we will need to do a similar ‘design workshop’ with them, to determine the requirements.
Question is, if in this case, we want to document these or not. Normally, we create the design first, and then do the implementation ourselves later, based on that design. In this case.. who will document the design?

I would say half , to 1 day, to go over design considerations and create an outline of what the design will be.

---

Installation – same question, who does it primarily? Us, with them watching, or them, based on the design, with our guidance?
If all the prerequisites are in place (we identify these during the design workshop), then the actual installation is not much work. For us would be a day or less now.
But if we do it with them, and they want explanation as we go, we should take 2 days. So that would cover:

Setup of AVI Controller (NSX ALB)
Setup (workload enablement) of both clusters for vSphere with Tanzu (TKGs) (Assuming we are gonna go with TKGs, as opposed to TKGm, but this is something we decide with them in the design workshop)

-----

If we are to add some standard packages to this.. like Harbor (container registry), then might need more time to explain this. This gets now into the area of how to use Kubernetes generally, which also has overlap with ‘daily operations & Maintenance’. I should note here: No one in ITQ has any actual experience yet with day2/day-to-day operations around TKGs or TKGm. So my intention would be to prepare some material/demonstration around this. (my time: 1 day)

Monitoring is a bit tricky. Its not an area I am very experienced with, and I am generally unfamiliar with Centreon and Elastic
That need not be a problem. If we approach this as ‘we will figure this out together’ , in that case, I would take both day2-operations + monitoring stuff as an additional 2 – 3 days.

So to summary, VERY rough estimation:

Scoping session: 2 hours
Design workshop: 1 day
(IF ITQ should document the design: 1 day (Robert))
Robert Day2 Ops prep: 1 day
Guided Installation and setup: 2 days
Day2 Ops + Kubernetes explanation/examples + Monitoring integration: 2 days

Is that enough for now?

Greets,
Robert