# ANSY SUBJECTS
## Information about the submissions

The submissions will be done by email at [e-mail], tag `[ANSY]`.

You can submit each part individually.
There is no restriction on the language used or the submission architecture, but please make it clear, and readable. The easier it is for me to correct you, the more likely I will be to fully appreaciate your work.

Recommended languages however includes:
- python
- bash
- c

I will also accept "normal" languages for the kind of tasks I'm asking you to do, but I reserve myself the right of refusing a submission if it's an obvious troll like brainfuck or ook!.

Please send files like:

TP1-<login>.zip
└── TP1-<login>/
    ├── README
    └── ...

In doubt, ask me :)

### Note

Please track the time you're spending on each part and provide the information in the submission. This will allow me to better estimate the workload of those subjets and adapt it eventually for next year

# ANSY 1 subject

## Rendu

Le rendu se fera par une archive .zip à m'envoyer à [e-mail].
Elle devra contenir :
- un README avec le résultat de "getting started" et le temps total passé sur le sujet 1
- le script demandé dans la deuxieme partie, avec l'historique des commandes

La date max de rendu est fixée au début du cours n°2

## getting started
1. Avec strace, trouver le syscall qui est executé plus de 10k fois par le binaire fourni, et me rapporter combien de fois il est appelé
2. Trouver et rapporter dans le README la phrase la plus importante, philosophie du developpement de linux, de l'échange houleux et très discutable sur la forme entre Linus Torvalds et Mauro Chehab (https://lkml.org/lkml/2012/12/23/75)

## Real stuff

1. écrire un programme qui permet d'executer le binaire fourni et faire en sorte qu'il fonctionne, c'est à dire qu'il retourne 0 en exit code et qu'il affiche la phrase de fin ("gg!"). Pour ça, il va falloir utiliser strace pour comprendre son fonctionnement, et faire en sorte qu'il puisse s'executer comme il devrait. Il y a 2-3 "pièges", en tout cas du bruit, pour essayer de se mettre dans une condition plus "réelle".
2. BONUS OPTIONNEL: Atteindre la seconde fin du programme (bonus, plus difficile)

Je vous demanderai aussi de conserver votre bash history (ou zsh history ou équivalent) pour me montrer les commandes et la réflexion que vous avez pu avoir pour ce TP

Le binaire à analyser est disponible sur https://zarak.fr/resources/straceme
Je vous demande de n'utiliser que strace pour l'analyse et pour le comprendre

# ANSY 2 subject

## How often is the scheduler waken up ?

### Setup the env

For this part, we're going to mess up with the scheduler a bit.

For the moment, let's keep the changes simple, we'll focus on bringing our own linux kernel alive, to explore the linux toolchain and build process.

For this subject, you need to have a Debian 12 VM (this is the recommended way, as I'm going to guide you with such setup. You may do this subject with another setup, but it might be harder for you, and I may not be able to debug you).

You can install a debian 12 VM with virtualbox or libvirt. If you're running AMD64, you can download it here : https://cloud.debian.org/images/release/current-live/amd64/iso-hybrid/ (debian-live-12.1.0-amd64-standard.iso)

Install the VM, install a few tools you might find useful, like vim or a ssh server.

### Prepare your linux build setup

You can follow guides like this: https://phoenixnap.com/kb/build-linux-kernel

But here are the things needed for this subject:

$ sudo apt-get install -y git fakeroot build-essential ncurses-dev xz-utils libssl-dev bc flex libelf-dev bison dwarves
$ uname -a
Linux ANSY 6.1.0-12-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.52-1 (2023-09-07) x86_64 GNU/Linux
$ echo Our current kernel is in version 6.1.0. Let's download the Linux source for this version
$ git clone https://github.com/torvalds/linux.git -b v6.1 --depth 1
$ cd linux
$ echo generate your own base kernel config with `make defconfig`
$ make defconfig
$ echo maybe you will need or want to change the configuration. Either edit the .config file newly created, or run make menuconfig
$ echo you can also copy your current linux config, but compilation will take more time
$ #cp /boot/config-6.1.0-12-amd64 .config

### Compile linux

You are now ready to compile linux. Simply run `make -j$(nproc)` to build the kernel.

You will also need the compressed version. You can get it with `make bzImage -j$(nproc)`. You can actually use exclusively this one to compile and build.

The kernel you just built may be slightly incorrect, and report itself as `6.1.0-dirty` (`uname(2)`). This is because you have built from a git repository with dirty changes -- non committed changed. To make your new kernel compatible, we'll add another build option:

$ uname -a
Linux ANSY 6.1.0-12-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.52-1 (2023-09-07) x86_64 GNU/Linux

My current kernel reports itself as 6.1.0-12-amd64. I'll force the kernel I'm building to report to this as well:

$ touch .scmversion && make bzImage -j$(nproc) LOCALVERSION="-12-amd64"

The first build will take quite some time. You may need some amounts of RAM and CPU. Be patient.

### Testing your linux

You should have in the end something like this in the output:

  BUILD   arch/x86/boot/bzImage
Kernel: arch/x86/boot/bzImage is ready  (#3)

You need to copy your newly built kernel in /boot:
`$ cp arch/x86/boot/bzImage /boot/vmlinuz-custom`

You don't want to replace the real kernel debian-provided youre currently using, as your newly built kernel may be broken. Instead, we're going to add an option in grub, when booting, to choose to boot from this kernel.

Edit the file `/boot/grub/grub.cfg`

Find the default menu entry. Mine looks like this:

menuentry 'Debian GNU/Linux' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-e06573ef-48b6-4890-b5c6-622eedf914b0' {
	load_video
	insmod gzio
	if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
	insmod part_msdos
	insmod ext2
	search --no-floppy --fs-uuid --set=root e06573ef-48b6-4890-b5c6-622eedf914b0
	echo	'Loading Linux 6.1.0-12-amd64 ...'
	linux	/boot/vmlinuz-6.1.0-12-amd64 root=UUID=e06573ef-48b6-4890-b5c6-622eedf914b0 ro  quiet
	echo	'Loading initial ramdisk ...'
	initrd	/boot/initrd.img-6.1.0-12-amd64
}

Copy your own menu entry (not mine ! it will not work) to have a second one right under the default with a few differences:

1c1
< menuentry 'Debian GNU/Linux' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-e06573ef-48b6-4890-b5c6-622eedf914b0' {
---
> menuentry 'Debian GNU/Linux with custom kernel' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-e06573ef-48b6-4890-b5c6-622eedf914b0' {
9c9
< 	linux	/boot/vmlinuz-6.1.0-12-amd64 root=UUID=e06573ef-48b6-4890-b5c6-622eedf914b0 ro  quiet
---
> 	linux	/boot/vmlinuz-custom root=UUID=e06573ef-48b6-4890-b5c6-622eedf914b0 ro  quiet

Change the name of the entry to show which one is yours and which one is the default one. Change the linux argument path to your own kernel.

Now, on reboot of the VM you should see a new entry in your grub menu to boot from your own kernel

### The assignment.

You will have to write a patch for linux. We're going to add a new behavior to the scheduler. We've been wondering in class how many times the scheduler was waken up to make a decision, to schedule.

Let's make our kernel expose this information. We're not going to do anything fancy yet as this is not the point of the exercise.

1. Setup the developement env
2. Write a patch to the scheduler so that every time it is called, a counter is increased. Also look at the time. If it's been more than a second since the last report, make a report. To make a report, write the number of times the scheduler has been called (your counter) to kernel logs, and reset the counter.
3. Test your new patch, and look in the kernel logs at the output of your scheduler

Hints:
- The scheduler function to schedule a task has a very straightforward name
- To log, use the function `printk`
- To get the time in ns, you can use `ktime_to_ns(ktime_get())`

### Submission

You will provide a `.patch` that contains the patch you've just written (i.e. the git diffs)
You will also provide a README with information you deem interesting to mention, but with at least:
- the amount of time spent on this
- an output of dmesg to show your scheduler logs

The deadline is for the 13th of october.

# ANSY 3

## Let's observe the CPU/Scheduler in action.

Write a program than can creates situations to prove how the niceness of 2 different processes interferes within each other.

The idea is to be able to provide an answer for such problem:
I have process A with a niceness of X. I have process be with a niceness of Y.
Both processes are started at the same time, and require the same amount of CPU in total. But they're fightining each other for said CPU. How much more real time A (or B) will need compared to the amount of time it needs if it runs alone ?

Answer can be summed up in an array like:

Niceness of A (X) | Niceness of B (Y) | Extra time needed for A | Extra time needed for B |
--------------|---------------|-------------------------|------------------------------|
0|10|0% (as fast as if it was alone) | 100% (needed twice more time)
0|0|50% |50%

Of course those values are example.

Create a program that takes as an input X and Y (argv), runs two processes in parallel with given niceness, measure how much time it needed to execute the two and output the % of extra time needed in comparison of a single execution.

### Tips

- You'll need to run a CPU program for this test to make sense
- You'll need to run in a single core those two processes to see how the CFS will make decisions. You'll have to force those 2 processes on one core
- Execution time will vary from a laptop to another, don't hardcore the referential execution time

## Extra challenge

This challenge is a bonus, it's quite difficult and may take quite some time to achieve. Don't feel forced to do it if you don't have the time.

Create a program that can create a very high cpu load -- let's say twice the amount of cores -- **without** using the CPU (CPU usage % shall remain below ~10%).

## Submission

The deadline for the main exercise will be for the 13th of october, at 14h. The deadline for the optionnal challenge is for the first of december.

As usual, the submission will be done by email, with a .zip described above.
I'm expecting a README with the amount of time spent, an array providing answer for (X=0,Y=0),(X=0,Y=10),(X=0,Y=15), and ofc the program(s)

# ANSY 4 Subject

**The deadline for this exercise will be the 25th of November 23:59.**

## Manipulating eBPF

This TP will make you write an eBPF program to monitor processes scheduling latency.

You will have to use BCC and write both a python script using BCC python API and an eBPF script (embedded in the python script).

The eBPF script will collect metrics about scheduling latency for each non-kernel process. The latency will be collected in ns, but needs to be displayed to the user in µs. The communication between the eBPF program and the python script will done via an eBPF map, to allow async collection of said metrics in python.

The python script will:
- Compile the eBPF script and load it
- Do an infinite loop:
- sleep 1s
- read the eBPF map
- format the data:
- Aggregate the values of each PID by TGID (average)
- Get the TGID's argv from /proc (think about implementing some cache maybe for this)
- Convert the units to µs if necessary
- display it on stdout

The eBPF script will:
- Compute each individual latency of scheduling
- Make some per-TGID aggregation of the latency (mean scheduling value)
- Also collect the maximum scheduling latency per-TGID
- Sum the number of scheduling events per-TGID
- For each TGID, optionally get the name of the task:
- COMM name
- exe file name
- Make those data available to userland via an eBPF map

The display format shall be the following:

`print(f'{tgid:<7} {average_us:>8} {max_us:>8} {number_of_events:>6} {comm:>16} {exe:>16} {argv}')`

Also add a delimiter between each sets of events every second.

### Useful tips

- Think about process time to live, and how to handle the death of a process
- To get you started, you can use the **runqslower** example provided by BCC as a layout. Don't bother with the TRACEPOINT, remove this part altogeter and keep only the eBPF part.
- https://github.com/iovisor/bcc/blob/master/docs/tutorial_bcc_python_developer.md
- https://github.com/iovisor/bcc/blob/master/docs/tutorial.md

# ANSY BONUS subject

This subject is a bonus, either to help you boost your grade if you believe you did not perform well on other subjects, or to help compensate a missing submission.

**The deadline for this exercise will be the 25th of November 23:59.**

## Manipulating block devices

The point of this exercise will be to write a small program/script to manipulate block devices, kernel modules related to block devices, filesystems and famous userland tools.

The suggested language is bash.

You will have to create a few scenarios and run benchmark to test them out.

The benchmark will be run with `fio`: `fio --name=random-write --rw=randwrite --bs=4k --numjobs=1 --size=1g --iodepth=256 --runtime=60 --time_based --end_fsync=1`

As you can read, you'll need a GiB of free space to run this test. It is therefore recommended to create each situation with 5 GiB to begin with, to be sure to have enough room in the end.

## Core

The core of your program will:
1. Create a file that will be used as a block device.
- *It's recommended to use a file rather than a disk partition as it's simpler for theses tests. But keep in mind that once this file has been setup to behave as a block device, things will be the same as if it's a "real" block device (a real hard drive, USB key, ...)*
1. Expose this file as a block device via a loop device. Check **losetup**
- *pro-tip: use `-f --show`*
3. You now have a block device than can be used. Depending on the arguments of your script, you will apply or not some transformation to it. Refer to the next section to know what transformation are expected here
4. After the optional transformation, you end up with a block device. This block device to be used needs a filesystem. Create an XFS filesystem on the block device
5. The filesystem to be accessed needs to be mounted. Mount it to the path specified by `--path` (defaults to `/mnt/final` if not specified)
6. If the user asked for it via `--bench`, run a benchmark on the final filesystem

## Transformations

Here are a list of flags your program can take and that will apply transformation to the block device at the step 3:
- `--loop` will take a number X as argument (default: 0) and will repeat the core of the program X times, meaning repeating steps 1 to 4 included, and step 5 with a path of `/mnt/loop-$i`.
- *If another transformation is asked, this transformation must be done at each loop iteration.*
- *The step 1 must be performed on the filesystem created at the previous iteration of course*
- For the last iteration of the loop, the name to be used on step 5 is `/mnt/final`
- With `--loop 0` (the default), one has one `/mnt/final`
- With `--loop 1`, one has `/mnt/loop-1` for the first iteration of the core loop, and `/mnt/final` for the last iteration
- `--basic-lvm` will create a LVM PV, VG and single LV taking all the space possible of the block device.
- *The names are up to you, but ``--loop` can be used, near that in mind. Refer to [archlinux LVM doc for help](https://wiki.archlinux.org/title/LVM)*
- `--crypt` will create a LUKS2 encrypted block device with cryptsetup. The password will be very secure of course, as it's going to be `password`
- *you will have to create the encrypted block device, but also open it to use it !*
- *The name of the devices is up to you. [archlinux doc](https://wiki.archlinux.org/title/dm-crypt/Device_encryption)*
- `--dm-linear` will create a block device from your whole block device in a linear way
- *[doc](https://docs.kernel.org/admin-guide/device-mapper/linear.html)*
- `--dm-delay` will perform as `--dm-linear`, but will add a delay of 1000 ms
- *[doc](https://docs.kernel.org/admin-guide/device-mapper/delay.html)*
- `--md` will create 2 block devices from your block device (split it in half with `dmsetup`'s linear option), and then a RAID 1 disk from the two block devices
- *[doc](https://wiki.archlinux.org/title/RAID)*

Each transformation shall take as an input 1 block device, and provide as an output 1 block device. They can be chained this way.

## Pro-tips

It is recommended to start simple. Create the `core` part of the program, and test it well.

Add a `--cleanup` flag to not do anything but remove what your program has created, this will help you a lot

Gradually add the transformations support. They may seems complex, but each one taken individually is actually fairly straight forward with the appropriate documentation

Reach me for help if things are unclear, or not working as expected.

You can use argbash.dev to simplify development if you chOose bash