[Development] Intercommunication between Linux drivers
===
###### tags: `development`, `C`, `driver`, `linux`, `LKM`, `KLM`, `KMOD`, `monolithic kernel`, `kernel modules`, `built-in module`,`external module`, `Module.symvers`
[ToC]
# Prerequisites
## Compiling directly into a kernel vs. modules[^ref1]
>You have two options for adding functionality to the kernel: building functions into the kernel (making a monolithic kernel) or adding them as modules.
>### Monolithic kernels:
>Building a function into the kernel directly ensures that that function will be available at all times. The downside is that it makes your kernel bigger, increasing boot time, and ultimately using that much more memory to hold the kernel. If you are compiling a kernel to fit on a floppy so that you can boot Linux off a rescue floppy, space will become an issue.
>### Modules:
>Building a function as a module allows that function to be loaded into memory as needed and unloaded when it is no longer needed. This helps keep your kernel small. It is very useful if, say, you are swapping hardware in and out of your system frequently. You could compile support for a variety of, say, sound cards as modules, and your Linux system will theoretically only load the driver that is appropriate for the hardware setup at the time.
Another benefit of building functions as modules is that parameters can be passed to the modules which can be useful in debugging your system if problems occur.
There are some considerations to be made when deciding if a kernel function should be modularized. A small performance penalty is paid as it takes a little time to get the module loaded and unloaded. There are some functions that are needed at boot time and these cannot be compiled as modules -- they need to be present in the kernel so your system can be loaded. For example, ext2/ext3/reiserfs file system support needs to be built into the kernel so that your partitions can be read, as you need to be able to read the filesystem to load modules. In my case, if I have PCMCIA support built into the kernel, then metworking works. If PCMCIA support is modularized, networking fails to start, probably because the PCMCIA support needs to be available very early in the boot process to set up networking.
>### General approach
>The way that I approached kernel building was to see which functions I would need all the time versus the ones that I would only use infrequently. If it was a function that I would be using a lot, I built it directly into the kernel. Otherwise, I compiled it as a module. As I got more comfortable with recompiling the kernel, I started making more functions as modules, and honestly, have not seen that much change in system performance. On the other hand, my hardware setup is pretty static -- my machine is a laptop, and the only hardware option that I have is whether or not I have a USB mouse plugged into it.

# Obejective
The intercommunication means that a caller calls to a callee here. The intercommunication between Linux drivers can be divided into three parts basically.
1. Intercommunication between monolithic kernel and LKM as shown in Figure 1.
a. From kernel to LKM.
b. From LKM to kernel.
2. Intercommunication between different LKMs as shown in Figure 1.
3. Intercommunication in monolithic kernel internally.

For instance, in Figure 1, there are two built-in kernel modules C.o and D.o, and two LKM X.ko and Y.ko, respectively. Assume we do the following function calls.
- 1.a C does a function call to X
- 1.b Y does a function call to D
- 2.　X does a function call to Y, and vice versa.
- 3.　C does functaion calls to each other.

So, how could we achieve these communication? Stay tuned to find out!
<br><center>

Figure 1 Intercommunication between drivers.
</center><br/>

## 1.a built-in kernel modules reference to symbols externally in LKM
In this case, the intercommunication is a little bit tricky. Symbols mention here are not only functions but variables in LKM. For instance, when C wants to call to X, he has to check if X exists. This implies C is dependent on X. Therefore, we can establish a machanism as show in.

First, assume the original arichecture is like these in C module:

/* in built-in module C, c_src.c*/
static struct list_head reg = {&reg, &reg};
typedef struct
{
    const char name[4];
    unsigned int weight;
    unsigned float volume;
} Someobject;

int register_xxx(const Someobject obj)
{
    //maybe need a lock to protect
    list_add((Someobject *)obj, &reg);
    return 1;
}
EXPORT_SYMBOL(register_xxx);

And we add an additional void pointer in Someobject.

/* in built-in module C, c_src.c*/
typedef struct
{
    struct list *nxt;
    struct list *pre; 
    const char name[4];
    unsigned int ID;
    unsigned float volume;
    void *extension; // An addition void pointer
} Someobject;

Besides, we define an extended structure for the void pointer.

/* in built-in module C, c_src.c*/
typedef struct {
    void (*callToLKM_C) (void);
    unsigned int (*getVarFromLKM_C) (void);
}C_extended;

And now, we start to deal with LKM part, i.e., X module.

/* in LKM module X, x_src.c*/
#include <linux/init.h>
#include <linux/kernel.h>

static int count = 32;
static Someobject x = {
    (struct list *) x,
    (struct list *) x,
    "X_Module",
    87,
    1893.32,
    NULL
}

void showColor(void){
    printk("Color is red\n");
}
unsigned int getCount(void){
    return count;
}

C_extended Xext = {
    showColor,
    getCount
}

static int __init X_init(void)
{
    x.extension = &Xext;
    register_xxx(&x);
    return 1;
}

static void __exit Xdeinit(void)
{
    deregister_xxx(&x);
}
module_init(X_init);
module_exit(X_deinit);
MODULE_LICENSE("GPL");

Finally, we show how to utilize this mechanism in built-in C module.

/* in built-in module C, c_src.c*/
int main(void){
    Someobject *tmp;
    tmp = getObjbyID(87); //we skip to implement this function here.
    
    printk("Starting to init ...\n");
    if(tmp){
        (C_extended*)(tmp->extension)->callToLKM_C();
        printk("count = %d\n", (C_extended*)(tmp->extension)->getVarFromLKM_C());
    }
    else
        printk("Cannot get module ID 87");
    return 1;
}

output:
insmod x.ko

Starting to init ...
Color is red
count = 32

## 1.b LKM referece to symbols in built-in module
This is an easy case. Fisrt, we define the term provider and consumer. A provider module might implement functions while a consumer module might call those functions. We can just export any symbols which may be referenced by other modules by MARCO EXPORT_SYMBOL in provider modules, and the consumer modules just extern those functions. In this way, these symbols and functions are trasparent for all modules.

## 2. LKM reference to symbols in LKM.
This case is the same with 1.b.

## 3. Symbols in built-in modules are referenced by each other
This case is the same with 1.b.

# Reference
[^ref1]: https://www.linuxquestions.org/questions/linux-software-2/built-in-or-module-any-difference-243255/

https://lkw.readthedocs.io/en/latest/doc/03_kernel_modules.html
http://yi-jyun.blogspot.com/2017/07/linux-kernel-modules.html