owned this note
owned this note
Published
Linked with GitHub
# Linux Memory Management - Pages
[TOC]
## Citations
**NOTICE NOTICE NOTICE**
The **Citations** section contains excerpts from the talks listed in the **References**. The origins are labeled in the end of the paragraphs, and the contents are attributed their respective authors.
### Pages: unit of physical memory
If we look into the RAM memory, it is subdivide into equally sized chunks. These chunks are called **pages**.
-- [27:55, Tutorial: Linux Memory Management and Containers](https://youtu.be/ql1axx--8sI?si=a3ZwzbBp2A5zu1N6&t=1675)
Kernel manages physical memory in unit of **pages**, not down to the byte level. Each page has a certain size, and the page size depends on the processor. In this talk, I'll primarily focus on x86-64 architecture, which has a base page size of 4k. So kernel takes all the memory available, chop it up to the size of 4K, and determines OK, how many pages I've got.
-- [15:19, Mentorship Session: Debugging Linux Memory Management Subsystem](https://youtu.be/fwLoPtTCmnw?si=ht6lnv0DQGmqiQH8&t=919)
This is how a **physical page** is represented in Linux (`struct page` in `include/ linux/mm_types.h`):
```c
struct page {
unsigned long flags; /* Page status, see <include/linux/page-flags.h>*/
struct list_head lru;
struct address_space *mapping;
pgoff_t index;
atomic_t _refcount; /* Usage count */
void *virtual; /* Kernel virtual address */
};
```
==This is by far a very simplified version of this struct.== This struct has a series of nested unions, but here I tried to zoom in the insteresting fields.
So as we can see, we have a pointer to the virtual page that is mapped to this physical page, flags telling you if this page is dirty, if it was swapped, if it is being used at all.
And yes, ==representing your memory consumes a little bit of memory.== In our example, with a 2G platform, we need roughly 64 bytes for each instance of this page struct, so to represent 2G of memory, we invest (a better terminology than "waste") 32 MB just to represent this memory.
-- [3:31, Inspecting and Optimizing Memory Usage in Linux - João Marcos Costa, Bootlin](https://youtu.be/pIR1H7ZyWe4?si=oihImhlqcKtkAg_7&t=211)
### Frame: a chunk in physical address space that worth a page
So, if you look through the console log, at some point after the kernel has run through the process of detecting all the pages, it prints a message saying what is the last **pfn** it saw.
```
[ 0.000000] tsc: Detected 3100.000 MHz processor
[ 0.000000] tsc: Detected 3072.000 MHz TSC
[ 0.000007] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000009] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000012] last_pfn = 0x880000 max_arch_pfn = 0x400000000
[ 0.000017] MTRR map: 8 entries (3 fixed + 5 variable; max 23), built from 10
variable MTRRs
[ 0.000019] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.000494] x2apic: enabled by BIOS, switching to x2apic ops
[ 0.000495] last_pfn = 0x62000 max_arch_pfn = 0x400000000
```
What is a **pfn**? `pfn` is physical **frame** number.
If you look at the memory map of a processor, it can (theoretically) have address from `0x0` all the way up to whatever the max the processor can address. And ==if you divide the entire address range into pages, in our case 4K page size, you divide the entire address range into 4K pages, each one of the address range is called a physical frame.==
A physical frame may has a physical memory backing it, or not. A physical memory frame may be used, but not by the physical memory. It might be used for something else. IO device map for example, so a physical frame can hold a physical page, and a physical frame can also be empty as well. The kernel will report what the last physical frame is saw that was populated.
-- [15:19, Mentorship Session: Debugging Linux Memory Management Subsystem](https://youtu.be/fwLoPtTCmnw?si=ht6lnv0DQGmqiQH8&t=919)
## References
### [Tutorial: Linux Memory Management and Containers - Gerlof Langeveld, AT Computing (27:34)](https://youtu.be/ql1axx--8sI?si=dvbiMMNsvJLNUx1C&t=1654)
{%youtube ql1axx--8sI %}
### [Mentorship Session: Debugging Linux Memory Management Subsystem](https://youtu.be/fwLoPtTCmnw)
{%youtube fwLoPtTCmnw %}
### [Inspecting and Optimizing Memory Usage in Linux - João Marcos Costa, Bootlin (3:31)](https://youtu.be/pIR1H7ZyWe4?si=oihImhlqcKtkAg_7&t=211)
{%youtube pIR1H7ZyWe4 %}
### [Kernel Recipes 2024 - All your memory are belong to… whom?](https://youtu.be/1RSKNTVf7tU)
{%youtube 1RSKNTVf7tU %}
### [Linux Memory Management - VMScan | DEEP LINUX](https://youtu.be/tpRlczF0pqw)
{%youtube tpRlczF0pqw %}