---
# PXE Network Booting
Why using PXE network booting? Using this method to boot server, the hard disk is not needed, but also the environment of server can be fixed.
The server that uses PXE to boot would automatically load the remote Linux kernel and remote initrd.
## 1. Setup PXE Server
### 1. Setup DHCP Service
1. Install DHCP server
`apt install -y isc-dhcp-server`
3. Add VM to virbr2. It will add a new netcard on VM.
`virsh attach-interface --domain ubt-20-pxe --type bridge --source virbr2 --target pxe-nic --model virtio --persistent`
3. Enable dhcp for virbr2.
* /etc/default/isc-dhcp-serve
```
INTERFACESv4="enp7s0" //netcard interface on VM
INTERFACESv6="enp7s0" //netcard interface on VM
```
4. Specific dhcpd setting
* /etc/dhcp/dhcpd.conf
```
allow bootp;
allow booting;
max-lease-time 1200;
default-lease-time 900;
log-facility local7;
option ip-forwarding false;
option mask-supplier false;
subnet 172.19.0.0 netmask 255.255.0.0 {
option routers 172.19.0.1;
option domain-name-servers 8.8.8.8;
range 172.19.0.100 172.19.0.200 ;
next-server 172.19.0.2;
filename "uefi/x86_64-efi/grubx64.efi";
#filename "bios/pxelinux.0";
}
```
5. Restart DHCP server to reload setting
`systemctl restart isc-dhcp-server`
### 2. Setup TFTP Service
1. Install TFTP server
`apt install -y tftpd-hpa`
2. Specific TFTP setting
* /etc/default/tftpd-hpa
```
TFTP_USERNAME="tftp"
TFTP_DIRECTORY="/var/lib/tftpboot"
TFTP_ADDRESS=":69"
TFTP_OPTIONS="--secure"
```
3. Restart TFTP server to reload setting
`systemctl restart tftpd-hpa`
4. Add folders for PXE service
```
mkdir /var/lib/tftpboot/uefi
mkdir /var/lib/tftpboot/bios
mkdir /var/lib/tftpboot/ramdisk
mkdir /var/lib/tftpboot/kernel
```
### 3. Setup HTTP servics
1. Install apache
`apt install -y apache2`
2. Specific Apache setting
* add following into <VirtualHost> element that in /etc/apache2/sites-available/000-default.conf
```
ServerAdmin webmaster@localhost
DocumentRoot /var/lib/tftpboot
<Directory /var/lib/tftpboot>
Options Indexes FollowSymLinks
AllowOverride None
Require all granted
</Directory>
```
3. Restart Apache to reload setting
`systemctl restart apache2`
### 4. Setup PXE Booter
* UEFI
1. Copy needed files
`cp -r /boot/grub/x86_64-efi /var/lib/tftpboot/uefi`
2. Generate the grub.efi that can support http trasmition
`grub-mkimage -O x86_64-efi -o /var/lib/tftpboot/uefi/x86_64-efi/grubx64.efi --prefix='(http,172.19.0.2)/uefi' efinet tftp http
`
3. Generate configure of grub
`vim /var/lib/tftpboot/uefi/grub.cfg`
```
default=0
timeout=10
function load_video {
if [ x$feature_all_video_module = xy ]; then
insmod all_video
else
insmod efi_gop
insmod efi_uga
insmod ieee1275_fb
insmod vbe
insmod vga
insmod video_bochs
insmod video_cirrus
fi
}
menuentry 'CentOS7.9' {
load_video
echo "Loading kernel..."
linuxefi /kernel/vmlinuz-3.10.0-1160.11.1.rt56.1145.el7.x86_64 console=tty0 console=ttyS0,115200n8 hugepages=8 hugepagesz=1G default_hugepagesz=1G hugepagesz=2M
echo "Loading initrd..."
initrdefi /ramdisk/ramdisk_centOS7.cpio.gz
}
menuentry 'Ubuntu20.04' {
load_video
echo "Loading kernel..."
linuxefi /kernel/vmlinuz-5.4.0-146-generic console=tty0 console=ttyS0,115200n8 hugepages=8 hugepagesz=1G default_hugepagesz=1G hugepagesz=2M
echo "Loading initrd..."
initrdefi /ramdisk/ramdisk_ubuntu20.cpio.gz
}
```
* BIOS
1. Install pxelinux
```
apt install -y pxelinux
cp -apv /usr/lib/PXELINUX/pxelinux.0 /var/lib/tftpboot/bios
```
2. Copy needed files
```
mount <vary_ubuntu20.iso> /mnt; cd /mnt
find /mnt/ \( -name "initrd" -o -name "vmlinuz" -o -name "ldlinux.c32" \) -exec cp -apv {} /var/lib/tftpboot/bios \;
umount /mnt
cp /usr/lib/syslinux/modules/bios/libcom32.c32 /var/lib/tftpboot/bios
cp /usr/lib/syslinux/modules/bios/libutil.c32 /var/lib/tftpboot/bios
```
3. Generate configure of pxelinux
`mkdir /var/lib/tftpboot/bios/pxelinux.cfg`
`vim /var/lib/tftpboot/bios/pxelinux.cfg/default`
```
display boot.msg
timeout=10
ontimeout focal-live-install-autoinstall
default CentOS7.9
prompt 0
LABEL CentOS7.9
KERNEL ../kernel/vmlinuz-3.10.0-1160.11.1.rt56.1145.el7.x86_64
INITRD ../ramdisk/ramdisk_centOS7.cpio.gz
APPEND console=tty0 console=ttyS0,115200n8 hugepages=8 hugepagesz=1G default_hugepagesz=1G hugepagesz=2M
LABEL Ubuntu20.04
KERNEL ../kernel/vmlinuz-5.4.0-146-generic
INITRD ../ramdisk/ramdisk_ubuntu20.cpio.gz
APPEND console=tty0 console=ttyS0,115200n8 hugepages=8 hugepagesz=1G default_hugepagesz=1G hugepagesz=2M
```
## 2. Create Initrd Ramdisk
### 1. Install OS
* **Ubuntu**
* Install Ubuntu on the target disk, and the install process is like as general, but the LVM option should be **disabled**, i.e, standard option.
* Install needed package for GV
```
apt install -y net-tools gcc cmake numactl pkg-config pciutils screen iperf3
```
* **CentOS**
* Install CentOS7.9 on the target disk, and the install process is like as general, but the partition type should set as **standard partition**.
* Add yum repo to support install the specific kernel (**rt-3.10.0-1160.11.1.rt56.1145.el7.x86_64**)
```
sudo tee /etc/yum.repos.d/CentOS-rt.repo >/dev/null <<EOF
# CentOS-rt.repo
[rt]
name=CentOS-7-rt
baseurl=http://mirror.centos.org/centos/\$releasever/rt/\$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
EOF
```
* Install specific kernel
```
yum install -y wget
wget https://linuxsoft.cern.ch/cern/centos/7/rt/x86_64/Packages/kernel-rt-3.10.0-1160.11.1.rt56.1145.el7.x86_64.rpm
yum install -y ./kernel-rt-3.10.0-1160.11.1.rt56.1145.el7.x86_64.rpm
```
* Install specific kernel develop library
```
yum install -y wget
wget https://linuxsoft.cern.ch/cern/centos/7/rt/x86_64/Packages/kernel-rt-devel-3.10.0-1160.11.1.rt56.1145.el7.x86_64.rpm
yum install -y ./kernel-rt-devel-3.10.0-1160.11.1.rt56.1145.el7.x86_64.rpm
```
* Install needed package for GV
```
yum install -y net-tools gcc cmake numactl-devel pkg-config rpm-build readline-devel redhat-lsb pciutils screen iperf3 vim
```
### 2. Generate Ramdisk Image
1. Move/plugin the target disk to other system, while **do not** let it be booting disk.
2. Mount target **rootfs** (root file system)
`mkdir -p /mnt/ramdisk`
`mount /dev/<partition> /mnt/ramdisk`
3. Prepare and Customize
1. `cd /mnt/ramdisk`
2. Remove unnecessary folder and procedure (**Non-essential Surgery**)
**NOTICE: This operate may cause that OS in the disk can not be successfully boot anymore. e.g., remove boot folder. Thus, I suggest copy them first and recover them after build ramdisk.**
* `rm -rf boot/*`
* `rm -rf swap.img`
* block all command in the etc/fstab `vim etc/fstab`
3. Generate **/init**
`ln -s sbin/init init`
init is the first user space process, and which will load rootfs.
Generally, we can generate we own init script, but we can also continue to use the default init.
**The following opertions are just needed for CentOS**
---
4. Disable SELinux
`vim /mnt/ramdisk/etc/selinux/config`
set value of `SELINUX` as `permissive` or `disable`.
5. Modify the Permission of ssh key
`chmod 600 /mnt/ramdisk/etc/ssh/*_key`
6. Modify the Permission of ping
`echo "net.ipv4.ping_group_range = 0 2147483647" >> /mnt/ramdisk/etc/sysctl.conf`
7. Create user
Generally, **home** folder is an independent partition on CentOS, so there would be no user folder in the ramdisk. We can create it by ourselves. However, to prevent any loss or mismatch any user resource, I prefer regenerate user.
```
chroot /mnt/ramdisk
userdel -r <username>
adduser <username>
passwd <username>
usermod -a -G wheel <username>
exit
```
---
4. Package
`cd /mnt/ramdisk`
`find . | cpio -o --format=newc | gzip -9 > ~/ramdisk.cpio.gz`
### 3. Modify Ramdisk (**Non-essential Surgery**)
1. Unpack Ramdisk
`mkdir ramdisk && cd ramdisk && cp ~/ramdisk.cpio.gz .`
`gunzip ramdisk.cpio.gz && rm ramdisk.cpio.gz`
`cpio -idmv < ramdisk.cpio`
2. Switch rootfs
`chroot .`
3. TODO
* Option: Add user
```
adduser <username>
passwd <username>
usermod -a -G wheel <username>
```
* Option: Install package
```
apt install -y <package name>
```
4. Update man database
`mandb -csp`
6. Repack Ramdisk
1. switch back orginal rootfs
`exit`
2. `find . | cpio -o --format=newc | gzip -9 > ~/ramdisk.cpio.gz`
## 3. Preparing on PXE server
### 1. Upload the ramdisk and kernel
* PXE server information
* **ip:** *192.168.1.164*
* **account:** *luser*
* **password:** *29597808*
* **tftp path:** */var/lib/tftpboot*
* Upload the ramdisk and kernel to PXE server
1. `scp ~/ramdisk.cpio.gz $account@$ip:/var/lib/tftpboot/kernel`
2. `scp /mnt/ramdisk/boot/vmlinuz-`<font color="#f00">X.X.X-XXX</font>`.-generic $account@$ip:/var/lib/tftpboot/ramdisk`
**After upload the file, please check the files' permission is 644.**
### 2. Edit PXE Booting Configure
* **UEFI**
**path:** */var/lib/tftpboot/uefi/grub.cfg*
```
default=0
timeout=10
function load_video {
if [ x$feature_all_video_module = xy ]; then
insmod all_video
else
insmod efi_gop
insmod efi_uga
insmod ieee1275_fb
insmod vbe
insmod vga
insmod video_bochs
insmod video_cirrus
fi
}
menuentry 'CentOS7.9' {
load_video
echo "Loading kernel..."
linuxefi /kernel/vmlinuz-3.10.0-1160.11.1.rt56.1145.el7.x86_64 console=tty0 console=ttyS0,115200n8 hugepages=8 hugepagesz=1G default_hugepagesz=1G hugepagesz=2M
echo "Loading initrd..."
initrdefi /ramdisk/ramdisk_centOS7.cpio.gz
}
menuentry 'Ubuntu20.04' {
load_video
echo "Loading kernel..."
linuxefi /kernel/vmlinuz-5.4.0-146-generic console=tty0 console=ttyS0,115200n8 hugepages=8 hugepagesz=1G default_hugepagesz=1G hugepagesz=2M
echo "Loading initrd..."
initrdefi /ramdisk/ramdisk_ubuntu20.cpio.gz
}
```
* If the ramdisk is the new one, you can add a new menuentry.
* Please fill in the relative path (from */var/lib/tftpboot*) of vmlinux(kernel file) following **linuxefi**.
* Please fill in the relative path (from */var/lib/tftpboot*) of ramdisk(initrd image) following **initrdefi**.
* If any kernel command is needed, please add them after the path of vmlinux.
* **BIOS**
**path:** */var/lib/tftpboot/bios/pxelinux.cfg/default*
```
display boot.msg
timeout=10
ontimeout focal-live-install-autoinstall
default CentOS7.9
prompt 0
LABEL CentOS7.9
KERNEL ../kernel/vmlinuz-3.10.0-1160.11.1.rt56.1145.el7.x86_64
INITRD ../ramdisk/ramdisk_centOS7.cpio.gz
APPEND console=tty0 console=ttyS0,115200n8 hugepages=8 hugepagesz=1G default_hugepagesz=1G hugepagesz=2M
LABEL Ubuntu20.04
KERNEL ../kernel/vmlinuz-5.4.0-146-generic
INITRD ../ramdisk/ramdisk_ubuntu20.cpio.gz
APPEND console=tty0 console=ttyS0,115200n8 hugepages=8 hugepagesz=1G default_hugepagesz=1G hugepagesz=2M
```
* Unlike UEFI, BIOS does not have a selection list (GRUB). It will automatically select the **default** option to boot.
* If the ramdisk is the new one, you can add a new LABEL.
* Please fill in the relative path (from */var/lib/tftpboot/bios*) of vmlinux(kernel file) following **KERNEL**.
* Please fill in the relative path (from */var/lib/tftpboot/bios*) of ramdisk(initrd image) following **INITRD**.
* If any kernel command is needed, please add them after APPEND keyword.
## 4. Setting PXE Networking Boot
1. Enter PXE client's **BIOS setting** (Delete or Esc)
2. Enable CSM Configuration: <font color="#f00">*(BIOS booting only)*</font>
Advanced -> CSM Configuration -> CSM Support (Enabled)
3. Set Network excution by ~~Legacy~~ (UEFI) <font color="#f00">*(BIOS booting only)*</font>
Advanced -> CSM Configuration -> Network (Legacy)
4. Enable the network stack
Advanced -> Network Stack Configuration -> Netwrk Stack (Enable)
Advanced -> Network Stack Configuration -> Ipv4 PXE Support (Enable)
5. Enable the LAN of network port
Advanced -> <the network device that connect to PXE server> -> NIC Configuration -> Wake on LAN (Enable)
7. Boot -> Boot Mode ~~(Legacy)~~(UEFI) <font color="#f00">*(BIOS booting only)*</font>
8. Boot -> Boot Option #1 <Network>
---
## REMARK
* **ERROR**: can't allocate initrd.
* Cause: This error is happend when your ramdisk is too huge that size is bigger than grub support. e.g., grub 2.02 which is default grub of CentOS7.9 is just supoort less than 900MB
* Fix: Upgrade grub version.
* **ERROR**: timeout reading `ramdisk.cpio.gz'
* Cause: This error is happend when waiting time for loading ramdisk is too much.
* Fix: Change other transmission protocol, e.g., http is faster than tftp.
* **INFO**: grub 2.02 will defaultly read the grub.cfg being in the same path, while grub 2.06 can be regerated to redirecte grub.cfg path by --prefix with grub-mkimage.
---
## REFERENCE
* https://www.linuxquestions.org/questions/slackware-14/problems-booting-installer-to-uefi-system-via-pxe-4175696093/
* https://www.vmvps.com/how-to-solve-the-error-caused-by-usr-bin-mandb.html
* https://magiclen.org/linux-ramdisk/
* https://sites.google.com/site/gyozapriate/Home/linux-island/boot/embedded-system/kernel-initrd
* http://www.study-area.org/tips/floppy-linux.htm
* https://hackmd.io/@Jason199567/SkrT-zutl
* https://blog.csdn.net/wbcuc/article/details/115921714
* https://www.debian.org/releases/jessie/amd64/ch04s05.html.en
* https://www.universalmediaserver.com/forum/viewtopic.php?t=15075
* https://blog.csdn.net/ZCShouCSDN/article/details/125777014
* http://hmli.ustc.edu.cn/doc/linux/centos-autoinstall.htm
Sign in with Wallet
Connect another wallet