Building a comfortable workflow for debugging an old version of the Linux kernel

When you want to work on the Linux kernel, for example to see how an exploit acts (like Dirty COW on kernel 4.7), you need to build a comfortable working environment.

I use:

• 
  Docker for compiling sources in their original version (with the GCC/LD versions corresponding to the source era).


• 
  QEMU for running the executables in a virtual machine.


• 
  VS Code as a debugger.

Through this article, I will show you how I configured my environment to achieve an efficient setup.

1) Compiling source code

The first step is to retrieve the kernel source code.

For relatively recent versions (beyond 5.10), the simplest way is to download the tar file from kernel.org. For older versions, the best solution is to fetch them via git.

For example, for 4.7, we start by retrieving the tags:

git ls-remote --tags git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git | grep "v4.7"

Then we clone it to get the version without downloading the entire history:

git clone --depth 1 --branch v4.7 git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 4.7

After a few minutes, you have a 4.7/ directory containing a working version of the 4.7 kernel sources.

However, these sources are not compilable as-is. Compiling a 4.7 kernel (released in 2016) with a modern compiler (GCC 13+) produces compilation errors due to changes in C language standards or security attribute handling. This is why you need to load versions of GCC and LD compatible with these sources (see the table below), and the easiest way to do this is via Docker.

We need to retrieve the Docker container corresponding to the kernel version we are interested in. For 4.7, it is Ubuntu 16.04.

To search for available containers, there are 2 solutions:

a) Run the command:

skopeo list-tags docker://docker.io/library/ubuntu

you should see something like

{

"Repository": "docker.io/library/ubuntu",

"Tags": [

"10.04",

"12.04",

"12.04.5",

"12.10",

"13.04",

"13.10",

"14.04",

"14.04.1",

...

...then run the command:

sudo docker pull ubuntu:16.04

b) Go to Docker Hub

and download it.

After a few minutes, you can run the following command to check that your container was downloaded

sudo docker images

and you should see

kernel-builder-515:latest 9f311ec05f16 580MB 147MB U


ubuntu:16.04 1f1a2d56de1d 195MB 46.5MB U

I always favor LTS (Long Term Support) versions like 16.04, because they guarantee library longevity and compilation tool stability, which is critical to avoid linking errors when generating vmlinux.

See the difference here: ubuntu:16.04 is the raw image we just downloaded, while kernel-builder-515 represents the kernel 5.15 image that I customized to include all the necessary compilation tools. We will now cover this customization step using the Dockerfile.

To customize the downloaded Docker container, we use a Dockerfile, here is an example:

### Example for kernel 4.7 (based on Ubuntu 16.04 - Xenial)

FROM ubuntu:16.04



### Install compilation dependencies

RUN apt-get update && apt-get install -y \

    build-essential \

    libncurses5-dev \

    bison \

    flex \

    libssl-dev \

    libelf-dev \

    gcc-5 g++-5 \

    && update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 100



WORKDIR /usr/src/linux

To create a personal container version, just run the command:

sudo docker build -t my-ubuntu:16.04 .

if we launch sudo docker images, you should see the following result

kernel-builder-515:latest 9f311ec05f16 580MB 147MB U


my-ubuntu:16.04 3aa5b81c120f 822MB 188MB U


ubuntu:16.04 1f1a2d56de1d 195MB 46.5MB U

You can find other Dockerfile examples at https://github.com/ddupard/low-level-security-research.

If you have followed all previous steps, you should be able to compile without issues by running these commands in a shell:

sudo docker run -it \

  -v ~/Desktop/KERNEL/4.7:/build \

  my-ubuntu:16.04



make x86_64_defconfig



scripts/config --enable CONFIG_DEBUG_INFO

scripts/config --disable CONFIG_DEBUG_INFO_REDUCED

scripts/config --enable CONFIG_GDB_SCRIPTS

scripts/config --enable CONFIG_DEBUG_KERNEL

scripts/config --disable CONFIG_RANDOMIZE_BASE



make olddefconfig

make scripts_gdb



make -j$(nproc)

et voila

2) Creating the runtime

This step consists of two parts:

• Retrieving and compiling BusyBox.


• Creating the initramfs.cpio.gz.

Unlike a complete Linux distribution that weighs gigabytes, BusyBox bundles essential utilities (ls, cat, sh, etc.) into a single binary executable.

We start by creating a shell script (build_busybox.sh) to download the BusyBox sources and compile them within Docker using my-ubuntu:16.04.

#!/bin/bash



BUSYBOX_VERSION="1.36.1"

BUSYBOX_TAR="busybox-$BUSYBOX_VERSION.tar.bz2"

BUILD_DIR="$(pwd)/busybox_build"



if [ ! -f "$BUSYBOX_TAR" ]; then

    echo "Downloading BusyBox $BUSYBOX_VERSION..."

    wget "https://busybox.net/downloads/$BUSYBOX_TAR"

fi



tar xvf "$BUSYBOX_TAR"

cd "busybox-$BUSYBOX_VERSION"



make defconfig

sed -i 's/.*CONFIG_STATIC.*/CONFIG_STATIC=y/' .config



### 4. Compilation

echo "Compilation of BusyBox..."

make -j$(nproc)

make install CONFIG_PREFIX="$BUILD_DIR"



echo "BusyBox compiled in : $BUILD_DIR"

then we launch the following command

sudo docker run -it \

  -v "$(pwd)":/build \

  -w /build \

  my-ubuntu:16.04 \

  /bin/bash ./build_busybox.sh

the compilation fails with the following message indicating that wget and other executables are missing

sudo docker run -it \

-v "$(pwd)":/build \

-w /build \

my-ubuntu:16.04 \

/bin/bash ./build_busybox.sh

Téléchargement de BusyBox 1.36.1...

./build_busybox.sh: line 11: wget: command not found

tar: busybox-1.36.1.tar.bz2: Cannot open: No such file or directory

tar: Error is not recoverable: exiting now

so we need to modify the Dockerfile in order to include these new executables

FROM ubuntu:16.04



RUN apt-get update && apt-get install -y \

    build-essential \

    gcc \

    make \

    libncurses5-dev \

    libssl-dev \

    bc \

    bison \

    flex \

    nano \

    git \

    cpio \

    openssh-client \

    wget \

    bzip2 \

    ca-certificates \

    linux-headers-generic \

    && rm -rf /var/lib/apt/lists/*





WORKDIR /opt

RUN git clone https://github.com/radareorg/radare2 \

    && radare2/sys/install.sh --install \

    && rm -rf radare2



WORKDIR /build

now we can recreate the container using

sudo docker build -t my-ubuntu:16.04 .

then we rebuild busybox with the following command

sudo docker run -it \

  -v "$(pwd)":/build \

  -w /build \

  my-ubuntu:16.04 \

  /bin/bash ./build_busybox.sh

however the compilation process wil fail with the following message

CC miscutils/mt.o

CC miscutils/nandwrite.o

CC miscutils/partprobe.o

CC miscutils/raidautorun.o

CC miscutils/readahead.o

CC miscutils/runlevel.o

CC miscutils/rx.o

CC miscutils/seedrng.o

miscutils/seedrng.c:45:24: fatal error: sys/random.h: No such file or directory

it's because you are compiling a recent version of BusyBox with an old Ubuntu 16.04 base.

in order to solve our problem, we will launch the following commands

sudo docker ps -a

in order to get the list of the last docker commands launched

then

sudo docker start 49c7144e4dbb

sudo docker exec -it 49c7144e4dbb /bin/bash

to restart the docker environment (root@49c7144e4dbb:/build#)

then the following commands

cd /build/busybox-1.36.1

sed -i 's/CONFIG_SEEDRNG=y/CONFIG_SEEDRNG=n/' .config



make clean

make -j$(nproc)

compilation should restart and go to the end.

then we launch the following command to make the install

make install CONFIG_PREFIX="$(pwd)/../busybox_build"

at the end a new directory should appear

Now that BusyBox has copiled, it's time to create initramfs.cpio.gz that sets up the file structure (/proc, /sys, etc.)

we are going to create and lauch a new shell script (create_initramfs.sh)

#!/bin/bash



BUILD_DIR="$(pwd)/busybox_build"

ROOTFS_DIR="$(pwd)/initramfs_staging"



rm -rf "$ROOTFS_DIR"

mkdir -p "$ROOTFS_DIR"

cp -a "$BUILD_DIR/"* "$ROOTFS_DIR/"



cd "$ROOTFS_DIR"

mkdir -p proc sys dev etc/init.d



cat <<EOF > init

#!/bin/sh

mount -t proc none /proc

mount -t sysfs none /sys

echo "--- Système démarré avec succès ---"

exec /bin/sh

EOF

chmod +x init



find . -print0 | cpio --null -ov --format=newc | gzip > ../initramfs.cpio.gz

at the end of its execution, we should see something like

./bin/mknod

./bin/more

./bin/zcat

./bin/pidof

./bin/egrep

./bin/watch

./bin/link

./bin/pwd

./bin/dd

./bin/netstat

./bin/fgrep

./bin/stty

./bin/mt

./bin/gunzip

./proc

./etc

./etc/init.d

./init

5556 blocks

and a new file in the root directory (initramfs.cpio.gz)

to check that everything is fine , we are going to launch the VM using the following command

qemu-system-x86_64 \

-kernel ./4.7/arch/x86/boot/bzImage \

-initrd ./initramfs.cpio.gz \

-append "console=ttyS0 nokaslr" \

-nographic

and something like below should appear

if it happens , congrats your environment is working now

3) Debugging setup

To debug the kernel, the base tool is GDB, but it isn't very user-friendly. The idea here is to use VS Code's overlay for GDB.

First, ensure you compile with debug options in your compile_4.7.sh script:

./scripts/config --enable CONFIG_DEBUG_INFO

./scripts/config --disable CONFIG_DEBUG_INFO_REDUCED

./scripts/config --enable CONFIG_GDB_SCRIPTS

./scripts/config --enable CONFIG_DEBUG_KERNEL

./scripts/config --disable CONFIG_RANDOMIZE_BASE

Verify with:

file ./4.7/vmlinux

(You should see a line containing "with debug_info")

Then, launch QEMU with the -s and -S options:

• 
  -s: Opens a GDB server on tcp::1234.


• 
  -S: Freezes QEMU at the first instruction.

qemu-system-x86_64 \

-kernel ./4.7/arch/x86/boot/bzImage \

-initrd ./initramfs.cpio.gz \

-append "console=ttyS0 nokaslr" \

-nographic \

-s -S

In VS Code, install the C/C++ extension from Microsoft and create a .vscode/launch.json file:

{

  "version": "0.2.0",

  "configurations": [

    {

      "name": "(gdb) Attacher au Kernel",

      "type": "cppdbg",

      "request": "launch",

      "program": "${workspaceFolder}/4.7/vmlinux",

      "miDebuggerServerAddress": "localhost:1234",

      "miDebuggerPath": "/usr/bin/gdb",

      "setupCommands": [

        { "text": "-enable-pretty-printing" }

      ],

      "cwd": "${workspaceFolder}"

    }

  ]

}

Debugging Workflow

1. Launch QEMU (using your script with -s -S).


2. In VS Code, go to the 4.7 directory and launch the "Debug Kernel LKM" configuration.


3. This triggers the kernel to start. You can now set breakpoints, view registers, and use the debug console to launch commands like the following

-exec info functions

-exec p/x $rax

-exec monitor xp/512gx 0x4786000

Note: If you load an LKM, you must load the symbol file and tell GDB where the source files are located using:

-exec add-symbol-file /path/to/my_attack.ko ...

-exec set substitute-path ...

Happy kernel debugging!

Appendix: Automation Script

To make the environment setup more efficient, I use a script (run_qemu.sh) to quickly switch between different kernel versions and automate the LKM injection process for the 5.15 LTS kernel.

You can save the following code into a file named run_qemu.sh in your ~/Desktop/KERNEL directory. Make sure to give it execution permissions with chmod +x run_qemu.sh.

#!/bin/bash



# This script runs on the host (Ubuntu) in ~/Desktop/KERNEL

cd "$HOME/Desktop/KERNEL"



echo "-------------------------------------------------------"

echo "  COGNITIVE FIREWALL LAB - TEST ENVIRONMENT"

echo "-------------------------------------------------------"

echo "Select the kernel version to test:"

echo "1) Kernel 4.7  (Dirty COW vulnerable)"

echo "2) Kernel 4.8  (Intro XDP)"

echo "3) Kernel 4.9  (Dirty COW patched)"

echo "4) Kernel 5.15 (LTS)"

echo "q) Quit"

echo "-------------------------------------------------------"



read -p "Your choice [1-4] : " choice



case $choice in

    1) VERSION="4.7"; KERNEL_PATH="./4.7/arch/x86/boot/bzImage" ;;

    2) VERSION="4.8"; KERNEL_PATH="./4.8/arch/x86/boot/bzImage" ;;

    3) VERSION="4.9"; KERNEL_PATH="./4.9/arch/x86/boot/bzImage" ;;        

    4)

        VERSION="5.15"

        KERNEL_PATH="./5.15/arch/x86/boot/bzImage"

        LKM2_PATH="./lkm/poc/my_attack.ko"



        if [ -f "$LKM2_PATH" ]; then

            echo "Building initramfs for Kernel 5.15..."

            mkdir -p .initramfs_root

            cd .initramfs_root

            zcat ../initramfs.cpio.gz | cpio -idmv &>/dev/null

            cp "../$LKM2_PATH" .



            # Automatic LKM injection script

            cat << 'EOF' > init

#!/bin/sh

mount -t proc proc /proc

mount -t sysfs sysfs /sys

mount -t devtmpfs devtmpfs /dev 2>/dev/null

if [ -f /my_attack.ko ]; then insmod /my_attack.ko; fi

exec /bin/sh

EOF

            chmod +x init

            find . -print0 | cpio --null -ov --format=newc 2>/dev/null | gzip -9 > ../initramfs.cpio.gz

            cd ..

            rm -rf .initramfs_root

        fi

        qemu-system-x86_64 -kernel "$KERNEL_PATH" -initrd "$HOME/Desktop/KERNEL/initramfs.cpio.gz" -nographic -append "console=ttyS0 loglevel=7 nosmep nosmap nokaslr" -s -S

        exit 0

        ;;

    q) exit 0 ;;

    *) echo "Invalid choice"; exit 1 ;;

esac



# Launch QEMU for versions 1, 2, 3

qemu-system-x86_64 \

  -kernel "$KERNEL_PATH" \

  -initrd "$HOME/Desktop/KERNEL/initramfs.cpio.gz" \

  -nographic \

  -append "console=ttyS0 nokaslr loglevel=7" \

  -s -S