Linux Kernel Modules That Explain How Podman Really Works

Hi, readers this time I want to show you how to build and run three Linux kernel modules that illustrate core Operating Systems concepts: kernel debug messages, character devices, and the relationship between kernel-level information and user-space processes/containers. Just like in my as our environment, and the . This gives us a disposable Ubuntu VM with root access, which is exactly what we need to build and load kernel modules (something you normally cannot do inside a regular container, since kernel modules run in the host's kernel space).

The source code for the three modules lives in the script that installs the required build dependencies (things like build-essential, linux-headers-$(uname -r), and make). Inside every module folder:

• 
  make all compiles the module and produces a .ko (kernel object) file.


• 
  make clean removes the build artifacts.

Statement

Build and run the 3 kernel modules, showing or explaining what is requested for each one.

How to install kernel headers and build dependencies for kernel modules

1. Clone the repository inside the Killercoda Ubuntu environment:

git clone https://github.com/sergioarmgpl/kernel-modules.git

cd kernel-modules

2. Install the dependencies needed to build kernel modules (kernel headers, compiler, make, etc.):

chmod +x ins_dep.sh

./ins_dep.sh

This script takes care of installing the Linux kernel headers that match the running kernel (linux-headers-$(uname -r)) plus the basic build toolchain, which is required because kernel modules must be compiled against the exact kernel version you are running.

How to install Podman on Ubuntu

Since the Killercoda Ubuntu playground doesn't ship with Podman preinstalled, the first thing we need to do is install it from the official Ubuntu repositories:

sudo apt-get update

sudo apt-get install -y podman

Verify the installation:

podman --version

You should see an output similar to:

podman version 4.9.3

Podman is daemonless: it doesn't need a long-running background service running as root to manage containers. Instead, it launches each container as a regular child process of the CLI itself, which fits nicely with what we are going to observe later with module3 when we inspect the kernel's process table.

How to compile a kernel module with make and load/unload it with insmod/rmmod, and read kernel debug messages with dmesg (Module 1)

module1 is the classic "Hello World" of Linux kernel programming. Its only job is to register init and exit functions and print a message to the kernel ring buffer (accessible with dmesg) when the module is loaded and unloaded, using macros like printk(KERN_INFO ...).

1. Compile the module:

cd module1

make all

2. Load the module into the running kernel:

sudo insmod module1.ko

3. Show the debug message printed by the module using the kernel ring buffer:

dmesg | tail -n 10

You should see an entry similar to:

[ 1234.567890] Module 1: Hello, Kernel! Module loaded successfully.

4. Once you are done, unload the module and clean the build:

sudo rmmod module1

make clean

dmesg shows you both the load message (printed inside the module's init function) and, after rmmod, the unload message (printed inside the exit function). This is the most basic proof that a piece of your own code is now running as part of the Linux kernel.

How to interact with a character device created by a kernel module (Module 2)

What does the module do? module2 goes one step further than module1: instead of only printing a message, it registers a character device in the kernel. A character device is one of the ways the kernel exposes functionality to user space as if it were a regular file, implementing callback functions such as open, read, write, and release. When the module is loaded, it creates a device node (for example /dev/module2 or similar, depending on the major/minor number it registers), and every time a user-space process interacts with that file, the corresponding callback inside the module gets executed.

1. Compile and load the module:

cd ../module2

make all

sudo insmod module2.ko

2. Check that the device was created (either automatically under /dev or by creating the node manually with mknod, depending on how the module registers itself), and look at the assigned major number:

dmesg | tail -n 10

cat /proc/devices | grep module2

ls -l /dev/module2

3. Write to the device created by the module:

echo "Hello from user space" | sudo tee /dev/module2

You can also read back from it to see how the module responds to the data it received:

sudo cat /dev/module2

4. Explaining the kernel messages of module 2:

Every time you load the module, write to the device, or read from it, module2 prints tracing messages through printk, which you can inspect with:

dmesg | tail -n 20

These messages typically show:

• A message when the device is opened, confirming a process obtained a file descriptor to /dev/module2.


• A message when data is written, usually including how many bytes were received and, in some implementations, echoing back the content that was sent.


• A message when the device is read, showing how many bytes were copied back to user space.


• A message when the device is released (closed).

This illustrates one of the core ideas of an Operating System: user-space processes never touch hardware or kernel memory directly, they go through system calls (open, read, write, close), and the kernel module is the piece of code that decides what happens on the other side of that system call.

5. Clean up:

sudo rmmod module2

make clean

How kernel-level process information relates to podman ps and OS process commands like ps (Module 3)

1. Before loading the module, let's create some containers so we have real workloads running on the system. Create an nginx, a redis, and a mongo container using Podman:

podman run -d --name web nginx

podman run -d --name cache redis

podman run -d --name db mongo

2. List the containers using Podman:

podman ps

You should see the three containers (web, cache, db) with their container IDs, image names, and status.

3. List the same workloads from the Operating System's point of view, using standard process commands. Since every container is ultimately just one (or more) Linux processes running in isolated namespaces and cgroups, you can find them with:

ps -ef | grep -E "nginx|redis|mongo"

pstree -p | grep -E "nginx|redis|mongod"

top

This shows that containers are not "magic": nginx, redis-server, and mongod show up as regular PIDs on the host, the same way any other Linux process would. This is especially visible with Podman, since it is daemonless and runs each container as a child process of the conmon/podman process tree itself, rather than hiding it behind a separate background service.

4. Now compile and load module3:

cd ../module3

make all

sudo insmod module3.ko

5. Inspect the information the module loads into the kernel log:

dmesg | tail -n 30

Explaining what module 3 loads and how it relates to the previous commands:

module3 walks the kernel's internal task list (the same in-kernel data structure that backs commands like ps and top) and prints information about the currently running processes: PID, process name (comm), and often parent PID or state. In other words, it is doing at the kernel level exactly what ps -ef does at the user-space level, except it reads the data directly from kernel structures (such as task_struct, traversed with helpers like for_each_process()) instead of going through /proc.

This is why the output of dmesg after loading module3 will include entries for nginx, redis-server, mongod, their conmon parents, and other system processes: they are all regular entries in the kernel's process table, the exact same list that podman ps indirectly depends on (Podman only adds a mapping between container IDs/names and the PIDs and namespaces the kernel is already tracking, without needing a central daemon to do so).

Putting it all together:

• 
  podman ps shows you the container abstraction: names, images, ports.


• 
  ps / pstree / top show you the OS process abstraction: PIDs, parents, CPU/memory usage.


• 
  module3 shows you the kernel data structures that make both of the previous views possible in the first place.

6. Clean up everything:

sudo rmmod module3

make clean

podman rm -f web cache db

Conclusion about kernel modules, virtualization, and processes

These three modules build on top of each other conceptually. module1 proves that your own code can run inside the kernel and log messages. module2 shows how the kernel exposes an interface to user space through a device file, using the same open/read/write model that every driver in Linux follows. module3 closes the loop by showing that container runtimes like Podman are built on top of the very same primitives the kernel already exposes for processes, namespaces and cgroups: there is no separate "container world", it's all Linux processes, all the way down — and Podman's daemonless design makes that especially visible, since every container you run stays traceable as an ordinary process tree.

Thanks for reading! Last to say, see you in my next blog post.

References