Access Nvidia gpu inside docker container in Arch linux

Why

• If you need gpu acceleration in a container or just have too much free time on your hands, you can run all kinds of things. For example Large Language Models or GUI apps.


• You can even create your own gpu accelerated docker image by using Nvidia ubuntu images
  

How

• We have to install the NVIDIA Container Toolkit


• 
  

  The package can be installed from the Extras repository
  
  

  
  

  sudo pacman -S nvidia-container-toolkit
  
  

  
  
  

1. 
   

   Configure the container runtime by using the nvidia-ctk command. The nvidia-ctk command modifies the /etc/docker/daemon.json file on the host. The file is updated so that Docker can use the NVIDIA Container Runtime.
   
   

   
   

   sudo nvidia-ctk runtime configure --runtime=docker
   
   

   
   
   


2. 
   

   For the change to take effect we have to restart the docker daemon.
   
   

   
   

   sudo systemctl restart docker
   
   

   
   
   


3. 
   

   You can add the gpu using docker compose:
   
   

   
   

   service_name:
   
     image: ....
   
     deploy:
   
       resources:
   
         reservations:
   
           devices:
   
             - driver: nvidia
   
               count: 1
   
               capabilities: [gpu]
   
   

   
   
   
   

   If you use the docker run command, you can do:
   
   

   
   

   sudo docker run --runtime=nvidia --gpus all ....
   
   

   
   
   

Resources

• NVIDIA Container Toolkit documentation


• Toolkit Arch package page