How to Set Up a Local Ubuntu Server to Host Ollama Models with a WebUI

Are you ready to set up a powerful local server to host Ollama models and interact with them via a sleek WebUI? This guide will take you through each step, from preparing your Ubuntu server to installing Ollama and integrating OpenWebUI for seamless interaction.

Whether you're a beginner or an experienced user, this comprehensive guide will make the process straightforward and error-free. Let's get started!

Installing Ubuntu Server on a PC

Before diving into the server setup, you need to install Ubuntu Server on your PC. Follow these steps to get started:

Step 1: Download Ubuntu Server ISO

1. Visit the Ubuntu Server Download Page.


2. Download the latest version of the Ubuntu Server ISO file.

Step 2: Create a Bootable USB Drive

Use tools like Rufus (Windows) or dd (Linux/Mac) to create a bootable USB drive:

• For Rufus: Select the ISO file and your USB drive, then click "Start."


• 
  

  For dd on Linux/Mac:
  
  

  
  

   sudo dd if=/path/to/ubuntu-server.iso of=/dev/sdX bs=4M status=progress
  
  

  
  
  
  

  Replace /dev/sdX with the appropriate USB device.

  
  

Step 3: Boot from USB and Install Ubuntu Server

1. Insert the USB drive into the PC and restart it.


2. Enter the BIOS/UEFI (usually by pressing DEL, F2, or F12 during startup).


3. Set the USB drive as the primary boot device and save the changes.


4. Follow the on-screen instructions to install Ubuntu Server.

• Select your language, keyboard layout, and network configuration.


• Partition the disk as needed (guided options work for most setups).


• Set up a username, password, and hostname for the server.

Complete the installation and reboot the system. Remove the USB drive during the reboot.

Setting Up Your Ubuntu Server

Step 1: Update and Install Essential Packages

To ensure your server is up-to-date and has the necessary tools, run the following commands:

sudo apt update && sudo apt upgrade -y

sudo apt install build-essential dkms linux-headers-$(uname -r) software-properties-common -y

Step 2: Add NVIDIA Repository and Install Drivers

If your server includes an NVIDIA GPU, follow these steps to install the appropriate drivers:

• Add the NVIDIA PPA:
  

   sudo add-apt-repository ppa:graphics-drivers/ppa -y

   sudo apt update

• Detect the recommended driver:
  

   ubuntu-drivers devices

Example output:

   driver   : nvidia-driver-560 - third-party non-free recommended

• Install the recommended driver:
  

   sudo apt install nvidia-driver-560 -y

   sudo reboot

• Verify the installation:
  

   nvidia-smi

This should display GPU details and driver version. If not, revisit the steps.

Step 3: Configure NVIDIA GPU as Default

If your system has an integrated GPU, disable it to ensure NVIDIA is the default:

• Identify GPUs:
  

   lspci | grep -i vga

• Blacklist the integrated GPU driver:
  

   sudo nano /etc/modprobe.d/blacklist-integrated-gpu.conf

Add the following lines based on your GPU type:

For Intel:

   blacklist i915

   options i915 modeset=0

For AMD:

   blacklist amdgpu

   options amdgpu modeset=0

• Update and reboot:
  

   sudo update-initramfs -u

   sudo reboot

Verify again with:

nvidia-smi

Installing and Setting Up Ollama

Step 1: Install Ollama

Download and install Ollama using the following command:

curl -fsSL https://ollama.com/install.sh | sh

Step 2: Add Models to Ollama

Ollama allows you to work with different models. For example, to add the llama3 model, run:

ollama pull llama3

Setting Up OpenWebUI for Seamless Interaction

To enhance your experience with Ollama, integrate OpenWebUI—a user-friendly interface for interacting with models:

• Run the following Docker command to set up OpenWebUI:
  

   sudo docker run -d --network=host -v open-webui:/app/backend/data \

       -e OLLAMA_BASE_URL=http://127.0.0.1:11434 \

       --name open-webui --restart always \

       ghcr.io/open-webui/open-webui:main

• 
  

  This command sets up a containerized WebUI with:

  
  
  
  
  
  • Data persistence via the open-webui volume.
  
  
  • Ollama base URL configuration for model interaction.
  
  
  
  
  
  




• Access the WebUI through your server's IP address.

Testing and Troubleshooting

Verify NVIDIA GPU Functionality

Use nvidia-smi to confirm the GPU is functioning properly. If you encounter errors like Command not found, revisit the driver installation process.

Common Errors and Fixes

Error: ERROR:root:aplay command not found

• Fix: Install alsa-utils:
  

  sudo apt install alsa-utils -y

Error: udevadm hwdb is deprecated. Use systemd-hwdb instead.

• Fix: Update system packages:
  

  sudo hwdb update

  sudo apt update && sudo apt full-upgrade -y

Optional: CUDA Setup for Compute Workloads

For advanced compute tasks, install CUDA tools:

• Install CUDA:
  

   sudo apt install nvidia-cuda-toolkit -y

• Verify CUDA installation:
  

   nvcc --version

Congratulations! You've set up a robust local Ubuntu server for hosting Ollama models and interacting with them via OpenWebUI. This setup is perfect for experimenting with AI models in a controlled, local environment.

If you encounter any issues, double-check the steps and consult the documentation. Enjoy exploring the possibilities of Ollama and OpenWebUI!