How to run Ollama using GPU acceleration only in WSL2?

Question

I am trying to run Ollama on WSL2 (Ubuntu 22.04) with GPU acceleration (CUDA), but it still heavily relies on CPU instead of utilizing only the NVIDIA GPU.

System Details:

• OS: Windows 10 (WSL2 with Ubuntu 24.04)
• GPU: NVIDIA RTX 3090 ti(Supports CUDA)
• CUDA Version: 12.8 (installed in /usr/local/cuda-12.8/)
• NVIDIA Drivers: Latest installed on Windows
• Ollama Version: 0.5.11
• Model Used: deepseek-r1:1.5b

Problem

I have installed CUDA and Nvidia drivers.

root@PC01:/mnt/c/Windows/system32# nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2025 NVIDIA Corporation Built on Wed_Jan_15_19:20:09_PST_2025 Cuda compilation tools, release 12.8, V12.8.61 Build cuda_12.8.r12.8/compiler.35404655_0

nvidia-smi

I tried changing the driver versions.

updating the sudo nano ~/.ollama/config.yaml to include use_cuda: true num_threads: 1

also exporting these values

echo 'export CUDA_VISIBLE_DEVICES=0' >> ~/.bashrc echo && 'export OLLAMA_USE_CUDA=1' >> ~/.bashrc && echo 'export OLLAMA_NUM_THREADS=1' >> ~/.bashrc && source ~/.bashrc