如果是在Ubuntu Docker NVIDIA 显卡上部署Ollama推荐直接使用官方 Docker 镜像并开启 GPU 支持。1. 检查显卡驱动确认 NVIDIA 驱动已安装nvidia-smi正常会看到类似-----------------------------------------------------------------------------| NVIDIA-SMI 550.xx Driver Version: 550.xx CUDA Version: 12.x |-----------------------------------------------------------------------------如果提示nvidia-smi: command not found或者没有识别显卡需要先安装驱动。2. 安装 DockerUbuntusudo apt updatesudo apt install -y ca-certificates curl gnupg lsb-releasecurl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpgecho deb [arch$(dpkg --print-architecture)signed-by/usr/share/keyrings/docker-archive-keyring.gpg] /spanIndex of linux/ubuntu/ /span$(lsb_release -cs) stable | /spansudo tee /etc/apt/sources.list.d/docker.listsudo apt updatesudo apt install -y docker-ce docker-ce-cli containerd.io启动sudo systemctl enable dockersudo systemctl start docker测试docker --version3. 安装 NVIDIA Container Toolkit让 Docker 能调用 GPU。添加仓库curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpgcurl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed s#deb https://#deb [signed-by/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list安装sudo apt updatesudo apt install -y nvidia-container-toolkit配置 Dockersudo nvidia-ctk runtime configure --runtimedocker重启 Dockersudo systemctl restart docker4. 验证 Docker GPU测试容器能否看到显卡docker run --rm --gpus all /spannvidia/cuda:12.6.0-base-ubuntu22.04 /spannvidia-smi如果容器内输出显卡信息说明 GPU 已正常透传。5. 部署 Ollama拉取镜像docker pull ollama/ollama启动docker run -d /span--name ollama /span--restart unless-stopped /span--gpus all /span-p 11434:11434 /span-v ollama:/root/.ollama /spanollama/ollama查看日志docker logs -f ollama6. 下载模型进入容器docker exec -it ollama bash下载模型Qwen3ollama pull qwen3:8bDeepSeek R1ollama pull deepseek-r1:8bLlama 3ollama pull llama3:8b7. 测试运行命令行docker exec -it ollama ollama run qwen3:8b输入你好即可对话。API测试curl http://localhost:11434/api/generate /span-d {model:qwen3:8b,prompt:介绍一下Ollama}8. 查看是否使用 GPU安装模型运行后docker exec -it ollama ollama ps或者直接观察watch -n 1 nvidia-smi运行推理时看到 GPU 显存增加例如GPU Memory Usage:12000 MiB / 24576 MiB说明 Ollama 已经在使用 GPU。Docker Compose 方式推荐docker-compose.ymlservices:ollama:image: ollama/ollama:latestcontainer_name: ollamarestart: unless-stoppedports:- 11434:11434volumes:- ollama:/root/.ollamadeploy:resources:reservations:devices:- driver: nvidiacount: allcapabilities: [gpu]volumes:ollama:启动docker compose up -d常见问题GPU 未识别检查docker run --rm --gpus all ubuntu nvidia-smi如果失败通常是NVIDIA 驱动未安装nvidia-container-toolkit 未安装Docker 未重启显存不足查看模型大小模型建议显存Qwen3 4B6GBQwen3 8B10GBDeepSeek-R1 8B12GBLlama3 8B10GBQwen3 32B24GB