Project

General

Profile

Actions

Idea #21926

open

AMD ROCm GPU support

Added by Peter Amstutz 6 months ago. Updated 3 days ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
Crunch
Target version:
Start date:
Due date:
Story points:
-

Description

docker run -it --device=/dev/kfd --device=/dev/dri/card0 --device=/dev/dri/renderD128 --group-add=video --network=host --ipc=host --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --shm-size 16G -v [directory binding options] --name [ollama-blablabla] ollama/ollama:rocm

https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/docker.html


Files

Actions #1

Updated by Peter Amstutz 6 months ago

  • Description updated (diff)
Actions #2

Updated by Peter Amstutz 6 months ago

  • Description updated (diff)
Actions #3

Updated by Peter Amstutz 6 months ago

  • Description updated (diff)
Actions #4

Updated by Peter Amstutz 4 months ago

  • Subject changed from ROCm GPU support to AMD ROCm GPU support
Actions #5

Updated by Peter Amstutz 4 days ago

It works without --network=host --ipc=host --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --shm-size 16G

Those options all reduce security, so if we can make it work with just this, that's way better:

docker run -it --device=/dev/kfd --device=/dev/dri/card0 --device=/dev/dri/renderD128 --group-add=video -v [directory binding options] --name [ollama-blablabla] ollama/ollama:rocm

Actions #6

Updated by Peter Amstutz 3 days ago

Attached package implements prototype ROCm support in crunch run.

If AMD_VISIBLE_DEVICES is set when crunch-run is executed (you can set AMD_VISIBLE_DEVICES before running crunch-dispatch-local) then crunch-run will make the GPU devices available to the container.

Actions

Also available in: Atom PDF