Skip to content

Vision Service: Support Intel GPU #12

@alexmi256

Description

@alexmi256

Describe what problem this solves and why this would be valuable to many users
As far as I can tell, the docker compose.yaml example is hardcoded for nvidia GPUs due to:

      NVIDIA_VISIBLE_DEVICES: "all"
      NVIDIA_DRIVER_CAPABILITIES: "compute,utility"
    ## Share host devices to enable CPU/GPU hardware acceleration (optional):
    # devices:
    #  - "/dev/dri:/dev/dri"
    #  - "/dev/nvidia0:/dev/nvidia0"
    #  - "/dev/nvidiactl:/dev/nvidiactl"
    #  - "/dev/nvidia-modeset:/dev/nvidia-modeset"
    #  - "/dev/nvidia-nvswitchctl:/dev/nvidia-nvswitchctl"
    #  - "/dev/nvidia-uvm:/dev/nvidia-uvm"
    #  - "/dev/nvidia-uvm-tools:/dev/nvidia-uvm-tools"
    deploy:
      resources:
        reservations:
          devices:
            - driver: "nvidia"
              capabilities: [ gpu ]
              count: "all"

This may mean that other GPUs are not supported which leaves out quite a few users. (does this mean everything is CPU run?)

I looked a bit into the docker deploy but don't really get how docker knows to reserve the right nvidia resources

Describe the solution you'd like

Add support for intel GPUs similar to what https://github.com/eleiton/ollama-intel-arc does which I believe is done using https://github.com/intel/ipex-llm

Describe alternatives you've considered

https://github.com/eleiton/ollama-intel-arc does provide an Ollama for Intel GPU service which can be used with the remote llama integration. While I have tested it individually, I have not yet tested it with photoprism-vision.

It may also be that the ollama performance is on par or better than the native models listed (Kosmos-2/VIT-GPT2/BLIP) in which case this may not be a huge issue as ollama-intel-arc could be used as a potential workaround

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions