Vision Service: Use an inference server or framework instead of custom microservices

[vigsterkr]

first of all kudos for photoprism! I think it is a great idea to factor out the ML part to a microservice, but i'm really wondering whether it is a good idea to develop an ML server from scratch. Especially looking at the current code, this one hard-codes models, and that is really not something that makes things scalable. there are plenty of mature ML serving development out there, TF has it's own TensorFlow Serving but that of course has its own problem, meaning it ties you to use a specific backend, namely TF.

https://github.com/roboflow/inference is something that comes to mind that is backend agnostic, but here's a quite good list of possible options: https://github.com/topics/inference-server

[lastzero]

My apologies for not getting back to you sooner! This is just a first version, and we wanted to integrate what we already have before making further changes to the vision service:

• AI: API to connect to external services for Image tagging / object recognition photoprism#1090
• Labels: Load and use a custom TensorFlow model photoprism#127 (comment)

Now that this is almost done, we can move on to improving the API implementation and models in this repository, e.g. by building on an existing inference server or framework. Any help with this would be very much appreciated! 🥇

[sgflt]

Just architecture question.

Is this module designed as proxy for photoprism/anyapp to other possible apis (roboflow, ollama, localai), or is it competetive API, or both?

photoprism -> photoprism-vision -+-> (other providers)
                                 |
                                 +-> local tensorflow

or

photoprism  -+-> (other providers)
             |
             +-> photoprism-vision     

or

photoprism -+-> (other providers) 
            | 
            +-> photoprism-vision -+-> (other providers)
                                   |
                                   +-> local tensorflow                                                           

[lastzero]

With the latest update, PhotoPrism (see main repository and release notes) can be configured to act as a model proxy, server, and client.

However, the API / server in this repository does not act as a proxy and using Python for this, when we can do it with Go (which we already have) probably makes no sense. We could however refactor the API to use a framework/server software specialized in this, and/or support standardized request and response formats, if there are any (we haven't found any suitable standards when we checked last year, but also didn't spend a lot of time doing research).