Ollama - Remote hosts

[Fried-Squid]

Background

Currently, AutoGPT only supports ollama servers running locally. Often, this is not the case as the ollama server could be running on a more suited instance, such as a Jetson board. This PR adds "ollama host" to the input of all LLM blocks, allowing users to select the ollama host for the LLM blocks.

Changes 🏗️

• Changes contained within blocks/llm.py:
  • Adding ollama host input to all LLM blocks
  • Fixed incorrect parsing of prompt when passing to ollama in the StructuredResponse block
  • Used ollama.Client instances to accomplish this.

Testing 🔍

Tested all LLM blocks with Ollama remote hosts as well as with the default localhost value.

Related issues

#8225

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 3 committers have signed the CLA.

✅ Fried-Squid
❌ Ace (Fried_Squid / therift)
❌ AceH-Joblogic

---

Ace (Fried_Squid / therift) seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codiumai-pr-agent-pro]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 Security concerns Potential remote code execution: The addition of the ollama_host parameter allows users to specify arbitrary hosts for Ollama connections. If not properly validated and sanitized, this could potentially be exploited to connect to unintended hosts or execute arbitrary code. It's crucial to implement strict input validation for the ollama_host parameter to ensure only authorized and safe connections are allowed.

⚡ Recommended focus areas for review Potential Security Risk The ollama_host parameter is set with a default value of "localhost:11434". This could potentially allow users to connect to arbitrary hosts if not properly validated. Error Handling The changes introduce new network calls to potentially remote Ollama hosts, but there's no visible error handling for network-related issues. Code Duplication The ollama_host parameter is added to multiple Input classes, which could lead to duplication and maintenance issues.

[netlify]

✅ Deploy Preview for auto-gpt-docs canceled.

Name	Link
🔨 Latest commit	25df9d6
🔍 Latest deploy log	https://app.netlify.com/sites/auto-gpt-docs/deploys/66fbeb64d49d8d00087af9bd

[Bentlybro]

This is a super nice change and its much needed 🙏 once CI tests pass it should be good to go!

[Fried-Squid]

CI/CD all passing now 😄
Just forgot to run the formatter and change a type lol

[ntindle]

@Bentlybro is there a way to conditionally show stuff like this if ollama is the selected model:

[Bentlybro]

@ntindle i dont think we have that setup at the moment but maybe that is something we should look into getting setup because i already see quite a lot of use-cases for that?

[Fried-Squid]

Just thinking, would it be worth splitting model selection into provider then model, rather than just model? Some providers have a wide variety of models (i.e. Ollama) which may overlap with other providers, so being able to choose the model and provider would let users have more control. It'd also make it a tad easier to have conditional inputs on the blocks I imagine as we wouldn't have to look up provider based on the model, it'd be in the block inputs.

Let me know and I can try and get a pr out for that functionality soon

[Bentlybro]

@Fried-Squid I think that's a good very good idea, it makes a lot more sense being able to do it that way, @ntindle @Torantulino what do we think? it should be pretty easy to do and it should just be changes to the block its self as far as I can see.

[Fried-Squid]

@Bentlybro the one issue I see with it is getting the frontend to display that properly - we'd need to make changes to the JSON schema which gets passed to the frontend to render the available model selection. Or we could just throw an error when model and provider don't match, but that seems pretty hostile to new users who might not understand the differences between the providers and models.