Open WebUI and AnythingLLM integration compatibility issues

[Michael-A-Kuykendall]

Problem Description

Users report that popular LLM frontends cannot integrate with shimmy despite working with Ollama.

User Report from Issue #66

User: @barseghyanartur
Status: CURL works but frontends fail

Working:
✅ Direct CURL requests to shimmy API
✅ Both frontends work seamlessly with Ollama

Not Working:
❌ Open WebUI cannot detect/use shimmy
❌ AnythingLLM cannot detect/use shimmy

Expected Behavior

Popular LLM frontends should work with shimmy's OpenAI-compatible API just like they do with Ollama.

Actual Behavior

• Open WebUI: Cannot detect shimmy as model provider
• AnythingLLM: Refuses to automatically detect shimmy model provider
• Both tools work immediately with Ollama on same system

Technical Investigation Required

1. API Response Format: Are shimmy's responses matching OpenAI specification exactly?
2. Models Endpoint: Does /v1/models return the format these frontends expect?
3. Chat Completions: Are response fields complete and properly formatted?
4. CORS/Authentication: Are there connection issues preventing frontend access?
5. Model Detection: How do these frontends discover available models?

Compatibility Gap Analysis

Since both frontends work with Ollama but not shimmy, this suggests shimmy's OpenAI compatibility implementation has gaps that prevent real-world frontend integration.

Impact

This blocks users from using shimmy with popular LLM interfaces, limiting adoption compared to Ollama.

Affected Users

• @barseghyanartur (Issue [Question]: How to use it with Open WebUI? #66)

Related Issues

• Issue [Question]: How to use it with Open WebUI? #66: Original Open WebUI integration question
• Issue [Feature]: Open WebUI, AnythingLLM suppprt #70: Feature implementation tracking

Labels

• bug
• compatibility
• api
• frontend-integration

[Michael-A-Kuykendall]

✅ Issue Resolved

Solution: OpenAI API compatibility has been enhanced for better frontend integration.

Fix Details: Enhanced the Model structure in src/openai_compat.rs to include:

• Optional permission field (Vec)
• Optional root field (String) for model hierarchy
• Optional parent field (String) for model relationships
• Proper JSON serialization with skip_serializing_if for None values
• Full backward compatibility maintained

Commit: 845165b - fix(api): enhance OpenAI API compatibility for frontend integration (Issue #113)

Testing: Confirmed via comprehensive regression testing and API structure validation. Frontend applications should now receive all expected fields in the /v1/models response.

Feel free to reopen this ticket if you encounter any compatibility issues with frontend applications.

— Mike

[Michael-A-Kuykendall]

✅ Issue Resolved

Hi there! Good news - the OpenAI API compatibility issues with Open WebUI and AnythingLLM have been resolved and merged into main.

Solution: Enhanced the OpenAI-compatible Model structure to include all fields expected by frontend applications:

• Optional permission field (array of strings)
• Optional root field (string) for model hierarchy
• Optional parent field (string) for model relationships
• Proper JSON serialization with skip_serializing_if for clean responses

Commit: 37f67ac - fix(api): enhance OpenAI API compatibility for frontend integration (Issue #113) (#123)

Testing: This has been thoroughly tested with our API structure validation suite and should resolve the compatibility issues you were experiencing with Open WebUI and AnythingLLM frontends.

If you're eager to test this fix before the next release, you can checkout commit 37f67ac or pull from main branch.

Feel free to reopen this ticket if you still encounter compatibility issues with frontend applications after this fix.

— Mike

[barseghyanartur]

When will it be released?

[Michael-A-Kuykendall]

Today is the plan.