Skip to content

[Research] Add 2 elite-tier projects to Inference Engines & Serving#131

Merged
alvinreal merged 1 commit intomainfrom
research/inference-engines-2026-04-07
Apr 7, 2026
Merged

[Research] Add 2 elite-tier projects to Inference Engines & Serving#131
alvinreal merged 1 commit intomainfrom
research/inference-engines-2026-04-07

Conversation

@alvinreal
Copy link
Copy Markdown
Owner

Project: One-API\n\n### Elite Criteria Checklist (ALL Required)\n\n- [x] Elite Criteria: ALL criteria met\n - ⭐ Stars: 31,512 (threshold: 1000+)\n - 🔄 Active: 2026-01-09 (within 6 months)\n - 🏭 Production: LLM API gateway with rate limiting, quota management, and cost tracking. Production deployments across multiple providers.\n - 📚 Quality: MIT license, comprehensive docs, stable releases\n\n### Evidence of Production Usage\n- https://github.com/songquanpeng/one-api - Used as unified API gateway for managing multiple LLM providers\n\n### Why This Belongs in Elite Tier\nOne-API solves a critical production need: unifying disparate LLM provider APIs under a single OpenAI-compatible interface. It includes rate limiting, quota management, and cost tracking - essential for production deployments.\n\n### Category\nInference Engines & Serving - High-performance Serving & API Servers\n\n---\n\n## Project: OpenLLM (BentoML)\n\n### Elite Criteria Checklist (ALL Required)\n\n- [x] Elite Criteria: ALL criteria met\n - ⭐ Stars: 12,273 (threshold: 1000+)\n - 🔄 Active: 2026-04-06 (within 6 months)\n - 🏭 Production: Enterprise-grade LLM serving platform used in production environments\n - 📚 Quality: Apache 2.0 license, full documentation, regular releases\n\n### Evidence of Production Usage\n- https://github.com/bentoml/OpenLLM - Deploy and serve LLMs in cloud environments with OpenAI-compatible API\n- https://bentoml.com - Commercial platform offering managed OpenLLM deployments\n\n### Why This Belongs in Elite Tier\nOpenLLM from BentoML provides a complete production-grade solution for running open-source LLMs. It supports 50+ models with built-in streaming, batching, and auto-acceleration. The project is backed by a commercial entity (BentoML) ensuring ongoing maintenance and support.\n\n### Category\nInference Engines & Serving - High-performance Serving & API Servers\n\n---\n\n## Summary\n\nThis PR adds 2 elite-tier inference/serving projects to the Inference Engines & Serving category:\n\n1. One-API (31,512 ⭐) - LLM API management and key redistribution system\n2. OpenLLM (12,273 ⭐) - Production-grade platform for running open-source LLMs\n\nBoth projects meet all elite-tier criteria and complement the existing inference ecosystem coverage.

## Projects Added

### One-API (songquanpeng/one-api)

- ⭐ Stars: 31,512 (threshold: 1000+)

- 🔄 Active: 2026-01-09 (within 6 months)

- 🏭 Production: LLM API gateway with rate limiting and quota management

- 📚 Quality: MIT license, full documentation

### OpenLLM (bentoml/OpenLLM)

- ⭐ Stars: 12,273 (threshold: 1000+)

- 🔄 Active: 2026-04-06 (within 6 months)

- 🏭 Production: Enterprise-grade LLM serving platform

- 📚 Quality: Apache 2.0 license, OpenAI-compatible API

Category: Inference Engines & Serving (§3)

Research Date: 2026-04-07
@alvinreal alvinreal merged commit bc000bb into main Apr 7, 2026
2 checks passed
@alvinreal alvinreal deleted the research/inference-engines-2026-04-07 branch April 7, 2026 08:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant