How does this project differ from Envoy Gateway?

[stevehipwell]

I'd like to understand how kgateway is different from the Envoy Gateway project? IIRC Envoy Gateway was kicked off as a cross company CNCF initiative to simplify the ecosystem and Tetrate were mentioned in the original announcement.

[linsun]

Hi Steve (@stevehipwell)!

Thank you for your great question! At KubeCon NA 2024, Solo.io announced its intention to donate the Gloo Gateway open source project as kgateway to the CNCF, to enrich the broader cloud native ecosystem. In case you are not familiar with Gloo Gateway, it is the most mature and widely deployed Envoy-based gateway in the market today. Built on open source and open standards, Gloo Gateway is Kubernetes-native and implements the Kubernetes Gateway API.

Kgateway is being donated to CNCF as a standalone project - a rebrand of a very mature open source gateway project that has been around since 2018. Envoy Gateway is a subproject of Envoy. Envoy became a CNCF graduated project in 2018 before Envoy Gateway was created. CNCF is known as a vendor neutral home for cloud native projects so it is common to have more than 1 API gateway projects within CNCF.

The following are key differences between Gloo/kgateway and Envoy Gateway:

Adopters: Based on both projects’ websites (https://kgateway.dev/ & https://gateway.envoyproxy.io/), Gloo/kgateway has a much larger list of adopters with many brand names such as T-Mobile, SAP, OfficeDepot, Carfax, ADP, BMW, and so on, handling large scale, mission critical enterprise systems API calls.

Features: Envoy gateway is primarily Kubernetes based while Gloo/kgateway supports not only Kubernetes but also other environments such as AWS Lambda. Gloo/kgateway supports rich API gateway features such as request transformation, CSRF or CORS or local rate limiting, Istio sidecar and ambient integration.

Maintainers: Kgateway currently has over 20 maintainers including 4 maintainers outside of solo.io. Envoy gateway has 5 active maintainers, with 6 emeritus maintainers. Gloo/kgateway was initially created in 2018 releasing over 1200 artifacts, while Envoy gateway was first established in 2022, releasing over 30 artifacts.

Architecture & Scale: Kgateway is the newest evolution of Gloo Gateway, taking ~7 years of lessons learned and refining our battle-proven Envoy control-plane to be able to handle enormous scale.

To learn more about kgateway and its future roadmap, read our blog that was published very recently. Feel free to let us know if you have any future questions or join one of our upcoming community meetings.

[stevehipwell]

OK, I'm familiar with the history and I'm not saying one project is better than the other. But given the current size of the CNCF landscape and the general sense of end-user (and maintainer) fatigue I was interested in why kgateway was a new project rather than a way to level up the existing project (which was backed by the Envoy project as well as the vendors). Having incompatible architecture and goals would be a good reason for there to be a new project, but without supporting documentation explaining "why" there is no way to validate this.

[linsun]

One addition point I wanted to add is a key principal of CNCF is no kingmakers and one size doesn't fit all, except Kubernetes. In this case, it would be odd for kgateway to join force with Envoy Gateway when kgateway is 4 years ahead on adoption, stability and have different architecture & extensibility. The goal of kgateway is to provide our users the best Envoy based API gateway in CNCF leveraging all 7 years' expertise building Envoy based API gateways and I look forward to seeing you in the kgateway community!

[thomaslepoix]

Envoy gateway is primarily Kubernetes based

I'd just like to amend this sentence, since 1.2.0 Envoy Gateway supports being deployed outside of Kubernetes ("Standalone" mode).

[teaglebuilt]

@stevehipwell Envoy Gateway does not have specific versioning yet so i cannot speak 100 percent certainly. However, recently i have used Envoy Gateway before moving to Kgateway. I will say it is fundamentally a different approach. At the time i used it, Envoy Gateway would generate the HTTPRoutes from the controller as KGateway does not abstract that away from you and just utilizes the inference extension from the cncf community. Adleast, this is what i have observed from my brief experience with kgateway as of today lol.

From my experience so far, is your looking for a more stable and scalable solution on k8s, I would look here momentarily.

[linsun]

@stevehipwell Envoy Gateway does not have specific versioning yet so i cannot speak 100 percent certainly. However, recently i have used Envoy Gateway before moving to Kgateway. I will say it is fundamentally a different approach. At the time i used it, Envoy Gateway would generate the HTTPRoutes from the controller as KGateway does not abstract that away from you and just utilizes the inference extension from the cncf community. Adleast, this is what i have observed from my brief experience with kgateway as of today lol.

From my experience so far, is your looking for a more stable and scalable solution on k8s, I would look here momentarily.

Hi @teaglebuilt thank you so much for weighing in and sharing your perspective. Would you be open sharing how you use kgateway in one of our upcoming community meetings? Is it mostly for the inference extension or something else? Thank you for your consideration, we'd love to hearing more from you!

+100 on your point about "more stable and scalable solution".

[mloskot]

First, thank you @stevehipwell for this thread and @linsun for chiming in.

@linsun

Gloo/kgateway supports rich API gateway features such as (...)

Well, it would be helpful if features are not listed but compared, so a kgateway newbie can figure what is missing.

In June, I've learned about kgateway. I'm also planning to migrate my clusters away from Kong and looking for Kong replacement, I asked around on the Slack and received good suggestions from @yuval-k on general pattern for migration. I'm still researching the alternatives in details, especially why kgateway and not Envoy Gateway or other implementations based on Envoy proxy.

I've just found the #12378 issue with request for API key authentication and it surprised me. Especially after reading above that kgateway is:

most mature and widely deployed Envoy-based gateway (...) years ahead on adoption

Isn't the API key authentication one of fundamental features of an API gateway?
Perhaps my API gateway perspective and API key auth requirement is unique and I should be looking elsewhere, but it's quite puzzling why it's not supported by such a mature project.

Please, don't take that as a critique, but the way I think here is this: if such a basic feature is missing, what other gaps am I going to find along the way during migration; am I going to discover any blockers for the planned migration. I'm not having a decade of experience with gateways for Kubernetes, so there is still more questions than answers and even small hurdles may impose a significant doubt.

[renanramonh]

I have the same issue. Some basic features for auth are missing but I think it comes from the time kgateway was still gloo gateway and you would need to use the enterprise features to get that, it seems to be still intentionally like that. It looks like the project is more focused on AI hype at the moment. If you compare the features actually Envoy Gateway is more complete and community driven.

[lgadban]

Apologies for the delay in response here and thank you for sharing your feedback!

It is true that the current state of kgateway has been largely shaped by the Gloo 1.x codebase when we were preparing for the CNCF donation.

However, the fact that some of these features are not currently in kgateway is simply the result of developer allocation and our roadmap thus far.

With the 2.0 release in April, we wanted to get the core architecture in place along with a basic set of features that will enable folks to begin exploring the project.

With the recent 2.1 release last month, we were mainly focused on UX improvements, ensuring that our architecture is solid and scales appropriately, adding lower-level features that are necessary for actually running in production etc. We also invested some effort into agentgateway, primarily due to how quickly that community is growing and its need for a K8s control plane.

Now that we are confident we have a solid foundation in place, we can begin shifting focus to higher-level features, including several of the features in question in this thread. Notably, JWT and API key authentication are high on the list and should be available in the upcoming 2.2 release.

For reference, these items are being tracked in the following issues:
API Key Auth
Envoy JWT Auth
Agentgateway JWT Auth

As for other gaps, we have been doing some analysis to see what else is missing, but would happily accept feedback or contributions to address any other concerns. This kind of feedback is invaluable as it helps us understand where to appropriately focus our efforts!

[lgadban]

Also @renanramonh if you ran into other gaps in the project, would you mind sharing them either here or by opening issues? (or even starting a convo on our Slack?).

I want to make sure we are appropriately tracking items that the community is looking for.