You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**tl;dr**: We've built a proof-of-concept (POC) routing capability that allows Kafka clients to produce and consume records to topics in multiple clusters. In other words, clients don't need to know where their topics live.
13
+
This is roughly the Kafka equivalent of what is often called _Data Virtualization_ for databases.
You can run this for yourself; check out the [routing POC branch](https://github.com/tombentley/kproxy/tree/routing/) on GitHub.
19
+
But feel free to run this for yourself by checking out the [routing POC branch](https://github.com/tombentley/kproxy/tree/routing/) on GitHub.
20
+
In instructions are in [`demo/topic-router/README.md`](https://github.com/tombentley/kproxy/tree/routing/demo/topic-router#readme).
19
21
20
22
## What's the backstory?
21
23
22
-
Nearly a year ago, I had an idea which I wrote up in what's become known as [The Routing Proposal](https://github.com/tombentley/kroxylicious-design/blob/8378acd7ef4c194cce5b60419bf92c8e8c7d8ea3/proposals/004-routing-api.md).
24
+
Nearly a year ago, I had an idea which I wrote up in what's become known as [The Routing Proposal](https://github.com/kroxylicious/design/pull/70/changes) PR.
23
25
(You know something might be important if it gets initial caps).
24
26
While a few people commented and it felt like a good concept, it was also a long way from the reality of the proxy codebase at that time.
25
27
And anyway, it's not as if this is the only good idea that we had, or the only thing we had to work on.
@@ -30,7 +32,7 @@ However, for this particular idea it was hard to forget about it for long. Engin
30
32
Some of our amazing contributors started the process with some initial refactorings.
31
33
Now, nearly a year later, we've found the time (and, I admit the tokens) to throw at building a proof of concept.
32
34
33
-
Right now the proof of concept (POC) branch is an interleaved mishmash of commits in each of these three areas. However, the take-away is that we believe we have an API which is capable of supporting a non-trivial router.
35
+
Right now the proof of concept branch is an interleaved mishmash of commits in each of these three areas. However, the take-away is that we believe we have an API which is capable of supporting a non-trivial router.
34
36
35
37
The router we've implemented allows clients to talk to multiple clusters.
36
38
@@ -59,11 +61,9 @@ This is at a POC stage. This blog post is announcing that we're in the middle ph
59
61
60
62
The big picture of what we've done breaks down like this:
61
63
62
-
1. Preparatory internal refactoring to break apart a single state machine into two: One state machine for the client-proxy interaction, and a separate one for the proxy-broker interaction.
64
+
* The work to add the Router API, and implement the runtime support for it.
63
65
64
-
2. The work to add the Router API, and implement the runtime support for it.
65
-
66
-
3. The work to implement the Topic Router plugin using that API.
66
+
* The work to implement the Topic Router plugin using that API.
67
67
68
68
This is where the Kroxylicious community comes in - turning those three big pieces into something that's actually usable and supportable.
0 commit comments