Reusable route groups with per-instance overrides

[Mounesh-S]

Hi Kaoto team,

We build and operate a number of Camel + Quarkus integrations where the same small set of
routes is repeated many times over. We'd like to check whether the needs below belong in
Kaoto — in core, or behind an extension point — before we build anything.

We're happy to implement whichever parts you'd welcome, and equally happy to be told this
sits outside Kaoto's scope.

The shape of the problem

We integrate external systems into a platform. Work arrives in domains, and each domain
always has the same fixed set of operations. Illustrative shape (names are generic):

Domain "Orders"         →  create · update · cancel · return
Domain "Notifications"  →  email · sms · push · webhook
Domain "Shipments"      →  book · track · cancel

Every newly integrated system needs that whole set again. And each set then runs for
several customers and several environments. So one definition becomes many instances:

Orders ─┬─ System A ─┬─ customer 1 · prod
        │            ├─ customer 1 · staging
        │            └─ customer 2 · prod
        └─ System B ──── customer 1 · prod

The important part: instances are mostly identical.

• The payload shape and the field mapping are a property of the integrated system —
  the same for every customer using it.
• Only endpoints, credentials, destinations and a few limits differ per customer and
  environment.

Today each route is a standalone .camel.yaml, so every instance is a full copy, and a fix
to a shared mapping has to be repeated in each one.

Requirements

R1 · A group is a first-class thing

What we need. A named, reusable definition that declares which routes it expects to
contain, and what they have in common — the data shape they must all produce, and a shared
error-handling policy. Each member still owns its own transformation and its own error
handling on top of that.

Without it. Related routes are unrelated files. Nothing records that they belong
together or that they must agree on an output shape.

Generically. The same idea as a project template that declares its expected contents.

R2 · Completion state within a group

What we need. A group knows it expects four members. Two are finished, one is in
progress, one hasn't been started — and the tool shows that. "Finished" should mean the
member passed its own checks, not that someone ticked a box.

Without it. Progress lives in a spreadsheet next to the editor, and it goes stale.

Generically. A checklist derived from the definition rather than maintained by hand.

R3 · Instantiate a whole group with parameters

What we need. Stamp out the entire group again with a different parameter set, in one
action, rather than recreating each route by hand.

Without it. Repeating a group is manual copying, which is where drift starts.

Generically. Camel already has the runtime half of this — RouteTemplates and Kamelets
are parameterised routes. What's missing is a UI story for instantiating a group of them
together, as a unit.

R4 · Instances on more than one axis, with a shared base and per-instance overrides

This is the requirement we'd most like your view on.

What we need. An instance is identified by more than one key — for us, integrated system
× customer × environment. Configuration should be authored once on the group and
inherited by every instance, with an instance overriding only the few values that genuinely
differ. The UI should make clear which values are inherited and which are overridden.

Without it. N instances means N copies of the same mapping, and one bug fix has to be
applied N times. This is the single biggest cost we're trying to avoid.

Generically. The same need Kustomize overlays or Spring profiles serve: one base
definition, many thin variants.

What we found in the codebase

We looked before proposing. Kaoto already has a rendering-anchor extension mechanism —
RenderingAnchorContext.registerComponent({ anchor, activationFn, component }) — and
Kaoto's own DataMapper, ComponentMode and GroupAutoStartupSwitch register through it.

Two observations:

• The three anchors (CanvasFormHeader, CanvasGroupTitlebar, CanvasNodeTopLeft) all sit
  inside the canvas node form, so there's no anchor above the level of a single route.
• RenderingAnchorContext isn't exported from public-api, and RegisterComponents holds
  its registrations in a local useRef, so a third party can't register without forking.

So even the parts that could reasonably live outside core have no supported seam today.

We'd rather build this as a plugin than in core

None of R1–R4 has to be in Kaoto's core, if the extension surface allows it. Our preference,
in order:

1. As an external plugin, if the extension points exist
2. In core, if you consider any of it broadly useful to Camel users
3. In our own application, if it fits neither — no hard feelings

For option 1, what we'd need from Kaoto is modest and not specific to us:

Needed	Why
Export RenderingAnchorContext, Anchors, IRegisteredComponent from public-api	So a plugin can register at all
Let RegisterComponents accept externally supplied registrations	Currently a hardcoded local useRef
Anchors above the node-form level — a toolbar, side panel, or view slot	R1–R4 all sit above the level of one route
A documented seam for a plugin to persist its own metadata	Group definitions and instance parameters need somewhere to live

We'd be glad to contribute that groundwork regardless of whether our own features ever land.

We know this overlaps Karavan

Karavan already covers project-level organisation and lifecycle, and "a group of
integrations" arguably belongs there rather than in the designer. If the answer is "take
this to Karavan," that's genuinely useful to hear.

Where we think it may still be Kaoto's: R3 is UI for an existing Camel concept.
RouteTemplates and Kamelets are already parameterised routes; a designer that instantiates
them with a parameter set seems like designer work.

What we're asking

1. Are R1, R2 and R4 in scope for Kaoto, better suited to Karavan, or out of scope for both?
2. Is R3 — instantiating a group of RouteTemplates/Kamelets with a parameter set — something
   Kaoto would want?
3. Would you consider making the rendering-anchor mechanism a supported public API, so work
   like this can live in a plugin rather than a fork?

We can bring a design sketch or a proof of concept to whichever is most useful.

Thanks for Kaoto — it's been good to work with.

[lordrip]

Hi @Mounesh-S , thanks for starting this discussion and we're glad that Kaoto helped you 🙏.

After reading the proposal, I think this is interesting, I would like to ask a few questions to help me understand better the goals. I'll be doing so on separate threads so we can focus on each topic (R1-4) on an individual basis.

[lordrip]

I think that in order to provide better answers, it would be beneficial to know:

1. When you think about these features, are you looking at Kaoto web, VS Code Kaoto or both?
2. Initially I read Quarkus, but in one of the last items you mentioned

Kustomize overlays or Spring profiles serve

Was that an example and you're looking for parameters from the application.properties files supported by Quarkus or are you looking at those configuration mechanisms?

3. Who would you say is the final user of these features? Are we talking about Kaoto being client-face or is it more like an internal tool to handle a service that will be ultimately client-facing?
4. If you have some sketches, for sure, please feel free to send them in :)

[Mounesh-S]

Hi @lordrip,

Thanks for the questions, a couple of them made us realise we had been vague. I'll answer them in order below.

1. Both, but our plan is Kaoto web self-hosted with our own customisations. We also think anyone supporting a lot of vendors or clients on similar formats hits the same thing, which is why we raised it here rather than patching our own copy.

2. Apologies, poor comparison on our part. We're on Quarkus with Camel and aren't asking Kaoto to touch application.properties or profiles.

3. Organisations maintaining or supporting many vendors and clients on similar formats, where the same route shape is repeated dozens of times and only a few values change. For us it's an internal tool — our integration engineers use it behind the firewall, and our customers see a separate application.

4. Attached some sketches.

5. Systems, groups, tenants

2. One group per tenant — the instance list; picking a tenant opens that instance on the canvas.

3. Drag an existing group — System A already runs orders, so adding it to System B is a drag rather than a rebuild.

4. Configure endpoints — what gets asked for after the drop: base URL, credentials, destination. Not the mapping, not the members.

[lordrip]

R1 · A group is a first-class thing

If I'm understanding correctly, this looks like a maven archetype, but for standalone routes, is that the case? I'm wondering if scaffolding another tenant through a template would help in this case.

In the past, we considered adding a "New route from a template" functionality, I think this case might be similar, but instead of a single file, it would compose a folder with multiple files in it

[Mounesh-S]

Yes, for creating new integrations, a template that scaffolds a folder with multiple route files is spot-on.

In our case, a domain is always a set of routes. For example, a Bills group consists of 4 routes (Invoice, Exchange, Return, Void), and a Notifications group consists of (OTP, Notification, Delivery Webhook, Delivery Response). When we onboard a new vendor or tenant, having a template generate that entire folder with those 4 routes ready to configure would already be a huge step forward.

The only gap with a standard archetype is maintenance and drift after creation:

• For a given vendor, the core field mapping and transformation logic are identical across 20 different tenants—only tenant-specific configs (endpoints, URLs, credentials) differ.
• With pure static scaffolding (generate and forget), if that vendor adds a new field to their invoice format, we have to manually open and update 20 separate copies. And nothing warns us if one tenant was missed or quietly diverged.

[lordrip]

I think this is related to R3 and perhaps both can be handled together. I recently added a template catalog to initialize individual files, like routes and kamelets. I believe we could create a new DynamicCatalog registry to dynamically fetch them. That would be your connection point, instead of loading what it is included, you would be loading what you want to provide.

That being said, at the moment, this feature falls short because it doesn't support multiple files, only one, so we could rework the template shape to instead of being a single file, an array of multiple files, so a single template entry (your domain) could create multiple files at once. For R3 this would mean enhancing the template mechanism a bit to show you a parametrized view to fill your parameters.

I think both R1 and R3 can be directly developed in Kaoto's core, this seems to be an interesting feature.

At the moment, we're reimplementing Kaoto's web version in the background (you can see it in the kaoto-web folder), this is where this feature should be initially developed because one of the goals is to support multiple files.

Are you open to contribute this feature? If the answer is yes, let me know so we file the plan in an issue so the feature land as smooth as possible.

[lordrip]

R2 · Completion state within a group

Do you mean like a checklist out of files presence?

[Mounesh-S]

File presence is definitely the starting point, but on its own it's only half the story.

In our real case:

• Presence tells us what is missing: If the Bills group expects 4 routes (Invoice, Exchange, Return, Void), but only Invoice and Exchange exist in the folder, the checklist immediately flags that Return and Void haven't been created yet. That alone is already very helpful.
• Why presence alone isn't enough: A developer could scaffold or create the Invoice route file, but leave the target endpoint URL blank or mandatory field mappings unconfigured. If completion is based only on file presence, the checklist turns green even though the integration is broken and cannot run.

So to us, 'completed' simply means: the expected route files are present, and Kaoto has no validation errors or missing required configurations on them.

[lordrip]

In this case, I'm not exactly sure how this could fit in Kaoto as-is. It doesn't mean it would never be, it just mean that at this point in time, I don't see how to connect this functionality with the rest of Kaoto in a way that helps with the diversity of integrations from the users.

That being said, to unblock this functionality, I think we could treat it as a plugin for now, so we could extend the RenderAnchor functionality to create the extension point needed, and from that point, you have the freedom to implement the validation in the way you exactly needed.

In the future, if we get a case in Kaoto where somethign similar is needed in the core, we can port the functionality generic parts and you use the customized variant.

Let me know if this would work for you

[lordrip]

R3 · Instantiate a whole group with parameters

Are we talking about creating the files or executing the integrations from the file group? If it is the latter, in VS Code Kaoto we have a "Run folder" functionality, but I would like to confirm the direction first.

[Mounesh-S]

We are talking about creating and configuring the files at design time, not executing the integrations at runtime.

1. Why design-time only:
   Our integrations run in our own backend (Quarkus + Camel) hosted on our servers. Kaoto is used as an authoring and visual configuration canvas for our integration engineers, not as the execution engine. ("Run Folder" in VS Code is a neat feature for local testing, but our execution and deployment happen outside Kaoto.)

2. What "instantiating with parameters" means in our case:
   When we onboard a new client for a Bills group, we need all 4 routes (Invoice, Exchange, Return, Void).

   Today, that requires creating 4 separate route files and configuring endpoints in each one by hand.

[lordrip]

I answered this as part of R1, but I believe we could extend the template catalog functionality to detect template parameters and show a parametrized view to fill the information and bootstrap the files. More details in R1

[lordrip]

R4 · Instances on more than one axis, with a shared base and per-instance overrides

This is interesting, do you mean launching the routes with different profiles and those profiles bringing different parameters? From where are you planning to launch the integrations? Through Kaoto?

[Mounesh-S]

No, we are not launching through Kaoto at all, and this is not about runtime profiles.

1. Where integrations are launched:
   We launch and run all integrations in our own backend runtime (Quarkus + Camel) on our servers. Kaoto is strictly a design-time authoring tool where our integration engineers visually build and configure routes.

2. Runtime profiles vs. Design-time inheritance:

   • Runtime values (like credentials, database URLs, or staging vs. prod endpoints) are already handled at runtime by Camel property placeholders (application.properties).
   • What R4 actually means is sharing the visually authored mapping logic across clients:
     • In our Bills group (Invoice, Exchange, Return, Void), the base payload structure and field mappings belong to the vendor (e.g., an ERP or POS system).
     • We onboard 20 clients on that vendor. 19 clients use the exact same mapping, but Client A needs one extra custom field mapped in Invoice.

[lordrip]

This requires more thought and consideration, if you don't mind, can we leave this one to a later stage?