using custom camel components

[gjongerh]

Hi community,
I do use camel a long time now and have developed our own custom components.
I did a test by adding components by hand in the kaoto catalog. That works as we want. Both methods did work: 1- serving the enhanced kaoto catalog by a webserver and copying the catalog into the extension folder. That take a lot time.

Better it is better to use the java catalog-generator project. But after staring for hours to the java code we did not find a good point to extend the code in using our custom components.json.

Is it possible to extend the camel catalog with own custom components before processing start by kaoto? Our java project does generate the camel component json for every component. And we have a maven plugin for collecting all the separated json files into one.

What is a good place to start extending the camel catalog?

[lhein]

Hello, let me ask a quick question. How does the data look like you want to inject into the catalog generation. Is it as described a json file containing your custom components data?

[gjongerh]

We use the standard camel custom component.
Does the example below answer your question.
Thank you for helping

{
  "component": {
    "kind": "component",
    "name": "vp-queue",
    "title": "VirtPoint Queue",
    "description": "Represents a VpQueue endpoint.",
    "deprecated": false,
    "firstVersion": "2.0.0",
    "label": "queue",
    "javaType": "com.virtorg.bus.component.queue.QueueComponent",
    "supportLevel": "Stable",
    "groupId": "com.virtorg.bus",
    "artifactId": "virtpoint-queue",
    "version": "3.0.6-SNAPSHOT",
    "scheme": "vp-queue",
    "extendsScheme": "",
    "syntax": "vp-queue:name",
    "async": false,
    "api": false,
    "consumerOnly": false,
    "producerOnly": false,
    "lenientProperties": false
  },
  "componentProperties": {
    "bridgeErrorHandler": { "kind": "property", "displayName": "Bridge Error Handler", "group": "consumer", "label": "consumer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored." },
    "lazyStartProducer": { "kind": "property", "displayName": "Lazy Start Producer", "group": "producer", "label": "producer", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing." },
    "autowiredEnabled": { "kind": "property", "displayName": "Autowired Enabled", "group": "advanced", "label": "advanced", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": true, "description": "Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc." }
  },
  "properties": {
    "name": { "kind": "path", "displayName": "Name", "group": "common", "label": "", "required": true, "type": "string", "javaType": "java.lang.String", "deprecated": false, "deprecationNote": "", "autowired": false, "secret": false, "defaultValue": "DUMMY", "description": "The Queue to read\/from messages from\/to" },
    "actionlog": { "kind": "parameter", "displayName": "Actionlog", "group": "common", "label": "", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": true, "description": "Create an actionlog." },
    "allTenant": { "kind": "parameter", "displayName": "All Tenant", "group": "common", "label": "", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "When false we must obey tenant parameter" },
    "body": { "kind": "parameter", "displayName": "Body", "group": "common", "label": "", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "Proces body as well." },
    "delay": { "kind": "parameter", "displayName": "Delay", "group": "common", "label": "", "required": false, "type": "integer", "javaType": "long", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 1000, "description": "If greater than 0, generate periodic events every period milliseconds." },
    "ended": { "kind": "parameter", "displayName": "Ended", "group": "common", "label": "", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "Message is finished." },
    "jpaFactory": { "kind": "parameter", "displayName": "Jpa Factory", "group": "common", "label": "", "required": false, "type": "object", "javaType": "com.virtorg.esb.jpa.service.IJpaServiceFactory", "deprecated": false, "autowired": false, "secret": false, "description": "JPA Service Factory" },
    "limit": { "kind": "parameter", "displayName": "Limit", "group": "common", "label": "", "required": false, "type": "integer", "javaType": "int", "deprecated": false, "autowired": false, "secret": false, "defaultValue": 10, "description": "Limit total messages processed per poll..." },
    "tenant": { "kind": "parameter", "displayName": "Tenant", "group": "common", "label": "", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "description": "Filter; only message for this tenant." },
    "update": { "kind": "parameter", "displayName": "Update", "group": "common", "label": "", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "Insert or Update message in the producer." },
    "bridgeErrorHandler": { "kind": "parameter", "displayName": "Bridge Error Handler", "group": "consumer (advanced)", "label": "consumer,advanced", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored." },
    "exceptionHandler": { "kind": "parameter", "displayName": "Exception Handler", "group": "consumer (advanced)", "label": "consumer,advanced", "required": false, "type": "object", "javaType": "org.apache.camel.spi.ExceptionHandler", "optionalPrefix": "consumer.", "deprecated": false, "autowired": false, "secret": false, "description": "To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored." },
    "exchangePattern": { "kind": "parameter", "displayName": "Exchange Pattern", "group": "consumer (advanced)", "label": "consumer,advanced", "required": false, "type": "object", "javaType": "org.apache.camel.ExchangePattern", "enum": [ "InOnly", "InOut", "InOptionalOut" ], "deprecated": false, "autowired": false, "secret": false, "description": "Sets the exchange pattern when the consumer creates an exchange." },
    "lazyStartProducer": { "kind": "parameter", "displayName": "Lazy Start Producer", "group": "producer (advanced)", "label": "producer,advanced", "required": false, "type": "boolean", "javaType": "boolean", "deprecated": false, "autowired": false, "secret": false, "defaultValue": false, "description": "Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing." }
  }
}

[lordrip]

Hi @gjongerh, thanks for your interest and for being willing to contribute this feature.

We have a medium-to-long-term idea about where we would like to move forward with this, and I think we can adapt it a bit to make it compatible with your use case.

Here's my proposal:

1. Generate the custom components JSON files (you're already doing this with the camel maven plugin)
2. Put them in a folder <custom-folder>
3. Run the catalog generator with a new flag --custom-components-folder=<custom-folder> flag
4. The Camel catalog generator will read those files and append them to all catalogs that are scheduled to be generated

The final command would look like the following:

$ java -jar ./target/catalog-generator-0.0.1-SNAPSHOT.jar -o ./dist/camel-catalog -k 4.12.0 -m 4.12.0 -cf `<custom-folder>` -n "My Catalog"

Now, from the catalog-generator side, we can implement the --custom-components-folder (--cf for short) to read the JSON files from there.

What do you think about this approach?

[lordrip]

If that works for you, here's how we could do it:

1. Extend the ComponentGenerator class.
2. Override the generate() method to read the files from the config flag if available
3. In the extended class, create a new method that receives an existing ObjectNode (the standard camel catalog) and appends what you just generated.
4. Register the new generator in the CamelCatalogProcessor class

[lordrip]

if you follow the CamelCatalogProcessor class, you'll notice a connection point with the CatalogGeneratorBuilder class that you could use to pass the folder information to the new class.

We could also think about being able to pass multiple folders at the same time by passing the --custom-components-folder=<custom-folder> many times, similar like we receive the -m class to generate multiple Camel versions.

[gjongerh]

We will try to implement this and come back to you.

[lordrip]

Awesome @gjongerh, feel free to reach out if you want another set of eyes 👍