Requirements

• This plugin is tested on Zabbix 5.4; given its simplicity, we don't expect major upgrade challenges.

• Your tenant must have the AI Process Engine enabled.

Installation + Configuration

This plugin requires initial configuration across the following systems:

• We will setup an Application Account (aka a technical user) for Zabbix in the Serviceware User Management
• We will prepare the workflow in the Serviceware AI Process Engine with an HTTP Start Event. This is where Zabbix will start Workflows.
• We will import the provided Zabbix Media Type into the Zabbix Instance.
• We will setup the minimum necessary configuration of the media type in Zabbix to function.
• Optionally, there will be a suggestion on setting up actions to report trigger events in Zabbix as tickets to the AI Process Engine.
• Optionally, we can also extend the media type script to set additional data fields in the AI Process Engine based on data from Zabbix.

This sounds like a lot, but this should not take much longer than an hour or two to go through. It's pretty normal integration work - a lot of small steps in a fair number of systems.

Setup an Application Account for Zabbix

1. Login to the Serviceware User Management.

2. Select the "Application Accounts" Tab.

3. Create a new application account

This opens up the dialog to configure the new application account:

• The "Name" is free-form, but will be used later on in the login user name. I recommend something including "Zabbix" in your own naming convention, for example "internal-zabbix" and "staging-zabbix".

• The "Description" is entirely free-form and used to document the purpose of this technical user. It is not technically used anywhere else. I recommend including a reference to the system using this application account.

• The "Permitted Modules" define the Serviceware Platform Modules the user is allowed to access. If a module is not allowed here, the Platform will reject any requests from this user immediately. This must enable "AI Process Engine", and for least-priviledge management should not allow anything else.

• The technical user does not need any Groups assigned.

• The technical user needs the Roles "Agent" and "Consumer" of the "AI Process Engine" Module. They are right below the "AI Process Engine" text and slightly indented.

Note down the internal name and the password it gives you (or copy them via the right-click menu). We will need those later.

If you need to rotate the password of the application user due to a leak or other security requirements, you can select the application account in this UI as well to obtain the handy "regenerate" button at the top.

For more information about application accounts, refer to the official documentation

Workflow configuration

Second, we will have to collect and adjust the workflow Zabbix is supposed to create instances.

Open up the workflow in the designer of the AI Process Engine. Go ahead and collect the Workflow ID into your notes.

.

Afterwards, add an HTTP Start event and route it into your workflow as necessary. Note down the following IDs and configurations:

• The Message ID. We need both the workflow ID and the message ID to point Zabbix at the right HTTP Start Event.

• Make sure to create a new object instance and choose the data object type that the rest of the workflow expects.

• Give it the object label that the rest of the workflow expects. Note down this object label as well.

Checkpoint: Serviceware Platform Configured

At this point, the Serviceware Platform is configured. You should now have a few values available for the Zabbix Configuration:

• Username + Password of the Application Account
• Workflow-ID of the target workflow
• Message-ID of the start event in the target workflow
• Object Label in the AIPE
• Finally, select two text fields for the summary and the message text in the data object.

Importing the Media Type

• Navigate to Administration -> Media Types or Alerts -> Media Types
• Use the Import button on the top right to import a new media type.

We need to configure and change a few things in the Media Type to get it running correctly. As such, I do not recommend to re-import and update an existing Media Type. The main changes + Fixes to this plugin should be changes in the script itself, which can be updated easily.

Minimum Configuration of the Media Type

• Open up the imported media type.
• If you want to connect multiple different AI Process Engine Tenants or multiple different workflows to this Zabbix Instance, update the Name of the Media Type, because this media type will be configured to target exactly one workflow in the AI Process Instance.
• And now we will have to set a few parameters:

Parameter	Value
aiProcessEngineRealmURL	This is the URL of your AI Process Engine tenant, normally https://<*TENANT*>.ai-process-engine.serviceware.cloud
applicationAccountPassword	Set this to the password of the application account, or use Zabbix Macros to supply this value from a secure location
applicationAccountUsername	Set this to the username of the application account, or use Zabbix Macros to pull this value from a secure location
dataModelSubjectField	Set this to the property of the data object in the AIPE the Trigger Subject should be placed in
dataModelMessageField	Set this to the property of the data object in the AIPE the Trigger Message should be placed in
realmName	Set this to the tenant name.
startEventId	Set this to the Message ID of the HTTP Start Event
workflowId	Set this to the ID of the workflow Zabbix is supposed to start
workflowObjectAlias	Set this to the label of the data object to create when starting the workflow

With all of this set, this media type should be able to create data objects and start workflows for them.

Setting up trigger actions to start workflows.

Note: This is how we have configured our Zabbix to create tickets in our AI Process Engine. We're no Zabbix experts, so there may be cleaner ways. It is functional and easy to select triggers to create tickets for though.

In our Zabbix, we have setup a user for the AI Process Engine and given this user the Media Type we just created. Tune the severities as preferred. The "Send To" value is ignored.

With this, you can now use trigger actions to start workflows for triggers:

Adding further data to data objects

It is possible to extend the media type to push further data from Zabbix into the AIPE. For example, in our internal production, we push the hostname and a bit more metadata to our AIPE to enable further automated incident classifications to run afterwards.

Our workflow for such extensions is as follows:

• Get the internal name of the property in the AI process engine. This is usually lower-case and kebab-case, like "hostname"

• Add two new parameters to the media type:

  • dataModel[NAME_OF_WHAT_YOU_ARE_ADDING] - set this to the internal property name from the AI Process Engine

  • [NAME_OF_WHAT_YOU_ARE_ADDIND} (for example: hostname). Set this to values you get from your host, like {HOST.HOST}.

• Open up the script and extend the function addDataFieldsToDataObject. Keeping the extensions inside these functions means that during an update, you can replace the entire rest of the script while keeping the addDataFieldsToDataObject modifications intact.

function addDataFieldsToDataObject(params, dataObject) {
    dataObject[params.dataModelHostnameField] = params.hostname;
    return dataObject;
}

Testing & Troubleshooting

Note The full function of the Zabbix - AI Process Engine integration can be impeded by various infrastructural topics we cannot know about, such as firewalling, egress protections, and so on. We cannot navigate these challenges on the way to the Serviceware Platform.

My current testing approach may be pragmatic. However, this media type has a lot of parameters to set just right, and the normal "Test" Button for a Media Type has given me false tests quite a few times by copy-pasting some secret wrong, or similar mistakes.

As such, I have a trigger action, which I enable whenever I have to test this media type, and I disable it afterward again. This trigger action filters for an operations internal test system and reports all problems on this test system to the AI Process Engine. This way, I can cause something non-critical on this system, such as shutting down a monitoring agent for a minute or placing a large file in /var/log via fallocate, generate an alert, and see what my media type does.

If this fails, the script is logging to the zabbix server log with the prefix '[ AIPE Webhook ].

Known and anticipated Errors:

• If there are timeouts when obtaining the access token, I'd advise checking if the media type on the Zabbix server is able to reach the Serviceware Platform, or if some firewall is blocking the outbound traffic. Outbound HTTP Traffic to auth.sso.serviceware.cloud and [YOUR REALM].ai-process-engine.serviceware.cloud are necessary for the functionality.

• If the access token derivation fails with a 401 - Unauthorized, the most likely errors are either an incorrect realm or incorrect credentials. The media type is logging the username and the first three characters of the password, because I have had issues in the past to shuffle the secret from our secret store correctly into the parameters.

• If the workflow start ends up with a 404 - Not Found, there may be typos in the realm name.

• If the workflow ends up with a 403 - Forbidden, the permitted modules or the roles of the application account may be incorrect, causing the AI Process Engine to reject the request based on permissions.

• If the workflow ends up with a 400 - Bad Request, the account should be setup correctly. However, the AI Process Engine might reject the workflow because either the workflow id, the message id, or any of the data object properties are incorrect. The response from the AI Process Engine should shed more light on this.