Workflow glue

[majamassarini]

This is a "research" on how we can glue multiple agents with simple scripts + redis.
I tried bash, python and ansible. Ansible to me feels like the best fit.

Drawbacks from my point of view are:

• when running goose inside ansible you can't see the output until the task is finished - since goose takes a lot of time to complete it can be annoying.
• goose exits always with a success code even though the recipe wasn't successfull
• some things have to be done inside the recipe (like removing the message from the redis queue after processing it) and other from inside ansible, like reacting to the redis message) this can lead to inconsistency.

Pros:

• I think it should be easy to load balance agents inside an Open Shift cluster using ansible targets like these.

[gemini-code-assist]

Code Review

The pull request introduces workflow automation capabilities using Goose, Redis, and Ansible. It includes changes to the Makefile, Docker Compose configuration, environment variables, and Goose recipes. The changes add support for managing and executing workflows related to package rebase and issue analysis. The review identified areas for improvement in error handling, documentation, and clarity of instructions.

[vrothberg]

Looking at it, maybe we can just keep the PR open?

Otherwise, we'll run into the game of running redis server etc and more. Just for prototyping with Goose, it's probably easier with the current main branch.

[vrothberg]

WDYT?

[majamassarini]

Looking at it, maybe we can just keep the PR open?

Otherwise, we'll run into the game of running redis server etc and more. Just for prototyping with Goose, it's probably easier with the current main branch.

Right, the compose now has all the redis stuff. Sure, for me it is ok.

[owtaylor]

I have to say that this feels like it is making the tech stack very big very quick. Do we need ansible here or could we keep it simpler?

I'm also not sure that we want agents to be directly manipulating the work-queue through redis-mcp. We wouldn't have a human do that. Seems like it's going to invite ending up with corrupted JSON in there.

[vrothberg]

I have to say that this feels like it is making the tech stack very big very quick. Do we need ansible here or could we keep it simpler?

I think we share the same thoughts. This PR was trying to figure out how we could glue multiple (goose) agents together. Part of the exercise is assessing the tools. Google's ADK may be better suited but let's see.

I'm also not sure that we want agents to be directly manipulating the work-queue through redis-mcp. We wouldn't have a human do that. Seems like it's going to invite ending up with corrupted JSON in there.

Agreed. We want/need "us" to do the DB manipulation, not an agent.

[majamassarini]

I have to say that this feels like it is making the tech stack very big very quick. Do we need ansible here or could we keep it simpler?

When I was writing the bash script for checking/reading the database and executing the next goose recipe, it didn't feel readable or reliable to me. I personally prefer Ansible over bash scripts.

As Valentin wrote, this was just a quick research prototype. As listed in the PR, I see several drawbacks and one advantage: the Ansible playbooks can easily be scaled in different ways in OpenShift if needed.

If we ever go back to working with Ansible, I think we should investigate Ansible's async capabilities further.