DM-38718: Arq Queue for Slack

[Fireye04]

No description provided.

[jonathansick]

• Be sure to add configurations for arq to the config module
• Be sure to initialize the redis dependency too for the FastAPI server in src/semaphore/main.py.
• You may want to also run Redis during tests. This can be done with tox-docker. check out https://github.com/lsst-sqre/times-square/blob/2ffb96e6de218d1bb61aaf84f2b6d97c4f19142e/tox.ini#L25-L47

Another note that this first commit kind of conflates two separate things. One is setting up arq, which is a redis-based queue. Another is setting up a redis client. Both use Redis, but they're actually separate things. We'll use the redis client to persist state.

[jonathansick]

To solve the CI fails, I think you'll need to add the tox-docker plugin. You can do that by modified the tox step in .github/workflows/ci.yaml to have a tox step that includes a tox-plugins line like this:

      - name: Run tox
        uses: lsst-sqre/run-tox@v1
        with:
          python-version: ${{ matrix.python }}
          tox-envs: 'lint,typing,py,coverage-report'
          tox-plugins: "tox-docker"

[Fireye04]

Made the suggested changes to ci.yaml, however it continues to fail, citing the following error, then timing out.

WARNING arq.connections:connections.py:266 redis connection error localhost:6379 ConnectionError Error connecting to localhost:6379. Multiple exceptions: [Errno 111] Connect call failed ('::1', 6379, 0, 0), [Errno 111] Connect call failed ('127.0.0.1', 6379).

[jonathansick]

Good start on this. So next you'll want to integrate this building block into the feature spec of sending Slack messages when there are new active messages for this environment.

One approach that you might take is to set up a cron task with Arq. The algorithm for that task is something like:

1. Compute the active broadcasts (like the web endpoint does)
2. Get the pre-exisiting state of active broadcasts.
3. If there are new active broadcasts compared to the state, send those to Slack.
4. Store the current active messages for the next cron iteration.

Probably the easiest way to store this state is in Redis with https://safir.lsst.io/user-guide/pydantic-redis.html

For that, you'll need to design a Pydantic class that describes the active broadcast messages. You can base that off the Pydantic models used by the web endpoint, like

semaphore/src/semaphore/handlers/v1/models.py

Line 44 in 75b8122

class BroadcastMessageModel(BaseModel):

[Fireye04]

Alright, so getting back into this, I found the documentation for mock slack webhooks, and decided to go about trying to implement it, with limited success. I believe I need to use the post_webhook function to log calls to the webhook, however I'm not certain how I'm supposed to get a request to put into it.