Document local relay-based setup; fix single-core default and shared-password auth

[nekrut]

Setup on this machine (Ubuntu 24.04)

I run a Pulsar endpoint that picks up jobs from test.galaxyproject.org through the pulsar-relay at relay.usegalaxy.org. The steps a new user has to follow:

1. Clone Pulsar, create a venv, pip install -e . (or pip install pulsar-app)
2. Install Docker engine, add the user to the docker group, log out and back in
3. Write app.yml with the relay URL, manager name, and a shared password issued by a Galaxy admin
4. Run pulsar-main -c .

My app.yml (password redacted):

conda_auto_init: false
conda_auto_install: false
message_queue_url: "https://relay.usegalaxy.org"
message_queue_username: anton
message_queue_password: <REDACTED>
managers:
  anton:
    staging_directory: /media/anton/data/staging
    type: queued_python

Once running, jobs from test.galaxyproject.org arrive over the relay, stage inputs via HTTPS, and run inside Docker. This works — log excerpt for one job:

INFO  Detected relay connection string, binding to pulsar-relay at https://relay.usegalaxy.org
INFO  Starting relay consumer for topics: ['job_setup_anton', 'job_status_request_anton', 'job_kill_anton']
INFO  Successfully authenticated with pulsar-relay, token expires in 3600 seconds
INFO  Processing setup message for job_id 1525331
DEBUG Staging input '...' to /media/anton/data/staging/1525331/inputs/...

Problems

1. Single core

type: queued_python runs one job at a time. The relay docs and the app.yml sample don't show how to raise concurrency for a relay-bound manager. A laptop or workstation joining as a Pulsar endpoint should pick up all cores by default, or the docs should show the right knob next to the relay example.

2. Auth is a shared password in plaintext

message_queue_password lives in plaintext in app.yml. Onboarding requires a Galaxy admin to mint a username and password and hand it over out of band. No rotation, no scoping, no self-service revoke.

3. No user-friendly auth flow

Claude Code (and gh auth login, and most modern CLIs) authenticate this way: the CLI opens a browser, the user clicks approve on the server's UI, and the CLI stores a refreshable token. Pulsar should do the same:

• pulsar-main login --server https://test.galaxyproject.org opens the browser
• User approves the endpoint under their Galaxy account
• Pulsar writes a token to a separate config file, not app.yml
• Token refreshes on its own; admin or user can revoke from the Galaxy UI

This drops the manual ticket step and gets the secret out of the config file.

Ask

• Document the relay-based local setup end to end (Docker prereqs, app.yml minimum, multi-core)
• Default to multi-core managers, or surface the setting in the relay example
• Browser-based endpoint enrollment, with revocable tokens

[martincarrere]

@nekrut This as an additional flow ? Not everyone uses the relay, and the AMQP/RabbitMQ path should stay as-is for operators running their own infrastructure. The goal of this issue is to offer a better option for this alongside it, not to replace what's already working. Their is other capability to manage the "queue" password such as certificate per runner, zero touch enrollment, etc. .

I dont have the full context of this issue I just wanted to flag my concern when I read "This drops the manual ticket step and gets the secret out of the config file."

[natefoo]

This won't replace AMQP operation, it's leveraging the Pulsar Relay (which is itself in early development) to allow local execution of some tasks from a public server. This issue is just identifying some of the blocking points for that as of now.