Description
I am initializing the app several times in my tests, and after upgrading to 0.23.0 I noticed that the tests have started to fail, with procrastinate.exceptions.TaskNotFound:
ERROR [procrastinate.worker] Task was not found:
Task cannot be imported.
Traceback (most recent call last):
File "/Users/krisb/Library/Caches/pypoetry/virtualenvs/file-processing-trigger--C-Yay8K-py3.9/lib/python3.9/site-packages/procrastinate/worker.py", line 206, in find_task
return self.app.tasks[task_name]
KeyError: 'mediabank_ingest:trigger_ingest'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/krisb/Library/Caches/pypoetry/virtualenvs/file-processing-trigger--C-Yay8K-py3.9/lib/python3.9/site-packages/procrastinate/worker.py", line 169, in process_job
await self.run_job(job=job, worker_id=worker_id)
File "/Users/krisb/Library/Caches/pypoetry/virtualenvs/file-processing-trigger--C-Yay8K-py3.9/lib/python3.9/site-packages/procrastinate/worker.py", line 213, in run_job
task = self.find_task(task_name=task_name)
File "/Users/krisb/Library/Caches/pypoetry/virtualenvs/file-processing-trigger--C-Yay8K-py3.9/lib/python3.9/site-packages/procrastinate/worker.py", line 208, in find_task
raise exceptions.TaskNotFound
procrastinate.exceptions.TaskNotFound:
Task cannot be imported.
I noticed when debugging the tests that my tasks had duplicated their prefixes:
I have a function scoped fixture that returns the app, which is created this way
def get_worker_app(connector):
app = procrastinate.App(connector=connector)
app.open()
app.add_tasks_from(debug_to_disk.bp, namespace=module_to_namespace(debug_to_disk.__name__))
app.add_tasks_from(mediabank_ingest.bp, namespace=module_to_namespace(mediabank_ingest.__name__))
return app
@pytest.fixture
def app():
app = get_worker_app(connector=InMemoryConnector())
yield app
# Reset the "in-memory pseudo-database" between tests:
app.connector.reset()So this gets called multiple times. It seems like the blueprint persists the name of the task, including the prefix.
Here's the blueprint:
bp = Blueprint()
@bp.task(name="write_to_file")
def write_to_file(path: str, content: str):
pathlib.Path(path).write_text(content)
logger.info(f"Wrote content to: {path}")I've been able to work around this by having a reset_blueprint fixture, that I'm using now:
@pytest.fixture
def reset_blueprints():
# rerun imports, to get a reset blueprints, else they will persist data from previous runs
importlib.reload(file_processing_trigger.tasks.debug_to_disk)
importlib.reload(file_processing_trigger.tasks.mediabank_ingest)
@pytest.fixture
def app(reset_blueprints):
...I'm not sure if this is a bug, or something else but importlib.reload seems like a workaround.
Sorry about no proper reproducer, let me if that's needed and I can try to set that up.