Safeguard for many status requests #97
Description
In i-VRESSE/haddock3-webapp#110 the queued/running page refreshes. This causes a lot more calls to the /api/job/{id} endpoint. And even more when there a bunch of people waiting for the job to complete.
Getting the state of a job from a scheduler like slurm or dirac can be expensive.
We should benchmark how stress-full it is for those schedulers to have lets say a 100(0) users each with 1 job queued or running being checked every 10 seconds.
This could be resolved in different ways:
rate limit
Moves problem to consuming webapp
cache
It would be nice if the state of a job is cached for a configurable duration like 1 minute. So if job state was fetched 5 seconds ago and you fetch again you would get the 5 second old state. If you wait a minute and fetch again you will get a fresh state from the scheduler. This would need saving a last checked timestamp in the database.