Improve API session expiration and renewal

[GibranAlfa]

Here's a suggestion of an alternative way to handle the API session expiration and renewal:

Suppose that the session is kept internally in the API indefinitely, and not destroyed when it has expired. A session is created the first time, with an expiry date, but when this expiry date arrives the session object is kept, instead of being destroyed as it is done today. The "existing session dependency" will return a failure due to the expiry date having passed, but the data is still there. The endpoint will return a 401 Unauthorized, and the GUI will call the session creation endpoint. When this endpoint is called, the post_session function runs and can simply update or refresh the session object, setting new create and expiry dates, and setting a new cookie. For the GUI it is treated like a new session, since there's a new cookie value and expiration date, but internally in the API it is just a matter of updating some fields in the session object (creation/expiration/cookie).

Such a solution would preserve data between sessions, since the actual session object persists internally and is not destroyed - the problem of transferring data from the expired session to the new session would be irrelevant. Whenever the client calls the create session endpoint, only the create/expiry/cookie values are renewed, and the context (data) for the session is preserved. It would be good to consider this solution, as I think it would solve the existing problem in a better way. This solution would be instead of changing the session duration from 20 minutes to 1 year. Compared to OAuth, the API token would be similar to a refresh token (for getting a new access token), with the cookie and short lived session similar to what the access token gives.

Originally posted by @joargr in #325 (comment)

To address that suggestion, we need to:

• Look into changing session expiry so expired sessions keep their internal context instead of being deleted right away.
• Expired sessions should still stop working for normal API calls, so protected endpoints still return 401 until the session is renewed.
• Make POST /session renew an expired session by reusing the stored context, instead of creating a totally new session and then moving data over.
• When a session is renewed, it should get a new session id and cookie, but keep the same underlying data/context from before expiry.
• Separate these two ideas more clearly:
  • the session data still exists in the backend
  • the session is active and allowed to handle requests
• Decide what should stay around when a session expires, like user/project context, and what should not.
• Decide what should happen to temporary/live things when a session expires or is renewed:
  • project locks
  • RMS sessions/workers
  • temporary access tokens
• Update tests and docs to match this flow, since it changes the session lifecycle quite a bit.
• Think about cleanup too, so expired stored sessions do not just pile up in memory forever.