Credits are lost if adding job to database fails

[jtherrmann]

As of #2045, dynamo.jobs.puts_jobs decrements the user's credits in the Users table before adding their jobs to the Jobs table. This was necessary to address a race condition described in https://asfdaac.atlassian.net/browse/TOOL-2558 (link accessible only to members of ASF) which allowed a user to submit jobs for free.

This means that if adding any of the jobs to the database fails, the user's credits have already been decremented and they have effectively lost the credits for the failed job submissions. In this case, I believe they would only see an Internal Server Error with no explanation.

Two options for improvement:

1. At a minimum, we could at least return a descriptive error message if any of the job submissions fail, apologizing for the lost credits and suggesting that they contact user support.

2. If we wanted to make the system more robust, we could investigate whether DynamoDB Transactions would allow us to decrement the credits and submit the jobs as a single transaction, so that if one of the database operations fails, then they all fail. We would have to decide whether to submit the entire batch of jobs as a single transaction, or use one transaction per job (and decrement the credits for each job individually as part of the job's transaction).

We're not aware of any cases of this issue occurring in the wild, so this is probably not a high-priority issue. It probably becomes more important for a pay-for-processing deployment in which users are actually paying for credits, but even then I'd suggest we start with option (1) and only explore (2) if we start seeing this occur in the wild.