AI-1849 Combat Growing Data & Backup Issues

[dbarkowsky]

Summary

AI-1849

Changes

• A document cleaning service that removes old documents and results based on a date threshold. We need to decide what the requirement for keeping this information actually is before merge. If users have injected this data into their own systems, what is the purpose of us keeping it?
• Changes to the backup and retention policy. I'm suggesting we keep 2 full backups, taken on a weekly basis, with incremental backups every 4 hours. Also open to debate. The goal is to reduce our backup footprint, which may only grow larger as more users are onboarded.
• Backups are now compressed with zstd. See this article for some good insight on the difference between this and gzip, the default: https://jothiprasath.com/blog/gzip-vs-zstd/

Testing

• We can test the document deletion locally, but we won't see the affects of the backup changes until they are in an OpenShift environment.

Checklist

By submitting this pull request, I acknowledge that I have attempted to meet the following:

• a self-review of my code
• commented code particularly in hard-to-understand areas
• corresponding changes to the documentation where required
• changes tested to the best of my ability
• no new errors or non-functional code