IOCs update infrastructure

[lsd-cat]

Currently, we use the same logic as for indicators updates, relying on the mvt-indicators repository. We parse the YAML file, which adds as a dependency, and then check every URL listed in it. On updates, we have to check each URL (expected to be a repository) to see if there are new commits, and update accordingly.

Furthermore, the current IOCs include ups.so, which causes false positives on all devices due to libcups.so. While this might be an issue with how we perform the matching (not sure if MVT has the same false positive), we might want to ensure before releasing IOCs that we don’t distribute ones that are difficult for end users to interpret.

In addition to the willingness to allow users to add custom IOCs endpoints (as in #15), we should design a better update mechanism.

I'll start by listing a few ideas and requirements.

Update server (the system that serves the updates)

• Should be hostable as static files (e.g. on Github Pages)
• Should serve aggregated files (a single HTTP request for update checks and downloads)
• Should provide signed, reproducible updates (e.g. signed and logged to Sigsum)
• Should support incremental updates (from one or two previous versions) to save bandwidth
• Updates and diffs should be compressed (or rely on transport-level compression if sufficient)
• Should support multiple update endpoints, so third parties can easily set up their own

Update software/pipeline (the process that builds and publishes updates)

• Should be able to run entirely from a CI environment
• Should allow filtering out false positives or other unwanted indicators during the build process
• Should make it easy for third parties to publish their own update endpoints and have the app support them
• We could optionally use an oblivious HTTP proxy (like Cloudflare or Fastly) to:
  • avoid easy network-based detection of activity
  • prevent us and other repositories from collecting identifiable user data