[HMA] Roadmap Discussion

[julietshen]

Copied from https://github.com/facebook/ThreatExchange/blob/main/hasher-matcher-actioner/docs/roadmap.md

Hasher Matcher Actioner Roadmap

This file gives some high level suggestions and ongoing direction for the Hasher Matcher Actioner (HMA) subproject in this repository, and larger projects if you are a developer interested in donating more time.
As of 9/2025, HMA has active interest, and some ongoing development. Some of these features might imply adding features to the underlying python-threatexchange library, also hosted in this repository.

Most Needed

These projects we think would most directly help with HMA in it's current state, either making it directly better for current adopters, or making it easier for new adopters to get started.

Hardening Functionality for Production Deployment

HMA is built on flask and python, though the original authors don’t have much experience running this technology stack at scale. We’ve noticed some problems when trying to run the software at our targeted scale of ~10M hashes indexed, such as memory leaks (#1813).

The target scale for a multi-instance HMA deployment is to be able to support at least 10M indexed hashes with 4k lookups per second, and for potential adopters to be able to test the technology with sampled traffic with only a single machine. Making sure there are ways to manage such a deployment in production compatible with a few different infrastructure choices is our top priority.

Functionality for Confirming/Deconfirming Signals from Exchanges

One of the key features of HMA is the integration with Exchanges, which allow receiving hashes from third parties. Sometimes, despite the best practices of all contributors to an exchange, it may turn out that a benign image that only exists on your platform will show as matching to one of these hashes. Even a single signal can unexpectedly create a flood of matches that might otherwise overwhelm your downstream review queue.

There are a suite of features that are needed to defend against this, but having them can improve the baseline capabilities of HMA as whole:

1. Configurable match distance: PDQ for example supports matching things from distance 0-31 by default. You can reduce the distance in order to increase match precision, at the expense of recall, which may make sense for some platforms.
2. Negative Banks: These banks instead suppress match events of specific other banks. Which banks suppress should be configurable as part of the original bank’s configuration.
3. Flood prevention & Review: Approximate match counts for individual hash ids can be maintained in memory, and if the number of matches exceeds an expected baseline, the signal can be quarantined, preventing match events, and potentially logging to a separate location.
4. Confirming / Deconfirming Third Party Signals: For banks that are from third parties, once you have manually reviewed a match and confirm it matches what you expect, you can move the signal to a “confirmed” bank, or otherwise treat it differently. We should provide an API that allows recording a piece of content was harmful even if we are not banking it, and recording that the signal(s) that led to that match were valuable. Signals that don’t match harmful content could also have their matching disabled temporarily or permanently.

Improving Explainability and Demo Features

HMA is meant to be an introduction to photo/video similarity hashing, and tools to help debug why things are matching or not matching are key for earlier experiences. A UI-centric experience for re-running a previous matches, debugging why something does or does not match is key to gaining confidence in the system.

Additionally, displaying more metadata about test matches, especially information from an exchange could help gain confidence in matches that are happening.

Areas of Exploration for Expanding HMA Capabilities

Content moderation is a very big place, and HMA is a narrow tool for a certain type of detection. These are projects demonstrate new ways HMA can be useful, or add functionality that helps improve the wider ecosystem.

Recently Seen Content Clustering & Retroactive Matching

It is possible to insert every item that is passed into HMA into its own index, which can be used to automatically cluster similar images. This can be useful for understanding content on a platform, but also allows you to match content uploaded before you learn a piece of content is of interest.

A ring buffer of in-memory indices with a UI page and API for querying previously uploaded content can provide a demonstrator, and unlock additional functionality.

Functionality for Sharing Match and Feedback with an Exchange

If Confirming/Deconfirming is implemented, then having a table for your instance’s feedback to the exchange, and allowing to sync that feedback via the exchanges API can provide value to the participants of the exchange. This will likely require adding feedback to the Exchange interface in python-threatexchange.

Functionality for Sharing the Content of a Bank with an Exchange

Many exchanges provide both read and write functionality, but only read behavior (i.e. copying the exchange into a local bank), is implemented. A symmetrical implementation that allows copying data out of a bank and maintaining it in the exchange would also be of interest, but similar to match and feedback would require changes to the exchange interface in python-threatexchange.

[Pezmc]

We’re using HMA at mid-scale at FetLife for NCMEC and STOP_NCII support.

The biggest challenge in getting the service running reliably was the lack of documentation.

I had to dig deep into the codebases of hasher-matcher-actioner and python-threatexchange to configure our banks and APIs, to build a Ruby API Wrapper, apply custom config, and to have the HMA container running reliably.

This was made harder by inconsistent API responses (#1827), which are not easily typesafe compatible since they change structure based on query. The proposed Swagger docs, albeit inaccurate, were useful at the time.

Now that the service is running, our ongoing issues are:

• Hashes that are not NCI or CP showing up as matches. This is mostly an upstream issue with STOP_NCII, since anyone can submit images and there’s little review.
• What looks like overzealous matching in some cases. We currently manually reject PDQ distances above a threshold after lookup.
• HMA occasionally locking up under hashing load (for example during backfills). Our crude workaround is to kill and restart the container when it happens.

All that said, we’d support ongoing development. We’d especially benefit from your proposed “Functionality for Confirming/Deconfirming Signals from Exchanges.” The other proposals are less relevant for us, but I’d strongly emphasize that better documentation would have the biggest impact.

Thanks for all the work you’ve already put into making this usable at scale.

[Dcallies]

Thanks for the feedback @Pezmc , this is super valuable!

Better documentation

What form do you want to get that documentation in? Do you have a link to a project you think has good documentation we can use for inspiration?

Hashes that are not NCI or CP showing up as matches

This is expected for StopNCII, as the hashes are user-submitted. Expect adversarial use (e.g. people uploading pictures of mountains or cats) as StopNCII cannot filter them. This is why feedback on hashes is so valuable, and you can choose to treat hashes differently depending on if other platforms.

What looks like overzealous matching in some cases. We currently manually reject PDQ distances above a threshold after lookup.

HMA1.0 let you tweak the default matching distance, and we do want to bring that back in HMA 2.0 so you can choose to trade off between precision and recall. It looks like bank management features like this aren't in the roadmap, but I do think should be mentioned, though I'm split on if they are Most-Needed or not. My gut says yes.

HMA occasionally locking up under hashing load (for example during backfills). Our crude workaround is to kill and restart the container when it happens.

New problem unrelated to the memory leaks tracked in #1813 ?

[julietshen]

Is adding support for CSAI Match part of the roadmap? If not, what's the reasoning?

[julietshen]

Also, should I create a new issue for "HMA occasionally locking up under hashing load (for example during backfills). Our crude workaround is to kill and restart the container when it happens." ?