Redactions

[Michael-F-Bryan]

Insta has a neat feature called redactions which let you omit certain values from the snapshot (e.g. a timestamp field which will be different on every run).

Is it possible to do something similar with pytest-insta?

[razzeee]

I would really like that, I'm employing a hack for this right now and it leads to me not being able to generate new snapshots easily, when I add new tests.

https://github.com/flathub-infra/website/blob/main/backend/tests/main.py#L29

[vberlier]

Built-in support for redactions is not really needed for pytest-insta. Python is a much more dynamic language than Rust, you can redact anything with an ordinary a line of code that sets a value to None or "(redacted)" before taking the snapshot.

e.g. @razzeee

def test_apps_by_category(client, snapshot):
    response = client.get("/category/Game")
    assert response.status_code == 200
    responseJson = response.json()
    responseJson["processingTimeMs"] = 123  # Overwrite with fixed number to ignore variations
    assert snapshot("test_apps_by_category.json") == responseJson

The key is modifying the actual value the snapshot is compared against, not the expected reference value returned from snapshot() like in your _assertAgainstSnapshotWithoutPerformance function. As you pointed out, if you try to modify the reference value it won't work when the snapshot gets initially created as there's no reference value to modify yet.

The original insta needs its own mechanism and DSL to let you perform redactions because Rust doesn't make it as convenient to just poke at the values you need to modify directly. In Python you'd write as much code configuring any baked-in redaction options as doing it yourself.

This means that you don't have to depend on a DSL that might not adapt to your requirements. And since redactions are just ordinary code, you can put them in regular functions to reuse them wherever you want!

def ignore_perf(responseJson):
    responseJson["processingTimeMs"] = 123
    return responseJson

def test_apps_by_category(client, snapshot):
    response = client.get("/category/Game")
    assert response.status_code == 200
    assert snapshot("test_apps_by_category.json") == ignore_perf(response.json())

If you want a DSL, make your own! It can be a simple helper like the one I shared the last time this was brought up #85

SUBSTITUTIONS = {
    ("id",): "",
    ("billing", "details"): {},  # can substitute entire json objects
    ("api_keys", "*", "token"): "",  # use "*" to select all elements in a list
}

def stable(data, subst, path=()):
    for suffix, replacement in subst.items():
        if path[-len(suffix) :] == suffix:
            return replacement
    if isinstance(data, dict):
        return {key: stable(value, subst, (*path, key)) for key, value in data.items()}
    if isinstance(data, list):
        return [stable(value, subst, (*path, "*")) for value in data]
    return data

assert snapshot("json") == stable(response.json(), SUBSTITUTIONS)

The advantage is that it's not external code that you'll have to bend to your will to make it do what you want to do.

I think I'll add a section about redactions to the README to explain why there's no baked-in utility. I probably should've done that before closing the original issue actually ^^"