frozenset[str] fields in API get shuffled

[srabraham]

It's kind of unfortunate that we end up shuffling collections in API fields. For example, I noticed this in the ranger_handles field in the incidents endpoint (like this one: https://ranger-ims-staging.burningman.org/ims/api/events/2023/incidents/), and it's presumably the same for incident_types (https://ranger-ims-staging.burningman.org/ims/api/incident_types/).

Even if you start with sorted values in _db.py, they eventually get shuffled before encoding, because frozenset[str] are unordered, e.g. below

FYI @wsanchez

>>> for val in frozenset(("a","b","c")):
...     print(val)
...
a
c
b

[srabraham]

This should be about as easy as replacing the freezeStrings function. frozenset(some_collection) jumbles up the order the collection. The below approach maintains the order and still returns an immutable, hashable collection with duplicates removed.

def freezeStrings(strings: Iterable[str]) -> tuple[str]:
    deduped = {s: None for s in strings}
    return tuple(s for s in deduped)

[srabraham]

Probably the right move is this change to _converters.py, so that fields can still be made immutable and hashable, but without losing ordering (or having duplicates removed).

This change might just work on its own, but there's a bunch of cleanup to do, mostly switching type hints. Also this would touch the auth component, which makes me want to tread carefully.

diff --git a/src/ims/model/_convert.py b/src/ims/model/_convert.py
index 14b817ca..500c78fe 100644
--- a/src/ims/model/_convert.py
+++ b/src/ims/model/_convert.py
@@ -20,19 +20,19 @@
 Converters
 """

-from collections.abc import Iterable
+from collections.abc import Iterable, Sequence
 from datetime import datetime as DateTime


 __all__ = ()


-def freezeIntegers(integers: Iterable[int]) -> frozenset[int]:
-    return frozenset(integers)
+def freezeIntegers(integers: Iterable[int]) -> Sequence[int]:
+    return tuple(i for i in integers)


-def freezeStrings(strings: Iterable[str]) -> frozenset[str]:
-    return frozenset(strings)
+def freezeStrings(strings: Iterable[str]) -> Sequence[str]:
+    return tuple(s for s in strings)

[wsanchez]

Yeah, this is partly me being a PITA to myself in that in clients care about the order, the clients should sort.

That said, I agree that the order should not change when deserializing data, because it means you shouldn't be using strong ETags and that you can't provide a useful hash of the result.