Display group hashes and identify groups from a description (LMFDB#5556)

[roed-math]

Surfaces the isomorphism-invariant group hash in the finite groups interface and adds a
group-identification tool. Abstract group homepages now show the stored hash (with a search
link) whenever it is informative (present and unequal to the label counter), there is an
optional off-by-default "Hash" search column, and the two hash placeholders in the subgroup
data popups are linkified. A new /Groups/Abstract/identify endpoint (with a form on the index
page and jump-box wiring for permutation input) lets a user enter permutation generators, a PC
code, or matrix generators; the server computes the LMFDB (Magma) hash in-process via libgap —
a faithful port of the FiniteGroups Hash.m — using strict parsers that never eval user input,
a 10^6 order cap, and alarm() timeouts. Results go through GAP identification (a proof), the
complete gps_smallhash tables (a unique match is a proof), then an (order, hash) lookup in
gps_groups, always with mathematically honest messaging that a hash match alone does not prove
isomorphism. Verified by reproducing many stored hashes against the database, exercising every
input/lookup/error/timeout path through the Flask test client, and running the abstract-groups
and browse-page test suites (pyflakes clean). Subgroup homepages are unchanged because the
subgroup/quotient hash columns are entirely NULL in the current data.

All hash resolution lives in one new module, lmfdb/groups/abstract/hash_lookup.py, so that
N#h means the structural hash in the find box, on group homepages, in the subgroup and
quotient popups, in the identification tool and in the optional column. That matters because
gps_groups.hash stores the label counter rather than a hash at six of the ten orders with a
complete gps_smallhash table (512, 1152, 1536, 1920, 2187, 15625), so searches at those orders
resolve through gps_smallhash, where the answer is provably complete and a unique match is a
proof of isomorphism. A stored counter is never displayed as a hash. Parsing, group construction
and the order computation share one alarm, and the order cap is checked before the order is
factored; GAP's own start-up is deliberately outside the alarm, since interrupting it leaves
libgap unusable for the whole worker. Matrix entries over GF(p^e) are integer codes whose base-p
digits are coordinates in the power basis of the Conway generator, matching the FiniteGroups
DecodeMat convention, so all of GF(q) can be entered and Mat(1,4):[[2]] is the generator of
GF(4).

Addresses LMFDB#5556.

🤖 Generated with Claude Code

[roed-math]

All three merge blockers from the review are addressed, plus one problem the work turned up.

1. One hash resolution layer

New file lmfdb/groups/abstract/hash_lookup.py (no dependency on main.py; the one call into web_groups is deferred) is now the only place that turns order#hash into a set of groups and the only place that decides which stored value may be shown as a hash:

• resolve_order_hash(order, value) -> HashResolution(order, value, labels, complete, source), with in_lmfdb() and unique_label() helpers;
• hash_constraint(order, values, qfield) for search queries;
• structural_hash(counter, stored) for display;
• SMALLHASH_ORDERS, hash_search_url, order_search_url moved here.

Everything user-facing goes through it: identify.py::_lookup, main.py::group_jump, main.py::parse_hashes, the group-homepage hash link, the subgroup and quotient popup links, and the optional search column. N#h now means the structural hash everywhere.

What the data actually looks like, since it decides the rule (verified over every row of each order):

orders	gps_groups.hash
512, 1152, 1536, 1920, 2187, 15625	the label counter, on 100% of rows (512.11 stores 11, its hash is 1584677793794603025)
6561, 16807, 78125, 161051	the real structural hash
everything else	the real structural hash (equal to the counter at identifiable orders, where the hash is the small-group index)

At all ten SMALLHASH_ORDERS the complete gps_smallhash table is now the source of truth for searching, so a unique match is a proof of isomorphism and a collision is reported as a complete cluster, including its members with no LMFDB row (78125#3521944227884464685 is {78125.82, 78125.335, 78125.340}; only the last two are in gps_groups). Where the missing member still has a GAP-computable homepage the candidate is linked and marked "not in the database"; at 6561 (3^8, Magma-only) it is not linked. A computed hash absent from a complete table now renders a red data-integrity note rather than a blank paragraph.

One deviation, and it needs a decision. The review asked for the public hash at those orders to be read from gps_smallhash by (order, counter) and bulk-loaded for search rows. That is not possible against this schema: gps_smallhash has 420,974,694 rows and exactly one index, on (order, hash). Going the other way is a sequential scan, measured at 389 s for a single (512, 11) lookup on devmirror, so it cannot back a page render at any batch size. The display side therefore suppresses the value at those orders rather than printing a counter under the group.hash knowl (homepage row hidden, column cell empty), and the module docstring says why. Making the real hash displayable there needs an (order, counter) index on gps_smallhash, which is a database-side change; say the word and I will open a separate issue for it.

2. Parsing, construction and the order computation are inside the guard

identify_group now runs _parse and G.Order() under a single alarm, converts parser-side GAPError/RuntimeError/TypeError into a normal error page, and checks ORDER_CAP before latex(N.factor()), so an over-cap order is never factored. Every path cancels the alarm; the module security note describes the boundary as implemented.

The problem this turned up: an alarm that fires while GAP is starting up leaves libgap unusable for the rest of the worker (every later call returns Aborted, which breaks live group pages too, not just this tool). A fresh worker takes about 20 s to start GAP, so with parsing inside a 5 s alarm the first identification after a restart would poison the process. prepare_gap() now forces that start-up outside the alarm; interrupting GAP once it is running recovers cleanly (checked explicitly). Timing out during GAP start-up was reachable on the previous revision too, through G.Order().

3. A real encoding for extension-field entries

entry_ring(q) documents and implements the code convention: for q = p^e with e > 1, an entry is an integer 0 <= x < q whose base-p digits (least significant first) are its coordinates in the basis 1, a, ..., a^(e-1) of the Conway generator a, which is the element GAP writes as Z(q). This is exactly the FiniteGroups DecodeMat convention, whose Basis(GF(q)) is that power basis; the equality is asserted element by element against libgap for q = 4, 8, 9 in the tests. Out-of-range and negative codes are refused with a message naming the convention. Prime fields and Z/q are unchanged, and the format table on the identify page documents the encoding beside the matrix format.

Mat(1,4):[[2]] is now the generator of GF(4), giving C_3 = 3.1, instead of being rejected as singular.

Tests

test_abstract_groups.py grows from 4 to 12 hash/identify tests, covering each numbered item in the review's three test lists: resolver unit tests (512#… unique, the 78125 cluster with a non-LMFDB member, 2016 unchanged, hash_constraint, structural_hash); jump redirect and search-page results at complete-table orders; subgroup and quotient popup hrefs; the identify page's "all groups with this order and hash" link resolving the same set it displayed; the missing-table diagnostic (monkeypatched resolver); the column display rule; the guard behaviours (parser-side AlarmInterrupt and RuntimeError, over-cap with a factorization spy, cancel_alarm counted on every path, HTTP 200 not 500); and the matrix encoding (Mat(1,4):[[2]], Mat(1,9):[[3]], prime-field and Z/n regressions, out-of-range codes, the GAP round trip).

sage -python -m pytest lmfdb/groups/abstract/test_abstract_groups.py -q   # 22 passed
sage -python -m pytest lmfdb/groups/abstract/test_browse_page.py -q       # passed
sage -python -m pyflakes lmfdb/groups/abstract/*.py                       # clean

[roed-math]

Both remaining items are addressed in c936653bc, plus one crash the route work exposed.

Complete hash searches now render the complete cluster

New route /Groups/Abstract/hash/<order>/<value> (by_hash, template abstract-hash.html) answers "which groups have this order and hash":

• at the ten complete-table orders it lists every such group, including the ones with no gps_groups row, states that the list is complete, links the labels that have a homepage (database row, or a GAP-computable page), marks the rest "not in the database", and redirects when the cluster determines one group that has a page;
• when the complete table has no such hash it says so, which at those orders is a proof of absence rather than an empty result;
• at every other order it forwards to the ordinary gps_groups search page, where the stored column is the hash and the usual search machinery (columns, refine, download) applies.

hash_search_url now points there, so all five link sources go through it: group homepages, the subgroup and quotient popups, the identification tool, the find box, and any search whose only constraints are an order and a hash (index forwards those; add a real filter and you get the ordinary filtered search, which is a subset by construction rather than by accident).

So ?hash=78125#3521944227884464685 shows all of 78125.82, 78125.335, 78125.340, with 78125.82 marked absent from the database — where before it showed the middle two only. The order-6561 cluster of 18 renders as 2 links and 16 plain labels, since 3^8 has no GAP small-group library to build a page from.

Also fixed: a unique complete-table match with neither a row nor a live page (the last bullet of the review's list) now displays its label instead of falling through to an empty search.

Counters no longer answer structural-hash searches

hash_constraint never consults gps_groups.hash at the six orders where it holds a counter, whatever the order constraint. With no single order it builds

{"$or": [{"hash": 1584677793794603025, "order": {"$nin": [512, 1152, 1536, 1920, 2187, 15625]}},
         {"order": 512, "counter": {"$in": [11]}}]}

from one indexed gps_smallhash query over those six orders (merged with collapse_ors, so it composes with an existing $or from the family parser, and any order range in the query still applies to every branch). Consequences, all now tested:

search	before	after
?hash=11	included 512.11	does not
?order=500-600&hash=11	included 512.11	does not
?hash=1584677793794603025 (no order)	missed 512.11	finds it
?order=512&hash=1584677793794603025,1718285292446712970	—	both, through gps_smallhash

I took the preferred option rather than rejecting unpinned hash searches, since the $or costs one extra indexed query and keeps the search box usable.

Optional column

The limitation is now documented in the hash_lookup module docstring: a row at one of the six counter-storing orders cannot be asked for its own hash without an (order, counter) index on gps_smallhash, so its homepage and its column cell stay empty rather than showing a counter. Where the search itself named one hash at one order, searched_hash recovers the value from the query and the column shows it (?order=512&hash=1584677793794603025&showcol=hash now prints the hash on the 512.11 row).

One crash found on the way

/Groups/Abstract/hash/512/99999999999999999999 reached Postgres and raised NumericValueOutOfRange (a 500). The route now refuses orders and values too large for the int4/int8 columns with a flash. The search-page path already returned a clean error page for the same input.

Tests and CI

test_abstract_groups.py is at 25 passed; the hash tests now cover each numbered item of both remaining lists: the three-label cluster page with its completeness message and absent-label marking, the empty complete result, the ordinary-order forward, the mocked unique-label-without-a-homepage rendering, the identification link (real and mocked-complete) landing on the same set it displayed, subgroup and quotient popup links landing on all three labels, the four false-positive cases, hash_constraint's $or shape, and the column's three display rules.

sage -python -m pytest lmfdb/groups/abstract/test_abstract_groups.py -q   # 25 passed
sage -python -m pytest lmfdb/groups/abstract/test_browse_page.py -q       # 71 passed
sage -python -m pyflakes lmfdb/groups/abstract/*.py                       # clean

On CI: the runs for 490df40ed sat queued for an hour, along with every other run on this fork, and were superseded before starting; the run for c936653bc has now begun, so the first CI validation of either round of tests will be this one.

[roed314]

GPT signed off.

[roed-math]

Superseded by LMFDB#7177, opened upstream from this same branch. Closing here; review continues upstream.