Filters for DNA Data Model

[iand]

This PR builds on #2295 and adds a range of filters for DNA Tests, DNA Matches and a few for People. Currently this PR is based on master which means it contains all the commits from the dna-core branch. Once #2295 is merged this PR will be a lot easier to review. The diff between the dna-core and dna-filters branches can be viewed here: iand/gramps@dna-core...iand:gramps:dna-filters

DNA Test filters

• AllDNATests - matches every DNA test (catch-all)
• HasIdOf - DNA test with a given Gramps ID
• DNATestPrivate - DNA tests marked private
• ChangedSince - DNA tests changed after a given date/time
• HasReferenceCountOf - DNA tests with a given reference count
• HasGallery - DNA tests with a given number of media
• HasNote - DNA tests with a given number of notes
• HasNoteRegexp - DNA tests with a note matching a regular expression
• HasNoteTag - DNA tests with a note bearing a given tag
• HasAttribute - DNA tests with a given DNA attribute
• MatchesFilter - DNA tests matching another saved DNA test filter
• HasYHaplogroup - DNA tests whose Y-DNA haplogroup contains text
• HasMtHaplogroup - DNA tests whose mtDNA haplogroup contains text
• HasPerson - DNA tests linked to a given person
• `MatchesPersonFilter - DNA tests linked to a person matching a saved person filter
• MatchesBothTestsOf - DNA tests that have a match with each of the two tests of a selected DNA match (matches of matches)

DNA Match filters

• AllDNAMatches - matches every DNA match (catch-all)
• HasIdOf - DNA match with a given Gramps ID
• DNAMatchPrivate - DNA matches marked private
• ChangedSince - DNA matches changed after a given date/time
• HasReferenceCountOf - DNA matches with a given reference count
• HasGallery - DNA matches with a given number of media
• HasNote - DNA matches with a given number of notes
• HasNoteRegexp - DNA matches with a note matching a regular expression
• HasNoteTag - DNA matches with a note bearing a given tag
• HasAttribute - DNA matches with a given DNA attribute
• MatchesFilter - DNA matches matching another saved DNA match filter
• HasSegment - DNA matches with a segment meeting combined criteria (chromosome, min cM, min SNPs, phase, IBD state)
• HasSegmentOverlapping - DNA matches with a segment overlapping a chromosome region
• SharesSegmentWithMatch - DNA matches sharing a segment with another given match
• HasSharedAncestor - DNA matches recording a given person as a shared ancestor (optional confidence)
• HasNoSharedAncestors - DNA matches with no shared-ancestor entries
• HasSubjectPerson - DNA matches whose subject kit belongs to a given person
• HasMatchPerson - DNA matches whose match kit belongs to a given person
• MatchesSubjectTestFilter - DNA matches whose subject kit matches a saved DNA test filter
• MatchesMatchTestFilter - DNA matches whose match kit matches a saved DNA test filter

Person filters

• HasDNATest - people with a DNA test meeting combined criteria (provider + test type, optional)
• HasDNATestWithTag - people with a DNA test bearing a given tag
• HasDNATestWithAttribute - people with a DNA test carrying a given DNA attribute
• HasYHaplogroup - people with a DNA test whose Y-DNA haplogroup contains text
• HasMtHaplogroup - people with a DNA test whose mtDNA haplogroup contains text
• HasDNAMatch - people with at least one DNA match
• HasDNAMatchWithSharedAncestor - people with a DNA match recording a given person as shared ancestor (optional confidence)
• IsSharedAncestor - people recorded as a shared ancestor in any DNA match (optional confidence)
• IsSharedAncestorInDNAMatchesOf - people recorded as a shared ancestor in the DNA matches of a given person

The following filters are already implemented in the dna-core branch and are in PR #2295

DNA Test filters (dnatest)

• HasDNATest - DNA tests whose fields contain a substring
• HasProvider - DNA tests from a given testing provider
• HasTestType - DNA tests of a given test type
• HasTag - DNA tests carrying a given tag
• RegExpIdOf - DNA tests whose ID matches a regular expression
• IsUnidentified - DNA tests with no linked person

DNA Match filters (dnamatch)

• HasDNAMatch - DNA matches whose fields contain a substring
• HasSharedCm - DNA matches with shared cM between a min and max
• HasTag - DNA matches carrying a given tag
• RegExpIdOf - DNA matches whose ID matches a regular expression

[dsblank]

I'm wondering about gramps/gen/filters/rules/dnamatch/_hasdnamatch.py

Here it looks like you are doing two filters: DNA match, plus it also filters on name. My main concern is that getting a person's display name is very expensive and slow. Wouldn't it be better to separate these two filters? The filter optimizer can work better when these two searches are separated. And with planned SQL implementations of filters, it may be possible to make the DNA match very fast, even with a lot of dna tests.

For more information on filter optimizations, see: #2350

[iand]

I'm wondering about gramps/gen/filters/rules/dnamatch/_hasdnamatch.py

Here it looks like you are doing two filters: DNA match, plus it also filters on name. My main concern is that getting a person's display name is very expensive and slow. Wouldn't it be better to separate these two filters? The filter optimizer can work better when these two searches are separated. And with planned SQL implementations of filters, it may be possible to make the DNA match very fast, even with a lot of dna tests.

For more information on filter optimizations, see: #2350

Thanks for the detailed look. This filter was added in the core PR to support sidebar searching on the DNA Match view. As you point out this is probably over broad and doesn't need to filter on the person's name. I'm aware of the filter optimisations idea which I think is great, so it would be good to play nice with it.

I'll take a look and experiment a bit with what I feel would be a good compromise. I think any fix will be applied against #2295 for consistency.