fix(dictionary): better handle conversion of hyphens to spaces - #417
Merged
Conversation
Cregeen prints 'thie veaghee'; the corpus says it once, as the Acts' 'thie-veaghee'. AttestationsOf already folded the two together, so the word page listed the attestation while greying the headword unattested. Attestation's phrase branch now asks the same folded count table before falling back to the phrase scan. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The lemma table folds hyphens to spaces, but the timeline scanned each
form literally: 'thie veaghee' never matched the corpus token
'thie-veaghee' and showed an empty timeline, and 'cur my ner' counted
40 of ~1,880 uses because the Bible writes 'Cur-my-ner'. Scan each
form with IgnoreHyphens, keeping affix queries ('aa-*') literal so the
wildcard is not split off the hyphen, and collapse forms differing
only in joining ('thie veaghee' / 'thieveaghee') into one scan so the
same token is not counted under each spelling.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
david-allison
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
thie veaghee==thie-veaghee