Skip to content

Commit 0aa5002

Browse files
tballisonCopilot
andauthored
Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
1 parent b7c6f1b commit 0aa5002

1 file changed

Lines changed: 7 additions & 4 deletions

File tree

  • tika-ml/tika-ml-junkdetect/src/main/java/org/apache/tika/ml/junkdetect

tika-ml/tika-ml-junkdetect/src/main/java/org/apache/tika/ml/junkdetect/JunkDetector.java

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1023,10 +1023,13 @@ private static double scorePairF1(int cpA, int idxA, int cpB, int idxB,
10231023
// pass through unchanged. Folding the edges too is what fully rescues short
10241024
// all-caps headings, whose ^X/X$ bigrams would otherwise floor on the rare
10251025
// uppercase-letter unigram backoff.
1026-
if ((Character.isUpperCase(cpA) || Character.isUpperCase(cpB))
1027-
&& !Character.isLowerCase(cpA) && !Character.isLowerCase(cpB)) {
1028-
int lcA = Character.isUpperCase(cpA) ? Character.toLowerCase(cpA) : cpA;
1029-
int lcB = Character.isUpperCase(cpB) ? Character.toLowerCase(cpB) : cpB;
1026+
boolean upperA = Character.isValidCodePoint(cpA) && Character.isUpperCase(cpA);
1027+
boolean upperB = Character.isValidCodePoint(cpB) && Character.isUpperCase(cpB);
1028+
boolean lowerA = Character.isValidCodePoint(cpA) && Character.isLowerCase(cpA);
1029+
boolean lowerB = Character.isValidCodePoint(cpB) && Character.isLowerCase(cpB);
1030+
if ((upperA || upperB) && !(lowerA || lowerB)) {
1031+
int lcA = upperA ? Character.toLowerCase(cpA) : cpA;
1032+
int lcB = upperB ? Character.toLowerCase(cpB) : cpB;
10301033
if (lcA != cpA || lcB != cpB) {
10311034
int lcIdxA = codepointToIndex(tables, lcA);
10321035
int lcIdxB = codepointToIndex(tables, lcB);

0 commit comments

Comments
 (0)