Lucene API comparison tool, #1022#1314
Open
paulirwin wants to merge 66 commits into
Open
Conversation
…s and trailing commas in IDE
…ort, fix some bugs
…pace instead of using config
Lucene.NET implements the dispose pattern: Java's zero-arg close() is exposed as protected Dispose(bool disposing) on the .NET side, with a public Dispose() inherited from IDisposable. Previously the bool overload showed up as an unmatched .NET-only method. Recognize the idiom in MethodsMatch and suppress the by-design visibility/parameter divergence so it isn't reported as a member difference. Clears 79 Java close() and 79 .NET Dispose(bool) entries from the diff. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Java enums always carry static/final (or static/abstract for enums with abstract methods, which Lucene.NET ports as a different shape). None of those modifiers apply to a .NET enum, which is implicitly sealed and can't carry static or abstract. Drop them from the comparison when both sides are enums so the comparison degenerates to public-only. Clears all 23 enum-type modifier mismatches; total type-modifier diffs drop from 38 to 15. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extend the ApiCheck comparison logic so the existing Int→Int32 / Long→Int64 / Short→Int16 / Float→Single / Comparator→Comparer rename rules work for type names too — including Java nested types flattened to Outer.Inner form (e.g. PackedInts.Header ↔ PackedInt32s.Header). Add a Field branch to the modifier comparison so Java public static final ↔ .NET public static readonly (and Java public static final ↔ .NET public const) match. Also override Parameter.ToString() so the HTML report renders parameter types instead of "Lucene.Net.ApiCheck.Models.Diff.Parameter". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The ApiCheck inference path now applies the same primitive-word renames (Int→Int32, Long→Int64, Short→Int16, Float→Single, Comparator→Comparer) to type names that it already applies to member names, so the explicit [LuceneType] markers on classes whose only divergence is one of those renames are no longer load-bearing. Remove them — and the now-unused Lucene.Net.Reflection usings — across Lucene.Net, Lucene.Net.Codecs, Lucene.Net.Facet, Lucene.Net.Queries, and Lucene.Net.Misc. Attributes are kept where the divergence isn't mechanical (Morfologik package mismatch, LRU casing, AbstractAllGroupHeadsCollector_GroupHead's flattened nesting, UpToTwoPositiveIntOutputs's non-standard Int→Int64 rename, SingularFunction, LuceneVersion, and Collector's interface-vs-class kind mismatch). Also flip NoLuceneEquivalentAttribute to Inherited = false so a base type marked NoLuceneEquivalent no longer suppresses inference on derived generic types (e.g. FieldComparer<T> would otherwise inherit the marker from the non-generic FieldComparer base). The attribute is meant to opt a specific declaration out, not its descendants. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- ModifierComparison: treat Java 'protected' as equivalent to .NET 'protected internal' on members and fields (Lucene.NET widens for test-framework access). - ModifierComparison: when the .NET declaring type is sealed, drop normalized 'sealed' so member-level final/override designators don't surface as diffs. - ModifierComparison: drop normalized 'sealed' on Java 'static final' methods since 'final' on a static method only blocks subclass hiding, with no .NET analogue. - DiffUtility: when a Java public field matches a .NET property by both name and compatible value type, treat as equivalent (no diff recorded). - DiffUtility/MemberComparison: silence both Dispose() and protected Dispose(bool) as the dispose-pattern siblings of a single Java close(), including when close is inherited via java.io.Closeable / java.lang.AutoCloseable. - TypeExtensions.FormatDisplayName: render BCL primitives, System.Void, string, and object as their C# keyword forms; recurse through array, by-ref, pointer, and generic argument wrappers. - TypeComparison: map java.io.PrintStream to System.IO.TextWriter (Lucene.NET porting equivalent). - Report: convert each per-assembly subsection to a collapsible <details> block with a custom rotating disclosure marker. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Upgrade Lucene.Net.ApiCheck and Lucene.Net.Tests.ApiCheck from net8.0 to net10.0. - Rename apicheck-config.json5 to apicheck-config.jsonc; the file uses only comments and trailing commas (the JSONC subset), both of which System.Text.Json already accepts via ReadCommentHandling=Skip and AllowTrailingCommas=true. - Drop committed src/java/lucene-api-extractor/.idea/* IDE files; the root .gitignore already covers .idea/. - Add trailing newline to src/java/lucene-api-extractor/.gitignore. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The lucene-api-extractor Java source is no longer vendored here; it now lives standalone at https://github.com/paulirwin/java-api-extractor and is published via GitHub Releases. - Remove src/java/lucene-api-extractor entirely. - Rewrite apicheck.ps1 to download the pinned v0.1.0 shaded ("all") fat jar from the external repo's GitHub Release via the gh CLI (cached under _artifacts/, re-downloaded with -clean) instead of building it with Maven. Checks for the gh CLI and reports an actionable error if it's missing. - Update the ApiCheck README to document the external tool and the gh CLI + Java 21 requirements. The published v0.1.0 schema predates the later enumConstants[] split, so enum constants still live in fields[]; the .NET Models/JavaApi records and JarToolIntegration remain compatible and are unchanged. Verified end-to-end: the extractor jar downloads, runs, and the diff/report generate for all 25 Lucene libraries. Also fixes two pre-existing LuceneDev6001 analyzer errors in MemberComparison.cs (missing StringComparison on StartsWith/EndsWith) that blocked the ApiCheck project from compiling. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The namespace->Java-package inference lowercases the whole suffix, so `Lucene.Net.QueryParsers.ComplexPhrase` inferred `...queryparser.complexphrase` and never matched Java's `org.apache.lucene.queryparser.complexPhrase` (note the interior capital P). As a result ComplexPhraseQueryParser showed up as both Lucene-only and Lucene.NET-only in the ApiCheck report. Add an explicit mapping that preserves the camelCase package segment, mirroring the existing Lucene.Net.Benchmarks.ByTask -> org.apache.lucene.benchmark.byTask mapping. ComplexPhraseQueryParser now matches; verified by regenerating the ApiCheck report. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
These 11 files had a single blank line inserted after the `// Lucene version compatibility level 4.8.1` comment with no other changes. Restore them to match master to keep the PR diff free of whitespace-only noise. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MaxFloatFunction, MinFloatFunction, and ProductFloatFunction differed from master only by a doc-comment grammar fix (it's -> its) and, for two of them, an added blank line after the version comment. No code changes. Restore them to master to keep the PR diff focused. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bump System.CommandLine from 2.0.0-beta4 to the stable 2.0.8 in Lucene.Net.ApiCheck and migrate Program.cs to the new API: - Option(name, aliases...) ctor with Description set via initializer; global options use Recursive = true instead of AddGlobalOption. - Command.SetAction(parseResult => ...) with parseResult.GetValue(option) replaces SetHandler. - rootCommand.Parse(args).InvokeAsync() replaces RootCommand.InvokeAsync(args). Verified the root/diff/report help views, argument parsing, and a full end-to-end apicheck.ps1 run; diff totals are unchanged. Also remove the report's index.js: it only contained an empty no-op onContentReady handler. Dropped the file, its EmbeddedResource, the copy entry in ReportCommand, and the <script> tag in index.handlebars. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Update the test project's NuGet packages and move from xUnit v2 to v3: - xunit 2.9.2 -> xunit.v3 3.2.2 - xunit.runner.visualstudio 2.8.2 -> 3.1.5 - Microsoft.NET.Test.Sdk 17.12.0 -> 18.6.0 xUnit v3 test assemblies are self-running executables on the Microsoft.Testing.Platform runner, so add OutputType=Exe and UseMicrosoftTestingPlatformRunner. The test sources only use Fact/Theory/ InlineData/Assert, which are source-compatible across v2 and v3, so no test code changes were needed. All 243 tests pass via both `dotnet test` and the direct MTP executable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lucene-analyzers-opennlp was excluded because it has no 4.8.1 release: the module was added to upstream Lucene long after 4.8, and Maven Central's earliest published version is 7.3.0. The Lucene.Net.Analysis.OpenNLP assembly is therefore mapped to 8.2.0 (via its LuceneMavenMapping) rather than 4.8.1 like every other module. The scan now runs cleanly against 8.2.0 (16 types matched, 0 type-level gaps, only a handful of member diffs), so enable it for coverage. Note its baseline is a newer API generation than the rest of the report. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CI's EditorConfig check failed with "Wrong character encoding (UTF-8-SIG instead of utf-8)" on seven files that were committed with a byte-order mark. The repo's .editorconfig requires charset = utf-8 (no BOM). Remove the BOM from each; no other content changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TestApiConsistency.TestPrivateFieldNames (run in the _J-S test shard) scans the whole Lucene.Net assembly and requires private fields to be camelCase (optionally _-prefixed) or ALL_CAPS. The new ReflectionTypeExtensions.LuceneNetNamespaceRegex was PascalCase and failed that check. Rename it to luceneNetNamespaceRegex (camelCase), matching the sibling packageMappingCache field. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address four defects in the ApiCheck comparison that produced spurious type/member differences against the Lucene 4.8.1 Java API: - InterfacesMatch: replace the equal-count + ".NET subset of Java" rule with a subset check (every Java interface must be represented on the .NET side) and drop the java.lang.Cloneable marker. .NET reports the full transitive interface set and intentionally enriches the surface (IFormattable, IEquatable<T>, IDisposable, transitive generic-collection interfaces), so the old rule could never line up. Cuts reported interface mismatches from 205 to 56 (remaining are genuine unmatched interfaces). - Method/constructor arity fallback: recompute HasTypeMismatch via MethodSignaturesMatch / ConstructorSignaturesMatch instead of hardcoding true, and skip recording the pair when nothing actually differs. The pair reached the fallback only because the strict match failed for an unrelated reason (e.g. a single de-nested type), so the signatures are often compatible. - Lucene.Net.Spatial: add the missing LucenePackageMapping for Lucene.Net.Spatial.Queries -> org.apache.lucene.spatial.query (the Java package is singular "query"); without it the namespace auto-lowercased to the wrong "spatial.queries" package and double-counted SpatialArgs, SpatialArgsParser, and SpatialOperation. - GetInferredLuceneTypeName: strip the generic-arity backtick suffix from each enclosing declaring-type name, not just the leaf, so nested types under a generic type match (e.g. PairOutputs.Pair, BlockTreeTermsReader.FieldReader). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Teach the comparison about four systematic Lucene.NET porting conventions so they no longer surface as spurious differences against Lucene 4.8.1: - De-nesting (H-1): match a Java nested type (Outer$Inner) to a .NET type that was promoted to namespace level (Occur, IndexOptions, OpenMode, TokenStreamComponents, ...) on the innermost segment, still gated on package and kind equality so a candidate must live in the same package. - Type equivalences (H-4): map Lucene.Net.Util.ICloseable to the Java close()-bearing markers (the apache#271 work item), reduce a closed generic (IComparable<Term>) to its open definition before the WellKnownEquivalentTypes lookup so it matches the IComparable<> key, and treat java.io.Serializable as a JVM marker interface with no .NET equivalent (alongside Cloneable). - Base-type idioms (H-6): new BaseTypesMatch helper that accepts interface-ification (Java abstract base -> .NET object + matching IFoo, e.g. the Collector cluster) and the generic/non-generic same-name split (FieldComparer<T> deriving from FieldComparer while Java derives from Object). - Modifier idioms (H-7): Java 'public abstract' holder class is equivalent to a .NET 'public static' class (StringHelper, PriorityQueue), and a .NET struct is the value-type port of a Java final/static-final nested class (drop the Java-only final/static that has no struct analogue). Effect vs the prior commit: interface mismatches 56 -> 26, base-type 40 -> 19, matched-member diffs 988 -> 894, Java/.NET types-not-in counts 177/383 -> 120/327. No types were added to either not-in list. The remaining residue is genuine divergence (handled later via suppress attributes) or out-of-scope heuristics. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e#1022) - BUG-5: the Impl-suffix strip in TypesMatch was one-directional (Java name only); GetInferredLuceneTypeName never strips Impl from the .NET name, so a type kept as 'FooImpl' identically on both sides failed to match. Compare the .NET name against both the stripped and un-stripped Java name. - H-8: tolerate a trailing CancellationToken parameter. Lucene.NET appends an optional CancellationToken to many methods (Search/Train/Lookup/Collect, "LUCENENET Specific"); ParameterListsMatch now drops a lone trailing CancellationToken and MethodNamesAndArityMatch counts effective arity, gated tightly on the parameter being exactly CancellationToken. - H-9: extend the type-equivalence map - non-generic System.Collections.IDictionary -> java.util.Map (the generic form was already present), StringBuilder, CultureInfo, TimeZoneInfo, Encoding, DateTime, PrintWriter, XmlElement, the J2N boxed-numeric / atomic / IO / BitSet forms, and a name-keyed map for Spatial4n/ICU4N types in assemblies ApiCheck does not reference at compile time. Also reduce a closed generic to its open definition before the lookup so IComparable<Term> matches the IComparable<> key. - H-10: known accessor renames where a Java zero-arg method maps to a differently named .NET property - size()/getSize() -> Count, getFilePointer() -> Position. (The next()/hasNext() -> MoveNext/Current iterator rename is intentionally left out: one Java method maps to two .NET members across heterogeneous types, too ambiguous to auto-match safely.) Adds/updates unit tests in TypeComparisonTests, MemberComparisonTests, and ModifierComparisonTests covering every case above (276 tests pass). Effect vs the prior commit: matched-member diffs 894 -> 642, Java/.NET members-not-in 993/1112 -> 874/1002, types-not-in 120/327 -> 118/323. No types added to either not-in list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…he#1022) Classified the member-not-in buckets by reason and addressed the four matcher-fixable findings (the rest are genuine divergence or intentional .NET enrichment, for a later suppress pass). Developed test-first. - F-1: FieldNamesMatch was case-sensitive while every other name matcher is case-insensitive, so a Java camelCase field never matched its .NET PascalCase const (defaultMaxEdits <-> DefaultMaxEdits). Make it case-insensitive and add lower_snake_case collapsing (scale_factor <-> ScaleFactor). The snake rule is restricted to all-lowercase names so a SCREAMING_SNAKE constant (MAX_LEVELS) is not de-snaked and conflated with a separate camelCase field (maxLevels) on the same type. - F-2: a .NET property and a standalone Set/Get method could both correspond to a Java get/set pair (RateLimiter.MbPerSec property + SetMbPerSec method <- getMbPerSec + setMbPerSec), but the property greedily consumed both Java accessors, orphaning the .NET method. DiffMethodsAndProperties now diverts an accessor to a name+arity-matching .NET method only when the property still retains at least one accessor, so a single Java accessor with both a .NET property and method (BooleanQuery.getClauses) stays with the property. - F-3: strip a leading 'Do' (followed by uppercase) on the .NET method name so Lucene.NET template/hook renames pair (lookup() <-> DoLookup(), checkIndex() <-> DoCheckIndex()). - F-4: accept a fluent setter - a Java setX(T) that returns the owner type rather than void - as a property setter, keeping the param-type guard so it can't over-match an unrelated method. Tests (TDD: red first, then green) in MemberComparisonTests cover each finding plus regression guards (SCREAMING_SNAKE/camelCase collision, fluent-setter type mismatch, Do-prefix boundary). 289 tests pass. Effect: Java/.NET members-not-in 874/1002 -> 821/950, with zero newly-orphaned members in either direction. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…pache#1022) Covers the BUG-4 fix in GetInferredLuceneTypeName (commit 9fc36a4) where the back-tick arity is stripped from each enclosing declaring-type name, not just the leaf, so a type nested under a generic outer (PairOutputs<A,B>.Pair) infers "PairOutputs.Pair" rather than "PairOutputs`2.Pair". This test lives in the main Lucene.Net test project alongside the other GetInferredLuceneTypeName cases, since the change is in the shipping Lucene.Net assembly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Pushed changes to this PR that significantly reduced mismatches. It's still ready for review, as subsequent improvements can be made in a separate PR. The latest updates include:
Net impact to mismatches:
What's leftThe remaining ~1,450 members and ~440 types are seemingly-genuine API differences after a thorough AI audit. These will be handled by suppression attributes separately. I think we've reached Pareto-optimal heuristics, although there might be a few long-tail opportunities still after this PR is merged. |
1 task
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new “API check” utility to compare Lucene.NET’s public API surface against the corresponding Java Lucene artifacts (via java-api-extractor JSON), along with reflection-based mapping/suppression attributes to handle intentional Java↔.NET shape differences.
Changes:
- Introduces
Lucene.Net.Reflectionattributes + extension helpers to infer/map Java package/type names and suppress known discrepancies. - Adds
src/dotnet/Lucene.Net.ApiCheck(CLI + diff engine + HTML report via Handlebars) and a dedicatedLucene.Net.Tests.ApiChecktest project. - Adds
apicheck.ps1+apicheck-config.jsoncand wires projects/scripts into the solution.
Reviewed changes
Copilot reviewed 111 out of 112 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Lucene.Net/Util/Version.cs | Adds [LuceneType] mapping for LuceneVersion. |
| src/Lucene.Net/Support/Reflection/ReflectionTypeExtensions.cs | New type→Java package/type inference helpers. |
| src/Lucene.Net/Support/Reflection/ReflectionAssemblyExtensions.cs | Adds cached assembly-level package mapping retrieval. |
| src/Lucene.Net/Support/Reflection/NoLuceneEquivalentAttribute.cs | Adds attribute to suppress Lucene equivalence comparisons. |
| src/Lucene.Net/Support/Reflection/LuceneTypeInfo.cs | Adds inferred/effective Java type info container. |
| src/Lucene.Net/Support/Reflection/LuceneTypeAttribute.cs | Adds explicit Java type mapping attribute. |
| src/Lucene.Net/Support/Reflection/LucenePackageMappingAttribute.cs | Adds assembly-level namespace→package mapping attribute. |
| src/Lucene.Net/Support/Reflection/LuceneModifierDifferenceAttribute.cs | Adds modifier-diff suppression attribute. |
| src/Lucene.Net/Support/Reflection/LuceneMavenMappingAttribute.cs | Adds assembly→Maven coordinates mapping attribute. |
| src/Lucene.Net/Support/Reflection/LuceneInterfaceDifferenceAttribute.cs | Adds interface-diff suppression attribute. |
| src/Lucene.Net/Support/Reflection/LuceneBaseTypeDifferenceAttribute.cs | Adds base-type-diff suppression attribute. |
| src/Lucene.Net/Support/Analysis/TokenAttributes/Extensions/CharTermAttributeExtensions.cs | Marks extensions as [NoLuceneEquivalent]. |
| src/Lucene.Net/Search/TotalHitCountCollector.cs | Adds base-type-diff suppression annotation. |
| src/Lucene.Net/Search/TimeLimitingCollector.cs | Adds base-type-diff suppression annotation. |
| src/Lucene.Net/Search/PositiveScoresOnlyCollector.cs | Adds base-type-diff suppression annotation. |
| src/Lucene.Net/Search/MultiCollector.cs | Adds base-type-diff suppression annotation. |
| src/Lucene.Net/Search/FieldComparator.cs | Marks non-generic helper as [NoLuceneEquivalent]. |
| src/Lucene.Net/Search/Collector.cs | Adds explicit mapping + modifier suppression for ICollector; marks helper class as [NoLuceneEquivalent]. |
| src/Lucene.Net/Properties/AssemblyInfo.cs | Adds Maven/package mappings for Lucene.Net core assembly. |
| src/Lucene.Net/Document/ShortDocValuesField.cs | Fixes XML doc cref for DocValues. |
| src/Lucene.Net/Codecs/Compressing/CompressingTermVectorsReader.cs | Adds interface-diff suppression annotation. |
| src/Lucene.Net/Codecs/BlockTreeTermsWriter.cs | Marks helper holder as [NoLuceneEquivalent]. |
| src/Lucene.Net.Tests/Support/Reflection/TestReflectionTypeExtensions.cs | Adds NUnit coverage for reflection inference helpers. |
| src/Lucene.Net.TestFramework/Properties/AssemblyInfo.cs | Adds Maven mapping for test framework. |
| src/Lucene.Net.Suggest/Properties/AssemblyInfo.cs | Adds Maven mapping for suggest module. |
| src/Lucene.Net.Spatial/Properties/AssemblyInfo.cs | Adds Maven + package mapping for spatial module. |
| src/Lucene.Net.Sandbox/Properties/AssemblyInfo.cs | Adds Maven mapping for sandbox module. |
| src/Lucene.Net.Replicator/Properties/AssemblyInfo.cs | Adds Maven mapping for replicator module. |
| src/Lucene.Net.QueryParser/Properties/AssemblyInfo.cs | Adds Maven + package mappings for queryparser module. |
| src/Lucene.Net.Queries/Properties/AssemblyInfo.cs | Adds Maven + package mapping for queries module. |
| src/Lucene.Net.Queries/Function/ValueSources/SingleFunction.cs | Fixes typo + adds [LuceneType] mapping for renamed type. |
| src/Lucene.Net.Misc/Util/Fst/UpToTwoPositiveIntOutputs.cs | Adds explicit [LuceneType] mappings for renamed types. |
| src/Lucene.Net.Misc/Properties/AssemblyInfo.cs | Adds Maven + package mapping for misc module. |
| src/Lucene.Net.Memory/Properties/AssemblyInfo.cs | Adds Maven mapping for memory module. |
| src/Lucene.Net.Join/Properties/AssemblyInfo.cs | Adds Maven mapping for join module. |
| src/Lucene.Net.Highlighter/Properties/AssemblyInfo.cs | Adds Maven mapping for highlighter module. |
| src/Lucene.Net.Grouping/Properties/AssemblyInfo.cs | Adds Maven + package mapping for grouping module. |
| src/Lucene.Net.Grouping/AbstractAllGroupHeadsCollector.cs | Adds explicit mapping for renamed nested type. |
| src/Lucene.Net.Facet/Taxonomy/WriterCache/NameHashIntCacheLRU.cs | Adds explicit [LuceneType] mapping for renamed type. |
| src/Lucene.Net.Facet/Properties/AssemblyInfo.cs | Adds Maven mapping for facet module. |
| src/Lucene.Net.Expressions/Properties/AssemblyInfo.cs | Adds Maven mapping for expressions module. |
| src/Lucene.Net.Demo/Properties/AssemblyInfo.cs | Adds Maven mapping for demo module. |
| src/Lucene.Net.Codecs/Properties/AssemblyInfo.cs | Adds Maven mapping for codecs module. |
| src/Lucene.Net.Classification/Properties/AssemblyInfo.cs | Adds Maven mapping for classification module. |
| src/Lucene.Net.Benchmark/Properties/AssemblyInfo.cs | Adds Maven + package mappings for benchmark module. |
| src/Lucene.Net.Analysis.Stempel/Properties/AssemblyInfo.cs | Adds Maven + package mapping for stempel analyzers. |
| src/Lucene.Net.Analysis.SmartCn/Properties/AssemblyInfo.cs | Adds Maven mapping for smartcn analyzers. |
| src/Lucene.Net.Analysis.Phonetic/Properties/AssemblyInfo.cs | Adds Maven mapping for phonetic analyzers. |
| src/Lucene.Net.Analysis.OpenNLP/Properties/AssemblyInfo.cs | Adds Maven mapping for OpenNLP analyzers. |
| src/Lucene.Net.Analysis.Morfologik/Properties/AssemblyInfo.cs | Adds Maven mapping for morfologik analyzers. |
| src/Lucene.Net.Analysis.Morfologik/Morfologik/TokenAttributes/MorphosyntacticTagsAttributeImpl.cs | Adds explicit [LuceneType] mapping for package mismatch. |
| src/Lucene.Net.Analysis.Morfologik/Morfologik/TokenAttributes/MorphosyntacticTagsAttribute.cs | Adds explicit [LuceneType] mapping for package mismatch. |
| src/Lucene.Net.Analysis.Morfologik/Lucene.Net.Analysis.Morfologik.csproj | Adjusts AssemblyInfo handling for added mappings. |
| src/Lucene.Net.Analysis.Kuromoji/Properties/AssemblyInfo.cs | Adds Maven mapping for kuromoji analyzers. |
| src/Lucene.Net.Analysis.Common/Properties/AssemblyInfo.cs | Adds Maven + package mappings for analysis-common. |
| src/dotnet/Lucene.Net.Tests.ApiCheck/Utilities/DiffUtilityTests.cs | Adds xUnit coverage for type enumeration. |
| src/dotnet/Lucene.Net.Tests.ApiCheck/Lucene.Net.Tests.ApiCheck.csproj | Adds new xUnit-based test project for ApiCheck. |
| src/dotnet/Lucene.Net.Tests.ApiCheck/Extensions/TypeExtensionsTests.cs | Adds xUnit coverage for display-name formatting. |
| src/dotnet/Lucene.Net.Tests.ApiCheck/Comparison/TypeComparisonTests.cs | Adds xUnit coverage for Java↔.NET type matching rules. |
| src/dotnet/Lucene.Net.Tests.ApiCheck/Comparison/ModifierComparisonTests.cs | Adds xUnit coverage for modifier equivalence rules. |
| src/dotnet/Lucene.Net.ApiCheck/Utilities/JarToolIntegration.cs | Adds Java extractor process integration. |
| src/dotnet/Lucene.Net.ApiCheck/Utilities/DiffUtility.cs | Implements main assembly/type/member diff computation. |
| src/dotnet/Lucene.Net.ApiCheck/ReportCommand.cs | Generates HTML report and copies embedded assets. |
| src/dotnet/Lucene.Net.ApiCheck/Report/index.handlebars | Adds Handlebars report template. |
| src/dotnet/Lucene.Net.ApiCheck/Report/index.css | Adds report CSS. |
| src/dotnet/Lucene.Net.ApiCheck/README.md | Documents tool usage and requirements. |
| src/dotnet/Lucene.Net.ApiCheck/Program.cs | Adds CLI entrypoint/commands (diff/report). |
| src/dotnet/Lucene.Net.ApiCheck/Models/MavenCoordinates.cs | Adds Maven coordinate parsing/model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/JavaApi/TypeMetadata.cs | Adds Java API type metadata model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/JavaApi/ParameterMetadata.cs | Adds Java parameter metadata model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/JavaApi/MethodMetadata.cs | Adds Java method metadata model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/JavaApi/LibraryResult.cs | Adds extractor output root model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/JavaApi/Library.cs | Adds (unused) library model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/JavaApi/FieldMetadata.cs | Adds Java field metadata model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/JavaApi/ConstructorMetadata.cs | Adds Java constructor metadata model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Diff/TypeReference.cs | Adds normalized type reference for reporting. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Diff/TypeDiff.cs | Adds per-type diff model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Diff/TypeDeclaration.cs | Adds type declaration model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Diff/PropertyReference.cs | Adds property member reference model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Diff/Parameter.cs | Adds parameter model for members. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Diff/ModifierSet.cs | Adds modifier set wrapper for display. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Diff/MethodReference.cs | Adds method member reference model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Diff/MemberReference.cs | Adds polymorphic member reference base model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Diff/MemberKind.cs | Adds member-kind enum. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Diff/MemberDiff.cs | Adds matched-member diff model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Diff/FieldReference.cs | Adds field member reference model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Diff/ConstructorReference.cs | Adds ctor member reference model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Diff/ComparisonPair.cs | Adds Java/.NET pair wrapper. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Diff/AssemblyDiff.cs | Adds per-assembly diff summary model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Diff/ApiDiffResult.cs | Adds root diff result model + totals. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Config/LibraryConfig.cs | Adds per-library config model. |
| src/dotnet/Lucene.Net.ApiCheck/Models/Config/ApiCheckConfig.cs | Adds root config model. |
| src/dotnet/Lucene.Net.ApiCheck/Lucene.Net.ApiCheck.csproj | Adds new ApiCheck tool project (deps + references + embedded assets). |
| src/dotnet/Lucene.Net.ApiCheck/Json.cs | Adds JSON (de)serialization helper/options. |
| src/dotnet/Lucene.Net.ApiCheck/GlobalOptions.cs | Adds global CLI options + config loader. |
| src/dotnet/Lucene.Net.ApiCheck/Extensions/TypeExtensions.cs | Adds reflection/type formatting + API-surface extraction helpers. |
| src/dotnet/Lucene.Net.ApiCheck/Extensions/StringExtensions.cs | Adds Java type reference formatting helper. |
| src/dotnet/Lucene.Net.ApiCheck/Extensions/PropertyInfoExtensions.cs | Adds property visibility/modifier helpers. |
| src/dotnet/Lucene.Net.ApiCheck/Extensions/MethodInfoExtensions.cs | Adds method modifier extraction helpers. |
| src/dotnet/Lucene.Net.ApiCheck/Extensions/FieldInfoExtensions.cs | Adds field modifier extraction helpers. |
| src/dotnet/Lucene.Net.ApiCheck/Extensions/ConstructorInfoExtensions.cs | Adds ctor modifier extraction helpers. |
| src/dotnet/Lucene.Net.ApiCheck/Extensions/CollectionExtensions.cs | Adds Java/.NET modifier sorting helpers. |
| src/dotnet/Lucene.Net.ApiCheck/DiffCommand.cs | Implements diff command JSON output. |
| src/dotnet/Lucene.Net.ApiCheck/Comparison/TypeComparison.cs | Implements Java↔.NET type matching logic. |
| src/dotnet/Lucene.Net.ApiCheck/Comparison/ModifierComparison.cs | Implements Java↔.NET modifier equivalence rules. |
| src/dotnet/Lucene.Net.ApiCheck/Comparison/MemberComparison.cs | Implements member/property/field/ctor matching logic. |
| Lucene.Net.sln.DotSettings | Updates IDE dictionary entries. |
| Lucene.Net.sln | Adds new projects + solution items. |
| apicheck.ps1 | Adds script to download extractor jar and run report. |
| apicheck-config.jsonc | Adds library/dependency configuration for API checks. |
| .gitignore | Adds ignore patterns (notably _artifacts/, .idea, etc.). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a comparison tool for comparing Lucene.NET's API surface to Lucene.
Fixes #1022
Description
After several months of effort, I present this PR for an API comparison tool against Lucene!
This tool allows you to extract the API surface of a corresponding version of Lucene for each library, compare it to the API surface of Lucene.NET, and generate a report in either JSON or HTML format. Many conventions were built-in to the diff tool to handle Java vs .NET idioms and casing differences. There are naturally cases where things don't match up exactly by convention, so this also supports adding attributes and finding them via reflection to handle discrepancies/suppressions.
Note that it is NOT a goal of this PR to make all suppressions; that would make the PR too large. We should analyze the discrepancies and create separate issues to do that work. I did add a few to the codebase to demonstrate their usage and go ahead and make a small dent in reducing the number of discrepancies, but this is not intended to be an exhaustive pass at this time.
The Java API surface is extracted using an Apache-licensed utility I created here: https://github.com/paulirwin/java-api-extractor
I originally was going to contribute this tool to this repo, but we realized that it would have usefulness with other projects like ICU4N that we depend on, and would be easier to use if it is a separate project. The separate project only handles extracting the Java API surface of specified Maven packages and exporting that to JSON; the remainder of the work of looking at Lucene.NET's API surface and doing the diff lives in this PR.
To run the report, run
./apicheck.ps1in PowerShell from the repo root. You must have Java 21 or later, and the GitHubghCLI (to pull the java-api-extractor release jar), installed. (Note that this tool is only intended to be used by Lucene.NET contributors, so those requirements seem reasonable.) This will generate both a JSON diff report (useful for programmatic analysis, or use by AI agents viajq), as well as an HTML report via Handlebars.Net. The HTML report is the easiest for human consumption, and gives you a summary at the top. Here is the current status of discrepancies:Note that the vast majority of those, from spot checking them, appear to be discrepancies that should be suppressed via attributes. However, there might be lurking in there legitimate missed ports (hence the goal of #1022).
The
apicheck-config.jsoncfile in the root of the repo tells the API check utility which libraries to analyze, and optionally which Maven packages are needed in order for java-api-extractor to successfully extract the API (typically when those third-party types are exposed in the public API). All libraries and their Maven packages where applicable have already been added to this file for analysis. This file is a.jsoncfile so that we can add comments (and this file extension makes the editor typically not complain about invalid JSON) which was helpful during the development process.There are various attributes added to the project to handle mappings or to suppress findings. These include:
[LuceneMavenMapping(groupId, artifactId, version)]- assembly-level attribute to specify the Maven coordinates for the equivalent Lucene library. I've already added these to each project where possible.[LucenePackageMapping(dotNetNamespace, javaPackage)]- maps a .NET namespace to a Java package, where it can't be mapped automatically by convention. This has been added already for all cases that I found.[NoLuceneEquivalent]- there is no Lucene equivalent for this type/member, and is thus Lucene.NET-specific[LuceneType(packageName, typeName)]- maps a single type whose name can't be inferred to a specific type in the given Java package[LuceneBaseTypeDifference]- the base type does not match (or one side is missing one) between Java and .NET[LuceneInterfaceDifference]- the set of interfaces does not match between Java and .NET. Note that this suppresses all interface differences for the type, and should only be added after carefully ensuring that we're not missing something that needs to be done in Lucene.NET.[LuceneModifierDifference]- the modifiers (i.e.public,final, etc.) of the type or member do not match between Java and .NET.Note that all but
LuceneMavenMappinghave an optional Justification argument that can be added to provide the justification where it's not obvious.In the future, we might add first-class support for detecting these attributes to our docs site generation.
Notes for PR Reviewers
The following things are NOT in scope for this PR, and should not be considered for review (as they can be done separately):
Due to the size of this PR already, please do not add comments for new features we could add or otherwise any things that can be done separately. Instead, please file new issues for them, mentioning issue #1022. I also kindly request that any subjective/style/bikeshedding concerns, or any other small/insignificant changes, are either committed directly here by any other maintainers to update this PR instead of commenting, or split out as a separate issue for follow-up later. Please focus on functionality correctness and maintainability issues only.