Add client-side async API for replication (IAsyncReplicator)

[NehanPathan]

Fixes:
Fixes #1181

Summary of the changes:
Implemented IAsyncReplicator for non-blocking replication operations.

---

Description

This PR introduces an async Task-based API for the replication support, allowing operations such as checking for updates, obtaining files, and releasing sessions to be executed asynchronously.

Key changes:

• Added IAsyncReplicator interface.
• Updated HttpReplicator to implement async versions of the operations (CheckForUpdateAsync, ObtainFileAsync, ReleaseAsync, PublishAsync).

This avoids synchronous HTTP calls that could deadlock or cause performance issues, while keeping IReplicationHandler synchronous, as Lucene.NET APIs currently do not have async equivalents.

Additional context:
This implementation has been tested and works successfully when using UpdateNowAsync in the GSoC project by referencing the Lucene.NET repository in the GSoC extensions project.

---

UPDATE by @paulirwin 6/24/2026: Async support for ReplicationClient has been removed, per discussion. We will reintroduce it separately in the Extensions repo/packages.

[paulirwin]

Thanks for the PR! I have some changes that need to be made, and a larger one about a refactoring that is up for discussion. Please share your thoughts on the refactoring before embarking on it!

[NehanPathan]

@paulirwin @NightOwl888
I’ve tried to cover most of the suggested changes, and the nullable warnings have also been resolved.
Please let me know if I made any mistakes — I apologize in advance, as there were quite a few nullable warnings to address.

[NehanPathan]

@NightOwl888
2 weeks ago, I accidentally merged the master branch into feature/async-replicator-client.
I’ve performed a force push on this branch to remove the merge commit from apache:master.

Reason: The merge brought in changes from master that we do not want in this feature branch. All intended feature commits are still intact.

This keeps the branch history clean and focused on the feature work.

If any changes from master are important to include, please let me know, and we can merge them properly.

[NehanPathan]

I’m seeing the check-editorconfig CI fail due to a “final newline expected” error. Locally, git diff --check doesn’t show any issues, so I suspect it might be related to line endings (CRLF vs LF) or some subtle trailing whitespace.

Could you please guide me on the safest way to fix this, such as how to identify files with extra trailing whitespace or missing newlines, so the CI passes without affecting other files?

[NightOwl888]

@NehanPathan

That error means that after all of the content in the file, there is no newline character. So, it just needs to be added.

I know it is a bit confusing - remove all trailing whitespace except at the end of the file, add a line break.

[paulirwin]

I have rebased this PR on latest master. Had to do #1221 to fix the build error due to an editorconfig-checker update. I'll look at resolving the remaining comments.

[Copilot]

Pull request overview

This PR adds a Task-based async API for client-side replication to avoid blocking/synchronous HTTP calls, while keeping the existing synchronous replication flow intact.

Changes:

• Introduces IAsyncReplicator and async counterparts to core replication operations (check/update, obtain file, release).
• Updates HttpReplicator/HttpClientBase with async HTTP helpers and stream-handling utilities.
• Extends ReplicationClient with UpdateNowAsync() and a periodic async update loop, plus nullability annotations across replicator-related APIs.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
src/Lucene.Net/Util/IOUtils.cs	Adjusts nullability for rethrow helpers (and minor doc text).
src/Lucene.Net.Tests.Replicator/Http/HttpReplicatorTest.cs	Adds async tests for client update and server error scenarios.
src/Lucene.Net.Replicator/Replicator.cs	Updates IReplicator.CheckForUpdate nullability annotations.
src/Lucene.Net.Replicator/ReplicationClient.cs	Adds async update APIs/loop and refactors locking to support async execution.
src/Lucene.Net.Replicator/IndexReplicationHandler.cs	Applies nullable reference annotations and modernizes string formatting.
src/Lucene.Net.Replicator/IndexAndTaxonomyReplicationHandler.cs	Applies nullable reference annotations and improves error formatting/guards.
src/Lucene.Net.Replicator/IAsyncReplicator.cs	Adds the new async replicator interface contract.
src/Lucene.Net.Replicator/Http/HttpReplicator.cs	Implements IAsyncReplicator with async HTTP request/stream handling.
src/Lucene.Net.Replicator/Http/HttpClientBase.cs	Adds async request helpers and response-stream ownership utilities.
Directory.Build.targets	Adds feature defines and formatting adjustments for framework-conditional compilation.

Comments suppressed due to low confidence (1)

src/Lucene.Net.Replicator/ReplicationClient.cs:528

• The comment in StopUpdateThread() says signaling stop will terminate the thread if it is “awaiting the lock”, but the stop event is only waited on during the sleep (stop.Wait(timeout)), not during updateLock.Wait(). With SemaphoreSlim, signaling stop won’t unblock a thread waiting to acquire updateLock. Please update the comment to reflect the actual behavior (or add logic to check stop before/while waiting on updateLock if that was the intent).

                    // this will trigger the thread to terminate if it awaits the lock.
                    // otherwise, if it's in the middle of replication, we wait for it to
                    // stop.
                    updateThread.stop.Signal();
                    try

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[paulirwin]

@NehanPathan I'll review Copilot's feedback here (as well as run it through a few other models) and will update the PR with any needed changes. We'll proceed with my latest commit for the constructor approach.

[NehanPathan]

@paulirwin
Thanks for taking this forward....

I’ll start through the Copilot comments (especially around async flow, disposal, and streaming), and will align with your direction on the constructor approach.

Let me know if there are specific areas you’d prefer I focus on.

[paulirwin]

I picked this back up again, trying to finish it up and ensure all the Copilot feedback is addressed properly, and discovered several likely bugs, particularly around synchronization (reentrant vs non-reentrant primitives, async deadlocks, disposal, task completion exceptions, etc). Additionally, "doing this right" while trying to adhere to some of the design principles discussed here (i.e. having sync and async versions in the same class) is going to result in a maintenance burden, I'm afraid, because it can't simply be done the same way "but just made async."

For example, the public synchronized void close() in Java calls public synchronized void stopUpdateThread(), which is allowed because Java's synchronized keyword is reentrant. Our UninterruptableMonitor is reentrant too, but it can't be used around an await since it might resume on a different thread. So now our Dispose/StopAsyncUpdateLoop methods have to dance around this and it seems from even the most generous of readings that this code is bug-prone now. Not to mention having to consider thread interrupt stuff.

My current thinking has gone back to thinking that we would actually be better served by splitting out a separate AsyncReplicationClient class, or even moving that support to the extensions project as a Microsoft.Extensions.Hosting BackgroundService. That way we can keep ReplicationClient very close to upstream (with no breaking changes for users), still support IAsyncReplicator in our Replicator library and its implementation in HttpReplicator as primitives for others to use if needed, but allow for our async-based ReplicationClient implementation to follow a different path.

I could go either way on whether to make it another class in our Replicator library, or to move to Extensions. Pros for each:

• Option A: Separate Class in Lucene.Net.Replicator:
  • Can see implementations side by side, and possibly even refactor out a common base class for some things
  • We can note in comments that future porting efforts need to update both classes
  • Will be easier to port upstream changes to each if they both look similar to upstream's ReplicationClient (just one with async-colored glasses)
• Option B: Move to Extensions:
  • Can use Microsoft.Extensions.Hosting's BackgroundService for a more modern-.NET idiomatic approach to running code in a background loop, which would likely result in significantly less code (no need to synchronize start/stop state, etc)
  • Can diverge from upstream and follow its own path
  • Supports Microsoft.Extensions.DependencyInjection
  • Would pair well with @NehanPathan's other extensions work (I still need to finish merging that btw)

And to clarify and emphasize what I said above: IAsyncReplicator and HttpReplicator's implementation of it would remain in this PR if we choose option B. I'm currently leaning towards that, but open to option A too if there's a strong consensus for that.

Also, it should be noted, async support for a replication client was primarily attempted out of consistency with the rest of our async support for replication. It's typically not strictly necessary, since you don't need to worry about hogging a thread for a background job which likely just has one thread anyways. Since this is proving to be more complex, difficult, and disruptive to future porting efforts than anticipated, I think we can pivot to Option B which would allow for a more modern IHostedService approach, still support the asynchronous code, and keep it out of the way for future porting efforts and bug fixes.

@NightOwl888 @NehanPathan @Shazwazza Please let me know your thoughts.

[NehanPathan]

@paulirwin
Thanks for digging deeper into this and explaining the concerns in more detail....

After thinking through the synchronization and lifecycle side more carefully, the new direction makes more sense to me as well.

Initially I was mostly looking at it from the perspective of “making async work” inside the existing ReplicationClient, and technically a lot of the locking issues can be worked around using approaches like SemaphoreSlim, avoiding await inside monitor locks, restructuring cleanup flow, etc. But the more I think about it, the more it feels like the complexity starts growing around the overall lifecycle management rather than any single async call itself.

Especially with things like:

• coordinating sync + async update loops
• disposal and cancellation interactions
• task completion/error handling
• synchronization assumptions from the original Java design
• and future maintenance/porting concerns

Even if we solve those correctly now, it still adds quite a bit of long-term complexity to a class that was originally designed around a synchronous polling/thread model.

Keeping IAsyncReplicator and the async support in HttpReplicator as reusable primitives while moving the hosted/background async behavior into Extensions using something like BackgroundService / IHostedService feels like a cleaner separation to me now as well. It also keeps the core replication library closer to upstream and probably makes future porting easier.

I also think this could give us more flexibility if we eventually decide to evolve beyond polling-based replication loops toward more event-driven approaches without complicating the core replication client further.

So overall I’m aligned with the new direction, and I’m happy to help with whichever approach we decide to move forward with.

[paulirwin]

I removed Async support from ReplicationClient per the discussion in the PR. We will move that to the Extensions repo, perhaps as a BackgroundService.

[NehanPathan]

@paulirwin
I reviewed the latest changes and they look good to me.

The updated architecture matches the direction we discussed earlier, with IAsyncReplicator remaining in the core library while the async replication client is moved out to the Extensions project. The cleanup around removing the async ReplicationClient implementation looks complete, and I didn't notice any dangling references or inconsistencies after the refactoring.
Thanks....

Feedback addressed, and re-reviewed by NehanPathan