Feature Request: Blue-Green Index Rebuilding

[tigreye007]

Written with a little help from my friend Claude

Is your feature request related to a problem? Please describe.

When a reindex operation is triggered and fails to complete (due to timeouts, exceptions, or resource constraints), the index is left in an incomplete state. This results in incomplete search results with no notification or warning that the rebuild failed. Users continue querying a partial index without knowing there's a problem.

Describe the solution you'd like

Implement a blue-green indexing strategy where:

1. A new index is built in a separate location while the existing index continues serving queries
2. The new index is validated upon completion (document count > 0, index integrity check)
3. Only after successful validation does an atomic swap occur, making the new index active
4. If the rebuild fails for any reason, the existing index remains intact and active
5. Rebuild status (success/failure, document count, duration) is logged to the Event Log

Describe alternatives you've considered

• Custom implementation: Wrapping the existing ILuceneClient with a custom blue-green manager that maintains two index slots and handles the swap logic. This works but requires forking or significantly extending the library.
• External monitoring: Adding health checks that compare expected vs. actual document counts post-rebuild, then alerting on mismatches. This detects the problem but doesn't prevent it.
• Retry logic: Implementing automatic retry on rebuild failure. This helps with transient issues but doesn't protect against partial indexes during the rebuild window.

Additional context

This is especially important for SaaS deployments where indexes must be rebuilt after each deployment. A failed rebuild during deployment currently leaves the site with broken search until someone notices and manually triggers another rebuild.

Related: The auto-reindexing documentation acknowledges indexes are lost on deployment but doesn't address the incomplete rebuild scenario.

[phixxo]

Hello @tigreye007 ,

This sounds very similar to the issue discussed here which also links some of the previous ones. The team is looking into this already.

[phixxo]

Hi again,
I just got some news from the team. They are already working on improving this - as described in the other, linked issues. Hopefully everything goes well and the improvements will be released in the near future