Request for review

[dpriskorn]

Hi 👋,

I’m working on a project called wikibase-backend — a clean-room, billion-scale Wikibase backend architecture based on immutable S3 snapshots and Vitess indexing:

👉 https://github.com/dpriskorn/wikibase-backend/

I would greatly appreciate your review, input, and guidance on the project from the perspective of the Scholia community and its use of Wikidata/Wikibase tooling.

---

Project Overview

wikibase-backend explores an alternative Wikibase backend architecture with a focus on:

• Immutable entity storage (S3 / object storage)
• High-throughput metadata and indexing via Vitess
• Strong auditability through immutable snapshots
• Decoupling MediaWiki from core storage concerns
• Scalability targets up to billions of entities

The repository primarily contains architecture documentation, conceptual models, and early implementation ideas rather than a drop-in replacement.

---

Feedback I’m Looking For

I would especially value your thoughts on:

1. Architecture & Design

• Does this approach make sense for alternative Wikibase backends?
• Are there known architectural pitfalls or constraints I should consider?

2. Compatibility with Scholia & Wikidata Tooling

• Potential issues or opportunities when consuming data from this backend in Scholia
• Assumptions Scholia makes today that might conflict with this design
• Ideas to improve interoperability with existing Wikidata-based tools

---

Current Status

• Architecture and design documentation is in place
• Consistency and write-flow strategies are described
• Some code and schemas exist, but this is not yet a production-ready system
• Interoperability testing with consumer tools (like Scholia) has not yet been done, but it shouldn't affect the SPARQL engine that consumes the RDF change stream at all.

Thank you very much for your time and expertise — any feedback would be extremely valuable 🙏