[docs] Add math and conformance to homepage (#2599)

Author: patrick-ogrady

docs/blogs/adb-any.html

@@ -87,7 +87,7 @@ <h1>The Simplest Database You Need</h1>
             <img src="/imgs/mmr.png" alt="Appending new elements to an MMR">
             <div class="image-caption">Figure 1: Appending new elements to an MMR</div>
         </div>
-        <p>Here, we'll dive into how the construction can be extended to create a lightweight <i>authenticated database</i> (ADB) capable of storing and proving mutable blockchain state with minimal amplification. We call the first of these MMR-powered ADBs <a href="https://docs.rs/commonware-storage/latest/commonware_storage/adb/any/index.html">adb::any</a>, and are excited to share that it is now available under an MIT/Apache-2 license.</p>
+        <p>Here, we'll dive into how the construction can be extended to create a lightweight <i>authenticated database</i> (ADB) capable of storing and proving mutable blockchain state with minimal amplification. We call the first of these MMR-powered ADBs <a href="https://docs.rs/commonware-storage/latest/commonware_storage/qmdb/any/index.html">adb::any</a>, and are excited to share that it is now available under an MIT/Apache-2 license.</p>
         <p>The primary limitation of the MMR compared to a more typical Merkle tree structure is its lack of support for removing elements from the list over which inclusion proofs can be provided. This restriction, at first glance, might appear to prohibit an MMR from being used to build a mutable database where keys take on values that change over time. However, append-only MMRs are a natural fit for authenticating log data. Instead of viewing our database as only a snapshot in time, the basic idea we'll build on is to maintain an MMR over the <i>historical log</i> of all database <i>operations</i> performed up until the present state.</p>
         <p>Let us consider two basic operations, <i>assigning</i> a key a value and <i>deleting</i> a key's value. Each operation applied to the database gets added to a persistent log, and its digest added to an MMR. To provide efficient lookups of a key's value, we employ a <a href="https://docs.rs/commonware-storage/latest/commonware_storage/index/index.html">memory-efficient index</a> that maps a compressed representation of each key to its latest value in the log.</p>
         <p>The figure below depicts a small example of this kind of ADB, showing only the updates for key “foo” which is assigned value 5 in log operation 3, deleted at operation 6, and later updated to value 6 at operation 9, which contains its active state. While the contents of the log are durably persisted to disk, the index depicted to the right is memory resident, mapping a key to the location of its currently active operation. Because keys in some applications may be large (e.g. 32 byte hashes), keys are transformed by a user-provided function into a more compact representation (e.g. an 8 byte hash prefix) for memory efficiency. (There is some additional complexity we don't describe here required for handling a small but non-negligible chance of collisions among these transformed keys.)</p>
@@ -96,10 +96,10 @@ <h1>The Simplest Database You Need</h1>
             <div class="image-caption">Figure 2: Appending new elements to adb::any</div>
         </div>
         <p>One challenge with this solution is the endlessly growing log of data which needs to be replayed to derive the current state of the database. Consider the MMR in the diagram above: it stores the old value for key "foo" in operation 3 even though its value is later reassigned in operations 6 and 9. To overcome this, we apply a <i>compaction</i> process that advances the inactivity floor, the index before which all operations are <i>inactive</i> (and therefore don't need to be stored to compute the current state). Starting from the old inactivity floor, we advance over the next n operations, where n is configurable. If the operation is already inactive, we simply drop the data. If it's active, we make it inactive by reapplying the same key-value in a new operation. This process guarantees that the number of inactive operations remaining ahead of the floor is never more than a small constant multiple of the number of active operations. Operations preceding the inactivity floor can be efficiently discarded from disk while still retaining the ability to authenticate any historical operation against the current root. Much as the log supports re-derivation of the database's state, it also allows us to re-derive the state of the pruned MMR as long as we durably persist (aka <i>pin</i>) a small (logarithmically sized) set of historical nodes.</p>
-        <p>A variety of designs are possible within this general framework for an authenticated database, each involving differing trade-offs between performance, memory efficiency, and authentication expressivity. Our initial release, <a href="https://docs.rs/commonware-storage/latest/commonware_storage/adb/any/index.html">adb::any</a>, implements the minimal construction described above over the existing <a href="https://docs.rs/commonware-storage/latest/commonware_storage/mmr/journaled/index.html">mmr::journaled</a> primitive. To limit the IO of both updates and deletes to a minimum, adb::any <b>only</b> supports proving that some key was assigned to a specific value at some point in its history (not that a key has some value now nor that a key has no value). This (lack of capability) achieves a unique balance between performance and functionality that we believe applications will find just useful enough. For example, adb::any can be used to power efficient state sync over the active range (indicated by the inactivity floor), drive message-based interoperability (where emitted messages are forever persisted), and equip users to verify the balance served to them by centralized infrastructure (constraining any future proof to be at an index higher than the last).</p>
+        <p>A variety of designs are possible within this general framework for an authenticated database, each involving differing trade-offs between performance, memory efficiency, and authentication expressivity. Our initial release, <a href="https://docs.rs/commonware-storage/latest/commonware_storage/qmdb/any/index.html">adb::any</a>, implements the minimal construction described above over the existing <a href="https://docs.rs/commonware-storage/latest/commonware_storage/mmr/journaled/index.html">mmr::journaled</a> primitive. To limit the IO of both updates and deletes to a minimum, adb::any <b>only</b> supports proving that some key was assigned to a specific value at some point in its history (not that a key has some value now nor that a key has no value). This (lack of capability) achieves a unique balance between performance and functionality that we believe applications will find just useful enough. For example, adb::any can be used to power efficient state sync over the active range (indicated by the inactivity floor), drive message-based interoperability (where emitted messages are forever persisted), and equip users to verify the balance served to them by centralized infrastructure (constraining any future proof to be at an index higher than the last).</p>
         <p>Another variant, which remains under development, augments this construction with <a href="https://docs.rs/commonware-storage/latest/commonware_storage/mmr/bitmap/index.html">an authenticated bitmap</a> over the activity status of each operation. This extension allows the ADB to additionally authenticate that a specific value is a key's <i>current</i> value – useful for applications requiring, for example, proving someone's account contains some balance, or that a user currently has the permission to execute some contract. <a href="https://arxiv.org/abs/2501.05262">QMDB</a>, the primary inspiration for this work, extends these concepts even further, adding the ability to prove <i>exclusion</i> (that some key is not currently assigned any value).</p>
         <p>In addition to <i>assign</i> and <i>delete</i> over keys, our implementation offers one additional database operation, <i>commit</i>, that synchronizes the compaction process with each batch update. In the context of a blockchain, the set of updates from executing all transactions in a block can be atomically committed to the database along with raising the inactivity floor a number of times necessary to prevent it from falling indefinitely behind (a useful marker in the log to inform state syncing clients where to start fetching from the log). This compaction process could be extended to charge any rewritten (still active) account at the inactivity floor some rent or to drop said accounts altogether from “hot” storage (requiring a proof to resurrect when needed again).</p>
-        <p><a href="https://docs.rs/commonware-storage/latest/commonware_storage/adb/any/index.html">adb::any</a> is available today with work ongoing to deliver both additional feature variants and a benchmarking suite to evaluate the impact of said variants. Like other primitives, adb::any is exhaustively tested with <a href="https://docs.rs/commonware-runtime/latest/commonware_runtime/deterministic/index.html">runtime::deterministic</a> over a number of failure scenarios (like unclean shutdown).</p>
+        <p><a href="https://docs.rs/commonware-storage/latest/commonware_storage/qmdb/any/index.html">adb::any</a> is available today with work ongoing to deliver both additional feature variants and a benchmarking suite to evaluate the impact of said variants. Like other primitives, adb::any is exhaustively tested with <a href="https://docs.rs/commonware-runtime/latest/commonware_runtime/deterministic/index.html">runtime::deterministic</a> over a number of failure scenarios (like unclean shutdown).</p>
         <p>We look forward to releasing a production-ready version of adb::any later this year.</p>
     </div>
 

docs/blogs/only-time-will-tell.html

@@ -115,7 +115,7 @@ <h2>BATTLEWARE: Proving BTLE is Practical</h2>
         <p>The entire implementation is <a href="https://github.com/commonwarexyz/battleware">open source</a> under both an MIT and Apache-2 license. The code is still a bit rough around the edges, but we were able to reach 77% test coverage (on ~11.2k lines of code). Consider it your opportunity to become a contributor.</p>
         <h2>Securing Player Access</h2>
         <p>Instead of trusting our API to provide access to game state, we opted to build a pipeline to provide visitors with authenticated access (like we did with <a href="https://alto.commonware.xyz">alto</a>). The big difference here being that BATTLEWARE players need to verify state and events against consensus outputs (rather than just consensus outputs directly).</p>
-        <p>To accomplish this, we inject state changes and execution events into our <a href="https://docs.rs/commonware-storage/latest/commonware_storage/adb/index.html">ADBs</a>. We then use <a href="https://docs.rs/commonware-consensus/latest/commonware_consensus/aggregation/index.html">consensus::aggregation</a> to generate a threshold signature over the computed roots of each ADB. Because execution relies on the output of a VRF not available during verification, we cannot just use the threshold signature emitted by threshold-simplex (unfortunately).</p>
+        <p>To accomplish this, we inject state changes and execution events into our <a href="https://docs.rs/commonware-storage/latest/commonware_storage/qmdb/index.html">ADBs</a>. We then use <a href="https://docs.rs/commonware-consensus/latest/commonware_consensus/aggregation/index.html">consensus::aggregation</a> to generate a threshold signature over the computed roots of each ADB. Because execution relies on the output of a VRF not available during verification, we cannot just use the threshold signature emitted by threshold-simplex (unfortunately).</p>
         <div class="image-container">
             <img src="/imgs/threshold-roots.png" alt="MMR state management">
             <div class="image-caption">Figure 5: After execution, the State ADB and Events ADB is updated and once wrapped by a threshold signature are pushed to Exoware.</div>

docs/index.html

@@ -82,12 +82,16 @@ <h3><a href="https://docs.rs/commonware-coding">coding</a></h3>
         <p>Encode data to enable recovery from a subset of fragments.</p>
         <h3><a href="https://docs.rs/commonware-collector">collector</a></h3>
         <p>Collect responses to committable requests.</p>
+        <h3><a href="https://docs.rs/commonware-conformance">conformance</a></h3>
+        <p>Automatically assert the stability of encoding and mechanisms over time.</p>
         <h3><a href="https://docs.rs/commonware-consensus">consensus</a></h3>
         <p>Order opaque messages in a Byzantine environment.</p>
         <h3><a href="https://docs.rs/commonware-cryptography">cryptography</a></h3>
         <p>Generate keys, sign arbitrary messages, and deterministically verify signatures.</p>
         <h3><a href="https://docs.rs/commonware-deployer">deployer</a></h3>
         <p>Deploy infrastructure across cloud providers.</p>
+        <h3><a href="https://docs.rs/commonware-math">math</a></h3>
+        <p>Create and manipulate mathematical objects.</p>
         <h3><a href="https://docs.rs/commonware-p2p">p2p</a></h3>
         <p>Communicate with authenticated peers over encrypted connections.</p>
         <h3><a href="https://docs.rs/commonware-resolver">resolver</a></h3>

examples/sync/README.md

@@ -2,7 +2,7 @@
 
  [![Crates.io](https://img.shields.io/crates/v/commonware-sync.svg)](https://crates.io/crates/commonware-sync)
 
-Continuously synchronize state between a server and client with [qmdb::any::Any](https://docs.rs/commonware-storage/latest/commonware_storage/adb/any/struct.Any.html) and [qmdb::immutable::Immutable](https://docs.rs/commonware-storage/latest/commonware_storage/adb/immutable/struct.Immutable.html)
+Continuously synchronize state between a server and client with [qmdb::any::Any](https://docs.rs/commonware-storage/latest/commonware_storage/qmdb/any/struct.Any.html) and [qmdb::immutable::Immutable](https://docs.rs/commonware-storage/latest/commonware_storage/qmdb/immutable/struct.Immutable.html)
 
 ## Components