Solution

.github/workflows/testing_action.yml

@@ -0,0 +1,25 @@
+name: Run Tests
+
+on:
+  pull_request:
+    branches:
+      - main
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v2
+
+      - name: Set up Node.js
+        uses: actions/setup-node@v2
+        with:
+          node-version: "14"
+
+      - name: Install dependencies
+        run: yarn install
+
+      - name: Run tests
+        run: yarn test

.gitignore

@@ -0,0 +1 @@
+node_modules

README.md

@@ -1,2 +1,55 @@
-# aztec-merkle-tree
-Aztec challenge: building a merkle tree.
+# Aztec Technical Challenge
+
+The test provides you an opportunity to demonstrate the following:
+
+- Your ability to write a data structure algorithm (in this case a merkle tree).
+- Your ability to write clean, idiomatic TypeScript.
+
+## Rationale
+
+A core data structure in the Aztec system is the merkle tree. It's a simple binary tree structure where the root node is represented by the hash of its two child hashes. Given any set of data in the leaves, this leads to a unique root. Furthermore, proof of existence of a piece of data can be represented by a hash path, a list of pairwise child hashes at each layer, from leaf to root. Aztec stores all of its notes in such data structures, and when proofs are generated they use hash paths to prove the data they are modifying exists.
+
+In this test you will be working on an implementation of a merkle tree.
+
+## Merkle Tree Structure
+
+- The merkle tree is of depth `32`, and is fully formed with leaves consisting of `64` zero bytes at every index.
+- When inserting an element of arbitrary length, the value must first be `hash`ed to `32` bytes using sha256.
+- Each node of the tree is computed by `compress`ing its left and right subtree hashes and taking the resulting sha256 hash.
+- For reference, an unpopulated merkle tree will have a root hash of `1c9a7e5ff1cf48b4ad1582d3f4e4a1004f3b20d8c5a2b71387a4254ad933ebc5`.
+
+The merkle tree is to be persisted in a key value store. `LevelUp` provides the basic key value store interface.
+
+## Building and Running
+
+After cloning the repo:
+
+```bash
+yarn install
+
+# To run all tests.
+yarn test
+
+# To run tests, watching for changes.
+yarn test --watch
+```
+### [WIP] Thinking Process
+
+Mantra: 1st make it work, 2nd make it right, 3rd make it fast.
+
+1st Attempt: Create one big buffer to store the entire tree and use array indexes for fast navigation. Problem: The tree was too large to fit in memory. Additionally, I realized I could only modify specific parts of the code and understood the necessity of using the DB.
+
+2nd Attempt: Use the configured DB and perform operations recursively. Problem: While specific changes were efficient (log2n), tree initialization caused max heap errors. I realized that since the root was deterministic, the entire tree was deterministic and could be built sequentially.
+
+3rd Attempt: Initialize the tree sequentially by calculating only necessary parts. For updates and path calculations, I kept recursion. The tests passed successfully.
+
+Other Problems I Encountered:
+- Infinite Loop: A bug in the update logic caused incorrect child settings, leading to infinite loops due to repeated values. Debugging this was challenging.
+- ShouldGoRight: Determining the correct path at each level was difficult. I found a formula online that worked, which involved traversing the tree from root to leaf and checking if the bit was 1 or 0.
+- Debugging: For multiple bugs, it was faster to create temporary functions to print the tree and paths for debugging. These functions were removed after tests passed for cleaner code.
+
+Future Work:
+- Add extensive tests. Multiple internal functions with logic need unit testing.
+- Reorganize the files: Separate the DB, tree, utilities, and tests into different files.
+
+Author: x.com/mgrabina

package.json

@@ -30,5 +30,6 @@
     "jest": "^24.8.0",
     "prettier": "^2.0.5",
     "ts-jest": "^24.0.2"
-  }
+  },
+  "packageManager": "[email protected]+sha512.84bec57763ecbf87ce884fc98d550b0e0f8b7241b4425ec83d4879c160bfe1412be96f9e17e5551167d1bba43ba9b3a7aa4efea98ec25d84922c17c92321255c"
 }

src/merkle_tree.ts

@@ -28,7 +28,43 @@ export class MerkleTree {
       throw Error('Bad depth');
     }
 
-    // Implement.
+    if (root) {
+      // Restore already saved tree state.
+      root.copy(this.root);
+    } else {
+      // Since initialization with zeros is deterministic, we do it sequentially for enhanced performance.
+      const hashPerLevel: Buffer[] = [];
+      for (let i = depth; i >= 0; i--) {
+        // Get deterministic hash for current level
+        if (this.depth === i) {
+          // Leaf nodes are initialized with zeros and hash is calculated from it.
+          const buffer = Buffer.alloc(64, 0);
+          const hash = this.hasher.hash(buffer);
+          hashPerLevel[i] = hash;
+        } else {
+          // For internal nodes, hash is calculated from children.
+          if (!hashPerLevel[i + 1]) {
+            throw new Error("Hash not found for level " + (i + 1));
+          }
+
+          const childHash = hashPerLevel[i + 1];
+          const combinedHash = this.hasher.compress(
+            childHash, // left
+            childHash // right
+          );
+          hashPerLevel[i] = combinedHash;
+
+          const levelBuffer = Buffer.alloc(64);
+          childHash.copy(levelBuffer, 0);
+          childHash.copy(levelBuffer, 32);
+
+          // Save nodes, just one instance per level needed.
+          this.db.put(combinedHash, levelBuffer);
+        }
+      }
+
+      hashPerLevel[0].copy(this.root); // Copy the new root hash.
+    }
   }
 
   /**
@@ -72,15 +108,72 @@ export class MerkleTree {
    *     d3:   [ ]         [ ]          [*]         [*]           [ ]         [ ]          [ ]        [ ]
    */
   async getHashPath(index: number) {
-    // Implement.
-    return new HashPath();
+    const getHashPathRecursively = async (
+      current: Buffer,
+      depth: number
+    ): Promise<Buffer[][]> => {
+      // DB values are stored as [left, right] pairs asigned to the parent hash.
+      const fromDb: Buffer = await this.db.get(current);
+      const leftHash = fromDb.slice(0, 32);
+      const rightHash = fromDb.slice(32, 64);
+
+      if (this.depth === depth + 1) {
+        // If next level is leaf, we add both children to the beginning of the path and return.
+        return [[leftHash, rightHash]];
+      }
+
+      const shouldGoRight = (index >> (this.depth - depth - 1)) & 1; // 1 if right, 0 if left. This works because we are traversing the tree from the root to the leaf.
+      const accPath = await getHashPathRecursively(
+        shouldGoRight ? rightHash : leftHash,
+        depth + 1
+      );
+
+      accPath.push([leftHash, rightHash]);
+
+      return accPath;
+    };
+
+    return new HashPath(await getHashPathRecursively(this.root, 0));
   }
 
   /**
    * Updates the tree with `value` at `index`. Returns the new tree root.
    */
   async updateElement(index: number, value: Buffer) {
-    // Implement.
+    const updateRecursively = async (
+      current: Buffer,
+      depth: number
+    ): Promise<Buffer> => {
+      if (this.depth === depth) {
+        // Leaf node, hash the value and return.
+        return this.hasher.hash(value);
+      }
+
+      const fromDb: Buffer = await this.db.get(current);
+      const shouldGoRight = (index >> (this.depth - depth - 1)) & 1; // 1 if right, 0 if left. This works because we are traversing the tree from the root to the leaf.
+
+      const leftHash = shouldGoRight
+        ? fromDb.slice(0, 32)
+        : await updateRecursively(fromDb.slice(0, 32), depth + 1);
+      const rightHash = shouldGoRight
+        ? await updateRecursively(fromDb.slice(32, 64), depth + 1)
+        : fromDb.slice(32, 64);
+
+      // Save new internal node. Hash of children is calculated and saved.
+      const newHash = this.hasher.compress(leftHash, rightHash);
+      const newChildrenBuffer = Buffer.alloc(64);
+      leftHash.copy(newChildrenBuffer, 0);
+      rightHash.copy(newChildrenBuffer, 32);
+      this.db.put(newHash, newChildrenBuffer);
+
+
+      return newHash;
+    };
+
+    (await updateRecursively(this.root, 0)).copy(this.root); // Update the root.
+
+    await this.writeMetaData(); // Persist the new root for future restores.
+
     return this.root;
   }
 }