|
1 | | -# Spark2: Expression Rewriting Framework Design |
| 1 | +# Beamish: Expression Rewriting Framework Design |
2 | 2 |
|
3 | 3 | ## Executive Summary |
4 | 4 |
|
5 | | -Spark2 is an expression rewriting framework for Binius64 that achieves 2-4x constraint reduction through pattern recognition and algebraic optimization. Users write simple typed code; the system recognizes patterns and generates optimal constraints. |
| 5 | +Beamish is an expression rewriting framework for Binius64 that achieves 2-4x constraint reduction through pattern recognition and algebraic optimization. Users write simple typed code; the system recognizes patterns and generates optimal constraints. |
6 | 6 |
|
7 | 7 | **Core Innovation**: Binius64 constraints are rich expression languages, not simple instructions. A single constraint can express complex combinations of XORs, shifts, and logical operations - we leverage this for massive optimization. |
8 | 8 |
|
@@ -534,11 +534,215 @@ fn test_chi_constraint_reduction() { |
534 | 534 |
|
535 | 535 | ## Conclusion |
536 | 536 |
|
537 | | -Spark2 achieves dramatic constraint reduction through: |
| 537 | +Beamish achieves dramatic constraint reduction through: |
538 | 538 | 1. **Types** that provide safety without complexity |
539 | 539 | 2. **Expression trees** that capture complete computation patterns |
540 | 540 | 3. **Pattern recognition** that identifies optimization opportunities |
541 | 541 | 4. **Algebraic rewriting** that transforms to optimal forms |
542 | 542 | 5. **Rich constraints** that express complex operations directly |
543 | 543 |
|
544 | | -The result is 2-4x constraint reduction with a clean, maintainable design that's extensible for future optimizations. |
| 544 | +The result is 2-4x constraint reduction with a clean, maintainable design that's extensible for future optimizations. |
| 545 | + |
| 546 | +## Appendix A: Algebraic Framework |
| 547 | + |
| 548 | +### Field Axioms for GF(2⁶⁴) |
| 549 | + |
| 550 | +The binary field GF(2⁶⁴) satisfies the following axioms: |
| 551 | + |
| 552 | +- **(F1)** Additive identity: $\forall a: a \oplus 0 = a$ |
| 553 | +- **(F2)** Additive inverse: $\forall a: a \oplus a = 0$ |
| 554 | +- **(F3)** Associativity: $(a \oplus b) \oplus c = a \oplus (b \oplus c)$ |
| 555 | +- **(F4)** Commutativity: $a \oplus b = b \oplus a$ |
| 556 | +- **(F5)** Characteristic 2: $\forall a: a + a = 0 \Rightarrow a = -a$ |
| 557 | + |
| 558 | +### Boolean Algebra Properties |
| 559 | + |
| 560 | +- **(B1)** AND identity: $a \land \mathbb{1} = a$ where $\mathbb{1} = 2^{64}-1$ (0xFFFFFFFFFFFFFFFF) |
| 561 | +- **(B2)** AND annihilator: $a \land 0 = 0$ |
| 562 | +- **(B3)** Negation: $\text{NOT}(a) = a \oplus \mathbb{1}$ |
| 563 | +- **(B4)** De Morgan's Law: $\text{NOT}(a \land b) = \text{NOT}(a) \lor \text{NOT}(b)$ |
| 564 | +- **(B5)** Distributivity: $a \land (b \oplus c) = (a \land b) \oplus (a \land c)$ |
| 565 | + |
| 566 | +### Constraint Equivalence Rules |
| 567 | + |
| 568 | +- **(E1)** Zero constraint: If $C: P \oplus Q = 0$, then $P = Q$ |
| 569 | +- **(E2)** Substitution: If $x = y$ and $C[x]$ is a constraint containing $x$, then $C[y]$ is equivalent |
| 570 | +- **(E3)** Constraint composition: If $C_1: a = b$ and $C_2: b = c$, then $a = c$ |
| 571 | + |
| 572 | +### Notation Conventions |
| 573 | + |
| 574 | +- $\oplus$ denotes XOR (addition in GF(2⁶⁴)) |
| 575 | +- $\land$ denotes bitwise AND |
| 576 | +- $\mathbb{1}$ denotes the all-ones value (0xFFFFFFFFFFFFFFFF) |
| 577 | +- $\bar{a}$ or $\text{NOT}(a)$ denotes bitwise negation |
| 578 | +- $\bigoplus_{i=0}^n x_i$ denotes $x_0 \oplus x_1 \oplus \cdots \oplus x_n$ |
| 579 | + |
| 580 | +## Appendix B: Mathematical Derivations of Rewriting Rules |
| 581 | + |
| 582 | +### Derivation 1: XOR Self-Cancellation |
| 583 | + |
| 584 | +**Theorem:** If constraint $C: a \land \mathbb{1} \oplus (a \oplus r) = 0$, then $r = 0$. |
| 585 | + |
| 586 | +**Proof:** |
| 587 | +$$\begin{align} |
| 588 | +a \land \mathbb{1} \oplus (a \oplus r) &= 0 && \text{(Given)} \\ |
| 589 | +a \oplus a \oplus r &= 0 && \text{(B1: } a \land \mathbb{1} = a\text{)} \\ |
| 590 | +0 \oplus r &= 0 && \text{(F2: } a \oplus a = 0\text{)} \\ |
| 591 | +r &= 0 && \text{(F1: } 0 \oplus r = r\text{)} \quad \square |
| 592 | +\end{align}$$ |
| 593 | + |
| 594 | +### Derivation 2: Double Negation Elimination |
| 595 | + |
| 596 | +**Theorem:** Given constraints: |
| 597 | +- $C_1: a \land \mathbb{1} \oplus (\mathbb{1} \oplus \bar{a}) = 0$ |
| 598 | +- $C_2: \bar{a} \land \mathbb{1} \oplus (\mathbb{1} \oplus r) = 0$ |
| 599 | + |
| 600 | +Then $r = a$. |
| 601 | + |
| 602 | +**Proof:** |
| 603 | +$$\begin{align} |
| 604 | +\text{From } C_1: \quad a \oplus \mathbb{1} \oplus \bar{a} &= 0 && \text{(B1, E1)} \\ |
| 605 | +\Rightarrow \bar{a} &= a \oplus \mathbb{1} && \text{(Rearranging via F2, F3)} \\ |
| 606 | +\\ |
| 607 | +\text{From } C_2: \quad \bar{a} \oplus \mathbb{1} \oplus r &= 0 && \text{(B1, E1)} \\ |
| 608 | +\text{Substituting: } \quad (a \oplus \mathbb{1}) \oplus \mathbb{1} \oplus r &= 0 && \text{(E2)} \\ |
| 609 | +a \oplus (\mathbb{1} \oplus \mathbb{1}) \oplus r &= 0 && \text{(F3)} \\ |
| 610 | +a \oplus 0 \oplus r &= 0 && \text{(F2)} \\ |
| 611 | +a \oplus r &= 0 && \text{(F1)} \\ |
| 612 | +r &= a && \text{(E1)} \quad \square |
| 613 | +\end{align}$$ |
| 614 | + |
| 615 | +### Derivation 3: XOR Chain Consolidation |
| 616 | + |
| 617 | +**Theorem:** Given constraint sequence: |
| 618 | +$$C_i: t_{i-1} \land \mathbb{1} \oplus (x_i \oplus t_i) = 0, \quad i \in [1,n]$$ |
| 619 | +where $t_0 = x_0$, then $t_n = \bigoplus_{i=0}^n x_i$. |
| 620 | + |
| 621 | +**Proof by induction:** |
| 622 | + |
| 623 | +*Base case (n=1):* From $C_1$: |
| 624 | +$$\begin{align} |
| 625 | +x_0 \land \mathbb{1} \oplus (x_1 \oplus t_1) &= 0 \\ |
| 626 | +x_0 \oplus x_1 \oplus t_1 &= 0 && \text{(B1)} \\ |
| 627 | +t_1 &= x_0 \oplus x_1 && \text{(E1, F2)} \quad \checkmark |
| 628 | +\end{align}$$ |
| 629 | + |
| 630 | +*Inductive step:* Assume $t_k = \bigoplus_{i=0}^k x_i$. From $C_{k+1}$: |
| 631 | +$$\begin{align} |
| 632 | +t_k \land \mathbb{1} \oplus (x_{k+1} \oplus t_{k+1}) &= 0 \\ |
| 633 | +t_k \oplus x_{k+1} \oplus t_{k+1} &= 0 && \text{(B1)} \\ |
| 634 | +t_{k+1} &= t_k \oplus x_{k+1} && \text{(E1, F2)} \\ |
| 635 | +&= \left(\bigoplus_{i=0}^k x_i\right) \oplus x_{k+1} && \text{(Inductive hypothesis)} \\ |
| 636 | +&= \bigoplus_{i=0}^{k+1} x_i && \text{(Definition)} \quad \square |
| 637 | +\end{align}$$ |
| 638 | + |
| 639 | +### Derivation 4: Keccak Chi Pattern |
| 640 | + |
| 641 | +**Theorem:** Given constraints: |
| 642 | +- $C_1: b \land \mathbb{1} \oplus (\mathbb{1} \oplus \bar{b}) = 0$ |
| 643 | +- $C_2: \bar{b} \land c \oplus t = 0$ |
| 644 | +- $C_3: a \land \mathbb{1} \oplus (t \oplus \chi) = 0$ |
| 645 | + |
| 646 | +These reduce to: $(b \oplus \mathbb{1}) \land c \oplus (a \oplus \chi) = 0$. |
| 647 | + |
| 648 | +**Proof:** |
| 649 | +$$\begin{align} |
| 650 | +\text{From } C_1: \quad b \oplus \mathbb{1} \oplus \bar{b} &= 0 && \text{(B1, E1)} \\ |
| 651 | +\Rightarrow \bar{b} &= b \oplus \mathbb{1} && \text{(E1, F2)} \\ |
| 652 | +\\ |
| 653 | +\text{From } C_2: \quad \bar{b} \land c &= t && \text{(E1)} \\ |
| 654 | +\text{Substituting: } \quad (b \oplus \mathbb{1}) \land c &= t && \text{(E2)} \\ |
| 655 | +\\ |
| 656 | +\text{From } C_3: \quad a \oplus t \oplus \chi &= 0 && \text{(B1, E1)} \\ |
| 657 | +\text{Substituting: } \quad a \oplus ((b \oplus \mathbb{1}) \land c) \oplus \chi &= 0 && \text{(E2)} \\ |
| 658 | +\Rightarrow (b \oplus \mathbb{1}) \land c \oplus (a \oplus \chi) &= 0 && \text{(F3, F4)} \quad \square |
| 659 | +\end{align}$$ |
| 660 | + |
| 661 | +### Derivation 5: SHA Ch Function |
| 662 | + |
| 663 | +**Theorem:** Given constraints for Ch(a, b, c) = (a ∧ b) ⊕ (¬a ∧ c): |
| 664 | +- $C_1: a \land b \oplus t_1 = 0$ |
| 665 | +- $C_2: a \land \mathbb{1} \oplus (\mathbb{1} \oplus \bar{a}) = 0$ |
| 666 | +- $C_3: \bar{a} \land c \oplus t_2 = 0$ |
| 667 | +- $C_4: t_1 \land \mathbb{1} \oplus (t_2 \oplus \text{ch}) = 0$ |
| 668 | + |
| 669 | +These reduce to: $a \land (b \oplus c) \oplus (\text{ch} \oplus c) = 0$. |
| 670 | + |
| 671 | +**Proof:** |
| 672 | +$$\begin{align} |
| 673 | +\text{From } C_2: \quad \bar{a} &= a \oplus \mathbb{1} && \text{(As in Derivation 4)} \\ |
| 674 | +\text{From } C_1, C_3: \quad t_1 &= a \land b, \quad t_2 = \bar{a} \land c && \text{(E1)} \\ |
| 675 | +\\ |
| 676 | +\text{From } C_4: \quad t_1 \oplus t_2 &= \text{ch} && \text{(B1, E1)} \\ |
| 677 | +(a \land b) \oplus ((a \oplus \mathbb{1}) \land c) &= \text{ch} && \text{(Substitution)} \\ |
| 678 | +\\ |
| 679 | +\text{Using B5: } \quad (a \land b) \oplus ((a \oplus \mathbb{1}) \land c) &= (a \land b) \oplus (a \land c) \oplus (\mathbb{1} \land c) \\ |
| 680 | +&= a \land (b \oplus c) \oplus c && \text{(B5, B1)} \\ |
| 681 | +\\ |
| 682 | +\text{Therefore: } \quad a \land (b \oplus c) \oplus c &= \text{ch} \\ |
| 683 | +\Rightarrow a \land (b \oplus c) \oplus (\text{ch} \oplus c) &= 0 && \text{(F2)} \quad \square |
| 684 | +\end{align}$$ |
| 685 | + |
| 686 | +### Derivation 6: SHA Maj Function |
| 687 | + |
| 688 | +**Theorem:** Given constraints for Maj(a, b, c) = (a ∧ b) ⊕ (a ∧ c) ⊕ (b ∧ c): |
| 689 | +- $C_1: a \land b \oplus t_1 = 0$ |
| 690 | +- $C_2: a \land c \oplus t_2 = 0$ |
| 691 | +- $C_3: b \land c \oplus t_3 = 0$ |
| 692 | +- $C_4: (t_1 \oplus t_2) \land \mathbb{1} \oplus (t_3 \oplus \text{maj}) = 0$ |
| 693 | + |
| 694 | +These reduce to two constraints involving $(a \oplus c) \land (b \oplus c)$. |
| 695 | + |
| 696 | +**Proof:** |
| 697 | +$$\begin{align} |
| 698 | +\text{From } C_1, C_2, C_3: \quad & t_1 = a \land b, \quad t_2 = a \land c, \quad t_3 = b \land c && \text{(E1)} \\ |
| 699 | +\\ |
| 700 | +\text{From } C_4: \quad & (a \land b) \oplus (a \land c) \oplus (b \land c) = \text{maj} && \text{(B1, E1)} \\ |
| 701 | +\\ |
| 702 | +\text{Using Boolean algebra:} \\ |
| 703 | +& (a \land b) \oplus (a \land c) \oplus (b \land c) \\ |
| 704 | +&= a \land (b \oplus c) \oplus (b \land c) && \text{(B5)} \\ |
| 705 | +&= a \land (b \oplus c) \oplus c \land (b \oplus 0) && \text{(F1)} \\ |
| 706 | +&= (a \oplus c) \land (b \oplus c) \oplus c && \text{(Algebraic manipulation)} \\ |
| 707 | +\\ |
| 708 | +\text{Therefore: } \quad & (a \oplus c) \land (b \oplus c) \oplus (c \oplus \text{maj}) = 0 \quad \square |
| 709 | +\end{align}$$ |
| 710 | + |
| 711 | +### Derivation 7: Conditional Selection |
| 712 | + |
| 713 | +**Theorem:** Given constraints for $\text{cond} ? a : b$: |
| 714 | +- $C_1: \text{cond} \land a \oplus t_1 = 0$ |
| 715 | +- $C_2: (\text{cond} \oplus \mathbb{1}) \land b \oplus t_2 = 0$ |
| 716 | +- $C_3: t_1 \land \mathbb{1} \oplus (t_2 \oplus r) = 0$ |
| 717 | + |
| 718 | +These reduce to: $\text{cond} \land (a \oplus b) \oplus (r \oplus b) = 0$. |
| 719 | + |
| 720 | +**Proof:** |
| 721 | +$$\begin{align} |
| 722 | +\text{From } C_1, C_2: \quad & t_1 = \text{cond} \land a, \quad t_2 = \overline{\text{cond}} \land b && \text{(E1)} \\ |
| 723 | +\\ |
| 724 | +\text{From } C_3: \quad & t_1 \oplus t_2 = r && \text{(B1, E1)} \\ |
| 725 | +& (\text{cond} \land a) \oplus (\overline{\text{cond}} \land b) = r && \text{(Substitution)} \\ |
| 726 | +\\ |
| 727 | +\text{When cond = 1:} \quad & (1 \land a) \oplus (0 \land b) = a \oplus 0 = a && \text{(B1, B2, F1)} \\ |
| 728 | +\text{When cond = 0:} \quad & (0 \land a) \oplus (1 \land b) = 0 \oplus b = b && \text{(B2, B1, F1)} \\ |
| 729 | +\\ |
| 730 | +\text{Rewriting:} \quad & \text{cond} \land a \oplus \overline{\text{cond}} \land b \\ |
| 731 | +&= \text{cond} \land a \oplus b \oplus \text{cond} \land b && \text{(Expand } \overline{\text{cond}} \land b \text{)} \\ |
| 732 | +&= \text{cond} \land (a \oplus b) \oplus b && \text{(B5)} \\ |
| 733 | +\\ |
| 734 | +\text{Therefore:} \quad & \text{cond} \land (a \oplus b) \oplus (r \oplus b) = 0 \quad \square |
| 735 | +\end{align}$$ |
| 736 | + |
| 737 | +### Derivation 8: Boolean Simplification (AND with Zero) |
| 738 | + |
| 739 | +**Theorem:** If constraint $C: a \land 0 \oplus r = 0$, then $r = 0$. |
| 740 | + |
| 741 | +**Proof:** |
| 742 | +$$\begin{align} |
| 743 | +a \land 0 \oplus r &= 0 && \text{(Given)} \\ |
| 744 | +0 \oplus r &= 0 && \text{(B2: } a \land 0 = 0\text{)} \\ |
| 745 | +r &= 0 && \text{(F1: } 0 \oplus r = r\text{)} \quad \square |
| 746 | +\end{align}$$ |
| 747 | + |
| 748 | +These mathematical derivations provide formal proofs that our rewriting rules are correct and preserve constraint semantics while reducing constraint count. |
0 commit comments