Skip to content

Add CalculationSink for Aggregate-Level Mathematical Operations#4763

Open
abdelaziz-mahdy wants to merge 5 commits intodevelopmentfrom
calcuation-sink
Open

Add CalculationSink for Aggregate-Level Mathematical Operations#4763
abdelaziz-mahdy wants to merge 5 commits intodevelopmentfrom
calcuation-sink

Conversation

@abdelaziz-mahdy
Copy link
Copy Markdown
Collaborator

Reopening due to need.

Problem

There was no way to perform mathematical operations on the results of aggregated sinks. Existing tools have gaps:

Tool Limitation
FScript Evaluates per-row before aggregation; cannot access SUM/COUNT/AVG results
Sequence (SEQ) Runs sinks in parallel but cannot perform cross-sink calculations
Formula (ADD/SUB/MUL/DIV) Works on property values per-row, not on sink outputs
GroupBy Groups data but needs a sink that can calculate across filtered aggregates

Use Case That Couldn't Be Solved:

For each category, calculate: income - expenses - adjustments
Where income/expenses/adjustments are filtered subsets with their own SUMs

Solution

Added CalculationSink - a sink that performs arithmetic operations on values from other sinks with proper operator precedence.

Key Features:

  • Chains sink results: FilteredSink(isIncome, SUM) - FilteredSink(isExpense, SUM)
  • Supports operator precedence: * and / before + and -
  • Two evaluation modes:
    • Total-level (default): Aggregate data in nested sinks, calculate at EOF
    • Row-level: Calculate per-row and accumulate results
  • Integrates with GroupBy, FilteredSink, LabeledSink

Example:

GroupBy(category, CalculationSink({
  operations: [
    { operand: FilteredSink(IS_INCOME, SUM(amount)), operation: ADD },
    { operand: FilteredSink(IS_EXPENSE, SUM(amount)), operation: SUBTRACT },
    { operand: FilteredSink(IS_ADJUSTMENT, SUM(amount)), operation: SUBTRACT }
  ]
}))

Data Flow:

Row 1 ──┬──► FilteredSink(income)     ──► SUM ──┐
Row 2 ──┤                                       │
Row 3 ──┤──► FilteredSink(expenses)   ──► SUM ──┼──► CALCULATE at EOF
Row 4 ──┤                                       │    (income - expenses - adj)
Row 5 ──┴──► FilteredSink(adjustments)──► SUM ──┘

Changes

  • Added CalculationSink class with support for ADD, SUBTRACT, MULTIPLY, DIVIDE operations
  • Added CalculationOperation class to pair operands with operations
  • Added ArithmeticOperation enum for supported operations
  • Added CalculationDAOAgent for reflow UI integration
  • Fixed: eof() now propagates to child sinks before evaluation
  • Fixed: toString() properly uses enum labels
  • Fixed: Java reduce() checks if operand is Reducible before casting
  • Fixed: Java getSinkValue_() handles nested CalculationSink
  • Added comprehensive test suite (JS and Java) covering:
    • Basic addition/subtraction
    • Operator precedence
    • Row-level vs total-level modes
    • FilteredSink integration
    • GroupBy integration
    • Edge cases (empty operations, division by zero)

Why Not Just Use FScript?

FScript is powerful for per-row expressions but operates before aggregation:

┌─────────────────────────────────────────────────────────────┐
│                    PROCESSING TIMELINE                      │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  FScript operates HERE                                      │
│        ↓                                                    │
│  [Row1] → [Row2] → [Row3] → ... → [RowN] → [AGGREGATION]   │
│                                                    ↑        │
│                              CalculationSink operates HERE  │
│                                                             │
└─────────────────────────────────────────────────────────────┘

FScript cannot:

  • Access SUM(amount) results (doesn't exist until after all rows)
  • Perform SUM(income) - SUM(expenses) (needs aggregate values)
  • Calculate across multiple filtered subsets of the same data

CalculationSink fills this gap by operating on sink outputs rather than row inputs.

@abdelaziz-mahdy abdelaziz-mahdy marked this pull request as draft January 27, 2026 22:50
… tests

Fix several issues in CalculationSink for aggregate-level math operations:
- Propagate eof() to child sinks before evaluation to ensure final values
- Fix toString() to show correct operators matching evaluation semantics
- Handle Property axioms in row-level mode via f() function check
- Fix Java reduce() to verify operand is Reducible before casting
- Support nested CalculationSink in getSinkValue_()
- Add Java toString() implementation for consistency
- Remove redundant default values for Boolean and Double properties

Add comprehensive test suites covering addition, subtraction, operator
precedence, division, row-level vs total-level modes, FilteredSink and
GroupBy integration, nested sinks, reset, and edge cases.
@abdelaziz-mahdy abdelaziz-mahdy marked this pull request as ready for review January 27, 2026 23:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant