diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000000..49462c3862d --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,162 @@ +# AGENTS.md — Open Food Network + +> **Status**: v0.3 — field-tested on Ubuntu 24.04, March 2026; process docs added May 2026 +> Maintained in [GOAT Agent Files](https://gitlab.com/our-sci/goat-agent-files). Improvements via MR. + + + +## OFN official process docs (read & follow these) + +OFN has its own contributor documentation. This file gives you project conventions and +pitfalls; it does **not** replace OFN's process. Read and follow OFN's own guides — when they +disagree with this file, OFN's docs win. The full forum → issue → PR → merge walkthrough is in +[`contributing-workflow.md`](goat-agents/contributing-workflow.md). Core sources: + +- [`CONTRIBUTING.md`](https://github.com/openfoodfoundation/openfoodnetwork/blob/master/CONTRIBUTING.md) and [`GETTING_STARTED.md`](https://github.com/openfoodfoundation/openfoodnetwork/blob/master/GETTING_STARTED.md) — fork/`upstream`, issue-numbered branch, TDD +- Wiki: [Making a great pull request](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Making-a-great-pull-request), [Making a great commit](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Making-a-great-commit), [Code Conventions](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Code-Conventions), [Testing & Rspec Tips](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Testing-and-Rspec-Tips) +- Wiki: [Review, test, merge & deploy](https://github.com/openfoodfoundation/openfoodnetwork/wiki/The-process-of-review%2C-test%2C-merge-and-deploy) — a PR needs **2 core reviewers** + a staging test stage before merge; releases are ~weekly. Don't expect an instant merge. +- **Discuss features first on the [community forum](https://community.openfoodnetwork.org)** ([Product Development backlog](https://community.openfoodnetwork.org/c/product-dev/5)); check the [user guide](https://guide.openfoodnetwork.org/) in case the feature already exists. +- **Write issues** with the right [template](https://github.com/openfoodfoundation/openfoodnetwork/tree/master/.github/ISSUE_TEMPLATE) (bug / story / feature / tech-debt) and a [bug-severity](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Bug-severity) label. +- Be civil and patient per the [Code of Conduct](https://github.com/openfoodfoundation/openfoodnetwork/blob/master/CODE_OF_CONDUCT.md) — reviewers and testers are volunteers. + +## Setup + +OFN uses native Ruby for local development. Docker exists but is too slow for active work +(20+ second page loads vs ~1-3s natively). + +```bash +# Clone +git clone https://github.com/openfoodfoundation/openfoodnetwork.git +cd openfoodnetwork + +# Install system deps (Ubuntu) — includes cmake, postgres, redis, imagemagick +bash script/install-system-deps.sh # requires sudo + +# Install Ruby via rbenv (compile ~5-10 min) +rbenv install $(cat .ruby-version) && rbenv global $(cat .ruby-version) + +# Install Node via nodenv +nodenv install $(cat .node-version) && nodenv global $(cat .node-version) + +# Run OFN's setup script (gems + db + seed data, ~15 min first time) +./script/setup + +# Start dev server (Rails + webpack + sidekiq) +bundle exec foreman start +``` + +App at http://localhost:3000 — login: `ofn@example.com` / `ofn123` + +Full runbook with troubleshooting: [`setup.md`](goat-agents/setup.md) + +## Running tests + +```bash +# Single spec file +bundle exec rspec spec/models/spree/order_spec.rb + +# By line number +bundle exec rspec spec/models/spree/order_spec.rb:42 + +# System specs (browser) +bundle exec rspec spec/system/ + +# Prepare test database (after schema changes) +bundle exec rake db:test:prepare +``` + +## Conventions + +- OFN is Ruby on Rails. It was originally built around Spree (a commerce engine) but is actively moving away from it — treat Spree as legacy context, not the primary architecture +- Models in `app/models/spree/` are OFN's own files, edited directly — not via decorators. The Spree gem is a dependency but OFN owns these model files. +- JavaScript: AngularJS (CoffeeScript, legacy — being phased out) lives in `app/assets/javascripts/`. Modern JS uses Stimulus + Hotwire/Turbo + CableReady (StimulusReflex). CableReady is planned for removal (reliability issues) — Turbo Streams is the preferred replacement. Some edge cases (e.g., report downloads) still use CableReady + polling; the future ideal is SSE, but not yet prioritised. There is no Vue.js. +- Stimulus controllers are in `app/webpacker/controllers/` (shared) and `app/components/*/` (component-scoped). Naming: `[name]_controller.js`. +- Database migrations go in `db/migrate/` — never modify existing migration files +- i18n: all user-facing strings use `I18n.t()` or Rails `t()` helper — never hardcode strings. Prefer lazy lookup (`t('.key')`) in views to avoid polluting the global namespace +- API is standard Rails (`ActionController::API`) at `app/controllers/api/v0/` and `app/controllers/api/v1/`. A separate DFC API engine lives at `engines/dfc_provider/`. Not Grape. + +## Responding to PR feedback + +Not all reviewer feedback carries equal weight. Calibrate your response based on who is giving the guidance. + +**Core maintainers** (as of March 2026 — verify with `gh pr list --state closed --limit 30 --json mergedBy`): +- `mkllnk` (Maikel) and `rioug` — primary maintainers, most active reviewers and mergers. Follow their guidance immediately; update GOAT and memory accordingly. +- `dacook` — active maintainer. High weight. + +**For core maintainer feedback**: apply it, update rules, and post a follow-up comment showing exactly what changed (show, don't tell). + +**For other contributors**: be positive and polite, but think it through before acting. Does the suggestion align with codebase conventions? Does it conflict with what maintainers have said? If uncertain, respond with a question or explain your reasoning rather than silently agreeing and changing course. + +## Before starting work on an issue + +Re-read the issue immediately before writing any code, even if you planned it earlier. Check: + +1. **Is it already claimed?** Look for a linked PR, an "In Progress" label, or a comment like "I'm working on this". If someone else has started, stop and pick a different issue. +2. **Is there an open PR?** Run `gh pr list -R openfoodfoundation/openfoodnetwork --search "closes #ISSUE_NUMBER"` or scan recent PRs. An open PR means the fix is already underway. +3. **Has the issue changed?** Scope, acceptance criteria, or approach may have been updated since you last read it. Skim all comments — the last one often supersedes earlier discussion. +4. **Is it still open?** If it was closed (merged fix or won't-fix), pick another issue. + +These checks take 30 seconds and prevent duplicating work that another contributor has already done. + +## Boundaries + +**Always**: +- Run tests before opening a PR: `bundle exec rspec spec/path/to/changed_spec.rb` +- Write or update specs for all changed behavior +- Run `./bin/rails db:migrate` after pulling changes with new migrations +- Use the PR template (`.github/PULL_REQUEST_TEMPLATE.md`) when opening PRs — fill in What/Why, What should we test, and the Changelog Category checkbox +- Only modify `config/locales/en.yml` — other locale files are managed by Transifex and will be overwritten on the next release +- **Confirm fixes in the browser** before considering work done (Playwright, Cypress, or manual): reproduce the bug, then show it fixed. Screenshots must go in the **PR description** (not committed to the repo). GitHub only allows image uploads via web UI drag-and-drop — save screenshots to a known local path and ask the human collaborator to paste them into the PR description. + +**Ask first**: +- Changes to database schema or existing migrations +- Modifications to Spree core behavior (the commerce engine) +- Changes to the v0 API endpoints — there is no OFN mobile app, but v0 must remain stable for external consumers +- Adding new Ruby gems + +**Never**: +- Modify existing migration files — create a new migration instead +- Edit files in `vendor/` or Spree gem source +- Hardcode environment-specific configuration — use ENV variables +- Merge directly to `master` — all changes go through PR with CI passing + +## Common pitfalls + +1. **Confusing Spree gem source with OFN's model files**: `app/models/spree/` contains OFN's own versions of these models — edit them directly. The actual Spree gem source is in the bundle cache, not in `app/`. +2. **Running `foreman start` instead of `bundle exec foreman start`**: Ubuntu ships a system `ruby-foreman` that uses system Ruby and can't find `bundle` → Always use `bundle exec foreman start` +3. **AngularJS vs Stimulus confusion**: Legacy admin JS is AngularJS (CoffeeScript in `app/assets/javascripts/`). New admin features use Stimulus + Turbo. There is no Vue.js in OFN — never has been. +4. **Modifying existing migrations**: Breaks reproducibility for other contributors → Create a new migration with the required change +5. **Missing i18n**: Hardcoded English strings break translations → Use `I18n.t('key')` and add the key to `config/locales/en.yml` +6. **RSpec global constant namespace pollution**: Constants defined at the top level of spec files (e.g. `MY_CONST = ...`) are truly global and leak across the test suite. Define them as local variables (`my_const = ...`) inside the `it` or `shared_examples` block where they are actually used. +7. **Run rubocop before pushing**: CI runs reviewdog which fails on style offenses. Catch them locally first: `bundle exec rubocop --only-recognized-file-types path/to/changed/file.rb`. Common issue: extra blank lines at block body end (`Layout/EmptyLinesAroundBlockBody`). +8. **Separate commits for different kinds of changes**: If a PR touches two unrelated concerns (e.g., fixing form label accessibility AND replacing display-only `label_tag nil` with `span`), put them in separate commits with a message explaining the distinction. Maintainers flag bundled changes that require different reasoning to review. +9. **No git history rewrites once a PR is under review**: Once a reviewer has left a comment on the PR, treat the branch as append-only until merge — no force-pushes, rebases, or amends. Rewriting detaches inline comments from the diff they referenced and makes the review thread harder for the next reviewer to follow. Push additive fixup commits instead; squash (if wanted) happens at merge time. Before any review starts, rebase/amend is fine. + +## Closing the loop + +Beyond *responding* to feedback (see "Responding to PR feedback" above), you commit to following each PR/issue through to merge, close, or explicit handoff. This applies whether the work was authored with Claude, Codex, Gemini, Copilot, an open-source model, or by hand — the maintainer is talking to you, not your tool. + +- **Watch what you open.** Subscribe to the GitHub PR (and the linked issue if separate). If your assistant supports a session-start activity check, wire it in so CI failures and review comments surface promptly. +- **Respond within ~1 week** to reviewer comments, especially from core maintainers. OFN reviewers are volunteers; silence pushes them to close work or stop reviewing AI-assisted contributions altogether. +- **CI failures are your responsibility.** If reviewdog flags style or the test suite fails, fix it before the maintainer has to ask. +- **Don't open and forget.** If you can't continue, post a note in the PR and either hand off or close. Stale PRs taint the appearance of the broader AI-assisted contribution effort. + +## Further reading + +- [`contributing-workflow.md`](goat-agents/contributing-workflow.md) — full forum → issue → PR → review/test/merge/deploy walkthrough, with links to OFN's official docs +- [`antipatterns.md`](goat-agents/antipatterns.md) — extended examples of common AI mistakes +- [`setup.md`](goat-agents/setup.md) — full dev environment runbook +- [`review-checklist.md`](goat-agents/review-checklist.md) — what OFN maintainers flag in PR review +- [`benchmarks/README.md`](goat-agents/benchmarks/README.md) — how to validate changes to this file + +## User-facing documentation (core context for any agent) + +When evaluating whether an OFN idea is already addressed by existing features, or when explaining how something currently works, **always consult the user-facing documentation first**: + +- **OFN User Guide** — https://guide.openfoodnetwork.org/ — the canonical guide for hub managers, producers, and shoppers. Covers basic features, hub-management tips, marketing tips, quick-start guides, FAQs, and a glossary. This is where to look when a proposed "feature" might already exist as a workflow. +- **OFN Super-Admin Guide** — https://ofn-user-guide.gitbook.io/ofn-super-admin-guide — for those running their own OFN instance. +- **OFN Developer Site** — https://dev.openfoodnetwork.org/ — developer-facing material outside of this AGENTS file. + +Both user guides are GitBook-hosted; their sitemaps can be fetched and the per-page content scraped. The `idea-refiner` tool (https://gitlab.com/our-sci/software/idea-refiner) caches the user guide and consults it before agents respond, to avoid recommending features that already exist. + +If an agent has no way to query the docs directly, it should at minimum *acknowledge* that the docs exist and tell the user to check there, rather than asserting "OFN does not have this feature." diff --git a/docs/superpowers/plans/2026-06-12-n8n-oc-stock-snapshot.md b/docs/superpowers/plans/2026-06-12-n8n-oc-stock-snapshot.md new file mode 100644 index 00000000000..9da291698af --- /dev/null +++ b/docs/superpowers/plans/2026-06-12-n8n-oc-stock-snapshot.md @@ -0,0 +1,764 @@ +# n8n OC Stock Snapshot — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build an n8n workflow that snapshots OFN product stock levels at OC open and close for a hub, writing opening stock, closing stock, units sold, and % sold per variant to a Google Sheet. + +**Architecture:** A single parameterized n8n workflow triggered twice per order cycle (once at `orders_open_at`, once at `orders_close_at`). A workflow variable `snapshot_type` determines which branch runs. The opening branch creates a new tab, writes stock data, and appends rows to the master sheet. The closing branch fetches current stock, fetches line item sales, reads back opening data from the sheet, calculates sold quantities in a Code node, and updates both the OC tab and master sheet. + +**Tech Stack:** n8n v1.x, PostgreSQL node, Google Sheets node (OAuth2), Code node (JavaScript ES2020), Merge node + +--- + +## File Structure + +``` +n8n/ + sql/ + stock_snapshot.sql # Stock snapshot query (both runs) + line_items_aggregate.sql # Line items aggregate (closing run only) + workflows/ + stock-snapshot.json # Exported n8n workflow JSON (produced in Task 9) + README.md # Credential setup + per-OC scheduling guide (produced in Task 9) +``` + +--- + +## Task 1: Write and verify SQL queries + +**Files:** +- Create: `n8n/sql/stock_snapshot.sql` +- Create: `n8n/sql/line_items_aggregate.sql` + +- [ ] **Step 1: Create the directory structure** + +```bash +mkdir -p n8n/sql n8n/workflows +``` + +- [ ] **Step 2: Write the stock snapshot query** + +Create `n8n/sql/stock_snapshot.sql`: + +```sql +-- Returns all variants in scope for a given hub and order cycle, with current stock levels. +-- Used for both the opening and closing snapshot runs. +-- $1 = hub_id (integer), $2 = order_cycle_id (integer) +SELECT + e_supplier.name AS supplier, + sp.name AS product, + COALESCE(sv.display_name, sv.sku, 'Default') AS variant, + sv.id AS variant_id, + COALESCE(vo.on_demand, sv.on_demand) AS on_demand, + COALESCE(vo.count_on_hand, ssi.count_on_hand) AS stock_level +FROM order_cycles oc +JOIN exchanges ex + ON ex.order_cycle_id = oc.id + AND ex.receiver_id = $1 + AND ex.incoming = false +JOIN exchange_variants ev ON ev.exchange_id = ex.id +JOIN spree_variants sv + ON sv.id = ev.variant_id + AND sv.deleted_at IS NULL +JOIN spree_products sp + ON sp.id = sv.product_id + AND sp.deleted_at IS NULL +JOIN enterprises e_supplier + ON e_supplier.id = sp.supplier_id +LEFT JOIN variant_overrides vo + ON vo.variant_id = sv.id + AND vo.hub_id = $1 + AND vo.permission_revoked_at IS NULL +LEFT JOIN spree_stock_items ssi + ON ssi.variant_id = sv.id + AND ssi.deleted_at IS NULL +WHERE oc.id = $2 +ORDER BY e_supplier.name, sp.name, sv.display_name; +``` + +- [ ] **Step 3: Write the line items aggregate query** + +Create `n8n/sql/line_items_aggregate.sql`: + +```sql +-- Returns total units sold per variant for an order cycle, excluding cancelled orders. +-- $1 = order_cycle_id (integer), $2 = hub_id (integer) +SELECT + li.variant_id, + SUM(li.quantity) AS units_sold +FROM spree_line_items li +JOIN spree_orders o ON o.id = li.order_id +WHERE o.order_cycle_id = $1 + AND o.distributor_id = $2 + AND o.state <> 'canceled' +GROUP BY li.variant_id; +``` + +- [ ] **Step 4: Verify the stock snapshot query against a real OC** + +Replace `` and `` with real values from your OFN database and run: + +```bash +psql $DATABASE_URL -c " +SELECT + e_supplier.name AS supplier, + sp.name AS product, + COALESCE(sv.display_name, sv.sku, 'Default') AS variant, + sv.id AS variant_id, + COALESCE(vo.on_demand, sv.on_demand) AS on_demand, + COALESCE(vo.count_on_hand, ssi.count_on_hand) AS stock_level +FROM order_cycles oc +JOIN exchanges ex ON ex.order_cycle_id = oc.id + AND ex.receiver_id = AND ex.incoming = false +JOIN exchange_variants ev ON ev.exchange_id = ex.id +JOIN spree_variants sv ON sv.id = ev.variant_id AND sv.deleted_at IS NULL +JOIN spree_products sp ON sp.id = sv.product_id AND sp.deleted_at IS NULL +JOIN enterprises e_supplier ON e_supplier.id = sp.supplier_id +LEFT JOIN variant_overrides vo ON vo.variant_id = sv.id + AND vo.hub_id = AND vo.permission_revoked_at IS NULL +LEFT JOIN spree_stock_items ssi ON ssi.variant_id = sv.id AND ssi.deleted_at IS NULL +WHERE oc.id = +ORDER BY e_supplier.name, sp.name, sv.display_name; +" +``` + +Expected: rows with `supplier`, `product`, `variant`, `variant_id`, `on_demand` (true/false), `stock_level` (integer or null). On-demand variants should have `on_demand = t` and may have a null `stock_level`. + +- [ ] **Step 5: Verify the line items aggregate query** + +```bash +psql $DATABASE_URL -c " +SELECT li.variant_id, SUM(li.quantity) AS units_sold +FROM spree_line_items li +JOIN spree_orders o ON o.id = li.order_id +WHERE o.order_cycle_id = + AND o.distributor_id = + AND o.state <> 'canceled' +GROUP BY li.variant_id; +" +``` + +Expected: one row per variant that has been ordered in the OC, with the total quantity. If no orders exist yet, zero rows is normal. + +- [ ] **Step 6: Commit** + +```bash +git add n8n/sql/ +git commit -m "feat: add n8n SQL queries for OC stock snapshot workflow" +``` + +--- + +## Task 2: Set up n8n credentials + +**Files:** n8n UI only + +- [ ] **Step 1: Create a PostgreSQL credential in n8n** + +In n8n: **Settings → Credentials → New → PostgreSQL** + +| Field | Value | +|---|---| +| Host | OFN database host | +| Database | OFN database name | +| User | A read-only database role (do not use the app user — read-only prevents accidental writes) | +| Password | Database password | +| Port | 5432 | +| SSL | Enable if required by your hosting provider | + +Save as **`OFN PostgreSQL (read-only)`**. + +Click **Test connection** — expected result: "Connection successful". + +- [ ] **Step 2: Create a Google Sheets OAuth2 credential in n8n** + +In n8n: **Settings → Credentials → New → Google Sheets OAuth2 API** + +1. In Google Cloud Console, create a project and enable the **Google Sheets API** +2. Create OAuth2 credentials: Application type = **Desktop app** +3. Copy the Client ID and Client Secret into n8n +4. Click **Connect my account** and authorise with the Google account that owns the target spreadsheet + +Save as **`Google Sheets (OFN stock)`**. + +- [ ] **Step 3: Create the target Google Spreadsheet** + +In Google Sheets, create a new spreadsheet. Note the spreadsheet ID from the URL: +`https://docs.google.com/spreadsheets/d//edit` + +Create a tab named exactly **`All Order Cycles`** — this is the master sheet. Leave it empty; the workflow will write headers on first use. + +--- + +## Task 3: Create workflow scaffold + +**Files:** n8n UI + +- [ ] **Step 1: Create a new workflow** + +In n8n: **Workflows → New**. Name it **`OC Stock Snapshot`**. + +- [ ] **Step 2: Set workflow variables** + +Open the **Variables** tab (⚙ icon in the workflow). Add: + +| Name | Type | Default | +|---|---|---| +| `hub_id` | String | *(blank)* | +| `order_cycle_id` | String | *(blank)* | +| `order_cycle_name` | String | *(blank)* | +| `google_sheet_id` | String | *(blank)* | +| `snapshot_type` | String | `opening` | + +These are set per scheduled trigger instance when scheduling for a specific OC (see Task 9 README). + +- [ ] **Step 3: Add a Schedule Trigger node** + +Add a **Schedule Trigger** node. Set the trigger time to any placeholder time — the real time will be configured per OC. Name this node **`Trigger`**. + +- [ ] **Step 4: Add an IF node to branch on snapshot_type** + +Add an **IF** node connected to `Trigger`: +- **Condition:** Value 1 = `{{ $vars.snapshot_type }}` / Operation = **equals** / Value 2 = `opening` +- **True** branch → opening run (Tasks 4 & 5) +- **False** branch → closing run (Tasks 6 & 7) + +Name this node **`Is Opening Run?`**. + +--- + +## Task 4: Opening run — fetch and format data + +**Files:** n8n UI + +- [ ] **Step 1: Add a PostgreSQL node to fetch opening stock** + +Add a **PostgreSQL** node on the **True** branch of `Is Opening Run?`: +- Credential: `OFN PostgreSQL (read-only)` +- Operation: **Execute Query** +- Query (uses `$1` and `$2` parameterized placeholders — safer than string interpolation): + +```sql +SELECT + e_supplier.name AS supplier, + sp.name AS product, + COALESCE(sv.display_name, sv.sku, 'Default') AS variant, + sv.id AS variant_id, + COALESCE(vo.on_demand, sv.on_demand) AS on_demand, + COALESCE(vo.count_on_hand, ssi.count_on_hand) AS stock_level +FROM order_cycles oc +JOIN exchanges ex + ON ex.order_cycle_id = oc.id + AND ex.receiver_id = $1 + AND ex.incoming = false +JOIN exchange_variants ev ON ev.exchange_id = ex.id +JOIN spree_variants sv ON sv.id = ev.variant_id AND sv.deleted_at IS NULL +JOIN spree_products sp ON sp.id = sv.product_id AND sp.deleted_at IS NULL +JOIN enterprises e_supplier ON e_supplier.id = sp.supplier_id +LEFT JOIN variant_overrides vo + ON vo.variant_id = sv.id + AND vo.hub_id = $1 + AND vo.permission_revoked_at IS NULL +LEFT JOIN spree_stock_items ssi + ON ssi.variant_id = sv.id AND ssi.deleted_at IS NULL +WHERE oc.id = $2 +ORDER BY e_supplier.name, sp.name, sv.display_name +``` + +- Additional Fields → **Query Parameters**: `{{ $vars.hub_id }},{{ $vars.order_cycle_id }}` + +Name this node **`Get Opening Stock`**. + +- [ ] **Step 2: Test the node** + +Set `hub_id` and `order_cycle_id` workflow variables to real values, then click **Test step** on `Get Opening Stock`. Verify it returns rows with `supplier`, `product`, `variant`, `variant_id`, `on_demand`, `stock_level`. + +- [ ] **Step 3: Add an IF node to halt on empty results** + +Add an **IF** node after `Get Opening Stock`: +- **Condition:** `{{ $items("Get Opening Stock").length }}` **greater than** `0` +- **True** → continue to format node +- **False** → error notification (wired up in Task 8) + +Name this node **`Has Variants?`**. + +- [ ] **Step 4: Add a Code node to format opening rows** + +Add a **Code** node on the **True** branch of `Has Variants?`: + +```javascript +// Format stock snapshot rows for writing to Google Sheets. +// On-demand variants show 'On Demand' for stock; stocked variants show the numeric count. +// Closing stock, units sold, and % sold are left blank — filled by the closing run. + +return items.map(item => { + const { variant_id, supplier, product, variant, on_demand, stock_level } = item.json; + const isOnDemand = on_demand === true || on_demand === 't'; + const ocId = String($vars.order_cycle_id); + const variantId = String(variant_id); + + return { + json: { + 'variant_id': variantId, + 'row_key': `${ocId}_${variantId}`, + 'Supplier': supplier ?? '', + 'Product': product ?? '', + 'Variant': variant ?? '(default)', + 'On Demand': isOnDemand ? 'Yes' : 'No', + 'Opening Stock': isOnDemand ? 'On Demand' : (stock_level ?? 0), + 'Closing Stock': '', + 'Units Sold': '', + '% Sold': '' + } + }; +}); +``` + +Name this node **`Format Opening Rows`**. + +--- + +## Task 5: Opening run — write to Google Sheets + +**Files:** n8n UI + +- [ ] **Step 1: Add a Google Sheets node to create the OC tab** + +Add a **Google Sheets** node after `Format Opening Rows`: +- Resource: **Spreadsheet** +- Operation: **Create Sheet (tab)** +- Spreadsheet ID: `{{ $vars.google_sheet_id }}` +- Title: `{{ $vars.order_cycle_name }}` + +Name this node **`Create OC Tab`**. + +- [ ] **Step 2: Add a Google Sheets node to write rows to the OC tab** + +Add a **Google Sheets** node after `Create OC Tab`: +- Resource: **Sheet** +- Operation: **Append or Update** +- Spreadsheet ID: `{{ $vars.google_sheet_id }}` +- Sheet Name: `{{ $vars.order_cycle_name }}` +- Column to Match On: `variant_id` +- Data Mode: **Auto-Map Input Data** + +Name this node **`Write Opening Rows to OC Tab`**. + +This writes each row from `Format Opening Rows` to the OC tab, using the JSON field names as column headers (first write creates headers automatically). + +- [ ] **Step 3: Add a Code node to enrich rows with OC metadata for the master sheet** + +Add a **Code** node after `Write Opening Rows to OC Tab`: + +```javascript +// Enriches formatted rows with order cycle metadata for the master sheet. +// oc_opens is captured at the time the opening run executes. + +const ocId = String($vars.order_cycle_id); +const ocName = String($vars.order_cycle_name); +const ocOpens = new Date().toISOString().replace('T', ' ').split('.')[0]; // YYYY-MM-DD HH:MM:SS + +return items.map(item => ({ + json: { + 'order_cycle_id': ocId, + 'row_key': item.json.row_key, + 'Order Cycle': ocName, + 'OC Opens': ocOpens, + 'OC Closes': '', + 'variant_id': item.json.variant_id, + 'Supplier': item.json['Supplier'], + 'Product': item.json['Product'], + 'Variant': item.json['Variant'], + 'On Demand': item.json['On Demand'], + 'Opening Stock': item.json['Opening Stock'], + 'Closing Stock': '', + 'Units Sold': '', + '% Sold': '' + } +})); +``` + +Name this node **`Enrich for Master Sheet (Opening)`**. + +- [ ] **Step 4: Add a Google Sheets node to append to the master sheet** + +Add a **Google Sheets** node after `Enrich for Master Sheet (Opening)`: +- Resource: **Sheet** +- Operation: **Append or Update** +- Spreadsheet ID: `{{ $vars.google_sheet_id }}` +- Sheet Name: `All Order Cycles` +- Column to Match On: `row_key` +- Data Mode: **Auto-Map Input Data** + +Name this node **`Append to Master Sheet`**. + +- [ ] **Step 5: Test the full opening run** + +Set workflow variables to a real hub_id, order_cycle_id, order_cycle_name, and google_sheet_id. Execute the workflow manually with `snapshot_type = opening`. Verify: +- A new tab is created in the spreadsheet with the OC name +- Rows are written with correct supplier/product/variant/stock data +- The `All Order Cycles` tab has the same rows with OC metadata prepended +- On-demand variants show "On Demand" for Opening Stock +- Closing Stock, Units Sold, % Sold are blank + +--- + +## Task 6: Closing run — fetch data and read opening rows + +**Files:** n8n UI + +- [ ] **Step 1: Add a PostgreSQL node to fetch closing stock** + +Add a **PostgreSQL** node on the **False** branch of `Is Opening Run?`. Copy-paste `Get Opening Stock` and rename: +- Credential: `OFN PostgreSQL (read-only)` +- Operation: **Execute Query** +- Query: same as Task 4 Step 1 (identical SQL and query parameters) + +Name this node **`Get Closing Stock`**. + +- [ ] **Step 2: Add a PostgreSQL node to fetch line items sold** + +Add a second **PostgreSQL** node also connected to the **False** branch of `Is Opening Run?` (runs in parallel with `Get Closing Stock`): +- Credential: `OFN PostgreSQL (read-only)` +- Operation: **Execute Query** +- Query: + +```sql +SELECT + li.variant_id, + SUM(li.quantity) AS units_sold +FROM spree_line_items li +JOIN spree_orders o ON o.id = li.order_id +WHERE o.order_cycle_id = $1 + AND o.distributor_id = $2 + AND o.state <> 'canceled' +GROUP BY li.variant_id +``` + +- Additional Fields → **Query Parameters**: `{{ $vars.order_cycle_id }},{{ $vars.hub_id }}` + +Name this node **`Get Line Items Sold`**. + +- [ ] **Step 3: Add a Google Sheets node to read the OC tab opening rows** + +Add a **Google Sheets** node also connected to the **False** branch of `Is Opening Run?` (runs in parallel): +- Resource: **Sheet** +- Operation: **Read Rows** +- Spreadsheet ID: `{{ $vars.google_sheet_id }}` +- Sheet Name: `{{ $vars.order_cycle_name }}` + +Name this node **`Read Opening Rows`**. + +This returns all rows from the OC tab as items, with column headers as JSON keys. The `variant_id` and `Opening Stock` fields are the ones the Code node will use. + +- [ ] **Step 4: Add a Merge node to synchronise the three parallel branches** + +Add a **Merge** node with three inputs: +- Input 1: `Get Closing Stock` +- Input 2: `Get Line Items Sold` +- Input 3: `Read Opening Rows` +- Mode: **Combine** → **All Input Data** (waits for all three to complete before continuing) + +Name this node **`Sync Closing Data`**. + +The Merge node output is not used directly — the next Code node reads from each source node by name using `$items()`. + +--- + +## Task 7: Closing run — calculate and update sheets + +**Files:** n8n UI + +- [ ] **Step 1: Add a Code node to calculate units sold and % sold** + +Add a **Code** node after `Sync Closing Data`: + +```javascript +// Merges closing stock, line items sold, and opening rows to produce final values. +// Uses $items() to access each data source by node name. + +const closingRows = $items("Get Closing Stock").map(i => i.json); +const lineItemsRows = $items("Get Line Items Sold").map(i => i.json); +const openingRows = $items("Read Opening Rows").map(i => i.json); + +// Build lookup maps keyed by variant_id (string for consistent comparison) +const lineItemsByVariant = {}; +for (const row of lineItemsRows) { + lineItemsByVariant[String(row.variant_id)] = Number(row.units_sold) || 0; +} + +const openingByVariant = {}; +for (const row of openingRows) { + openingByVariant[String(row.variant_id)] = row['Opening Stock']; +} + +const ocId = String($vars.order_cycle_id); +const ocCloses = new Date().toISOString().replace('T', ' ').split('.')[0]; + +return closingRows.map(row => { + const variantId = String(row.variant_id); + const isOnDemand = row.on_demand === true || row.on_demand === 't'; + const closingStock = row.stock_level; + const lineItemsSold = lineItemsByVariant[variantId] ?? 0; + const openingStockRaw = openingByVariant[variantId]; // undefined = mid-cycle addition + const isMidCycleAddition = openingStockRaw === undefined || openingStockRaw === ''; + + let openingDisplay, closingDisplay, unitsSold, percentSold; + + if (isOnDemand) { + openingDisplay = 'On Demand'; + closingDisplay = 'On Demand'; + unitsSold = lineItemsSold; + percentSold = '—'; + } else if (isMidCycleAddition) { + openingDisplay = '— (Added mid-cycle)'; + closingDisplay = closingStock ?? 0; + unitsSold = lineItemsSold; + percentSold = '—'; + } else { + const opening = Number(openingStockRaw); + const closing = Number(closingStock ?? 0); + openingDisplay = opening; + closingDisplay = closing; + unitsSold = opening - closing; + percentSold = opening === 0 ? '—' : `${Math.round((unitsSold / opening) * 100)}%`; + } + + return { + json: { + 'variant_id': variantId, + 'row_key': `${ocId}_${variantId}`, + 'Supplier': row.supplier ?? '', + 'Product': row.product ?? '', + 'Variant': row.variant ?? '(default)', + 'On Demand': isOnDemand ? 'Yes' : 'No', + 'Opening Stock': openingDisplay, + 'Closing Stock': closingDisplay, + 'Units Sold': unitsSold, + '% Sold': percentSold, + 'OC Closes': ocCloses + } + }; +}); +``` + +Name this node **`Calculate Sold`**. + +- [ ] **Step 2: Add a Google Sheets node to update the OC tab** + +Add a **Google Sheets** node after `Calculate Sold`: +- Resource: **Sheet** +- Operation: **Append or Update** +- Spreadsheet ID: `{{ $vars.google_sheet_id }}` +- Sheet Name: `{{ $vars.order_cycle_name }}` +- Column to Match On: `variant_id` +- Data Mode: **Auto-Map Input Data** + +Name this node **`Update OC Tab`**. + +This updates the existing row matched by `variant_id`, filling in Closing Stock, Units Sold, and % Sold. Mid-cycle additions (no existing row) are appended as new rows. + +- [ ] **Step 3: Add a Code node to enrich rows for the master sheet update** + +Add a **Code** node after `Update OC Tab`: + +```javascript +// Adds master-sheet-specific columns for updating the All Order Cycles tab. +const ocId = String($vars.order_cycle_id); +const ocName = String($vars.order_cycle_name); + +return items.map(item => ({ + json: { + 'order_cycle_id': ocId, + 'row_key': item.json.row_key, + 'Order Cycle': ocName, + 'OC Closes': item.json['OC Closes'], + 'variant_id': item.json.variant_id, + 'Supplier': item.json['Supplier'], + 'Product': item.json['Product'], + 'Variant': item.json['Variant'], + 'On Demand': item.json['On Demand'], + 'Opening Stock': item.json['Opening Stock'], + 'Closing Stock': item.json['Closing Stock'], + 'Units Sold': item.json['Units Sold'], + '% Sold': item.json['% Sold'] + } +})); +``` + +Name this node **`Enrich for Master Sheet (Closing)`**. + +- [ ] **Step 4: Add a Google Sheets node to update the master sheet** + +Add a **Google Sheets** node after `Enrich for Master Sheet (Closing)`: +- Resource: **Sheet** +- Operation: **Append or Update** +- Spreadsheet ID: `{{ $vars.google_sheet_id }}` +- Sheet Name: `All Order Cycles` +- Column to Match On: `row_key` +- Data Mode: **Auto-Map Input Data** + +Name this node **`Update Master Sheet`**. + +- [ ] **Step 5: Test the full closing run** + +With the opening snapshot already written (from Task 5 test), change `snapshot_type` to `closing` and execute manually. Verify: +- Closing stock is filled in on the OC tab +- Units Sold = Opening Stock − Closing Stock for stocked variants +- % Sold is calculated correctly (e.g. opening 40, closing 12 → 28 sold → 70%) +- On-demand variants show units sold from line items and `—` for % Sold +- Mid-cycle additions appear with `— (Added mid-cycle)` for Opening Stock +- `All Order Cycles` master sheet is updated with OC Closes and all calculated values + +--- + +## Task 8: Error handling + +**Files:** n8n UI + +- [ ] **Step 1: Wire up the empty-results error notification** + +Connect the **False** branch of `Has Variants?` (from Task 4 Step 3) to a **Send Email** (or **Slack**) node: +- To: your operations email / Slack channel +- Subject: `⚠ OC Stock Snapshot failed — no variants found` +- Body: + +``` +The opening stock snapshot for order cycle "{{ $vars.order_cycle_name }}" (ID: {{ $vars.order_cycle_id }}) returned no variants for hub ID {{ $vars.hub_id }}. + +Check that hub_id and order_cycle_id are correct and that the hub has outgoing exchange variants in this OC. +``` + +Name this node **`Notify: No Variants Found`**. + +- [ ] **Step 2: Add a global error workflow in n8n** + +In n8n: **Settings → Error Workflows**. Create a new workflow named **`OC Stock Snapshot — Error Handler`** that: +1. Receives the error trigger (Error Trigger node) +2. Sends a notification containing `{{ $json.execution.error.message }}` and `{{ $json.execution.url }}` + +Set this as the error workflow for `OC Stock Snapshot` (in workflow settings → Error Workflow). + +- [ ] **Step 3: Add a parallel warning for missing opening snapshot** + +`Calculate Sold` already handles missing opening rows via the `isMidCycleAddition` logic — the closing run completes regardless. The warning is a side-channel notification, not a flow gate (connecting a notification back into the main flow in n8n requires all inputs to complete, which would stall execution). + +Add a parallel branch from `Sync Closing Data`: +1. Add an **IF** node connected to `Sync Closing Data` (in parallel with `Calculate Sold`): + - Condition: `{{ $items("Read Opening Rows").length }}` **equals** `0` + - True: connect to a **Send Email/Slack** node with message: + + ``` + ⚠ OC Stock Snapshot closing run for "{{ $vars.order_cycle_name }}" found no opening snapshot in the sheet. All variants will be treated as mid-cycle additions and units sold will be sourced from line items only. + ``` + + - False: connect to a **No-Op** node (or leave disconnected) + +Name this node **`Has Opening Snapshot?`**. This branch runs in parallel with `Calculate Sold` and sends a warning without affecting the main flow. + +--- + +## Task 9: End-to-end test, export, and README + +**Files:** +- Create: `n8n/workflows/stock-snapshot.json` +- Create: `n8n/README.md` + +- [ ] **Step 1: Run a full end-to-end test with a real OC** + +Choose an order cycle that is either currently active or upcoming. Run: +1. Opening run (manually, or wait for scheduled trigger) — verify the OC tab and master sheet +2. Wait for at least one order to be placed in the OC +3. Closing run (manually) — verify closing stock, units sold, % sold + +Check edge cases: +- At least one on-demand variant exists in the OC → Units Sold from line items, % Sold = `—` +- At least one stocked variant with stock_level > 0 → Units Sold and % Sold calculated correctly +- At least one variant with opening_stock = 0 → % Sold = `—` + +- [ ] **Step 2: Export the workflow JSON** + +In n8n: open the `OC Stock Snapshot` workflow → **⋮ → Export → Download** + +Save the downloaded file to `n8n/workflows/stock-snapshot.json`. + +- [ ] **Step 3: Write the README** + +Create `n8n/README.md`: + +```markdown +# OC Stock Snapshot — n8n Workflow + +Snapshots product stock levels for a hub at OC open and close, writing results to Google Sheets. + +## Prerequisites + +- n8n v1.x instance +- Read-only PostgreSQL access to the OFN database +- Google Sheets OAuth2 access (see credential setup below) +- A Google Spreadsheet with an `All Order Cycles` tab + +## Import the workflow + +1. In n8n: Workflows → Import → upload `n8n/workflows/stock-snapshot.json` +2. Set up credentials (see below) + +## Credential setup + +### PostgreSQL +Settings → Credentials → New → PostgreSQL +- Name: `OFN PostgreSQL (read-only)` +- Use a read-only database role + +### Google Sheets +Settings → Credentials → New → Google Sheets OAuth2 API +- Name: `Google Sheets (OFN stock)` +- Follow the OAuth2 flow with the account that owns the target spreadsheet + +## Scheduling a new order cycle + +For each order cycle, create **two** scheduled trigger instances of the `OC Stock Snapshot` workflow: + +**Opening trigger** (set to fire at the OC's `orders_open_at` time): +``` +hub_id: +order_cycle_id: +order_cycle_name: +google_sheet_id: +snapshot_type: opening +``` + +**Closing trigger** (set to fire at the OC's `orders_close_at` time): +``` +hub_id: +order_cycle_id: +order_cycle_name: +google_sheet_id: +snapshot_type: closing +``` + +## Finding OC IDs + +In the OFN admin, the order cycle ID is visible in the URL: +`/admin/order_cycles//edit` + +The hub enterprise ID is visible at: +`/admin/enterprises//edit` + +## Google Sheet structure + +Each run creates or updates: +- **One tab per OC** (tab name = `order_cycle_name`) +- **`All Order Cycles` tab** — master sheet with all OCs + +Columns: `variant_id` (hidden), `row_key` (hidden), Supplier, Product, Variant, On Demand, Opening Stock, Closing Stock, Units Sold, % Sold + +To hide the `variant_id` and `row_key` columns: right-click column header → Hide column. +``` + +- [ ] **Step 4: Commit** + +```bash +git add n8n/ +git commit -m "feat: add n8n OC stock snapshot workflow and setup documentation" +``` diff --git a/docs/superpowers/specs/2026-06-12-n8n-stock-snapshot-as-built.md b/docs/superpowers/specs/2026-06-12-n8n-stock-snapshot-as-built.md new file mode 100644 index 00000000000..54b3a613373 --- /dev/null +++ b/docs/superpowers/specs/2026-06-12-n8n-stock-snapshot-as-built.md @@ -0,0 +1,386 @@ +# n8n OC Stock Snapshot — As-Built Reference + +**Date:** 2026-06-12 +**Status:** Built and tested + +This document captures the actual working implementation, including fixes discovered during the build that differ from the original design spec. + +--- + +## Two workflows (duplicates with different snapshot_type) + +- **OC Stock Snapshot — Opening** (cron: `58 16 * * 4`, fires Thursday 16:58 Toronto time) +- **OC Stock Snapshot — Closing** (cron: `0 17 * * 4`, fires Thursday 17:00 Toronto time) + +--- + +## Node structure + +``` +Trigger + → Base Config (Set node) + → Get Current OC (PostgreSQL) + → Config (Code node — merges Base Config + OC lookup) + → Is Opening Run? (IF node: snapshot_type == 'opening') + TRUE branch (opening): + → Get Opening Stock (PostgreSQL) + → Has Variants? (IF node) + → Format Opening Rows (Code) + → Create OC Tab (Google Sheets) + → Restore Row Data (Code) + → Write Opening Rows to OC Tab (Google Sheets — Append Row) + → Enrich for Master Sheet (Code) + → Append to Master Sheet (Google Sheets — Append Row) + FALSE branch (closing): + ├── Get Closing Stock (PostgreSQL) ──────┐ + ├── Get Line Items Sold (PostgreSQL) ─────→ Sync Closing Data (Merge) + └── Read Opening Rows (Google Sheets) ───┘ + → Calculate Sold (Code) + → Format for OC Tab (Code) + → Update OC Tab (Google Sheets — Append or Update Row, match: variant_id) + → Enrich for Master Sheet Closing (Code) + → Update Master Sheet (Google Sheets — Append or Update Row, match: row_key) +``` + +--- + +## Base Config (Set node) + +Fields (only these three — order_cycle_id and order_cycle_name come from DB lookup): + +| Field | Value | +|---|---| +| `hub_id` | Enterprise ID of the hub | +| `google_sheet_id` | Full Google Sheets URL | +| `snapshot_type` | `opening` or `closing` | + +--- + +## Get Current OC (PostgreSQL) + +**Opening workflow:** +```sql +SELECT oc.id AS order_cycle_id, oc.name AS order_cycle_name +FROM order_cycles oc +JOIN exchanges ex ON ex.order_cycle_id = oc.id + AND ex.receiver_id = {{ $('Base Config').first().json.hub_id }}::integer + AND ex.incoming = false +WHERE oc.orders_open_at BETWEEN NOW() AND NOW() + INTERVAL '30 minutes' +ORDER BY oc.orders_open_at ASC +LIMIT 1 +``` + +**Closing workflow:** +```sql +SELECT oc.id AS order_cycle_id, oc.name AS order_cycle_name +FROM order_cycles oc +JOIN exchanges ex ON ex.order_cycle_id = oc.id + AND ex.receiver_id = {{ $('Base Config').first().json.hub_id }}::integer + AND ex.incoming = false +WHERE oc.orders_close_at BETWEEN NOW() - INTERVAL '30 minutes' AND NOW() + INTERVAL '5 minutes' +ORDER BY oc.orders_close_at DESC +LIMIT 1 +``` + +--- + +## Config (Code node) + +Merges Base Config and OC lookup so all downstream nodes reference a single source: + +```javascript +const base = $('Base Config').first().json; +const oc = $('Get Current OC').first().json; + +return [{ + json: { + hub_id: base.hub_id, + google_sheet_id: base.google_sheet_id, + snapshot_type: base.snapshot_type, + order_cycle_id: String(oc.order_cycle_id), + order_cycle_name: oc.order_cycle_name + } +}]; +``` + +--- + +## Stock snapshot SQL (used in both Get Opening Stock and Get Closing Stock) + +Query Parameters: `{{ $('Config').first().json.hub_id }},{{ $('Config').first().json.order_cycle_id }}` + +```sql +SELECT + e_supplier.name AS supplier, + sp.name AS product, + COALESCE(sv.display_name, sv.sku, 'Default') AS variant, + sv.id AS variant_id, + COALESCE(vo.on_demand, false) AS on_demand, + COALESCE(vo.count_on_hand, ssi.count_on_hand) AS stock_level +FROM order_cycles oc +JOIN exchanges ex + ON ex.order_cycle_id = oc.id + AND ex.receiver_id = $1 + AND ex.incoming = false +JOIN exchange_variants ev ON ev.exchange_id = ex.id +JOIN spree_variants sv ON sv.id = ev.variant_id AND sv.deleted_at IS NULL +JOIN spree_products sp ON sp.id = sv.product_id AND sp.deleted_at IS NULL +JOIN enterprises e_supplier ON e_supplier.id = sv.supplier_id +LEFT JOIN variant_overrides vo + ON vo.variant_id = sv.id AND vo.hub_id = $1 AND vo.permission_revoked_at IS NULL +LEFT JOIN spree_stock_items ssi + ON ssi.variant_id = sv.id AND ssi.deleted_at IS NULL +WHERE oc.id = $2 +ORDER BY e_supplier.name, sp.name, sv.display_name +``` + +**Key fix from original design:** `sv.supplier_id` not `sp.supplier_id`, and `on_demand` does not exist on `spree_variants` — use `COALESCE(vo.on_demand, false)`. + +--- + +## Line items aggregate SQL (Get Line Items Sold — closing only) + +Query Parameters: `{{ $('Config').first().json.order_cycle_id }},{{ $('Config').first().json.hub_id }}` + +```sql +SELECT + li.variant_id, + SUM(li.quantity) AS units_sold +FROM spree_line_items li +JOIN spree_orders o ON o.id = li.order_id +WHERE o.order_cycle_id = $1 + AND o.distributor_id = $2 + AND o.state <> 'canceled' +GROUP BY li.variant_id +``` + +--- + +## Format Opening Rows (Code) + +Filters out variants with zero or null stock (on-demand variants always included): + +```javascript +return items + .filter(item => { + const isOnDemand = item.json.on_demand === true || item.json.on_demand === 't'; + const stockLevel = item.json.stock_level; + return isOnDemand || (stockLevel !== null && stockLevel > 0); + }) + .map(item => { + const { variant_id, supplier, product, variant, on_demand, stock_level } = item.json; + const isOnDemand = on_demand === true || on_demand === 't'; + const ocId = String($('Config').first().json.order_cycle_id); + const variantId = String(variant_id); + + return { + json: { + 'variant_id': variantId, + 'row_key': `${ocId}_${variantId}`, + 'Supplier': supplier ?? '', + 'Product': product ?? '', + 'Variant': variant ?? '(default)', + 'On Demand': isOnDemand ? 'Yes' : 'No', + 'Opening Stock': isOnDemand ? 'On Demand' : (stock_level ?? 0), + 'Closing Stock': '', + 'Units Sold': '', + '% Sold': '' + } + }; + }); +``` + +--- + +## Restore Row Data (Code) + +Required after Create OC Tab because the Google Sheets Create Sheet node does not pass input items through: + +```javascript +return $('Format Opening Rows').all(); +``` + +--- + +## Enrich for Master Sheet (Code — opening only) + +```javascript +const ocId = String($('Config').first().json.order_cycle_id); +const ocName = String($('Config').first().json.order_cycle_name).trim(); +const ocOpens = new Date().toISOString().replace('T', ' ').split('.')[0]; + +return items.map(item => ({ + json: { + 'order_cycle_id': ocId, + 'row_key': item.json.row_key, + 'Order Cycle': ocName, + 'OC Opens': ocOpens, + 'OC Closes': '', + 'variant_id': item.json.variant_id, + 'Supplier': item.json['Supplier'], + 'Product': item.json['Product'], + 'Variant': item.json['Variant'], + 'On Demand': item.json['On Demand'], + 'Opening Stock': item.json['Opening Stock'], + 'Closing Stock': '', + 'Units Sold': '', + '% Sold': '' + } +})); +``` + +--- + +## Calculate Sold (Code — closing only) + +```javascript +const closingRows = $('Get Closing Stock').all().map(i => i.json); +const lineItemsRows = $('Get Line Items Sold').all().map(i => i.json); +const openingRows = $('Read Opening Rows').all().map(i => i.json); + +const lineItemsByVariant = {}; +for (const row of lineItemsRows) { + lineItemsByVariant[String(row.variant_id)] = Number(row.units_sold) || 0; +} + +const openingByVariant = {}; +for (const row of openingRows) { + openingByVariant[String(row.variant_id)] = row['Opening Stock']; +} + +const ocId = String($('Config').first().json.order_cycle_id); +const ocName = String($('Config').first().json.order_cycle_name).trim(); +const ocCloses = new Date().toISOString().replace('T', ' ').split('.')[0]; + +return closingRows + .filter(row => { + const variantId = String(row.variant_id); + const hasOpeningRow = openingByVariant[variantId] !== undefined && openingByVariant[variantId] !== ''; + const hasSales = (lineItemsByVariant[variantId] ?? 0) > 0; + return hasOpeningRow || hasSales; + }) + .map(row => { + const variantId = String(row.variant_id); + const isOnDemand = row.on_demand === true || row.on_demand === 't'; + const closingStock = row.stock_level; + const lineItemsSold = lineItemsByVariant[variantId] ?? 0; + const openingStockRaw = openingByVariant[variantId]; + const isMidCycleAddition = openingStockRaw === undefined || openingStockRaw === ''; + + let openingDisplay, closingDisplay, unitsSold, percentSold; + + if (isOnDemand) { + openingDisplay = 'On Demand'; + closingDisplay = 'On Demand'; + unitsSold = lineItemsSold; + percentSold = '—'; + } else if (isMidCycleAddition) { + openingDisplay = '— (Added mid-cycle)'; + closingDisplay = closingStock ?? 0; + unitsSold = lineItemsSold; + percentSold = '—'; + } else { + const opening = Number(openingStockRaw); + const closing = Number(closingStock ?? 0); + openingDisplay = opening; + closingDisplay = closing; + unitsSold = opening - closing; + percentSold = opening === 0 ? '—' : `${Math.round((unitsSold / opening) * 100)}%`; + } + + return { + json: { + 'variant_id': variantId, + 'row_key': `${ocId}_${variantId}`, + 'order_cycle_id': ocId, + 'order_cycle_name': ocName, + 'Supplier': row.supplier ?? '', + 'Product': row.product ?? '', + 'Variant': row.variant ?? '(default)', + 'On Demand': isOnDemand ? 'Yes' : 'No', + 'Opening Stock': openingDisplay, + 'Closing Stock': closingDisplay, + 'Units Sold': unitsSold, + '% Sold': percentSold, + 'OC Closes': ocCloses + } + }; + }); +``` + +--- + +## Format for OC Tab (Code — closing only) + +Strips master-sheet-only fields before updating the OC tab: + +```javascript +return items.map(item => ({ + json: { + 'variant_id': item.json.variant_id, + 'row_key': item.json.row_key, + 'Supplier': item.json['Supplier'], + 'Product': item.json['Product'], + 'Variant': item.json['Variant'], + 'On Demand': item.json['On Demand'], + 'Opening Stock': item.json['Opening Stock'], + 'Closing Stock': item.json['Closing Stock'], + 'Units Sold': item.json['Units Sold'], + '% Sold': item.json['% Sold'] + } +})); +``` + +--- + +## Enrich for Master Sheet Closing (Code — closing only) + +```javascript +return items.map(item => ({ + json: { + 'order_cycle_id': item.json.order_cycle_id, + 'row_key': item.json.row_key, + 'Order Cycle': item.json.order_cycle_name, + 'OC Closes': item.json['OC Closes'], + 'variant_id': item.json.variant_id, + 'Supplier': item.json['Supplier'], + 'Product': item.json['Product'], + 'Variant': item.json['Variant'], + 'On Demand': item.json['On Demand'], + 'Opening Stock': item.json['Opening Stock'], + 'Closing Stock': item.json['Closing Stock'], + 'Units Sold': item.json['Units Sold'], + '% Sold': item.json['% Sold'] + } +})); +``` + +--- + +## Google Sheets node settings + +| Node | Operation | Sheet | Match Column | +|---|---|---|---| +| Create OC Tab | Create Sheet | `{{ $('Config').first().json.order_cycle_name.trim() }}` | — | +| Write Opening Rows to OC Tab | Append Row | `{{ $('Config').first().json.order_cycle_name.trim() }}` | — | +| Append to Master Sheet | Append Row | `All Order Cycles` | — | +| Update OC Tab | Append or Update Row | `{{ $('Config').first().json.order_cycle_name.trim() }}` | `variant_id` | +| Update Master Sheet | Append or Update Row | `All Order Cycles` | `row_key` | + +--- + +## Known edge cases + +| Case | Behaviour | +|---|---| +| Variant with zero stock at opening | Filtered out — not included in output | +| On-demand variant | Opening/Closing = "On Demand", Units Sold from line items, % Sold = — | +| Mid-cycle addition with sales | Opening = "— (Added mid-cycle)", Units Sold from line items | +| Mid-cycle addition with zero sales | Excluded (filtered out as no opening row and no sales) | +| Opening stock = 0 | % Sold = — | + +--- + +## To add a new hub + +Duplicate both workflows, update `hub_id` in `Base Config`, activate. diff --git a/docs/superpowers/specs/2026-06-12-n8n-stock-snapshot-design.md b/docs/superpowers/specs/2026-06-12-n8n-stock-snapshot-design.md new file mode 100644 index 00000000000..ffaff524763 --- /dev/null +++ b/docs/superpowers/specs/2026-06-12-n8n-stock-snapshot-design.md @@ -0,0 +1,188 @@ +# n8n Stock Snapshot Workflow — Design Spec + +**Date:** 2026-06-12 +**Status:** Approved + +## Overview + +An n8n workflow that snapshots product stock levels at the opening and closing of an OFN order cycle for a distributor/hub enterprise, writing results to Google Sheets. Outputs units sold and % sold per variant per order cycle. + +--- + +## Scope + +- **Enterprise role:** Distributor/hub only +- **Directionality:** Going forward only (no historical backfill) +- **Trigger:** Manually scheduled exact triggers, one pair (open + close) per order cycle +- **Output:** Google Sheets — one tab per OC plus a running master sheet + +--- + +## Data Model + +### Row grain + +Each row is a unique combination of: **supplier × product × variant × order cycle** + +### Stock source + +Stock level = `COALESCE(variant_overrides.count_on_hand, spree_stock_items.count_on_hand)` + +The `variant_overrides` record is keyed by `(variant_id, hub_id)` and represents the hub's override of the producer's base stock. If no override exists, fall back to `spree_stock_items.count_on_hand`. + +On-demand flag = `COALESCE(variant_overrides.on_demand, spree_variants.on_demand)` + +### Variant discovery + +Variants in scope for a given OC and hub are found via: + +``` +order_cycles + → exchanges (incoming = false, receiver_id = hub_id) + → exchange_variants + → spree_variants (deleted_at IS NULL) + → spree_products (deleted_at IS NULL, supplier_id → enterprises) +``` + +### Units sold (on-demand and mid-cycle additions) + +For variants where opening stock cannot be subtracted (on-demand, or added after OC opened): + +``` +SUM(spree_line_items.quantity) + WHERE spree_orders.order_cycle_id = $order_cycle_id + AND spree_orders.distributor_id = $hub_id + AND spree_orders.state NOT IN ('canceled') + GROUP BY spree_line_items.variant_id +``` + +--- + +## Google Sheet Structure + +### Per-OC tab (tab name = OC name) + +| variant_id *(hidden)* | Supplier | Product | Variant | On Demand | Opening Stock | Closing Stock | Units Sold | % Sold | +|---|---|---|---|---|---|---|---|---| +| 42 | Farm A | Tomatoes | 500g | No | 40 | 12 | 28 | 70% | +| 67 | Farm B | Lettuce | Head | Yes | On Demand | On Demand | 15 | — | +| 91 | Farm C | Herbs | Basil 50g | No | — (Added mid-cycle) | 8 | 5 | — | + +`variant_id` is written as the first column and hidden; the closing run uses it to match rows for updating. + +### Master sheet (tab name: `All Order Cycles`) + +| order_cycle_id *(hidden)* | variant_id *(hidden)* | Order Cycle | OC Opens | OC Closes | Supplier | Product | Variant | On Demand | Opening Stock | Closing Stock | Units Sold | % Sold | +|---|---|---|---|---|---|---|---|---|---|---|---|---| + +Same row grain as the OC tab. `order_cycle_id` and `variant_id` are written as hidden columns; the closing run matches on both to update the correct rows. + +--- + +## Workflow Architecture + +### Workflow variables (set per scheduled trigger pair) + +| Variable | Description | +|---|---| +| `hub_id` | Enterprise ID of the distributor | +| `order_cycle_id` | ID of the order cycle | +| `google_sheet_id` | Target Google Spreadsheet ID | +| `snapshot_type` | `opening` or `closing` | + +### Opening run (triggered at `orders_open_at`) + +1. **PostgreSQL node** — fetch all in-scope variants with current stock levels and on-demand flags +2. **Function node** — flag on-demand variants; format rows (on-demand stock shown as "On Demand") +3. **Google Sheets node** — create new tab named after the OC +4. **Google Sheets node** — write header row + all variant rows (opening stock filled; closing/sold/% blank) +5. **Google Sheets node** — append rows to master sheet (opening stock filled; closing/sold/% blank) + +### Closing run (triggered at `orders_close_at`) + +1. **PostgreSQL node** — fetch same variant list with current stock levels (closing snapshot) +2. **PostgreSQL node** — fetch `SUM(line_items.quantity)` grouped by `variant_id` for non-cancelled orders in this OC +3. **Google Sheets node** — read existing rows from the OC tab (to retrieve opening stock) +4. **Function node** — merge by `variant_id`: + - Stocked variant with opening row: units sold = opening − closing; % sold = units sold ÷ opening × 100 + - On-demand variant: units sold = line items sum; % sold = `—` + - Mid-cycle addition (no opening row): opening stock = `—`; units sold = line items sum; % sold = `—` +5. **Google Sheets node** — update OC tab rows with closing stock, units sold, % sold (rows matched by `variant_id` column, which is written but hidden in both sheets) +6. **Google Sheets node** — update master sheet rows with the same (rows matched by `variant_id` + `order_cycle_id`) + +--- + +## Calculations + +| Case | Units Sold | % Sold | +|---|---|---| +| Stocked variant, opening snapshot exists | Opening − Closing | Units Sold ÷ Opening × 100 | +| On-demand variant | Line items aggregate | — | +| Added mid-cycle (no opening snapshot) | Line items aggregate | — | +| Opening stock = 0 | Opening − Closing | — (avoid divide-by-zero) | + +--- + +## Error Handling + +| Scenario | Behaviour | +|---|---| +| DB query returns no variants | Workflow halts; n8n error workflow sends notification; no empty tab created | +| Closing run finds no opening rows in sheet | Writes a warning row to master sheet flagged `⚠ Opening snapshot missing`; continues with mid-cycle logic | +| Mid-cycle variant addition | Detected as variant present in closing DB result but absent from opening sheet rows; written with opening = `—`, units sold from line items | +| On-demand line items query returns null | Treated as 0 units sold | +| Opening stock = 0 for stocked variant | % sold shown as `—` | +| Google Sheets API rate limit | n8n built-in retry with exponential backoff | + +--- + +## SQL Queries + +### Stock snapshot query (used for both opening and closing runs) + +```sql +SELECT + e_supplier.name AS supplier, + sp.name AS product, + sv.display_name AS variant, + sv.id AS variant_id, + COALESCE(vo.on_demand, sv.on_demand) AS on_demand, + COALESCE(vo.count_on_hand, ssi.count_on_hand) AS stock_level +FROM order_cycles oc +JOIN exchanges ex + ON ex.order_cycle_id = oc.id + AND ex.receiver_id = $hub_id + AND ex.incoming = false +JOIN exchange_variants ev ON ev.exchange_id = ex.id +JOIN spree_variants sv + ON sv.id = ev.variant_id + AND sv.deleted_at IS NULL +JOIN spree_products sp + ON sp.id = sv.product_id + AND sp.deleted_at IS NULL +JOIN enterprises e_supplier + ON e_supplier.id = sp.supplier_id +LEFT JOIN variant_overrides vo + ON vo.variant_id = sv.id + AND vo.hub_id = $hub_id + AND vo.permission_revoked_at IS NULL +LEFT JOIN spree_stock_items ssi + ON ssi.variant_id = sv.id + AND ssi.deleted_at IS NULL +WHERE oc.id = $order_cycle_id +ORDER BY e_supplier.name, sp.name, sv.display_name; +``` + +### Line items aggregate query (closing run only) + +```sql +SELECT + li.variant_id, + SUM(li.quantity) AS units_sold +FROM spree_line_items li +JOIN spree_orders o ON o.id = li.order_id +WHERE o.order_cycle_id = $order_cycle_id + AND o.distributor_id = $hub_id + AND o.state <> 'canceled' +GROUP BY li.variant_id; +``` diff --git a/goat-agents/antipatterns.md b/goat-agents/antipatterns.md new file mode 100644 index 00000000000..d352dc2b35e --- /dev/null +++ b/goat-agents/antipatterns.md @@ -0,0 +1,162 @@ +# Open Food Network — AI Anti-Pattern Catalog + +Specific mistakes AI assistants commonly make when working on OFN. Kept separate from `AGENTS.md` so that file stays short. `AGENTS.md` links here. + +--- + +## AP-001: Using decorators for `app/models/spree/` files + +**What happens**: AI treats `app/models/spree/*.rb` files as Spree gem source and avoids editing them directly, instead creating decorator files (e.g., `app/models/spree/order_decorator.rb`) or using `class_eval`/`prepend` to avoid "touching Spree." + +**Why it's wrong**: OFN *owns* these files. They live under version control in `app/models/spree/` and are OFN's own significantly-forked versions — not the Spree gem. The actual Spree gem source lives in the bundle cache (not in `app/`). OFN is actively moving away from Spree, so these files are OFN code, not upstream gem code. + +**Correct approach**: Edit `app/models/spree/order.rb` (or whichever file) directly, like any other OFN model. + +```ruby +# CORRECT: edit directly in app/models/spree/order.rb +class Spree::Order < ApplicationRecord + def my_custom_method + # OFN-specific behavior here + end +end +``` + +Note: Changes to critical Spree-namespaced code (checkout, payments) still warrant extra care and an "ask first" before touching — but the mechanism is direct editing, not decorators. + +--- + +## AP-002: Using Docker for active development + +**What happens**: AI sets up OFN using Docker (`docker compose up`, `./docker/build`) and runs all commands via `docker compose run --rm web`. + +**Why it's wrong**: OFN development via Docker produces 20+ second page load times due to Docker's file system overhead and Rails' dev-mode code reloading. Native setup produces ~1-3 second loads. OFN's own `docker/README.md` states: *"Docker is not commonly used by developers at this time."* + +**Symptoms**: +- Every page request takes 15-25 seconds +- Development iteration is impractically slow +- `Completed 200 OK in 21861ms` in logs + +**Correct approach**: Use native Ruby setup via rbenv + `./script/setup`. See `setup.md`. + +--- + +## AP-008: Running Docker before native setup + +**What happens**: AI runs `./docker/build` or `docker compose up` before (or instead of) setting up natively. Later, `./script/setup` fails with `Permission denied` errors on `log/`, `tmp/`, `public/packs/`, and `node_modules/`. + +**Why it's wrong**: Docker runs as root. All files it creates are owned by root. The native `./script/setup` runs as the current user and cannot delete or write to root-owned files. + +**Symptoms**: +``` +rm: cannot remove 'public/packs/static/logo.png': Permission denied +rake aborted! Errno::EACCES: Permission denied @ rb_sysopen - log/development.log +error Error: EACCES: permission denied, unlink 'node_modules/.yarn-integrity' +``` + +**Correct approach**: Fix all file ownership before running native setup: +```bash +sudo chown -R $USER:$USER /path/to/openfoodnetwork +``` +Then re-run `./script/setup`. + +--- + +## AP-003: Modifying existing migration files + +**What happens**: AI edits an existing file in `db/migrate/` to change a column type, add an index, or fix a mistake. + +**Why it's wrong**: Other contributors have already run this migration. Modifying it doesn't change their database — they'd need to roll back and re-run, which is error-prone and breaks reproducibility. + +**Symptoms**: +- Developer's local DB diverges from schema +- CI passes, staging fails +- "Works on my machine" + +**Correct approach**: Create a new migration. +```bash +bundle exec rails generate migration AddIndexToOrdersEmail +``` + +--- + +## AP-004: Hardcoded strings + +**What happens**: AI writes user-facing strings directly in views or controllers. + +**Wrong**: +```ruby +flash[:notice] = "Order confirmed!" +``` + +**Why it's wrong**: OFN supports multiple languages. Hardcoded English strings break translations and fail string audit tooling. + +**Correct approach**: +```ruby +flash[:notice] = I18n.t('order.confirmed') +``` +Then add to `config/locales/en.yml`: +```yaml +en: + order: + confirmed: "Order confirmed!" +``` + +--- + +## AP-005: Mixing AngularJS and Stimulus code (or assuming Vue.js exists) + +**What happens**: AI writes Stimulus controller syntax inside an AngularJS template, or writes AngularJS `$scope`/`ng-` directives in a Stimulus context. Or AI assumes OFN uses Vue.js and writes Vue components. + +**Why it's wrong**: OFN has two distinct JS layers that cannot be mixed. There is **no Vue.js** in OFN — never has been. + +**How to identify which layer is in use**: +- AngularJS (legacy): `ng-` attributes, `$scope`, `angular.module()`, files in `app/assets/javascripts/` +- Stimulus (modern): `data-controller=`, `data-action=`, files in `app/webpacker/controllers/` or `app/components/` + +**Rule of thumb**: Bug fixes in existing AngularJS code should stay AngularJS. New features should use Stimulus + Turbo. + +--- + +## AP-006: Missing specs for changed behavior + +**What happens**: AI implements a feature or fixes a bug but doesn't update or add specs. + +**Why it's wrong**: OFN has strong test coverage requirements. PRs without tests are blocked by CI and will be rejected in review. + +**Correct approach**: For every behavior change: +1. Write or update a model spec in `spec/models/` +2. Write or update a system spec in `spec/system/` if the change has UI +3. Run: `bundle exec rspec spec/models/spree/order_spec.rb` + +--- + +## AP-009: Starting work on a claimed or updated issue + +**What happens**: AI plans work on an issue, then later starts coding without re-reading the issue — missing that another contributor has opened a PR, been assigned, or the scope has changed. + +**Why it's wrong**: Open source projects move fast. Between planning and coding, any of these can happen: +- Another contributor opens a PR (fixing the same thing in parallel = wasted effort) +- A maintainer assigns the issue to someone else +- The issue discussion changes the accepted approach +- The issue is closed as won't-fix or duplicate + +**Symptoms**: +- Opening a PR and seeing "closes the same issue as #XXXX" +- A maintainer commenting "someone is already working on this" +- Implementing an approach that the issue thread already rejected + +**Correct approach**: Immediately before writing the first line of code (not when planning — when *starting*), re-read the full issue including all comments, and check for open PRs: +```bash +gh pr list -R openfoodfoundation/openfoodnetwork --search "closes #ISSUE_NUMBER" +``` +If anything has changed, pause and reassess before proceeding. + +--- + +## AP-007: Assuming OFN uses Grape for its API + +**What happens**: AI adds API endpoints to a Grape DSL file or looks for `app/webpages/api/` as the API location. + +**Why it's wrong**: OFN does **not** use Grape. The API is standard `ActionController::API` at `app/controllers/api/v0/` and `app/controllers/api/v1/`. There is also a separate DFC API engine at `engines/dfc_provider/` — touch this only if working on DFC-specific features. + +**Correct approach**: Add or modify controllers in `app/controllers/api/v0/` or `v1/`, following the existing `ActionController::API` patterns there. diff --git a/goat-agents/contributing-workflow.md b/goat-agents/contributing-workflow.md new file mode 100644 index 00000000000..69cc38800f0 --- /dev/null +++ b/goat-agents/contributing-workflow.md @@ -0,0 +1,122 @@ +# OFN Contribution Workflow — forum → issue → PR → merge + +OFN has its own contributor documentation. **This file does not replace it — it points you +at it and summarises what an AI-assisted contributor needs to follow.** When OFN's own docs +and this file disagree, OFN's docs win. Read [`AGENTS.md`](AGENTS.md) for project conventions +and pitfalls; read this file for *process*. + +Canonical OFN sources (read these, don't just rely on this summary): + +- [`CONTRIBUTING.md`](https://github.com/openfoodfoundation/openfoodnetwork/blob/master/CONTRIBUTING.md) — the contributor guide +- [`GETTING_STARTED.md`](https://github.com/openfoodfoundation/openfoodnetwork/blob/master/GETTING_STARTED.md) — dev environment setup +- [`CODE_OF_CONDUCT.md`](https://github.com/openfoodfoundation/openfoodnetwork/blob/master/CODE_OF_CONDUCT.md) +- [Developer wiki](https://github.com/openfoodfoundation/openfoodnetwork/wiki) — best-practice pages linked below +- [Community forum (Discourse)](https://community.openfoodnetwork.org) — where features are discussed + +--- + +## 1. Discuss the idea first (forum / Discourse) + +OFN is a community-governed project. **New features and significant changes are vetted on the +forum before they become issues**, not in a pull request. + +- Forum: https://community.openfoodnetwork.org + - [Product Development](https://community.openfoodnetwork.org/c/product-dev/5) — the "backlog/icebox" where feature ideas are proposed and prioritised. The feature issue template **requires a link to a Product Development backlog item.** + - [Software Improvement](https://community.openfoodnetwork.org/c/software-impovement/21) *(the URL typo is the real slug)* + - [Support](https://community.openfoodnetwork.org/c/support/8) +- **Before proposing a "new feature":** check the forum and the [user guide](https://guide.openfoodnetwork.org/) — it may already exist as a workflow, or already be under discussion. +- **After your PR merges:** post back to the related forum thread with a link, to close the loop for the people who raised it. +- Be civil and patient — follow the [Code of Conduct](https://github.com/openfoodfoundation/openfoodnetwork/blob/master/CODE_OF_CONDUCT.md). Reviewers and testers are volunteers across many timezones. + +## 2. Write a good issue + +Use the right GitHub issue template (`.github/ISSUE_TEMPLATE/`) — don't open a blank issue: + +| Template | Use it for | +|----------|-----------| +| `bug_report.md` | Something is broken. Fill in Description, Expected vs Actual, **Steps to Reproduce**, screenshot/gif, environment, and a **Severity** label (see below). | +| `story-template.md` | A small, deliverable chunk of work. Use "As a … / On page … / I want …" plus **Acceptance Criteria & Tests** a tester can follow. | +| `feature-template.md` | A larger feature epic. Must link to its [Product Development backlog](https://community.openfoodnetwork.org/c/product-dev/5) item on the forum. | +| `tech-debt-template.md` | Refactors that aid future development. | + +**Bug severity** ([wiki](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Bug-severity)) — apply the right label: + +- `bug-s1` critical feature broken (checkout, payments, signup, login) +- `bug-s2` non-critical feature broken, no workaround +- `bug-s3` broken but a workaround exists +- `bug-s4` annoying but usable +- `bug-s5` minor, few users impacted + +**Picking an issue to work on:** new contributors should start from the +[Welcome New Developers board](https://github.com/orgs/openfoodfoundation/projects/5). Before +you start, run the "Before starting work on an issue" checks in [`AGENTS.md`](AGENTS.md) +(is it claimed? is there an open PR? has scope changed? is it still open?). + +## 3. Set up and make the change + +Full setup is in [`setup.md`](setup.md) and OFN's +[`GETTING_STARTED.md`](https://github.com/openfoodfoundation/openfoodnetwork/blob/master/GETTING_STARTED.md). +The contribution-specific git setup OFN expects: + +```bash +# Fork on GitHub, then: +git clone git@github.com:YOUR_USERNAME/openfoodnetwork.git +cd openfoodnetwork +git remote add upstream git@github.com:openfoodfoundation/openfoodnetwork.git +git fetch upstream master + +# Branch off upstream/master, named after the issue number: +git checkout -b 1234-short-description --no-track upstream/master +``` + +OFN convention: **`upstream` = main repo, `origin` = your fork.** Branch names start with the +issue number (e.g. `3727-first-credit-card-default`). + +Make the change test-first (OFN recommends TDD). Follow the project conventions and pitfalls in +[`AGENTS.md`](AGENTS.md) and these wiki pages: + +- [Code Conventions](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Code-Conventions) +- [Testing and Rspec Tips](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Testing-and-Rspec-Tips) +- [Internationalisation (i18n)](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Internationalisation-%28i18n%29) — only edit `config/locales/en.yml`; other locales are managed by Transifex + +## 4. Open the pull request + +Follow OFN's [Making a great pull request](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Making-a-great-pull-request) +and [Making a great commit](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Making-a-great-commit) guides: + +- **Keep the PR small and single-focus.** Maintainers will ask you to split a sprawling PR. +- **Tests live in the same commit as the code they cover.** Maintain a clean, readable history; rebase (not merge) to tidy it *before* review starts. +- **Fill in the [PR template](https://github.com/openfoodfoundation/openfoodnetwork/blob/master/.github/PULL_REQUEST_TEMPLATE.md)** completely: + - **What? Why?** with `Closes #ISSUE` + - **What should we test?** — concrete steps for the tester (the PR title becomes the release note) + - **Release notes / Changelog Category** checkbox (User facing / API / Technical only / Feature toggled) + - Dependencies and Documentation updates sections +- **Watch CI.** Tests + reviewdog/Rubocop run within ~10 min. Fix style/test failures yourself — don't make a maintainer ask. See [Continuous Integration](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Continuous-Integration). +- Confirm the fix in the browser and put screenshots in the **PR description** (see [`AGENTS.md`](AGENTS.md) Boundaries). + +## 5. Review → Test → Merge → Deploy + +Know the lifecycle ([The process of review, test, merge and deploy](https://github.com/openfoodfoundation/openfoodnetwork/wiki/The-process-of-review%2C-test%2C-merge-and-deploy)) so you don't expect an instant merge: + +1. **Review** — a PR needs approval from **two core-team reviewers**. Address feedback as *additional* commits (don't force-push once review has started — see [`AGENTS.md`](AGENTS.md) pitfall on history rewrites). +2. **Test** — most PRs go to a **Test Ready** stage and are tested on a staging environment by someone other than the author and reviewer. Make their job easy with a clear "What should we test". +3. **Merge & Deploy** — core developers merge to `master`. +4. **Release** — happens roughly **weekly**. + +**Your responsibility through all of this:** respond to reviewers within ~1 week (sooner if CI +is red or a reviewer is blocked), follow the work through to merge/close, and weigh feedback by +reviewer authority. See "Responding to PR feedback" and "Closing the loop" in [`AGENTS.md`](AGENTS.md). + +--- + +## Quick links + +| Stage | OFN doc | +|-------|---------| +| Discuss | [Community forum](https://community.openfoodnetwork.org) · [Product Dev backlog](https://community.openfoodnetwork.org/c/product-dev/5) | +| Conduct | [Code of Conduct](https://github.com/openfoodfoundation/openfoodnetwork/blob/master/CODE_OF_CONDUCT.md) | +| Issue | [Issue templates](https://github.com/openfoodfoundation/openfoodnetwork/tree/master/.github/ISSUE_TEMPLATE) · [Bug severity](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Bug-severity) · [Welcome New Devs board](https://github.com/orgs/openfoodfoundation/projects/5) | +| Setup | [GETTING_STARTED](https://github.com/openfoodfoundation/openfoodnetwork/blob/master/GETTING_STARTED.md) · [`setup.md`](setup.md) | +| Code | [CONTRIBUTING](https://github.com/openfoodfoundation/openfoodnetwork/blob/master/CONTRIBUTING.md) · [Code Conventions](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Code-Conventions) · [Testing & Rspec Tips](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Testing-and-Rspec-Tips) · [i18n](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Internationalisation-%28i18n%29) | +| PR | [PR template](https://github.com/openfoodfoundation/openfoodnetwork/blob/master/.github/PULL_REQUEST_TEMPLATE.md) · [Great PR](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Making-a-great-pull-request) · [Great commit](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Making-a-great-commit) · [CI](https://github.com/openfoodfoundation/openfoodnetwork/wiki/Continuous-Integration) | +| Lifecycle | [Review, test, merge & deploy](https://github.com/openfoodfoundation/openfoodnetwork/wiki/The-process-of-review%2C-test%2C-merge-and-deploy) | diff --git a/goat-agents/review-checklist.md b/goat-agents/review-checklist.md new file mode 100644 index 00000000000..fda91d2a0d5 --- /dev/null +++ b/goat-agents/review-checklist.md @@ -0,0 +1,85 @@ +# OFN Maintainer Review Checklist + +What OFN maintainers typically check in PR review. Use as a pre-flight before submitting. + +Based on OFN's CONTRIBUTING.md, PR review patterns, and known maintainer preferences. + +--- + +## Before opening a PR + +- [ ] All tests pass: `bundle exec rspec` (native — see AGENTS.md; Docker exists but is too slow for active work) +- [ ] No existing tests broken +- [ ] JavaScript tests pass: `yarn test` +- [ ] New/changed behavior covered by specs +- [ ] No debugging code left in (`binding.pry`, `debugger`, `console.log`, etc.) +- [ ] Migrations run cleanly: `bundle exec rails db:migrate` +- [ ] i18n keys added to `config/locales/en.yml` for any new strings +- [ ] PR description explains what changed and why (link to issue) + +--- + +## Code review checklist + +### Ruby / Rails patterns + +- [ ] Models in `app/models/spree/` are edited directly — OFN owns these files, no decorators +- [ ] No existing migration files modified — new migration created for schema changes +- [ ] User-facing strings use `I18n.t()` or `t()` helper +- [ ] No hardcoded environment values — use ENV variables with defaults +- [ ] ActiveRecord queries use scopes or class methods, not raw SQL where possible +- [ ] N+1 queries avoided (check with `bullet` gem output in development) +- [ ] No `rescue Exception` — rescue specific error classes + +### Testing + +- [ ] Model specs test the model's logic (not the controller) +- [ ] Feature specs test user-visible behavior via the UI +- [ ] Test uses factories (`FactoryBot`) for test data, not hardcoded fixtures +- [ ] Test data uses realistic values (not `"test test test"` or `nil` everywhere) +- [ ] Shared examples used where appropriate (not copy-pasted test blocks) + +### JavaScript + +- [ ] New frontend features use Stimulus + Hotwire/Turbo (not AngularJS, not Vue — there is no Vue in OFN) +- [ ] Existing AngularJS code: changes stay in AngularJS unless explicitly migrating +- [ ] Prefer Turbo Streams over CableReady (CableReady is planned for removal) +- [ ] Stimulus controllers placed in `app/webpacker/controllers/` (shared) or `app/components/*/` (component-scoped); naming `[name]_controller.js` +- [ ] Stimulus controllers have corresponding tests + +### Security + +- [ ] No raw SQL string interpolation (use parameterized queries) +- [ ] User input sanitized before rendering in views +- [ ] Permissions checked before action (authorization, not just authentication) +- [ ] No sensitive data logged + +--- + +## Common reasons for PR revision requests + +1. **Missing tests** — especially for feature specs covering new UI behavior +2. **Spree core files edited directly** — maintainers will ask to use decorators +3. **Missing i18n keys** — hardcoded strings are caught in CI/review +4. **Migration file modified** — must create a new migration +5. **PR too large** — OFN prefers focused, reviewable PRs; maintainers may ask to split +6. **No issue linked** — OFN uses GitHub issues; PRs should reference one +7. **N+1 queries** — especially in views that list orders or products +8. **Missing documentation** for new feature or API change + +--- + +## Performance notes + +- OFN serves producers and shoppers simultaneously — database queries in list views must be efficient +- The `/shop` path is high traffic — be cautious about adding queries there +- Background jobs (Delayed::Job) for anything that shouldn't block a web request + +--- + +## Tone and communication + +- OFN is an international open source project with contributors across many timezones +- If a reviewer asks a question, answer it directly before pushing changes +- Maintainers appreciate small, well-explained PRs over large "everything fixed" submissions +- If your PR has been waiting for review, a polite ping after 1 week is appropriate diff --git a/goat-agents/setup.md b/goat-agents/setup.md new file mode 100644 index 00000000000..ff221c7aec8 --- /dev/null +++ b/goat-agents/setup.md @@ -0,0 +1,162 @@ +# Open Food Network Dev Environment Setup + +Step-by-step runbook for OFN local development on **Ubuntu 24.04 x86_64**. + +> **Do not use Docker for active development.** First page loads take 20+ seconds inside +> Docker vs ~1-3 seconds natively. OFN's own docker/README.md notes Docker "is not commonly +> used by developers at this time." See Method 2 if you only want to run the app briefly. + +--- + +## Method 1: Native Ruby (recommended for contributors) + +### Prerequisites (system packages) + +Run once with sudo. A script at `script/install-system-deps.sh` would be useful here but does not yet exist in the OFN repo — worth contributing upstream. Install manually if missing: + +```bash +sudo apt-get update -qq +sudo apt-get install -y \ + git curl cmake \ + libssl-dev libreadline-dev zlib1g-dev \ + autoconf bison build-essential libyaml-dev \ + libncurses5-dev libffi-dev libgdbm-dev \ + libsqlite3-dev libxml2-dev libxslt1-dev \ + postgresql postgresql-contrib libpq-dev \ + redis-server \ + imagemagick libmagickwand-dev libvips-dev \ + wkhtmltopdf \ + chromium-browser chromium-chromedriver \ + ruby-foreman +``` + +> **cmake is required** by the `rugged` gem. It is not listed in OFN's contributor docs +> but `bundle install` will fail without it. + +Create the database user: +```bash +sudo -u postgres psql -c "CREATE USER ofn WITH SUPERUSER CREATEDB PASSWORD 'f00d';" +``` + +### rbenv + Ruby + +```bash +git clone https://github.com/rbenv/rbenv.git ~/.rbenv +git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build +echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc +echo 'eval "$(rbenv init - bash)"' >> ~/.bashrc +source ~/.bashrc + +rbenv install $(cat .ruby-version) +rbenv global $(cat .ruby-version) +``` + +### nodenv + Node + +```bash +git clone https://github.com/nodenv/nodenv.git ~/.nodenv +git clone https://github.com/nodenv/node-build.git ~/.nodenv/plugins/node-build +echo 'export PATH="$HOME/.nodenv/bin:$PATH"' >> ~/.bashrc +echo 'eval "$(nodenv init - bash)"' >> ~/.bashrc +source ~/.bashrc + +nodenv install $(cat .node-version) +nodenv global $(cat .node-version) +``` + +### Run the setup script + +```bash +./script/setup +``` + +This installs gems, yarn deps, creates/migrates the database, and seeds sample data. +Ends with: `WELCOME TO OPEN FOOD NETWORK! email: ofn@example.com / password: ofn123` + +### Start the dev server + +```bash +bundle exec foreman start +``` + +> Use `bundle exec foreman`, not plain `foreman`. Ubuntu ships a system `ruby-foreman` +> apt package that uses system Ruby and cannot find `bundle`. + +App at http://localhost:3000 + +--- + +## Method 2: Docker (not recommended for active development) + +Use this only if you need a quick one-off run and don't want to install Ruby. + +```bash +# Build and seed (first run: 15-20 min) +./docker/build + +# Start the server +./docker/server +``` + +App at http://localhost:3000 — expect 20+ second page load times. + +### If Docker was run before native setup + +Docker runs as root. Its files must be fixed before running `./script/setup`: + +```bash +sudo chown -R $USER:$USER /path/to/openfoodnetwork +``` + +--- + +## Running tests (native) + +```bash +# Single spec file +bundle exec rspec spec/models/spree/order_spec.rb + +# By line number +bundle exec rspec spec/models/spree/order_spec.rb:42 + +# System specs (browser — slower, needs chromium-chromedriver) +bundle exec rspec spec/system/ + +# Prepare test database (after schema changes) +bundle exec rake db:test:prepare +``` + +--- + +## Daily commands + +```bash +bundle exec foreman start # start dev server (./bin/rails s also works, Spring is faster) +./bin/rails console # Rails console (Spring-backed, faster than bundle exec) +./bin/rails db:migrate # run pending migrations +./bin/rails generate migration Name # create migration +./bin/setup # standard Rails post-pull command (gems + migrations + setup) +``` + +> `./bin/rails` uses Spring (process pre-loader) and is faster than `bundle exec rails` for repeated commands. + +--- + +## Common setup problems + +| Error | Cause | Fix | +|-------|-------|-----| +| `CMake is required to build Rugged` | cmake not installed | `sudo apt-get install cmake` | +| `Permission denied` on log/tmp/public | Docker left root-owned files | `sudo chown -R $USER:$USER /path/to/repo` | +| `bundle: not found` when running foreman | System apt foreman used | Use `bundle exec foreman start` | +| `ruby:X.Y.Z-alpineX.Y: not found` | Docker Hub tag missing | Update both `Dockerfile` and `.ruby-version` to match | +| `Your Ruby version is X but Gemfile specified Y` | Dockerfile + .ruby-version mismatch | Update `.ruby-version` to match Dockerfile | +| `yarn: command not found` | nodenv not initialized | Run `source ~/.bashrc` or open new terminal | + +--- + +## References + +- [OFN CONTRIBUTING.md](https://github.com/openfoodfoundation/openfoodnetwork/blob/main/CONTRIBUTING.md) +- [OFN Developer Wiki](https://github.com/openfoodfoundation/openfoodnetwork/wiki) +- [Spree Documentation](https://guides.spreecommerce.org/) diff --git a/lib/reporting/reports/bulk_coop/base.rb b/lib/reporting/reports/bulk_coop/base.rb index cfc72b4ace7..c86a5289ded 100644 --- a/lib/reporting/reports/bulk_coop/base.rb +++ b/lib/reporting/reports/bulk_coop/base.rb @@ -34,8 +34,7 @@ def order_permissions def report_line_items @report_line_items ||= Reporting::LineItems.new( order_permissions, - @params, - CompleteVisibleOrdersQuery.new(order_permissions).call + @params ) end diff --git a/spec/lib/reports/bulk_coop_report_spec.rb b/spec/lib/reports/bulk_coop_report_spec.rb index 47b0c46cd24..48599121a08 100644 --- a/spec/lib/reports/bulk_coop_report_spec.rb +++ b/spec/lib/reports/bulk_coop_report_spec.rb @@ -29,12 +29,12 @@ module BulkCoop expect(subject.table_items).to eq([li1]) end - it 'shows canceled orders' do + it 'excludes canceled orders' do o2 = create(:order, state: 'canceled', completed_at: 1.day.ago, order_cycle: oc1, distributor: d1) line_item = build(:line_item_with_shipment) o2.line_items << line_item - expect(subject.table_items).to include(line_item) + expect(subject.table_items).not_to include(line_item) end end @@ -47,16 +47,27 @@ module BulkCoop expect(subject.table_items).to eq([li1]) end - it 'shows canceled orders' do + it 'excludes canceled orders' do o2 = create(:order, state: 'canceled', completed_at: 1.day.ago, order_cycle: oc1, distributor: d1) line_item = build(:line_item_with_shipment) o2.line_items << line_item - expect(subject.table_items).to include(line_item) + expect(subject.table_items).not_to include(line_item) end end end + context "filtering by order state" do + it 'excludes cancelled orders' do + o2 = create(:order, state: 'canceled', completed_at: 1.day.ago, order_cycle: oc1, + distributor: d1) + cancelled_li = build(:line_item_with_shipment) + o2.line_items << cancelled_li + + expect(subject.table_items).not_to include(cancelled_li) + end + end + context "filtering by date" do it do user = create(:admin_user)