Skip to content

new dev area for 15.0.7#8444

Open
bewest wants to merge 221 commits intomasterfrom
dev
Open

new dev area for 15.0.7#8444
bewest wants to merge 221 commits intomasterfrom
dev

Conversation

@bewest
Copy link
Copy Markdown
Member

@bewest bewest commented Mar 3, 2026

This initializes development after releasing 15.0.6.

bewest and others added 30 commits January 19, 2026 13:09
Update project documentation and startup script for Nightscout v15.0.4, including Replit specific configurations like INSECURE_USE_HTTP=true and webpack bundling.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 748e831e-92ae-4927-9569-47665f47f29c
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 6e1d3764-af75-4a2b-a75f-f7252955326d
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/7ea4278b-5c6c-4065-9cb8-f1013771318d/748e831e-92ae-4927-9569-47665f47f29c/FO8cDm7
Replit-Helium-Checkpoint-Created: true
Enhance `replit.md` to include comprehensive documentation on API endpoints (v1, v2, v3), authentication methods (API_SECRET, JWT), OpenAPI specs, Socket.IO channels, and environment variables.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 748e831e-92ae-4927-9569-47665f47f29c
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 759a1536-7b1b-4ce2-9972-168703ba4f7d
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/7ea4278b-5c6c-4065-9cb8-f1013771318d/748e831e-92ae-4927-9569-47665f47f29c/FO8cDm7
Replit-Helium-Checkpoint-Created: true
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 06715f81-f852-4156-8ab0-cf9e0aa3564d
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 79dc8047-74ae-47e7-85a6-8c7255d9603d
Create a new RFC document and associated JSON schemas defining an agentic control plane for automated insulin delivery systems, including event envelopes, capabilities models, delivery tracking, and conflict rules.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 06715f81-f852-4156-8ab0-cf9e0aa3564d
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: 010cdc4a-2c86-4d2e-9768-bf2769a89953
Replit-Helium-Checkpoint-Created: true
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 06715f81-f852-4156-8ab0-cf9e0aa3564d
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: a40997df-4db2-4767-9c35-abb942a5fca2
Replit-Helium-Checkpoint-Created: true
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 0dc64f83-dd07-4193-899d-2998f5eb61c8
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 31a7d26d-5532-4bb7-90f2-0c00bf18017e
Replit-Helium-Checkpoint-Created: true
Create a proposal document detailing the current state of the test suite, dependency analysis, and a phased strategy for modernization, including migrating client tests to Jest.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 0dc64f83-dd07-4193-899d-2998f5eb61c8
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: 560c5b85-bd12-4a58-bc27-d8f6612621a6
Replit-Helium-Checkpoint-Created: true
Update replit.md to include a new proposal for modernizing test infrastructure, detailing a three-phase strategy and critical dependency updates.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 0dc64f83-dd07-4193-899d-2998f5eb61c8
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 3fb6e201-20e1-4478-9e20-72faa517ec04
Replit-Helium-Checkpoint-Created: true
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 6bd7e260-440b-4cf7-bd4f-6c49f415222f
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 003afa1a-8cd2-4c59-81d5-c8169366a6cf
Replit-Helium-Checkpoint-Created: true
…ral goals

Revise the testing modernization proposal to include interview findings, a new three-track approach, and scope control guardrails.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 6bd7e260-440b-4cf7-bd4f-6c49f415222f
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: d3a329ea-f588-48b4-ab9f-72250d9f0d39
Replit-Helium-Checkpoint-Created: true
Revise testing modernization proposal to include UI modernization, architecture goals, and a three-track approach.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 6bd7e260-440b-4cf7-bd4f-6c49f415222f
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 6175529f-d563-44d9-a0d8-39ef3f387bd8
Replit-Helium-Checkpoint-Created: true
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 6faed36e-345f-45eb-8006-45dc67e8451e
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 8124194f-045a-4644-b523-2697d2284678
Replit-Helium-Checkpoint-Created: true
…ity measures

Corrects inaccuracies in the Security Audit, Architecture Overview, and Modernization Roadmap documentation regarding rate limiting, Node.js versions, and code examples.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 6faed36e-345f-45eb-8006-45dc67e8451e
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: 207079d8-1a78-4ade-ad5c-d33397f8c775
Replit-Helium-Checkpoint-Created: true
Add a comprehensive system audit documentation section to replit.md, detailing findings on rate limiting, dependencies, bundle size, and Node.js support, along with a list of 9 audit documents.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 6faed36e-345f-45eb-8006-45dc67e8451e
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: a4473413-4cff-4847-b61c-6fc7e4d66b0a
Replit-Helium-Checkpoint-Created: true
Modify docs/modernization-roadmap.md to replace time intervals with effort (low/high) and complexity (straightforward/complicated) descriptions for each phase and task.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 6faed36e-345f-45eb-8006-45dc67e8451e
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 8aa5ae46-3944-4b57-9c5f-7f7df957037e
Replit-Helium-Checkpoint-Created: true
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 6faed36e-345f-45eb-8006-45dc67e8451e
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 02e98f82-c851-43a4-b518-90eccf9df206
Replit-Helium-Checkpoint-Created: true
Update architecture, security, and modernization documentation to clarify existing brute-force protection, introduce OIDC/OAuth2 integration plans, and address rate limiting gaps.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 6faed36e-345f-45eb-8006-45dc67e8451e
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: 483af0ac-57e5-44ad-89c0-d2374643791a
Replit-Helium-Checkpoint-Created: true
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: 6faed36e-345f-45eb-8006-45dc67e8451e
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 0a9ae46c-4b3a-4953-b56d-f33d485ea783
Replit-Helium-Checkpoint-Created: true
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: d9c34e72-a841-40c0-9e5f-c5e839501957
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 151d4417-e878-49d8-8040-5dcaf8a7da08
Replit-Helium-Checkpoint-Created: true
Add new sections for Pagination Metadata, Date Format Normalization, and Observability Plan to the API query normalization proposal document, and renumber existing sections accordingly.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: d9c34e72-a841-40c0-9e5f-c5e839501957
Replit-Commit-Checkpoint-Type: intermediate_checkpoint
Replit-Commit-Event-Id: 9d148101-339f-436a-bbbe-56da2f911abb
Replit-Helium-Checkpoint-Created: true
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: d9c34e72-a841-40c0-9e5f-c5e839501957
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 147a1f33-7ada-4bb0-b6c6-9c89c8c0f7dc
Replit-Helium-Checkpoint-Created: true
Replace timeline-based phases with complexity and risk ratings in the proposal document.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: d9c34e72-a841-40c0-9e5f-c5e839501957
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: aef39c0d-7cd4-4b3c-87f7-303fcc84a898
Replit-Helium-Checkpoint-Created: true
…t_db_updates""

This reverts commit 2e65b1d.
Try to bring back #8026.
Provide some tests for the profile API.
Refactor `devicestatus.js` to use `async.eachSeries` for sequential processing and update WebSocket `dbAdd` handler to process array inputs sequentially, ensuring correct multi-document write support.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: fbbf36df-818c-4b8c-8760-4975515f38e6
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 522116f1-cf99-4504-9c93-bc47b4276585
Replit-Helium-Checkpoint-Created: true
Replit-Commit-Author: Agent
Replit-Commit-Session-Id: fbbf36df-818c-4b8c-8760-4975515f38e6
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 8a0072a5-b8d3-4d98-977e-5f56d3d1d195
Replit-Helium-Checkpoint-Created: true
bewest and others added 17 commits March 17, 2026 17:11
fix: gate UUID_HANDLING on both write and read paths; fix doc accuracy
First describe block was missing its closing });

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NightscoutKit (Loop) sends profiles wrapped in arrays: [profile].
The MongoDB driver migration changed insert() to insertOne(), breaking
array support.

Changes:
- API layer: normalize input to array, purify each item
- Storage layer: use insertMany() instead of insertOne()
- Tests: verify single, array, and empty array handling

This matches the proven pattern from treatments API.

Fixes array handling regression introduced in d46c5b4.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The API layer was calling purifyObject() on the raw req.body without
handling arrays. When NightscoutKit sends [status], only the outer
array would be purified (no-op), not the individual status objects.

Now normalizes to array and purifies each devicestatus object,
matching the treatments pattern.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Extract profile structure from NightscoutKit Swift source:
- ProfileSet.swift: dictionaryRepresentation structure
- LoopSettings.swift: loopSettings with overrides, dosing params
- TemporaryScheduleOverride.swift: override presets format

Fixtures include:
- minimal: Minimal valid profile
- fullLoop: Full profile with loopSettings and schedules
- withActiveOverride: Profile with active schedule override
- mmol: mmol/L units variant
- singleArray: Single profile in array (most common case)
- batchArray: 3 profiles for batch upload testing
- generateUniqueProfile(): Helper for dedup testing

NightscoutKit always sends profiles as arrays (NightscoutClient.swift:404).

Sources:
  externals/NightscoutKit/Sources/NightscoutKit/Models/ProfileSet.swift
  externals/NightscoutKit/Sources/NightscoutKit/Models/LoopSettings.swift
  externals/NightscoutKit/Sources/NightscoutKit/Models/TemporaryScheduleOverride.swift

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Extract devicestatus structure from NightscoutKit Swift source:
- DeviceStatus.swift: top-level structure with device, created_at, identifier
- LoopStatus.swift: name, version, iob, cob, predicted, enacted, failureReason
- PumpStatus.swift: pumpID, manufacturer, model, reservoir, battery
- IOBStatus.swift: iob, basaliob with timestamp
- COBStatus.swift: cob with timestamp
- PredictedBG.swift: startDate, values array, optional COB/IOB curves
- LoopEnacted.swift: rate, duration (minutes), received, bolusVolume
- UploaderStatus.swift: name, battery percentage
- BatteryStatus.swift: percent, voltage, status

Fixtures include:
- minimal: Just device and timestamp
- loopWithIOBCOB: Loop status with IOB/COB
- fullLoop: Complete status with pump, predictions, enacted
- withEnacted: Status with active temp basal
- withFailure: Status with failureReason
- withAutoBolus: Status with automatic dose recommendation
- withMultiplePredictions: COB and IOB prediction curves
- singleArray: Single status in array format
- batchArray: 3 statuses for batch upload testing
- helpers: Factory functions for building custom fixtures

NightscoutKit sends devicestatus as arrays (NightscoutClient.swift:646-647).

Sources:
  externals/NightscoutKit/Sources/NightscoutKit/Models/DeviceStatus.swift
  externals/NightscoutKit/Sources/NightscoutKit/Models/LoopStatus.swift
  externals/NightscoutKit/Sources/NightscoutKit/Models/PumpStatus.swift
  externals/NightscoutKit/Sources/NightscoutKit/Models/IOBStatus.swift
  externals/NightscoutKit/Sources/NightscoutKit/Models/COBStatus.swift
  externals/NightscoutKit/Sources/NightscoutKit/Models/PredictedBG.swift
  externals/NightscoutKit/Sources/NightscoutKit/Models/LoopEnacted.swift

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Extract treatment fixture patterns from NightscoutKit Swift source:
- BolusNightscoutTreatment (Correction Bolus)
- CarbCorrectionNightscoutTreatment (Carb Correction)
- TempBasalNightscoutTreatment (Temp Basal)
- OverrideTreatment (Temporary Override)
- MealBolusNightscoutTreatment (Meal Bolus)
- BGCheckNightscoutTreatment (BG Check)
- NoteNightscoutTreatment (Note)
- Site Change, Sensor Start

Includes:
- Individual treatment objects for each type
- Array-wrapped formats (what NightscoutKit actually sends)
- Batch upload arrays for testing
- Helper functions for generating unique fixtures
- syncIdentifier field handling (client-provided dedup ID)

Source files:
- externals/NightscoutKit/Sources/NightscoutKit/Models/Treatments/*.swift
- externals/NightscoutKit/Sources/NightscoutKit/NightscoutClient.swift

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Extends api.shape-handling.test.js with:

Profile API tests:
- Single profile object POST
- Single-element array POST (NightscoutKit format)
- Multi-element array POST (batch upload)
- Response count equals input count validation
- Empty array handling
- Response shape consistency (always returns array)

NightscoutKit Fixtures Integration tests:
- Bolus treatments with syncIdentifier
- Carb entries with absorption time
- Temp basal treatments
- Mixed batch arrays
- Loop devicestatus with IOB/COB
- Batch devicestatus arrays
- Loop profile with loopSettings
- Historical profile batch sync

Test matrix coverage per spec in profile-api-array-regression.md:
| API | Single | Array | Batch | Empty | Response=Array | _id present |
|-----|--------|-------|-------|-------|----------------|-------------|
| Treatments | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| DeviceStatus | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Entries | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Profile | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add validation for _id field in profile API:
- POST: validates each document's _id before storage
- PUT: validates _id format before update
- DELETE: validates _id parameter before removal

Accepts: undefined, null, or 24-character hex string
Rejects: UUIDs, short strings, numbers, objects with 400 Bad Request

This prevents 500 errors from BSONError when clients send
UUID-style _ids (e.g., NightscoutKit).

Tests added for all validation cases.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add validation for _id field in devicestatus API:
- POST: validates each document's _id before storage
- DELETE: validates _id parameter (allows wildcard '*')

Accepts: undefined, null, or 24-character hex string
Rejects: UUIDs, short strings, numbers, objects with 400 Bad Request

Previously, invalid _id values were silently stored as strings instead
of ObjectIds, causing inconsistent data and query issues.

Tests added for all validation cases.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add validation for _id field in activity and food APIs:
- activity: POST, PUT, DELETE now validate _id format
- food: POST, PUT, DELETE now validate _id format

Accepts: undefined, null, or 24-character hex string
Rejects: UUIDs, short strings, numbers, objects with 400 Bad Request

Previously:
- activity: 500 crash on invalid _id in save/remove
- food: silently replaced invalid _id with new ObjectId (data loss)

Tests added covering all validation cases.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add array normalization to food API POST endpoint:
- API layer: normalize single object to array (like activity/profile)
- Storage layer: use replaceOne loop with upsert (same as activity pattern)
- Storage layer: accept both single object and array for backward compat

Previously POST /api/food/ with array input would crash:
  insertOne([{...}]) → MongoDB error

Now supports both single object and array input consistently.
Response format is now array (matching treatments pattern).

Fixes #8447

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace sequential replaceOne calls with bulkWrite for batch upserts.
This improves performance when inserting multiple documents at once.

- activity.js: Use bulkWrite with replaceOne ops instead of forEach loop
- food.js: Same optimization for consistency

Both maintain exact same upsert behavior (query by _id+created_at or doc).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add tests for POST with array of foods
- Add tests for PUT with array of foods
- Add test for empty array returning empty array
- Fix PUT endpoint to normalize array input like POST
- Fix food.save() storage to handle arrays with bulkWrite
- Rename test file to follow *.test.js convention

Validates fix from ef7bff3 for complete array handling.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add test for single activity returns array with one item
- Add test for activity array returns array
- Add test for empty array returns empty array
- Rename test file to follow *.test.js convention

Validates array normalization behavior for activity API.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add test for single entry returns array with one item
- Add test for empty array returns empty result
- Validates response format consistency for entries API

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wip/test improvements - extends same fixes for data shape across all remaining api surface areas and includes test coverage across the test matrix spectrum.
bewest and others added 5 commits March 18, 2026 19:49
Adds multi-layer protection against running destructive tests on production:

1. Pre-flight check (hooks.js): Verifies NODE_ENV=test before any DB connection
2. Database name check: Requires 'test' substring in database name
3. Entry count threshold: Refuses if database has >100 entries (configurable)

Environment Variables:
- TEST_SAFETY_MAX_ENTRIES: Max entries before refusing (default: 100)
- TEST_SAFETY_REQUIRE_TEST_DB: Require 'test' in DB name (default: true)
- TEST_SAFETY_SKIP: Emergency bypass for all checks (default: false)

Files:
- tests/lib/production-safety.js: Core safety check module
- tests/00_production-safety.test.js: Runs first to gate test suite
- tests/production-safety.test.js: Unit tests for safety module
- tests/hooks.js: Updated to use new module

This addresses concerns about users with 'test' in production DB names
by adding the entry count threshold as a secondary safety measure.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The ctx.entries module isn't always available depending on boot context.
Access the entries collection directly via ctx.store.db.collection().

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Change 'FAILED' to 'ACTIVATED' - this is protective, not a failure
- Explain that tests WILL DELETE data in the database
- Explain the purpose: preventing accidental production data loss
- List all override options clearly

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bewest
Copy link
Copy Markdown
Member Author

bewest commented Mar 19, 2026

The purpose of this release is to continue ratcheting node engines up to modern LTS versions, as well as enable continued updates to the storage layer. These changes were started by Sulka 3 years ago and are needed to continue maintenance work. We've implemented hundreds of tests capturing behavior across AAPS, Trio, Loop, tracking differences in usage of NightscoutKit, and xdrip+ among many other apps. We've also tested for single object vs array data shape across entries, treatments, profiles, devicestatus, food, and activity APIs. Finally, we during the course of integration testing we found that we needed to mitigate against invalid inputs to ObjectId() for entries and treatments, specifically.
I was able to extract many test fixtures after analysis of other open source apps with additional tooling and prepare additional analysis available here:

So far the full integration testing is uneventful for me.
What is left to do:

  • more integration testing: are things working with your favorite apps?
  • Is the documentation satisfactory? Env variables?
  • Changed behavior: the tests are destructive to the test database. NODE_ENV=test is required, as well as some arbitrary name checks for "test", as well bail if there are more than 100 entries in the database.

We added a quirk to handle the cases where Loop sends UUID to _id field in override treatments, as well as where Trio sends UUID to _id field for entries. These are intended to help address non-working behavior and potentially assist in using objectIds, letting Nightscout set the _id, or using a different field for their client UUID.

bewest and others added 6 commits March 19, 2026 17:31
Inserts a treatment directly into MongoDB with UUID as _id (no identifier
field) — the shape of overrides created before normalizeTreatmentId(). All 3
tests fail as expected:

- DELETE: responds 200 but deletedCount=0 (silent no-op)
- PUT: creates duplicate document instead of updating in place
- GET: returns 0 results (query rewrite misses legacy doc)

These tests document the legacy data gap and will pass once updateIdQuery()
is updated to use a $or fallback: {identifier: UUID} || {_id: UUID}.

Relates to: #6923, #8450

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
MongoDB driver 5.x ObjectId has no enumerable properties, so
_.isEmpty(new ObjectId()) returns true. This breaks cache.js

7 tests confirm:
- _.isEmpty(ObjectId) returns true (regression)
- filterForAge rejects ObjectId _id documents
- processRawDataForRuntime mitigates by converting to string
- Proposed fix (_id != null) works correctly

Currently mitigated in V1 paths by processRawDataForRuntime, but
unmitigated in API V3 mongoCachedCollection path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
_.isEmpty(new ObjectId()) returns true on driver 5.x because ObjectId
no longer has enumerable own properties (Object.keys returns []).
This caused filterForAge() to silently drop all entries with ObjectId
_id from the server cache.

check. This correctly accepts ObjectId instances and strings while
still rejecting null, undefined, and empty string.

16 tests confirm the fix:
- 3 root cause tests (_.isEmpty regression)
- 7 filterForAge logic tests (old vs fixed behavior)
- 6 integration tests (actual cache.js with data-update events)

Fixes AAPS backfill display bug where entries were in MongoDB but
invisible on the chart when using API V3 (mongoCachedCollection
emits data-update with raw ObjectId _id).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Strategy C: leading-edge debounce + concurrency guard.

- First data-received fires dataloader immediately (zero delay)
- Rapid events (AAPS batch upload) coalesced by 1s debounce
- Concurrency guard prevents overlapping dataloader.update() on shared ddata
- maxWait: 5s ensures data appears within 5s under sustained load
- Pending flag guarantees one final re-run after burst completes

Without this, N uploads → N concurrent dataloader runs → N×9 MongoDB
queries racing on the same ddata object. With it: N uploads → 2-3 runs.

9 tests confirm: leading-edge, coalescing (50 events → 2 runs),
no overlapping runs, trailing edge, maxWait guarantee.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
updateIdQuery() and upsertQueryFor() now use
  {$or: [{identifier: UUID}, {_id: UUID}]}
instead of only {identifier: UUID}. This matches both:
- New documents (UUID in identifier field, ObjectId in _id)
- Legacy documents (UUID directly in _id, no identifier field)

Gated behind env.uuidHandling (UUID_HANDLING env var, default true).

All 30 treatment tests pass:
- 3 legacy UUID tests (issue-6923): DELETE, PUT, GET all work
- 12 gap-treat-012 tests: new data paths unaffected
- 15 uuid-handling tests: edge cases, UUID_HANDLING=false still works

Fixes #6923 (unable to edit/save/delete overrides for legacy data)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants