Skip to content

perf: cache program rule variable option-set mappings (2.41) - #24852

Draft
jason-p-pickering wants to merge 4 commits into
2.41from
perf/2.41-programrulevariable-optionset-cache
Draft

perf: cache program rule variable option-set mappings (2.41)#24852
jason-p-pickering wants to merge 4 commits into
2.41from
perf/2.41-programrulevariable-optionset-cache

Conversation

@jason-p-pickering

@jason-p-pickering jason-p-pickering commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Problem

DefaultProgramRuleEntityMapperService.getOptions(ProgramRuleVariable) called optionSet.getOptions() fresh on every tracker import that touched the owning program, for every option-set-backed rule variable with useCodeForOptionSet false. OptionSet.options has no L2 collection cache (deliberately, to avoid a different N+1), so every call re-queried optionvalue and re-populated the Option entity cache without ever reading from it — visible on a live instance as ehcache_puts_total{cache="Option"} climbing steadily against a flat ehcache_hits_total, correlated with ProgramRuleAction cache hits, while the option set itself hadn't changed in over a week.

Also directly visible in query-stats profiling as a SELECT ... FROM optionset WHERE optionsetid = ? fired once per rule-variable-mapping pass per option set, because any non-id method call on a Hibernate proxy (like .getOptions()) forces the proxy to fully initialize itself first.

Fix

Memoize the mapped (name, code) list per option set id via a new CacheProvider-backed cache. On a cache hit, the code now only calls optionSet.getId() — which Hibernate special-cases to never trigger proxy initialization — so the optionset row load is skipped entirely instead of just its options collection.

Cache invalidation

Option writes can arrive via DefaultOptionService, the generic metadata CRUD controller, or a metadata import — none of which funnel through a single call site this class could hook into directly to invalidate the cache itself. So invalidation happens at the Hibernate level instead: OptionCacheInvalidationListener is a PostCommit insert/update/delete listener for the Option entity, registered globally via OptionCacheInvalidationListenerConfigurer — the same registration pattern DeletedObjectListenerConfigurer already uses elsewhere in the codebase. On any Option write, it evicts just that option's owning option-set entry.

This is same-node only. A write committed on another node in a cluster isn't seen by this listener, so cross-node staleness still relies on the pre-existing 1h TTL as a backstop.

What this does not fix

Worth being explicit about scope: this only caches the OptionSetOption leg. It does not touch the separate DataElement proxy load that also shows up in the same query-stats profile (SELECT ... FROM dataelement WHERE dataelementid = ?) — toRuleVariable()/toMappedValueType() call prv.getDataElement().getUid()/.getValueType()/.hasOptionSet() unconditionally on every mapping pass, uncached, regardless of whether an option set is involved. That's a separate N+1 (no cache wraps toMappedProgramRuleVariables() as a whole) and isn't addressed here.

Testing

  • ProgramRuleEntityMapperServiceTest updated for the new CacheProvider constructor dependency (was relying on @InjectMocks passing null for anything unmocked, which broke once the constructor started calling cacheProvider.createProgramRuleVariableOptionsCache() eagerly — fixed with explicit construction stubbed with a real SimpleCacheBuilder-backed cache rather than a mock, since a mock stubbed after @InjectMocks already ran is too late).
  • New OptionCacheInvalidationListenerTest covers the listener's type/null-guard logic in isolation; mutation-tested (broke the instanceof Option check, confirmed all 5 tests fail correctly, restored).
  • Full dhis-service-program-rule module: 41 tests, 0 failures.

🤖 AI Assisted

jason-p-pickering and others added 2 commits August 10, 2026 12:45
DefaultProgramRuleEntityMapperService.getOptions(ProgramRuleVariable) called
optionSet.getOptions() fresh on every tracker import that touched the owning
program, for every option-set-backed rule variable with useCodeForOptionSet
false. OptionSet.options has no L2 collection cache (deliberately, to avoid
a different N+1), so every call re-queried optionvalue and re-populated the
Option entity cache without ever reading from it - visible on a live
instance as ehcache_puts_total{cache="Option"} climbing steadily against a
flat ehcache_hits_total, correlated with ProgramRuleAction cache hits, while
the option set itself hadn't changed in over a week.

Memoize the mapped (name, code) list per option set id via a new
CacheProvider-backed cache, matching this module's existing
programRuleVariablesCache pattern: TTL-bounded (1 hour) rather than
event-invalidated on Option/OptionSet writes, since option sets change far
less often than that in practice.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…r dependency

The prior commit added a CacheProvider constructor param and called
createProgramRuleVariableOptionsCache() eagerly in the constructor, but
left the test's @Injectmocks wiring untouched. Since no @mock CacheProvider
existed, Mockito's constructor injection passed null, so the constructor
threw an NPE for every test in the class (previously masked by running
against a stale .m2 build during initial verification).

Switch to explicit construction with a real SimpleCacheBuilder-backed
cache instead of a mock, since stubbing a mock's method after @Injectmocks
already ran is too late - the field value is already captured.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@jason-p-pickering
jason-p-pickering requested a review from a team August 11, 2026 11:20
jason-p-pickering and others added 2 commits August 11, 2026 13:38
The mapping cache added previously was TTL-only (1h), which meant an
edited/added/removed Option could take up to an hour to show up in
rule-variable evaluation. Option/OptionSet writes arrive via
DefaultOptionService, the generic metadata CRUD controller, or a
metadata import - none of which funnel through a single call site
DefaultProgramRuleEntityMapperService could hook into directly, so
invalidation has to happen at the Hibernate level instead.

Add OptionCacheInvalidationListener, a PostCommit insert/update/delete
listener for the Option entity, registered globally via
OptionCacheInvalidationListenerConfigurer - the same registration
pattern DeletedObjectListenerConfigurer already uses. This is same-node
only; a write committed on another node in a cluster still relies on
the existing 1h TTL as a backstop.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Hibernate's PostInsert/PostUpdate/PostDeleteEventListener interfaces
all extend Serializable, making this class transitively Serializable
even though it's just a Spring singleton registered with Hibernate's
EventListenerRegistry and never actually serialized. SonarQube flagged
the non-transient, non-serializable field (S1948); transient satisfies
that contract without changing runtime behaviour.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant