Fix/optionset preheat n1 master (#24850)

* feat: add valid-option-code storage to TrackerPreheat

Adds per-option-set valid-code and resolved-set tracking so validators
and the ASSIGN-action pipeline can check option codes without
materializing the full OptionSet.options collection.

* feat: add OptionValueSupplier, preheat only referenced option codes

Queries optionvalue directly for the (option set, code) pairs actually
referenced by the import payload, in chunks, instead of relying on
OptionSetMapper to hydrate entire option collections. Cost is bounded
by distinct codes referenced, not option set size.

Includes the NOSONAR suppression for the two-phase collect/query loop
(false positive: confirmed is populated by the query immediately
before it's read).

* feat: validate option codes against preheated OptionValueSupplier data

Switches validateOptionSet() to check TrackerPreheat's valid-code cache
instead of scanning OptionSet.getOptions(), with a diagnostic log.warn
when an option set never got resolved during preheat. Updates all
three call sites (tracked entity, enrollment, and event attribute/data
value validators) and their tests accordingly.

* perf: stop preheat from fully materializing OptionSet.options

OptionSetMapper no longer maps the options collection, which was
forcing a full lazy-collection load (and JSONB deserialization of
every option) for every OptionSet touched during tracker preheat.
Callers now go through OptionValueSupplier/TrackerPreheat instead.

* fix: resolve option codes introduced by program rule ASSIGN actions

ASSIGN actions can introduce data element/attribute values that were
never part of the original import payload, so OptionValueSupplier's
single preheat pass can't have seen their option codes. Runs the
assigned values back through OptionValueSupplier as a synthetic
TrackerObjects before validation, so option-set validation still sees
them as preheated.

Adds getter access to the executors' resolved data element/attribute
UID and value, and asserts no errors in the previously-unchecked
option-value ASSIGN warning test.

* test: remove redundant public visibility modifier in AttributeValidatorTest

SonarQube java:S5786 -- JUnit5 lifecycle methods don't need public visibility.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test: prove OptionValueSupplier can't validate a fabricated cross-option-set pair

The two-column IN/IN query in queryChunk() can return rows whose optionsetid
and code each satisfy one of the two independent IN-lists without the pair
itself ever having been requested. Correctness depends entirely on the
in-memory confirmed.contains(pair) check filtering those out before anything
is marked valid. Add a regression test that mocks a query result covering
that exact cross-contamination shape and confirms the fabricated pairs stay
unvalidated while a genuine pair still passes; verified by temporarily
weakening the check and watching this test fail.

* fix: use unnest parallel-array join in OptionValueSupplier

Switches the (optionsetid, code) lookup from IN (...) AND IN (...) to a
join against unnest(?::bigint[], ?::text[]), per @teleivo/@ameenhere's
review on #24850. The IN/IN form only sargs on the leading index column
once cardinality grows, downgrading `code` to a post-scan Filter; the
unnest join keeps both columns in the Index Cond via
optionvalue_unique_optionsetid_and_code regardless of scale.

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

Author: jason-p-pickering

dhis-2/dhis-test-integration/src/test/java/org/hisp/dhis/tracker/imports/programrule/ProgramRuleAssignActionTest.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2004-2022, University of Oslo
+ * Copyright (c) 2004-2026, University of Oslo
  * All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -308,6 +308,8 @@ void shouldImportWithWarningWhenDataElementOfTypeOptionWithValidValueIsAssignedB
     assertHasOnlyWarnings(importReport, E1308);
     if (!hasValidValue) {
       assertHasOnlyErrors(importReport, E1125);
+    } else {
+      assertNoErrors(importReport);
     }
   }
 

dhis-2/dhis-tracker/src/main/java/org/hisp/dhis/tracker/imports/bundle/DefaultTrackerBundleService.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2004-2022, University of Oslo
+ * Copyright (c) 2004-2026, University of Oslo
  * All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -34,25 +34,37 @@
 import com.google.common.collect.Lists;
 import java.util.Date;
 import java.util.List;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
 import java.util.stream.Stream;
 import javax.annotation.Nonnull;
 import lombok.RequiredArgsConstructor;
 import org.hisp.dhis.common.UID;
+import org.hisp.dhis.dataelement.DataElement;
 import org.hisp.dhis.feedback.ForbiddenException;
 import org.hisp.dhis.feedback.NotFoundException;
 import org.hisp.dhis.program.UserInfoSnapshot;
+import org.hisp.dhis.trackedentity.TrackedEntityAttribute;
 import org.hisp.dhis.tracker.imports.ParamsConverter;
 import org.hisp.dhis.tracker.imports.TrackerImportParams;
 import org.hisp.dhis.tracker.imports.bundle.persister.CommitService;
 import org.hisp.dhis.tracker.imports.bundle.persister.PersistenceException;
 import org.hisp.dhis.tracker.imports.bundle.persister.TrackerObjectDeletionService;
 import org.hisp.dhis.tracker.imports.bundle.persister.TrackerPersister.PersistResult;
+import org.hisp.dhis.tracker.imports.domain.Attribute;
+import org.hisp.dhis.tracker.imports.domain.DataValue;
+import org.hisp.dhis.tracker.imports.domain.Enrollment;
 import org.hisp.dhis.tracker.imports.domain.TrackerDto;
+import org.hisp.dhis.tracker.imports.domain.TrackerEvent;
 import org.hisp.dhis.tracker.imports.domain.TrackerObjects;
 import org.hisp.dhis.tracker.imports.notification.EntityNotifications;
 import org.hisp.dhis.tracker.imports.preheat.TrackerPreheat;
 import org.hisp.dhis.tracker.imports.preheat.TrackerPreheatService;
+import org.hisp.dhis.tracker.imports.preheat.supplier.OptionValueSupplier;
 import org.hisp.dhis.tracker.imports.programrule.ProgramRuleService;
+import org.hisp.dhis.tracker.imports.programrule.executor.enrollment.AssignAttributeExecutor;
+import org.hisp.dhis.tracker.imports.programrule.executor.event.AssignDataValueExecutor;
 import org.hisp.dhis.tracker.imports.report.PersistenceReport;
 import org.hisp.dhis.tracker.imports.report.TrackerTypeReport;
 import org.hisp.dhis.user.UserDetails;
@@ -77,6 +89,8 @@ public class DefaultTrackerBundleService implements TrackerBundleService {
 
   private final TrackerObjectDeletionService deletionService;
 
+  private final OptionValueSupplier optionValueSupplier;
+
   private final ObjectMapper mapper;
 
   @Nonnull
@@ -97,9 +111,96 @@ public TrackerBundle create(
   public TrackerBundle runRuleEngine(@Nonnull TrackerBundle trackerBundle) {
     programRuleService.calculateRuleEffects(trackerBundle, trackerBundle.getPreheat());
 
+    optionValueSupplier.preheatAdd(
+        collectRuleAssignedValues(trackerBundle), trackerBundle.getPreheat());
+
     return trackerBundle;
   }
 
+  /**
+   * Collects the values {@code ASSIGN} rule actions are going to apply, shaped as a synthetic
+   * {@link TrackerObjects} payload the {@link OptionValueSupplier} can resolve option codes from.
+   *
+   * <p>Rule engine validation rejects unknown option codes based on {@link
+   * TrackerPreheat#isValidOptionCode(Long, String)}, but {@code ASSIGN} actions can add or
+   * overwrite data values and attributes that were not in the original payload, so the codes they
+   * introduce were never resolved during preheat and valid data would be rejected with E1125.
+   *
+   * <p>The values are already final here: they are the rule engine's evaluated output, captured in
+   * the executors {@code calculateRuleEffects} just built, so nothing evaluated later can change
+   * them.
+   *
+   * <p>All assigned values are gathered onto a single synthetic event and enrollment. The supplier
+   * only looks at (data element, value) and (attribute, value) pairs, so which or how many real
+   * entities the values belong to does not matter.
+   */
+  private TrackerObjects collectRuleAssignedValues(TrackerBundle bundle) {
+    TrackerPreheat preheat = bundle.getPreheat();
+
+    Set<DataValue> assignedDataValues =
+        bundle.getEventRuleActionExecutors().values().stream()
+            .flatMap(List::stream)
+            .filter(AssignDataValueExecutor.class::isInstance)
+            .map(AssignDataValueExecutor.class::cast)
+            .map(executor -> toDataValue(preheat, executor))
+            .filter(Objects::nonNull)
+            .collect(Collectors.toSet());
+
+    List<Attribute> assignedAttributes =
+        bundle.getEnrollmentRuleActionExecutors().values().stream()
+            .flatMap(List::stream)
+            .filter(AssignAttributeExecutor.class::isInstance)
+            .map(AssignAttributeExecutor.class::cast)
+            .map(executor -> toAttribute(preheat, executor))
+            .filter(Objects::nonNull)
+            .toList();
+
+    return TrackerObjects.builder()
+        .events(
+            List.of(
+                TrackerEvent.builder()
+                    .event(UID.generate())
+                    .dataValues(assignedDataValues)
+                    .build()))
+        .enrollments(
+            List.of(
+                Enrollment.builder()
+                    .enrollment(UID.generate())
+                    .attributes(assignedAttributes)
+                    .build()))
+        .build();
+  }
+
+  /**
+   * Executors only know the target's UID, while the supplier looks metadata up by the payload's id
+   * scheme, so the identifier has to be converted the same way the executors themselves do when
+   * they apply the value.
+   */
+  private DataValue toDataValue(TrackerPreheat preheat, AssignDataValueExecutor executor) {
+    DataElement dataElement = preheat.getDataElement(executor.getDataElementUid().getValue());
+    if (dataElement == null) {
+      return null;
+    }
+
+    return DataValue.builder()
+        .dataElement(preheat.getIdSchemes().toMetadataIdentifier(dataElement))
+        .value(executor.getValue())
+        .build();
+  }
+
+  private Attribute toAttribute(TrackerPreheat preheat, AssignAttributeExecutor executor) {
+    TrackedEntityAttribute attribute =
+        preheat.getTrackedEntityAttribute(executor.getAttributeUid().getValue());
+    if (attribute == null) {
+      return null;
+    }
+
+    return Attribute.builder()
+        .attribute(preheat.getIdSchemes().toMetadataIdentifier(attribute))
+        .value(executor.getValue())
+        .build();
+  }
+
   @Nonnull
   @Override
   @Transactional

dhis-2/dhis-tracker/src/main/java/org/hisp/dhis/tracker/imports/config/TrackerPreheatConfig.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2004-2022, University of Oslo
+ * Copyright (c) 2004-2026, University of Oslo
  * All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -38,6 +38,7 @@
 import org.hisp.dhis.tracker.imports.preheat.supplier.EnrollmentsWithAtLeastOneEventSupplier;
 import org.hisp.dhis.tracker.imports.preheat.supplier.EventCategoryOptionComboSupplier;
 import org.hisp.dhis.tracker.imports.preheat.supplier.FileResourceSupplier;
+import org.hisp.dhis.tracker.imports.preheat.supplier.OptionValueSupplier;
 import org.hisp.dhis.tracker.imports.preheat.supplier.OrgUnitValueTypeSupplier;
 import org.hisp.dhis.tracker.imports.preheat.supplier.PreheatStrategyScanner;
 import org.hisp.dhis.tracker.imports.preheat.supplier.PreheatSupplier;
@@ -56,6 +57,7 @@ public class TrackerPreheatConfig {
   private final List<Class<? extends PreheatSupplier>> preheatOrder =
       List.of(
           ClassBasedSupplier.class,
+          OptionValueSupplier.class,
           DefaultsSupplier.class,
           TrackedEntityEnrollmentSupplier.class,
           EnrollmentsWithAtLeastOneEventSupplier.class,

dhis-2/dhis-tracker/src/main/java/org/hisp/dhis/tracker/imports/preheat/TrackerPreheat.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2004-2022, University of Oslo
+ * Copyright (c) 2004-2026, University of Oslo
  * All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -138,6 +138,42 @@ private Pair<String, Set<MetadataIdentifier>> toCategoryOptionComboCacheKey(
     return Pair.of(categoryCombo.getUid(), categoryOptions);
   }
 
+  /**
+   * Set of (option set id, option code) pairs confirmed to exist, populated by {@code
+   * OptionValueSupplier}. Only pairs actually referenced by the import payload are present here —
+   * this is not the full set of options for any given option set.
+   *
+   * <p>Preheated {@link org.hisp.dhis.option.OptionSet} instances deliberately carry no populated
+   * {@code options} collection (see {@code OptionSetMapper} for why). {@link
+   * #isValidOptionCode(Long, String)} and {@link #addValidOptionCode(Long, String)} are the
+   * supported way to check option code validity against preheated data. Do not call {@code
+   * OptionSet#getOptions()} on a preheated option set expecting real data — it is always empty.
+   */
+  private final Set<Pair<Long, String>> validOptionCodes = new HashSet<>();
+
+  /**
+   * Option sets {@code OptionValueSupplier} attempted to resolve. Lets callers tell "this code was
+   * checked against the database and does not exist" apart from "this option set was never resolved
+   * at all", which would be an internal bug rather than user error.
+   */
+  private final Set<Long> resolvedOptionSets = new HashSet<>();
+
+  public void addValidOptionCode(Long optionSetId, String code) {
+    this.validOptionCodes.add(Pair.of(optionSetId, code));
+  }
+
+  public boolean isValidOptionCode(Long optionSetId, String code) {
+    return this.validOptionCodes.contains(Pair.of(optionSetId, code));
+  }
+
+  public void addResolvedOptionSet(Long optionSetId) {
+    this.resolvedOptionSets.add(optionSetId);
+  }
+
+  public boolean isOptionSetResolved(Long optionSetId) {
+    return this.resolvedOptionSets.contains(optionSetId);
+  }
+
   /**
    * Check if a category option combo for given category combo and category options has been stored
    * using {@link #putCategoryOptionCombo}. Returns true if null and a non-null category option

dhis-2/dhis-tracker/src/main/java/org/hisp/dhis/tracker/imports/preheat/mappers/OptionSetMapper.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2004-2022, University of Oslo
+ * Copyright (c) 2004-2026, University of Oslo
  * All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -29,15 +29,19 @@
  */
 package org.hisp.dhis.tracker.imports.preheat.mappers;
 
-import java.util.List;
-import org.hisp.dhis.option.Option;
 import org.hisp.dhis.option.OptionSet;
 import org.mapstruct.BeanMapping;
 import org.mapstruct.Mapper;
 import org.mapstruct.Mapping;
-import org.mapstruct.Named;
 import org.mapstruct.factory.Mappers;
 
+/**
+ * {@code options} is deliberately left unmapped — mapping it would force Hibernate to fully
+ * materialize the {@code OptionSet.options} collection (including JSONB attribute values) on every
+ * preheat, regardless of how many option codes the import actually references. {@link
+ * org.hisp.dhis.tracker.imports.preheat.supplier.OptionValueSupplier} preheats only the specific
+ * {@code (option set, code)} pairs the payload references instead.
+ */
 @Mapper
 public interface OptionSetMapper extends PreheatMapper<OptionSet> {
   OptionSetMapper INSTANCE = Mappers.getMapper(OptionSetMapper.class);
@@ -47,9 +51,5 @@ public interface OptionSetMapper extends PreheatMapper<OptionSet> {
   @Mapping(target = "uid")
   @Mapping(target = "name")
   @Mapping(target = "code")
-  @Mapping(target = "options", qualifiedByName = "options")
   OptionSet map(OptionSet optionSet);
-
-  @Named("options")
-  List<Option> mapOptionValues(List<Option> options);
 }