Skip to content

SAK-52336 Samigo improve bulk delete efficiency#14364

Open
ottenhoff wants to merge 6 commits intosakaiproject:masterfrom
ottenhoff:SAK-52336
Open

SAK-52336 Samigo improve bulk delete efficiency#14364
ottenhoff wants to merge 6 commits intosakaiproject:masterfrom
ottenhoff:SAK-52336

Conversation

@ottenhoff
Copy link
Contributor

@ottenhoff ottenhoff commented Feb 6, 2026

Summary by CodeRabbit

  • Bug Fixes

    • Deletions now run as batched operations with robust validation, null checks, grouped cleanup of locks, calendar events, tasks/notifications, and improved error handling.
  • Refactor

    • Assessment and published-assessment removal reworked into a typed, batched, guarded flow to improve reliability and reduce per-item operations and redundant event postings.
  • New Features

    • Background scheduling for bulk removal of assessment-related search index documents; single consolidated unindexing events instead of per-item postings.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 6, 2026

Walkthrough

Collects assessment and published-assessment targets, then performs batched removals, group-unlocks, calendar/task cleanups, event postings, and notification updates; changes published-assessment authorization to use settings-only fetch with null checks; adds assessment-aware bulk document deletion scheduling to the question search index builder.

Changes

Cohort / File(s) Summary
Assessment removal listeners
samigo-app/src/java/org/sakaiproject/tool/assessment/ui/listener/author/RemoveAssessmentListener.java, samigo-app/src/java/org/sakaiproject/tool/assessment/ui/listener/author/RemovePublishedAssessmentListener.java
Refactored deletion flow to collect targets into typed batched collections (assessments, published IDs, release-group mappings, calendar event IDs, error tracking) and perform grouped removals and cleanups. Added helper methods (collectGroupLockRemovals, clearGroupDeletionLocks, updateInactivePublishedAssessments), null checks, and moved to typed collections. Changed isUserAllowedToDeletePublishedAssessment(AuthorBean, PublishedAssessmentFacade) signature. RemovePublishedAssessmentListener now uses getSettingsOfPublishedAssessment(...) and consolidates UNINDEXITEM posting.
Search index builder — assessment deletes
samigo-impl/src/java/org/sakaiproject/samigo/search/QuestionElasticSearchIndexBuilder.java
Added assessment/published-assessment resource tokens and path prefixes, a synchronized pendingAssessmentDeletes set, and methods deleteAllDocumentForAssessment(...) and scheduleDeleteAllDocumentForAssessment(...). DELETE handling now schedules bulk deletion for assessment-prefixed resources (with duplicate-schedule guard) and maps resource names containing assessment IDs to standardized assessment paths during validation/dispatch.
🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main objective of the pull request: improving the efficiency of bulk deletion operations in Samigo by refactoring deletion flow to use batched processing instead of per-item deletions.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Fix all issues with AI agents
In
`@samigo/samigo-app/src/java/org/sakaiproject/tool/assessment/ui/listener/author/RemoveAssessmentListener.java`:
- Around line 159-167: In RemoveAssessmentListener's loop over
deleteableAssessments, get the siteId by calling
assessmentService.getAssessmentSiteId(assessmentId) (or fallback to
AgentFacade.getCurrentSiteId()) before calling
assessmentService.removeAssessment(assessmentId), then use that stored siteId
when posting the EventTrackingService.post events and adding to
removedAssessmentIds; do not call getAssessmentSiteId after removeAssessment to
avoid null/exception on hard delete.

In
`@samigo/samigo-app/src/java/org/sakaiproject/tool/assessment/ui/listener/author/RemovePublishedAssessmentListener.java`:
- Around line 85-86: The call to
assessmentService.getSettingsOfPublishedAssessment(assessmentId.toString()) can
return null and will NPE when calling assessment.getCreatedBy(); update
RemovePublishedAssessmentListener to (1) call getSettingsOfPublishedAssessment
using assessmentId directly (remove the redundant .toString()), (2) check the
returned PublishedAssessmentFacade (e.g., variable assessment) for null and bail
out/continue the loop similarly to RemoveAssessmentListener before accessing
assessment.getCreatedBy(), and (3) keep existing logic unchanged otherwise.

In
`@samigo/samigo-impl/src/java/org/sakaiproject/samigo/search/QuestionElasticSearchIndexBuilder.java`:
- Around line 363-374: In deleteAllDocumentForAssessment change the log.error
call to use SLF4J parameterized logging instead of string concatenation; replace
the current concatenated message with something like log.error("Failed to remove
all documents from question index for assessment: {}", assessmentId, e) so the
assessmentId is a placeholder and the IOException (e) is passed as the throwable
to preserve stacktrace.
🧹 Nitpick comments (4)
samigo/samigo-app/src/java/org/sakaiproject/tool/assessment/ui/listener/author/RemoveAssessmentListener.java (4)

170-170: Redundant PublishedAssessmentService instantiation.

publishedAssessmentService is already created on line 94 and used on line 132. This creates a second unnecessary instance. Reuse the existing one.

Proposed fix
-            PublishedAssessmentService pubAssessmentService = new PublishedAssessmentService();
             for (String assessmentId : deleteablePublishedAssessmentIds) {
                 log.debug("assessmentId = {}", assessmentId);
 
-                pubAssessmentService.removeAssessment(assessmentId, "remove");
+                publishedAssessmentService.removeAssessment(assessmentId, "remove");

106-107: Misleading map variable name: groupIdsByAssessmentId actually maps groupId → Set<assessmentId>.

Looking at collectGroupUnlocks (line 262), the key inserted is groupId and the value set contains assessmentIds. The name groupIdsByAssessmentId implies the opposite mapping. This propagates through lines 106, 157, 177, 195, 271.

Consider renaming to assessmentIdsByGroupId for clarity.

Also applies to: 256-269


304-325: Use typed collections in new helper method.

Lines 309–310 use raw List types in newly added code. This is inconsistent with the typed collections used earlier in this PR (lines 99–107) and can be improved for type safety.

Proposed fix
     private void updateInactivePublishedAssessments(AuthorBean author, Set<String> removedPublishedAssessmentIds) {
         if (removedPublishedAssessmentIds.isEmpty()) {
             return;
         }
 
-        List inactivePublishedAssessmentList = author.getInactivePublishedAssessments();
-        List inactiveList = new ArrayList();
+        `@SuppressWarnings`("unchecked")
+        List<PublishedAssessmentFacade> inactivePublishedAssessmentList = (List<PublishedAssessmentFacade>) author.getInactivePublishedAssessments();
+        List<PublishedAssessmentFacade> inactiveList = new ArrayList<>();
         boolean isAnyAssessmentRetractForEdit = false;
 
-        for (int i = 0; i < inactivePublishedAssessmentList.size(); i++) {
-            PublishedAssessmentFacade pa = (PublishedAssessmentFacade) inactivePublishedAssessmentList.get(i);
+        for (PublishedAssessmentFacade pa : inactivePublishedAssessmentList) {
             if (!removedPublishedAssessmentIds.contains(pa.getPublishedAssessmentId().toString())) {
                 inactiveList.add(pa);
                 if (Integer.valueOf(3).equals(pa.getStatus())) {

256-269: Consider computeIfAbsent to simplify the map-population logic.

Simplification
     private void collectGroupUnlocks(String assessmentId, Set<String> selectedGroupIds, Map<String, Set<String>> groupIdsByAssessmentId) {
         if (selectedGroupIds == null || selectedGroupIds.isEmpty()) {
             return;
         }
 
         for (String groupId : selectedGroupIds) {
-            Set<String> assessmentIds = groupIdsByAssessmentId.get(groupId);
-            if (assessmentIds == null) {
-                assessmentIds = new HashSet<>();
-                groupIdsByAssessmentId.put(groupId, assessmentIds);
-            }
-            assessmentIds.add(assessmentId);
+            groupIdsByAssessmentId.computeIfAbsent(groupId, k -> new HashSet<>()).add(assessmentId);
         }
     }

Comment on lines 85 to 86
// get settings without loading sections/items
PublishedAssessmentFacade assessment = assessmentService.getSettingsOfPublishedAssessment(assessmentId.toString());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Missing null check for getSettingsOfPublishedAssessment result.

RemoveAssessmentListener (lines 132–135) guards against a null return from getSettingsOfPublishedAssessment with a continue. Here, no such guard exists — if the published assessment was already removed or the ID is stale, this will NPE on line 89 (assessment.getCreatedBy()).

Also, assessmentId is already a String (line 80), so .toString() on line 86 is redundant.

Proposed fix
       // get settings without loading sections/items
-      PublishedAssessmentFacade assessment = assessmentService.getSettingsOfPublishedAssessment(assessmentId.toString());
+      PublishedAssessmentFacade assessment = assessmentService.getSettingsOfPublishedAssessment(assessmentId);
+      if (assessment == null) {
+          log.warn("Published assessment not found for id {}", assessmentId);
+          return;
+      }
🤖 Prompt for AI Agents
In
`@samigo/samigo-app/src/java/org/sakaiproject/tool/assessment/ui/listener/author/RemovePublishedAssessmentListener.java`
around lines 85 - 86, The call to
assessmentService.getSettingsOfPublishedAssessment(assessmentId.toString()) can
return null and will NPE when calling assessment.getCreatedBy(); update
RemovePublishedAssessmentListener to (1) call getSettingsOfPublishedAssessment
using assessmentId directly (remove the redundant .toString()), (2) check the
returned PublishedAssessmentFacade (e.g., variable assessment) for null and bail
out/continue the loop similarly to RemoveAssessmentListener before accessing
assessment.getCreatedBy(), and (3) keep existing logic unchanged otherwise.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In
`@samigo/samigo-app/src/java/org/sakaiproject/tool/assessment/ui/listener/author/RemoveAssessmentListener.java`:
- Around line 171-194: The batch removal loop in RemoveAssessmentListener calls
publishedAssessmentService.removeAssessment and removeFromGradebook (and then
does cleanup like collectGroupUnlocks, calendarService.removeCalendarEvent,
taskService.removeTaskByReference,
samigoAvailableNotificationService.removeScheduledAssessmentNotification)
without per-iteration error handling, so one exception aborts the whole batch;
wrap the body of the loop that iterates deleteablePublishedAssessmentIds in a
try/catch around the sequence that calls
publishedAssessmentService.removeAssessment(assessmentId, "remove") and
removeFromGradebook(assessmentId), log the exception (including assessmentId)
and continue to the next assessment, and only add assessmentId to
removedPublishedAssessmentIds after successful completion of the cleanup steps;
apply the same per-iteration try/catch pattern to the earlier assessment removal
loop that calls publishedAssessmentService.removeAssessment and
removeFromGradebook so one failure doesn’t skip remaining assessments.
🧹 Nitpick comments (1)
samigo/samigo-impl/src/java/org/sakaiproject/samigo/search/QuestionElasticSearchIndexBuilder.java (1)

606-611: Chained if (not else if) for resource token rewriting is consistent with existing pattern but has a subtle ordering dependency.

Lines 600–611 use sequential if blocks, so each condition operates on the (potentially already-rewritten) resourceName. In practice the tokens are mutually exclusive in a single event resource, so this works. Flagging only because an else if chain would make the mutual exclusivity explicit and avoid a future maintenance pitfall.

}
}

private void unlockGroupsForDeletion(Map<String, Set<String>> assessmentIdsByGroupId) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When a group assessment is being removed it is correct to remove any locks on the group

  • the method name seems in correct because it's not unlocking the group it is simply removing a lock reference, is the group being deleted here, doesn't appear that way
  • if all that is happening is removing a lock reference, there probably is no need to save the site which is a heavy operation, just save the authz group if it was updated
  • also the authz group deals in references this appears to use an assessmentId? is this correct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Renamed methods to match behavior:
    - collectGroupLockRemovals(...) at .../RemoveAssessmentListener.java:291
    - clearGroupDeletionLocks(...) at .../RemoveAssessmentListener.java:301

    • Reduced heavy save cost:
      • Only call siteService.save(site) if at least one lock reference was actually changed (siteUpdated flag) at .../RemoveAssessmentListener.java:317 and .../RemoveAssessmentListener.java:332.
    • Kept lock key as assessment id intentionally:
      • We clear with group.setLockForReference(assessmentId, RealmLockMode.NONE) at .../RemoveAssessmentListener.java:326.
      • This matches how locks are created on publish with published assessment id at samigo/samigo-app/src/java/org/sakaiproject/tool/assessment/ui/listener/author/PublishAssessmentListener.java:278.

    I did not switch to direct authz-group save because that is a broader behavior change; current Samigo pattern already uses site save for these group lock updates, and this patch keeps that pattern while avoiding unnecessary saves.

Copy link
Contributor

@ern ern Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

example from assignments, again site.save is a very heavy operation in comparison:

        for (String groupReference : assignment.getGroups()) {
            try {
                AuthzGroup group = authzGroupService.getAuthzGroup(groupReference);
                group.setLockForReference(reference, AuthzGroup.RealmLockMode.NONE);
                authzGroupService.save(group);
            } catch (GroupNotDefinedException | AuthzPermissionException e) {
                log.warn("Exception while removing lock for assignment {}, {}", assignment.getId(), e.toString());
            }
        }

also I suggest:

  • clearRealmLocks or removeRealmLocks
  • getRealmLocksForRemoval

…t/ui/listener/author/RemoveAssessmentListener.java:

  - Added per-item failure isolation for draft deletions:
      - Core delete is wrapped in try/catch and continue on failure (:164-:170).
      - Event posting is best-effort with separate try/catch (:172-:177).
  - Added per-item failure isolation for published deletions:
      - Core delete path (removeAssessment + removeFromGradebook) is wrapped in try/catch and continue on failure (:184-:191).
      - removedPublishedAssessmentIds is updated after successful core delete (:187), not blocked by cleanup failures.
      - Cleanup operations are individually best-effort and logged:
          - group unlock collection (:193-:197)
          - calendar event removal (:201-:205)
          - event posts (:208-:213)
          - task removal (:217-:221)
          - scheduled notification removal (:223-:227)
      - collectGroupLockRemovals(...) at .../RemoveAssessmentListener.java:291
      - clearGroupDeletionLocks(...) at .../RemoveAssessmentListener.java:301
  - Reduced heavy save cost:
      - Only call siteService.save(site) if at least one lock reference was actually changed (siteUpdated flag) at .../RemoveAssessmentListener.java:317 and .../RemoveAssessmentListener.java:332.
  - Kept lock key as assessment id intentionally:
      - We clear with group.setLockForReference(assessmentId, RealmLockMode.NONE) at .../RemoveAssessmentListener.java:326.
      - This matches how locks are created on publish with published assessment id at samigo/samigo-app/src/java/org/sakaiproject/tool/assessment/ui/listener/author/PublishAssessmentListener.java:278.

  I did not switch to direct authz-group save because that is a broader behavior change; current Samigo pattern already uses site save for these group lock updates, and this patch keeps that pattern while avoiding unnecessary saves.
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In
`@samigo/samigo-app/src/java/org/sakaiproject/tool/assessment/ui/listener/author/RemoveAssessmentListener.java`:
- Around line 184-191: After successfully calling
publishedAssessmentService.removeAssessment(assessmentId, "remove") in
RemoveAssessmentListener, add removedPublishedAssessmentIds.add(assessmentId)
immediately (before calling removeFromGradebook) so the in-memory list reflects
the actual DB deletion even if removeFromGradebook or
SpringBeanLocator.getBean() throws; keep removeFromGradebook(assessmentId) and
subsequent cleanup logic the same but ensure any cleanup relying on
removedPublishedAssessmentIds still runs for that assessment.

…i/listener/author/RemoveAssessmentListener.java so that in the published-delete loop:

  - removedPublishedAssessmentIds.add(assessmentId) now happens immediately after successful:
      - publishedAssessmentService.removeAssessment(assessmentId, "remove")
  - and before:
      - removeFromGradebook(assessmentId)

  This ensures UI/in-memory removal tracking reflects actual DB deletion even if gradebook cleanup throws.
Comment on lines +83 to +84
private SiteService siteService = ComponentManager.get(SiteService.class);
private ToolManager toolManager = ComponentManager.get(ToolManager.class);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another improvement you could do is (although you don't have to) is have this class extend SpringBeanAutowiringSupport and then you can use @Autowired to wire up your services managed by spring, this removes the need for ComponentManager all together :).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants