Skip to content

o/devicemgmtstate: remove sequence entry after unsequenced message is processed - #17371

Merged
st3v3nmw merged 7 commits into
canonical:masterfrom
st3v3nmw:remove-seq-entry-after-unsequenced-message-is-processed
Jul 29, 2026
Merged

o/devicemgmtstate: remove sequence entry after unsequenced message is processed#17371
st3v3nmw merged 7 commits into
canonical:masterfrom
st3v3nmw:remove-seq-entry-after-unsequenced-message-is-processed

Conversation

@st3v3nmw

Copy link
Copy Markdown
Member

Once an unsequenced message completes the pipeline, its entry in Sequences was being left behind. removeRequestMessage kept entries intact to preserve Applied progress for sequenced messages, but unsequenced messages carry no such progress. Over time, these stale entries accumulate in state.

removeRequestMessage now deletes the sequence entry when the processed message is unsequenced and no other messages remain under that base ID.

@st3v3nmw
st3v3nmw requested a review from Copilot July 15, 2026 11:12
@st3v3nmw st3v3nmw added the Simple 😃 A small PR which can be reviewed quickly label Jul 15, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR prevents unbounded growth of devicemgmtstate persisted state by deleting Sequences[baseID] entries once an unsequenced request-message has fully completed processing, since unsequenced messages don’t carry Applied progress that needs to be preserved.

Changes:

  • Update removeRequestMessage to delete the Sequences map entry when removing the last unsequenced message for a base ID.
  • Add/adjust unit tests to cover both sequenced (preserve Applied) and unsequenced (delete sequence entry) behaviors.
  • Minor test refactors/renames to clarify intent around sequenced vs unsequenced paths.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
overlord/devicemgmtstate/devicemgmtmgr.go Deletes stale Sequences entries after unsequenced messages are processed.
overlord/devicemgmtstate/devicemgmtmgr_test.go Adds coverage asserting sequences are preserved for sequenced messages and removed for unsequenced messages.

Comment on lines +199 to +202
// Unsequenced messages have no Applied progress to carry forward.
if msg.SeqNum == 0 && len(seq.Messages) == 0 {
delete(ms.Sequences, msg.BaseID)
}

@st3v3nmw st3v3nmw Jul 15, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, but the suggestion doesn't make sense here. To make it concrete, say we have mesg-1 and mesg-2 already in flight. Both parse to BaseID = "mesg", so they share Sequences["mesg"] and the sequence "mesg" is in SequenceLRU.

Now mesg (unsequenced) arrives with the same BaseID. Since its SeqNum = 0, enqueueRequestMessages skips the LRU update, and that's fine since "mesg" is already in the LRU from the sequenced messages, & it'd mess with the sequence LRU order. When mesg completes and removeRequestMessage is called, we shouldn't drop SequenceLRU since the sequence is still in progress and we want to know the true LRU ordering.

That said, "you" are pointing at a real underlying issue: a base ID shared between sequenced and unsequenced messages leads to odd behavior, especially during dispatch. Tracking that here.

CC: @miguelpires @pedronis

@codecov

codecov Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.88%. Comparing base (d3e3858) to head (70d4e4c).

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #17371      +/-   ##
==========================================
+ Coverage   78.81%   78.88%   +0.06%     
==========================================
  Files        1405     1397       -8     
  Lines      197076   196895     -181     
  Branches     2498     2498              
==========================================
- Hits       155334   155325       -9     
+ Misses      32379    32217     -162     
+ Partials     9363     9353      -10     
Flag Coverage Δ
unittests 78.88% <100.00%> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@st3v3nmw
st3v3nmw requested review from miguelpires and pedronis July 15, 2026 12:10
@github-actions

github-actions Bot commented Jul 15, 2026

Copy link
Copy Markdown

Wed Jul 29 14:02:41 UTC 2026
The following results are from: https://github.com/canonical/snapd/actions/runs/30434148971

Failures:

Preparing:

  • openstack:ubuntu-26.10-64:tests/main/interfaces-posix-mq
  • openstack:ubuntu-26.10-64:tests/main/lxd-postrm-purge

Executing:

  • openstack:opensuse-tumbleweed-selinux-64:tests/main/manpages
  • openstack:opensuse-tumbleweed-64:tests/main/manpages
  • openstack:opensuse-tumbleweed-64:tests/main/services-watchdog
  • openstack:ubuntu-26.10-64:tests/main/bad-interfaces-warn
  • openstack:ubuntu-26.10-64:tests/main/upgrade-from-release
  • openstack:ubuntu-26.10-64:tests/main/broken-seeding
  • openstack:ubuntu-26.10-64:tests/main/interfaces-location-control
  • openstack:ubuntu-26.10-64:tests/main/lxd
  • openstack:ubuntu-26.10-64:tests/main/xdg-settings

Restoring:

  • openstack:ubuntu-26.10-64:tests/main/apparmor-prompting-integration-tests:write_read_multiple_actioned_by_other_pid_allow_deny
  • openstack:ubuntu-26.10-64:tests/main/apparmor-prompting-integration-tests:write_read_multiple_actioned_by_other_pid_deny_allow
  • openstack:ubuntu-26.10-64:tests/main/
  • openstack:ubuntu-26.10-64:
  • openstack:ubuntu-26.10-64:tests/main/
  • openstack:ubuntu-26.10-64:

Skipped tests from snapd-testing-skip

If you wish to have any of the below tests run in your PR, in your PR description, add 'unskip:' followed by a copy-and-pasted list of the below tests you wish to run (unskip plus test list must be valid yaml)

  • openstack-arm:ubuntu-24.04-arm-64:tests/main/i18n
  • openstack-arm:ubuntu-core-24-arm-64:tests/main/i18n
  • openstack:debian-sid-64:tests/main/interfaces-network-status-classic
  • openstack:debian-sid-64:tests/main/interfaces-xdg-portal-permission-store
  • openstack:ubuntu-24.04-64:tests/main/apparmor-prompting-flag-restart
  • openstack:ubuntu-24.04-64:tests/main/apparmor-prompting-integration-tests
  • openstack:ubuntu-24.04-64:tests/main/apparmor-prompting-prompt-restoration
  • openstack:ubuntu-24.04-64:tests/main/apparmor-prompting-smoke
  • openstack:ubuntu-24.04-64:tests/main/apparmor-prompting-snapd-startup
  • openstack:ubuntu-24.04-64:tests/main/apparmor-prompting-support
  • openstack:ubuntu-24.04-64:tests/main/i18n
  • openstack:ubuntu-24.04-64:tests/main/interfaces-requests-activates-handlers
  • openstack:ubuntu-26.04-64:tests/main/apparmor-prompting-flag-restart
  • openstack:ubuntu-26.04-64:tests/main/apparmor-prompting-integration-tests
  • openstack:ubuntu-26.04-64:tests/main/apparmor-prompting-prompt-restoration
  • openstack:ubuntu-26.04-64:tests/main/apparmor-prompting-smoke
  • openstack:ubuntu-26.04-64:tests/main/apparmor-prompting-snapd-startup
  • openstack:ubuntu-26.04-64:tests/main/apparmor-prompting-support
  • openstack:ubuntu-26.04-64:tests/main/i18n
  • openstack:ubuntu-26.04-64:tests/main/interfaces-requests-activates-handlers

@pedronis pedronis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx

@st3v3nmw st3v3nmw added the Auto rerun spread Auto reruns spread up to 4 times in non-draft PRs w/ >=1 approval and <20 fails in any fund. system label Jul 20, 2026
@st3v3nmw

Copy link
Copy Markdown
Member Author

Spread test failures are unrelated to these changes.

Copilot summary:

  • The job failed due to 3 spread tests on openSUSE Tumbleweed: tests/main/manpages (on both opensuse-tumbleweed-64 and opensuse-tumbleweed-selinux-64) and tests/main/services-watchdog (on opensuse-tumbleweed-64). The manpages failures are likely caused by a man cache issue on openSUSE where the -u --where flags may not behave as expected after a recent Tumbleweed rolling update. The services-watchdog failure appears to be a timing issue where the watchdog-bad service didn't transition to a failed state within the 20-second polling window on that platform.
  • The job failed due to 3 spread tests on openSUSE Tumbleweed: tests/main/manpages (on both opensuse-tumbleweed-64 and opensuse-tumbleweed-selinux-64) and tests/main/services-watchdog (on opensuse-tumbleweed-64). The manpages failures are likely caused by a man cache issue on openSUSE where the -u --where flags may not behave as expected after a recent Tumbleweed rolling update. The services-watchdog failure appears to be a timing issue where the watchdog-bad service didn't transition to a failed state within the 20-second polling window on that platform.

@st3v3nmw
st3v3nmw merged commit d7c520b into canonical:master Jul 29, 2026
486 of 512 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Auto rerun spread Auto reruns spread up to 4 times in non-draft PRs w/ >=1 approval and <20 fails in any fund. system Simple 😃 A small PR which can be reviewed quickly

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants