Skip to content

[Nexthop] Dynamic FDB entries should be skipped during warm boot if aged out in h/w - #1528

Open
vvasavada-nexthop wants to merge 1 commit into
facebook:mainfrom
nexthop-ai:dynamic-fdb-entries-should-be-skipped
Open

[Nexthop] Dynamic FDB entries should be skipped during warm boot if aged out in h/w#1528
vvasavada-nexthop wants to merge 1 commit into
facebook:mainfrom
nexthop-ai:dynamic-fdb-entries-should-be-skipped

Conversation

@vvasavada-nexthop

Copy link
Copy Markdown

Pre-submission checklist

  • I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running pip install -r requirements-dev.txt && pre-commit install
  • pre-commit run

Summary

Warm boot crashed the hw agent with SIGABRT failing warm_boot.AgentMacLearningAndNeighborResolutionTest/1.learnMacProgramNeighborsAndAgeMac:

[fdb] Failed to get sai attribute: FdbEntry(switch:..., mac:02:00:00:00:00:01) ... ITEM NOT FOUND
terminate called after throwing an instance of 'facebook::fboss::SaiApiError'
  ... SaiObjectStore<SaiFdbTraits>::reload() -> SaiStore::reload() -> SaiSwitch::initLocked()

The warm boot adapter-key file is treated as faithful snapshot of h/w, however, h/w deletes dynamic FDB entries on its own. Every other FDB path already tolerates this race - ignoreMissingInHwOnDelete_ in SaiObject::remove(), the ITEM_NOT_FOUND catch in SaiFdbManager::changeMac()/createSaiObject(), and removeUnclaimedDynanicEntries(), which exists to discard exactly these stale entries but runs from initialStateApplied(), i.e. after reload() has already killed the process. Reload was the only path that treated vanished dynamic FDB entry as fatal.

We fix this by:

  • New SaiObjectMayBeMissingInHw trait in api/Traits.h (default false), specialized true for SaiFdbTraits in api/FdbApi.h.
  • SaiObjectStore::reload() goes through a new getObjectIfInHw() which catches SaiApiError, rethrows if the error is anything but SAI_STATUS_ITEM_NOT_FOUND, otherwise skips with warning log, for the cases where trait evaluates to true.

Test Plan

Running all tests took 0:04:53.548095 between 2026-08-17 18:40:14.465554 and 2026-08-17 18:45:08.013649
[ PASSED ] cold_boot.AgentMacLearningAndNeighborResolutionTest/0.learnMacProgramNeighborsAndAgeMac (30533 ms)
[ PASSED ] warm_boot.AgentMacLearningAndNeighborResolutionTest/0.learnMacProgramNeighborsAndAgeMac (26595 ms)
[ PASSED ] cold_boot.AgentMacLearningAndNeighborResolutionTest/1.learnMacProgramNeighborsAndAgeMac (30531 ms)
[ PASSED ] warm_boot.AgentMacLearningAndNeighborResolutionTest/1.learnMacProgramNeighborsAndAgeMac (26578 ms)
[ PASSED ] cold_boot.AgentMacLearningAndNeighborResolutionTest/2.learnMacProgramNeighborsAndAgeMac (29530 ms)
[ PASSED ] warm_boot.AgentMacLearningAndNeighborResolutionTest/2.learnMacProgramNeighborsAndAgeMac (26640 ms)
[ PASSED ] cold_boot.AgentMacLearningAndNeighborResolutionTest/3.learnMacProgramNeighborsAndAgeMac (30531 ms)
[ PASSED ] warm_boot.AgentMacLearningAndNeighborResolutionTest/3.learnMacProgramNeighborsAndAgeMac (26599 ms)
Summary:
   PASSED : 8
   FAILED : 0
   SKIPPED : 0
   TIMEOUT : 0

… h/w

Signed-off-by: Vishrant Vasavada <vvasavada@nexthop.ai>
@vvasavada-nexthop
vvasavada-nexthop requested a review from a team as a code owner August 19, 2026 02:02
@meta-cla meta-cla Bot added the CLA Signed label Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant