HPCC-35546 Change storage plane -> dali group update semanics #20781
Conversation
|
Jira Issue: https://hpccsystems.atlassian.net//browse/HPCC-35546 Jirabot Action Result: |
There was a problem hiding this comment.
Pull request overview
This PR changes the semantics of how Dali groups are synchronized with storage plane definitions. Storage planes now become the single source of truth for storage group layout, with automatic updates of corresponding Dali groups when plane definitions change. A new @protectGroup configuration option allows individual planes to opt out of automatic updates to preserve existing group layouts that logical files may reference.
Key changes:
- Storage plane definitions now automatically overwrite corresponding Dali groups (unless protected)
- Stale Dali groups without corresponding planes are automatically removed
- New
@protectGroupboolean field in plane configuration to prevent automatic overwrites
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| helm/hpcc/values.schema.json | Adds protectGroup boolean field to plane schema to control automatic group updates |
| dali/base/dadfs.cpp | Implements new group synchronization logic: removes stale groups, changes protection semantics, and refactors initialization to handle containerized vs bare-metal environments differently |
Ensure Dali groups are kept in sync with their corresponding storage/planes definitions. Delete stale old groups that no longer have a corresponding plane Also unify initClusterGroups and initClusterAndStoragePlaneGroups Signed-off-by: Jake Smith <[email protected]>
jakesmith
force-pushed
the
HPCC-35546-planes-update-dali-groups
branch
from
d375e98 to
2c3abed
Compare
ghalliday
left a comment
ghalliday
left a comment
There was a problem hiding this comment.
@jakesmith change looks good. One question about what happens the first time this is run.
Also slightly concerned by the target version in case it introduces a problem.
| } | ||
| } | ||
| for (auto &group: toRemove) | ||
| groupsRoot->removeTree(group); |
There was a problem hiding this comment.
Will this mean that the first time it is run on a bare-metal system all groups will be removed?
There was a problem hiding this comment.
Yes. New groups will be created based on the environment that will be @Protected (except dropzone groups), so that next time, they will not be in this remove list.
So in theory, there could be an old group in use by logical files (that is not @Protected), that this process will now delete...
Perhaps as a guard, I should move them (/GroupsDisassociated/) instead of delete them?
2 participants
Ensure Dali groups are kept in sync with their corresponding storage/planes definitions.
Delete stale old groups that no longer have a corresponding plane
Type of change:
Checklist:
Smoketest:
Testing: