Skip to content

feat: add dynamic realm and shard to AccoundIds #1783

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 40 commits into
base: main
Choose a base branch
from

Conversation

Ivo-Yankov
Copy link
Contributor

@Ivo-Yankov Ivo-Yankov commented Apr 2, 2025

Description

  • Introduces the ability to create deployments with non-zero realms and shards on the solo side. A newer version of Consensus Node (>= v0.61.0) is required to create such a deployment, but that introduces an issue with NodeUpdateTransaction, which breaks most of our E2E tests. The bump to the Consensus Node and Mirror Node versions should be done in a separate PR to unblock the merging of this one.
  • Adds new flags to solo deployment create: --shard and --realm with default values 0.
  • Modifies LocalConfig to store the shard and realm values, which are now used to build all EntityIds.
  • Adds a new schema migration for LocalConfig
  • Fixes some compatibility issues with M4 chips and Java.
  • New tests should be added as per this issue

Related Issues

Pull request (PR) checklist

  • This PR added tests (unit, integration, and/or end-to-end)
  • This PR updated documentation
  • This PR added no TODOs or commented out code
  • This PR has no breaking changes
  • Any technical debt has been documented as a separate issue and linked to this PR
  • Any package.json changes have been explained to and approved by a repository manager
  • All related issues have been linked to this PR
  • All changes in this PR are included in the description
  • When this PR merges the commits will be squashed and the title will be used as the commit message, the 'commit message guidelines' below have been followed

Testing

  • This PR added unit tests
  • This PR added integration/end-to-end tests
  • These changes required manual testing that is documented below
  • Anything not tested is documented

The following manual testing was done:

  • TBD

The following was not tested:

  • TBD
Commit message guidelines We use 'Conventional Commits' to ensure that our commit messages are easy to read, follow a consistent format, and for automated release note generation. Please follow the guidelines below when writing your commit messages:
  1. BREAKING CHANGE: a commit that has a footer BREAKING CHANGE:, or appends a ! after the type/scope, introduces a breaking API change (correlating with MAJOR in Semantic Versioning). A BREAKING CHANGE can be part of commits of any type. NOTE: currently breaking changes will only bump the MAJOR version.
  2. The title is prefixed with one of the following:
Prefix Description Semantic Version Update Captured in Release Notes
feat: a new feature MINOR Yes
fix: a bug fix PATCH Yes
perf: performance PATCH Yes
refactor: code change that isn't feature or fix none No
test: adding missing tests none No
docs: changes to documentation none Yes
build: changes to build process none No
ci: changes to CI configuration none No
style: formatting, missing semi-colons, etc none No
chore: updating grunt tasks etc; no production code change none No

@Ivo-Yankov Ivo-Yankov self-assigned this Apr 2, 2025
Signed-off-by: Ivo Yankov <[email protected]>
Signed-off-by: Ivo Yankov <[email protected]>
Signed-off-by: Ivo Yankov <[email protected]>
Copy link
Contributor

github-actions bot commented Apr 4, 2025

Unit Test Results - Linux

24 tests  ±0   24 ✅ ±0   0s ⏱️ ±0s
13 suites ±0    0 💤 ±0 
 1 files   ±0    0 ❌ ±0 

Results for commit e619bba. ± Comparison against base commit 484f219.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Apr 4, 2025

Unit Test Results - Windows

  1 files  ±0   70 suites  ±0   3s ⏱️ +2s
290 tests ±0  290 ✅ ±0  0 💤 ±0  0 ❌ ±0 
295 runs  ±0  295 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit e619bba. ± Comparison against base commit 484f219.

♻️ This comment has been updated with latest results.

Signed-off-by: Ivo Yankov <[email protected]>
Signed-off-by: Ivo Yankov <[email protected]>
Signed-off-by: Ivo Yankov <[email protected]>
Copy link
Contributor

github-actions bot commented Apr 7, 2025

E2E Test Report

 17 files  ±0  117 suites  ±0   1h 28m 15s ⏱️ + 1m 19s
309 tests ±0  309 ✅ ±0  0 💤 ±0  0 ❌ ±0 
326 runs  ±0  326 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 0209cd6. ± Comparison against base commit 2a340f6.

Copy link
Contributor

github-actions bot commented Apr 7, 2025

E2E Test Report

 17 files  ±0  119 suites  ±0   1h 27m 46s ⏱️ +37s
310 tests ±0  310 ✅ ±0  0 💤 ±0  0 ❌ ±0 
326 runs  ±0  326 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit e619bba. ± Comparison against base commit 484f219.

♻️ This comment has been updated with latest results.

Copy link

codacy-production bot commented Apr 7, 2025

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
+0.20% (target: -1.00%) 87.88%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (484f219) 34816 28961 83.18%
Head commit (e619bba) 35216 (+400) 29363 (+402) 83.38% (+0.20%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#1783) 594 522 87.88%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

Copy link

codecov bot commented Apr 7, 2025

Codecov Report

Attention: Patch coverage is 86.53199% with 80 lines in your changes missing coverage. Please review.

Project coverage is 82.37%. Comparing base (484f219) to head (e619bba).
Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
src/commands/mirror-node.ts 51.85% 52 Missing ⚠️
.../migration/impl/local/local-config-v2-migration.ts 80.43% 5 Missing and 4 partials ⚠️
src/commands/network.ts 86.20% 4 Missing ⚠️
src/core/config/local/local-config.ts 87.50% 4 Missing ⚠️
src/core/profile-manager.ts 93.93% 4 Missing ⚠️
src/commands/node/tasks.ts 96.29% 3 Missing ⚠️
src/commands/node/configs.ts 92.00% 2 Missing ⚠️
src/commands/account.ts 95.00% 1 Missing ⚠️
src/core/helpers.ts 94.11% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1783      +/-   ##
==========================================
+ Coverage   82.18%   82.37%   +0.19%     
==========================================
  Files         243      244       +1     
  Lines       34816    35216     +400     
  Branches     2956     3075     +119     
==========================================
+ Hits        28612    29009     +397     
+ Misses       6051     6038      -13     
- Partials      153      169      +16     
Files with missing lines Coverage Δ
src/commands/deployment.ts 68.50% <100.00%> (+0.44%) ⬆️
src/commands/flags.ts 68.39% <100.00%> (+0.29%) ⬆️
src/commands/node/handlers.ts 92.93% <100.00%> (+2.35%) ⬆️
src/commands/relay.ts 85.56% <100.00%> (ø)
src/core/account-manager.ts 87.72% <100.00%> (+0.84%) ⬆️
src/core/config/local/local-config-data-wrapper.ts 96.39% <100.00%> (+0.06%) ⬆️
src/core/constants.ts 97.58% <100.00%> (ø)
src/core/error-messages.ts 88.09% <100.00%> (ø)
...etwork-models/genesis-network-node-data-wrapper.ts 97.26% <100.00%> (+0.15%) ⬆️
src/core/network-nodes.ts 93.08% <100.00%> (+0.04%) ⬆️
... and 13 more

... and 20 files with indirect coverage changes

Impacted file tree graph

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Ivo-Yankov Ivo-Yankov marked this pull request as ready for review April 8, 2025 08:35
@Ivo-Yankov Ivo-Yankov requested review from a team as code owners April 8, 2025 08:35
Signed-off-by: Ivo Yankov <[email protected]>
@Ivo-Yankov Ivo-Yankov marked this pull request as draft April 8, 2025 11:51
@Ivo-Yankov Ivo-Yankov marked this pull request as ready for review April 14, 2025 12:51
@jeromy-cannon jeromy-cannon added PR: Checks Failed A pull request where the checks have failed. PR: Merge Conflicts A pull request that has merge conflicts that need to be resolved. and removed PR: Checks Failed A pull request where the checks have failed. labels Apr 14, 2025
@@ -567,3 +550,12 @@ export async function getAppleSiliconChipset(logger: SoloLogger) {
return ['unknown'];
}
}

export async function requiresJavaSveFix(logger: SoloLogger) {
const chipSet = await getAppleSiliconChipset(logger);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is incorrect in certain scenarios. For example, if a Hashsphere SRE uses Solo from his personal Macbook to deploy to a AWS cluster, then it will fail. Solo will be on a Macbook, whereas the Mirror Node, or anything else using this getAppleSiliconChipset() function will incorrectly assume that because Solo is running with a Silicon chipset, that the K8s cluster is also running on a Silicon chipset, which is not true in all cases.

We should probably delete the getAppleSiliconChipset() function, and instead write logic to query the cluster to get the information required to determine the chipset.

@@ -6,10 +6,14 @@ deployments:
- e2e-cluster-1
- e2e-cluster-2
namespace: dual-cluster-full
realm: 1
Copy link
Contributor

@jeromy-cannon jeromy-cannon Apr 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file is meant to represent v0.35.1, v0.35.1 did not have a realm or shard property in it.

I see you are trying to use this for testing the migration. I recommend cloning this file, and have the clone named schema-v1-local-config.yaml. The only thing is that it also should not have realm and shard, since realm and shard don't appear until schema version 2.

Comment on lines +52 to +53
argv.setArg(flags.realm, 0);
argv.setArg(flags.shard, 0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we set these to non-zero values to ensure the functionality is working correctly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-zero realms and shards require a bump to the network-node version, which breaks most of our e2e tests.

Comment on lines +517 to +518
realm: 0,
shard: 0,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we set these to non-zero values to help verify more scenarios?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will add scenarios with non-zero values in a following PR

@@ -2341,14 +2366,18 @@ export class NodeCommandTasks {

const k8 = this.k8Factory.getK8(context);

const archiveCommand = (await requiresJavaSveFix(this.logger))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering why we can just always use zip, instead of using jar? Additionally, since we own this container we could just update the container to install zip during the docker build process.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure, maybe we should discuss this in the next meeting.

@jeromy-cannon jeromy-cannon added the PR: Unresolved Comments A pull request where there are comments and they need to be resolved. label Apr 14, 2025
…r-accounts

# Conflicts:
#	docs/site/content/User/StepByStepGuide.md
Signed-off-by: Ivo Yankov <[email protected]>
Signed-off-by: Ivo Yankov <[email protected]>
@Ivo-Yankov Ivo-Yankov removed the PR: Merge Conflicts A pull request that has merge conflicts that need to be resolved. label Apr 17, 2025
@Ivo-Yankov Ivo-Yankov added PR: Needs Team Approval A pull request that needs review from a team member. and removed PR: Unresolved Comments A pull request where there are comments and they need to be resolved. labels Apr 17, 2025
@jeromy-cannon jeromy-cannon added PR: Unresolved Comments A pull request where there are comments and they need to be resolved. PR: Merge Conflicts A pull request that has merge conflicts that need to be resolved. and removed PR: Needs Team Approval A pull request that needs review from a team member. labels Apr 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
PR: Merge Conflicts A pull request that has merge conflicts that need to be resolved. PR: Unresolved Comments A pull request where there are comments and they need to be resolved.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

dynamic shared and realm for accounts
2 participants