io_properties: fix --update-aws-params and add RunByUser tagging#973
Open
fruch wants to merge 3 commits into
Open
io_properties: fix --update-aws-params and add RunByUser tagging#973fruch wants to merge 3 commits into
fruch wants to merge 3 commits into
Conversation
Sort instance types by family (letter prefix, generation number, suffix) then by size within each family. Standardize sub-key ordering to alphabetical (read_bandwidth, read_iops, write_bandwidth, write_iops). This ensures that updating a single instance type won't cause unrelated diff noise from re-ordering.
Add get_username() to identify who launched the instance, matching the pattern used in scylla-cluster-tests. Tags all launched instances with RunByUser for cost tracking and ownership identification. Also adds the missing 'import getpass' and 'import subprocess' that the function requires.
Problems fixed: - update_single_instance_params stored the raw io_properties.yaml structure (with 'disks' list wrapper and 'mountpoint') instead of the flat 4-value format used in aws_io_params.yaml - update_aws_params_yaml re-read and re-wrote the entire file on each instance in a loop, and used sort_keys=False causing unstable output - Both functions produced diffs on unrelated entries when updating a single instance type Changes: - Add extract_io_params() to consistently extract the 4 IO values from raw io_properties structure - Add write_aws_params_yaml() that uses sort_aws_instance_types for consistent top-level ordering and sorted sub-keys - update_aws_params_yaml now reads once, updates all instances, writes once - update_single_instance_params now extracts flat values and uses the same write function - Remove unused sort_instances_by_size function
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix the
--update-aws-paramsflag inget_scylla_io_properties.pyto produce minimal, correct diffs when updating instance IO parameters.Changes
Sort aws_io_params.yaml for stable diffs
read_bandwidth,read_iops,write_bandwidth,write_iops)Add RunByUser tag to EC2 instances
get_username()to identify who launched the instance (matches scylla-cluster-tests pattern)RunByUserfor cost tracking and ownershipimport getpassandimport subprocessFix --update-aws-params to write correct format
update_single_instance_paramswas storing the rawio_properties.yamlstructure (withdiskslist wrapper andmountpoint) instead of the flat 4-value formatupdate_aws_params_yamlwas re-reading/re-writing the entire file per instance in a loop withsort_keys=Falseextract_io_params()to consistently extract just the 4 IO valueswrite_aws_params_yaml()usingsort_aws_instance_typesfor consistent orderingsort_instances_by_sizefunction