aws: add i8g instance type #676
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
roydahan
reviewed
Mar 20, 2025
| read_iops: 527429 | ||
| read_bandwidth: 4797424981 | ||
| write_iops: 139669 | ||
| write_bandwidth: 3421133482 |
There was a problem hiding this comment.
it's the same as i8g.4xlarge, while all previous ones scaled quite linearly.
There was a problem hiding this comment.
I see that it's the top for write starting 4xlarge and above.
Does it make sense?
There was a problem hiding this comment.
Yes, it seems like i8g.4xlarge and above are same performance, when measuring per single disk (number of disks will increase when instance size become larger, so RAID0 performance probably will increase).
(see detail of the result at #676 (comment))
Maybe we should make these entries into one, something like (we have such virtual instance size named "ALL"):
i8g.ALL:
read_iops: 553607
read_bandwidth: 4797597013
write_iops: 140814
write_bandwidth: 3419948373
There was a problem hiding this comment.
@avikivity does it make sense to you?
It seems like it would be better to use multiple i8g.4x than scale up to i8g.8x and above.
Adding preset io parameters of i8g to scylla_cloud_io_setup, and also added i8g to supported instance type on aws_instance class. All preset values are measured by iotune on target instances. Here's measurement environment details: - Measured on i8g.* instances with latest version of Ubuntu 24.04 LTS AMI (We cannot use Scylla AMI since we do want to measure single drive performance) - Measured single local SSD w/o RAID0, since we simulate RAID0 performance on scylla_cloud_io_setup script from single drive performance - Use iotune for the measurement, executed 3 times for each instance size and used average of the results - Automated measurement by script: https://github.com/syuu1228/ec2_run_script Here's raw output of iotune: - i8g.large (0/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 399 MB/s (deviation 14%) Measuring sequential read bandwidth: 577 MB/s (deviation 43%) Measuring random write IOPS: 45598 IOPS (deviation 31%) Measuring random read IOPS: 82269 IOPS (deviation 29%) Writing result to /etc/scylla.d/io_properties.yaml - i8g.large (1/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 399 MB/s (deviation 14%) Measuring sequential read bandwidth: 577 MB/s (deviation 43%) Measuring random write IOPS: 45601 IOPS (deviation 31%) Measuring random read IOPS: 82266 IOPS (deviation 29%) Writing result to /etc/scylla.d/io_properties.yaml - i8g.large (2/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 399 MB/s (deviation 14%) Measuring sequential read bandwidth: 577 MB/s (deviation 43%) Measuring random write IOPS: 45596 IOPS (deviation 31%) Measuring random read IOPS: 82269 IOPS (deviation 29%) Writing result to /etc/scylla.d/io_properties.yaml - i8g.xlarge (0/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 799 MB/s (deviation 14%) Measuring sequential read bandwidth: 1160 MB/s (deviation 43%) Measuring random write IOPS: 90444 IOPS (deviation 19%) Measuring random read IOPS: 164432 IOPS (deviation 29%) Writing result to /etc/scylla.d/io_properties.yaml - i8g.xlarge (1/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 799 MB/s (deviation 14%) Measuring sequential read bandwidth: 1160 MB/s (deviation 43%) Measuring random write IOPS: 90490 IOPS (deviation 20%) Measuring random read IOPS: 164456 IOPS (deviation 29%) Writing result to /etc/scylla.d/io_properties.yaml - i8g.xlarge (2/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 799 MB/s (deviation 14%) Measuring sequential read bandwidth: 1160 MB/s (deviation 43%) Measuring random write IOPS: 90405 IOPS (deviation 19%) Measuring random read IOPS: 164439 IOPS (deviation 29%) Writing result to /etc/scylla.d/io_properties.yaml - i8g.2xlarge (0/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 1609 MB/s (deviation 14%) Measuring sequential read bandwidth: 2325 MB/s (deviation 41%) Measuring random write IOPS: 135841 IOPS Measuring random read IOPS: 328308 IOPS (deviation 23%) Writing result to /etc/scylla.d/io_properties.yaml - i8g.2xlarge (1/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 1609 MB/s (deviation 14%) Measuring sequential read bandwidth: 2325 MB/s (deviation 41%) Measuring random write IOPS: 133803 IOPS (deviation 3%) Measuring random read IOPS: 328216 IOPS (deviation 22%) Writing result to /etc/scylla.d/io_properties.yaml - i8g.2xlarge (2/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 1609 MB/s (deviation 14%) Measuring sequential read bandwidth: 2325 MB/s (deviation 41%) Measuring random write IOPS: 133992 IOPS Measuring random read IOPS: 328302 IOPS (deviation 23%) Writing result to /etc/scylla.d/io_properties.yaml - i8g.4xlarge (0/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3260 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 141234 IOPS Measuring random read IOPS: 554281 IOPS Writing result to /etc/scylla.d/io_properties.yaml - i8g.4xlarge (1/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3262 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 140699 IOPS Measuring random read IOPS: 552544 IOPS Writing result to /etc/scylla.d/io_properties.yaml - i8g.4xlarge (2/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3262 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 140510 IOPS Measuring random read IOPS: 553998 IOPS Writing result to /etc/scylla.d/io_properties.yaml - i8g.8xlarge (0/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3262 MB/s (deviation 8%) Measuring sequential read bandwidth: 4574 MB/s (deviation 26%) Measuring random write IOPS: 139456 IOPS Measuring random read IOPS: 528086 IOPS Writing result to /etc/scylla.d/io_properties.yaml - i8g.8xlarge (1/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3262 MB/s (deviation 8%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 140425 IOPS Measuring random read IOPS: 527157 IOPS Writing result to /etc/scylla.d/io_properties.yaml - i8g.8xlarge (2/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3262 MB/s (deviation 9%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 139128 IOPS Measuring random read IOPS: 527045 IOPS Writing result to /etc/scylla.d/io_properties.yaml - i8g.12xlarge (0/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3263 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 139198 IOPS Measuring random read IOPS: 512264 IOPS Writing result to /etc/scylla.d/io_properties.yaml - i8g.12xlarge (1/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3262 MB/s (deviation 11%) Measuring sequential read bandwidth: 4566 MB/s (deviation 25%) Measuring random write IOPS: 139031 IOPS Measuring random read IOPS: 509359 IOPS Writing result to /etc/scylla.d/io_properties.yaml - i8g.12xlarge (2/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3262 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 139333 IOPS Measuring random read IOPS: 512320 IOPS Writing result to /etc/scylla.d/io_properties.yaml - i8g.16xlarge (0/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3262 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 140104 IOPS Measuring random read IOPS: 506129 IOPS Writing result to /etc/scylla.d/io_properties.yaml - i8g.16xlarge (1/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3262 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 139681 IOPS Measuring random read IOPS: 504401 IOPS Writing result to /etc/scylla.d/io_properties.yaml - i8g.16xlarge (2/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3261 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 140213 IOPS Measuring random read IOPS: 506485 IOPS Writing result to /etc/scylla.d/io_properties.yaml - i8g.24xlarge (0/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3265 MB/s (deviation 14%) Measuring sequential read bandwidth: 4574 MB/s (deviation 26%) Measuring random write IOPS: 139638 IOPS Measuring random read IOPS: 517821 IOPS Writing result to /etc/scylla.d/io_properties.yaml - i8g.24xlarge (1/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3262 MB/s (deviation 10%) Measuring sequential read bandwidth: 4574 MB/s (deviation 26%) Measuring random write IOPS: 140312 IOPS Measuring random read IOPS: 517178 IOPS Writing result to /etc/scylla.d/io_properties.yaml - i8g.24xlarge (2/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3265 MB/s (deviation 14%) Measuring sequential read bandwidth: 4574 MB/s (deviation 26%) Measuring random write IOPS: 140032 IOPS Measuring random read IOPS: 519382 IOPS (deviation 3%) Writing result to /etc/scylla.d/io_properties.yaml - i8g.metal-24xl (0/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3260 MB/s (deviation 10%) Measuring sequential read bandwidth: 4569 MB/s (deviation 25%) Measuring random write IOPS: 138682 IOPS Measuring random read IOPS: 513462 IOPS Writing result to /etc/scylla.d/io_properties.yaml - i8g.metal-24xl (1/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3263 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 139715 IOPS Measuring random read IOPS: 510749 IOPS Writing result to /etc/scylla.d/io_properties.yaml - i8g.metal-24xl (2/3) Starting Evaluation. This may take a while... Measuring sequential write bandwidth: 3260 MB/s (deviation 9%) Measuring sequential read bandwidth: 4555 MB/s (deviation 24%) Measuring random write IOPS: 139076 IOPS Measuring random read IOPS: 511639 IOPS Writing result to /etc/scylla.d/io_properties.yaml Fixes scylladb#560
syuu1228
force-pushed
the
aws_i8g_instance_type
branch
from
e97e6dd to
d7c67e5
Compare
yaronkaikov
approved these changes
Mar 24, 2025
There was a problem hiding this comment.
LGTM - Let's wait for @roydahan to return from his vacation before merging this
|
These are the specs for i8g published by AWS:
It seems like our numbers aren't aligned with these once we reach to 4xlarge, both for read IOPS, but especially for write IOPS which looks like reaching a limit. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adding preset io parameters of i8g to scylla_cloud_io_setup, and also added i8g to supported instance type on aws_instance class.
All preset values are measured by iotune on target instances.
Here's measurement environment details:
Here's raw output of iotune:
i8g.large (0/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 399 MB/s (deviation 14%) Measuring sequential read bandwidth: 577 MB/s (deviation 43%) Measuring random write IOPS: 45598 IOPS (deviation 31%) Measuring random read IOPS: 82269 IOPS (deviation 29%) Writing result to /etc/scylla.d/io_properties.yaml
i8g.large (1/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 399 MB/s (deviation 14%) Measuring sequential read bandwidth: 577 MB/s (deviation 43%) Measuring random write IOPS: 45601 IOPS (deviation 31%) Measuring random read IOPS: 82266 IOPS (deviation 29%) Writing result to /etc/scylla.d/io_properties.yaml
i8g.large (2/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 399 MB/s (deviation 14%) Measuring sequential read bandwidth: 577 MB/s (deviation 43%) Measuring random write IOPS: 45596 IOPS (deviation 31%) Measuring random read IOPS: 82269 IOPS (deviation 29%) Writing result to /etc/scylla.d/io_properties.yaml
i8g.xlarge (0/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 799 MB/s (deviation 14%) Measuring sequential read bandwidth: 1160 MB/s (deviation 43%) Measuring random write IOPS: 90444 IOPS (deviation 19%) Measuring random read IOPS: 164432 IOPS (deviation 29%) Writing result to /etc/scylla.d/io_properties.yaml
i8g.xlarge (1/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 799 MB/s (deviation 14%) Measuring sequential read bandwidth: 1160 MB/s (deviation 43%) Measuring random write IOPS: 90490 IOPS (deviation 20%) Measuring random read IOPS: 164456 IOPS (deviation 29%) Writing result to /etc/scylla.d/io_properties.yaml
i8g.xlarge (2/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 799 MB/s (deviation 14%) Measuring sequential read bandwidth: 1160 MB/s (deviation 43%) Measuring random write IOPS: 90405 IOPS (deviation 19%) Measuring random read IOPS: 164439 IOPS (deviation 29%) Writing result to /etc/scylla.d/io_properties.yaml
i8g.2xlarge (0/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1609 MB/s (deviation 14%) Measuring sequential read bandwidth: 2325 MB/s (deviation 41%) Measuring random write IOPS: 135841 IOPS
Measuring random read IOPS: 328308 IOPS (deviation 23%) Writing result to /etc/scylla.d/io_properties.yaml
i8g.2xlarge (1/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1609 MB/s (deviation 14%) Measuring sequential read bandwidth: 2325 MB/s (deviation 41%) Measuring random write IOPS: 133803 IOPS (deviation 3%) Measuring random read IOPS: 328216 IOPS (deviation 22%) Writing result to /etc/scylla.d/io_properties.yaml
i8g.2xlarge (2/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 1609 MB/s (deviation 14%) Measuring sequential read bandwidth: 2325 MB/s (deviation 41%) Measuring random write IOPS: 133992 IOPS
Measuring random read IOPS: 328302 IOPS (deviation 23%) Writing result to /etc/scylla.d/io_properties.yaml
i8g.4xlarge (0/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3260 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 141234 IOPS
Measuring random read IOPS: 554281 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
i8g.4xlarge (1/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3262 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 140699 IOPS
Measuring random read IOPS: 552544 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
i8g.4xlarge (2/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3262 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 140510 IOPS
Measuring random read IOPS: 553998 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
i8g.8xlarge (0/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3262 MB/s (deviation 8%) Measuring sequential read bandwidth: 4574 MB/s (deviation 26%) Measuring random write IOPS: 139456 IOPS
Measuring random read IOPS: 528086 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
i8g.8xlarge (1/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3262 MB/s (deviation 8%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 140425 IOPS
Measuring random read IOPS: 527157 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
i8g.8xlarge (2/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3262 MB/s (deviation 9%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 139128 IOPS
Measuring random read IOPS: 527045 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
i8g.12xlarge (0/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3263 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 139198 IOPS
Measuring random read IOPS: 512264 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
i8g.12xlarge (1/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3262 MB/s (deviation 11%) Measuring sequential read bandwidth: 4566 MB/s (deviation 25%) Measuring random write IOPS: 139031 IOPS
Measuring random read IOPS: 509359 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
i8g.12xlarge (2/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3262 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 139333 IOPS
Measuring random read IOPS: 512320 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
i8g.16xlarge (0/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3262 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 140104 IOPS
Measuring random read IOPS: 506129 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
i8g.16xlarge (1/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3262 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 139681 IOPS
Measuring random read IOPS: 504401 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
i8g.16xlarge (2/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3261 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 140213 IOPS
Measuring random read IOPS: 506485 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
i8g.24xlarge (0/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3265 MB/s (deviation 14%) Measuring sequential read bandwidth: 4574 MB/s (deviation 26%) Measuring random write IOPS: 139638 IOPS
Measuring random read IOPS: 517821 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
i8g.24xlarge (1/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3262 MB/s (deviation 10%) Measuring sequential read bandwidth: 4574 MB/s (deviation 26%) Measuring random write IOPS: 140312 IOPS
Measuring random read IOPS: 517178 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
i8g.24xlarge (2/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3265 MB/s (deviation 14%) Measuring sequential read bandwidth: 4574 MB/s (deviation 26%) Measuring random write IOPS: 140032 IOPS
Measuring random read IOPS: 519382 IOPS (deviation 3%) Writing result to /etc/scylla.d/io_properties.yaml
i8g.metal-24xl (0/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3260 MB/s (deviation 10%) Measuring sequential read bandwidth: 4569 MB/s (deviation 25%) Measuring random write IOPS: 138682 IOPS
Measuring random read IOPS: 513462 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
i8g.metal-24xl (1/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3263 MB/s (deviation 10%) Measuring sequential read bandwidth: 4575 MB/s (deviation 26%) Measuring random write IOPS: 139715 IOPS
Measuring random read IOPS: 510749 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
i8g.metal-24xl (2/3) Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 3260 MB/s (deviation 9%) Measuring sequential read bandwidth: 4555 MB/s (deviation 24%) Measuring random write IOPS: 139076 IOPS
Measuring random read IOPS: 511639 IOPS
Writing result to /etc/scylla.d/io_properties.yaml
Fixes #560