Skip to content

Conversation

@vsawake
Copy link

@vsawake vsawake commented Nov 13, 2022

Create zones with weights depend on current utilization sum previous weight +(1-U)*WIDENING and choose worker depend on random int from 0 to max weight

tests: not sure how to test with random: pls advice, what I need to cover by tests
changelog.md: will update if PR will generally approved

  • tests and linter show no problems (npm t)
  • tests are added/updated for bug fixes and new features
  • code is properly formatted (npm run fmt)
  • description of changes is added in CHANGELOG.md
  • update .d.ts typings

create zones with weights depend on current utilization sum previous weight +(1-U)*WIDENING and choose worker depend on random int from 0 to max weight
@tshemsedinov
Copy link
Member

See this https://youtu.be/qipIRQptP_4

1. balance based on ELU diff
2. review width and width creation
@vsawake
Copy link
Author

vsawake commented Nov 19, 2022

https://youtu.be/qipIRQptP_4 ~38:00 (more 1:18:00)

  1. First balancing makes the calculation of weights without using ELU. Example [250,500,750,1000] for 4 worker threads
  2. Next balancing uses (1.01 - delta ELU).
  3. Balancing extends working intervals to [0,1000(WIDENING)] for easy selection, so we always receive [X,Y,Z,1000] for 4 worker threads. Example:

Reason for using 1.01 (possible 1.1 would be better)

  • negative values of delta
  • In case all worker threads has 1 utilization => [0,0,0,1000](or [Nan,Nan,NaN,1000]) and that means last worker receives all tasks

Test (calculate checksum for all files in /user):

without: ~57s

old version: ~55s
files per thread: [ 28800, 26300, 27277, 23000 ]
active: [ 18740.3154, 7710.0067, 9252.7903, 6951.1388]
idle: [ 35079.1447, 46123.5110, 44574.0218, 46858.2439 ]

new: ~36s
files per thread: [ 26522, 37433, 29558, 11897 ]
active: [ 24503.9996, 19966.9119, 19960.0115, 28230.7962 ]
idle: [ 12138.4605, 16691.6788, 16691.0947, 8414.3918 ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants