Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add new view to balancing #29

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

add new view to balancing #29

wants to merge 2 commits into from

Conversation

vsawake
Copy link

@vsawake vsawake commented Nov 13, 2022

Create zones with weights depend on current utilization sum previous weight +(1-U)*WIDENING and choose worker depend on random int from 0 to max weight

tests: not sure how to test with random: pls advice, what I need to cover by tests
changelog.md: will update if PR will generally approved

  • tests and linter show no problems (npm t)
  • tests are added/updated for bug fixes and new features
  • code is properly formatted (npm run fmt)
  • description of changes is added in CHANGELOG.md
  • update .d.ts typings

create zones with weights depend on current utilization sum previous weight +(1-U)*WIDENING and choose worker depend on random int from 0 to max weight
@tshemsedinov
Copy link
Member

See this https://youtu.be/qipIRQptP_4

1. balance based on ELU diff
2. review width and width creation
@vsawake
Copy link
Author

vsawake commented Nov 19, 2022

https://youtu.be/qipIRQptP_4 ~38:00 (more 1:18:00)

  1. First balancing makes the calculation of weights without using ELU. Example [250,500,750,1000] for 4 worker threads
  2. Next balancing uses (1.01 - delta ELU).
  3. Balancing extends working intervals to [0,1000(WIDENING)] for easy selection, so we always receive [X,Y,Z,1000] for 4 worker threads. Example:

Reason for using 1.01 (possible 1.1 would be better)

  • negative values of delta
  • In case all worker threads has 1 utilization => [0,0,0,1000](or [Nan,Nan,NaN,1000]) and that means last worker receives all tasks

Test (calculate checksum for all files in /user):

without: ~57s

old version: ~55s
files per thread: [ 28800, 26300, 27277, 23000 ]
active: [ 18740.3154, 7710.0067, 9252.7903, 6951.1388]
idle: [ 35079.1447, 46123.5110, 44574.0218, 46858.2439 ]

new: ~36s
files per thread: [ 26522, 37433, 29558, 11897 ]
active: [ 24503.9996, 19966.9119, 19960.0115, 28230.7962 ]
idle: [ 12138.4605, 16691.6788, 16691.0947, 8414.3918 ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants