Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed flaky circuitBreaker unit test. #893

Merged

Conversation

nhtruong
Copy link
Collaborator

@nhtruong nhtruong commented Oct 30, 2024

Here's the actual code in Transport.js:

    const MAX_STRING_LENGTH = buffer.constants.MAX_STRING_LENGTH;
    const HEAP_SIZE_LIMIT = require('v8').getHeapStatistics().heap_size_limit;
    ...
    const shouldApplyCircuitBreaker = (contentLength) => {
      if (!this.memoryCircuitBreaker || !this.memoryCircuitBreaker.enabled) return false;
      const maxPercentage = validateMemoryPercentage(this.memoryCircuitBreaker.maxPercentage);
      const heapUsed = process.memoryUsage().heapUsed;
      return contentLength + heapUsed > HEAP_SIZE_LIMIT * maxPercentage;
    };

However the current unit test sets the contentLength off of MAX_STRING_LENGTH instead of HEAP_SIZE_LIMIT. This results in the test failing on earlier versions of node.js where heapUsed is smaller and doesn't cause contentLength + heapUsed > HEAP_SIZE_LIMIT * maxPercentage to return true.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

#
Signed-off-by: Theo Truong <[email protected]>
#
Signed-off-by: Theo Truong <[email protected]>
Comment on lines -1044 to -1060
const contentLength = buffer.constants.MAX_STRING_LENGTH - 1;
const percentage = 0.01;
const HEAP_SIZE_LIMIT = require('v8').getHeapStatistics().heap_size_limit;
const contentLength = Math.round(HEAP_SIZE_LIMIT * percentage + 1);
const memoryCircuitBreaker = {
enabled: true,
maxPercentage: percentage,
};
// Simulate allocation of bytes
const memoryAllocations = [];
while (process.memoryUsage().heapUsed + contentLength <= HEAP_SIZE_LIMIT_WITH_BUFFER) {
const allocation = 50 * 1024 * 1024; // 50MB
const numbers = allocation / 8;
const arr = [];
arr.length = numbers;
for (let i = 0; i < numbers; i++) {
arr[i] = i;
}
memoryAllocations.push(arr);
}
Copy link
Collaborator Author

@nhtruong nhtruong Oct 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing this block of code because memoryAllocations is not being used anywhere.

@nhtruong nhtruong marked this pull request as ready for review October 30, 2024 20:06
CHANGELOG.md Outdated Show resolved Hide resolved
#
Signed-off-by: Theo Truong <[email protected]>
@nhtruong nhtruong merged commit 5f55f06 into opensearch-project:main Oct 31, 2024
51 of 52 checks passed
@nhtruong nhtruong deleted the fix_flaky_circuit_breakter_test branch October 31, 2024 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants