Code cache allocation changes with Large pages enabled

[SajinaKandy]

Issue or pull request number:
eclipse-openj9/openj9#18342

Overview:
Where Large page/huge page support is available, if the configured large page size is greater than the size of the total code cache size for JIT, then the page size used for code cache allocation will be selected from the next available lower page size available on the platform.

For e.x, if the large page size is configured as 1 GB and the total code cache size is not modified (using -Xcodecachetotal), then the next available page size value lower than 256MB (default code cache total size) will be used for code cache allocation.

Release target:
Eclipse OpenJ9 0.43

Applies to the following JDK versions:
8 and later

Applies to the following platforms:
All platforms

[SajinaKandy]

I think the following pages should have the updated information:
https://eclipse.dev/openj9/docs/xlp/
https://eclipse.dev/openj9/docs/xlpcodecache/
https://eclipse.dev/openj9/docs/configuring/#configuring-large-page-memory-allocation

[SajinaKandy]

@Sreekala-Gopakumar Please let me know if further details are needed regarding this change.

[Sreekala-Gopakumar]

@SajinaKandy - What was the issue because of which this change had to be made? What was happening before? Thanks!

[SajinaKandy]

@Sreekala-Gopakumar
-Xcodecachetotal<size> is the total JIT code cache size. Assuming it's left to default values, 256 MB (for a 64-bit VM) and large page size of 1GB is enabled on the machine. Then,
Before this fix:
The code cache was allocated as 1 GB, as multiple of the available page size. Which is incorrect as the code cache should have been 256 MB.
After the fix:
If large pages are enabled, then the page size will be recalculated if the current page size is greater than the total code cache size. Ex in this case 1GB > 256 MB. Next available lower size page size will be used to allocate 256 MB for the codecache. Ex if there is 2MB page size available then the page size will be set to 2MB and this page size will be used to allocate the 256 MB for code cache.

[Sreekala-Gopakumar]

@Sreekala-Gopakumar -Xcodecachetotal<size> is the total JIT code cache size. Assuming it's left to default values, 256 MB (for a 64-bit VM) and large page size of 1GB is enabled on the machine. Then, Before this fix: The code cache was allocated as 1 GB, as multiple of the available page size. Which is incorrect as the code cache should have been 256 MB. After the fix: If large pages are enabled, then the page size will be recalculated if the current page size is greater than the total code cache size. Ex in this case 1GB > 256 MB. Next available lower size page size will be used to allocate 256 MB for the codecache. Ex if there is 2MB page size available then the page size will be set to 2MB and this page size will be used to allocate the 256 MB for code cache.

In this comment, when you say that "If large pages are enabled, then the page size will be recalculated if the current page size is greater than the total code cache size... Ex if there is 2MB page size available then the page size will be set to 2MB and this page size will be used to allocate the 256 MB for code cache." then:

• What does "the page size will be set to 2MB" mean? Does the original large page size remain 1 GB or is that changed to 2 MB? If the original large page size is not changed then what is it used for, if not allocated for JIT code cache?
• If the next available page size lower than the original large page size is 2 MB and that is used to allocate the 256 MB for code cache, then does that mean 128 pages (of 2 MB each) are allocated for the code cache?

Thanks!

[SajinaKandy]

@Sreekala-Gopakumar

1. Correct we do not change the large page size. We only change the page size used for allocating the Code cache. The code caches are aligned during allocation using the original large page size.
2. To map 256 MB of memory, with 2MB page size, 128 pages will be needed. I would say that means your assumption with regards to code cache should be correct.
   @mpirvu do you think this is correct.

[Sreekala-Gopakumar]

@Sreekala-Gopakumar

1. Correct we do not change the large page size. We only change the page size used for allocating the Code cache. The code caches are aligned during allocation using the original large page size.
2. To map 256 MB of memory, with 2MB page size, 128 pages will be needed. I would say that means your assumption with regards to code cache should be correct.
   @mpirvu do you think this is correct.

Thanks! What do you mean by "The code caches are aligned during allocation using the original large page size."?

[mpirvu]

Thanks! What do you mean by "The code caches are aligned during allocation using the original large page size."?

I think she meant that the allocation would be aligned on a 1GB boundary (assuming 1GB pages are available), but we may end-up using 2 MB or 4 KB pages.