Add guarded virtual memory support for z/OS by VermaSh · Pull Request #8279 · eclipse-omr/omr

VermaSh · 2026-06-09T18:07:53Z

Add guarded virtual memory support to the z/OS vmem path so that object heap and sparse virtual memory reservations can use guarded storage for Balanced GC. This reduces MEMLIMIT pressure by keeping reserved regions guarded until they are committed for use.

Introduce the OMRPORT_VMEM_MEMORY_MODE_GUARDED flag (currently supported only for Balanced GC) to indicate guarding of VLHGC heap and sparse virtual memory reservations. Guarded 1 MB pageable and 4 KB above-the-bar allocations are supported. Guard and unguard operations are implemented using IARV64 guarded storage support.

Signed-off-by: Shubham Verma shubhamv.sv@gmail.com

VermaSh · 2026-06-10T14:51:08Z

Waiting for my test builds to finish before removing WIP tag.

VermaSh · 2026-06-11T16:26:51Z

Almost all test jobs failed because they were unable to find needed jars. While they don't look related but will try running them locally to confirm.

VermaSh · 2026-06-16T18:45:53Z

Don't see any relevant failures in my local tests. Sanity (minus native lib tests) bucket ran without failures. Doing another run of sanity bucket with native lib tests. I'll set the PR ready for review once my local tests finish.

Opened an issue to investigate why previous build failed.

VermaSh · 2026-06-19T16:32:01Z

My local sanity tests for functional code didn’t have any failures related to this PR. This is ready for review. @dmitripivkine, @joransiu / @r30shah

dmitripivkine · 2026-06-22T17:40:51Z

+				rc = omrremove_guard((void *)alignedAddress, numSegments);
+			}
+
+			if (0 == rc || ( 4 == rc && identifier->pageSize == FOUR_K)) {


Would it be better to define return codes? As minimum we should add comment to explain what code 4 means.

Please put brackets around logical conditions, do not rely on operations order. Also put FOUR_K at the left:

if ((0 == rc) || ((4 == rc) && (FOUR_K == Identifier->pageSize))) {

I would not mention adding extra brackets to the code below to not pollute it too much, please review.

Good point, I have updated the method comment with return code description

dmitripivkine · 2026-06-22T17:42:03Z

+			}
+
+			if (0 == rc || ( 4 == rc && identifier->pageSize == FOUR_K)) {
+				ptr = (void*)alignedAddress;


please add space between void and star.

dmitripivkine · 2026-06-22T19:12:31Z

+	/* Skip low memory allocation (malloc31) when OMRPORT_VMEM_MEMORY_MODE_GUARDED flag */
+	/* is set to avoid MEMLIMIT issues. malloc31() cannot provide guarded memory and */
+	/* counts against MEMLIMIT. When OMRPORT_VMEM_MEMORY_MODE_GUARDED is set, route all */
+	/* allocations directly to above-the-bar allocation which avoids MEMLIMIT issues. */


please use multiline comment format:

/* * ... * ... */

dmitripivkine · 2026-06-23T15:27:57Z

@@ -231,12 +251,42 @@ void *
 omrvmem_commit_memory(struct OMRPortLibrary *portLibrary, void *address, uintptr_t byteAmount, struct J9PortVmemIdentifier *identifier)
 {
 	void *ptr = NULL;
+#if defined(OMR_ENV_DATA64)
+	uintptr_t numSegments = 0;


There is no need to define this variable for full function scope. It can be define inside if statement close to use.

dmitripivkine · 2026-06-23T15:41:19Z

+			intptr_t rc = -1;
+			uintptr_t alignedByteAmount = ROUND_UP_TO_POWEROF2(byteAmount, ONE_M);
+			uintptr_t alignedAddress = ROUND_DOWN_TO_POWEROF2((uintptr_t)address, ONE_M);
+			if (rangeIsValid(identifier, (void *)alignedAddress, alignedByteAmount)) {


I think this check is not necessary. We can rely on guarded allocation pattern: actual allocation for guarded memory happen to be done in 1m pages. So rounding up to 1m should be valid regardless requested size. And yes, we need comment about this here explaining why rounding to 1m is necessary. The same point is applicable to decommit function as well.

dmitripivkine · 2026-06-23T15:57:28Z

+					if (0 == result && OMR_ARE_ANY_BITS_SET(identifier->mode, OMRPORT_VMEM_MEMORY_MODE_GUARDED)) {
+						uintptr_t alignedByteAmount = ROUND_DOWN_TO_POWEROF2(byteAmount, ONE_M);
+						uintptr_t alignedAddress = ROUND_UP_TO_POWEROF2((uintptr_t)address, ONE_M);
+						if (rangeIsValid(identifier, (void *)alignedAddress, alignedByteAmount)) {


Same comment as for commit above

Add guarded virtual memory support to the z/OS vmem path so that object heap and sparse virtual memory reservations can use guarded storage for Balanced GC. This reduces MEMLIMIT pressure by keeping reserved regions guarded until they are committed for use. Introduce the OMRPORT_VMEM_MEMORY_MODE_GUARDED flag (currently supported only for Balanced GC) to indicate guarding of VLHGC heap and sparse virtual memory reservations. Guarded 1 MB pageable and 4 KB above-the-bar allocations are supported. Guard and unguard operations are implemented using IARV64 guarded storage support. Signed-off-by: Shubham Verma <shubhamv.sv@gmail.com>

joransiu · 2026-06-23T17:00:37Z

+	__asm(" IARV64 PLISTVER=MAX,MF=(L,SGETSTOR)":"DS"(wgetstor));
+
+	segments = *numMBSegments;
+	wgetstor = ngetstor;


The ngetstor should be referencing the sgetstor you declared globally for this function.

joransiu · 2026-06-23T17:05:38Z

+ */
+void *omrallocate_1M_pageable_pages_guarded_above_bar(int *numMBSegments, int *userExtendedPrivateAreaMemoryType, const char *ttkn) {
+	long segments;
+	long origin;


We should initialize these variables with a default value.

joransiu · 2026-06-23T17:06:19Z

+
+	switch (useMemoryType) {
+	case ZOS64_VMEM_ABOVE_BAR_GENERAL:
+		break;


Since iarv64_rc is initialized to 0, won't we return a potentially garbage (uninitialized) value of origin?

joransiu · 2026-06-23T17:06:29Z

+ */
+void *omrallocate_4K_pages_guarded_in_userExtendedPrivateArea(int *numMBSegments, int *userExtendedPrivateAreaMemoryType, const char *ttkn) {
+	long segments;
+	long origin;


These variables should be initialized.

joransiu · 2026-06-23T17:06:51Z

+	__asm(" IARV64 PLISTVER=MAX,MF=(L,EGETSTOR)":"DS"(wgetstor));
+
+	segments = *numMBSegments;
+	wgetstor = rgetstor;


rgetstor should be egetstor.

VermaSh · 2026-06-23T17:10:37Z

@dmitripivkine I have updated the PR based on your feedback. It's ready for another review.

dmitripivkine · 2026-06-23T18:42:59Z

@@ -534,7 +602,7 @@ reservePagesAboveBar(struct OMRPortLibrary *portLibrary, J9PortVmemIdentifier *i
 	uintptr_t userExtendedPrivateAreaMemoryType = ZOS64_VMEM_ABOVE_BAR_GENERAL;
 	uintptr_t userExtendedPrivateAreaMemoryMax = 0;

-	if (OMRPORT_VMEM_ZOS_USE2TO32G_AREA == (OMRPORT_VMEM_ZOS_USE2TO32G_AREA & options)) {
+	if (OMR_ARE_ANY_BITS_SET(mode, OMRPORT_VMEM_MEMORY_MODE_GUARDED) || OMRPORT_VMEM_ZOS_USE2TO32G_AREA == (OMRPORT_VMEM_ZOS_USE2TO32G_AREA & options)) {


you can use OMR_ARE_ANY_BITS_SET for OMRPORT_VMEM_ZOS_USE2TO32G_AREA flag in options as well for consistency

dmitripivkine · 2026-06-23T18:47:36Z

 /* omrvmem_support_above_bar.s */
 #pragma linkage(omrallocate_4K_pages_in_userExtendedPrivateArea,OS_NOSTACK)
 void * omrallocate_4K_pages_in_userExtendedPrivateArea(int numMBSegments, int userExtendedPrivateAreaMemoryType, const char * ttkn);

+/* omrvmem_support_above_bar.s */
+#pragma linkage(omrallocate_4K_pages_guarded_in_userExtendedPrivateArea,OS_NOSTACK)
+void * omrallocate_4K_pages_guarded_in_userExtendedPrivateArea(int numMBSegments, int userExtendedPrivateAreaMemoryType, const char * ttkn);


There is no need to have extra space after star (two occurrences). You can fix formatting in the previous pragma as well if you like.

dmitripivkine · 2026-06-23T19:03:16Z

@@ -280,6 +328,26 @@ omrvmem_decommit_memory(struct OMRPortLibrary *portLibrary, void *address, uintp
 				case OMRPORT_VMEM_RESERVE_USED_J9ALLOCATE_4K_PAGES_ABOVE_BAR: /* FALLTHROUGH */
 				case OMRPORT_VMEM_RESERVE_USED_MOSERVICES:
 					result = omrdiscard_data((void *)address, byteAmount >> ZOS_REAL_FRAME_SIZE_SHIFT);
+					if (0 == result && OMR_ARE_ANY_BITS_SET(identifier->mode, OMRPORT_VMEM_MEMORY_MODE_GUARDED)) {


please add extra brackets around logical conditions

dmitripivkine · 2026-06-23T19:03:26Z

+						/* determine number of 1MB segments required */
+						uintptr_t numSegments = alignedByteAmount / ONE_M;
+						result = omradd_guard((void *)address, numSegments);
+						if (4 == result && FOUR_K == identifier->pageSize) {


please add extra brackets around logical conditions

dmitripivkine · 2026-06-23T19:03:57Z

+			uintptr_t numSegments = alignedByteAmount / ONE_M;
+			rc = omrremove_guard((void *)alignedAddress, numSegments);
+
+			if (0 == rc || (4 == rc && FOUR_K == identifier->pageSize)) {


please add extra brackets around logical conditions

VermaSh force-pushed the zos_off-heap_memlimit branch from 2bc7269 to e2cdcb2 Compare June 9, 2026 18:27

VermaSh force-pushed the zos_off-heap_memlimit branch from e2cdcb2 to 2343772 Compare June 19, 2026 16:29

VermaSh changed the title ~~WIP Add guarded virtual memory support for z/OS~~ Add guarded virtual memory support for z/OS Jun 19, 2026

VermaSh marked this pull request as ready for review June 19, 2026 16:32

VermaSh requested review from 0xdaryl, babsingh and mstoodle as code owners June 19, 2026 16:32

dmitripivkine reviewed Jun 22, 2026

View reviewed changes

VermaSh force-pushed the zos_off-heap_memlimit branch 2 times, most recently from 2b2a8b6 to ee6aa1f Compare June 23, 2026 15:23

dmitripivkine reviewed Jun 23, 2026

View reviewed changes

VermaSh force-pushed the zos_off-heap_memlimit branch 2 times, most recently from f955b57 to 5c7ef76 Compare June 23, 2026 15:38

dmitripivkine reviewed Jun 23, 2026

View reviewed changes

VermaSh force-pushed the zos_off-heap_memlimit branch from 5c7ef76 to d9d4d5b Compare June 23, 2026 15:47

dmitripivkine reviewed Jun 23, 2026

View reviewed changes

VermaSh force-pushed the zos_off-heap_memlimit branch from d9d4d5b to eb891af Compare June 23, 2026 17:09

joransiu suggested changes Jun 23, 2026

View reviewed changes

dmitripivkine reviewed Jun 23, 2026

View reviewed changes

Uh oh!

Conversation

VermaSh commented Jun 9, 2026

Uh oh!

VermaSh commented Jun 10, 2026

Uh oh!

VermaSh commented Jun 11, 2026

Uh oh!

VermaSh commented Jun 16, 2026

Uh oh!

VermaSh commented Jun 19, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmitripivkine Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

VermaSh commented Jun 23, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dmitripivkine Jun 23, 2026 •

edited

Loading