Skip to content

OCPBUG 6952: Modified 'understanding how to override JVM max heap size' section #68617

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 33 additions & 72 deletions modules/nodes-cluster-resource-configure-jdk.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -7,110 +7,71 @@
= Understanding OpenJDK settings for {product-title}

The default OpenJDK settings do not work well with containerized
environments. As a result, some additional Java memory
settings must always be provided whenever running the OpenJDK in a container.
environments. As a result, some additional Java memory settings must always be provided whenever running the OpenJDK in a container.
Comment on lines 9 to +10
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be fully unwrapped?


The JVM memory layout is complex, version dependent, and describing it in detail
is beyond the scope of this documentation. However, as a starting point for
running OpenJDK in a container, at least the following three memory-related
tasks are key:
The JVM memory layout is complex, version dependent, and describing it in detail is beyond the scope of this documentation. However, as a starting point for running OpenJDK in a container, at least the following three memory-related tasks are key:

. Overriding the JVM maximum heap size.

. Encouraging the JVM to release unused memory to the operating system, if
appropriate.
. Encouraging the JVM to release unused memory to the operating system, if appropriate.

. Ensuring all JVM processes within a container are appropriately configured.

Optimally tuning JVM workloads for running in a container is beyond the scope of
this documentation, and may involve setting multiple additional JVM options.
Optimally tuning JVM workloads for running in a container is beyond the scope of this documentation, and may involve setting multiple additional JVM options.

[id="nodes-cluster-resource-configure-jdk-heap_{context}"]
== Understanding how to override the JVM maximum heap size

For many Java workloads, the JVM heap is the largest single consumer of memory.
Currently, the OpenJDK defaults to allowing up to 1/4 (1/`-XX:MaxRAMFraction`)
of the compute node's memory to be used for the heap, regardless of whether the
OpenJDK is running in a container or not. It is therefore *essential* to
override this behavior, especially if a container memory limit is also set.

There are at least two ways the above can be achieved:

* If the container memory limit is set and the experimental options are
supported by the JVM, set `-XX:+UnlockExperimentalVMOptions
-XX:+UseCGroupMemoryLimitForHeap`.
+
[NOTE]
====
The `UseCGroupMemoryLimitForHeap` option has been removed in JDK 11. Use `-XX:+UseContainerSupport` instead.
====
+
This sets `-XX:MaxRAM` to the container memory limit, and the maximum heap size
(`-XX:MaxHeapSize` / `-Xmx`) to 1/`-XX:MaxRAMFraction` (1/4 by default).

* Directly override one of `-XX:MaxRAM`, `-XX:MaxHeapSize` or `-Xmx`.
+
This option involves hard-coding a value, but has the advantage of allowing a
safety margin to be calculated.
OpenJDK defaults to using a maximum of 25% of available memory (recognising any container memory limits in place) for "heap" memory. This default value is conservative, and in a properly-configured container environment, this value would result in 75% of the memory assigned to a container being mostly unused. A much higher percentage for the JVM to use for heap memory, such as 80%, is more suitable in a container context where memory limits are imposed on the container level.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
OpenJDK defaults to using a maximum of 25% of available memory (recognising any container memory limits in place) for "heap" memory. This default value is conservative, and in a properly-configured container environment, this value would result in 75% of the memory assigned to a container being mostly unused. A much higher percentage for the JVM to use for heap memory, such as 80%, is more suitable in a container context where memory limits are imposed on the container level.
OpenJDK defaults to using a maximum of 25% of available memory (recognizing any container memory limits in place) for "heap" memory. This default value is conservative and, in a properly-configured container environment, this value would result in 75% of the memory assigned to a container being mostly unused. A much higher percentage for the JVM to use for heap memory, such as 80%, is more suitable in a container context where memory limits are imposed on the container level.


Most of the Red Hat containers replace the OpenJDK default by means of a startup script - included in those containers - that set updated values when the JVM launches.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do this for any instances of "Red Hat"

Suggested change
Most of the Red Hat containers replace the OpenJDK default by means of a startup script - included in those containers - that set updated values when the JVM launches.
Most of the Red{nbsp}Hat containers replace the OpenJDK default by means of a startup script - included in those containers - that set updated values when the JVM launches.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe rephrase this a bit?

Suggested change
Most of the Red Hat containers replace the OpenJDK default by means of a startup script - included in those containers - that set updated values when the JVM launches.
Most of the Red Hat containers include a startup script that replaces the OpenJDK default by setting updated values when the JVM launches.


For example, the Red Hat build of OpenJDK containers have a default value of 80%. This value can be set to a different percentage by defining the following environment variable:

[source,terminal]
----
JAVA_MAX_RAM_RATIO
----
Comment on lines +29 to +34
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need a block? it's not showing a command, as written

Suggested change
For example, the Red Hat build of OpenJDK containers have a default value of 80%. This value can be set to a different percentage by defining the following environment variable:
[source,terminal]
----
JAVA_MAX_RAM_RATIO
----
For example, the Red{nbsp}Hat build of OpenJDK containers have a default value of 80%. This value can be set to a different percentage by defining the `JAVA_MAX_RAM_RATIO` environment variable.


For other OpenJDK deployements, the default value of 25% can be changed using the following command:

.Example
[source,terminal]
----
java -XX:MaxRAMPercentage=80.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

commands need a $ prompt unless it's run as root (in which case it gets a #)

Suggested change
java -XX:MaxRAMPercentage=80.0
$ java -XX:MaxRAMPercentage=80.0

----


[id="nodes-cluster-resource-configure-jdk-unused_{context}"]
== Understanding how to encourage the JVM to release unused memory to the operating system

By default, the OpenJDK does not aggressively return unused memory to the
operating system. This may be appropriate for many containerized Java
workloads, but notable exceptions include workloads where additional active
processes co-exist with a JVM within a container, whether those additional
processes are native, additional JVMs, or a combination of the two.
By default, OpenJDK does not aggressively return unused memory to the operating system. This may be appropriate for many containerized Java workloads, but notable exceptions include workloads where additional active processes co-exist with a JVM within a container, whether those additional processes are native, additional JVMs, or a combination of the two.

Java-based agents can use the following JVM arguments to encourage the JVM
to release unused memory to the operating system:
Java-based workloads can use the following JVM arguments to encourage the JVM to release unused memory to the operating system:

[source,terminal]
----
-XX:+UseParallelGC
-XX:MinHeapFreeRatio=5 -XX:MaxHeapFreeRatio=10 -XX:GCTimeRatio=4
-XX:AdaptiveSizePolicyWeight=90.
-XX:AdaptiveSizePolicyWeight=90
----

These arguments are intended to return heap
memory to the operating system whenever allocated memory exceeds 110% of in-use
memory (`-XX:MaxHeapFreeRatio`), spending up to 20% of CPU time in the garbage
collector (`-XX:GCTimeRatio`). At no time will the application heap allocation
be less than the initial heap allocation (overridden by `-XX:InitialHeapSize` /
`-Xms`). Detailed additional information is available
link:https://developers.redhat.com/blog/2014/07/15/dude-wheres-my-paas-memory-tuning-javas-footprint-in-openshift-part-1/[Tuning Java's footprint in OpenShift (Part 1)],
link:https://developers.redhat.com/blog/2014/07/22/dude-wheres-my-paas-memory-tuning-javas-footprint-in-openshift-part-2/[Tuning Java's footprint in OpenShift (Part 2)],
and at
link:https://developers.redhat.com/blog/2017/04/04/openjdk-and-containers/[OpenJDK
These arguments are intended to return heap memory to the operating system whenever allocated memory exceeds 110% of in-use memory (`-XX:MaxHeapFreeRatio`), spending up to 20% of CPU time in the garbage collector (`-XX:GCTimeRatio`). At no time will the application heap allocation be less than the initial heap allocation (overridden by `-XX:InitialHeapSize` /
`-Xms`). Detailed additional information is available in link:https://developers.redhat.com/blog/2014/07/15/dude-wheres-my-paas-memory-tuning-javas-footprint-in-openshift-part-1/[Tuning Java's footprint in OpenShift (Part 1)], link:https://developers.redhat.com/blog/2014/07/22/dude-wheres-my-paas-memory-tuning-javas-footprint-in-openshift-part-2/[Tuning Java's footprint in OpenShift (Part 2)], and at link:https://developers.redhat.com/blog/2017/04/04/openjdk-and-containers/[OpenJDK
and Containers].
Comment on lines +50 to 61
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably out of scope for this PR but it would be nice to organize this as a list of arguments and what they do instead of a block that lists them and then a paragraph under - it's a bit hard to follow

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unwrap this line too?


[id="nodes-cluster-resource-configure-jdk-proc_{context}"]
== Understanding how to ensure all JVM processes within a container are appropriately configured

In the case that multiple JVMs run in the same container, it is essential to
ensure that they are all configured appropriately. For many workloads it will
be necessary to grant each JVM a percentage memory budget, leaving a perhaps
substantial additional safety margin.
In the case that multiple JVMs run in the same container, it is essential to ensure that they are all configured appropriately. For many workloads it will be necessary to grant each JVM a percentage memory budget, leaving a perhaps substantial additional safety margin.

Many Java tools use different environment variables (`JAVA_OPTS`, `GRADLE_OPTS`, and so on) to configure their JVMs and it can be challenging to ensure
that the right settings are being passed to the right JVM.
Many Java tools use different environment variables (`JAVA_OPTS`, `GRADLE_OPTS`, and so on) to configure their JVMs and it can be challenging to ensure that the right settings are being passed to the right JVM.

The `JAVA_TOOL_OPTIONS` environment variable is always respected by the OpenJDK,
and values specified in `JAVA_TOOL_OPTIONS` will be overridden by other options
specified on the JVM command line. By default, to ensure that these options are
used by default for all JVM workloads run in the Java-based agent image, the {product-title} Jenkins
Maven agent image sets:
The `JAVA_TOOL_OPTIONS` environment variable is always respected by the OpenJDK, and values specified in `JAVA_TOOL_OPTIONS` will be overridden by other options specified on the JVM command line. By default, to ensure that these options are used by default for all JVM workloads run in the Java-based agent image, the {product-title} Jenkins Maven agent image sets:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The `JAVA_TOOL_OPTIONS` environment variable is always respected by the OpenJDK, and values specified in `JAVA_TOOL_OPTIONS` will be overridden by other options specified on the JVM command line. By default, to ensure that these options are used by default for all JVM workloads run in the Java-based agent image, the {product-title} Jenkins Maven agent image sets:
The `JAVA_TOOL_OPTIONS` environment variable is always respected by the OpenJDK, and values specified in `JAVA_TOOL_OPTIONS` will be overridden by other options specified on the JVM command line. By default, to ensure that these options are used by default for all JVM workloads run in the Java-based agent image, the {product-title} Jenkins Maven agent image sets the following variable:


[source,terminal]
----
JAVA_TOOL_OPTIONS="-XX:+UnlockExperimentalVMOptions
-XX:+UseCGroupMemoryLimitForHeap -Dsun.zip.disableMemoryMapping=true"
JAVA_TOOL_OPTIONS="-Dsun.zip.disableMemoryMapping=true"
----

[NOTE]
====
The `UseCGroupMemoryLimitForHeap` option has been removed in JDK 11. Use `-XX:+UseContainerSupport` instead.
====

This does not guarantee that additional options are not required, but is
intended to be a helpful starting point.
This does not guarantee that additional options are not required, but is intended to be a helpful starting point.