Java heap space

[yifangt]

While running "phg create-maf-vcf " I hit the warning and error:

SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
SLF4J: Class path contains SLF4J bindings targeting slf4j-api versions 1.7.x or earlier.
SLF4J: Ignoring binding found at [jar:file:/storage/proj/Pea_hapmap/yifang/data/phg2/phg/lib/logback-classic-1.2.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See https://www.slf4j.org/codes.html#ignoredBindings for an explanation.
getVariantContextsfromMAF: Processing a single genome
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at java.base/java.lang.String.valueOf(String.java:4653)
        at biokotlin.genome.MAFToGVCF.buildSNP(MAFToGVCF.kt:1080)
        at biokotlin.genome.MAFToGVCF.buildTempVariants(MAFToGVCF.kt:609)

After search for those messages, it seems the SLF4J part is just warning that could be ignored. The actual problem is with the message OutOfMemoryError: java heap space, I believe.
I tried different numbers with:
export 'JVM_ARGS=-Xms4g -Xmx64g'
but the error stayed the same.
I am not familiar with JAVA, and this may be basically wrong to modify the java heap space.
What is the correct way to fix this problem?
Thanks a lot!

[zrm22]

Hello,

You are correct that you can ignore the SLF4J warning and the real error is the OutOfMemoryError.

As defined here: https://phg.maizegenetics.net/installation/#setting-memory
I think you have it set correctly. You may need to up the -Xmx value as high as it can go on your machine to have things process. This step can require a moderate amount of RAM. We are looking to improve this a bit and will hopefully have that ready to go in the near future. You can probably also remove the -Xms as it will likely need more than the minimum as well.

[yifangt]

Thanks a lot!
With you help, I increased the parameter to a crazy number as export 'JVM_ARGS=-Xmx1024g' and it still gave me the same error.
I was thinking maybe the assemblies number (8 plus Ref, each ~3.5Gbp) are too many for this practice, as 3 lines of the assemblies are almost complete as the reference genome. Does this make the sites too many to be handled in a single run? Or, is it still valid if I create individual graph compared with the Reference? Thanks again.

[tcasstevens]

I think you need to set the memory like this export JAVA_OPTS="-Xmx500g" And consider using some less than the total amount the machine has

…

On Mon, Mar 10, 2025 at 12:41 PM Yifang Tan ***@***.***> wrote: Thanks a lot! With you help, I increased the parameter to a crazy number as *export 'JVM_ARGS=-Xmx1024g'* and it still gave me the same error. I was thinking maybe the assemblies number (8 plus Ref, each ~3.5Gbp) are too many for this practice, as 3 lines of the assemblies are almost complete as the reference genome. Does this make the sites too many to be handled in a single run? Or, is it still valid if I create individual graph compared with the Reference? Thanks again. — Reply to this email directly, view it on GitHub <#288 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADUI6AWJOBMBYPGCQNA65L2TW6FLAVCNFSM6AAAAABYSF5ZWKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTENBVGE4DMOA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.*** com>

[zrm22]

Also just a quick update. I am working on some changes to the underlying biokotlin code to address this. In initial testing with Maize it reduced the RAM needed for one line from 84GB to 28GB(3x) and this likely will see similar or even better improvements with longer genomes. I am verifying that the resulting files are equivalent and will hopefully get this change into PHGv2 this week.

[yifangt]

Thank you!

[zrm22]

Please try the new version of PHGv2. We added in the changes which should help address this. There may be some additional changes we need to implement though but see if it works for you. If it does not please open up an issue.

[yifangt]

Testing is on the way, which was smooth so far except the agc-compress step, which did not finish after overnight run (>24hr now). Is this step supposed to take long time?

[yifangt]

Tried many times without luck. It seems to me there might be some issues with anchorwave for my samples, as there was not problem with the test dataset or single chromosome (tried so far).