GC overhead limit exceeded

[jatinagarwal]

Even after using 13g of memory I get "GC overhead limit exceeded" exception with following stack trace:
Picked up _JAVA_OPTIONS: -Xms2g -Xmx13g -Xss100M -XX:MaxPermSize=3g
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Arrays.copyOfRange(Arrays.java:2694)
at java.lang.String.(String.java:203)
at java.lang.String.substring(String.java:1913)
at java.lang.String.split(String.java:2288)
at java.lang.String.split(String.java:2355)
at be.tarsos.lsh.util.FileUtils.readCSVFile(Unknown Source)
at be.tarsos.lsh.LSH.readDataset(Unknown Source)
at be.tarsos.lsh.CommandLineInterface.parseArguments(Unknown Source)
at be.tarsos.lsh.LSH.main(Unknown Source)

My dataset size is 4GB, it contains 15201 vectors. I run with following options "java -jar TarsosLSH-0.7.jar -f l2 -h 8 -t 32 -n 15 -d tfVectorFor105G -q tfVectorQueriesOn105g"

[JorenSix]

This seems indeed to be a problem related to memory. The -X params should be able to correct things. How large (dimensions) are your vectors?

[jatinagarwal]

Vectosr have 66669 dimensions.