Skip to content

Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded #198

Open
@jialu-stellar-xia

Description

I have this issue when importing the data to the format for LDA. I tried enlarge the MALLET_MEMORY=128G (the memory of my server is also 128G), but it still does not work.
My data contains 6,712,484 documents in one .txt file and its size is 3.07G
I sampled 100 documents to test the script for importing data, it works well. But keep popping this error message when importing my entire data.
Could you please help to figure out the problem? Really appreciate your help!!
截屏2021-04-11 下午8 14 08

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions