Skip to content

Conversation

@Pangjing-Wu
Copy link

No description provided.

@dangmingcong
Copy link

In the data preprocessing part, did you use 200G of RAM to process one-tenth of the data?

@Pangjing-Wu
Copy link
Author

In the data preprocessing part, did you use 200G of RAM to process one-tenth of the data?

I can completely load the data with about 120G RAM

@dangmingcong
Copy link

Did you handle this with unmodified code, or did you modify the code to load the relationships between different files in sequence?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants