Possible Data Corruption in Criteo 1 TB Click Logs Dataset

[IFOnlyC]

I’m preparing the dataset for the MLPerf Training v5.0 - DLRM v2 benchmark and have encountered repeated errors when processing the Criteo 1 TB Click Logs dataset hosted on Hugging Face.

After downloading all 24 daily archives, I invoke the preprocessing script as documented:

bash ./scripts/process_Criteo_1TB_Click_Logs_dataset.sh \
    ./criteo_1tb/raw_input_dataset_dir \
    ./criteo_1tb/temp_intermediate_files_dir \
./criteo_1tb/numpy_contiguous_shuffled_output_dataset_dir

For every file (day_0.gz through day_23.gz), the script halts with:

gzip: day_.gz: unexpected end of file

This suggests each .gz archive may be truncated or otherwise corrupted on download. I have verified:

• Tried multiple download methods (Hugging Face CLI, wget, browser) on different machines and networks—same result.
• Re-running the script and re-downloading for each individual day zip file yields the same error.

[matthew-frank]

There is a discussion on the HuggingFace forums where other people are also seeing this problem. https://discuss.huggingface.co/t/unable-to-extract-the-criteo-criteoclicklogs-dataset/157616

[HiIcy]

i met the same issue, Any solution have?

[ShriyaRishab]

Hi, we are working on hosting the dataset in MLC and should have instructions to download available soon.

[ShriyaRishab]

Hi all,
We have uploaded the dataset to mlcommons-storage. Can you follow those instructions and see if you are able to access the data?

Note that its already preprocessed and thus can be used for training directly.

[ShriyaRishab]

Closing as clean data is now uploaded to MLC storage