Comprehensive collection of open datasets, dataset tools, and data repositories for ML/AI training, research, and benchmarking.
| Repository | Description | Stars |
|---|---|---|
| awesomedata/awesome-public-datasets | Topic-centric list of high-quality public datasets across every domain |  |
| huggingface/datasets | 50,000+ datasets for ML — unified API, streaming, fast loading |  |
| openml/openml-python | Python API for OpenML — 20,000+ datasets with benchmarks |  |
| fivethirtyeight/data | Data and code behind FiveThirtyEight articles |  |
| datahub-project/datahub | Metadata platform for data discovery and governance |  |
| niderhoff/nlp-datasets | Alphabetical list of free NLP datasets with descriptions |  |
| datasets/awesome-data | Curated awesome datasets — economics, finance, geo, and more |  |
| google-research-datasets | Google Research open datasets — official releases |  |
| Repository | Description | Stars |
|---|---|---|
| cocodataset/cocoapi | MS COCO — detection, segmentation, keypoints, captions |  |
| cvdfoundation/open-images-dataset | Google Open Images — 9M images, 600 classes, bounding boxes |  |
| pytorch/vision | TorchVision — CIFAR, ImageNet, MNIST, STL10, VOC, and more |  |
| ultralytics/ultralytics | YOLO dataset tools — COCO, Objects365, OpenImagesV7 support |  |
| openvinotoolkit/datumaro | Dataset management — convert between all CV dataset formats |  |
| google/mediapipe | Cross-platform ML pipeline with face, hand, pose datasets |  |
| visipedia/inat_comp | iNaturalist — 2.7M images of plants and animals |  |
| fastai/imagenette | Subset of 10 easily classified ImageNet classes — fast experimentation |  |
| ieee8023/covid-chestxray-dataset | COVID-19 chest X-ray dataset for medical imaging research |  |
| waymo-research/waymo-open-dataset | Waymo Open Dataset — lidar and camera data for autonomous driving |  |
| nutonomy/nuscenes-devkit | nuScenes — full autonomous driving dataset with devkit |  |
| commaai/comma2k19 | 2000 hours of commute data for self-driving research |  |
| facebookresearch/detectron2 | Facebook detection platform — includes dataset loaders for major CV datasets |  |
| Labelbox/labelbox-python | Data labeling platform SDK — annotation + dataset management |  |
| Repository | Description | Stars |
|---|---|---|
| google-research/bert | BERT pre-training scripts — Wikipedia + BookCorpus pipeline |  |
| google-research-datasets/natural-questions | Real Google user queries + Wikipedia answers | |
| allenai/allennlp | NLP research library — dataset readers for 50+ datasets |  |
| explosion/spacy-models | spaCy trained models + dataset pipeline tools |  |
| EleutherAI/the-pile | 825GB diverse open-source text for LLM training |  |
| togethercomputer/RedPajama-Data | Open reproduction of LLaMA training dataset — 1.2T tokens |  |
| tatsu-lab/stanford_alpaca | Stanford Alpaca — 52K instruction-following examples |  |
| sahil280114/codealpaca | Code Alpaca — 20K instruction-following examples for code |  |
| Instruction-Tuning-with-GPT-4/GPT-4-LLM | GPT-4 generated instruction-following dataset |  |
| bigscience-workshop/xP3 | Multilingual dataset collection for instruction tuning |  |
| smashew/NameDatabases | Large name database from 182 countries |  |
| karpathy/char-rnn | Character-level datasets — Shakespeare, Wikipedia, Linux kernel |  |
| Repository | Description | Stars |
|---|---|---|
| openai/whisper | 680K hours of multilingual speech — Whisper training data |  |
| mozilla/DeepSpeech | Mozilla Common Voice + DeepSpeech training pipeline |  |
| coqui-ai/STT | Open speech-to-text engine with training datasets |  |
| magenta/magenta | Google's music and audio generation — MIDI datasets included |  |
| facebookresearch/encodec | High-fidelity neural audio codec — audio datasets and tools |  |
| suno-ai/bark | Text-to-audio model — training data references included |  |
| jameslyons/python_speech_features | Speech feature extraction — MFCC, filterbank tools |  |
| Repository | Description | Stars |
|---|---|---|
| deepmind/alphafold | AlphaFold — protein structure prediction dataset and model |  |
| deepmind/deepmind-research | Implementations and datasets from DeepMind research papers |  |
| allenai/s2orc | Semantic Scholar Open Research Corpus — 200M+ paper abstracts |  |
| allenai/science-parse | Parse scientific papers — AI2 research dataset tools |  |
| nasa/NASA-Open-APIs | NASA open APIs and datasets — space, climate, imagery |  |
| owid/covid-19-data | Our World in Data — COVID-19 dataset updated daily |  |
| beoutbreakprepared/nCoV2019 | COVID-19 epidemiological data — global case tracking |  |
| Repository | Description | Stars |
|---|---|---|
| ranaroussi/yfinance | Yahoo Finance market data — stocks, crypto, forex |  |
| JerBouma/FinanceDatabase | 300,000+ financial instruments — stocks, ETFs, funds |  |
| philipperemy/financial-news-dataset | Reuters and Bloomberg financial news dataset |  |
| jealous/stockstats | Stock statistics utilities + dataset tools |  |
| Repository | Description | Stars |
|---|---|---|
| gboeing/osmnx | Download and analyze OpenStreetMap geospatial data |  |
| microsoft/GlobalMLBuildingFootprints | 1.1B building footprints worldwide — Bing Maps ML dataset |  |
| awslabs/open-data-registry | AWS Open Data Registry — satellite, climate, genomics |  |
| openclimatefix/open-climate-fix | Open climate datasets for ML and research |  |
| openmapflow/openmapflow | Crop mapping datasets using satellite imagery |  |
| Repository | Description | Stars |
|---|---|---|
| github/CodeSearchNet | 6M functions from open source code — code search benchmark |  |
| microsoft/CodeXGLUE | Benchmark dataset for code intelligence tasks |  |
| bigcode-project/starcoder | The Stack — 6.4TB of permissively licensed source code |  |
| openai/human-eval | HumanEval — hand-crafted programming problems for LLM evaluation |  |
| THUDM/CodeGeeX | Multilingual code generation dataset and model |  |
| google-research/google-research | Google Research code and datasets across all domains |  |
| Repository | Description | Stars |
|---|---|---|
| openai/evals | OpenAI evaluation framework + benchmark datasets for LLMs |  |
| EleutherAI/lm-evaluation-harness | 200+ NLP benchmark datasets in one framework |  |
| google/BIG-bench | Beyond Imitation Game — 200+ tasks for LLM evaluation |  |
| facebookresearch/mmf | Multimodal framework — VQA, image captioning benchmarks |  |
| paperswithcode/sota-extractor | State-of-the-art results extraction — benchmarks tracker |  |
| mwaskom/seaborn-data | Clean tabular datasets used in seaborn — great for quick experiments |  |
| Repository | Description | Stars |
|---|---|---|
| heartexlabs/label-studio | Open source data labeling — images, text, audio, video |  |
| opencv/cvat | Computer Vision Annotation Tool by Intel/OpenCV |  |
| iterative/dvc | Data Version Control — manage large datasets like code |  |
| activeloopai/deeplake | Dataset storage for deep learning — stream to PyTorch/TF |  |
| cleanlab/cleanlab | Find and fix label errors in datasets — data-centric AI |  |
| snorkel-team/snorkel | Build training datasets programmatically — no manual labeling |  |
| albumentations-team/albumentations | Fast image augmentation library for expanding CV datasets |  |
| joke2k/faker | Generate fake data for testing — names, addresses, emails |  |
| ydataai/ydata-profiling | Dataset profiling — full EDA report in one line |  |
| great-expectations/great_expectations | Data quality validation — test and document your datasets |  |