A curated survey of datasets and benchmarks for Quranic and Islamic Question Answering research.
Detailed per-dataset analysis, limitations, and observations: datasets.md
| # | Dataset | Year | Size | Language | Format | Main Question Addressed | Dataset Link | Paper |
|---|---|---|---|---|---|---|---|---|
| 1 | AyaTEC | 2020 | 207 Q · 1,762 answers | Arabic | Verse-based QA | Which Quranic verses directly answer a given question (factoid + non-factoid, including zero-answer)? | GitLab · Task site | ACM TALLIP |
| 2 | QRCD v1.1 | 2022 | 1,093 QP pairs · 1,289 QPA triplets | Arabic | Extractive MRC (SQuAD-style) | Can a model extract the correct answer span from a given Quranic passage? | GitLab · GitHub | ACL · arXiv |
| 3 | QRCD v1.2 | 2023 | 1,155 QP pairs · 1,399 QPA triplets (~15% unanswerable) | Arabic | Extractive MRC + unanswerable | Can a model detect answer spans and correctly abstain when no answer exists in the Quran? | GitLab · HuggingFace | ACL |
| 4 | Al-Bayan | 2014 | Not reported | Arabic | Retrieval QA over Quran + Tafseer | Can a system retrieve the correct Quranic verse and Tafseer passage for a given question? | GitHub | ACL |
| 5 | AQQAC | 2018 | Not reported | Arabic | Ontology-based QA | Can an ontology-driven system answer questions over the Arabic Quran? | Leeds Repository | Semantic Scholar |
| 6 | QUQA | 2023 | Not reported | Arabic | Passage QA | Does a model understand the semantic and interpretative diversity of Quranic text in QA? | GitHub | ACL / RANLP 2023 |
| 7 | Aljaji et al. 2025 | 2025 | Not reported | Arabic | MCQ + Verse identification | Can LLMs answer Quranic MCQs stratified by cognitive difficulty (Bloom's taxonomy) and verse familiarity? | Not public | OpenReview / NeurIPS 2025 |
| 8 | Comprehensive Islamic QA (Qamar et al.) | 2024 | 73,000+ QA pairs | Arabic | Non-factoid QA | Can models answer complex interpretive questions over Quranic Tafsir and Hadith, and do automatic metrics reliably evaluate them? | HuggingFace | arXiv |
| 9 | Quran Tafseer Dataset | ~2023 | 84 tafseer books | Arabic | Knowledge base (RAG) | Does a RAG pipeline correctly retrieve and reason over Quranic commentary? | HuggingFace (MohamedRashad) · HuggingFace (tarteel) | No paper |
| 10 | Hajj-FQA | 2025 | 2,826 QA pairs (800 fatwas) | Arabic | Extractive QA over fatwas | Can models answer questions about Hajj based on expert-annotated Islamic legal rulings? | Not public | Springer |
| 11 | FiqhQA | 2025 | 960 QA pairs | Arabic + English | MCQ | Can LLMs answer Islamic jurisprudence questions across the four Sunni schools in a bilingual setting? | Not public | arXiv · AAAI AIES |
| 12 | QIAS 2025 | 2025 | From 32,000 fatwas | Arabic | MCQ (3 difficulty levels) | Can models reason over Islamic inheritance law at varying levels of complexity? | CodaBench · GitHub | ACL |
| 13 | MizanQA | 2025 | ~1,700 MCQ | Arabic | MCQ | Can models recall and reason over Maliki jurisprudence in the Moroccan Islamic legal tradition? | HuggingFace | arXiv |
| 14 | ARCD | 2019 | 1,395 questions | Arabic (MSA) | Extractive MRC | Can models perform open-domain Arabic reading comprehension over Wikipedia articles? | HuggingFace · GitHub | ACL · arXiv |
| 15 | Arabic-SQuAD | ~2019 | ~87,000+ QA pairs (MT) | Arabic | Extractive QA | Can models trained on machine-translated English QA generalize to Arabic reading comprehension? | HuggingFace · GitHub | ACL · arXiv |
| 16 | ArabicaQA | 2024 | 89,000+ answerable · 3,700 unanswerable | Arabic | Extractive QA + unanswerable | Can models answer Arabic Wikipedia questions and correctly identify unanswerable ones? | GitHub | arXiv · SIGIR 2024 |
| 17 | TyDi QA (Arabic) | 2020 | ~200K total; Arabic subset | Arabic + multilingual | Information-seeking QA | Can models answer information-seeking questions in typologically diverse languages including Arabic? | HuggingFace · GitHub | arXiv |
| 18 | Indonesian Quranic QA | 2014–2016 | ~222 ontology concepts | Indonesian | Factoid QA (ontology/rule-based) | Can a system answer Who/When/Where questions over the Indonesian Quran translation using ontologies? | Not public | Semantic Scholar (2014) |
| 19 | QASiNa | 2023 | Not reported | Indonesian | Extractive QA | Can multilingual transformers and LLMs answer questions over Sirah Nabawiyah (Prophet's biography)? | GitHub · HuggingFace | arXiv |
| 20 | IslamicPCQA | 2023 | 12,282 QA pairs | Persian | Multi-hop QA | Can models perform multi-step reasoning across multiple Islamic encyclopedic documents in Persian? | Not public | arXiv · IEEE |
| Domain | Datasets |
|---|---|
| Quran (direct text / verses) | AyaTEC, QRCD v1.1, QRCD v1.2, Al-Bayan, AQQAC, QUQA, Aljaji 2025 |
| Tafseer / Quranic commentary | Al-Bayan, Comprehensive Islamic QA, Quran Tafseer Dataset |
| Hadith | Comprehensive Islamic QA |
| Islamic jurisprudence (Fiqh) | Hajj-FQA, FiqhQA, QIAS 2025, MizanQA |
| Sirah (Prophet's biography) | QASiNa |
| General Arabic (used as baseline) | ARCD, Arabic-SQuAD, ArabicaQA, TyDi QA |
| Non-Arabic Islamic texts | Indonesian Quranic QA (Indonesian), QASiNa (Indonesian), IslamicPCQA (Persian) |
- No hallucination benchmark — no dataset is designed to measure LLM hallucinations on Quranic content
- Automatic metrics unreliable — ROUGE scores agree with Islamic scholars only ~11–20% of the time (Qamar et al., 2024)
- No generative QA evaluation — most benchmarks are extractive (span) or MCQ; free-form answer quality is unmeasured
- No morpho-syntactic-semantic evaluation — complex Arabic linguistic queries combining morphology + syntax + semantics are not covered
- Small sizes — all Quranic-specific datasets are under 10,000 QA pairs, insufficient for LLM fine-tuning
- Licensing undocumented — nearly all datasets lack explicit license information
Based on literature review covering 2014–2025. Last updated: March 2026.