Skip to content

data

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

data

README.md

Data

This folder hosts the data used in the different tutorials and hands-on sessions. Short descriptions are provided below, please refer to the original sources for further information.

The preprocessing folder contains pre-processing scripts used to obtain the presented data files. It is included for transparency and to allow reproduction, but was not developed for pedagogical purposes (-> the code is likely quick and dirty).

Week 1 — 29.10

Tutorial 1: Word Sense Disambiguation

  • File(s): word_sense/CoarseWSD-20.csv
  • Description:
    • Dataset based on Wikipedia articles excerpts, developed to evaluate and train word disambiguation models based on a selection of 20 expert-selected words (apple, club, bow, bank, crane, square, chair, java, bass, seal, pitcher, arm, mole, pound, deck, trunk, spring, hood, yard, digit).
    • columns: text, target, label, label_sense, split
  • Source: Daniel Loureiro, Kiamehr Rezaee, Mohammad Taher Pilehvar, & Jose Camacho-Collados. (2021). Analysis and Evaluation of Language Models for Word Sense Disambiguation. arXiv

Hands-on 1: Semantic Shifts

Data from the Living With Machines programme. Please check their website if interested in digital history, and to discover their research on rethinking the impact of technology on people's lives during the Industrial Revolution.

  • File(s): word_sense/lwm_annot.csv
  • Description:
    • Excerpts of british historical newspaper articles (1818-1920), OCR texts, annotated for word sense for four target words (trolley, car, bike, coach).
    • columns: text, annot, newspaper_date, newspaper_title, newspaper_place, image_url, image_url_conflict_values, target, year.
  • Source: Ridge, M., Pedrazzini, N., Vieira, J. M. M., Ciula, A., & McGillivray, B. (2024). Language of mechanisation crowdsourcing datasets from the living with machines project. Journal of Open Humanities Data, 10, 33. Link ; Data Repository

Week 2 — 05.11

Tutorial 2: Topic Modeling

Nineteenth century American Recipes

  • File(s): topic_data/nineteenth_recipes.csv
  • Description:
    • A collection of 1,034 nineteenth-century American recipes, extracted from cookbooks available through the Project Gutenberg. Each entry includes the recipe text and its corresponding Project Gutenberg book ID.
    • columns: text, ids (Project Gutenberg ID of the source book)
  • Source: Blankenship, A. (2021). A Dataset of Nineteenth-Century American Recipes. Viral Texts: Mapping Networks of Reprinting in 19th-Century Newspapers and Magazines. GitHub Repository

4Newsgroups

  • File(s): topic_data/4newsgroups.csv
  • Description:
    • The 20 Newsgroups data set is a collection of approximately 18,000 newsgroups posts, partitioned (nearly) evenly across 20 different newsgroups. Here, a filtered version is provided: it is reduced to only 4 topics, and 1,969 documents.
    • columns: text, label (original source / topic)
  • Source: Lang, K. (1995). NewsWeeder: learning to filter netnews. In Proceedings of the Twelfth International Conference on International Conference on Machine Learning (pp. 331–339). Morgan Kaufmann Publishers Inc. Link ; Data loaded from scikit-learn handle.

Hands-on 2:

  • File(s): topic_data/ungdc.csv, ``
  • Description:
    • The UN General Debate Corpus (UNGDC) contains 10,952 speeches delivered by representatives from 202 countries (including historical states) during the annual UN General Assembly General Debate from 1946 to 2024. Each speech captures national perspectives on global political issues and provides a longitudinal dataset for the study of international relations, political discourse, and global affairs. An alternative version with coarse heuristic-based paragraphs split is provided as well.
    • columns:
      • ungdc.csv : text, country_iso, year, session
      • ungdc_coarse_paragraphs.csv: text, country_iso, year, session, paragraph_id
  • Source: Jankin, S., Baturo, A., & Dasandi, N. (2024). Words to unite nations: The complete UN General Debate Corpus, 1946–present. Journal of Peace Research. DOI: 10.1177/00223433241275335; Data Repository.

Week 3 — 12.11

Tutorial 3: Book Genre Classification

  • File(s): literary_sft/sampled_chunks.csv | (Metadata: literary_sft/retained_books.csv )
  • Description:
    • 5-sentences excerpts from books sourced from Project Gutenberg. The original dataset contains nearly 6M chunks, here it is reduced to ~19.5K chunks randomly extracted from 1,000 books balanced over 4 genres (detective and mystery stories; science fiction; adventure stories; and children's stories). The data is furthered split into train, validation and test samples.
    • Columns:
      • sampled_chunks.csv: book_id, chunk_id, text, author_gender, label, split
      • retained_books.csv: book_id, title, author, author_gender, author_birth_year, author_death_year, pg_subjects, topic
    • Note: to prevent important data leakage, there cannot be chunks from the same book in different data splits.
  • Source: Despina Christou & Grigorios Tsoumakas. (2025). Artificial Relationships in Fiction: A Dataset for Advancing NLP in Literary Domains. In Proceedings of the 9th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL 2025) (pp. 130–147). Association for Computational Linguistics. Paper | Source Dataset

Hands-on 3: Canonicity Prediction

  • File(s): canon_challenge/dataset
  • Description:
    • This dataset is based on the ANRChapitres Corpus: A corpus of 2031 copyright-free French-language novels published between the early XIXth to mid XXth century. It was devised as part of the ANR Chapitres project. Notably, this coprus was annotated with "canonicity" tags.
    • The canon tags are affixed at the author-level. They are based on academic interests in an author's work, and determined via the number of occuerences on fabula.org, an academic literature research website (see metadata discussion).
    • The provided version is a sampled version of the original corpus, where text chunks are extracted from the book. The excerpts are five-sentence long segments extracted at the beginning of (at maximum) 10 random chapters of the books. It is further partitionned into train/validation/test splits. It is provided as an arrow-formatted dataset.
    • Columns: author, author_gender, book_title, publication_year, text, label
    • Note: Partition is made at the book level. The labels of the test set in canon_challenge/dataset are kept secret!
  • Source: ANRChapitres/2000romans19e20e: Corpus Chapitres (Leblond et al., 2023). Data Repository: DOI

Week 4 — 19.11

Hands-on 4: Machine Behavior

No big data for this session! Only a .json formatted questionnaire is provided as an example (more for the format than for the content) for the hands-on evaluation session.

  • File(s): questionnaires/example_questionnaire.json
  • Description:
    • MCQ complying with the json format used by the Questionnaire class used in the hands-on session
    • Fields:
      • "categories" (List[str]): names of the categories
      • "min_labels" (List[str]): labels for low scores for each category
      • "max_labels" (List[str]): labels for high scores for each category
      • "data" (dict[str]): content of the questionnaire indexed by item IDs. Each item has:
        • "question" (str): question (which will be included in the prompt template)
        • "choices" (List[str]): list of possible responses (which will be included in the prompt template)
        • "scores" (List[List]): scores associated with each choice. Score for an individual choice is a distribution over the categoires (e.g., a score entry [0.5, 1, -0.2], if associated option is selected, would add +0.5 to the first category, +1 to the second one, and -0.2 to the third one).
  • Source: —