Skip to content

Attributes to add/modify #663

Description

@zaidalyafeai
  • Partial Attribute for datasets that might not include all attributes. For example, some social media datasets require re-clawring.
  • UnitAdd conversations in for tha attribute.
  • Source the source of the non-annotated data. ['web pages', 'public datasets', 'books', 'blogs', 'scientific articles', 'wikipedia', 'TV channels', 'LLM']
  • Test Split rename to Has_Splits
  • Domain is for the domain of the dataset i.e., ['Law', 'Science', 'News', 'General', 'Quran', 'Culture', 'General']
  • Collection Style should be either ['crawling', 'manual curation']
  • Annotation Style to better represent how the dataset was annotated, and add another option for no annotation, for example, public datasets are inheritly annotated annotated ['Machine', 'Human', 'Inehrited']
  • Define a list of venues for easier annotation
  • Rename Subsets to Dialect Subsets for better representation
  • Preprocessing rather than Tokenized this general attributes define the preprocessning steps for the dataset i.e. ['tokenized','normalized','remove diacritization', 'other']
  • Benchmarks are ill-defined in our definition; also, some datasets might include subsets other than dialects

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions