A collection of complementary taxonomies for describing, annotating, and classifying human vocal audio. Together they cover speech characteristics, emotional expression, and non-speech vocalizations — plus an applied layer of hard delivery cases that composes all three.
| Taxonomy | Scope | Categories | Scale |
|---|---|---|---|
| VoiceNet | Speech dimensions (how someone speaks) | 59 dimensions | 7-point (0-6) per dimension |
| EmoNet | Emotional categories (what someone feels) | 40 categories | Categorical with valence/arousal |
| VocalBurst | Non-speech sounds (vocal bursts) | 82 categories | Categorical across 16 groups |
| Acting Edge Cases | Hard expressive-delivery situations to generate on purpose | 266 cases / 7 families | Director's coaching + detector-token intentions |
59 perceptual dimensions for describing speech along orthogonal axes.
Organized into 10 groups: Rhythm & Timing, Social & Interpersonal, Speaker Identity, Emotion & Affect, Physical Production, Spectral & Timbral, Temporal Dynamics, Language & Recording, Resonance Placement, and Speaking Style.
Each dimension uses a 7-point scale with detailed anchor descriptions. Example:
TEMP (Tempo): Speed of word/syllable production
0 = Glacially slow, syllables stretched to breaking point
3 = Standard conversational tempo, balanced and natural
6 = Hyper-accelerated, blistering speed (auctioneer/panic)
40 emotion categories with associated keywords, valence, and arousal annotations.
Spans the full emotional spectrum from positive states (Amusement, Elation, Contentment) through neutral (Concentration, Contemplation) to negative (Fear, Anger, Sadness) and mixed states (Longing, Teasing).
Each category includes:
-
Valence classification (positive/negative/neutral/mixed)
-
Arousal tendency (low to high)
-
4-10 descriptive keywords per category
-
Source: EmoNet Phase Paper
82 non-speech vocal sound categories across 16 semantic groups.
Covers: Laughter (8), Crying & Distress (5), Breathing (7), Sighs (4), Gasps (3), Groans & Moans (4), Grunts (3), Throat Sounds (7), Humming (4), Coughing & Sneezing (7), Whistling (5), Mouth & Lip Sounds (9), Tongue Clicks (4), Eating & Drinking (6), Screams (2), Hand & Body Sounds (4).
266 hard expressive-delivery situations — each written as a short direction an actor could take, with what it should do to the signal and which measurable dimensions ought to move. Unlike the three base taxonomies, this is an applied layer: every case's detector_tokens reference VoiceNet dimension codes, EmoNet emotions and VocalBurst types — so a case is also a falsifiable claim about what a detector should see. Built to drive prompt generation for a synthetic voice-acting corpus, where the dialogue is deliberately neutral and the performance lives only in the delivery.
Seven families: Emotion, Masking & the Social Game (84) · Voice-Craft, Bursts & Delivery (58) · Life, Language & Society (32) · Body, Illness, Pain & Substances (31) · Physical Exertion & Environment (26) · Consciousness, Shock & Altered States (25) · Crisis, Emergency & Survival (10).
┌─────────────────────────────────────────────────────────────┐
│ Human Vocal Audio │
├─────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ VoiceNet │ │ EmoNet │ │ VocalBurst │ │
│ │ │ │ │ │ │ │
│ │ HOW someone │ │ WHAT someone│ │ NON-SPEECH sounds │ │
│ │ speaks │ │ feels │ │ someone makes │ │
│ │ │ │ │ │ │ │
│ │ 59 dims │ │ 40 emotions │ │ 82 sound types │ │
│ │ continuous │ │ categorical │ │ categorical │ │
│ └─────────────┘ └─────────────┘ └─────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
- VoiceNet answers: How fast, loud, rough, warm, formal, etc. is this speech?
- EmoNet answers: What emotion is being expressed?
- VocalBurst answers: What non-speech sound is this?
- Acting Edge Cases sit on top: hard situations to generate on purpose, each expressed as a combination of VoiceNet, EmoNet and VocalBurst tokens.
All taxonomies are available as JSON files for programmatic access:
import json
# Load any taxonomy
with open("voicenet/voicenet_taxonomy.json") as f:
voicenet = json.load(f)
with open("emonet/emonet_taxonomy.json") as f:
emonet = json.load(f)
with open("vocalburst/vocalburst_taxonomy.json") as f:
vocalburst = json.load(f)
with open("acting_edge_cases/edge_cases.json") as f:
edge_cases = json.load(f) # list of 266 casesCC-BY-4.0
- Voice Annotation Data v2: https://huggingface.co/datasets/TTS-AGI/voice-annotation-data-v2
- Vocal Burst Database: https://huggingface.co/datasets/laion/vocal-burst-db
- LAION: https://laion.ai