About Piano Music Composer

This repository implements a compact Transformer-based Composer for piano event sequences and includes utilities to build datasets, train a model, save/load checkpoints, and generate playable 20-second piano pieces.

Data preprocessing and how the model learns melody

This project uses an event-level representation (implemented in midi2seq.py) to convert MIDI into compact token sequences that explicitly encode timing, note events, and velocity changes. Below is a step-by-step summary of how the data is used to train a melody model and how the model's outputs become a 20‑second musical excerpt.

Event vocabulary (see midi2seq.py):
- shift tokens encode short time deltas (decoded as val/100 + 0.01 seconds),
- down tokens represent note-on for a MIDI pitch,
- up tokens represent note-off for a MIDI pitch,
- velo tokens change expressive velocity bins.
Preprocessing pipeline (high level):
1. Raw MIDI files are read and converted to event sequences by midi2seq.process_midi_seq(...).
2. Long sequences are segmented into fixed-length segments (e.g. 100 tokens) suitable for batching and training. These segments are saved into dataset.npy for quick loading.
3. Each training example is a sequence of tokens where the model sees tokens [t0, t1, ..., t_{n-1}] as input and predicts the next-token targets [t1, t2, ..., t_n] (next-token / teacher-forcing training).
Training contract (what the model learns):
- The SimpleTransformer (in hw2.py) is trained with CrossEntropyLoss on next-token prediction. By learning to predict the immediate next event given a sequence of past events, the model implicitly learns local melodic and rhythmic structure (timing and pitch transitions) and to continue motifs present in the primer context.
- Typical training loop: create a DataLoader from dataset.npy, convert batches to torch.LongTensor, and call Composer.train(batch) which performs forward, loss, backward, gradient clipping and optimizer step. See train_from_dataset(...) and Composer.train_loop(...) in hw2.py.
Primer usage at generation time:
- A primer (start sequence) can be supplied to Composer.compose(start_sequence=...). The model conditions on the primer tokens (the last max_len tokens) and continues autoregressively.
- If no primer is provided, a canonical segment-start token is used so the model learns to generate from an implicit segmentation context.
Sampling to produce melody (how generation maps to 20s music):
1. The sampler runs autoregressively: at each step the model predicts logits for the next token from the last max_len tokens.

Logits are adjusted using temperature, a repetition penalty, and optional primer jitter (small Gaussian noise applied for the first few generated tokens to create variation around the primer).
Probability filtering is applied (top-k and/or top-p nucleus sampling). The sampler supports extra heuristics: diversity-boost (exclude argmax early) and diversity-exclude-k (remove top-k probabilities for initial steps) to encourage distinct continuations.
The sampler accumulates time when shift tokens are generated and stops once the accumulated time >= 20.0 s (and a minimum token count is met).
The raw event sequence is postprocessed:
- _trim_sequence_to_duration(...) ensures the sequence represents exactly 20.0 seconds (by splitting or padding shifts and dropping downs that would start at/after the cutoff),
- sanitize_sequence(...) removes unmatched up events and inserts minimal shifts to prevent zero-duration notes, and
- _melodic_cleanup(...) optionally removes very short down/up pairs, collapses tiny shifts, and normalizes velocity bins to reduce jitter.

Execution

python3 hw2.py --compose --load --n_variations 7 --out_template saurav_final_final_final_test_{i}.mid --temp 1.0 --top_k 200 --diversity_exclude_k 12 --diversity_exclude_steps 8 --primer_jitter_sigma 0.5 --primer_jitter_steps 8 --temp_jitter 0.12
timidity final_final_final_test_5.mid