About Piano Music Composer

This repository implements a compact Transformer-based Composer for piano event sequences and includes utilities to build datasets, train a model, save/load checkpoints, and generate playable 20-second piano pieces.

Data preprocessing and how the model learns melody

This project uses an event-level representation (implemented in midi2seq.py) to convert MIDI into compact token sequences that explicitly encode timing, note events, and velocity changes. Below is a step-by-step summary of how the data is used to train a melody model and how the model's outputs become a 20‑second musical excerpt.

• Event vocabulary (see midi2seq.py):

  • shift tokens encode short time deltas (decoded as val/100 + 0.01 seconds),
  • down tokens represent note-on for a MIDI pitch,
  • up tokens represent note-off for a MIDI pitch,
  • velo tokens change expressive velocity bins.

• Preprocessing pipeline (high level):

  1. Raw MIDI files are read and converted to event sequences by midi2seq.process_midi_seq(...).
  2. Long sequences are segmented into fixed-length segments (e.g. 100 tokens) suitable for batching and training. These segments are saved into dataset.npy for quick loading.
  3. Each training example is a sequence of tokens where the model sees tokens [t0, t1, ..., t_{n-1}] as input and predicts the next-token targets [t1, t2, ..., t_n] (next-token / teacher-forcing training).

• Training contract (what the model learns):

  • The SimpleTransformer (in hw2.py) is trained with CrossEntropyLoss on next-token prediction. By learning to predict the immediate next event given a sequence of past events, the model implicitly learns local melodic and rhythmic structure (timing and pitch transitions) and to continue motifs present in the primer context.
  • Typical training loop: create a DataLoader from dataset.npy, convert batches to torch.LongTensor, and call Composer.train(batch) which performs forward, loss, backward, gradient clipping and optimizer step. See train_from_dataset(...) and Composer.train_loop(...) in hw2.py.

• Primer usage at generation time:

  • A primer (start sequence) can be supplied to Composer.compose(start_sequence=...). The model conditions on the primer tokens (the last max_len tokens) and continues autoregressively.
  • If no primer is provided, a canonical segment-start token is used so the model learns to generate from an implicit segmentation context.

• Sampling to produce melody (how generation maps to 20s music):

  1. The sampler runs autoregressively: at each step the model predicts logits for the next token from the last max_len tokens.

2. Logits are adjusted using temperature, a repetition penalty, and optional primer jitter (small Gaussian noise applied for the first few generated tokens to create variation around the primer).
3. Probability filtering is applied (top-k and/or top-p nucleus sampling). The sampler supports extra heuristics: diversity-boost (exclude argmax early) and diversity-exclude-k (remove top-k probabilities for initial steps) to encourage distinct continuations.
4. The sampler accumulates time when shift tokens are generated and stops once the accumulated time >= 20.0 s (and a minimum token count is met).
5. The raw event sequence is postprocessed:
   • _trim_sequence_to_duration(...) ensures the sequence represents exactly 20.0 seconds (by splitting or padding shifts and dropping downs that would start at/after the cutoff),
   • sanitize_sequence(...) removes unmatched up events and inserts minimal shifts to prevent zero-duration notes, and
   • _melodic_cleanup(...) optionally removes very short down/up pairs, collapses tiny shifts, and normalizes velocity bins to reduce jitter.

Execution

1. python3 hw2.py --compose --load --n_variations 7 --out_template saurav_final_final_final_test_{i}.mid --temp 1.0 --top_k 200 --diversity_exclude_k 12 --diversity_exclude_steps 8 --primer_jitter_sigma 0.5 --primer_jitter_steps 8 --temp_jitter 0.12

2. timidity final_final_final_test_5.mid

Piano Music Samples

Sample_1

---

Sample_2

---

Sample_3

---

Sample_4

---

Sample_5

---

Sample_6

---

Sample_7

---

Sample_8

---

Technical documentation