Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 14 additions & 10 deletions PLANS.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,9 @@ M1 should include:

Use this section as the current execution order for agent work. Feature document numbering describes the planned product sequence, but actual issue order may change as dependencies, review feedback, or implementation risks become clearer.

1. #19 Pitched instrument selection and Iowa Piano sustain -> `docs/features/08-pitched-instruments-and-sustain.md` (PR #22 in review)
1. #25 Tempo control and live BPM updates -> `docs/features/10-tempo-control-and-bpm-slider.md`
2. #26 Arrangement view UI shell -> `docs/features/11-arrangement-view-ui-shell.md`
3. #23 Sampler for advanced sustain -> `docs/features/09-sampler-advanced-sustain.md`

## Planned Milestones

Expand All @@ -39,22 +41,24 @@ Use this section as the current execution order for agent work. Feature document
4. Drum step sequencer.
5. Basic piano roll.
6. Transport pause/resume and editor playhead.
7. Pitched instrument selection and Iowa Piano sustain.
8. Hybrid clip loop playback.
9. Sample import.
10. Project export/import.
11. IndexedDB autosave.
12. Arrangement data model.
13. Arrangement view.
14. Mixer and basic effects.
7. Pitched instrument selection and Iowa Piano one-shot playback.
8. Tempo control and live BPM updates.
9. Arrangement view UI shell.
10. Sampler advanced sustain.
11. Hybrid clip loop playback.
12. Sample import.
13. Project export/import.
14. IndexedDB autosave.
15. Arrangement data model.
16. Arrangement editing and playback.
17. Mixer and basic effects.

## Backlog

- Keyboard shortcuts for transport and editing.
- Basic undo and redo.
- Velocity editing for drum and note events.
- Clip duplication.
- BPM and transport controls.
- Starter project template.
- Metronome.
- Quantize utilities.
Expand Down
35 changes: 33 additions & 2 deletions docs/audio-engine.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,20 +76,37 @@ Sample-based pitched playback should:

The first Iowa Piano implementation should avoid sustain looping. The bundled one-second piano samples are not reliable loop-ready sustain samples yet, and early loop points can make long notes sound like repeated strikes. Iowa Piano notes should currently play the sample once from the configured start offset, then stop or naturally decay.

A future sampler sustain task may add explicit `loopStartSeconds` and `loopEndSeconds`; if those values are present, the source node may use `loop = true` with those loop points. Loop metadata is serializable instrument/sample metadata, but decoded buffers and active source nodes are runtime-only.
A future sampler sustain task may add explicit sustain metadata with `loopStartSeconds` and `loopEndSeconds`; if those values are present and valid, the source node may use `loop = true` with those loop points. Loop metadata is serializable instrument/sample metadata, but decoded buffers and active source nodes are runtime-only.

Do not attempt automatic loop point detection in the first pass. If loop points are added later, update the explicit metadata and document the decision.

Current Iowa Piano sample zones use explicit `sampleStartSeconds` offsets because the bundled one-second C4-C5 WAV files contain leading silence before the audible note attack. The engine should pass that value as the second argument to `AudioBufferSourceNode.start(when, offset)`.

Current Iowa Piano sample zones intentionally omit `loopStartSeconds` and `loopEndSeconds`, so the audio engine does not enable `AudioBufferSourceNode.loop` for Iowa Piano.
Current Iowa Piano sample zones intentionally omit sustain metadata, so the audio engine does not enable `AudioBufferSourceNode.loop` for Iowa Piano.

The audio engine should load sample zones needed by selected note events before handing them to the lookahead scheduler. If a sample-based instrument has no zone for a note, the engine may fall back to `Default Synth` behavior for that note rather than storing any runtime fallback state in project data.

Each scheduled note event carries an `instrumentId`. The scheduler should play all note events in the selected clip, not only the currently visible piano roll lane, so multiple pitched instruments can sound together.

When `decodeAudioData` cannot decode a bundled PCM WAV file, the browser engine may fall back to a local PCM WAV decoder and create an `AudioBuffer` manually. This fallback is runtime-only and does not change project data.

## Sampler Sustain

Sampler sustain should be a reusable audio-engine capability for sample-based pitched instruments, not an `Iowa Piano` special case.

Sampler sustain should:

- Read serializable sample-zone metadata such as `sampleStartSeconds`, `sampleEndSeconds`, sustain loop mode, loop start/end points, optional crossfade length, and envelope settings.
- Treat sample-local offsets, loop points, and envelope lengths as seconds because they describe positions inside a decoded sample buffer.
- Keep musical event positions and durations in ticks and convert them to seconds only at scheduling time.
- Validate loop metadata before enabling looping.
- Fall back to one-shot sample playback when loop metadata is missing or invalid.
- Apply release behavior at note end, transport pause, and transport stop so sustained voices do not remain stuck.

The first implementation should prefer explicit metadata over automatic analysis. Automatic loop point detection, visual loop point editing, velocity layers, round-robin selection, and full sampler preset management belong in later tasks.

If a zone uses a simple forward loop, the engine may use `AudioBufferSourceNode.loop`, `loopStart`, and `loopEnd`. If a future task adds crossfaded looping, it may need multiple scheduled source nodes and gain ramps because Web Audio buffer source looping does not provide native loop crossfade behavior.

## Lookahead Scheduler Concept

Do not rely on UI timers for exact playback. Sequenced playback should use a timer that wakes frequently, looks ahead by a short scheduling window, and schedules Web Audio events against `AudioContext.currentTime`.
Expand Down Expand Up @@ -139,6 +156,20 @@ Pause and stop have different meanings:

The audio engine should expose pause separately from stop. A pause operation preserves the scheduler object and its current tick snapshot; a stop operation clears the active loop and returns the transport to the loop start.

## Tempo Control

Tempo is musical project state, not UI-only state. The transport BPM slider should update the current `tempoBpm` value used by audio scheduling.

Tempo changes should follow these rules:

- When stopped, the next playback start uses the current BPM.
- When paused, resume uses the current BPM from the preserved paused tick.
- When playing, BPM changes should affect future lookahead scheduler windows.
- Existing musical event positions stay in ticks. Tempo changes alter tick-to-seconds conversion, not event tick positions.
- React may own the visible slider value, but exact event scheduling must continue to use audio-engine transport state and `AudioContext.currentTime`.

Prefer an audio-engine API that can update active scheduler tempo while preserving the current runtime tick. If the first implementation restarts the selected clip loop from the current tick to apply tempo changes, document the limitation and keep the schedule-ahead window short enough for interactive use.

## Looping Behavior

For M1, loop playback targets a selected 1-bar clip. The default loop range is 0 to 1920 ticks.
Expand Down
41 changes: 38 additions & 3 deletions docs/data-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,14 @@ For 4/4:

Seconds are derived at playback time from ticks and tempo. Do not store seconds as the primary event position.

## Tempo

Project tempo should be represented as serializable BPM data, using a field such as `tempoBpm` on the project or current app-level project state until full persistence exists.

Changing tempo must not rewrite clip event positions. Drum events and note events keep their `startTick` and `durationTicks`; the audio engine converts those ticks to seconds using the current `tempoBpm` at scheduling time.

The initial transport UI range is 60 to 180 BPM. Implementations should validate or clamp tempo values before passing them to scheduler or tick/time conversion utilities.

## Core Entities

- `Project`: top-level serializable project document.
Expand Down Expand Up @@ -130,16 +138,29 @@ export interface SampleZone {
midiNote: number;
rootMidiNote: number;
sampleStartSeconds?: number;
sampleEndSeconds?: number;
sustain?: SamplerSustainMeta;
envelope?: SamplerEnvelopeMeta;
}

export interface SamplerSustainMeta {
mode: "none" | "forward-loop" | "crossfade-loop";
loopStartSeconds?: number;
loopEndSeconds?: number;
crossfadeSeconds?: number;
}

export interface SamplerEnvelopeMeta {
attackSeconds?: number;
releaseSeconds?: number;
}
```

The `loopStartSeconds` and `loopEndSeconds` fields are serializable metadata. They describe how the runtime audio engine may configure `AudioBufferSourceNode.loopStart` and `loopEnd` for sustained sample playback.
The `sustain` fields are serializable metadata. They describe how the runtime audio engine may configure sample sustain playback. A simple `forward-loop` mode may map to `AudioBufferSourceNode.loopStart` and `loopEnd`; a future `crossfade-loop` mode may require additional scheduled source nodes and gain ramps.

The `sampleStartSeconds` field skips leading silence before note attack. The bundled Iowa Piano WAV files contain substantial leading silence, so sample zones use explicit start offsets.
The `sampleStartSeconds` field skips leading silence before note attack. `sampleEndSeconds`, sustain loop points, crossfade length, and envelope values are sample-local seconds because they describe positions or durations inside a sample, not musical event time.

Current Iowa Piano sample zones intentionally omit `loopStartSeconds` and `loopEndSeconds`. The samples play once from their configured start offsets and do not loop in the initial implementation. Advanced sampler sustain may add explicit loop metadata later if the loop points are tuned well enough to avoid repeated-strike artifacts.
Current Iowa Piano sample zones intentionally omit `sustain` metadata. The samples play once from their configured start offsets and do not loop in the initial implementation. Advanced sampler sustain may add explicit loop metadata later if the loop points are tuned well enough to avoid repeated-strike artifacts.

## Initial Piano Roll Implementation

Expand Down Expand Up @@ -247,8 +268,22 @@ export interface SampleZone {
sampleId: string;
midiNote: number;
rootMidiNote: number;
sampleStartSeconds?: number;
sampleEndSeconds?: number;
sustain?: SamplerSustainMeta;
envelope?: SamplerEnvelopeMeta;
}

export interface SamplerSustainMeta {
mode: "none" | "forward-loop" | "crossfade-loop";
loopStartSeconds?: number;
loopEndSeconds?: number;
crossfadeSeconds?: number;
}

export interface SamplerEnvelopeMeta {
attackSeconds?: number;
releaseSeconds?: number;
}
```

Expand Down
2 changes: 1 addition & 1 deletion docs/features/08-pitched-instruments-and-sustain.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Feature: 08 Pitched Instruments and Sustain

## Status
In Review
Done

## Goal

Expand Down
130 changes: 130 additions & 0 deletions docs/features/09-sampler-advanced-sustain.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
# Feature: 09 Sampler Advanced Sustain

## Status
Planned

## Goal

Add a reusable sampler sustain path for sample-based pitched instruments.

The first target is `Iowa Piano`, but the implementation should not be hard-coded to piano. Future bundled or imported sample instruments should be able to use the same sampler metadata and playback path.

## Context

`Default Synth` can hold long notes because it uses an oscillator. `Iowa Piano` currently plays bundled WAV samples once from explicit `sampleStartSeconds` offsets so the engine skips leading silence before the attack.

Earlier sustain looping was deferred because simple loop points made long notes sound like repeated strikes. The next sampler step should support longer sample-based notes without coupling React UI to timing and without storing runtime audio objects in project data.

This feature should add a general, metadata-driven sampler sustain mechanism. It is not a full sampler product.

## Scope

Included:
- Define serializable sampler zone metadata for sample start offset, loop region, loop mode, optional crossfade duration, and basic envelope values.
- Implement reusable sampled-note scheduling for pitched sample instruments.
- Use the general sampler path for `Iowa Piano`.
- Keep one-shot sample playback as the fallback when a zone has no valid sustain metadata.
- Support explicit loop metadata only.
- Use gain envelopes for attack and release so note starts and note-offs avoid clicks.
- Stop sustained sample voices cleanly when note duration ends, transport pauses, or transport stops.
- Keep note start and duration in ticks and convert to seconds at scheduling time.
- Keep decoded buffers, source nodes, gain nodes, and active voice state runtime-only.
- Add focused tests for sampler metadata validation, loop-region calculation, fallback behavior, and release timing helpers where practical.
- Update audio engine, data model, and testing docs.

Excluded:
- User sample import UI.
- Visual loop point editor.
- Automatic loop point detection or sample analysis.
- Full sampler preset browser.
- Multisample velocity layers.
- Round-robin sample selection.
- Time-stretching.
- Pitch-shifting beyond playback-rate mapping from `rootMidiNote`.
- Arrangement playback changes.
- New runtime dependencies unless the implementation clearly justifies them.

## Constraints

- React UI must not own exact audio timing.
- Schedule playback against `AudioContext.currentTime`.
- Store musical event time in ticks, not seconds.
- Sample-local metadata such as start offsets, loop points, and envelope lengths may be stored in seconds.
- Keep project data serializable.
- Do not store `AudioBuffer`, `AudioNode`, object URLs, or decoded sample data in project JSON.
- Preserve `Default Synth` behavior.
- If sampler sustain metadata is invalid or sounds unsafe, fall back to one-shot sample playback instead of producing stuck notes.

## Sampler Metadata Approach

Sample zones may evolve toward this shape:

```ts
export interface SampleZone {
sampleId: string;
midiNote: number;
rootMidiNote: number;
sampleStartSeconds?: number;
sampleEndSeconds?: number;
sustain?: SamplerSustainMeta;
envelope?: SamplerEnvelopeMeta;
}

export interface SamplerSustainMeta {
mode: "none" | "forward-loop" | "crossfade-loop";
loopStartSeconds?: number;
loopEndSeconds?: number;
crossfadeSeconds?: number;
}

export interface SamplerEnvelopeMeta {
attackSeconds?: number;
releaseSeconds?: number;
}
```

Use these as planning types. The implementation may refine names, but any semantic changes should update `docs/data-model.md`.

## Playback Approach

- Load and decode the required sample before scheduling the note.
- Start playback at `sampleStartSeconds` when present.
- If `sustain.mode` is `none` or loop metadata is missing, play the sample once with the note envelope.
- If `sustain.mode` is `forward-loop`, use `AudioBufferSourceNode.loop`, `loopStart`, and `loopEnd` after validating the region.
- If `sustain.mode` is `crossfade-loop`, implement a simple scheduled crossfade strategy only if it can be done safely in this feature. Otherwise document the limitation and keep `forward-loop` as the first pass.
- Always apply release behavior at the note end so sustained voices do not stay stuck.
- Transport pause and stop must clear active sampler voices.

## Done when

- Sample-based pitched instruments use shared sampler sustain logic instead of Iowa-Piano-only sustain code.
- `Iowa Piano` can use explicit sampler sustain metadata through the shared path.
- Long Iowa Piano notes sustain more smoothly than one-shot playback when valid metadata is present.
- Long notes do not sound like repeated attacks as much as the available sample material allows.
- Invalid or missing loop metadata falls back to one-shot sample playback.
- Note release and transport stop clear sustained sample voices.
- Serializable metadata is documented and runtime audio objects remain runtime-only.
- Relevant docs and tests are updated.

## Verification

Run:
- `npm run typecheck`
- `npm run lint`
- `npm run test`
- `npm run build`

Manual check:
- Create short and long Iowa Piano notes.
- Confirm short notes still start at the audible attack.
- Confirm long notes sustain without obvious repeated-strike artifacts when sustain metadata is valid.
- Confirm invalid or missing sustain metadata still plays as one-shot sample playback.
- Stop playback and confirm no sustained sample voice remains stuck.
- Confirm `Default Synth` behavior is unchanged.

## PR notes

- Explain the sampler metadata fields added or changed.
- Explain whether the implementation uses forward looping, crossfaded looping, or a documented fallback.
- List any Iowa Piano samples whose loop metadata still sounds weak.
- Mention manual audio checks and any remaining sample-material limitations.
Loading
Loading