Bug: Transformer generates an extra B3 note at the end of measure 3

[JoeZhang42]

Bug: Transformer generates an extra B3 note at the end of the 3rd measure of this page that doesn't exist in the source image

Environment

• homr version: 0.7.0
• Command: homr --coreml-encoder test_score02.jpg
• Hardware: Apple M3, 16GB
• Output format: MusicXML

Description

When processing a standard B♭ major piano score (digital score screenshot), the MusicXML output contains an extra B3 eighth note in the right hand (staff 1, voice 1) at the end of the 3rd measure of this page. This note does not exist in the original image.

Location

• Score: test_score02 (B♭ major piano score)
• Measure: 3
• Part: Right hand (staff 1, voice 1)
• Position: End of measure
• Wrong note: B3 (step=B, octave=3), eighth note, duration=2

Context (correct musical passage)

The correct right hand sequence in measure 3 should be:

C5 (16th) → G4 (16th) → F4 (16th) → B♭4 (quarter) → (no note here) → measure 4 begins

But homr outputs an extra B3 eighth note after the B♭4 quarter, then proceeds to measure 4.

Reproducibility

100% reproducible. Running homr twice on the same image produces the exact same wrong note at the exact same position. This indicates it is a systematic misrecognition, not a random hallucination.

Root Cause Hypothesis

There are no musical notes at that position in the original image. However, other printed elements (fingering markers, pedal markings, dynamic signs, or layout artifacts) may have been misrecognized by the Transformer as a B3 eighth note.

Expected Behavior

The model should not generate notes at positions where no notes exist in the source image.

Attachment

Source image file: test_score02.jpg

(The image is a screenshot of an digital piano score in B♭ major, containing melody, chord accompaniment, pedal markings, etc.)

[liebharc]

Hi thanks for the report! To set overall expectations: OMR systems aren't perfect, state-of-the-art error rates reported in literature are usually still >5% for real world scenarios, which means that 1 out of 20 symbols has an error. That's also what you can reasonably expect from homr.

Still you have a really interesting case here in measure 3. What seems to confuse homr are the numbers. I still haven't understood where this sensitivity to numbers comes from, but we have also seen it in #97. Later homr ignores the octave shift, which is expected as it doesn't support that, however it's interesting that here the numbers cause no confusion.

PS: The tempo marking in the final bar is placed at the worst spot for any OMR system - I hope whoever notated this feels a bit guilty ;)

[JoeZhang42]

Hi, thanks for the detailed explanation!

You're right about the numbers. In the first three measures there are fingering markings, and on the last B♭4 (quarter) in measure 3 there are even two optional fingerings (4 and 5), followed by a breath mark right after. So I can see why that might confuse the model.

This score is designed for arranger keyboard beginners. Besides fingerings, it also includes chord symbols for left-hand control and section markers for accompaniment styles. It is a pretty busy layout compared to standard piano scores. So, feel free to keep this open for reference, or close it if you think it's covered by the known limitations.

All that said, I totally understand that homr focuses on core musical content, and I have to say it's by far the best open-source OMR system I've tried. Really impressive work, hats off to all the contributors!

[liebharc]

It looks like a relevant case which I want to understand better. Don't expect quick progress, but I will certainly leave this open until we have a better understanding. Thanks again for bringing this up!

[weixlu]

Hi, I have reproduced the issue you mentioned, a redundant note at the end of a measure.

I have a different idea. I once checked the dataset (.tokens) and found similar cases where there were extra notes in a measure. Claude said that out of the total 200k files (for training), about 10k have this issue. But I haven’t manually verified them yet.

Later, if I have time, I’ll explore this direction — for example, filtering out those files and retraining to see whether it resolves the problem.

[liebharc]

@weixlu , that sound promising. One other thought I had was that perhaps cutting of notes during data augmentation makes the model start hallucinating notes. But you lead sounds more promising.

[weixlu]

Oh yes, that makes sense. Data augmentation should also be done carefully

[liebharc]

While checking our datasets I came around to make an interesting experiment:

1. A model only trained on the Lieder dataset still has the extra B3 note. That was surprising, I just would have assumed that the B3 note is a more or less random artifact (or chaotic: reproducible but any change to the model will make it disappear for this particular image), instead this indicates that the issue is learned during the training of Lieder. It's consistently wrong even in multiple training runs.
2. A model trained on Lieder+pdmx+musetrainer with the fixes of MuseScore allows to change the y-position of a note #115 makes the B3 note disappear (see image)
3. Going back to the model on main and looking at the center of attention of the B3 note: It's really off, the model focus is outside the staff

A possible explanation for this could be the model mainly predicts the B3 note on a learned sequence pattern and less on visual data. Or in other words: During training it established a pattern that the B3 must be there given the previous sequence of tokens, even if there is little visual basis for that. So possibly the issue resolves with more training data (meaning adding pdmx+musetrainer). However the experiment allows alternative explanations, e.g. perhaps the changes of #115 cause the difference.

[weixlu]

Interesting finding! Let me try the current dataset (no pdmx nor musetrainer) with #115, and see if it also solves this issue

[weixlu]

Good news! The B3 note is gone. (trained 10 epochs)

[liebharc]

Should be fixed with #107. Thanks again for spotting this error, it turned out to be really interesting.