Add automated feedback tips & outlier/comparison detection

[wustep]

Examples:

• Highlight sections of high discrepancy between events in each graph, e.g. velocity mismatch (probably not tho since musical issues here), note frequency mismatch, master note graph timing mismatch

[jrg94]

You mentioned it already here, but I figure it would be helpful if notes on the master graph had reduced opacity if they're correct. There are probably other ways to highlight errors as well. Maybe add a border to the note group.

[wustep]

This is probably the toughest issue left, since:

• Music is subjective, different plays will have differences that aren't necessarily wrong
  • e.g. if you're playing a piece quieter, but the whole piece is quieter, this is fine.
  • BUT if you play a forte section more piano, then this is bad
• A lot of mistakes/differences observed from our graphs aren't necessarily bad
  • if you hold a note for longer than you should, this is usually OK if the difference is slight (0.5s) and unnoticeable
• If you're a solo player and you go off-time but correct yourself, everything after this point is offset.

And lots of other issues. So exploring the idea of "objective mistakes" will be interesting.. I think it's worth having some discussions as to what qualifies here, what kind of feedback to give, etc. before diving into code.

Objectively:

• if you played more/less notes than expected, this is bad
  • note that if you played the extra notes at a very small (<20) velocity, this is unnoticeable
• if your volume goes from LOUD to SOFT sections when it should go SOFT to LOUD, this is bad
  • but this is difficult to mathematically examine. maybe each MIDI file has a "baseline" volume and there's sections classified as LOUD or SOFT based on that, but this is tough

goal: find more low hanging fruit :)

[jrg94]

For practical purposes, this one is really hard. But, I think there is some low hanging fruit here. For instance, fading pairs of bars that are equivalent in the note frequency graph. This makes bars that are not the same stand out more (meaning you play some note more/less than expected).

Just saw you catch this one above.

[jrg94]

You might be able to set a threshold for the velocity problem. If the curve differs by some delta then we flag that section as an error. Of course, this has some limits. The delta method wouldn't be able to handle time offsets. For instance, if a section goes from soft to loud rapidly, there's very little room for error. If the player gets loud more gradually, we'd be calling them out for not being loud enough when instead they just took too long to get loud.

Just thinking out loud here...

[wustep]

Yeah I think delta is good here, at least within multiple esections. If the average velocity for notes in a 3-measure section is 30, vs. another playthrough being 180, this is bad. But picking the parameters (offset, number of sections) carefully is key. Another thing is that we won't have a lot of reference, human data. I'll try to take some recordings later

A lot more work to be done on the velocity graph too tho which will affect this #26

[jrg94]

True, true. Let me take a swing at fading bars in the note histogram first. Then, I might try to rework the velocity graph. That one is tough. We might need to do some downsampling or averaging of the data to improve that graph a bit.