Incorrect parsing of ASD files saved on macOS Ableton 10.1.30

[wyhinton]

I'm running the provided example and having a hard time understanding the values of Clip.start_marker/Clip.end_marker. The time values returned seem to be very large, what units are they in? Is there a way to convert this value to seconds?

import abletonparsing
import librosa

def test():
    bpm = 130.
    audio_path = 'test.wav'
    clip_path = audio_path + '.asd'

    audio_data, sr = librosa.load(audio_path, sr=None, mono=False)
    num_samples = audio_data.shape[1]

    clip = abletonparsing.Clip(clip_path, sr, num_samples)
    print(clip.start_marker)
    # 5.742306885616792e+72
test()

[DBraun]

Hi, thanks for trying it and filing the issue. What version of Ableton Live created the file? I've only tested Ableton 9 and 10. Do you observe the same issue with an audio file which you could share along with the asd file? Feel free to email me directly at [braun][at symbol][ccrma.stanford.edu].

The units are beats, like quarter notes, so 4.0 indicates 4 quarter notes. To convert to seconds in the output audio, you can multiply by (seconds per quarter note = 60/ target bpm), but if the target BPM is changing over time then it's more complicated. Then it's like an integral equation.

[wyhinton]

I see, thanks for your response. I've created a repo where you should be able to reproduce my results here: https://github.com/wyhinton/asdparsingissue. Ableton version is 10.1.30. Hmm even with the units being beats, it seems like the values I'm getting back are very large. It does look like the parser is picking up that the end marker is a greater value than the start marker.

Debugging the clip I get the following:

ic| clip.loop_on: False
ic| clip.start_marker: 5.742306885616792e+72
ic| clip.end_marker: 1.9271279672093702e+87
ic| clip.loop_end: 2.5933863806803696e+161
ic| clip.hidden_loop_start: 1.366487708774349e+241
ic| clip.hidden_loop_end: 3.3328279543797095e+178
ic| clip.warp_markers: []
ic| clip.warp_on: False
ic| clip.sr: 44100
ic| clip: <abletonparsing.Clip object at 0x7fad18174a00>

[DBraun]

Thanks for trying this and sharing your files. I'm also using Ableton 10.1.30 on Windows. With your script, I see the same weird output. If I just resave the asd as-is, then it works correctly:

ic| clip.loop_on: False
ic| clip.start_marker: 0.0
ic| clip.end_marker: 183.34639266983018
ic| clip.loop_end: 183.34639266983018
ic| clip.hidden_loop_start: 0.0
ic| clip.hidden_loop_end: 4.0
ic| clip.warp_markers: [WarpMarker(seconds=2.01687074829932,beats=0.0),
                        WarpMarker(seconds=9.589433106575964,beats=12.0),
                        WarpMarker(seconds=12.132517006802722,beats=16.0),
                        WarpMarker(seconds=17.2624716553288,beats=24.0),
                        WarpMarker(seconds=19.893242630385487,beats=28.0),
                        WarpMarker(seconds=27.736485260770976,beats=40.0),
                        WarpMarker(seconds=30.369138321995464,beats=44.0),
                        WarpMarker(seconds=33.047278911564625,beats=48.0),
                        WarpMarker(seconds=38.482063492063496,beats=56.0),
                        WarpMarker(seconds=41.24324263038549,beats=60.0),
                        WarpMarker(seconds=46.8072335600907,beats=68.0),
                        WarpMarker(seconds=49.57424036281179,beats=72.0),
                        WarpMarker(seconds=52.35569160997733,beats=76.0),
                        WarpMarker(seconds=57.718435374149664,beats=84.0),
                        WarpMarker(seconds=63.03895691609978,beats=92.0),
                        WarpMarker(seconds=65.67399092970521,beats=96.0),
                        WarpMarker(seconds=68.33784580498866,beats=100.0),
                        WarpMarker(seconds=71.06770975056689,beats=104.0),
                        WarpMarker(seconds=73.81721088435374,beats=108.0),
                        WarpMarker(seconds=76.55598639455782,beats=112.0),
                        WarpMarker(seconds=82.0044671201814,beats=120.0),
                        WarpMarker(seconds=84.71399092970522,beats=124.0),
                        WarpMarker(seconds=90.0758276643991,beats=132.0),
                        WarpMarker(seconds=92.7704081632653,beats=136.0),
                        WarpMarker(seconds=95.48408163265306,beats=140.0),
                        WarpMarker(seconds=98.17766439909298,beats=144.0),
                        WarpMarker(seconds=100.83952380952381,beats=148.0),
                        WarpMarker(seconds=103.52834467120182,beats=152.0),
                        WarpMarker(seconds=106.25814058956917,beats=156.0),
                        WarpMarker(seconds=109.05580498866213,beats=160.0),
                        WarpMarker(seconds=111.90873015873017,beats=164.0),
                        WarpMarker(seconds=114.81714285714285,beats=168.0),
                        WarpMarker(seconds=117.7562358276644,beats=172.0),
                        WarpMarker(seconds=120.79011337868481,beats=176.0),
                        WarpMarker(seconds=120.81421432114513,beats=176.03125)]
ic| clip.warp_on: True
ic| clip.sr: 44100
ic| clip: <abletonparsing.Clip object at 0x000001CF8F987760>
0.0
0.0

I guess my hex hackery is not as reliable as I had hoped 😢
What stands out is that there are no WarpMarker text segments inside the asd you've shared. Once I re-save it, they appear.

Does this asd have any features that I may have missed? I didn't see automation of parameters or anything else to complicate it.

[wyhinton]

Thanks for testing. I see you have several warp markers. Did you try saving it when warp was turned off for the sample? That's how I was using it.

By resave, do you mean open the .asd in ableton? How are you exporting the .asd files?

See how I have warp turned off:

[DBraun]

If I clone your repo and drag the clip in, it looks like this. Warp seems to be on in the file in the repo, but I think you're saying that on your side it's not on.

If I run the test.py with the original file, then the numbers are junk as you've observed.

If I use the save button (to the right of Edit and to the left of Rev. in the image you just attached), then test.py produces good values. If I turn off warp and save, and then run test.py, it produces the same values, but warp is correctly indicated as off.

I think that there is a difference in the way our systems are creating the ASDs and I haven't learned how to parse this other format.

[wyhinton]

Odd that it looks like that when you drag it in as warp is not enabled on the clip when I load it. I do have automatic warping turned off in my Ableton Preferences though.

After some more tests thought it looks like warp being on/off is not the cause of the problem. Testing a clip with warp enabled, I'm still getting the odd numbers for everything but the warp markers:

ic| clip.loop_on: False
ic| clip.start_marker: 5.742306885616792e+72
ic| clip.end_marker: 1.9271279672093702e+87
ic| clip.loop_end: 2.5933863806803696e+161
ic| clip.hidden_loop_start: 1.366487708774349e+241
ic| clip.hidden_loop_end: 3.3328279543797095e+178
ic| clip.warp_markers: [WarpMarker(seconds=14.0,beats=0.0),
                        WarpMarker(seconds=14.015625,beats=0.03125)]
ic| clip.warp_on: False
ic| clip.sr: 44100
ic| clip: <abletonparsing.Clip object at 0x7f9c4805ca00>

So it seems like parsing the hex data for the WarpMarkers is working regardless of the OS (index = asd_bin.find(b'WarpMarker')) but not for SampleOverview (index = asd_bin.find(b'SampleOverViewLevel')).

Oddly it seems that warp_on is False, but that's impossible because we have a list of warp markers, right?

I see, I'm on macOS Big Sur with an M1 Chip, I wonder if that has anything to do with it. I'm not familiar with reverse engineering file formats, but any thoughts on what could be going on here?

[wyhinton]

I think for my purposes I'm just going to record the start and end positions of the clips manually. But, as a note, I just realized that if a clip does not have warping activated, then the sample viewer uses minutes-seconds-milliseconds instead of bars-beats-sixteenths for the start/end of the clip. So I guess that's something people should be aware of when trying to do conversions.

See the "Playing and Scrubbing Clips" section from the ableton docs:

You can also adjust the clip start and end numerically using the respective value fields. For warped audio clips, these fields display values as bars-beats-sixteenths; for unwarped audio clips, the display is in minutes-seconds-milliseconds.

[DBraun]

Lots of the parameters/fields inside the ASD file work independently of each other. I think that Ableton will toggle the field for warp_on without destructively removing warp markers, for example.

I think that Ableton always has at least two warp marker for a clip and those implicitly define the BPM. It's actually ok if these values are very wrong, but there might always be at least two warp markers. This is why you see beats=0.0 and beats=0.03125. This is the internal data representation. This also allows Ableton to store many of the fields as values in units of beats rather than milliseconds. For example, the milliseconds for start_marker do not seem to be stored in the ASD file because they don't need to be. You can store a sample_offset to the loop_start, and the loop_start's milliseconds can be figured out because of the assumption of always having two warp markers and an implicit BPM.

I'll keep this macOS issue on my radar. Thanks again for pointing it out.

[riban-bw]

I am running Ableton Live Lite 11 on Windows 10 and am seeing similarly inappropriate values. This of course is likely to be due to a different version of Live but one might expect a degree of compatibility between versions. I used one of the built in samples.

>>> clip.loop_end
9.346091110231955e-307
>>> clip.loop_start
8.011012841951456e-307
>>> clip.start_marker
2.091712705903652e-306