toml package fails to parse valid toml - breaks prefect_test_harness

[hredestig]

Prefect depends on toml which claims to pass the toml test suite. But, it fails to parse this example

potpourri = ["flower", 1749, { symbol = "X", color = "blue" }, 1994-02-14]

cat blub.toml | python -c "import sys; import toml; toml.load(sys.stdin)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/henning/proj/benchling/.venv/sandbox/lib/python3.11/site-packages/toml/decoder.py", line 156, in load
    return loads(f.read(), _dict, decoder)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/henning/proj/benchling/.venv/sandbox/lib/python3.11/site-packages/toml/decoder.py", line 511, in loads
    ret = decoder.load_line(line, currentlevel, multikey,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/henning/proj/benchling/.venv/sandbox/lib/python3.11/site-packages/toml/decoder.py", line 778, in load_line
    value, vtype = self.load_value(pair[1], strictly_valid)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/henning/proj/benchling/.venv/sandbox/lib/python3.11/site-packages/toml/decoder.py", line 880, in load_value
    return (self.load_array(v), "array")
            ^^^^^^^^^^^^^^^^^^
  File "/Users/henning/proj/benchling/.venv/sandbox/lib/python3.11/site-packages/toml/decoder.py", line 1002, in load_array
    a[b] = a[b] + ',' + a[b + 1]
                        ~^^^^^^^
IndexError: list index out of range

Python's native tomllib parses it without issues.

cat blub.toml | python -c "import sys; import tomllib; print(tomllib.load(sys.stdin.buffer))"
{'potpourri': ['flower', 1749, {'symbol': 'X', 'color': 'blue'}, datetime.date(1994, 2, 14)]}

Currently, I can't use the prefect_test_harness since it fails when trying to read my pyproject.toml where I store my tox config that make use of syntax like that.

Suggest to drop the toml package in favour of tomllib.

[zzstoatzz]

hi @hredestig - tomllib was introduced in python 3.11, so as long as we support 3.9 and 3.10, I think we can't use it (broadly) yet. similar to pendulum (used below 3.13) we could conditionally use it when the user's python installation is 3.11+ though

[hredestig]

Ah okay - then my suggestion is not good - but problem remains. Edited the title to reflect.