Describe the bug

When running a sort with a low memory limit, DataFusion can run into an internal error. I noticed this with the SHOW ALL; command, which is converted to SELECT name, value FROM information_schema.df_settings ORDER BY name and hence runs a sort.

To Reproduce

cargo run --bin=datafusion-cli -- -m=10m -c="show all"

Will error with

Error: Internal error: Should be called after `spill_append`.
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker

Note, that -m=20m will succeed again.

Expected behavior

I'd instead expect the sort to either work (if necessary with spilling) or to return a non-internal error that's actionable to the user.

Additional context

Tested this on main.

I'd be happy to dig a little bit deeper into this over the weekend :)