chore: upgrade to delta-kernel-rs 0.16.0 and remove more dependencies #10231

Job	Run time
PySpark Integration Tests	6m 49s
Python Build (Python 3.10 Unity Catalog Integration tests)	6m 29s
test-minimal	42s
Python Build (Python 3.10 LakeFS Integration tests)	6m 34s
Python Build (Python 3.10 Unity Catalog Integration tests)	6m 17s
Running with Python 3.11	6m 1s
Running with Python 3.10	5m 42s
Running with Python 3.9	5m 51s
Running with Python 3.12	5m 42s
Python Build (Python 3.10 PyArrow latest)	4m 19s
	54m 26s

Provide feedback