chore: upgrade to delta-kernel-rs 0.16.0 and remove more dependencies #10224

Job	Run time
test-minimal	2m 8s
Python Build (Python 3.10 LakeFS Integration tests)	6m 9s
Python Build (Python 3.10 PyArrow latest)	3m 19s
Python Build (Python 3.10 Unity Catalog Integration tests)	5m 55s
Python Build (Python 3.10 Unity Catalog Integration tests)	6m 8s
PySpark Integration Tests	6m 40s
Running with Python 3.9	5m 47s
Running with Python 3.12	5m 49s
Running with Python 3.10	5m 36s
Running with Python 3.11	5m 48s
	53m 19s

Provide feedback