Plan to remove Spark 2.X in the next release

[wudanzy]

We are planning to drop the support for Spark 2.4 in the next release.

The last Spark 2.4 release (2.4.8) is back in May 2021.

Please leave any comments if you have any concerns with dropping the support.

[grazy27]

Wikipedia says the last supported version for Spark is 3.3, shall we also drop all previous versions including 3.2?
This would simplify codebase significantly

https://en.wikipedia.org/wiki/Apache_Spark#history

[wudanzy]

I think that can be our next step, we can have more frequent release in the future.