Issue Description:
Hello.
I have discovered a performance degradation in the df.ewm function of pandas version 1.2.2 when ewm passed with times and it is so slow compared to ewm pased with int. And I notice that CI/requirements-wheel.txt, shows that it can depend on pandas version 1.2.2. I am not sure whether this performance problem in pandas will affect this repository. I found some discussions on GitHub related to this issue, including #39784 and #40072.
I also found that python/xorbits/_mars/dataframe/window/ewm/core.py used the influenced api. There may be more files used the influenced api.
Suggestion
I would recommend deleting the pandas 1.2.2 version.
Any other workarounds or solutions would be greatly appreciated.
Thank you!