We still don't use BFDOT on macOS CPU because Apple compiler used for PyTorch wheel is outdated

[swolchok]

🚀 The feature, motivation and pitch

See pytorch/pytorch#143913 . This is blocking improved CPU performance for bfloat16 decoding on Mac; setting up an issue on torchchat side to track.

Alternatives

robust setup for decoding using accelerators on Mac

Additional context

No response

RFC (Optional)

No response

[Jack-Khuu]

Pulling up context from the chain:

Per @huydhn

We don't plan to take any action for this right now and will wait for Apple to release a new clang/xcode version on MacOS