Skip to content

Remove Meta Llama's mixing pattern of cos/sin values in tt_transformers #37605

@adrian-pascual-bernal

Description

@adrian-pascual-bernal

Is your feature request related to a problem? Please describe

Meta Llama style cos/sin interleaving pattern is planned to be removed from tt_transformers once we have investigated that the HF style mixing pattern performance and accuracy meet the requirements.
HF mixing pattern for cos/sin support, using ttnn.experimental.rotary_embedding applying the RoPE, is introduced in #32454.

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions