[Questions] Context length extension.

[iPRET]

Hi!

I have a couple questions about increasing context lengths during training.

1. Does the framework support increasing the context length during training when using rotary position embeddings?
2. If it does, does it support position embedding interpolation? Or just extrapolation?

Thanks!
Ingus