Skip to content

Commit 8f09858

Browse files
Merge remote-tracking branch 'origin/main' into HEAD
2 parents 29aed0b + 5dd51e4 commit 8f09858

3 files changed

Lines changed: 143 additions & 290 deletions

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222

2323
## 🎉 What's New
2424

25-
**⚗️ DistillationTrainer is now stable:** [`DistillationTrainer`](https://huggingface.co/docs/trl/distillation_trainer) graduates to the stable API — on-policy knowledge distillation that matches a teacher's full next-token distribution with a memory-efficient chunked JSD loss and vLLM-powered generation.
25+
**📜 Training beyond 1M tokens:** A new [long context guide](https://huggingface.co/docs/trl/long_context_training) walks through the four things that break as sequences grow — the loss, the positions, the activations and the memory of a single GPU — and ends on an example that trains Qwen3-8B on million-token sequences on one 8-GPU node.
2626

2727
## Overview
2828

docs/source/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ The library is integrated with 🤗 [transformers](https://github.com/huggingfac
1212

1313
## 🎉 What's New
1414

15-
**⚗️ DistillationTrainer is now stable:** [`DistillationTrainer`](distillation_trainer) graduates to the stable API — on-policy knowledge distillation that matches a teacher's full next-token distribution with a memory-efficient chunked JSD loss and vLLM-powered generation.
15+
**📜 Training beyond 1M tokens:** A new [long context guide](long_context_training) walks through the four things that break as sequences grow — the loss, the positions, the activations and the memory of a single GPU — and ends on an example that trains Qwen3-8B on million-token sequences on one 8-GPU node.
1616

1717
## Taxonomy
1818

0 commit comments

Comments
 (0)