Add contributor skills at .agents/skills

.agents/skills/update-paper-index/SKILL.md

@@ -0,0 +1,36 @@
+---
+name: update-paper-index
+description: Add or review a paper entry in TRL's paper index. Use when a PR implements a method, algorithm, or training approach from a research paper, or when reviewing such a PR.
+---
+
+# Update the paper index
+
+Any PR that implements a method, algorithm, or training approach from a research paper must add a corresponding subsection to `docs/source/paper_index.md`. When reviewing such a PR, check that the file was updated.
+
+## Entry format
+
+The file is organized as one `##` section per method family (usually one per trainer), each holding `###` subsections, one per paper.
+
+Each entry contains:
+
+1. The paper title as the `###` heading.
+2. A paper link line, using the Hugging Face paper page (same ID as arXiv), never an arxiv.org link:
+   ```
+   **📜 Paper**: https://huggingface.co/papers/<id>
+   ```
+3. A few sentences on what the paper introduces and how it maps to TRL.
+4. A Python snippet showing the TRL config that reproduces the paper's setting, with the paper's hyperparameters quoted in comments:
+   ```python
+   from trl import GRPOConfig, GRPOTrainer
+
+   training_args = GRPOConfig(
+       beta=0.001,  # "the KL coefficient to 0.001"
+       num_generations=16,  # "For each question, we sample 16 outputs..."
+   )
+   ```
+   When the paper doesn't specify hyperparameters, say so in a comment rather than inventing values.
+
+## Placement
+
+- If the paper belongs to an existing method family, add it under that `##` section.
+- If it introduces a new trainer or family, add a new `##` section with a one-line pointer to the trainer, e.g. `Papers relating to the [`GRPOTrainer`].`

.ai/AGENTS.md

@@ -34,6 +34,21 @@ This is intentional: each trainer must be readable, modifiable, and evolvable in
 
 **When modifying duplicated code**: if you change a pattern that exists in multiple trainers (e.g., the vLLM generation path in `_generate_single_turn`), apply the same change to all other trainers. A fix in GRPO often implies the same fix in RLOO, and vice versa. Not propagating a change is a bug.
 
+Find every copy by grepping a distinctive line of the block:
+
+```sh
+grep -rn "self._last_loaded_step" trl/trainer/ trl/experimental/
+```
+
+After propagating, diff the corresponding regions to confirm they stayed aligned:
+
+```sh
+diff <(sed -n '/def _generate_single_turn/,/def /p' trl/trainer/grpo_trainer.py) \
+     <(sed -n '/def _generate_single_turn/,/def /p' trl/trainer/rloo_trainer.py)
+```
+
+Remaining diffs must all be semantic divergences, not drift.
+
 **When reviewing**: if a PR touches duplicated logic, verify that all copies are updated consistently. A common mistake is fixing one trainer and forgetting the others.
 
 ### Simplicity

.gitignore

@@ -147,5 +147,5 @@ wandb/
 uv.lock
 
 # AI agent generated artifacts
-/.agents/skills
-/.claude/skills
+/.claude/skills
+/.agents/skills/local-*/

CONTRIBUTING.md

@@ -421,10 +421,12 @@ By following this classification, you ensure that warnings, information, and exc
 
 ## Coding with AI agents
 
-This repository keeps AI-agent configuration in `.ai/` and exposes local agent files via symlinks.
+This repository keeps AI-agent configuration in `.ai/` and skills in `.agents/skills/`.
 
-Skills can be exposed to agents by running `make codex` or `make claude`
+`AGENTS.md`, `CLAUDE.md`, and `.cursor/BUGBOT.md` all point to `.ai/AGENTS.md`. Cursor reads `AGENTS.md` and Bugbot reads `.cursor/BUGBOT.md`.
 
-`AGENTS.md`, `CLAUDE.md`, and `.cursor/BUGBOT.md` all point to `.ai/AGENTS.md`.
+Codex, Cursor and Gemini CLI read `.agents/skills/` directly. Claude Code reads `.claude/skills/`, so run `make claude` to symlink it to `.agents/skills/` (and `make clean-ai` to remove it).
 
-Cursor reads `AGENTS.md` and Bugbot reads `.cursor/BUGBOT.md`. Cursor reads skills from Claude or Codex paths, so setting up the repository for Claude or Codex will work for Cursor.
+`.agents/skills/` holds contributor skills maintained by the repository. Personal skills belong in `~/.agents/skills`, which Codex, Cursor, Gemini CLI and Copilot all read as user scope; for a personal skill that has to be repo-scoped, `.agents/skills/local-*/` is git-ignored.
+
+A skill that describes a workflow is updated in the PR that changes that workflow, the same rule that applies to `paper_index.md`.

Makefile

@@ -1,4 +1,4 @@
-.PHONY: test precommit common_tests slow_tests tests_gpu test_experimental codex claude clean-ai
+.PHONY: test precommit common_tests slow_tests tests_gpu test_experimental claude clean-ai
 
 check_dirs := examples tests trl
 
@@ -22,15 +22,13 @@ slow_tests:
 test_experimental:
 	pytest -n auto -s -v tests/experimental
 
-codex:
-	mkdir -p .agents
-	rm -rf .agents/skills
-	ln -snf ../.ai/skills .agents/skills
-
 claude:
 	mkdir -p .claude
 	rm -rf .claude/skills
-	ln -snf ../.ai/skills .claude/skills
+	ln -snf ../.agents/skills .claude/skills
 
+# The `.agents/skills` line removes a leftover symlink from the old `make codex` setup; the tracked
+# directory is left alone.
 clean-ai:
-	rm -rf .agents/skills .claude/skills
+	[ -L .agents/skills ] && rm .agents/skills || true
+	rm -rf .claude/skills