You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**CapRL 3.0 series (CapRL++)**: **CapRL-Video-4B** has been released!
52
+
**CapRL 3.0 series (CapRL++)**: **CapRL-Video-4B** has been released! CapRL++ extends the original image-caption RL framework to a unified image and video captioning paradigm with verifiable rewards.
32
53
33
54
We are excited to release the **CapRL 2.0 series**: **CapRL-Qwen3VL-2B** and **CapRL-Qwen3VL-4B**. These models feature fewer parameters while delivering even more powerful captioning performance.
34
55
Notably, **CapRL-Qwen3VL-2B outperforms both CapRL-Qwen2.5VL-3B and Qwen2.5VL-72B in captioning tasks, while CapRL-Qwen3VL-4B further demonstrates a significant performance leap over the 2B version.**
@@ -41,16 +62,17 @@ This guide will help you choose the most suitable model for your specific needs:
|🤗[CapRL-Video-4B](https://huggingface.co/internlm/CapRL-Video-4B)|4B|Dense Video Captioning, Temporal Grounding|
44
66
45
67
Now you can try out CapRL with your own images🎨! ➡️ [🌈CapRL-Qwen2.5VL-3B Space](https://huggingface.co/spaces/yuhangzang/caprl) and [🌈CapRL-Qwen3VL-4B Space](https://huggingface.co/spaces/yuhangzang/CapRL-Qwen3VL-4B).
46
68
47
69
48
70
## 📢 News
49
71
We are working on even stronger base models and upgrading our training recipe — stay tuned!
50
-
- 🔥 [05/25/2026] We have released the training and evaluation code for CapRL++. See mroe in `CapRL++` folder.
72
+
- 🔥 [05/25/2026] We have released the training and evaluation code for CapRL++. See more in `CapRL++` folder.
51
73
- 🔥 [05/22/2026] We have released the **[CapRL-Video-QA-20K](https://huggingface.co/datasets/internlm/CapRL-Video-QA-20K)** dataset for CapRL++ training and
52
74
the **[CapRL-Video-178K](https://huggingface.co/datasets/internlm/CapRL-Video-178K)** dataset (recaptioned by **[CapRL-Video-4B](https://huggingface.co/internlm/CapRL-Video-4B)** from LLaVA-Video-178K)!
53
-
- 🔥 [05/22/2026]**CapRL++** is coming! We have released the **[CapRL-Video-4B](https://huggingface.co/internlm/CapRL-Video-4B)** model designed for video captioning! Demo is [here](https://internlm.github.io/CapRL/demo/)
75
+
- 🔥 [05/22/2026]**CapRL++** is coming! We have released the **[CapRL-Video-4B](https://huggingface.co/internlm/CapRL-Video-4B)** model (trained on Qwen3-VL-4B) designed for video captioning! Demo is [here](https://internlm.github.io/CapRL/demo/).
54
76
- 🔥 [04/16/2026] We have released the **[CapRL-QA-75K](https://huggingface.co/datasets/internlm/CapRL-QA-75K)** training dataset!
55
77
- 🔥 [2/9/2026] We release the CapRL training code.
56
78
- 🔥 [1/27/2026] CapRL is accepted by ICLR2026! We are working on cleaning training code, and will release everything as soon as possible!
@@ -64,8 +86,11 @@ the **[CapRL-Video-178K](https://huggingface.co/datasets/internlm/CapRL-Video-17
64
86
65
87
66
88
## Introduction
67
-
🌈We are excited to introduce <strong>CapRL-3B</strong>, a lightweight 3B image captioner that achieves perception capabilities comparable to Qwen2.5-VL-72B.
68
-
By employing CapRL training framework, initializing with the Qwen2.5-VL-3B model, and using a carefully filtered 75K QA dataset as the training set, we obtained a highly capable captioner, CapRL-3B.
89
+
🌈 We are excited to introduce the **CapRL series**, a family of dense captioning models trained with reinforcement learning rather than conventional supervised caption imitation.
90
+
91
+
The original **CapRL** framework focuses on dense image captioning. It optimizes an LVLM captioner with QA-derived rewards: a caption is considered high quality when a text-only model can answer visual questions using only that caption. With this recipe, the lightweight **CapRL-3B** achieves perception capabilities comparable to Qwen2.5-VL-72B.
92
+
93
+
**CapRL++** further generalizes this idea from static images to dynamic videos. It trains a Qwen3-VL-based captioner with a unified RLVR pipeline, where generated captions are evaluated by their downstream utility for multiple-choice visual question answering. For videos, CapRL++ adds timestamp-format rewards and length-aware regularization so the model learns dense, temporally grounded, and non-redundant descriptions.
69
94
70
95
71
96
@@ -82,23 +107,88 @@ By employing CapRL training framework, initializing with the Qwen2.5-VL-3B model
82
107
83
108
84
109
## 💡 Highlights
85
-
- 🔥 **Remarkable visual understanding for Chart, Infographics and Document**: CapRL-3B achieves perception accuracy and visual information coverage comparable to Qwen2.5-VL-72B.
86
-
- 🔥 **Well-organized output**: The outputs of CapRL-3B are relatively well-structured, making them clear and easy to understand.
87
-
- 🔥 **Detailed description for natural images**: The outputs of CapRL-3B can perfectly cover all valid visual information while containing fewer hallucinations.
110
+
- 🔥 **Unified dense caption RL for images and videos**: CapRL++ applies the same QA-utility reward philosophy to both image and video captioning, avoiding dependence on a single reference caption.
111
+
- 🔥 **Verifiable reward design**: CapRL++ combines visual utility reward, timestamp-format reward, and length-aware penalty to optimize accuracy, temporal structure, and information efficiency.
112
+
- 🔥 **Strong temporal grounding**: CapRL-Video-4B generates explicit timestamped video descriptions and improves downstream video understanding when used as caption data.
113
+
- 🔥 **Remarkable visual understanding for charts, infographics, and documents**: CapRL-3B achieves perception accuracy and visual information coverage comparable to Qwen2.5-VL-72B.
114
+
- 🔥 **Well-organized dense output**: CapRL models generate structured captions that cover fine-grained objects, attributes, OCR content, relations, and events.
88
115
89
116
## Model Card
90
117
- Based on the same recipe as CapRL-3B, we used InternVL3.5-8B as the policy model and obtained CapRL-InternVL3.5-8B through CapRL.
91
118
- CapRL-3B-GGUF is static quants version, and CapRL-3B-i1-GGUF is weighted/imatrix quants version. Thanks for their contribution!
119
+
- CapRL-Video-4B is trained from Qwen3-VL-4B with CapRL++ for dense video captioning. It is designed to describe both spatial details and temporal event changes with timestamped structure.
92
120
93
121
94
122
## 👨💻 Todo
95
123
96
124
- ✅ Release 75k QA dataset.
97
125
126
+
## CapRL++: Unified Image and Video Caption RL
127
+
128
+
CapRL++ is the video-oriented extension of CapRL. It keeps the central principle of CapRL: **a caption should be rewarded by how useful it is for downstream visual question answering**. Instead of comparing a generated caption with a fixed reference, CapRL++ lets the policy model generate captions, then asks a separate vision-free LLM to answer curated multiple-choice questions using only those captions. The answer accuracy becomes a verifiable reward for RL training.
129
+
130
+
### Reward Design
131
+
132
+
For a sampled caption `c`, CapRL++ uses a multidimensional reward:
-**Visual utility reward (`R_acc`)**: measures whether a text-only LLM can answer image/video MCQs from the generated caption alone. Options are shuffled and sampled multiple times to reduce answer-position bias.
139
+
-**Temporal format reward (`R_format`)**: used for video captions. It encourages valid timestamp brackets and chronological ordering, helping the model produce temporally grounded narratives.
140
+
-**Length-aware reward (`R_len`)**: discourages reward hacking through overly long or repetitive captions, pushing the model toward high information density.
141
+
142
+
### Static-to-Dynamic Bootstrapping
143
+
144
+
CapRL++ uses **S2D-Boot**, a two-stage image-to-video training recipe:
145
+
146
+
1.**Image stage**: train on static images with visual utility and length rewards to strengthen fine-grained spatial perception, OCR, attributes, and relation extraction.
147
+
2.**Video stage**: initialize from the image-stage checkpoint and train on video data with the full reward space, including timestamp-format reward, so optimization can focus on event ordering and temporal localization.
148
+
149
+
This progressive strategy preserves strong image captioning ability while improving video understanding.
150
+
151
+
### CapRL++ Datasets
152
+
153
+
-**[CapRL-Video-QA-20K](https://huggingface.co/datasets/internlm/CapRL-Video-QA-20K)**: multiple-choice video QA data for CapRL++ reward training.
154
+
-**[CapRL-Video-178K](https://huggingface.co/datasets/internlm/CapRL-Video-178K)**: LLaVA-Video-178K videos recaptioned by **[CapRL-Video-4B](https://huggingface.co/internlm/CapRL-Video-4B)** with dense, timestamped descriptions.
155
+
-**CapRL-Image-5M**: the CapRL++ paper also studies large-scale image recaptioning by combining ShareGPT4V-1M, DenseFusion-1M, and filtered web images, then annotating them with a CapRL++ image captioner.
156
+
157
+
### Key Results from CapRL++
158
+
159
+
- In image Prism evaluation, CapRL++ trained from Qwen2.5-VL-3B reaches an average score of **48.3**, matching Qwen2.5-VL-72B while substantially outperforming LVLM-as-a-judge reward variants.
160
+
- On HAT image captioning evaluation, CapRL++ improves coverage and reaches an average score comparable to Qwen2.5-VL-72B.
161
+
- In video Prism evaluation, CapRL++ trained from Qwen3-VL-4B achieves the best average score among compared general and specialized video models, surpassing Qwen3-VL-32B and Qwen3-VL-235B-A22B in the reported setting.
162
+
- As video pretraining data, CapRL-Video-178K improves Molmo2 downstream video understanding more than GPT-4o, ShareGPT4Video-8B, and Tarsier2-7B caption sources in the paper's continual-pretraining experiments.
163
+
- S2D-Boot gives the best balance between image and video performance, retaining strong spatial captioning while improving temporal reasoning.
164
+
165
+
### Code Entry Points
166
+
167
+
The CapRL++ implementation is in [`CapRL++`](CapRL++):
168
+
169
+
```text
170
+
CapRL++/
171
+
├── train/
172
+
│ ├── scripts/ # reward service and verl training launch scripts
173
+
│ └── verl/ # bundled verl backend with video caption RL recipe
2. Install the OpenRLHF-based training framework and remaining dependencies via `pip install -e .`
113
203
204
+
For CapRL++ video training and evaluation:
205
+
206
+
```bash
207
+
cd CapRL/CapRL++/train
208
+
conda create -n caprl python=3.10 -y
209
+
conda activate caprl
210
+
pip install -r scripts/requirements.txt
211
+
pip install -e ./verl
212
+
```
213
+
214
+
Video Prism evaluation dependencies are installed separately:
215
+
216
+
```bash
217
+
cd CapRL/CapRL++/eval
218
+
pip install -r requirements.txt
219
+
```
220
+
114
221
## ⭐️ Quick Start
115
222
If you want to use **CapRL-3B** for captioning, you can directly follow the exact same inference approach as in [Qwen2.5-VL-series](https://github.com/QwenLM/Qwen3-VL/tree/d2240f11656bfe404b9ba56db4e51cd09f522ff1).
116
223
117
224
The prompt we use for training and evaluation is `Please describe this image in detail.`
118
225
119
226
We recommend using **vLLM** to speed up inference.
120
227
228
+
For **CapRL-Video-4B**, use the Qwen3-VL video inference interface or the Prism evaluation scripts under `CapRL++/eval`. A typical video caption prompt is:
229
+
230
+
```text
231
+
Please describe this video in detail with timestamps.
0 commit comments