my_research_feed.xml

<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0">
  <channel>
    <title>Small RL Paper Radar</title>
    <link>https://github.com/careyourcake/my-research-feed</link>
    <description>Daily reinforcement learning and robotics papers. Hugging Face: ok (11 items); Papers with Code: unavailable (JSONDecodeError); arXiv: ok (48 items)</description>
    <language>en</language>
    <lastBuildDate>Thu, 20 Aug 2026 00:57:00 +0000</lastBuildDate>
    <ttl>1440</ttl>
    <item>
      <title>Hydra-0: Action Flow for Generalist World Modeling and Control</title>
      <link>http://arxiv.org/abs/2608.18077v1</link>
      <guid isPermaLink="false">2608.18077</guid>
      <pubDate>Tue, 18 Aug 2026 17:59:30 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Hongyu Li, Bowen Wen, Xinghao Zhu, Yixuan Wang, Yilun Du, Yunzhu Li, George Konidaris, Stan Birchfield, Soha Pouya, Chenran Li, Yan Chang</p><p>We introduce Hydra-0, a generalist world model conditioned on action flow, which represents robot actions as pixel motion. This shared visual interface enables generalist world modeling and control by learning action consequences across embodiments, tasks, environments, and video-generation backbones. Our best configuration achieves 90.4% lower robot-motion error and 60.2% lower object-motion error than our action-conditioned baseline, while supporting zero-shot composition and data-efficient adaptation. On the RoboLab benchmark, Hydra-0 achieves a Pearson correlation of r=0.96 between replayed and reference success rates. Finally, we uncover an emergent inverse mode of this interface: a world action model that predicts compatible robot motion from desired object flow transferred from a human demonstration. A trained action head maps the resulting latent features to executable actions without requiring task-specific expert robot demonstrations. Together, these results demonstrate the potential of action flow as a shared control interface connecting heterogeneous training data, open-loop policy evaluation, and robot control.</p>]]></description>
    </item>
    <item>
      <title>Policy-Invariant Reward Shaping from LLM Feedback: A Framework for Hybrid RL Agents</title>
      <link>http://arxiv.org/abs/2608.18008v1</link>
      <guid isPermaLink="false">2608.18008</guid>
      <pubDate>Tue, 18 Aug 2026 16:55:46 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Christophe D. Hounwanou, John Emeka Eze, Yaé U. Gaba</p><p>Combining large language models with reinforcement learning is increasingly explored, yet the theoretical status of LLM-derived reward signals is often left implicit. We formalize the hybrid LLM-planner and RL-controller architecture as a Goal-Augmented Markov Decision Process and show that when the LLM per-state progress score is used as a bounded potential function, the resulting shaping term preserves the optimal policy set even when the LLM scores are inaccurate. This guarantee is stronger than what general LLM-as-reward approaches provide. We verify the result numerically on a small MDP under four potential configurations, including an adversarial one scaled to twenty times the base reward magnitude.</p>]]></description>
    </item>
    <item>
      <title>Towards Zero-Shot Task Transfer with Neurosymbolic World Models</title>
      <link>http://arxiv.org/abs/2608.17959v1</link>
      <guid isPermaLink="false">2608.17959</guid>
      <pubDate>Tue, 18 Aug 2026 16:12:40 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Isidoro Tamassia, Lennert De Smet, Giuseppe Marra</p><p>State-of-the-art model-based reinforcement learning methods learn neural world models that allow policy improvement by planning in a latent space, without assumptions on the structure of the underlying environment. While expressive, these models are generally task-dependent: they learn uninterpretable latent representations that are tied to the training task and thus hard to generalize to new tasks. In this work, we present a novel world model formulation where the reward prediction only depends on a subset of structured, symbolic components of the whole latent state. Decoupling observation reconstruction and reward prediction allows us to learn world models that can adapt zero-shot, i.e. without further environment interactions, to new reward functions defined over the same symbolic state space. We discuss the main advantages and challenges of learning these neurosymbolic world models and demonstrate the strong generalisation properties of our approach over purely neural methods.</p>]]></description>
    </item>
    <item>
      <title>An Omitted Mode Is a Rare Rule: The Sampling-Verification Danger Law in Continuous Code World Models</title>
      <link>http://arxiv.org/abs/2608.17956v1</link>
      <guid isPermaLink="false">2608.17956</guid>
      <pubDate>Tue, 18 Aug 2026 16:09:51 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Javier Aguilar Martín</p><p>In the Code World Model paradigm an LLM synthesizes an executable world model that a classical planner searches, and the model is accepted when it reproduces sampled transitions. We ask what that acceptance certifies in continuous control. We define the pipeline&#x27;s danger as an expected risk and isolate its exact factor: the probability that N i.i.d. gate rollouts all miss a critical event of probability r is exactly (1-r)^N; an independent acceptance sample adds its budget to the exponent. On three hybrid instruments the accepted mode-blind model is exploited: the planner is pinned at the mode boundary at a regret of nearly the whole attainable return. We prove a localization budget, valid at boundary points: models with Lipschitz constant at most L differing by eta at a point disagree above tolerance eps on a region of volume at least kappa((eta-eps)/L)^(d+m); the discontinuous reset modes studied pay no such budget. With real LLM synthesis, GPT-5.x repairs an omitted 1D clamp in 105 of 111 mode-containing draws -- every attempt exact on 50 of 56 instrument-stream blocks (95% CI [0.781, 0.960]). On 2D regions no artifact recovers the rule (0/156); eight targeted interventions leave the failure in place, and positive controls locate it: a located rule is not induced, while given form and location the constants follow exactly. A version-space certificate proves identification is class-relative: at the widest dose the declared fit succeeds in 20/20 blocks and every sample-consistent circle is within tolerance in 18/20. We prove a class of entry rules exactly consistent with every sample yet harmless at play, so identifiability is a measurable property of the instrument. Re-scoring all 1034 artifacts on independent samples confirms acceptance certifies sample consistency and no more: where the gate is provably informative it covers about two percent of the exploited planner&#x27;s queries.</p>]]></description>
    </item>
    <item>
      <title>Efficient RLVR Scheduling via Graph-Structured Online Difficulty Estimation</title>
      <link>http://arxiv.org/abs/2608.17941v1</link>
      <guid isPermaLink="false">2608.17941</guid>
      <pubDate>Tue, 18 Aug 2026 16:01:00 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Zhizhao Liu, Zhiliang Tian, Xi Wang, Zhihua Wen, Yihang Xiong, Zhiquan Lai, Dongsheng Li</p><p>Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models but relies on costly rollout exploration. Assigning the same exploration budget to samples with different difficulty levels is inefficient: easy samples may receive redundant rollouts, whereas difficult but learnable samples may receive too little exploration. Existing adaptive schedulers address this mismatch through curriculum-based sample selection or non-uniform rollout allocation based on estimated sample difficulty. However, obtaining reliable online difficulty estimates remains challenging: dedicated probing adds substantial generation overhead, whereas history-based estimators face a cold start with no initial observations and stale feedback, and typically ignore relations among samples. To address these limitations, we propose a plug-and-play graph-based online difficulty estimator that shares rollout feedback across related samples and continuously updates their difficulty estimates, mitigating cold start and staleness without dedicated probing. Specifically, we first construct a difficulty-aware sample graph based on semantic and reasoning similarities. Based on this graph, we introduce latent difficulty states and use a Potts prior to encourage neighboring samples to share the same state. We then employ a state-level Beta-Binomial model to aggregate the rollout outcomes associated with each state. Finally, we use an online mean-field variational algorithm to continuously update the latent-state assignments and state-level difficulty as new feedback arrives. Our framework can be integrated into sample-selection and rollout-allocation schedulers, enabling difficulty-adaptive exploration without dedicated probing. Experiments across multiple base models, RL schedulers, and benchmarks demonstrate that our framework achieves better performance.</p>]]></description>
    </item>
    <item>
      <title>Debate Training Reduces Reward Hacking in RLAIF</title>
      <link>http://arxiv.org/abs/2608.17776v1</link>
      <guid isPermaLink="false">2608.17776</guid>
      <pubDate>Tue, 18 Aug 2026 13:40:29 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Zachary Kenton, Lili Janzer, Rory Greig, Tian Huey Teh, Kirill Tyshchuk, Jonah Brown-Cohen, Harri Edwards, Senthooran Rajamanoharan, Noah Y. Siegel, Natasha Jaques, Rohin Shah</p><p>We demonstrate that RL finetuning an LLM using debate, a two-player adversarial game between a generator and a critic adjudicated by a weaker LLM judge, reduces reward hacking compared to a reinforcement learning from AI feedback (RLAIF) baseline. Reward hacking is a central obstacle in RLAIF: as training progresses, the policy learns to exploit systematic errors in its AI judge, degrading task performance, a problem that worsens precisely when the judge is weaker than the policy, the setting most relevant to overseeing increasingly capable AI systems. We study mathematics tasks, where final-answer correctness is verifiable, allowing us to measure reward hacking dynamics. We train a Gemini~2.5 Flash-class policy with a frozen, weaker Gemini~2.5 Flash Lite judge, comparing a single-player RLAIF baseline against debate. While the baseline quickly hacks the judge, debate maintains judge performance throughout training, leading to a higher peak validation accuracy (45\% performance gap recovered) that persists through many RL steps. Additional experiments show that: 1) further weakening the judge leads to faster hacking, but this can be compensated by adding an additional debate round; 2) debate incentives override prompted misalignment; 3) RL using an LLM judge has a smaller train/validation reward gap than RL from verifiable rewards; 4) learning to critique to convince the judge using ground truth labels is possible but slow. Taken together, our results are a positive update on the feasibility of debate, while highlighting that balancing multi-agent training is critical: without player constraints, adversarial training risks defaulting to critic judge-hacking. We show that critique word limits (effective up to 150 words) successfully balance the game and avoid judge hacking, though this introduces a trade-off by restricting critic expressive clarity.</p>]]></description>
    </item>
    <item>
      <title>Electromagnetic World Model for 6G: A Unified Framework for Joint Environment Reconstruction and Channel Prediction</title>
      <link>http://arxiv.org/abs/2608.17769v1</link>
      <guid isPermaLink="false">2608.17769</guid>
      <pubDate>Tue, 18 Aug 2026 13:32:17 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Yizhu Zhao, Li Yu, Jianhua Zhang, Yuxiang Zhang, Zhen Zhang, Guangyi Liu</p><p>The integration of sensing, communication, and intelligence is becoming a key enabler for sixth generation (6G) wireless systems, where intelligent terminals are expected to simultaneously support efficient link establishment and reliable environmental sensing. However, existing studies mainly exploit sensing information or communication information to address a single task, such as channel prediction or environment reconstruction. Motivated by the shared dependence of optical and radio-frequency signals on the surrounding environment, we propose the electromagnetic world model (EMWM), the first unified framework for joint environment reconstruction and channel prediction. EMWM learns a common electromagnetic representation with the potential to provide a modeling foundation for 6G tasks. Specifically, partial channel state information (CSI) and multi-view red-green-blue (RGB) images are encoded into CSI and visual tokens and jointly processed by a hierarchical world-model backbone with local and global aggregation. Based on the learned representation, a mixture-of-experts (MoE)-based CSI prediction head reconstructs the complete CSI, while a depth prediction head estimates multi-view depth maps that are further converted into three-dimensional (3D) point clouds. Moreover, a large-scale multi-modal dataset is constructed based on a campus digital twin. Experimental results show that EMWM outperforms conventional neural network and large language model (LLM) baselines in both CSI prediction and environment reconstruction, achieving a squared generalized cosine similarity (SGCS) of 0.9699 for CSI prediction while demonstrating robustness across different signal-to-noise ratio (SNR) conditions and zero-shot generalization at 28 GHz.</p>]]></description>
    </item>
    <item>
      <title>Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See</title>
      <link>http://arxiv.org/abs/2608.17744v1</link>
      <guid isPermaLink="false">2608.17744</guid>
      <pubDate>Tue, 18 Aug 2026 13:09:03 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Ayoub Kirouane, Christos Petrocheilos</p><p>Take three frontier mixture-of-experts models (Alibaba, OpenAI, NVIDIA; 3.6-4.0B active parameters each) and fine-tune them to reason in a low-resource language. On accuracy benchmarks almost nothing happens, and the benchmark itself is noise at this scale: changing only the random seed moves the score by 7.7 points, more than every data and recipe effect we measured. That null is our first result. The real changes live where accuracy cannot see. Base models never think in Greek: 0 of 1,000 reasoning traces, even when the question is Greek, so the model answers correctly while reasoning in a form its user cannot read, audit, or correct. After supervised fine-tuning (SFT), every released checkpoint reasons in the language of the question on ~98% of items, one family at 3x fewer tokens, with judged grammaticality improving on all four models and general ability within a few points of each base: nothing was forgotten, and fluency was gained. We propose six behavioural dimensions that make such changes measurable, each gated to reject any metric that correlates with output length, and we report how our own instruments lied: six failures, each caught by a control. What SFT cannot do is fix its own defects: a quarter of answers skip the requested format, answers leak into the reasoning channel, and an explicit &quot;think in English&quot; is obeyed under half the time. Reinforcement learning with verifiable rewards, pre-registered before training, fixes the first two outright (fallback 24% to 2.5%, leak 3.5% to 0.0%, both against a flat random-reward control) and moves the third (+9.1pp), while the Greek reasoning habit survives an accuracy-only gradient untouched. We release five checkpoints. The instruments, the controls and the pre-registration travel to any low-resource language; Greek is the case that let us measure them.</p>]]></description>
    </item>
    <item>
      <title>Offline Multi-Agent Reinforcement Learning with a Physics-Informed World Model for Cooperative Mixed Traffic Control</title>
      <link>http://arxiv.org/abs/2608.17739v1</link>
      <guid isPermaLink="false">2608.17739</guid>
      <pubDate>Tue, 18 Aug 2026 13:03:51 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Lu Liu, Chi Xie, Xi Xiong</p><p>This study investigates cooperative control of connected and automated vehicles (CAVs) at partially observable highway bottlenecks in mixed traffic, aiming to mitigate congestion without relying on complete global traffic states or online trial-and-error. We propose a physics-informed world model-based offline multi-agent reinforcement learning framework that reconstructs a physically interpretable global traffic state from local CAV observation-action histories, with coupled macroscopic-microscopic traffic dynamics providing physics-based supervision. A probabilistic ensemble world model learns traffic-state transitions and system rewards, while model disagreement quantifies epistemic uncertainty. Multi-step imagined rollouts with pessimistic rewards and uncertainty-driven truncation are then used for offline policy learning. Experiments in a SUMO-based on-ramp bottleneck using approximately $1\times10^6$ offline transitions show that physics supervision improves state reconstruction and world-model prediction accuracy.</p>]]></description>
    </item>
    <item>
      <title>rl-triton: High-Performance Triton GPU Kernels for Reinforcement Learning Credit Assignment</title>
      <link>http://arxiv.org/abs/2608.17641v1</link>
      <guid isPermaLink="false">2608.17641</guid>
      <pubDate>Tue, 18 Aug 2026 10:58:20 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Lars Simon Zehnder</p><p>We present rl-triton, an open-source library of high-performance GPU kernels for reinforcement learning credit assignment, implemented in Triton. The core contribution is a unified associative scan framework that recasts seven distinct RL estimation algorithms - Generalized Advantage Estimation (GAE), V-Trace, Retrace($λ$), TD($λ$) returns, discounted returns, eligibility traces, and episodic prefix sums - as instances of a single first-order linear recurrence solved in $O(\log T)$ parallel steps. All algorithms share the same associative scan operator, with algorithm-specific fused Triton kernels constructing their recurrence coefficients on-chip. We verify the associative operator algebraically and define the treatment of terminated and truncated episodes explicitly. Benchmarks show a 1.6-5.70$\times$ full-call speedup over a vectorized torch.compile baseline in the massively parallel simulation regime (thousands of environments, short rollouts). The reported range covers all seven algorithms on both GPUs, both with and without per-step truncation handling. For most algorithms, speedups increase at longer sequence lengths, as the baseline requires more scan stages as $\log T$ grows, each adding an intermediate HBM round-trip. The library is available at https://github.com/simonsays1980/rl-triton.</p>]]></description>
    </item>
    <item>
      <title>Iterative Grasp Pose Refinement: A Deep Reinforcement Learning Approach for 2D Vision</title>
      <link>http://arxiv.org/abs/2608.17628v1</link>
      <guid isPermaLink="false">2608.17628</guid>
      <pubDate>Tue, 18 Aug 2026 10:44:58 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Amir Arsalan Nematollahi, Shayan Ahmadi, Mehdi Tale Masouleh, Ahmad Kalhor</p><p>Developing robots capable of understanding and manipulating objects requires compact, interpretable, and generalizable representations. This work proposes a reinforcement learning-based framework for robotic grasp refinement, integrating keypoint-based object representations with a Deep Q-Network (DQN). Using 2D overhead images captured in a simulated environment, a geometric-based algorithm generates initial grasp candidates, which are iteratively refined by the proposed framework, transforming failed grasps into successful ones. Experiments conducted on 300 objects from the Dex-Net dataset using a UR5 manipulator demonstrate the framework&#x27;s effectiveness, achieving a 100% success rate on objects previously deemed ungraspable by geometrical methods. The framework&#x27;s sim-to-real transferability is further validated through physical experiments on a Delta parallel robot, where a refined grasp successfully manipulates an object that was previously ungraspable. The findings underscore the effectiveness of reinforcement learning in addressing challenges in robotic grasping, offering a scalable and adaptable solution for contact-rich manipulation tasks.</p>]]></description>
    </item>
    <item>
      <title>LIBERO-VIFO: Benchmarking the Capability and Safety of Visual Cue Following in Vision-Language-Action Models</title>
      <link>http://arxiv.org/abs/2608.17600v1</link>
      <guid isPermaLink="false">2608.17600</guid>
      <pubDate>Tue, 18 Aug 2026 10:07:17 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Zhengyan Qian, Rui Yan, Alex Jinpeng Wang, Jinhui Tang</p><p>Visual cues are increasingly adopted to guide robot learning, but whether Vision-Language-Action (VLA) models can reliably follow authorized cues while disregarding unauthorized ones remains unclear. Existing work covers only a narrow range of cue forms and focuses on final task success, providing only a coarse assessment of cue-following capability. Treating all visual cues as authorized also leaves safety risks of unauthorized following unexplored. To address these gaps, we introduce LIBERO-VIFO, a benchmark to evaluate both the capability and safety of visual cue following in VLA models. LIBERO-VIFO defines eight visual cue families spanning diverse forms. A total of four protocols in two parts are defined: Part I tests cue understanding and authorized following, while Part II evaluates unauthorized visual cue following under language-cue conflict and empty language conditions. Evaluating seven VLA models reveals that although visual cue understanding does not reliably translate into execution, current VLAs are able to execute cue-indicated tasks without language instruction, exposing an emerging risk of unauthorized visual cue following. Extended experiments on scene-instantiated cues, safety-critical settings, and real-robot deployment corroborate these findings. LIBERO-VIFO brings both the capability and safety of visual cue following into systematic evaluation, establishing visual-centric safety as a new perspective for the VLA community.</p>]]></description>
    </item>
    <item>
      <title>tinyDSM: A Framework for Skill Modeling and Development for Resource-Constrained Millirobots</title>
      <link>http://arxiv.org/abs/2608.17596v1</link>
      <guid isPermaLink="false">2608.17596</guid>
      <pubDate>Tue, 18 Aug 2026 10:03:56 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Markus D. Kobelrausch, Michael Miedler, Axel Jantsch</p><p>In this study, we investigate developmental mechanisms that enable small, resource-constrained systems such as cm-sized millirobots to autonomously explore, learn, and adapt their capabilities throughout their lifespan. Reinforcement learning algorithms guide the agent&#x27;s skill acquisition and adaptation through the interplay of our proposed tinyDSM, which integrates intrinsic motivation and fitness-based assessment. We strive for minimal, hard-wired skills while encouraging the open-ended development of new skills. A key emphasis in our approach is to encode minimal a-priori general knowledge, which serves as a foundational starting point for the system as it further learns system-specific dependencies from the initial knowledge provided. Thus, by design, our approach attempts to cover very generic application domains. The methodology is based on (a) developmental mechanism with intrinsic motivation, and (b) a cognitive architecture (knowledge, reasoning, learning), while (c) utilizing minimal resources. It uses a hierarchical knowledge graph and kinematic reasoners to model and evaluate simple and advanced motion related skills. In our experiments, we use a resource-constrained millirobot with a volume of 36 cm^3 with a Raspberry Pi Pico 32-bit microcontroller (RP2040) that integrates all described features and capabilities except the camera system in 9 kB. Starting with learning the most elementary motor skills the millirobot autonomously progresses from simple linear and angular movements to complex geometric patterns within 15 minutes. To complement the physical experiments, we perform a simulation-based analysis that enables systematic comparisons across learning algorithms and intrinsic motivation parameters.</p>]]></description>
    </item>
    <item>
      <title>Write, Execute, Refine: From Skill Followers to Skill Optimizers via Reinforcement Learning from Execution Feedback</title>
      <link>http://arxiv.org/abs/2608.17587v1</link>
      <guid isPermaLink="false">2608.17587</guid>
      <pubDate>Tue, 18 Aug 2026 09:52:48 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Kang Peng, Zhiwei Zhang, Yichen Zhang, Zezhong Wang, Yiming Du, Geng Tu, Baojun Wang, Bin Liang, Ruifeng Xu, Kam-Fai Wong</p><p>Expert-written natural language skills can improve tool-using agents, yet agent-authored skills perform 8-11 points worse than using no skill. This gap suggests that following procedural guidance and improving it from execution evidence are distinct capabilities. Inference time loops can repair skills but do not improve the model that writes the next one. We study how to organize execution experience from intermediate skills into training states for an optimizer. We introduce WER (Write, Execute, and Refine), a multi-phase framework that trains a Skill Optimizer outside a frozen executor. The optimizer proposes skills, a frozen agent executes each repeatedly, and a programmatic verifier scores the outcomes. The scores provide relative credit and select mixed-outcome records. Matched successful and failed trajectories from these records form the next phase&#x27;s refinement states, so the optimizer learns from the consequences of its earlier outputs. On BFCL v4 multi-turn and tau2-bench, WER improves average Pass@1 over the no-skill baseline by 7.80 and 3.85 points, respectively. Under an identical refinement workflow, it outperforms the same backbone without optimizer training by 9.35 and 10.29 points. The trained 4B optimizer reaches 76.63 percent on BFCL v4, outperforming all evaluated off-the-shelf general-purpose models used as skill optimizers on average.</p>]]></description>
    </item>
    <item>
      <title>HODAgent: Towards On-Demand, Responsive Humanoids for Physical World Human Interaction</title>
      <link>http://arxiv.org/abs/2608.17584v1</link>
      <guid isPermaLink="false">2608.17584</guid>
      <pubDate>Tue, 18 Aug 2026 09:49:16 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Wang Warren Chen, Jiahao Zhang, Zhenjiang Li, Mingxu Wang, Lei Yi, Yuchen Kang, Shuo Sun, Ziping Chen, Jie Chen</p><p>We propose HODAgent, a System-2 embodied agent for humanoid robots in service settings, addressing situated intent, responsive execution, task revision, and outcome verification. Its semi-duplex architecture integrates an Env-Interactor, Planner, Executor, and hierarchical Memory to maintain coherent interaction, planning, and task state during service episodes. This allows handling new requests during motion, retaining progress, revising actions, and grounding closure in execution outcomes. A shared interface connects simulation and physical robots (Unitree G1), isolating platform-specific control. In an interactive simulation with 164 cases, HODAgent achieves 84.8% and 91.5% Joint Success under two VLM backbones, outperforming baselines by 9.8 and 18.9 points. On physical robots, pass rates are 92% (atomic), 72% (composite), and 63.3% (complete tasks). On multiple embodied benchmarks, it improves over baselines by 0.7-9.0 points. Results show a unified System-2 agent enables adaptive humanoid service across simulation and reality.</p>]]></description>
    </item>
    <item>
      <title>No Gaussian Required: Contrastive Inverse Dynamics for JEPA World Models</title>
      <link>http://arxiv.org/abs/2608.17542v1</link>
      <guid isPermaLink="false">2608.17542</guid>
      <pubDate>Tue, 18 Aug 2026 09:03:35 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Jack Boylan, Chris Hokamp</p><p>Joint-Embedding Predictive Architectures (JEPAs) learn world models by predicting future embeddings, but the objective admits a trivial solution of a constant encoder, so every practical system adds an anti-collapse mechanism (LeCun, 2022; Assran et al., 2023; Bardes et al., 2022; 2024). LeWorldModel (LeWM) prevents collapse with SIGReg, a regularizer that forces the latent distribution to match an isotropic Gaussian: the representation is stabilized by prescribing what it must look like, independently of the environment it models. We argue that the anti-collapse pressure can instead come from the transition data itself. Action-Contrastive Masked Transition Modeling (AC-MTM) keeps LeWM&#x27;s forward latent-prediction objective and adds a training-only inverse-dynamics head trained with Action-NCE: each latent transition must identify the action that produced it among the other actions in the batch, a discrimination task that a collapsed encoder provably fails. The inverse branch is discarded after training, leaving test-time encoding, forward prediction, planning, and compute identical to LeWM. On four standard pixel-control tasks under a matched planning protocol, AC-MTM trains stably from scratch and matches SIGReg on average. On the harder multi-object OGBench Visual Scene task, results are consistent with the prescribed geometry becoming a bottleneck: AC-MTM reaches 80.0$\pm$2.0% success versus 58.0$\pm$2.0% for SIGReg, improving by 20-24 points in each training seed. A single 50-episode random-policy run gives a 52% baseline estimate. Contrastive inverse dynamics thus provides a distribution-free anti-collapse signal that requires no target network, stop-gradient, pretrained encoder, or reconstruction objective, and we characterize the action-space and observability assumptions under which it holds. We make our code available at https://github.com/jackboyla/action-contrastive-jepa</p>]]></description>
    </item>
    <item>
      <title>Evaluating RL Explainability Methods by How Much They Help Fix Bugs in Agents</title>
      <link>http://arxiv.org/abs/2608.17524v1</link>
      <guid isPermaLink="false">2608.17524</guid>
      <pubDate>Tue, 18 Aug 2026 08:46:26 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Ram Rachum, Yotam Amitai, Bálint Gyevnár, Reuth Mirsky, Cameron Allen</p><p>This preliminary paper outlines a planned evaluation benchmark for Explainable Reinforcement Learning (XRL) methods. Current evaluations rely on functionally-grounded metrics like faithfulness and compactness, and on human-grounded proxies like subjective ratings or prediction accuracy. We suggest evaluating XRL methods by how effectively their generated explanations help to diagnose and fix malfunctioning reinforcement learning (RL) agents. We propose EvalXRL, a benchmark in which a Large Language Model (LLM) coding agent uses different XRL methods to diagnose a held-out malfunction in an RL agent, and then repair it. Our proposed benchmark iterates across (environment $\times$ malfunction $\times$ XRL method) tuples and uses the reward signal of the RL agents to form a final score for each XRL method. The coding agent may use the method interactively: invoke the XRL method, process its output, form new hypotheses on what is broken, and invoke the method again with parameters adjusted for testing these hypotheses. This closed-loop structure may be described as a simplified version of the scientific method. Some XRL methods provide self-evaluations that follow this pattern; we propose the first head-to-head comparison of multiple XRL methods in closed-loop usage.</p>]]></description>
    </item>
    <item>
      <title>Towards Better Agents for Multi-Turn User Interaction: The Next User Turn Is More Than Context</title>
      <link>http://arxiv.org/abs/2608.17499v1</link>
      <guid isPermaLink="false">2608.17499</guid>
      <pubDate>Tue, 18 Aug 2026 08:25:21 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Yiwen Zhao, Zhihao Wen, Yuchen Mao, Mingxuan Jiang, Yihao Hu, Pan Wang, Xin Zhang, Wei Wu</p><p>User-facing tool agents must coordinate dialogue and tool use as user goals unfold over multiple turns. Yet interactive reinforcement learning typically reduces each rollout to a terminal reward, assigning the same credit to effective elicitation, errors, and later repair. The next user turn is more than context: it also provides noisy, temporally local evidence about the preceding user-to-user segment. We introduce \textbf{F}eedback-\textbf{A}ware \textbf{C}redit \textbf{A}ssignment (\textsc{FACA}), which aligns each reaction with that segment, derives a locally normalized reaction advantage, and adds it to verified terminal outcome advantage without an extra critic or rollout. Against an outcome-only Interactive GRPO control matched in simulator, visible dialogue, initialization, rollout, and optimization, \textsc{FACA} improves the nine-domain $τ$-family average across three independently trained runs by 5.91 and 10.22 percentage points at 8B and 14B, respectively. Gains concentrate in Telecom; at 8B, randomizing reaction polarity removes the Telecom gain. The same ordering holds zero-shot on Pare-Bench and Co-Gym. These results demonstrate that next-turn user reactions provide actionable local credit for improving multi-turn user-interacting agents.</p>]]></description>
    </item>
    <item>
      <title>Calibrated Predictive Safety for Heterogeneous Robots: An Action-Conditioned JEPA Framework with Model-Based Safety Shields</title>
      <link>http://arxiv.org/abs/2608.17496v1</link>
      <guid isPermaLink="false">2608.17496</guid>
      <pubDate>Tue, 18 Aug 2026 08:24:08 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Kaiming Zhong, Tianhua Liu, Yue Wang</p><p>Vision-language-action policies generalize broadly but provide no execution-time guarantees; classical model-based planners respect kinematic and geometric constraints but generalize poorly. We study whether an action-conditioned Joint-Embedding Predictive Architecture (JEPA) world model can predict, before execution, both task progress and physical risk for candidate action chunks, and whether coupling these predictions to an embodiment-specific model-based safety shield yields a deployable pipeline for heterogeneous robots. We propose a receding-horizon decision pipeline: (1) a proposer produces K candidate action chunks; (2) an action-conditioned JEPA rolls each candidate forward in a frozen-encoder latent space conditioned on an embodiment embedding; (3) calibrated risk and progress heads score each rollout and report uncertainty; (4) a deterministic per-embodiment safety shield filters inadmissible candidates; (5) a fallback ladder handles empty-admissible-set cases. The learned ranking only reorders admissible candidates; enforcement guarantees come from the deterministic shield and fallback ladder. We evaluate with a pre-registered protocol in simulation (LIBERO-Long). In 600-episode configurations the full framework improved success over a shield-only baseline and reduced collision false negatives at matched recall. Deployment-efficiency measurements on target on-robot and edge accelerators are included. Real-robot experiments and an offline reranking significance test remain future work; see the paper for disclosures.</p>]]></description>
    </item>
    <item>
      <title>Optimal control of a swimming robot based on Purcell&#x27;s microswimmer model</title>
      <link>http://arxiv.org/abs/2608.17455v1</link>
      <guid isPermaLink="false">2608.17455</guid>
      <pubDate>Tue, 18 Aug 2026 07:34:57 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Noam Berkovich Lahav, Oren Wiezel, Yizhar Or</p><p>Purcell&#x27;s swimmer is a well-known planar model of a swimming microorganism, governed by low Reynolds number hydrodynamics, which is comprised of three rigid links connected by actuated rotary joints. This model has been analyzed as a robotic locomotion system governed by first-order nonlinear dynamics with a periodic input (gait) of the two joint angles. In this work, we present a robotic macro-scale realization of this three-link swimmer moving in a highly viscous fluid. We propose a simple variant of Purcell&#x27;s theoretical model with non-slender links and a central rigid sphere which represents the added drag of the robot&#x27;s central flotation block, and calibrate the model&#x27;s parameters to fit experimental measurements. Next, we apply optimal control formulation based on Pontryagin&#x27;s Maximum Principle (PMP) in order to find optimal gaits that maximize the displacement per cycle under bounds on the joint angles. Employing a differential geometric method that transforms the problem to area integral enclosed by the gait trajectory in the plane of joint angles, enables visual interpretation which explains topological changes in displacement-optimal gaits upon varying the bound on the joint angles. We then apply PMP formulation to the problem of maximizing Lighthill&#x27;s energy efficiency in order to obtain a boundary value problem (BVP) whose solution gives efficiency-optimal gaits for Purcell&#x27;s swimmer model, as well as its variant with a central sphere. Finally, we utilize numerical methods such as parameterizing the input gait as a truncated Fourier series, as well as GPOPS-II solver, to produce sufficient initial guess values for solving the BVPs and obtaining efficiency-optimal gaits.</p>]]></description>
    </item>
    <item>
      <title>EATR-Stereo: Embodiment-Aware Routing of Paired Stereo Evidence for Humanoid Vision-Language-Action Control</title>
      <link>http://arxiv.org/abs/2608.17453v1</link>
      <guid isPermaLink="false">2608.17453</guid>
      <pubDate>Tue, 18 Aug 2026 07:32:50 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Songwei Wu, Rui Zhao, Fan Yang, Zhongqiang Nie, Zhiduo Jiang, Wandong Sun, Yuwei Li, Yang Liu, Hong Liu</p><p>Long-horizon humanoid vision--language--action (VLA) control with head-mounted stereo cameras requires visual interfaces that can exploit complementary views while maintaining compatibility with pretrained representations. Existing interfaces often discard complementary stereo evidence or fuse additional observations without preserving the native primary-view pathway and adapting auxiliary information to robot embodiment. We present EATR-Stereo, an embodiment-aware token-routing framework that retains primary-view tokens and constructs primary-aligned Cross-View Auxiliary Tokens (CVATs) by querying the synchronized auxiliary-view token sequence. A body-segmented proprioceptive encoder further conditions token-wise auxiliary usage on robot configuration history, enabling selective incorporation of stereo evidence during action generation. The routed auxiliary stream augments the language and primary-visual context of a pretrained VLA while keeping its vision--language model frozen. On a 33-DoF physical humanoid with a 37-D proprioceptive state, we evaluate nine configurations in over-100-s search--approach--grasp--place--return tasks. EATR-Stereo achieves 60.0% full-task success, 100.0% grasp success, and 80.0% stage success. Under severe asymmetric occlusion, it improves recovery to 80% compared with 30% for CVAT alone. Ablation studies further show the importance of preserving primary tokens and combining cross-view auxiliary features with structured proprioceptive routing. These results demonstrate that selectively routed paired stereo evidence improves spatial grounding for reliable long-horizon humanoid VLA control.</p>]]></description>
    </item>
    <item>
      <title>UniReflex: Plug-and-Play Force Control for Pretrained Generative Policies via Fast-Slow Reflex</title>
      <link>http://arxiv.org/abs/2608.17432v1</link>
      <guid isPermaLink="false">2608.17432</guid>
      <pubDate>Tue, 18 Aug 2026 06:54:22 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Yan Huang, Shoujie Li, Ziwu Song, Wenbo Ding</p><p>Generative imitation learning policies excel at trajectory planning but lack closed-loop force regulation, while directly incorporating force modalities often requires redesigning or retraining the network. We present UniReflex, a universal plug-and-play framework that equips frozen generative policies with variable impedance control (VIC) for contact regulation, guided by force-direction intent collected during demonstration, without further slow-backbone fine-tuning. By non-invasively intercepting deep latent representations from the action head, UniReflex drives a fast reflex network that decouples active force exertion from external interaction response. This scheme predicts normalized anisotropic stiffness directions for directional compliance allocation. Furthermore, UniReflex integrates an adaptive gating mechanism that enables seamless transitions between position-dominant planning and force-dominant execution. Real-world bimanual experiments demonstrate that UniReflex significantly improves contact stability and success rates while preserving original position accuracy. Our approach achieves 25-66x lower per-step backward latency relative to joint training strategies on the evaluated backbones.</p>]]></description>
    </item>
    <item>
      <title>Prism-GRPO: Faster VLA Policy Optimization via Splitting Same-outcome Groups</title>
      <link>http://arxiv.org/abs/2608.17423v1</link>
      <guid isPermaLink="false">2608.17423</guid>
      <pubDate>Tue, 18 Aug 2026 06:44:16 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Zeyun Deng, Yuzhe Lu, Yawei Wang, Linbo Liu, Qing Ping, Han Ding, Guande Wu, Panpan Xu, Jun Huan</p><p>GRPO is increasingly used for reinforcement learning of vision-language-action (VLA) policies because, unlike PPO, it does not require training a critic. This simplification comes with a sampling cost: group-relative advantages require multiple rollouts from each scene. Under binary success rewards, groups whose rollouts all succeed or all fail have zero advantage and are discarded by dynamic sampling. These groups are especially common early in training, when most rollouts fail, wasting much of the expensive robotic rollout budget. We introduce Prism-GRPO, which augments binary outcome reward with a weighted trajectory-level execution-quality score. By splitting same-outcome groups into a quality spectrum, Prism-GRPO recovers training signal while ensuring that every success still outranks every failure. Quality scores can be derived from simulator contacts, executed actions, or visual observations, avoiding task-specific progress rewards. We prove that Prism-GRPO never increases the probability that a sampled group is discarded for having zero advantages, and derive a gradient-alignment condition under which its combined update remains a local ascent direction for task success. Across four RoboTwin tasks spanning different horizons and coordination patterns, Prism-GRPO improves success and quality at matched rollout budgets and reaches target success rates with up to 56% fewer rollouts. It also suppresses a reward-hacking shortcut, with the cleaner behavior transferring under direct deployment to a real robot. Through ablations, we show consistent gains across contact-, smoothness-, and VLM-derived quality signals.</p>]]></description>
    </item>
    <item>
      <title>REChart: Reasoning-Efficient Chart Editing with Large Reasoning Models</title>
      <link>http://arxiv.org/abs/2608.17414v1</link>
      <guid isPermaLink="false">2608.17414</guid>
      <pubDate>Tue, 18 Aug 2026 06:26:45 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Yuanbang Liu, Chenxi Ruan, Yihan Hou, Qiong Luo, Wei Zeng</p><p>Chart editing requires inferring and modifying visualization code from a reference chart image based on an editing instruction, challenging fine-grained visual reasoning, instruction following, and executable code synthesis capabilities of MLLMs. Large reasoning models (LRMs) with extended Chain-of-Thought (CoT) reasoning are suitable for tackling such complex multimodal tasks. However, our preliminary study reveals an ``inverted-U&#x27;&#x27; relationship between reasoning length and chart-editing performance: Excessive reasoning often leads to ``overthinking,&#x27;&#x27; where models drift toward hallucinated visual details or get stuck in redundant reasoning loops. To address the gap, we introduce REChart, a two-stage training framework that provides process-level supervision over intermediate reasoning steps, improving both editing fidelity and reasoning efficiency. First, we synthesize 200k high-quality reasoning trajectories for supervised fine-tuning from a large image-instruction-code pool, using a role-specialized agentic Reason-Score-Refine workflow that iteratively refine the chart code toward higher quality. Second, we optimize the model via reinforcement learning with two complementary rewards: a \emph{fidelity} reward evaluating code correctness, visual fidelity, and structural consistency, and an \emph{efficiency} reward that assigns each rollout a random thinking budget, truncates the reasoning process, and credits the final reasoning segment according to its contribution to the output. On the ChartEdit and ChartMIMIC benchmarks, our model achieves state-of-the-art chart-editing performance among open-source models of comparable scale, while mitigating overthinking and reducing average reasoning token usage by 79.0\% under a maximum thinking budget of 16,384 tokens compared with the base model.</p>]]></description>
    </item>
    <item>
      <title>LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents</title>
      <link>http://arxiv.org/abs/2608.17393v1</link>
      <guid isPermaLink="false">2608.17393</guid>
      <pubDate>Tue, 18 Aug 2026 05:34:35 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Yiming Du, Yuxin Jiang, Tao Yuan, Jianbo Dai, Shaowei Wang, Jierun Chen, Chaofan Tao, Xianzhi Yu, Lifeng Shang, Kam-Fai Wong, Xiaohui Li, Haoli Bai</p><p>Reinforcement learning for coding agents increasingly relies on long-running agent harnesses to manage tool integration, repository contexts, and execution feedback. However, the native execution environments of these harnesses are inherently misaligned with policy-gradient training: environmental crashes and reward hacking corrupt outcome signals, while train-inference discrepancies decouple rollout behavior from policy updates. To address this, we present LEGO-RL, a framework that bridges native coding-agent harnesses with scalable policy-gradient optimization without modifying their internal control flow. LEGO-RL is built upon three pillars: (1) faithful optimization via in-process LLM proxying that captures raw generation streams for token-level alignment and robust trainer-side log-probability recomputation, even under harness-side compaction or re-serialization; (2) reliable execution via scalable sandbox orchestration featuring image caching and stage-wise defenses to mitigate reward hacking; and (3) observable training through an integrated plugin that automates validation and monitoring, paired with a Live UI for granular trajectory diagnostics. We evaluate LEGO-RL by training the sparse MoE model Qwen3.5-35B-A3B with GSPO across three native coding-agent harnesses. LEGO-RL improves Qwen3.5-35B-A3B across OpenHands SDK (64.0% to 70.4%), Claude Code (62.4% to 68.2%), and OpenCode (57.2% to 66.6%) on SWE-bench Verified, while maintaining a rollout-training probability correlation above 0.99.</p>]]></description>
    </item>
    <item>
      <title>Integrating Novelty and Surprise for Experience Prioritization and Exploration in Image-Based Reinforcement Learning</title>
      <link>http://arxiv.org/abs/2608.17373v1</link>
      <guid isPermaLink="false">2608.17373</guid>
      <pubDate>Tue, 18 Aug 2026 05:07:58 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Hoda Yamani, Henry Williams, Bruce A. MacDonald</p><p>Sample efficiency is a central challenge in reinforcement learning (RL), particularly in image-based domains where agents must learn from high-dimensional visual inputs. Traditional sampling often relies on random or suboptimal experience selection, leading to redundant updates and slow learning. Improving efficiency requires mechanisms that prioritize informative experiences while also encouraging effective exploration. Prioritized Experience Replay (PER) addresses part of this challenge by reusing high-value transitions, while intrinsic rewards promote the exploration of novel or uncertain states. However, their integration has not been extensively studied. This paper introduces Novelty and Surprise Prioritized Experience Replay (NSPER), which uses novelty to capture underrepresented states and surprise to expose gaps in the agent&#x27;s understanding of the environment. We further extend this with NSPER+R, integrating these signals as intrinsic rewards to jointly improve replay quality and exploration. Experiments on DeepMind Control Suite tasks show that NSPER and NSPER+R improve training efficiency and convergence speed compared to existing methods in image-based RL.</p>]]></description>
    </item>
    <item>
      <title>Repetition as Reinforcement: Enhancing Sample Efficiency via Instant Episode Repetition in Reinforcement Learning</title>
      <link>http://arxiv.org/abs/2608.17347v1</link>
      <guid isPermaLink="false">2608.17347</guid>
      <pubDate>Tue, 18 Aug 2026 04:11:50 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Hoda Yamani, Yuning Xing, Koen van Rijnsoever, Bruce A. MacDonald, Henry Williams</p><p>Repetition is a fundamental mechanism in human learning, where revisiting successful experiences strengthens memory, consolidates skills, and improves future performance. Motivated by this biological principle, we introduce Instant Episode Repetition (IER), a simple and novel mechanism that improves sample efficiency by immediately repeating action sequences from successful episodes during environment interaction. Unlike conventional approaches such as Experience Replay and Self-Imitation Learning (SIL), which passively reuse past experience during training updates, IER directly influences the data collection process. Upon identifying a high-reward episode, the agent repeats its action sequence for a fixed number of subsequent episodes, reinforcing valuable behaviors through renewed interaction with the environment. We integrate IER into state-of-the-art SAC and TD3 algorithms and evaluate its effectiveness on continuous-control benchmarks, including MuJoCo, the DeepMind Control Suite, and a real-world dynamic object translation task with a robotic manipulator. Experimental results demonstrate that this simple mechanism improves learning performance over standard and self-imitation-based baselines.</p>]]></description>
    </item>
    <item>
      <title>ORPA: Online Residual Policy Adaptation for Robot Manipulation Control with Human Feedback</title>
      <link>http://arxiv.org/abs/2608.17323v1</link>
      <guid isPermaLink="false">2608.17323</guid>
      <pubDate>Tue, 18 Aug 2026 03:30:12 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Muhammad A. Muttaqien, Tomohiro Motoda, Ryo Hanai, Yukiyasu Domae</p><p>Robotic manipulation policies trained via imitation learning, such as Action Chunking with Transformers (ACT), can achieve strong performance under ideal conditions but often remain sensitive to small execution errors and distribution shifts. Correcting these failures typically requires dataset aggregation and full-policy retraining, which is computationally expensive and unsuitable for real-time deployment. In this work, we propose Online Residual Policy Adaptation (ORPA), a framework that enables immediate, feedback-driven correction of robot actions without modifying the underlying policy parameters. ORPA augments a pretrained control policy with a lightweight, feedback-conditioned module that predicts residual adjustments directly in joint space, allowing the system to adapt its behavior at runtime. We evaluate ORPA on a set of precision-sensitive manipulation tasks using the ALOHA platform, demonstrating improvements in success rate and recovery from small perturbations compared to baseline control policies and rule-based inverse kinematics corrections.</p>]]></description>
    </item>
    <item>
      <title>Robust Brachiation on a Life-Sized Dual-Arm Robot Using Waypoint-Guided Reinforcement Learning</title>
      <link>http://arxiv.org/abs/2608.17320v1</link>
      <guid isPermaLink="false">2608.17320</guid>
      <pubDate>Tue, 18 Aug 2026 03:24:50 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Ayumu Iwata, Kento Kawaharazuka, Keita Yoneda, Takahiro Hattori, Kei Okada</p><p>Brachiation is a form of locomotion in which primates move primarily using their arms, enabling traversal in environments without footholds. However, this motion requires highly coordinated whole-body movement and precise timing control for bar grasping and release. As a result, achieving robust behavior on life-sized robotic platforms remains challenging. In this study, we present a reinforcement learning-based method to realize brachiation on a life-sized dual-arm robot. The core of the proposed approach is Waypoint-Guided Reinforcement Learning (WGRL), a learning framework for inducing non-linear and complex motions. For high-difficulty tasks where imitation learning data are unavailable, WGRL guides behavior acquisition by sparsely specifying waypoints for the end-effector trajectory, while whole-body motion is generated through reinforcement learning. In addition, by integrating the waypoint-following guidance with rewards based on task success and mechanical energy, and training in an environment designed for Sim-to-Real transfer, the proposed method achieves both forward progression and motion stability. The acquired behavior is evaluated through Sim-to-Sim experiments under monkey-bar environments with geometric variations and hardware experiments, confirming robust brachiation including failure recovery behavior. This study provides effective learning design guidelines for realizing arm-based locomotion on life-sized robotic hardware and expanding the traversable workspace of robots.</p>]]></description>
    </item>
    <item>
      <title>Agentic ESOpt: Fine-Tuning Long-Horizon LLM Agents with Minimal GPU Requirements</title>
      <link>http://arxiv.org/abs/2608.17310v1</link>
      <guid isPermaLink="false">2608.17310</guid>
      <pubDate>Tue, 18 Aug 2026 03:03:53 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Zhi Zheng, Rongsheng Chen, Yunpeng Ba, Zhenkun Wang, Yee Whye Teh, Wee Sun Lee</p><p>Reinforcement Learning (RL) has been promising in single-turn LLM fine-tuning. However, long-horizon agentic reasoning introduces increasingly branching interactions and sparse rewards, exposing several limitations of RL: its heavyweight backpropagation-based training stack makes it impractical to fine-tune larger LLMs, and longer-horizon trajectories make credit assignment in RL substantially harder. This paper argues that evolution strategies (ES) can be a better choice for fine-tuning long-horizon LLM agents. Compared with agentic RL, ES offers three key advantages: 1) Model Scalability: ES enables full-parameter optimization with only minimal, inference-level GPU memory, making it possible to fine-tune large LLMs. 2) Flexibility: its lightweight, black-box feedback interface makes ES fine-tuning easy to compose with prompt-space evolution (e.g., skill optimization &amp; test-time compute); and 3) Long-Horizon Scalability: ES performs trajectory-level parameter attribution without decomposing rewards across horizons, yielding better scalability than Agentic RL as the horizon length grows. Based on this insight, we propose Agentic ESOpt, a full-parameter agentic fine-tuning framework tailored to flexible parameter--context co-evolution. At each step, Agentic ESOpt samples perturbations around the current LLM parameters, evaluates the resulting agents with rewards, and applies an online reward-weighted update. To improve the exploration--adaptation trade-off, Agentic ESOpt further introduces a cosine decay schedule of the perturbation scale $σ$. On WebArena-Lite, full-parameter optimization of Qwen-3.5-27B improves the No Skill baseline by 6.69%. In test-time automatic heuristic design, Agentic ESOpt performs online prompt--parameter co-evolution, improving its matched baseline in 28 of 36 settings.</p>]]></description>
    </item>
    <item>
      <title>SignalReasoner: Assessing the Upper Bound of 3B Models for Signal Mathematical Reasoning</title>
      <link>http://arxiv.org/abs/2608.17301v1</link>
      <guid isPermaLink="false">2608.17301</guid>
      <pubDate>Tue, 18 Aug 2026 02:52:41 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Guozheng Sun</p><p>Post-training with supervised chain-of-thought fine-tuning and reinforcement learning from verifiable rewards has substantially improved the mathematical reasoning capabilities of large language models (LLMs). However, their application to signal processing problems remains relatively under-explored. This report investigates reinforcement fine-tuning strategies for adapting Qwen2.5-3B-Base to graduate-level signal mathematical problems from WirelessMATHBench-XL, a comprehensive benchmark for mathematical reasoning in this domain. We examine two training paradigms: (i) direct reinforcement learning (RL) on WirelessMATHBench-XL with verifiable rewards; and (ii) supervised fine-tuning (SFT) on a distilled wireless-domain chain-of-thought corpus, followed by the same domain-specific RL stage. Across both paradigms, we benchmark Group Relative Policy Optimization (GRPO), Group Sequence Policy Optimization (GSPO), and Geometric-Mean Policy Optimization (GMPO). We aim to assess whether domain-aware CoT SFT serves as an effective initialization for subsequent RL, and whether GSPO or GMPO offer advantages in stability or accuracy over GRPO for signal reasoning tasks. Our best model achieves an overall accuracy of 39.12\%, representing a more than threefold improvement over the untrained Base model (12.37\%).</p>]]></description>
    </item>
    <item>
      <title>The Road Less Traveled: Congestion-Aware NoC Placement and Packet Routing for FPGAs</title>
      <link>http://arxiv.org/abs/2608.17266v1</link>
      <guid isPermaLink="false">2608.17266</guid>
      <pubDate>Tue, 18 Aug 2026 01:51:14 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Soheil Gholami Shahrouz, Vaughn Betz</p><p>To help scale to ever-larger and more complex designs, recent FPGA architectures now integrate network-on-chips (NoCs). NoCs help transfer high-bandwidth data over long distances within the chip without using scarce low-delay long routing wire segments. While NoC-enhanced FPGAs aid system integration and design reuse, they also complicate FPGA computer-aided design (CAD) flows by introducing new constraints and metrics. Placement and routing need to optimize NoC metrics like latency and bandwidth utilization and avoid link oversubscription (congestion), while simultaneously optimizing the programmable routing resource usage of the design modules attached to NoC routers. In this work, we develop several new approaches to reduce NoC congestion while minimizing the impact on other design metrics. First, we incorporate a NoC link congestion cost into the placement engine of the open-source CAD flow, versatile place &amp; route (VPR). Second, we integrate turn model NoC routing algorithms into the placement engine to leverage path diversity to further reduce congestion. On average over a suite of 29 benchmarks, combining placement congestion modeling with turn model packet routing reduces NoC congestion by 90.7% at the cost of increasing aggregate bandwidth demand by 4%. In cases where the enhanced placement engine and NoC routing fail to fully resolve congestion, we formulate NoC routing as a Boolean satisfiability (SAT) problem. This approach yields significant additional improvements; the combined algorithm reduces congestion by 95.1% compared to the baseline placement. Finally, we enhance the reinforcement learning (RL) agent in VPR&#x27;s placement engine by introducing a NoC-aware move type, resulting in an 8.8% reduction in wirelength on designs that make extensive use of the NoC.</p>]]></description>
    </item>
    <item>
      <title>A Hybrid End-to-End and Modular Control Architecture Toward Safe Vehicle Lateral Control: Combining Soft Actor-Critic with Model Predictive Control</title>
      <link>http://arxiv.org/abs/2608.17258v1</link>
      <guid isPermaLink="false">2608.17258</guid>
      <pubDate>Tue, 18 Aug 2026 01:31:51 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Farzaneh Tatari</p><p>Connected and automated vehicles demand lateral controllers that are simultaneously accurate, low-effort, and safe under model error and sensor noise. Modular controllers such as model predictive control (MPC) are interpretable and constraint-aware but rely on accurate models and hand-tuned weights. End-to-end learned policies, in particular continuous-action deep reinforcement learning, are adaptable and require no hand-designed control law, but offer no intrinsic safety guarantees and limited interpretability. This paper presents a hybrid architecture that combines an end-to-end Soft Actor-Critic (SAC) policy with a constrained linear MPC into a single steering command, using the MPC&#x27;s first-step optimum as the model-based anchor and a single monotone blending coefficient that interpolates between the two paradigms. The architecture is evaluated on a linearized lateral bicycle model against a PID baseline, a tuned linear MPC, and a stand-alone SAC policy, across nominal, single-axis robustness, and multi-initial-condition ensemble experiments. The hybrid retains the tracking quality of stand-alone SAC while remaining inside the MPC&#x27;s actuator envelope and preserving a deterministic, model-based contribution to every steering command. The architecture provides an actuator-envelope guarantee by construction but does not establish recursive feasibility or terminal invariance, and the closed-form blend does not prevent all corner-case divergences at the boundary of the training distribution. A corner-case analysis shows that the blend attenuates but cannot prevent failure under distribution shift, motivating a connectivity-aware extension in which the blending coefficient is scheduled by vehicle-to-everything (V2X) signals to restore model-based authority. Limitations and a path toward a constrained-QP predictive safety filter are discussed.</p>]]></description>
    </item>
    <item>
      <title>Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL</title>
      <link>http://arxiv.org/abs/2608.17253v1</link>
      <guid isPermaLink="false">2608.17253</guid>
      <pubDate>Tue, 18 Aug 2026 01:16:02 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Yunhao Yang, Yuexin Bian, Yunjie Tian, Di Fu, Tianjin Huang, Yuanyuan Shi, Ziang Xiao, Nuno Vasconcelos, Yijiang Li</p><p>Reinforcement learning (RL) has emerged as a powerful approach for improving reasoning in language and vision-language models, yet its strongest successes still depend heavily on ground-truth supervision (e.g., verifiable reward). Such annotations are costly to obtain and become increasingly scarce as reasoning capabilities advance beyond what humans can reliably evaluate. Self-rewarding RL reduces this dependence by enabling models to derive reward signals from their own completions. However, training solely on self-generated feedback can reinforce existing biases and suboptimal behaviors, reduce response diversity, and ultimately lead to homogenized responses and training collapse. In this work, we show that unsupervised reasoning can emerge through cooperative multi-agent training. We introduce Co-RL, a framework in which multiple decoupled models, sharing no parameters, are simultaneously optimized through RL using rewards derived from their peers. We further show that increasing cohort diversity, through heterogeneous model families, sizes, and rephrased training samples, reduces the correlated errors that drive self-reinforcing feedback loops. This diversity consistently improves reasoning performance, maintains behavioral diversity, and mitigates training collapse. Across text-only and multimodal domains, Co-RL consistently outperforms the base models and prior label-free approaches, while matching or surpassing supervised methods, without access to any ground-truth labels. Concretely, Co-RL yields average gains of 3.0-8.6% across seven text-only benchmarks for LLMs and 2.3-7.2% across four multimodal benchmarks for VLMs. Code is available at https://github.com/DrStranded/Co-RL.</p>]]></description>
    </item>
    <item>
      <title>Safe Deep Reinforcement Learning for Energy-Efficient HVAC Control in Multi-Zone Residential Buildings</title>
      <link>http://arxiv.org/abs/2608.17235v1</link>
      <guid isPermaLink="false">2608.17235</guid>
      <pubDate>Tue, 18 Aug 2026 00:49:04 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Oussama Ziadi, Abdelilah Rochd, Samir Idrissi Kaitouni, Mohamed Oualid Mghazli, Adnane Saoud</p><p>HVAC systems represent a major share of building energy consumption. Traditional control strategies are limited in coordinating energy-comfort tradeoffs across multiple zones simultaneously. Reinforcement learning (RL) offers adaptive, data-driven control that optimizes performance over time. However, deploying learned neural network controllers in safety-critical building systems remains challenging due to lack of formal safety guarantees. We propose a safety-certified deep RL framework for multi-zone residential HVAC control. Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) agents are trained in an EnergyPlus/Sinergym simulation to minimize energy consumption while maintaining thermal comfort. Post-training safety certification is performed on the PPO policy using Lipschitz-based forward invariance analysis, building on existing tools for the computation of Lipschitz constants for neural networks, to guarantee constraint satisfaction. Both agents are evaluated over an annual simulation cycle in an eight-zone variable refrigerant flow (VRF) testbed. The PPO agent achieves 67\% comfort violation reduction compared to rule-based control, while the SAC agent achieves 27.6\% energy savings. The PPO policy satisfies formal safety certification with a margin of $2.003^\circ$C. These results demonstrate the feasibility of combining reinforcement learning with post-training safety verification for multi-zone building control.</p>]]></description>
    </item>
    <item>
      <title>An O-RAN-Assisted MARL Approach for Dynamic Sidelink and Infrastructure Selection in V2X Communications</title>
      <link>http://arxiv.org/abs/2608.17210v1</link>
      <guid isPermaLink="false">2608.17210</guid>
      <pubDate>Mon, 17 Aug 2026 23:46:39 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Maria Katarine Santana Barbosa, Kelvin Lopes Dias</p><p>Future applications in the 6G-based Internet of Vehicles will leverage sidelink (SL) transmissions in Vehicle-to-Everything (V2X) scenarios. However, SL-based direct communication can significantly increase interference among vehicles and between vehicles and other entities of the Intelligent Transportation System. Thus, both Vehicle-to-Vehicle communications and Vulnerable Road Users (VRUs) uplink resources may be degraded or subject to starvation. Existing solutions primarily focus on improving resource allocation and pair selection. Nonetheless, they lack a comprehensive approach to tackle the communication modes and the entire network. To address these challenges, this paper leverages Open RAN to manage V2X communication and proposes a multi-agent reinforcement learning (MARL) resource-aware system. Open RAN provides control loops through a global view of the network and also an open interface-based framework for machine learning models applied to resource decision-making. Meanwhile, the MARL model aims to mitigate interference, optimize resource usage, and enhance quality of service by optimally selecting between sidelink and network transmissions. To reduce system complexity, this work employs a clustering strategy. Each agent manages a group of pairs, rather than assigning one agent to each pair. The solution supports this design by adopting a centralized training with decentralized execution approach, empowered by Open RAN. The strategy uses offline training and an off-policy approach, in which each agent stores experience for fine-tuning. Results indicate that the MARL approach reduces average loss by 21% and latency by 19% in Vehicle-only scenarios. In coexistence VRU scenarios, loss and latency drop by 18% and 30%, respectively, compared to the single-agent approach.</p>]]></description>
    </item>
    <item>
      <title>Teach and Grow: An Agent-Centered Architecture for General Robot Learning</title>
      <link>http://arxiv.org/abs/2608.17209v1</link>
      <guid isPermaLink="false">2608.17209</guid>
      <pubDate>Mon, 17 Aug 2026 23:45:21 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Chang Nie, Zhe Liu, Hesheng Wang</p><p>End-to-end vision-language-action (VLA) and world-action models offer an elegant route to general-purpose robotics, but their reliability is bounded by validated physical coverage. When an unfamiliar object, sensor, embodiment, or contact falls outside that coverage and no validated fallback exists, correcting the failure requires new robot data, a policy update, and regression testing. This recurring burden is the retraining tax. Unlike text, embodied data must often be created by operating machines. We present Teach-and-Grow Learning (TGL), an agent-centered architecture for general robot learning. In its general form, a multimodal agent turns a few successful demonstrations into reusable Skill Blocks: closed-loop behaviors for meaningful subgoals. In a new scene, the agent grounds and composes these blocks, selects learned or geometric tools, observes the physical outcome, and revises the route when execution departs from intent. A Skill Library stores executable behavior, while structured Experience Memory carries forward success, failure, and repair. New tasks are acquired without task-specific policy retraining. Our LIBERO evaluation attains state-of-the-art performance; controlled studies expose skill induction, persistent reuse, and agent-directed adaptation. Finally, we propose the Teach-and-Grow scaling-law hypothesis: if X denotes effective reusable experience, future-task error and teaching demand should approach irreducible floors as power laws in X. The architecture therefore treats deployment as a period of continued learning, in which one task can make the next easier.</p>]]></description>
    </item>
    <item>
      <title>Reinforcement Learning as (Discrete) Potential Theory</title>
      <link>http://arxiv.org/abs/2608.17181v1</link>
      <guid isPermaLink="false">2608.17181</guid>
      <pubDate>Mon, 17 Aug 2026 22:43:51 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Christopher Connolly</p><p>Reinforcement learning (RL) theory fundamentally depends on probability theory through the Markov chain. There is a deep connection between probability theory and potential theory. This paper reviews that connection and explores the potential-theoretic viewpoint for core reinforcement learning representations and algorithms under a fixed-policy assumption. This viewpoint may offer a path for improved sample efficiency and formal constraints that can be applied to RL. When the fixed-policy assumption is relaxed, the linear potential theory framework can be naturally extended to the nonlinear case.</p>]]></description>
    </item>
    <item>
      <title>Task Specialization Fine-Tuning for Contextual Reinforcement Learning</title>
      <link>http://arxiv.org/abs/2608.17180v1</link>
      <guid isPermaLink="false">2608.17180</guid>
      <pubDate>Mon, 17 Aug 2026 22:42:01 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Jianan Zhou, Jung-Hoon Cho, Tianyue Zhou, Han Zheng, Jie Zhang, Roy Dong, Yining Ma, Cathy Wu</p><p>Contextual Reinforcement Learning (CRL) seeks to generalize classical RL by maximizing task coverage across a context space of related tasks. While prior works often train from scratch and rely on either multi-task learning for a single policy or strategically training multiple policies, we advocate for a unified alternative: pretraining a single policy with good initial performance, followed by fine-tuning multiple policies for task specialization. This new paradigm, however, introduces unique challenges, such as heterogeneous marginal returns and sample inefficiency. This raises a critical research question: given a pretrained policy and a constrained budget, how much fine-tuning should each task region receive to enable sample-efficient CRL? To this end, we propose Task Specialization Fine-Tuning (TSFT), an online framework that predicts fine-tuning performance with a simple parametric model and exactly solves the resulting discrete budget allocation problem via integer linear programming. Extensive experiments across diverse decision domains, including combinatorial optimization, continuous control, and LLM fine-tuning, demonstrate that TSFT significantly outperforms baselines in task coverage and approaches oracle performance. Our work charts a new direction for model-based CRL, aligning with the modern pretrain-finetune era.</p>]]></description>
    </item>
    <item>
      <title>Q-Learning With World Models</title>
      <link>http://arxiv.org/abs/2608.17163v1</link>
      <guid isPermaLink="false">2608.17163</guid>
      <pubDate>Mon, 17 Aug 2026 22:00:42 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Perry Dong, Yueru Jia, Chelsea Finn, Dorsa Sadigh</p><p>Off-policy reinforcement learning (RL) has become increasingly sample-efficient, enabling applications such as RL fine-tuning of Vision-Language-Action models into reliable, high-performing policies. World models offer a further lever for sample efficiency, as they predict state changes rather than actions alone, but their success has largely been confined to supervised policy learning. Prior model-based RL methods often optimize the policy or value function directly on imagined rollouts, which is prone to compounding bias and struggles to scale to large, high-dimensional problems such as real-world robotics, a problem that worsens with task horizon and visual complexity. In this work, we instead ask whether we can leverage world models directly on top of standard Q-learning to improve performance, while remaining trained and grounded in the real, online setting. We propose QWM, a framework that leverages world models to perform test-time search over imagined trajectories on top of Q-learning to select high-value actions during both online rollouts and evaluation. Since the policy and value function are trained only on real transitions, QWM avoids compounding model bias while still gaining the sample-efficiency benefits of predictive search. On challenging manipulation benchmarks Robomimic and LIBERO, QWM significantly outperforms strong prior state-of-the-art methods on both sample efficiency and performance.</p>]]></description>
    </item>
    <item>
      <title>PROBE: Manipulation-Grounded Visual Question Answering with VLM Agents</title>
      <link>http://arxiv.org/abs/2608.17129v1</link>
      <guid isPermaLink="false">2608.17129</guid>
      <pubDate>Mon, 17 Aug 2026 21:03:31 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Vineet Bhat, Siyi Chen, Alex Zook, Xuning Yang, Stan Birchfield, Valts Blukis, Jonathan Tremblay</p><p>Vision-language Models (VLMs) excel at 2D grounding, spatial reasoning and agentic tool-based planning in static scenes. However, consider asking a home robot &quot;Is my medication still in the cabinet?&quot; The answer may be physically hidden behind a row of containers that must first be moved aside. Answering such questions in real-world cluttered environments requires reasoning in dynamic scenes: distractors must be manipulated to reveal occluded objects, and each action changes the scene the model must reason over. We formalize this setting as Manipulation-Grounded Visual Question Answering (MG-VQA) and introduce PROBE, a framework for benchmarking and finetuning VLM agents on such tasks. We first develop PROBE-Sim, a high-fidelity tabletop simulator with everyday objects and a robot manipulator equipped with grasping and pushing tools. PROBE-Sim is used to create PROBE-Bench: an evaluation suite of 150 tasks across 6 question types on cluttered tabletop scenes, where a VLM perceives, picks up or pushes objects before answering. We observe consistent trend across all frontier VLMs: agentic tool-based methods outperform their perception-only baselines (8.0% on average) across all task types. We further design PROBE-Agent, a finetuning recipe to distill successful trajectories from a powerful teacher foundation model to a smaller open-weight model using a mixed data recipe that encourages manipulation-efficient question answering. PROBE Agent finetuned models outperform their off-the-shelf agent baseline (11.5% on average) and demonstrate positive transfer to unseen objects and a held-out task. We validate sim-to-real transfer by deploying PROBE-Agent finetuned policies in real-world tabletop environments.</p>]]></description>
    </item>
    <item>
      <title>Physics-informed Reinforcement Learning for Stochastic Reach-Avoid Analysis</title>
      <link>http://arxiv.org/abs/2608.17117v1</link>
      <guid isPermaLink="false">2608.17117</guid>
      <pubDate>Mon, 17 Aug 2026 20:50:33 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Hikaru Hoshino, Yorie Nakahira</p><p>Stochastic reach-avoid analysis of controlled dynamical systems is an important tool for safety-critical control under uncertainty, in which the reach-avoid probability is characterized by a Hamilton-Jacobi partial differential equation (PDE). However, solving this PDE using conventional numerical methods becomes computationally intractable as the system dimension increases. Physics-informed neural networks (PINNs) may converge to inaccurate local minima when trained primarily through PDE-residual minimization. Reinforcement learning (RL) offers a scalable alternative, but its learned value functions may be inaccurate or inconsistent with the governing PDE. This paper proposes a physics-informed RL (PIRL) framework that combines the complementary strengths of PINNs and RL for stochastic reach-avoid analysis. We develop a scheduled PIRL algorithm in which temporal-difference actor-critic learning first guides the critic toward a meaningful approximation of the reach-avoid value function. PDE-residual and boundary-condition losses are then introduced progressively to enforce consistency with the governing PDE and its boundary conditions. The proposed method mitigates the failure modes of conventional PINN techniques while achieving accuracy comparable to that of successfully trained PINNs. The effectiveness of the proposed framework is demonstrated through two case studies.</p>]]></description>
    </item>
    <item>
      <title>Inference-Time Attention Steering for Vision-Language-Action Driving Models</title>
      <link>http://arxiv.org/abs/2608.17095v1</link>
      <guid isPermaLink="false">2608.17095</guid>
      <pubDate>Mon, 17 Aug 2026 20:03:38 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Darshan Nagendra Prasad, Lars Ullrich, Knut Graichen</p><p>Vision-language-action (VLA) driving models couple a reasoning stage with a diffusion-based trajectory decoder, but do not give a direct way to redirect attention toward safety-critical actors at inference time without retraining. We studied a bounded additive pre-softmax attention bias on the visual tokens of detector localized traffic actors on Alpamayo-R1&#x27;s Qwen3-VL backbone. It is applied as a fail open forward pre-hook with no weight changes. On 50 lane-change scenarios from the Physical AI World Model Synthetic dataset. The trajectory decoder shows a monotonic dose response in the bias magnitude, separate from a paired zero bias control at every tested magnitude. It reaches $\approx 17$\,cm mean displacement with lateral shifts up to $\sim 140$\ cm at the clamp. A layer ablation places the action-relevant signal in late layers, where the effect increases with the number of hooked layers (2.0cm for the first 8 layers; 67.6cm for all 36). A per call injection audit explains why the Chain-of-Causation text never changes. The mask based bias never reaches the reasoning pathway in this serving stack, so the invariance is verified exposure, not robustness. Steered trajectories tend to shift toward the attended actor, suggesting the bias governs where the model looks rather than encoding a target behavior.</p>]]></description>
    </item>
    <item>
      <title>Lambda-Hold Control: Human-Like Movement Emerges from a Minimal Task Reward in Predictive Musculoskeletal Simulation</title>
      <link>http://arxiv.org/abs/2608.17030v1</link>
      <guid isPermaLink="false">2608.17030</guid>
      <pubDate>Mon, 17 Aug 2026 18:28:09 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Jun Hyuk Lee, Chihyeong Lee, Jooeun Ahn</p><p>The massive overactuation in the human musculoskeletal system makes it challenging to train musculoskeletal models to generate human-like motion via reinforcement learning, primarily because exploration in the resulting high-dimensional and redundant action space is extremely inefficient. To address this problem, we propose the $λ$-hold controller, inspired by the equilibrium-point (EP) hypothesis, which has been widely supported by extensive evidence from human motor control studies. The policy&#x27;s control variable is the per-muscle EP threshold length $λ$, from which a stretch-reflex recruitment law computes the muscle excitations automatically. Holding each $λ$ over an interval of the gait phase also sharply reduces the frequency at which the policy must be queried. Consequently, the controller, to our knowledge for the first time, enables a muscle-actuated skeletal model to learn human-like sprinting using only a minimal reward within an hour of training. The efficient exploration through the proposed $λ$-hold controller is not merely an engineering trick but an approach grounded in physiology, bringing together the EP hypothesis, intermittent control, and optimal feedback control. Beyond encapsulating human-like behavior in predictive simulation, this achievement contributes to developing a learnable model of the human motor controller.</p>]]></description>
    </item>
    <item>
      <title>FetchMan: Learning Visual Humanoid Loco-Manipulation Policies from Simulated Experiences</title>
      <link>http://arxiv.org/abs/2608.17027v1</link>
      <guid isPermaLink="false">2608.17027</guid>
      <pubDate>Mon, 17 Aug 2026 18:24:41 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Omar Rayyan, Zhi Li, Max Argus, Yuxin Jiang, Chang Yu, Chenfanfu Jiang, Yuchen Cui</p><p>Visual loco-manipulation policies that can generalize to novel scenes and objects have long been a goal of robotics research. However, today&#x27;s data-hungry algorithms make collecting sufficient demonstrations a struggle for tabletop manipulation, and even more so for humanoids that must also walk and balance. Learning from simulated data and transferring that behavior to the real world, as is commonly done in locomotion, sidesteps this struggle, so we replicate that recipe for loco-manipulation. In doing so, we find that cloning synthetic demonstrations results in a low performance ceiling no matter the amount of training data. Reinforcement learning breaks through it, and refining the cloned policy with Flow-GRPO on a single sparse reward yields performance that synthetic behavior cloning cannot match. Together, these stages form our end-to-end sim-to-real pipeline spanning more than 150,000 scenes, which we use to train FetchMan. We evaluate it on FetchMan-Bench, a simulation benchmark we release, and deploy it zero-shot on a real Unitree G1, where our single-object reach-and-pick policy walks to and grasps a target across unseen scenes at 73.3% success. Finally, we extend this recipe to multi-object training, a first step toward loco-manipulation generalist policies at this data scale.</p>]]></description>
    </item>
    <item>
      <title>Q-based Variational Inverse Reinforcement Learning</title>
      <link>http://arxiv.org/abs/2608.16888v1</link>
      <guid isPermaLink="false">2608.16888</guid>
      <pubDate>Mon, 17 Aug 2026 17:59:55 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Ondrej Bajgar, Peter Tisnikar, Alessandro Abate, Konstantinos Gatsis, Maike Osborne</p><p>The development of safe and beneficial AI requires that systems can learn and act in accordance with human preferences. However, explicitly specifying these preferences by hand is often infeasible. Inverse reinforcement learning (IRL) addresses this challenge by inferring preferences, represented as reward functions, from expert behaviour. We introduce Q-based Variational IRL (QVIRL), a novel Bayesian IRL method that recovers a posterior distribution over rewards from expert demonstrations via primarily learning a variational distribution over optimal Q-values. Unlike previous approaches, QVIRL combines scalability with uncertainty quantification, important for safety-critical applications as well as active learning. We demonstrate QVIRL&#x27;s strong performance in apprenticeship learning across various tasks, including gridworlds, Lunar Lander, the Highway Environment, and two ATARI games both with static expert data and with active learning. It is the first method for Bayesian IRL that demonstrates training from raw pixel observations.</p>]]></description>
    </item>
    <item>
      <title>$τ_0$-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation</title>
      <link>http://arxiv.org/abs/2608.16885v1</link>
      <guid isPermaLink="false">2608.16885</guid>
      <pubDate>Mon, 17 Aug 2026 17:59:11 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Xiaowei Cai, Yunuo Cai, Bingao Chen, Jingxiao Chen, Zhi Chen, Siyuan Feng, Tengyu Hou, Jingshun Huang, Han Jiang, Runkun Ju, Dong Li, Mingxiang Li, Shaowei Li, Xinchen Li, Yifan Li, Yi Liu, Zhongyuan Liu, Jianlan Luo, Junwen Miao, Ruiqi Ni, Buqing Nie, Mingjie Pan, Xinlin Ren, Jianheng Song, Jiaxu Wang, Peiqi Wang, Sen Wang, Xiaoyan Wang, Dafeng Wei, Dongming Wu, Pengwei Xie, Pu Yang, Hangjian Ye, Xiangyu Yue, Jinyu Zhang, Qinglin Zhang, Xueyong Zhao, Pengfei Zhou, Yue Zhou</p><p>Long-horizon robot manipulation requires a robot to both execute individual skills reliably and sequence them coherently over extended tasks. Most hierarchical vision-language-action (VLA) models make each such decision with a single forward pass, leaving no mechanism to allocate additional computation to difficult or consequential choices. We introduce $τ_0$-VLA, a hierarchical robot foundation model that formulates high-level subtask generation as a compute-scalable inference problem through world-model-guided test-time computation. At each inference step, the high-level policy uses execution memory to generate a subtask and, when needed, searches over alternatives before committing to its output. A low-level policy then executes the generated subtask across multiple robot embodiments. The policy is trained on 40,115 hours of heterogeneous real-world data with multimodal co-training. Across in-domain and distribution-shifted settings, allocating additional test-time computation substantially improves next-subtask prediction accuracy, and these gains translate into higher closed-loop success on long-horizon robot manipulation tasks.</p>]]></description>
    </item>
    <item>
      <title>HarnessEval-W: Agentifying the Evaluation of Visual Worlds</title>
      <link>http://arxiv.org/abs/2608.16859v1</link>
      <guid isPermaLink="false">2608.16859</guid>
      <pubDate>Mon, 17 Aug 2026 17:43:24 +0000</pubDate>
      <category>arXiv</category>
      <description><![CDATA[<p><strong>Source:</strong> arXiv</p><p><strong>Authors:</strong> Weiliang Chen, Haowen Sun, Jun Gao, Jiawei Chi, Hanyang Wang, Qiyu Dai, Yihao Li, Hao Li, Jingnan Gao, Yi-Hsin Hung, Xingzhuo Guo, Shangchen Miao, Zhiyuan Shi, Xiang Li, Fengrui Tian, Weihua Du, Ziqi Huang, Shenyuan Gao, Siqiao Huang, Mingyu Liu, Yifei Li, Shizun Wang, Xi Wang, Tianqi Zhang, Xue Luo, Xiyin Ren, Jinshan Ren, Xiaoyang Shen, Xiaobo Hu, Zhiyang Dou, Mingyu Ding, Yichao Yan, Xinchao Wang, Yizhou Wang, Shilong Liu, Wenzhao Zheng, Yueqi Duan, Yuan Gong, Ziwei Liu, Ming-Yu Liu, Jialong Wu, Jiangran Lyu, Fangfu Liu</p><p>A benchmark should deliver more than a scalar score: what makes an evaluation trustworthy is the reasoning that justifies the score. This is especially critical for world models, where judging a rollout requires understanding whether physics, causality, and world state evolve correctly. Humans spot such violations naturally, yet no existing benchmark automates this capability: metrics are computed brute-force, leaving no reasoning chain that can be examined or verified. We introduce HarnessEval-W, an agentified evaluation pipeline that brings the harness paradigm from the LLM ecosystem to world model benchmarking. Rather than applying a fixed rubric, HarnessEval-W interprets the context of each evaluation case, decomposes the evaluation question into measurable subproblems, and spawns specialized sub-agents, each equipped with tailored context and diagnostic tools to reason over its own subproblem. The parent agent then validates the gathered evidence and summarizes it into the final verdict. This hierarchical workflow turns every evaluation into a transparent evidence tree whose complete reasoning chain justifies the result. We apply HarnessEval-W to 18 representative world models over 330 evaluation cases. Its judgments closely align with human preferences while providing verifiable, fine-grained diagnoses of every generated rollout. We open-source the full pipeline as a live benchmark and invite the broad community to contribute to grow new skills and evaluation cases as world models evolve.</p>]]></description>
    </item>
    <item>
      <title>StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understanding</title>
      <link>https://arxiv.org/abs/2608.16320</link>
      <guid isPermaLink="false">2608.16320</guid>
      <pubDate>Sun, 16 Aug 2026 20:00:00 +0000</pubDate>
      <category>Hugging Face Daily Papers</category>
      <description><![CDATA[<p><strong>Source:</strong> Hugging Face Daily Papers</p><p><strong>Authors:</strong> Keming Wu, Baoyi Wang, Kaichen Zhang, Xiang An, Zuhao Yang, Sudong Wang, Haowei Zhu, Tingxuan Huang, Hongcheng Gao, Bin Wang</p><p>Streaming video understanding demands direct responses from the causally observed prefix of an unfolding video. Existing systems add inference-time memory, retrieval, and compression, yet a training-free sliding-window baseline already matches them. We therefore fix a memory-free recent-window protocol and ask how far post-training alone can go. Reinforcement learning with verifiable rewards fits this regime poorly, encouraging long ``think-then-answer&#x27;&#x27; generations, while on-policy distillation (OPD) supplies dense token-level teacher supervision on student trajectories but is stable only when both models train in thinking mode. These observations lead to StreamOPD, a recipe combining verifiable streaming-video data, thinking-mode OPD, and instruct-mode deployment. It raises StreamingBench from 77.9% to 83.9%---within 0.3 points of the 9B teacher---and improves OVO-Bench excluding its hallucination-detection subtask (HLD) by 9.1 points under unchanged inference. As a teacher-privilege extension, Spatio-Temporal CueGate (ST-CueGate) aggregates cue-versus-no-cue teacher likelihood ratios into a group-relative response score that reweights OPD. It reaches 71.9% on OVO-Bench (excluding HLD) and 64.9% on Video-MME, and is the only variant that stays above the base model on all four benchmarks. Replacing the teacher with a frozen copy of the student&#x27;s initial policy---on-policy self-distillation---retains most of these gains and lifts HLD to 57.0%, above both the untrained student and the 9B teacher, so abstention loss is not intrinsic to the recipe. We provide a transparent and reproducible reference for open-source streaming-video research.</p>]]></description>
    </item>
    <item>
      <title>Learn What&#x27;s Left, Not What&#x27;s Mastered: Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization</title>
      <link>https://arxiv.org/abs/2608.16072</link>
      <guid isPermaLink="false">2608.16072</guid>
      <pubDate>Sun, 16 Aug 2026 20:00:00 +0000</pubDate>
      <category>Hugging Face Daily Papers</category>
      <description><![CDATA[<p><strong>Source:</strong> Hugging Face Daily Papers</p><p><strong>Authors:</strong> Yixuan Wang, Yifei Chen, Haichao Zhang, Haozheng Luo, Xander Wu, Jie Ni, Yun Fu, Nuno Vasconcelos, Yijiang Li</p><p>Reinforcement learning (RL) with group-relative advantages has become the de facto standard for post-training language model reasoners. However, when optimizing multiple reward objectives, existing methods typically scalarize the reward vector with a fixed weighted sum before group-wise standardization. We show that this design leads to two fundamental problems: rollouts with distinct reward profiles can receive identical advantages, and all objectives are optimized with fixed relative weights regardless of their current level of saturation. As a result, training continues to allocate gradient budget to already-solved objectives instead of focusing on those with greater remaining headroom. We introduce Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization (SA-MRPO), which standardizes each reward objective independently and adaptively discounts its contribution according to a batch-level estimate of objective saturation. This dynamically reallocates optimization effort toward under-optimized objectives while empirically maintaining performance on those that are already well satisfied. We further show that saturation-aware reweighting can reverse the sign of an update, rather than merely rescale its magnitude. Across mathematical reasoning with two- and three-objective reward combinations, SA-MRPO improves the harder correctness objective over GDPO in 12 of 15 benchmark comparisons, with gains of up to 5% on AIME24. On adaptive reasoning it improves accuracy on all five benchmarks, by 3.8% on average and up to 9.2 % on AMC23, and on coding benchmarks it improves pass rate by up to 2.3%, while in all settings maintaining the easier objectives near their already satisfied levels.</p>]]></description>
    </item>
    <item>
      <title>Drive, Pack, Fly: The Travelling Thief Problem with Drone</title>
      <link>https://arxiv.org/abs/2608.16435</link>
      <guid isPermaLink="false">2608.16435</guid>
      <pubDate>Sun, 16 Aug 2026 20:00:00 +0000</pubDate>
      <category>Hugging Face Daily Papers</category>
      <description><![CDATA[<p><strong>Source:</strong> Hugging Face Daily Papers</p><p><strong>Authors:</strong> Kabir Murjani, Abhay Sobhanan</p><p>In collection operations, accumulating payload progressively slows the vehicle, imposing a cumulative penalty on routing efficiency. An onboard drone can offset this penalty by retrieving outlying items, thereby shortening the makespan and increasing operational profit. However, travel time remains load-dependent, and each item collected by the ground vehicle shifts the arrival times that govern the drone&#x27;s launch and rendezvous points. This paper introduces the Travelling Thief Problem with Drone (TTP-D), which maximises the collected profit, net of a time-based rental cost, by jointly optimising item selection, vehicle routing, and flight synchronisation. We formulate a mixed-integer linear program that solves small instances to optimality, and develop both metaheuristics and an attention-based Deep Reinforcement Learning (DRL) policy for larger instances. We further propose a learner-initialised hybrid solver, in which the DRL policy constructs an initial solution that a short annealing run subsequently refines. On two benchmark sets, this hybrid recovers most of the metaheuristic baseline&#x27;s quality at a fraction of its computational budget, although the largest instances still require the baseline at its full budget. Finally, a sensitivity analysis reveals that the rental ratio is the primary driver of profitability, whereas the fleet parameters affect profit only at the margin.</p>]]></description>
    </item>
    <item>
      <title>ClawGym II: Exploring Black-Box RL on Agent Harness</title>
      <link>https://arxiv.org/abs/2608.16798</link>
      <guid isPermaLink="false">2608.16798</guid>
      <pubDate>Sun, 16 Aug 2026 20:00:00 +0000</pubDate>
      <category>Hugging Face Daily Papers</category>
      <description><![CDATA[<p><strong>Source:</strong> Hugging Face Daily Papers</p><p><strong>Authors:</strong> Huatong Song, Fei Bai, Ming Yang, Renyuan Li, Jia Deng, Jujie He, Zhange Zhang, Daixuan Cheng, Yan Xing, Qi Yun, Xuxing Chen, Danyang Li, Feng Chang, Chuan Hao, Ran Tao, Jian Yang, Bryan Dai, Wayne Xin Zhao, Mingjie Tang, Ji-Rong Wen</p><p>Agent harnesses have substantially improved performance on long-horizon tasks by coordinating agent interactions with the environment. However, reinforcement learning through complex harnesses remains largely unexplored, as scaling such training to long-horizon agent tasks introduces fundamental challenges. In this work, we present a unified black-box RL framework for stable and scalable optimization of general agents through complex harnesses. Concretely, we first build a sandbox-based execution infrastructure that isolates task environments and harnesses within temporary sandboxes for large-scale concurrent rollouts. We then decouple policy optimization from opaque harness execution and place a serving proxy at the model boundary to capture model calls. To reconstruct multi-turn trajectories and improve training efficiency, we organize the captured calls into prefix trees and further adapt both critic-based PPO and critic-free GRPO to optimize over the recovered tree structure. Meanwhile, we maintain training-inference consistency throughout the optimization process. Finally, we introduce mix-harness training, allowing a single model to be jointly optimized by heterogeneous harnesses. With Qwen3-30A3B, black-box RL improves Pass@1 on ClawGym-Bench by 9.98 and 14.81 points through OpenClaw and Claude Code, respectively, while remaining stable over 200-400 optimization steps. Moreover, the framework yields consistent gains on more challenging tasks such as JobBench and OfficeQA. Overall, our framework enables effective, stable, and scalable optimization of general agents through black-box harnesses, supporting unified training across heterogeneous execution systems.</p>]]></description>
    </item>
    <item>
      <title>Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning</title>
      <link>https://arxiv.org/abs/2608.15869</link>
      <guid isPermaLink="false">2608.15869</guid>
      <pubDate>Sat, 15 Aug 2026 20:00:00 +0000</pubDate>
      <category>Hugging Face Daily Papers</category>
      <description><![CDATA[<p><strong>Source:</strong> Hugging Face Daily Papers</p><p><strong>Authors:</strong> Xiaoyu Zhu, Xinke Deng, Suresh Taddewadikar, Arnab Kumar Mondal, Zhongyu Jiang, Ian Fasel, Joerg Liebelt</p><p>Multimodal large language models increasingly use visual chain-of-thought (Visual CoT) to reason about spatial, temporal, and embodied environments. By generating intermediate reasoning images, Visual CoT provides an intuitive mechanism for visual foresight but introduces substantial inference overhead, which is particularly problematic for proactive video reasoning. We ask whether models can learn to think visually during training while reasoning directly at inference. We introduce Internalized Visual Thinking (IVT), a post-training framework that jointly optimizes textual prediction and next-embedding prediction over unlabeled videos. Given a partially observed video, IVT predicts latent representations of future frames together with the target textual answer, encouraging the model to capture motion, object transitions, interactions, and latent intent. At inference, IVT generates the answer directly without synthesizing or re-encoding future frames. We conduct controlled studies across target representations, decoder designs, prediction horizons, data mixtures, training curricula, and predictive objectives. IVT improves over direct-answer fine-tuning on all six evaluation settings while retaining the same inference pathway. Compared with explicit Visual CoT, IVT achieves comparable or better performance and reduces average end-to-end latency by more than 5x. Together, our findings suggest that explicit pixel-space generation at inference time, as used in visual chain-of-thought, may not be necessary for effective proactive video reasoning. Predictive world modeling can be internalized during training to produce multimodal reasoners that are both more accurate and substantially more efficient.</p>]]></description>
    </item>
    <item>
      <title>MathForm: Scaling Mathematical Autoformalization with Knowledge Retrieval and Verification-Guided Refinement</title>
      <link>https://arxiv.org/abs/2608.14221</link>
      <guid isPermaLink="false">2608.14221</guid>
      <pubDate>Fri, 14 Aug 2026 07:51:12 +0000</pubDate>
      <category>Hugging Face Daily Papers</category>
      <description><![CDATA[<p><strong>Source:</strong> Hugging Face Daily Papers</p><p><strong>Authors:</strong> Lushi Pu, Weiming Zhang, Xinheng Xie, Zixuan Fu, Bingxiang He, Hengyu Zhao, Hongya Lyu, Xin Li, Jie Zhou, Yudong Wang</p><p>Autoformalization is commonly framed as translating natural-language mathematical statements into machine-verifiable formal languages such as Lean 4. However, faithful formalization requires more than translation. Models must map mathematical concepts to the complex hierarchy of types and definitions in formal libraries such as Mathlib, while ensuring that generated statements preserve the meaning of the source propositions. Existing approaches struggle because they rely heavily on the model&#x27;s parametric memory for library-specific knowledge, while common data construction pipelines often resort to filtering single-pass outputs and lack mechanisms for feedback-driven revision. To address these challenges, we introduce MathForm, an autoformalization framework for constructing verified training data through Mathlib knowledge retrieval and verification-guided iterative refinement. Before generation, a retrieval planner gathers relevant definitions and existing formalizations from Mathlib to guide the formalization generator. Generated statements are then revised using compiler diagnostics and semantic-consistency feedback. Using this framework, we construct FormalVerse, a Lean 4 dataset containing approximately 367K verified examples across diverse mathematical domains and sources. We then train MathForm-8B through supervised fine-tuning followed by reinforcement learning. Across six benchmarks, MathForm-8B achieves average Pass@8 rates of 88.06% under Syntax Check (SC) and 72.37% under Consistency Check (CC), outperforming multiple specialized 32B autoformalizers. On the challenging FATE-H and FATE-X subsets, it attains CC pass rates of 63% and 37%, exceeding the strongest specialized baselines in both cases.</p>]]></description>
    </item>
    <item>
      <title>HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark</title>
      <link>https://arxiv.org/abs/2608.13555</link>
      <guid isPermaLink="false">2608.13555</guid>
      <pubDate>Wed, 12 Aug 2026 20:00:00 +0000</pubDate>
      <category>Hugging Face Daily Papers</category>
      <description><![CDATA[<p><strong>Source:</strong> Hugging Face Daily Papers</p><p><strong>Authors:</strong> Dairu Liu, Zekun Qi, Jiayu Zeng, Ruixi Yu, Yu Guan, Yintianrun Zhang, Xuchuan Chen, Sikai Liang, Zekai Li, Chenghuai Lin, Xinqiang Yu, Wenyao Zhang, He Wang, Li Yi</p><p>Humanoid motion tracking is central to teleoperation and whole-body imitation, yet evaluation often disagrees with what people perceive in videos. Kinematic errors average per-frame pose differences but miss the physical artifacts that matter most, particularly unstable support and incorrect contacts such as foot skating and mistimed touch-downs. Meanwhile, widely used test suites are small and lack the diversity needed to stress contact-rich, long-horizon behaviors. We introduce HumanTracker to make humanoid tracking evaluation both perceptually aligned and scalable. The HumanTracker benchmark contains approximately 153 hours of optical motion trajectories from multiple professional performers, organized into four motion families with text labels for fine-grained diagnosis. We further propose HumanScore, a preference-aligned metric trained on 12K motion pairs containing 24K motions. Across representative state-of-the-art trackers, HumanScore better predicts human preferences and reveals contact and stability failures that kinematic metrics often miss.</p>]]></description>
    </item>
    <item>
      <title>Verifier-Induced Support Reshaping in On-Policy Optimization</title>
      <link>https://arxiv.org/abs/2608.00220</link>
      <guid isPermaLink="false">2608.00220</guid>
      <pubDate>Thu, 30 Jul 2026 20:00:00 +0000</pubDate>
      <category>Hugging Face Daily Papers</category>
      <description><![CDATA[<p><strong>Source:</strong> Hugging Face Daily Papers</p><p><strong>Authors:</strong> Shaohang Wei, Zikun Su, Feifan Song, Wen Luo, Wei Li, Guangyue Peng, Houfeng Wang</p><p>We show that on-policy reinforcement learning with verifiable rewards (RLVR) can improve the current objective while making successful behaviors for later objectives too rare to sample and reinforce. We call this verifier-induced support reshaping and define effective rewardable support as successful trajectories reachable within a fixed rollout budget. Across two model families, we study this effect through repeated verifier-scored sampling and bidirectional training on mathematical reasoning and constrained instruction following, including sequential training with the opposite verifier. Math-RLVR raises average instruction-following success but reduces the number of prompts with any successful response under repeated sampling. On IFEval with Qwen3-8B-Base, pass@1 rises by 6.5 percentage points while best@32 falls by 9.8 percentage points, and the same divergence appears across both models and IF benchmarks. Conversely, IF-RLVR shifts math responses from step-by-step openings toward direct answers, lowers best@k across sampling budgets, and reduces reward variation for later Math-RLVR. Token-distribution analyses and controlled opening interventions show that these changes concentrate in the first few response tokens. RLVR mainly reranks openings already available in the base policy, and the selected opening causally affects math searchability. The tested reference-policy constraints, routing priors, and on-policy distillation preserve cross-task support only partially; MathIF and ReasonIF show that marginal gains translate only partly into responses that are both correct and constraint-following. Therefore, endpoint improvements do not guarantee future trainability or joint capability under on-policy optimization. Code is available at https://github.com/sylvain-wei/verifier-induced-support-reshaping</p>]]></description>
    </item>
  </channel>
</rss>