Kedi Chen
Publications
ReNIO: Reweighting Negative Trajectory Importance for LLM On-Policy Distillation
On-policy distillation (OPD) improves LLM reasoning by training a student model on its own generated outputs, but standard OPD treats all student-generated outputs (SGOs) equally regardless of their informativeness. We observe a consistent asymmetry in controlled filtering experiments: in both OPD and on-policy self distillation (OPSD), training only on incorrect SGOs outperforms training only on correct ones. Our further analysis suggests that models trained on correct-only SGOs tend to generate shorter reasoning traces and show weaker reflection behavior, while incorrect SGOs better preserve exploratory reasoning near the model's capability boundary. To exploit this signal without requiring full answer-containing rollouts, we introduce ReNIO, which Reweights Negative trajectory Importance for LLM On-policy distillation. By using the student-to-teacher probability ratio, ReNIO identifies pivotal tokens leading to wrong reasoning traces and aggregates their information into a normalized sample weight, inherently assigning larger weights to likely negative trajectories without observing the correctness of final-answer. Since Re-NIO only uses prefix-conditioned token probabilities, it preserves OPD's prefix training advantage over full-rollout reinforcement learning. Across both mathematical reasoning and code generation tasks, ReNIO improves both OPD and OPSD, with representative relative gains of up to 8.90% for Qwen3-1.7B and 10.00% for R1-Distill-Qwen-7B on mathematical reasoning benchmarks. Code repo: https://github.com/BDML-lab/ReNIO.
Dynamic Multimodal Activation Steering for Hallucination Mitigation in Large Vision-Language Models
Large Vision-Language Models (LVLMs) exhibit outstanding performance on vision-language tasks but struggle with hallucination problems. Through in-depth analysis of LVLM activation patterns, we reveal two key findings: 1) truthfulness and visual perception capabilities predominantly engage different subsets of attention heads within the model architecture; and 2) truthfulness steering vectors vary significantly across different semantic contexts. Based on these observations, we propose Dynamic Multimodal Activation Steering, a training-free approach for hallucination mitigation. Our method constructs a semantic-based truthfulness steering vector database and computes visual perception steering vectors, enabling context-aware interventions during inference by dynamically selecting the most relevant steering vectors based on input semantic similarity and applying them to the most influential attention heads. We conduct comprehensive experiments across multiple models and datasets, demonstrating that our approach significantly enhances model performance, outperforming existing state-of-the-art methods.
InternAgentHarness: A Scalable Synthetic Environment for Enhancing LLM Agentic Abilities
Large language models (LLMs) are increasingly expected to act as generalist agents capable of solving complex real-world problems. Training such agents, however, requires stable and diverse environments that support repeated interaction with stateful, tool-augmented tasks and provide verifiable feedback. Despite recent progress, the development of robust LLM agents remains limited by the lack of realistic, scalable, and executable training environments. We present InternAgentHarness, a scalable synthetic environment for improving the agentic capabilities of LLMs. Built upon the InternBootcamp training framework~\citep{internbootcampv1}, InternAgentHarness instantiates executable agent environments through a four-layer interface that unifies prompt generation, tool execution, interaction control, and reward computation. We further introduce \bootcampcli, an agent harness that automatically converts diverse agentic tasks into a Bootcamp-trainable paradigm. Unlike static benchmarks, InternAgentHarnessmakes evaluation actionable: observed failures can be systematically converted into new synthetic tasks, filtered trajectories, reinforcement learning rollouts, and subsequent re-evaluation under the same executable interface. We instantiate InternAgentHarness on a suite of 10 tasks covering both text-only and vision-based agent scenarios. Starting from Qwen3-VL-30B-A3B-Thinking, both supervised fine-tuning (SFT) and reinforcement learning (RL) on InternAgentHarness substantially improve performance over untuned model. These results suggest that InternAgentHarness provides a practical foundation for scalable synthetic agent environments and enables the continuous improvement of LLM agents through an iterative cycle of evaluation, synthesis, training, and refinement.