Qwen3-VL Technical Report

Core Contributors

• Qwen3-VL is the most powerful vision-language model in the Qwen series to date.
• I focus on enhanced multimodal reasoning: Excels in STEM/Math—causal analysis and logical, evidence-based answers.
• Specifically, I lead the innovation of data synthesis pipelines and training methodologies to systematically enhance perceptual and reasoning capabilities across all stages—from continual pretraining and supervised fine-tuning to reinforcement fine-tuning and reinforcement learning.

From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning

Ruilin Luo*, Chufan Shi*, Yizhen Zhang*, Cheng Yang, Songtao Jiang, Tongkun Guan, Ruizhe Chen, Ruihang Chu, Peng Wang, Mingkun Yang, Lei Wang, Yujiu Yang, Junyang Lin, Zhibo Yang

• We propose the VAS attention metric and find that the reasoning performance of Multimodal Large Reasoning Models (MLRM) is highly correlated with VAS.
• We introduce the AVAR training pipeline, which enhances the effectiveness of multimodal reasoning data training by scaling VAS through three stages: data construction, cold-start initialization, and reinforcement learning.
• Our model achieves state-of-the-art results on reasoning and perception benchmarks at the 7B scale.

Unlocking Multimodal Mathematical Reasoning via Process Reward Model

Ruilin Luo*, Zhuofan Zheng*, Yifan Wang, Xinzhe Ni, Zicheng Lin, Songtao Jiang, Yiyao Yu, Chufan Shi, Ruihang Chu, Lei Wang, Jin zeng, Yujiu Yang

[Paper] | [Code] | [HuggingFace]

• We are the first to propose leveraging a process reward model (PRM) to provide process-level optimization in multimodal mathematical reasoning.
• We introduce a training framework addressing three key stages: reasoning data scarcity, reward data scaling, and online PRM-integrated reinforcement learning (RL).
• Our data is used to train Seed1.5-VL, and we open-source URSA-8B-PS-GRPO, a model whose reasoning capability matches that of the InternVL3 series.

PTD-SQL: Partitioning and Targeted Drilling with LLMs in Text-to-SQL

Ruilin Luo, Liyuan Wang, Binghuai Lin, Zicheng Lin, Yujiu Yang

[Paper] | [Code]

• We propose a code agent framework for the text-to-SQL task that leverages structured grammars for question-type classification, followed by question-type-specific automated question bank construction and a two-layer few-shot retrieval mechanism.
• We observe that stronger models achieve greater improvements on harder question types when provided with better-matched few-shot chain-of-thought (CoT) prompts, reflecting a learning pattern aligned with human cognition.
• Our method achieves state-of-the-art (SOTA) performance on benchmarks such as Spider and BIRD.

Fair Text-Attributed Graph Representation Learning

Ruilin Luo, Tianle Gu, Lin Wang, Yunfeng Zhou, Lei Wang, Yujiu Yang

[Paper]

• We are the first to identify the bias amplification effect between language model embeddings and GNNs in TAG representation learning.
• We propose solutions from the perspectives of fine-tuning and offline reinforcement learning.
• We provide a theoretical foundation for both the amplification of unfairness and our proposed mitigation approaches.

Prior Bilinear Based Models for Knowledge Graph Completion

Jiayi Li*, Ruilin Luo*, Jiaqi Sun, Jing Xiao, Yujiu Yang

[Paper]

• We propose the principle of identity in knowledge graph completion (KGC).
• We introduce a bilinear KGC method that explicitly models the law of identity and outperforms classical approaches such as RESCAL and ComplEx.
• We provide a theoretical derivation for modeling the law of identity.

Prior Relational Schema Assists Effective Contrastive Learning for Inductive Knowledge Graph Completion

Ruilin Luo, Jiayi Li, Jianghangfan Zhang, Jing Xiao, Yujiu Yang

[Paper] | [Code]