Qwen-Planner-Agent
A Closed-Loop AI-for-AI Framework for Real-World Mobile Planner Agents
MAI Team, Alibaba Token Hub, Alibaba GroupMobilePA-BenchScores (%) · Higher is better
Qwen-Planner-Agent 27B77.05% Overall · $2.41 / 1K tasks
Examples
Six tasks, from user request to confirmed result.
User request
Agent EnvironmentReadyPaused for reading
Recorded replay · Illustrative device UI
Original trajectory Full source & interaction history ↓
Methodology
Qwen-Planner-Agent couples a trained Planner Model with a unified Harness for tools, Skills, Memory, and execution feedback. Its AI-for-AI framework connects task construction, competence-adaptive learning, and evidence-driven Harness refinement.
Figure 2. An AI-for-AI lifecycle linking data generation, competence-adaptive learning, and Model–Harness refinement through execution feedback.
01
AI for Data
An agent-driven data flywheel turns capability gaps into executable training tasks.
- Executable task constructionAgents define goals, tools, initial states, and completion criteria to exercise planning, state tracking, and recovery.
- Evidence-preserving curationVerified demonstrations support the planning cold start; resettable tasks support online learning, with actions, feedback, and outcomes retained.
- Feedback-directed coverageFailure diagnosis guides targeted task generation and sampling adjustments, connecting each reviewed data release to specific capability gaps.
02
AI for Training
CARE adapts learning to model competence, balancing task completion with execution efficiency.
- Planning-oriented learningCold-start training masks erroneous actions and retains verified recovery; online learning combines sandboxes, LLM simulation, and selected real-device sessions.
- Competence-adaptive rewardsGroup success rates shift emphasis from progress to completion and efficiency, with an LLM controller configuring predefined reward schedules.
- Quality-preserving calibrationA success-derived normalization floor limits the influence of small efficiency differences, keeping task completion central as execution becomes more economical.
03
AI for Harness
Runtime context adapts to each request; execution evidence guides model and Harness refinement.
- Tool-conditioned SkillsThe Scenario Adapter matches guidance to available tools, preserving action dependencies, operating procedures, and multi-turn requirements.
- Evidence-grounded MemoryAI-assisted consolidation retains source evidence, checks provenance and conflicts, and explicitly supersedes outdated facts.
- Model–Harness co-evolutionHeld-out feedback guides alternating policy updates and LLM-assisted instruction revisions, shaping the context and trajectories for subsequent training.
Execution Evidence Connects the Loop
Execution evidence guides reviewed, versioned updates to data, model, and Harness. During serving, the Harness adapts context while model parameters remain fixed.
Benchmark Details
Mobile planning, long-history memory, and performance beyond mobile tasks.
Capability Comparison
Qwen-Planner-Agent 27B vs. Qwen Baseline 27B
Agent Baseline0–100%
Complete MobilePA-Bench Results
Qwen-Planner-Agent includes the deployment Harness. Overall follows the benchmark’s aggregation protocol.
Persistent Memory
Long-History Memory
The memory Harness retrieves relevant evidence from histories that exceed the active context. The comparison below uses the same planner checkpoint with two ways of accessing that history.
| Benchmark | Direct contextQwen-Planner-Model | Memory HarnessQwen-Planner-Agent | Changepercentage points |
|---|
Direct context uses a 256K-token budget, truncating the earliest history when needed. The Harness processes the full history and retrieves evidence for each question. LoCoMo and LongMemEval report LLM-judge accuracy; BEAM reports average rubric score. All scores are percentages.
Complete Memory Results
† Mem0 (Official) scores are externally reported reference results. — indicates an unreported score. History-access regimes differ in input-token and processing costs.
Planner Generalization
Beyond Mobile Planning
Qwen-Planner-Model improves mean performance across seven non-mobile agentic benchmarks, while general knowledge and instruction-following scores remain close to the corresponding Qwen baselines.
Qwen-Planner-Model is the trained planner evaluated without the deployment Harness. Each Mean is the unweighted arithmetic average within its benchmark group. Individual benchmark scores follow the report’s general agentic capability evaluation.
Citation
The official citation will be available when the report is released.
% Qwen-Planner-Agent technical report % Official BibTeX will be added upon release.