Mobile · Computer · Web
MAI-Gym
One open, all-in-one platform to evaluate and train GUI agents.
- 78.4%
- MobileWorld
- 76.2%
- OSWorld-Verified
- 10
- Benchmarks
- 20K
- Parallel envs
Technical Report 
MAI-Gym-9B Open research resources
- Training & evaluation infra
- App & task synthesis skills
- 49 resettable apps
- 2,000 synthetic tasks
- MAI-Gym-9B checkpoint
01Platform
One rollout path. Three workloads.
02Overview
Evaluate, Diagnose, Synthesize, Retrain.
Open releaseFrameworkSynthesis skills49 mocked apps2,000 verified tasksMAI-Gym-9B weights
03Real-world clones
Mocked Apps: Our Clones vs. the Real Ones
Drag across the phone to compare the real app with its mock. Arrow keys move in small steps; Home and End reveal one full image.
04Novel apps
Mocked Apps: 42 Novel Apps
05Main results
MAI-Gym-9B leads open GUI agents under 10B.
| Model | Size | Access | Mobile | Computer | |||||
|---|---|---|---|---|---|---|---|---|---|
| Task OOD | Env OOD | Task OOD | Env OOD | ||||||
| MobileWorldGUI-only | AndroidWorld | MobileGym | OSWorld-Verified | OSWorld 2.0 | MacAgentBench | MyPCBench | |||
| Frontier models | |||||||||
| Claude Opus 4.8 | – | API | 69.2 | 75.0 | 50.7 | 83.4 | 20.3/54.8 | 58.4 | 62.0 |
| GPT-5.5 | – | API | 62.4 | 74.1 | 48.1 | 78.7 | 13.9/47.5 | 66.7 | 47.3 |
| Qwen3.8-Max | 2.4T-A95B | API | 77.8 | 85.3 | 61.3 | 86.1 | 19.4/46.7 | 74.3 | 48.6 |
| Qwen3.8-27B | 27B | Open | 70.1 | 81.9 | 57.5 | 84.3 | 19.4/48.0 | 65.4 | 28.5 |
| GUI agent models | |||||||||
| GUI-Owl-1.5-8B | 8B | Open | 38.2 | 71.6 | 15.1 | 52.9 | 0.0/1.6 | 10.2 | 1.8 |
| GUI-Owl-1.5-32B | 32B | Open | 43.9 | 69.8 | 20.3 | 56.5 | 0.9/2.5 | 17.2 | 6.3 |
| UI-Venus-2-9B | 9B | Open | 75.2 | 80.2 | 52.7 | 70.8 | 0.0/7.5 | 36.2 | 6.3 |
| SCALECUA-Qwen3.5-9B | 9B | Open | 12.8 | 47.4 | 13.5 | 68.7 | 0.0/2.2 | 3.9 | 1.1 |
| Qwen-CUA | 397B-A17B | Open | – | – | – | 86.2 | 18.5/48.4 | 69.2 | 58.7 |
| Ours | |||||||||
| Qwen3.5-9B base model | 9B | Open | 18.0 | 57.8 | 9.0 | 41.8 | 0.0/2.5 | 20.7 | 6.5 |
| MAI-Gym-9B | 9B | Open | 78.4 | 75.9 | 46.5 | 76.2 | 3.7/11.3 | 41.3 | 15.2 |
| (+60.4) | (+18.1) | (+37.5) | (+34.4) | (+3.7/+8.8) | (+20.6) | (+8.7) | |||
Citation
@article{maigym2026,
title = {MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation
and RL Training across Mobile and Computer Environments},
author = {Panrong Tong and Quyu Kong and Pengxiang Li and Tianyu Xia and
Hanzhang Zhou and Lei Wang and Chenglin Cai and Long Li and Long Chen and
Jianan Zhang and Zhengwen Zeng and Xu Zhang and Gongjie Zhang and
Yue Wang and Steven Hoi},
journal = {arXiv preprint arXiv:2610.20939},
year = {2026},
url = {https://arxiv.org/abs/2610.20939}
}