I am a Research Scientist at Meta, working on multimodal agents and computer-use agents for Muse Spark. Previously, I was a Member of Technical Staff at xAI working on computer-use agents, and before that I worked on multimodal post-training and UI agents for Apple Foundation Models. I completed my Ph.D. in Computer Science at UCLA, advised by Prof. Nanyun (Violet) Peng, focusing on multimodal learning. During my Ph.D., I received the Amazon PhD Fellowship and the JPMC PhD Fellowship. I earned my M.S. at Carnegie Mellon University under Prof. Graham Neubig, specializing in NLP, and my B.S. from the KYM Honors School at Nanjing University.
Experience
Meta MSL, 04/2026-
Research Scientist working on multimodal and computer-use agents for Muse Spark
xAI, 11/2025-03/2026
Member of Technical Staff working on mid-training and post-training for computer-use agents
Apple AI/ML, 01/2025-11/2025
Research Scientist working on multimodal post-training and agents for Apple Foundation Models
Publications ( show selected )
MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer
Yanghao Li, Rui Qian, Bowen Pan, Haotian Zhang, Haoshuo Huang, Bowen Zhang, Jialing Tong, Haoxuan You, Xianzhi Du, Zhe Gan, Hyunjik Kim, Chao Jia, Zhenbang Wang, Yinfei Yang, Zi-Yi Dou, Wenze Hu, Chang Gao, Dongxu Li, Philipp Dufter, Zirui Wang, Guoli Yin, Zhengdong Zhang, Chen Chen, Yang Zhao, Ruoming Pang, Zhifeng Chen
Re-ReST: Reflection-Reinforced Self-Training for Language Agents
Zi-Yi Dou, Cheng-Fu Yang, Xueqing Wu, Kai-Wei Chang, Nanyun Peng
EMNLP 2024
VALOR-EVAL: Holistic Coverage and Faithfulness Evaluation of Large Vision-Language Models
Haoyi Qiu*, Wenbo Hu*, Zi-Yi Dou, Nanyun Peng
ACL-Findings 2024
DesCo: Learning Object Recognition with Rich Language Descriptions
Liunian Harold Li*, Zi-Yi Dou*, Nanyun Peng, Kai-Wei Chang
NeurIPS 2023
Masked Path Modeling for Vision-and-Language Navigation
Zi-Yi Dou, Feng Gao, Nanyun Peng
EMNLP-Findings 2023
FOAM: A Follower-aware Speaker Model for Vision-and-Language Navigation
Zi-Yi Dou, Nanyun Peng
NAACL 2022
Explainaboard: An Explainable Leaderboard for NLP
Pengfei Liu, Jinlan Fu, Yang Xiao, Weizhe Yuan, Shuaicheng Chang, Junqi Dai, Yixin Liu, Zi-Yi Dou, Graham Neubig
ACL 2021 Demo Best Demo
awesome-align: Word Alignment by Fine-tuning Embeddings on Parallel Corpora
Zi-Yi Dou, Graham Neubig
EACL 2021
CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems
Yiran Chen, Pengfei Liu, Ming Zhong, Zi-Yi Dou, Danqing Wang, Xipeng Qiu, Xuanjing Huang
EMNLP-Findings 2020
TICO-19: the Translation Initiative for COvid-19
Antonios Anastasopoulos, Alessandro Cattelan, Zi-Yi Dou, Marcello Federico, Christian Federman, Dmitriy Genzel, Francisco Guzmán, Junjie Hu, Macduff Hughes, Philipp Koehn, Rosie Lazar, Will Lewis, Graham Neubig, Mengmeng Niu, Alp Öktem, Eric Paquin, Grace Tang, Sylwia Tur
EMNLP 2020 Workshop on NLP-COVID
A Deep Reinforced Model for Zero-Shot Cross-Lingual Summarization with Bilingual Semantic Similarity Rewards
Zi-Yi Dou, Sachin Kumar, Yulia Tsvetkov
ACL 2020 Workshop on Neural Generation and Translation
Domain Differential Adaptation for Neural Machine Translation
Zi-Yi Dou, Xinyi Wang, Junjie Hu, Graham Neubig
EMNLP 2019 Workshop on Neural Generation and Translation
Information Aggregation for Multi-Head Attention
Jian Li, Baosong Yang, Zi-Yi Dou, Xing Wang, Michael R. Lyu, Zhaopeng Tu
NAACL 2019
Exploiting Deep Representations for Neural Machine Translation
Zi-Yi Dou, Zhaopeng Tu, Xing Wang, Shuming Shi, Tong Zhang
EMNLP 2018
Unsupervised Bilingual Lexicon Induction via Latent Variable Models
Zi-Yi Dou, Zhi-Hao Zhou, Shujian Huang
EMNLP 2018
SkipNet: Learning Dynamic Routing in Convolutional Networks
Xin Wang, Fisher Yu, Zi-Yi Dou, Trevor Darrel, Joseph E. Gonzalez
ECCV 2018
Education
University of California, Los Angeles, 08/2020-01/2025
Ph.D. in Computer Science
Carnegie Mellon University, 08/2018-08/2020
Master of Science in Language Technologies
Nanjing University, 09/2014-07/2018
Bachelor of Science in Computer Science, Minor in Statistics
Service
Area Chair for ACL, EMNLP, ACL Rolling Review
Reviewer for ACL, EMNLP, NAACL, ACL Rolling Review, ICML, NeurIPS, ICLR, CVPR, ICCV, ECCV, AAAI