Mingkai Deng
mingkaid [at] cs [dot] cmu [dot] edu

I am a PhD student at the Language Technologies Institute at Carnegie Mellon University, working with Prof. Eric P. Xing. I also work closely with Prof. Zhiting Hu.

My research centers on world models and agent models, including topics like multimodality, reasoning & planning, representation, self-supervised learning, and reinforcement learning. World models are generative models that simulate future states of the world given proposed actions, while agent models are reasoning systems that select actions to achieve specific goals. I'm particularly interested in how the interplay between these models can help us move towards general-purpose intelligence.

I completed my MS at CMU's Machine Learning Department. Before that, I worked as a data scientist in industry following an internship at Baidu Research, where I was advised by Prof. Hui Xiong. I earned my undergraduate degree from Columbia University, with a double major in Mathematics-Statistics and Computer Science.

CV  /  LinkedIn  /  Twitter  /  GitHub

• 2025-07 Our position paper "Critiques of World Models" is released on Arxiv.
• 2025-05 Started internship at MBZUAI IFM working on training agent models with RL
• 2025-02 Our project "SimuRA: Towards General Goal-Oriented Agent via Simulative Reasoning Architecture with LLM-Based World Model" won the Fundamental Track at Berkeley's LLM Agents Hackathon.
• 2025-02 Released ReasonerAgent, a fully open source, ready-to-run agent that does research in a web browser. Check out the demo here
• 2024-09 Our paper "Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs" is accepted by NeurIPS 2024 Datasets and Benchmarks Track.

This position paper critiques several schools of thoughts on world modeling, and proposes a new architecture for a general-purpose world model, with an outlook of a Physical, Agentic, and Nested (PAN) AGI system enabled by such a model.

A general architecture for optimal goal-oriented agent based on simulation with LLM-based world model. The web agent based on SimuRA, ReasonerAgent-Web, shows up to 124% improvement in web browsing tasks.

A 65-billion parameter, 360-degree open-source LLM that surpasses LLaMA-65B and rivals LLaMA2-70B, while requiring fewer FLOPs and tokens.

A large-scale webpage-to-code dataset which enhances the webpage understanding and HTML code translation abilities of MLLMs without compromising general visual capabilities.

An efficient and flexible framework for using RL to optimize prompts of discrete text that enable pre-trained LMs (e.g., BERT, GPT-2) to perform diverse NLP tasks. Experiments on few-shot classification and unsupervised text style transfer show superior performance to a wide range of existing methods.

A general framework that helps solve the difficulty of evaluating natural language generation (NLG) with a single unified operation. Inspired evaluation metrics improve over SOTA metrics for diverse NLG tasks. Our metrics are available as library on PyPI and GitHub.

An open-source web agent that operates in a Chromium-based browser and can perform a broad range of realistic tasks that require complex, long-range, and goal-achieving behavior, such as searching for flights, compiling online shopping options, and researching news coverage.

A data-driven narrative of NYC gentrification patterns over time and their second-order impact on the city's inhabitants. Construct significant leading predictors using resources from NYC OpenData.

Combine structured point-of-interest (POI, e.g., parking lots and buildings) data with unstructured satellite image data to automatically identify and draw the boundaries of urban areas-of-interest (e.g., residential areas and campuses).

Formulate the Transformer model as an ordinary differential equation (ODE) and perform forward-backward operations using an ODE solver. The resulting continuous-depth model has fewer parameters, converges faster, and shows promising performance in empirical experiments.

The source code of this website is adapted from here