Ph.D. student in Computer Science · Data Systems Group · University of Waterloo
I am a Ph.D. student in the Data Systems Group at the Cheriton School of Computer Science, University of Waterloo, advised by Prof. M. Tamer Özsu. I work on multimodal and agentic data management: natural-language interfaces to databases, complex analytical tasks over heterogeneous federated systems, computer-use agents, and multimodal learning.
I was a research intern at Salesforce AI Research in 2026 and a visiting researcher at ServiceNow Research from 2024 to 2025. Before Waterloo, I received my M.Sc. in Data Science from City University of Hong Kong, advised by Prof. Yu Yang, working on data mining and graph learning. Off the clock, I follow Manchester City closely.
xiangru=# SELECT position, status xiangru-# FROM job_market xiangru-# WHERE available_from = 'May 2027';
| position | status |
|---|---|
| Tenure-track Assistant Professor | open |
| Postdoctoral Researcher | open |
| Industry Research Scientist | open |
(3 rows)
I expect to complete my Ph.D. in May 2027 and would be glad to talk about faculty, postdoc, and research scientist roles. Email me for the most recent version of my CV.
I build systems that let LLM agents answer hard analytical questions over data that is scattered, heterogeneous, and multimodal: relational and graph databases, documents, videos, and the desktop software where much of that data actually lives.
Thesis (tentative) A Multi-Agent Architecture for Complex Analytical Tasks over Heterogeneous Federated Systems
Natural-language interfaces to relational and graph databases, multi-agent analytics over federated data lakes, and the interplay between LLMs and data management systems.
Benchmarks, large-scale human demonstrations, grounding models, and RL training for agents that operate real desktop software and tools (MCP).
Vision-language models for documents, posters, and molecules; LLMs and vision models on graphs; representation learning for tables.
RA-SQL: Relational Algebra as Deterministic Chain-of-Thought for Natural Language to SQL accepted by SIGMOD 2027.
TRL-Bench, a cross-paradigm benchmark for tabular encoders, accepted by the NeurIPS 2026 Datasets & Benchmarks Track. [project]
FORGE, fine-grained multimodal evaluation for manufacturing, accepted by Findings of EMNLP 2026. [project]
Talk at Peking University: Answering Complex Analytical Questions over Heterogeneous Federated Systems.
Wrapped up my research internship at Salesforce AI Research (Singapore) on computer-use and MCP tool-use agents, mentored by Junnan Li.
Preprints released: MCP-Universe RL (RL for MCP tool-use agents, [project]) and TEXAS (task-expert-aware supervision for MoE LLM adaptation).
Preprint released: StateAct, program state before pixels for long-horizon computer-use agents.
MolEmb appears at the ICML 2026 Workshop on Multi-modal Foundation Models and LLMs for Life Sciences; CUA-Suite appears at the ICLR 2026 Workshop on Lifelong Agents.
The Duality between Large Language Models and Data Management Systems published in IEEE Data Engineering Bulletin 50(1).
InteracSPARQL accepted by Findings of ACL 2026.
CUA-Suite released and ranked #1 Paper of the Day on Hugging Face. It unifies UI-Vision, GroundCUA and VideoCUA (55 hours of expert video across 87 professional apps); the series has 500K+ downloads on Hugging Face.
Invited talk at Tencent Hunyuan: Recent Efforts on CUAs' Data, Training, Evaluation, and (Potentially) Beyond.
Spatio-temporal traffic accident detection via graph-based GAN published in Engineering Applications of Artificial Intelligence.
Two papers accepted by ICLR 2026: GraphOmni and GroundCUA (Grounding Computer Use Agents on Human Demonstrations).
Invited talks at Snowflake (Dedicated Multi-Agent System over Heterogeneous Data Lakes) and LlamaIndex (Multimodal Learning on Documents and Beyond).
LazyVLM, a neuro-symbolic approach to video analytics, accepted by ICDE 2026 (Demo).
Talk at ONDBD 2025: An Interactive Tool for SPARQL Query Refinement Using Natural Language Explanations.
Three papers accepted by NeurIPS 2025: Paper2Poster (3.9k ★ on GitHub), vision models for graph structural understanding, and AlignVLM.
UI-Vision, a desktop-centric GUI benchmark, accepted by ICML 2025.
BigDocs accepted by ICLR 2025; spectral augmentation for graph SSL accepted by TMLR.
Started a visiting researcher internship at ServiceNow Research (Montreal), working on multimodal learning and GUI understanding (through May 2025).
Two papers accepted by EMNLP 2023: Balance Act (Oral) and InvGC (Findings).
Paper on loss landscapes of PDE-solving networks accepted by IEEE BigData 2023 (Oral).
Decentralized online DR-submodular maximization accepted by CIKM 2023.
* equal contribution† project lead✉︎ corresponding author
ACM SIGMOD International Conference on Management of Data (SIGMOD)
Serializes queries as relational-algebra operator trees and uses them as a deterministic chain of thought, so the model reasons in RA before it writes SQL.
NeurIPS 2026 Datasets and Benchmarks Track · * core contributors
Probes row, column, and table embeddings from 20 tabular encoders across 16 tasks and 87 datasets, and finds that encoder quality is capability-specific rather than a single leaderboard.
Findings of EMNLP 2026
A benchmark of real 2D images and 3D point clouds annotated down to exact model numbers; across 18 MLLMs, missing domain knowledge, not visual grounding, is the main bottleneck.
arXiv preprint
A code-first multi-agent harness that grounds computer-use agents in program state and calls a GUI agent only when needed, raising OSWorld 2.0 binary success from 20.6% to 26.9% (Claude Opus 4.8) at about 9× lower cost per task.
arXiv preprint · short version at the ICLR 2026 Workshop on Lifelong Agents
About 10K expert desktop tasks across 87 professional apps (55 hours of 30 fps video with cursor traces and reasoning), unified with UI-Vision and GroundCUA into one ecosystem for training and evaluating CUAs.
ICML 2026 Workshop on Multi-modal Foundation Models and LLMs for Life Sciences
IEEE Data Engineering Bulletin, 50(1):40–58
A two-way view of LLMs for databases and databases for LLMs, from LLM-synthesized database engines to data systems that become agentic data environments.
Findings of ACL 2026
A training-free refinement loop that explains generated SPARQL in natural language and lets users and LLMs fix it, improving accuracy on QALD-9 and QALD-10.
International Conference on Learning Representations (ICLR)
Benchmarks LLM reasoning on graph problems across task, graph type, serialization, and prompt scheme, with an RL-inspired selector that finds strong configurations cheaply.
International Conference on Learning Representations (ICLR)
GroundCUA: 3.56M human-verified UI element annotations across 87 desktop apps. GroundNext models trained on it reach state of the art at the 3B and 7B scales on five grounding benchmarks with a tenth of the usual data.
IEEE International Conference on Data Engineering (ICDE), Demo Track
Decomposes multi-frame video queries into fine-grained operations and pushes most of the work into relational query execution and vector search, so video analytics scales beyond what VLMs alone can do.
Engineering Applications of Artificial Intelligence, vol. 165
NeurIPS 2025 Datasets and Benchmarks Track · Oral at the ICML 2025 MAS Workshop
The first benchmark and metric suite for academic poster generation, plus PosterAgent, a visual-in-the-loop multi-agent pipeline that turns papers into editable posters.
Advances in Neural Information Processing Systems (NeurIPS)
Advances in Neural Information Processing Systems (NeurIPS)
International Conference on Machine Learning (ICML)
A license-permissive desktop GUI benchmark over 83 applications with dense human annotations for element grounding, layout grounding, and action prediction.
International Conference on Learning Representations (ICLR)
A 7.5M-sample, license-permissive dataset across 30 document and code tasks (e.g., screenshot-to-HTML, table-to-LaTeX), released with the BigDocs-Bench benchmark.
Transactions on Machine Learning Research (TMLR)
Engineering Applications of Artificial Intelligence, vol. 151
Conference on Empirical Methods in Natural Language Processing (EMNLP)
IEEE International Conference on Big Data (BigData)
ACM International Conference on Information and Knowledge Management (CIKM)
Under review
No papers in this view. Try “All”.
Much of my work builds the data that agents and multimodal models learn from and are measured on. Every number below comes from the paper or the official release.
A unified ecosystem of expert video demonstrations and dense annotations for training and evaluating desktop computer-use agents across 87 professional applications. The series has 500K+ downloads on Hugging Face.
The largest open expert video corpus for desktop computer use, with cursor traces and multi-layered reasoning annotations.
Pixel-precise desktop grounding data built entirely by human annotators; it trains the GroundNext 3B/7B models, state of the art at their scale on five grounding benchmarks.
A license-permissive desktop benchmark for element grounding, layout grounding, and action prediction.
An open, license-permissive dataset for document understanding and code generation, plus BigDocs-Bench, 10 new tasks such as Screenshot2HTML, Table2LaTeX, and Image2SVG.
The first benchmark and metric suite for academic poster generation, with PosterAgent, a multi-agent pipeline that turns papers into editable posters. Oral at the ICML 2025 MAS Workshop.
A benchmark of LLM reasoning on graph-theoretic problems that varies graph type, serialization format, and prompt scheme across three difficulty levels.
A benchmark of holistic graph-structure perception (archetypes, symmetry, connectivity strength, critical elements), tested on progressively larger out-of-distribution graphs.
A representation-level benchmark that probes frozen row, column, and table embeddings from tabular encoders with shared lightweight heads, across three suites: TRL-CTbench, TRL-Rbench, and TRL-DLTE.
A manufacturing benchmark of real 2D images and 3D point clouds annotated down to exact model numbers, covering workpiece verification, surface inspection, and assembly verification.
A molecular context-aware retrieval benchmark from the MolEmb paper: it tests whether task instructions let molecular embeddings retrieve the right context when molecule identity alone is not enough.
Research Intern · Singapore
Computer-use and MCP tool-use agents. Contributed to StateAct and MCP-Universe RL; responsible for data development and reinforcement-learning training of the internal SFR-CUA model. Mentor: Junnan Li
Visiting Researcher (Internship) · Montreal, Canada
Multimodal learning and GUI understanding: BigDocs, UI-Vision, GroundCUA, and CUA-Suite. Mentor: Sai Rajeswar
Research Assistant · Hong Kong SAR
Order prediction for industrial partners including Alibaba and HKTV Mall.
Research Assistant · Hong Kong SAR
Loss landscapes of deep neural networks.
Ph.D., Computer Science
2022 – 2027 (expected)
Advisor: M. Tamer Özsu
M.Sc., Data Science
2020 – 2021
Advisor: Yu Yang · Graduated with distinction, ranked 1st
B.Eng., Civil Engineering; minor in Mathematics
2014 – 2019
Outstanding Graduate (top 5%)