Yu Mao

Yu Mao

Research Scientist at ByteDance

Publications

← Selected publications on the homepage

LLM Capability & Agents

Probe with Participation Trophies: Random-Reward RL as a Probe of LLM Capability
Probe with Participation Trophies: Random-Reward RL as a Probe of LLM Capability
Yu Mao, L. Yu, Z. Zhu, Y. Zheng, H. Li, F. Shi, Y. Yin, Z. Wang, J. Niu
Preprint 2026
arXiv PDF Code
AgentPProf: Semantic Profiler for AI Agents
AgentPProf: Semantic Profiler for AI Agents
Yusheng Zheng, Chaokun Chang, Yu Mao, Tianyuan Wu, Yuxi Huang, Tao Ma, Wenan Mao, Shuyi Cheng, Andi Quinn, Wei Wang
AgenticOS @ SOSP 2026 · Best Paper Award
PDF Code
LLM Agent Capabilities Should Follow Task Intent and Context Source
LLM Agent Capabilities Should Follow Task Intent and Context Source
Yusheng Zheng, Wenhui Zhang, Yu Mao
AgenticOS @ SOSP 2026
arXiv PDF

Compression & Representation

ECM: Enhancing Compressibility of Quantized Vision Encoder and LLM for Large Vision-Language Models
ECM: Enhancing Compressibility of Quantized Vision Encoder and LLM for Large Vision-Language Models
W. Wang, Yu Mao, D. Tang, N. Guan, C.J. Xue
ICASSP 2026 Corresponding
PDF
WISE: A Framework for Gigapixel Whole-Slide-Image Lossless Compression
WISE: A Framework for Gigapixel Whole-Slide-Image Lossless Compression
Yu Mao, J. Wang, N. Guan, C.J. Xue
CVPR 2025
PDF
Easz: A Transformer-based Image Compression Framework for IoT
Easz: A Transformer-based Image Compression Framework for IoT
Yu Mao, J. Li, J. Wang, H. Xu, T.W. Kuo, N. Guan, C.J. Xue
DAC 2025
PDF
Lossless Compression of Large Language Model-Generated Text via Next-Token Prediction
Lossless Compression of Large Language Model-Generated Text via Next-Token Prediction
Yu Mao, H. Pirk, C.J. Xue
Preprint 2025
arXiv PDF
When Compression Meets Model Compression: Memory-Efficient Double Compression for Large Language Models
When Compression Meets Model Compression: Memory-Efficient Double Compression for Large Language Models
W. Wang, Yu Mao, D. Tang, H. Du, N. Guan, C.J. Xue
EMNLP Findings 2024 Corresponding
PDF
On the Compressibility of Quantized Large Language Models
On the Compressibility of Quantized Large Language Models
Yu Mao, W. Wang, H. Du, N. Guan, C.J. Xue
Preprint 2024
arXiv PDF KV Cache Benchmark
Faster and Stronger Lossless Compression with Optimized Autoregressive Framework
Faster and Stronger Lossless Compression with Optimized Autoregressive Framework
Yu Mao, J. Li, Y. Cui, C.J. Xue
DAC 2023
PDF Code
Accelerating General-purpose Lossless Compression via Simple and Scalable Parameterization
Accelerating General-purpose Lossless Compression via Simple and Scalable Parameterization
Yu Mao, Y. Cui, T.-W. Kuo, C.J. Xue
ACM MM 2022
PDF
Trace: A Fast Transformer-based General-purpose Lossless Compressor
Trace: A Fast Transformer-based General-purpose Lossless Compressor
Yu Mao, Y. Cui, T.-W. Kuo, C.J. Xue
The Web Conference (WWW) 2022
PDF Code

AI Systems & Inference

DATE
Parallel-SA: Point Cloud Processing Acceleration via Parallel Set Abstraction
D. Tang, W. Wang, Yu Mao, W. Xie, N. Guan, T.-W. Kuo, C.J. Xue
DATE 2026 Corresponding
PMR: Fast Application Response via Parallel Memory Reclaim on Mobile Devices
PMR: Fast Application Response via Parallel Memory Reclaim on Mobile Devices
W. Li, L.-P. Chang, Yu Mao, L. Shi
USENIX ATC 2025
PDF
DAWN: Accelerating Point Cloud Object Detection
DAWN: Accelerating Point Cloud Object Detection
D. Tang, Yu Mao, W. Wang, N. Guan, T.W. Kuo, C.J. Xue
DAC 2025 Corresponding
PDF
STEM: Streaming-based FPGA Acceleration for Large-Scale Compactions in LSM KV
STEM: Streaming-based FPGA Acceleration for Large-Scale Compactions in LSM KV
D. Tang, W. Wang, Yu Mao, J. Yu, T.-W. Kuo, C.J. Xue
ICDE 2024 Corresponding
PDF
Moby: Empowering 2D Models for Efficient Point Cloud Analytics on the Edge
Moby: Empowering 2D Models for Efficient Point Cloud Analytics on the Edge
J. Li, Y.H. Cai, L. Liu, Yu Mao, C.J. Xue, H. Xu
ACM MM 2023
arXiv PDF

Applied ML

BAHOP: Similarity-based Basin Hopping for A fast hyper-parameter search in WSI classification
BAHOP: Similarity-based Basin Hopping for A fast hyper-parameter search in WSI classification
J. Wang, Yu Mao, Y. Cui, N. Guan, C.J. Xue
ICASSP 2026 Corresponding
arXiv PDF
SHAP-CAT: An Interpretable Multi-Modal Framework Enhancing WSI Classification via Virtual Staining and Shapley-Value-Based Multimodal Fusion
SHAP-CAT: An Interpretable Multi-Modal Framework Enhancing WSI Classification via Virtual Staining and Shapley-Value-Based Multimodal Fusion
J. Wang, Yu Mao, N. Guan, C.J. Xue
Preprint 2024 Corresponding
arXiv PDF
Advances in Multiple Instance Learning for Whole Slide Image Analysis: Techniques, Challenges, and Future Directions
Advances in Multiple Instance Learning for Whole Slide Image Analysis: Techniques, Challenges, and Future Directions
J. Wang, Yu Mao, N. Guan, C.J. Xue
Preprint 2024 Corresponding
PDF
Variational Nested Dropout
Variational Nested Dropout
Y. Cui, Yu Mao, Z. Liu, Q. Li, A.B. Chan, X. Liu, T.-W. Kuo, C.J. Xue
IEEE TPAMI 2023
PDF