Primary Host: zeyu.tw Secondary Host: zeyu-ai.net (CN Users)

Zeyu丶泽宇's CV

Return Home
PFP

Zeyu Zhang

Email: qxc4fh@virginia.edu / zhang@zeyu.tw

INTRODUCTION

I am a PhD student at the University of Virginia (UVA), focusing on systems for training, inference, and evaluation of Large Language Models (LLMs) and recommendation models. My research primarily centers on optimizing long-context models and improving the communication, computation, and memory efficiency of KV cache. I also work on mitigating straggler issues in large-scale Machine Learning (ML) training. Prior to my PhD, I worked on network communication optimization, including user-space networking stacks and Network Function Virtualization (NFV). Earlier in my career, I also conducted research in recommender systems and algorithms.

EXPERIENCES

ByteDance, San Jose, California, USA (08/2026-Present)

  • Software Engineer Intern (Inference Infrastructure)
  • Working on the open-source project AIBrix.

OppyAI, USA (05/2026-08/2026)

  • Intern (Split Machine Learning)
  • Worked on resource optimization for split machine learning inference.

Meta (Formerly Facebook), Sunnyvale, California, USA (06/2025-12/2025)

  • Research Scientist Intern (AI Systems Machine Learning)
  • Worked on systems for large language and recommendation models.

Harvard University, Boston, Massachusetts, USA (03/2024-08/2024)

  • Visiting Researcher
  • Worked on systems for Large Language Models (LLMs). The main topic was LLM KV cache quantization, and we proposed homomorphic quantization for LLM KV cache to deal with communication, computation, and memory issues in disaggregated LLM serving.

Microsoft, Seattle, Washington, USA (05/2023-08/2023, 09/2024-12/2024)

  • Visiting Researcher
  • Worked with DeepSpeed on LLM training, especially on long-context-model training.
  • Worked with Azure on multi-modality model serving.

University of Virginia, Charlottesville, Virginia, USA (08/2021-Now)

  • PhD in Computer Science
  • Working on systems for AI.
  • Worked on LLM KV cache optimization and proposed ZACK to reduce KV cache size in the hidden size dimension, which is orthogonal to quantization and token eviction based methods. I also enhanced the self-attention kernel used for ZACK.
  • Worked on long-context-model inference and proposed CSPS and PecSched for efficient long-context-model serving.
  • Also worked on straggler problems in Machine Learning (ML) training.

Intel, Shanghai, China (06/2019-11/2019)

  • Intern in Network and Custom Logic Group (NCLG)
  • Worked on user-space networking stack (collaborating with Cisco). I optimized NGINX based on open-source high-performance packet processing framework VPP to increase its throughput, achieve good scalability, reduce latency and reduce CPU usage.

Shanghai Jiao Tong University, Shanghai, China (09/2017-03/2020)

  • Master in Software Engineering
  • Worked on Network Function Virtualization (NFV).

Wuhan University, Wuhan, China (09/2013-06/2017)

  • Bachelor in Software Engineering
  • Made identification of consumer groups in shopping malls and promoted the maximization of merchants' profits by predicting consumer groups' group behaviors.
  • I designed an improved Apriori Algorithm that is able to efficiently conduct trajectory prediction in large shopping malls.

RESEARCH PAPERS

* Equal contribution.

  • To Appear Zeyu Zhang and Haiying Shen. 2026. FastDC: Fast KV Dimensionality Compression for Decompression-Free and Attention-Accelerated LLM Inference. In Proceedings of the 2026 ACM Symposium on Cloud Computing (Singapore) (SoCC ’26). Association for Computing Machinery, New York, NY, USA. [Link]
  • To Appear Zeyu Zhang, Haiying Shen, Shay Vargaftik, Ran Ben Basat, Michael Mitzenmacher, and Minlan Yu. 2026. HACK: Homomorphic Acceleration via Compression of the Key-Value Cache for Disaggregated LLM Inference. In Proceedings of the 2026 ACM SIGOPS Annual Technical Conference (Hong Kong, China) (ATC ’26). Association for Computing Machinery, New York, NY, USA. [Link]
  • Zeyu Zhang and Haiying Shen. 2026. Straggler Tolerant and Resilient DL Training on Homogeneous GPUs. In 2026 35th International Conference on Computer Communications and Networks (ICCCN) (Honolulu, HI, USA). IEEE, 1–9. https://doi.org/10.1109/ICCCN69946.2026.11662727
  • Zeyu Zhang and Haiying Shen. 2026. PEACE: Preemptive and Efficient Cluster Scheduling for LLM Inference with Mixed Prompts. In 2026 IEEE International Parallel and Distributed Processing Symposium (IPDPS) (New Orleans, LA, USA). IEEE, 1272–1286. https://doi.org/10.1109/IPDPS65963.2026.00105
  • Zhaoyuan Su, Zeyu Zhang, Tingfeng Lan, Zirui Wang, Haiying Shen, Juncheng Yang, and Yue Cheng. 2026. MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing. In Proceedings of Machine Learning and Systems (Bellevue, WA, USA), A. Chowdhery and Z. Jia (Eds.), Vol. 8. MLSys, 1056–1075. [Link]
  • Haoran Qiu, Anish Biswas, Zihan Zhao, Jayashree Mohan, Alind Khare, Esha Choukse, Íñigo Goiri, Zeyu Zhang, Haiying Shen, Chetan Bansal, Ramachandran Ramjee, and Rodrigo Fonseca. 2025. ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving. In Proceedings of the 2025 ACM Symposium on Cloud Computing (Online, USA) (SoCC ’25). Association for Computing Machinery, New York, NY, USA, 817–830. https://doi.org/10.1145/3772052.3772254
  • Short Paper Zeyu Zhang, Haiying Shen, Shay Vargaftik, Ran Ben Basat, Michael Mitzenmacher, and Minlan Yu. 2025. HACK: Homomorphic Acceleration via Compression of the Key-Value Cache for Disaggregated LLM Inference. In Proceedings of the ACM SIGCOMM 2025 Conference (São Francisco Convent, Coimbra, Portugal) (SIGCOMM ’25). Association for Computing Machinery, New York, NY, USA, 1245–1247. https://doi.org/10.1145/3718958.3750481 [Link]
  • Suraiya Tairin*, Zeyu Zhang*, and Haiying Shen. 2025. Revisiting the Straggling Problem in GPU-based Distributed Deep Learning Training. In 2025 34th International Conference on Computer Communications and Networks (ICCCN) (Tokyo, Japan). IEEE, 1–9. https://doi.org/10.1109/ICCCN65249.2025.11133842
  • Haiying Shen and Zeyu Zhang. 2025. Deep Learning Training Job Scheduling for Proactive Straggler Reduction. In 2025 IEEE 25th International Symposium on Cluster, Cloud and Internet Computing (CCGrid) (Tromsø, Norway). IEEE, 1–12. https://doi.org/10.1109/CCGRID64434.2025.00038
  • Preprint Zeyu Zhang and Haiying Shen. 2025. ZACK: Zero-Overhead LLM Inference Acceleration via Dimensionality Compression of the Key-Value Cache. arXiv:2408.04107v2 [cs.LG] [Link]
  • Preprint Zeyu Zhang and Haiying Shen. 2024. CSPS: A Communication-Efficient Sequence-Parallelism based Serving System for Transformer based Models with Long Prompts. arXiv:2409.15104v1 [cs.DC] [Link]
  • Suraiya Tairin, Haiying Shen, and Zeyu Zhang. 2023. Embracing Uncertainty for Equity in Resource Allocation in ML Training. In Proceedings of the 52nd International Conference on Parallel Processing (Salt Lake City, UT, USA) (ICPP 2023). Association for Computing Machinery, New York, NY, USA, 423–432. https://doi.org/10.1145/3605573.3605583
  • Zeyu Zhang and Weiping Zhu. 2017. Location and Motion Prediction of Consumers in a Large Shopping Mall. In 2017 Fifth International Conference on Advanced Cloud and Big Data (CBD) (Shanghai, China). IEEE, 250–255. https://doi.org/10.1109/CBD.2017.50

PATENTS

  • A Network Request Processing System and Method (Jian Li, Zeyu Zhang, Haibing Guan) (Chinese Patent Number: 202010059255.0)

Kuro Usagi