avatar

Jianhong Tu

Ph.D. Student
University of California, Santa Cruz
tovitusaikou (at) gmail.com


About Me

I am a second-year PhD student in Computer Science at the University of California, Santa Cruz, advised by Dr. Chenguang Wang, and a visiting researcher at Scale AI. I have worked on evaluation of frontier AI agents, with a special interest in software engineering (SWE) agents on challenging cybersecurity tasks. I am drawn to the question of when agents can be trusted with real systems: how to build evaluations that are realistic and hard to game, how to measure what agents can actually do rather than what they have memorized, and how to turn their failures into better training. I am also fortunate to work with Prof. Dawn Song at Berkeley RDI.

Before UCSC, I earned my B.S. in Data Science from Washington University in St. Louis, where I worked on AI for Science at the Hengen Lab with Dr. Keith Hengen and Dr. James McGregor.

Publications

  1. Under Review
    CyberArena: Benchmarking AI Agents in Live Symmetric Attack-and-Defense Combat
    Bangya Liu*, Jianhong Tu*, Yueyang Pan*, Hanhong Zhao*, Zhe Ye, Hongwei Li, Saastha Vasan, Katherine Lambert, Junmin Zhu, Hao Wang, Jienan Chen, Yunze Xiao, Minglai Yang, Zihan Wang, Hairong Yin, Hao Wang, Zhiwen Fan, Tianneng Shi, Zhun Wang, Wenbo Guo, Chenguang Wang, Dawn Song (*Equal contribution)
    Under review at ICLR 2027.
    arXiv Project Page

  2. Preprint
    SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
    Xiangyi Li, Yimin Liu, Wenbo Chen, ..., Jianhong Tu, Chenguang Wang, Yizhou Sun, Wei Wang, ..., Dawn Song
    Show all 78 authorsXiangyi Li, Yimin Liu, Wenbo Chen, Bingran You, Zonglin Di, Yifeng He, Shenghan Zheng, Kyoung Whan Choe, Jiankai Sun, Shuyi Wang, Chujun Tao, Binxu Li, Xuandong Zhao, Hejia Geng, Xiaojun Wu, Junwei Zhou, Xiaokun Chen, Hanwen Xing, Yubo Li, Qunhong Zeng, Di Wang, Yuanli Wang, Roey Ben Chaim, Penghao Jiang, Haotian Shen, Luyang Kong, Xinyi Liu, Runhui Wang, Xuanqing Liu, Jiachen Li, Xin Lan, Yueqian Lin, Wengao Ye, Junwei He, Songlin Li, Yue Zhang, Yipeng Gao, Yijiang Li, Ze Ma, Liqiang Jing, Tianyu Wang, Kaixin Li, Yiqi Xue, Haoran Lyu, Yizhuo He, Yuchen Tian, Shutong Wu, Bowei Wang, Yixuan Gao, Bo Chen, Litong Liu, Sikai Cheng, Jiajun Bao, Shuaicheng Tong, Shuwen Xu, Terry Yue Zhuo, Tinghan Ye, Qi Qi, Miao Li, Longtai Liao, Zelin Tan, Chang Shi, Xilin Tang, Srinath Tankasala, Boqin Yuan, Yaoyao Qian, Jianhong Tu, Chenguang Wang, Yizhou Sun, Wei Wang, Aaron Taylor, Ziyue Yang, Changkun Guan, Zhikang Dong, Xinyu Zhang, Steven Dillmann, Han-chung Lee, Dawn Song
    Preprint, arXiv:2602.12670, 2026.
    arXiv Project Page

  3. Preprint
    AgentBeats: Agentifying Agent Assessment for Openness, Standardization, and Reproducibility
    Xiaoyuan Liu, Jianhong Tu, Yuqi Chen, Siyuan Xie, Sihan Ren, Tianneng Shi, Gal Gantar, Evan Sandoval, Donghyun Lee, Daniel Miao, Peter J. Gilbert, Nick Hynes, Mauro Staver, Warren He, David Marn, Andrew Low, Xi Zhang, Elron Bandel, Michal Shmueli-Scheuer, Siva Reddy, Alexandre Drouin, Alexandre Lacoste, Ramayya Krishnan, Elham Tabassi, Yu Su, Victor Barres, Chenguang Wang, Wenbo Guo, Dawn Song
    Preprint, arXiv:2606.13608, 2026.
    arXiv Project Page

  4. NeurIPS
    Agents' Last Exam
    Yiyou Sun, Xinyang Han, Weichen Zhang, ..., Xiaoyuan Liu, ..., Jianhong Tu, ..., Chenguang Wang, ..., Dawn Song
    Show all 379 authorsYiyou Sun, Xinyang Han, Weichen Zhang, Yuanbo Pang, Tianyu Wang, Yuhan Cao, Yixiao Huang, Xiang Liu, Zonglin Di, Chris Duroiu, Haoyun Zhang, Jeffrey Lin, Weishu Zhang, Tyler Zeng, Ying Yan, Bo Liu, Hanson Wen, Mingyang Xu, Xiaoyuan Liu, Zimeng Chen, Zhaowei Zhang, Weiyan Shi, Amanda Dsouza, Vincent Sunn Chen, Jingxu Xie, Dewen Lei, Hanxuan Zhang, Yinhe Wang, Xinran Zhang, Chao Zhou, Sheldon Huang, Patrick Bryant, Carl Boettiger, Yamini Rangan, Bradley Rothenberg, Kyle Steinfeld, Arvind Rao, Tapio Schneider, Georgios Yannakakis, Laure Zanna, Kaan Ozbay, Ida Sim, Tarek Zohdi, George Em Karniadakis, Jack Gallant, Teresa Head-Gordon, Chris Yuhao Liu, Rongwang Hu, Yafei Cheng, Aras Bacho, Zengyi Qin, Alan Yuille, Yixiong Chen, Hao Liu, Lin Zeng, Bo Chen, Zhendong Li, Litian Gong, Shashank Muralidhar Bharadwaj, Thorben Jansen, Haixiang Tang, Jianhong Tu, Jixian Liu, Maojia Song, Ruheng Wang, Shengcao Cao, Yanxin Zhang, Yingxuan Yang, Zongzheng Zhang, Honglin Bao, Hyunjun Kim, Peiran Xu, Shuo Lu, Zheng Zhang, Zhonghua Wang, Aditya Prakash, Bohan Lyu, Chang Ma, Haoxuan Chen, Hengduan Fan, Molei Tao, Roman Sitelew, Russell Alan Poldrack, Yang Liu, Yang Liu, Yichen Wu, Yuanze Lin, Zhenglu Li, Zibu Wei, Zijiao Chen, Abderrahim Bendahi, Alexander Schubert, Chase Broderick, Chengzhi Liu, Haoran Sun, Haoyang Li, Haoyue Hua, Jiayuan Ding, Jing Huang, Shangding Gu, Tuan Vinh, Xiatao Sun, Yanqiong Jiang, Yiran Hu, Zechen Xu, Ziao Yang, Chenguang Wang, Costas Spanos, Dongrong Yang, Haocheng Wang, Jiayi Cheng, Jicheng Wang, Junbo Peng, Kangqi Zhang, Keying Kuang, Lichao Sun, Ming Yin, Orson Xu, Vincent Siu, Wanzhe Liao, Wenqi Shi, Wenxi Deng, Xiaopan Zhang, Yangjunfeng Zhu, Yifan Shen, Yi Jiang, Yinsheng Li, Youbang Pan, Yuan Li, Yuchen Zhu, Yunshi Zhang, Zhenning Yang, Ziyan Rao, Alireza Rafiei, Andrew Glover, Bixuan Ren, Boqin Yuan, Chonghao Cai, Deren Lei, Hanwen Xing, Haoran Jin, Hector Gomez, Ishaan Singh Chandok, Jiaming Hu, Jianxin Sun, Justin Purnomo, Kunyang Sun, Lei Ding, Longtai Liao, Lukas Kim, Ming Liu, Qingpo Wuwu, Ran Jin, Ruiqi Zhang, Ting Sun, Wenbo Huang, Xuandong Zhao, Peter Chen, Chuanyang Jin, Qiming Bao, Xuhuan Shen, Yifan Zhang, Yi Li, Yingze Wang, Yixin Wang, Yushi Yang, Zhen Dong, Zhiling Yan, Zhongxing Xu, Zibo Zhao, Zixin Jiang, Ali Emami, Andy Zeyi Liu, Angela Wu, Aylin Caliskan, Baihe Huang, Baixiang Huang, Benjamin Yang, Benran Zhang, Brian Zhao, Carlo Bosio, Charlie Guo, Che Liu, Chenchen Du, Chen Dai, Chenlong Zhang, Chenyu Li, Chenyu Zhu, Chi Zhang, Core Francisco Park, Daofeng Li, Dehao Wu, Dowling Wong, Erin Kasson, Fangyu Wu, Fan Xu, Fiona Shen, Haimin Hu, Haitao Li, Hang Zhou, Hannah Szlyk, Hanning Zhang, Hanwen Zhang, Hao Fan, Haoran Ye, HaoYang Shang, Haoyue Bai, Hengyu Fu, Hongwei Li, Huiqi Wang, Jiachen Li, Jialu Wang, Jiaming Liu, Jiawei Shen, Jierui Zuo, Jingxuan Fan, Jingyan Li, Jirong Yang, Jonah So, Jorin Overwiening, Joseph Cavanagh, Joseph Lee, Junge Zhang, Kaiyuan Liu, Kaustubh Dhole, Kevin Qinghong Lin, Kunpeng Zhang, Kyle Montgomery, Layna Paraboschi, Liana, Lihui Liu, Liwei Zhou, Lu Lu, Mengyuan "Millie" Wu, Mikko Tolonen, Mingxuan Yan, Minqi Shi, Murphy Niu, Pan Lu, Patricia Cavazos-Rehg, Qijian Zhao, Qiuyang Mang, Rimika Jaiswal, Ruiling Xu, Run-Ze Fan, Ruoxi Ning, Santanu Bosu Antu, Scott Rankin, Sen Yang, Shi Gu, Shi Qiu, Sichen Zhu, Sihan Xu, Siqi Zhu, Song Zhang, Sophia Sun, Stefano De Castro, Swati Rajwal, Tianneng Shi, Tianxing Chen, Tianyue Zhou, Ting Wang, Ventus Shaw, Vladislav Susoy, Weihang Xiao, Wei Wang, Wenbo Lv, Wenjin Zhang, Wenshuo Zhang, Wenyu Liang, Xiaohu Yu, Xiao Li, Xiaoli Yang, Xin Lan, Xinyu Wang, Xuan Liu, Xuewen Chen, Xu Pan, Yan Zhu, Yicheng Lyu, Yifan Gu, Yimin Zhao, Ying Zhang, Yin Ren, Yipu Chen, Yiren Rong, Yi Shao, Yishu Zhu, Yizhi Zhang, Yongyi Xiong, Yuangang Li, Yuchen Wu, Yuhaohua Zheng, Yuhao Wu, Yunfu Deng, Yushan Li, Yu Wu, Yuzheng Xu, Yuzhou Nie, Zehao Wang, Zhehang Du, Zhengming Yu, Zhihan Jiang, Zhi Li, Zhixu Li, Zhuoran Yi, Tao Sun, Zhun Wang, Justin Xu, Shi Bo, Helena Casademunt, Wenjia Chen, Cheng Cheng, Nawen Deng, Rao Fu, Tianfu Fu, Yifan Han, Zhenyu He, Qiao Jin, Langlang Li, Yuetai Li, Sylvia Liu, Subhabrata Mukherjee, Yunqi Ouyang, He Ren, Dawei Shi, Haoran Wu, Zhiyue Wu, Hannah Yao, Jenny Yu, Rhea Zhan, Luqing Zhou, Blake Zhu, Junfan Zhu, Sanjiv Das, Yanjun Gao, Wenbo Guo, Erika J Schneider, Bing Liu, Zhiyong Lu, Yian Ma, Mark Mueller, Radha Poovendran, Somayeh Sojoudi, Huaxiu Yao, Yinglun Zhu, Károly Zsolnai-Fehér, Yegor Denisov-Blanch, Changran Hu, Vincent Koc, Hezheng Yin, Naomi Xia, Xinyi Bai, Arsh Shah Dilbagi, Jing Gu, Yutong Liu, Lin Shi, Qizheng Zhang, Yuexing Hao, Xiaoming Li, Xiangyi Li, Bingran You, Wenjun Ke, Daoan Zhang, Ari Nemet, Tianyi Zhang, Xingda Lyu, Pranit Chawla, Genta Indra Winata, Maosen Tang, Purva Chiniya, Qiuyi Chen, Steven Dillmann, Wanghan Xu, Wenqi Yang, Xiaojun Wu, Xin Luo, Yue Huang, Dawn Song
    Conference on Neural Information Processing Systems (NeurIPS), Evaluations and Datasets Track, 2026.
    arXiv Project Page

  5. ICML-W
    FaultLoc: Evaluating Coding Agents for Fault Localization
    Jianhong Tu, Shubham Gaur, Rathik Murtinty, Zhun Wang, Tianneng Shi, Dawn Song, Chenguang Wang
    Second Workshop on Agents in the Wild: Safety, Security, and Beyond (AIWILD), ICML, 2026.
    PDF

  6. Under Review
    A Comprehensive Survey of Multimodal Foundation Model Evaluation: A Hierarchical Perspective and Extensive Applications
    Ye Yuan, Junyu Luo, Guancheng Wan, Jinsheng Huang, Chengwu Liu, Junwei Yang, Yifang Qin, Zhiping Xiao, Qingqing Long, Meng Xiao, Yiqiao Jin, Jianhong Tu, Yuqi Chen, Wei Ju, Zhongwei Wan, Yusheng Zhao, Xiao Luo, Yiwei Fu, Chenguang Wang, Ming Zhang
    Under review at Information Fusion, 2026.
    PDF

  7. ICML
    CyberGym-E2E: Scalable Real-World Benchmark for AI Agents' End-to-End Cybersecurity Capabilities
    Tianneng Shi, Robin Rheem, Dongwei Jiang, Mona Wang, Francisco De La Riega, Zhun Wang, Jingzhi Jiang, Alexander Cheung, Sean Tai, Jonah Cha, Jianhong Tu, Gabriel Han, Chenguang Wang, Jingxuan He, Wenbo Guo, Dawn Song
    International Conference on Machine Learning (ICML), 2026.
    PDF

  8. ACL Findings
    FICO: Evaluating Vision-Language Models under Visual Fidelity and Compression at Scale
    Jianhong Tu, Nicholas Crispino, Kyle Montgomery, Chenguang Wang, Dawn Song
    Findings of the Association for Computational Linguistics (ACL), 2026.
    PDF

  9. ACL-W
    MLAN: Language-Based Instruction Tuning Preserves and Transfers Knowledge in Multimodal Language Models
    Jianhong Tu, Zhuohao Ni, Nicholas Crispino, Zihao Yu, Michael Bendersky, Beliz Gunel, Ruoxi Jia, Xin Liu, Lingjuan Lyu, Dawn Song, Chenguang Wang
    3rd Workshop on Towards Knowledgeable Foundation Models (KnowFM), ACL, 2025.
    arXiv

  10. ACL-W
    Predicting Task Performance with Context-Aware Scaling Laws
    Kyle Montgomery, David Park, Jianhong Tu, Michael Bendersky, Beliz Gunel, Dawn Song, Chenguang Wang
    3rd Workshop on Towards Knowledgeable Foundation Models (KnowFM), ACL, 2025.
    arXiv

  11. Preprint
    Environmental complexity reveals memory-guided search as the locus of learning in prey capture
    Aidan M. Schneider, James N. McGregor, Mark Song, Jacob L. Amme, Samantha Zheng, Doran Wu, Jianhong Tu, Gavin Yao, Emmalee Eslinger, Jay Chitalia, Jack Powers, Varun Sinha, Eva L. Dyer, Daniel Levenstein, Keith B. Hengen
    Preprint, bioRxiv, 2026.
    PDF

  12. Neuron
    Failure in a population: Tauopathy disrupts homeostatic set-points in emergent dynamics despite stability in the constituent neurons
    James N. McGregor, Clayton A. Farris, Sahara Ensley, Aidan Schneider, Leandro J. Fosque, Chao Wang, Elizabeth I. Tilden, Yuqi Liu, Jianhong Tu, Halla Elmore, Keenan D. Ronayne, Ralf Wessel, Eva L. Dyer, Kiran Bhaskaran-Nair, David M. Holtzman, Keith B. Hengen
    Neuron, 2024.
    PDF Cover Paper

Experience

Honors and Awards

Service

Open Source

Teaching


Powered by Jekyll and Minimal Light theme.