🎓 I am on the job market! I am looking for full-time research scientist / applied scientist positions in industry, starting fall 2027. Feel free to reach out at qianqi@ucsb.edu.

I am Qianqi “Jackie” Yan, a Ph.D. candidate in Computer Science @University of California, Santa Barbara, advised by Prof. Xin (Eric) Wang as a member of the UC Santa Barbara AI Group. I have interned as an Applied Scientist at Amazon, Oracle Health AI, and eBay.

I obtained my bachelor’s degrees from the University of Michigan, Ann Arbor (B.S.E. in Computer Science, Summa cum laude) and Shanghai Jiao Tong University (B.S.E. in Electrical and Computer Engineering), where I worked with Prof. Joyce Chai and Prof. Stella Yu.

Research Interests

My research focuses on trustworthy multimodal LLMs and agents. I build benchmarks and attribution methods to measure and mitigate failures of modern multimodal models, including:

  • Hallucination & faithfulness: probing evaluations and inconsistency/underspecification reasoning for multimodal LLMs (ProbMed, MMIR, iReason)
  • Attribution & auditing: training-free, generation-time evidence attribution across text, image, audio, and video (OmniTrace, clinical summarization auditing)
  • Agent safety: deceptive behavior of tool-using LLM agents under pressure (AgentDecept) and agent harness safety
  • Generative media forensics & understanding: forgery detection for diffusion-based document edits, and locating reasoning in image/video generators

Please don’t hesitate to email me if you are interested in these areas. I am happy to chat ;)

News

  • [2026.11] 🎓 I am on the job market for full-time research/applied scientist roles starting fall 2027!
  • [2026.09] Finished my Applied Scientist internship at Amazon, working on generative document forgery detection. Work submitted to CVPR 2027.
  • [2026.09] I will be co-organizing the 5th ALVR workshop at ACL 2027.
  • [2026.01] PhyWorldBench accepted to ICLR 2026 as an Oral presentation!
  • [2025.09] I will be hosting the 4th ALVR workshop at ACL 2026.
  • [2025.08] One paper accepted to EMNLP 2025! I will be attending EMNLP 2025 in Suzhou, China. Please feel free to reach out for a chat!
  • [2025.05] Two papers accepted to ACL 2025!
  • [2025.04] I will join Oracle Health AI as an applied scientist intern in 2025 summer.
  • [2024.04] I will join eBay as an applied scientist intern in 2024 summer.

Experience

Amazon logo
Amazon, Applied Scientist Intern
San Diego, CA · Jun. 2026 – Sep. 2026
Paired forgery synthesis pipeline and factorized benchmark for detecting diffusion-based text edits in document images.
Oracle logo
Oracle Health AI, Applied Scientist Intern
Santa Clara, CA · Jun. 2025 – Sep. 2025
Training-free, attention-guided attribution for multimodal clinical summarization; up to 15% citation F1 gain. Published at ML4H 2025 and filed as a U.S. patent.
eBay logo
eBay, Applied Scientist Intern
San Jose, CA · Jun. 2024 – Sep. 2024
Multimodal retrieval combining image embeddings with fine-grained title representations; +12.1% MAR@1 and +26.0% MAR@10, robust to single-modality dropout.

Preprints & Under Review

  • AgentDecept: Benchmarking Deceptive Behavior in Tool-Using LLM Agents under Pressure
    Qianqi Yan, Yehan Zhang, Hang Yin, Ching-Chen Kuo, Shan Jiang, Ming Ouyang, Yijie Liu, Xin Eric Wang
    Under review

  • OmniTrace: A Unified Framework for Generation-Time Attribution in Omni-Modal LLMs
    Qianqi Yan, Yichen Guo, Ching-Chen Kuo, Shan Jiang, Hang Yin, Yang Zhao, Xin Eric Wang
    Under review

  • Do Generators Think in Pixels? Locating Reasoning in Image and Video Generators
    Qianqi Yan, Jinru Han, Kaizhi Zheng, Yuhang Gan, Xin Eric Wang
    Under review

  • DocTwin: A Paired, Factorized Dataset for Generative Document Forgery Detection
    Qianqi Yan, Richard Y. Chen, Chi Zhang, Daniel Cociorva, Xin Eric Wang
    Under review

  • Auditing Agent Harness Safety
    Chengzhi Liu, Yichen Guo, Yepeng Liu, Yuzhe Yang, Qianqi Yan, et al.
    Under review

  • Skill-CMIB: Multimodal Agent Skill for Consistent Action via Conditional Multimodal Information Bottleneck
    Zihan Huang, Junda Wu, Tong Yu, Qianqi Yan, et al.
    Under review

Publications

  • PhyWorldBench: A Comprehensive Evaluation of Physical Realism in Text-to-Video Models
    Jing Gu, Xian Liu, Yu Zeng, Ashwin Nagarajan, Fangrui Zhu, Daniel Hong, Yue Fan, Qianqi Yan, Kaiwen Zhou, Ming-Yu Liu, Xin Eric Wang
    ICLR, 2026 Oral

  • From Attribution to Abstention: Training-Free Attention-Based Auditing for Clinical Summarization
    Qianqi Yan, Huy V. Nguyen, Sumana Srivatsa, Hari Bandi, Xin Eric Wang, Krishnaram Kenthapadi
    ML4H 2025

  • Hidden in Plain Sight: Reasoning in Underspecified and Misspecified Scenarios for Multimodal LLMs
    Qianqi Yan, Hongquan Li, Shan Jiang, Yang Zhao, Xinze Guan, Ching-Chen Kuo, Xin Eric Wang
    EMNLP 2025

  • Multimodal Inconsistency Reasoning (MMIR): A New Benchmark for Multimodal Reasoning Models
    Qianqi Yan, Yue Fan, Hongquan Li, Shan Jiang, Yang Zhao, Xinze Guan, Ching-Chen Kuo, Xin Eric Wang
    ACL 2025

  • Worse than Random? An Embarrassingly Simple Probing Evaluation of Large Multimodal Models in Medical VQA
    Qianqi Yan, Xuehai He, Xiang Yue, Xin Eric Wang
    ACL 2025

  • Muffin or Chihuahua? Challenging Large Vision-Language Models with Multipanel VQA
    Yue Fan, Jing Gu, Kaiwen Zhou, Qianqi Yan, Shan Jiang, Ching-Chen Kuo, Xinze Guan, Xin Eric Wang
    ACL 2024

  • Med-HVL: Automatic Medical Domain Hallucination Evaluation for Large Vision-Language Models
    Qianqi Yan, Xuehai He, Xin Eric Wang
    AAAI 2024 Spring Symposium on Clinical Foundation Models

Service

Organizer

Editor

  • Proceedings of the 4th Workshop on Advances in Language and Vision Research (ALVR), ACL 2026

Reviewer

  • Main Conference: NeurIPS 2026, ICLR 2024-2026
  • Workshop: ALVR @ACL 2024, CLVL @ICCV 2023

Education

  • Ph.D. in Computer Science, University of California, Santa Barbara, 2025 - Jun. 2027 (expected)
  • Ph.D. in Computer Science, University of California, Santa Cruz, 2023 - 2025 (transferred with advisor)
  • B.S.E. in Computer Science, Summa cum laude, University of Michigan (Ann Arbor), 2021 - 2023
  • B.S.E. in Electrical and Computer Engineering, Shanghai Jiao Tong University, 2019 - 2023
  • Visiting Student, Technische Universität Berlin, 2020

Teaching

  • CS165A: Artificial Intelligence, University of California, Santa Barbara
  • CSE101: Introduction to Data Structures and Algorithms, University of California, Santa Cruz
  • EECS 448: Human-Centered Machine Learning, University of Michigan, Ann Arbor
  • VE 215: Intro to Circuits, Shanghai Jiao Tong University
  • VG 101: Intro to Computers and Programming, Shanghai Jiao Tong University