OmniTrace

A Unified Framework for Generation-Time Source Attribution in Omni-Modal LLMs

Qianqi Yan1 Yichen Guo1 Ching-Chen Kuo2 Shan Jiang2 Hang Yin2 Yang Zhao2 Xin Eric Wang1
1University of California, Santa Barbara  ·  2eBay
Oct 2026 · v2. New method (attention mass over source units), a causal test for audio and video, and three models. The code and the paper are updated.
OmniTrace overview: video, audio and document inputs are cut into source units; while the decoder writes a sentence, OmniTrace sums attention mass on each unit and labels the units that stand out; right: sentences with their sources, and faithfulness against extra forward passes.
Left: video, audio, and documents enter one input sequence as source units: seconds, passages, and images. While the decoder writes a sentence, OmniTrace sums the attention mass on each unit (bars) and labels the units that stand out within their modality (orange). Top right: every sentence comes with its sources. Here most labelled seconds fall inside the span that annotators marked, although no annotation was used. Bottom right: under a causal test, OmniTrace reaches 77–96% of the effect of leave-one-out with no extra forward pass. Perturbation needs 16 to 41 passes per question.
0
extra forward passes. Perturbation methods need 16–41 per question.
77–96%
of leave-one-out's causal effect, from the generation pass alone.
9 / 9
model × modality settings where the selected units beat random units at every budget.
0.83
AUROC against human-marked document evidence. The best baseline reaches 0.76.
The problem

What did the model actually use?

Omni-modal models answer questions about long videos, recordings, and documents, but they do not show where an answer came from. Two ways to find out exist today, and they answer different questions.

Ask

Prompt the model to cite

  • Cheap: one more generation.
  • What it says it used. Qwen2.5-Omni cites a valid span in only 31% of video and 38% of audio answers.
  • Hiding OmniTrace's audio seconds costs 3.1× more than hiding the same number of cited ones. Asking for citations also drops video accuracy from 72% to 55%.
Measure

Remove parts, rerun

  • What it used. Leave-one-out and ContextCite.
  • Costly: 16–41 extra forward passes per question, each over the full clip.
  • Defined for text. On audio and video they need a way to hide a second without shifting the rest.
Read (OmniTrace)

Read it from the generation pass

  • Free: no extra pass, no training. Every sentence is attributed while it is written.
  • Faithful: 77–96% of leave-one-out's effect under the same causal test.
  • Any modality: seconds of video and audio, passages, images. One setting for three models.
Method

Evidence in continuous inputs is diffuse. Read it as mass, not peaks.

Every generated token attends over the whole input, so the generation pass already holds a distribution over the sources for every token. The question is how to read it once the sources are seconds of sound and video rather than sentences.

Cut the input into source units

Use the structure the input already has: about one second per unit for video and audio, following the encoder's own time grid, and one unit per text passage or image. The question, the options, and the template are never candidates.

Sum late-layer attention mass per unit

Average attention over all heads of the last third of the decoder layers. A unit's score for a sentence is the attention mass the sentence's tokens put on it. Scores add up, so one pass gives sentence- and response-level attributions.

sk(Sj) = Σt∈Ck Σi∈Sj at(i)

Label the units that stand out

A unit is labelled when its score is ≥ 1.5× the mean of the other units of its own modality, so text cannot crowd out images. Flat scores give no label, and the sentence abstains. Time units merge into intervals.

Optional: OmniTrace-trained

Where ablation data are available, learn one weight per attention head. It is still a single pass at test time.

Illustration with synthetic numbers. On 1,000 NExT-GQA dev questions, ranking seconds by argmax votes agrees with human evidence at chance (AUROC 0.50–0.52); mass reaches 0.60.
Interactive examples

See what each sentence relied on

Pick a modality and a model, then click a sentence. Cells are source units, and darker cells hold more attention mass. Orange outlines are the units OmniTrace labels. Click a cell to play the clip from there.

Input
Model

These examples were selected (all three models answer correctly). The scores, labels, and causal numbers are the unedited outputs of the runs in the paper. Video: NExT-GQA, frames only. Audio: MMAU. Documents: MMDocRAG, 15 quotes.

Results

Tested, not assumed

We hide units at the encoder input behind content-free placeholders of the same length (black frames, silence, a black image, blank tokens), replay the model's own response, and measure how much its log-probability drops. Each method is compared with hiding the same number of random units. The same operation lets leave-one-out and ContextCite run on audio and video, so this is a direct comparison of single-pass and perturbation attribution on continuous inputs.

The selected units matter, everywhere

On all three models and all three modalities, hiding OmniTrace's units hurts more than hiding random units at every budget. This also holds for single sentences: a sentence's units matter for that sentence more than for the others.

Most of the faithfulness, none of the cost

From one pass, OmniTrace reaches 96% of leave-one-out's lift on video, 77% on audio, and 95% on documents, and it has about twice the lift of embedding similarity.

Stated evidence is not used evidence

On all three models, the audio seconds a model cites itself matter less than the ones OmniTrace finds. On Qwen2.5-Omni, citations are often missing, sometimes copy the prompt's example span, and asking for them changes the answer.

One setting, three models

The setting was fixed on Qwen2.5-Omni video and carried unchanged to MiniCPM-o 4.5 and Qwen3-Omni-30B. Where the default falls short (MiniCPM-o documents), trained head weights recover 96% of leave-one-out's lift, still in one pass.

Drop in log-probability when OmniTrace's units vs random units are hidden, per modality and model
Hiding what OmniTrace selects (orange) hurts the response more than hiding random units of the same count (blue), in every modality, at every budget, and on every model. Solid: Qwen2.5-Omni-7B. Dashed: MiniCPM-o 4.5. Dotted: Qwen3-Omni-30B. Bands are 95% bootstrap intervals.
VideoAudioImage–text
MethodFwdLiftS-liftFwdLiftS-liftFwdLiftS-lift
Baselines on Qwen2.5-Omni-7B
Embedding similarity00.0080.00900.0340.03100.2160.216
Self-citation10.0130.01210.0400.03610.1840.349
Leave-one-out*410.0160.024170.1000.133160.3680.404
ContextCite*330.0160.027280.0990.139330.3890.397
OmniTrace on Qwen2.5-Omni-7B
OmniTrace00.0160.01500.0770.11000.3510.376
OmniTrace-trained00.0140.01600.0810.11600.3740.385
OmniTrace on MiniCPM-o 4.5
OmniTrace00.0240.02700.1360.10200.1050.159
OmniTrace-trained00.0670.06500.1690.12000.4320.706
OmniTrace on Qwen3-Omni-30B-A3B
OmniTrace00.0330.03200.1120.09500.3160.448
Fwd: extra forward passes per question. Lift: extra drop in the response's log-probability (nats/token) when a method's top units are hidden, compared with the same number of random units (20% of the duration for video and audio, 2 of 15 quotes for documents). S-lift: the same for single sentences. Bold: largest per column. *Defined for text in the original work. Here they run on audio, video, and images with OmniTrace's units and placeholders.
Lift as a share of leave-one-out against extra forward passes
Faithfulness against cost: each method's lift as a share of leave-one-out's, against the extra forward passes it needs. Both OmniTrace variants sit at zero.
Share of responses that cite a source, and lift of cited units vs OmniTrace's units
The model's own citations. Left: share of responses that cite at least one valid source when asked to (OmniTrace attributes every response). Right: lift of the cited units and of OmniTrace's units of the same count.
Agreement with human evidence (Qwen2.5-Omni-7B)Video AUROCVideo Hit@1Docs AUROCDocs Hit@1
Random0.5000.2150.5000.194
Embedding similarity0.6040.2880.7630.527
Leave-one-out0.5490.3090.7210.657
ContextCite0.5660.3420.7430.642
OmniTrace0.6370.2300.8290.698
OmniTrace-trained0.6260.3210.8150.695
AUROC: probability that an evidence unit is ranked above a non-evidence unit. Hit@1: the top unit is evidence. On Qwen3-Omni-30B, for correctly answered document questions, OmniTrace's AUROC is 0.91 and its labels match the gold quotes with an F1 of 0.61, against 0.39 for the model's own citations.
Code

Attribute any answer in a few lines

Supports Qwen2.5-Omni, MiniCPM-o 4.5, and Qwen3-Omni-30B-A3B. Any attribution method that ranks the same units can be scored with the same causal test.

# install (one environment per model family)
git clone https://github.com/UCSB-AI/OmniTrace.git
cd OmniTrace
pip install -r requirements.txt
python -m spacy download en_core_web_sm

# the three examples on this page
python scripts/demo.py

# paper runs + causal test
bash scripts/run_mmau.sh full
bash scripts/run_mmau.sh causal
from omnitrace import OmniTrace

ot = OmniTrace("qwen2.5-omni-7b")
out = ot.trace({
  "question": "What sport is the first speaker getting a scholarship for?",
  "options": ["Basketball", "Soccer", "Baseball", "Softball"],
  "sources": [{"sid": "a0", "modality": "audio",
               "path": "examples/media/427e55eb-4ab4-4bb0-8c8f-574f10097087.wav"}],
})
for s in out["sentences"]:
    print(s["text"], s["spans"] or s["sources"])

BibTeX

@misc{yan2026omnitrace,
  title         = {OmniTrace: A Unified Framework for Generation-Time Source Attribution in Omni-Modal LLMs},
  author        = {Qianqi Yan and Yichen Guo and Ching-Chen Kuo and Shan Jiang and Hang Yin and Yang Zhao and Xin Eric Wang},
  year          = {2026},
  eprint        = {2604.13073},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2604.13073}
}