TAAF: A Trace Abstraction and Analysis Framework Synergizing Knowledge Graphs and LLMs

Abstract

Execution traces are a critical source of information for understanding, debugging, and optimizing complex software systems. However, traces from OS kernels or large-scale applications like Chrome or MySQL are massive and difficult to analyze. Existing tools rely on predefined analyses, and custom insights often require writing domain-specific scripts, which is an error-prone and time-consuming task. This paper introduces TAAF (Trace Abstraction and Analysis Framework), a novel approach that combines time-indexing, knowledge graphs (KGs), and large language models (LLMs) to transform raw trace data into actionable insights. TAAF constructs a time-indexed KG from trace events to capture relationships among entities such as threads, CPUs, and system resources. An LLM then interprets query-specific subgraphs to answer natural-language questions, reducing the need for manual inspection and deep system expertise. To evaluate TAAF, the authors introduce TraceQA-100, a benchmark of 100 questions grounded in real kernel traces. Experiments across three LLMs and multiple temporal settings show that TAAF improves answer accuracy by up to 31.2%, particularly in multi-hop and causal reasoning tasks. The paper also analyzes where graph-grounded reasoning helps and where limitations remain.

Source: Official arXiv:2601.02632 abstract; public preprint version; related ACM DOI signal retained separately.

Problem and motivation

Kernel and large-application traces are too large, multidimensional, and temporally structured for direct manual inspection or raw LLM ingestion. Existing stateful tools support predefined queries, but flexible questions still require domain-specific scripts and expert knowledge.

Method and contribution

TAAF implements a three-layer trace-to-answer pipeline: LTTng events are transformed into a time-indexed Trace Compass-style State System; a natural-language query selects a compact temporal knowledge graph with typed entities, weighted relations, and time scope; a schema-conditioned LLM answers from the graph. The pipeline is evaluated with TraceQA-100, a benchmark of expert-authored questions over real kernel traces.

Findings and evidence

TraceQA-100 contains 100 questions in explanatory, multiple-choice, and true/false formats, split across single- and multi-hop reasoning. Across the core GPT-4.1 nano/GPT-4o/o4-mini grid and 1/10/100-second windows, graph grounding improves weighted accuracy by a reported mean 21.5 percentage points and up to 31.17 points; o4-mini with TAAF reaches 95.5% at 1 second and 90.17% at 100 seconds. Supplying the graph schema adds 8.1 points in the focused GPT-4o test.

Limitations and future directions

Limitations: The benchmark is hand-crafted from SciMark 2.0 traces under Linux and uses a three-level score, author labeling, three samples per configuration, API models, and a raw-State-System baseline only. Long windows, explanatory multi-hop arithmetic, global queries, model drift, and other kernels/workloads remain difficult. The methods section accounts for 5,400 Phase-1 plus 2,100 Phase-2 outputs (7,500), while the conclusion says 7,800; preserve this as an unresolved paper-internal inconsistency.

Future work: Broaden question generation and annotation, use finer numeric-tolerance metrics and independent raters, support temporal/production-scale traces, hierarchical or retrieval-based prompting, autonomous-agent integration, and optimize KG construction with partitioning, streaming, batching, and early exit.

Sources and identifiers

• Publication DOI listed on arXiv page publication_signal
• Public preprint · PDF public_full_text
• arXiv version arXiv 2601.02632

When to cite this paper

Cite this paper when your work uses or compares the State System -> query-specific temporal knowledge graph -> LLM architecture for grounded kernel-trace question answering.

• the State System -> query-specific temporal knowledge graph -> LLM architecture for grounded kernel-trace question answering.
• TraceQA-100, including expert-authored temporal, multi-entity, numeric, single-hop, and multi-hop questions over SciMark 2.0 kernel traces.
• the measured benefit of graph grounding, especially the reported +21.5-point mean gain and up-to +31.17-point gain over raw State-System output.
• the schema and temporal-window ablations when discussing how explicit structure and shorter windows affect LLM trace reasoning.

Citation

@misc{ezzatiJivan2026taafa,
  author = {Alireza Ezaz and Ghazal Khodabandeh and Majid Babaei and Naser Ezzati-Jivan},
  title = {TAAF: A Trace Abstraction and Analysis Framework Synergizing Knowledge Graphs and LLMs},
  year = {2026},
  howpublished = {arXiv / CoRR; arXiv page reports acceptance to ICSE 2026},
  volume = {abs/2601.02632},
  doi = {10.1145/3744916.3787832},
  url = {https://doi.org/10.1145/3744916.3787832}
}

Ezaz, A., Khodabandeh, G., Babaei, M., & Ezzati-Jivan, N. (2026). TAAF: A Trace Abstraction and Analysis Framework Synergizing Knowledge Graphs and LLMs. arXiv / CoRR; arXiv page reports acceptance to ICSE 2026. https://doi.org/10.1145/3744916.3787832

A. Ezaz, G. Khodabandeh, M. Babaei, and N. Ezzati-Jivan, "TAAF: A Trace Abstraction and Analysis Framework Synergizing Knowledge Graphs and LLMs," in arXiv / CoRR; arXiv page reports acceptance to ICSE 2026, vol. abs/2601.02632, 2026, doi: 10.1145/3744916.3787832