Enhancing empirical software performance engineering research with kernel-level events: A comprehensive system tracing approach

Problem and motivation

Application-level metrics can show degradation without exposing scheduling, I/O, network, memory, and interrupt behavior. Performance-engineering research therefore needs realistic trace data and reproducible noise conditions rather than isolated benchmark numbers.

Method and contribution

The artifact records Linux kernel events and system calls in an Elasticsearch/Kibana-oriented workflow, with light- and heavy-load scenarios and CPU, I/O, network, and memory noise injection. The paper describes use for performance monitoring, noise detection/root-cause analysis, and software-phase detection. The evaluated environment includes Ubuntu 22.04, OpenJDK 18, LTTng 2.13.9, and Trace Compass 8.2.0; the collected artifact contains 24,263,691 events in a 924 MB trace, with sched_switch representing approximately 16.2% of events. Workloads use stress-ng CPU matrix, I/O, socket, and memory options, plus Edge-IIoTset-derived activity.

Findings and evidence

The artifact provides light/heavy execution conditions and several resource-noise dimensions that can be reused to test monitoring, anomaly/root-cause analysis, and phase-detection methods. The paper's detailed counts make the collection concrete, but the artifact should be cited as an evidence/benchmark resource rather than as proof that one detector dominates alternatives. The paper's setup references Elasticsearch 6 in one place while linked materials reference 8.8; this is a reproducibility issue worth retaining.

Limitations and future directions

Limitations: The workloads and noise injections represent selected Linux conditions and may not cover other kernels, hardware, applications, or production interference. Event volume and instrumentation overhead require independent measurement for each use case, and the artifact does not by itself provide ground-truth root causes for every event sequence.

Future work: Add heterogeneous hardware and kernels, richer ground-truth labels, standardized trace schemas, controlled overhead studies, container/orchestrator workloads, and reproducible baseline analyses that quantify how much the artifact improves generalization.

Resources

• Data

Sources and identifiers

• Published version published

When to cite this paper

Cite this paper when you need a reusable Linux kernel-event artifact with injected resource noise for performance-engineering evaluation.

• The light/heavy workload and CPU/I/O/network/memory noise design.
• The 24,263,691-event, 924 MB artifact and its analysis applications.
• The linked dataset repository and reproducibility materials.
• The selected Linux environment and version-consistency caveats.

Citation

@article{ezzatiJivan2024enhancingempirical,
  author = {Morteza Noferesti and Naser Ezzati-Jivan},
  title = {Enhancing empirical software performance engineering research with kernel-level events: A comprehensive system tracing approach},
  year = {2024},
  journal = {Journal of Systems and Software},
  volume = {216},
  pages = {112117},
  eid = {112117},
  publisher = {Elsevier BV},
  issn = {0164-1212},
  doi = {10.1016/j.jss.2024.112117},
  url = {https://doi.org/10.1016/j.jss.2024.112117}
}

Noferesti, M., & Ezzati-Jivan, N. (2024). Enhancing empirical software performance engineering research with kernel-level events: A comprehensive system tracing approach. Journal of Systems and Software, 216, 112117. https://doi.org/10.1016/j.jss.2024.112117

M. Noferesti and N. Ezzati-Jivan, "Enhancing empirical software performance engineering research with kernel-level events: A comprehensive system tracing approach," Journal of Systems and Software, vol. 216, Art. no. 112117, 2024, doi: 10.1016/j.jss.2024.112117