Multi-Dimensional Bias Analysis in LLMs Using Hierarchical and Interaction Models

Catalog abstract summary

The official abstract describes a five-layer framework for analyzing interacting bias dimensions in large language models, with ChatGPT and Gemini used as illustrative systems and context-dependent ranking of bias dimensions.

Source: Author-supplied five-page PDF reviewed locally; the complete abstract is not reproduced in the public catalog because reuse permission for the conference paper was not established.

Problem and motivation

One-dimensional bias checks and static fairness metrics can miss dependencies between data, model behavior, generated outputs, deployment context, user interaction, and societal feedback. Without a layered representation, it is difficult to trace a biased output back to possible origins or to reason about intersectional amplification.

Method and contribution

The conceptual framework separates five causally linked layers: Data Bias, Algorithmic Bias, Surface-Level Bias, Operational Bias, and Societal Influence. It models direct propagation between layers, cascading multi-step effects, feedback loops, and cross-dimensional interactions such as gender and ethnicity. Proposed measurements include representation ratios and stereotype frequency, feature-importance or attention distributions, sentiment and stereotype-polarity scores, engagement metrics, social-network or echo-chamber analysis, Pearson and Spearman correlations, mutual information, Granger-causality and structural-equation models. The paper aggregates layer scores and interaction effects as B(S) = sum(alpha_i L_i) + I(S), initially uses equal layer weights, normalizes the final score to a 0-10 scale, and discusses sensitivity analysis and k-fold validation. Its preliminary case study uses the LLMBI tool with ChatGPT/GPT-4 and Gemini 1.5, four bias dimensions, 18 neutral/ambiguous prompts per category, temperature adjustment, instruction guiding, and 100 iterations per scenario.

Findings and evidence

The preliminary table reports ChatGPT scores of -1.6 for gender, -11.6 for nationality, -4.9 for ethnicity, and -9.3 for religion, while Gemini 1.5 scores are +41.2, +40.0, +39.7, and +44.8 for the same dimensions. The paper interprets positive values as stronger stereotype alignment and negative values as reduced or counter-stereotypical tendencies under its scoring convention. The results show distinct profiles in this small case study and illustrate how the proposed framework can present multiple dimensions together; they do not establish that one model is universally less biased or that the proposed layers identify causal effects.

Limitations and future directions

Limitations: This is an Emergent Research Forum paper and the framework is primarily conceptual with preliminary two-model validation. The evaluation uses LLMBI prompts rather than a broad released benchmark, covers four dimensions and two model families, and does not provide a detailed independent ground-truth or causal-identification protocol. The paper mentions equal weights, sensitivity analysis, and k-fold validation but does not give enough procedural detail to reproduce a complete calibrated ranking system from the paper alone. Prompt wording, model versions, stochastic settings, and the interpretation of negative scores can affect the reported values.

Future work: The paper proposes evaluation across more LLMs, cultural contexts, and intersectional scenarios; comparison with StereoSet and Wino-Bias; tracking bias across model versions; qualitative case studies; user evaluations of perceived fairness; and additional dimensions including age, disability, and socioeconomic background. It also identifies layer-specific mitigation as a future extension.

Sources and identifiers

• Institutional publication record publication_signal
• Official abstract page public_abstract

When to cite this paper

Cite this paper when framing LLM bias as a layered, interacting, and feedback-sensitive phenomenon rather than a single scalar fairness score.

• Five-layer decomposition spanning data, algorithmic, surface-level, operational, and societal influences.
• Direct propagation, cascades, feedback loops, and cross-dimensional/intersectional interactions.
• The proposed layer/interactions score and the LLMBI case study comparing GPT-4 and Gemini 1.5.
• Preliminary evaluation limits and future use of StereoSet, Wino-Bias, broader models, and cultural contexts.

Citation

@inproceedings{ezzatiJivan2025multidimensional,
  author = {Basil Syed and Daniel Arana Charlebois and Naser Ezzati-Jivan and Leila Tahmooresnejad and Anteneh Ayanso},
  title = {Multi-Dimensional Bias Analysis in LLMs Using Hierarchical and Interaction Models},
  year = {2025},
  booktitle = {AMCIS 2025, Data Science / SIG DSA (ERF)},
  url = {https://aisel.aisnet.org/amcis2025/data_science/sig_dsa/15/}
}

Syed, B., Charlebois, D. A., Ezzati-Jivan, N., Tahmooresnejad, L., & Ayanso, A. (2025). Multi-Dimensional Bias Analysis in LLMs Using Hierarchical and Interaction Models. In AMCIS 2025, Data Science / SIG DSA (ERF). https://aisel.aisnet.org/amcis2025/data_science/sig_dsa/15/

B. Syed, D. A. Charlebois, N. Ezzati-Jivan, L. Tahmooresnejad, and A. Ayanso, "Multi-Dimensional Bias Analysis in LLMs Using Hierarchical and Interaction Models," in AMCIS 2025, Data Science / SIG DSA (ERF), 2025, [Online]. Available: https://aisel.aisnet.org/amcis2025/data_science/sig_dsa/15/