Problem Statement and Scholarly Relevance

The increasing integration of AI systems into organizational decision-making processes marks a profound shift in the design of sociotechnical systems. While technological capabilities continue to advance, the scholarly grounding of governance structures and algorithmic responsibility (accountability) remains underdeveloped in many respects. In this context, algorithmic responsibility is understood as the traceability, assignability, and controllability of algorithmic decisions.

For the information systems discipline, this gives rise to a central research question: How can human oversight and algorithmic autonomy be configured in hybrid decision architectures such that efficiency gains are realized without violating regulatory and ethical requirements? From a behavioral IS perspective in particular, this entails examining how individual and organizational behavior patterns, attitudes, and trust shape the use and control of AI-based decision systems.

The disciplinary relevance is substantial, as information systems research has traditionally occupied the interface between technological innovation and organizational value creation. Whereas prior work has primarily focused on efficiency gains, the debate is increasingly shifting toward transparency, explainability (Explainable AI, XAI), and risk control in sensitive application domains such as financial services and public administration Rai, 2020.

Theoretical Concepts and Current Debates

At the core of behavioral IS research on AI systems lies the tension between algorithmic efficiency and human agency. Established adoption and use models such as the Technology Acceptance Model (TAM) and its extensions (e.g., UTAUT2) provide a structured lens for analyzing perceived usefulness, expected use benefits, trust, and habit as drivers of system use Davis, 1989; Venkatesh et al., 2012. For AI-based decision systems, however, a purely acceptance-oriented perspective is insufficient, as perceptions of control, opportunities to intervene, and attributions of responsibility must also be modeled.

An increasingly influential guiding concept in this context is meaningful human control, which specifies the conditions under which autonomous systems remain subject to accountable human control Santoni de Sio & van den Hoven, 2018. The notions of tracking and tracing enable normative requirements regarding control and responsibility to be integrated with behavioral constructs such as perceived control, willingness to assume responsibility, and trust.

In parallel, XAI research emphasizes that interpretability is not only a technical property but also a behaviorally relevant feature that shapes attitudes and usage patterns Rai, 2020; Doshi-Velez & Kim, 2017. For behavioral IS, this creates the challenge of empirically linking constructs such as trust, perceived risk and fairness, and responsibility attribution to concrete XAI features and governance mechanisms.

Methodological Approaches and Empirical Research

Investigating algorithmic responsibility from a behavioral IS perspective requires a methodological repertoire that captures both individual and organizational behavior patterns and technical artifacts:

• Quantitative surveys and experiments: Causal and correlational analyses of the effects of XAI features and governance mechanisms on trust, intention to use, perceived control, and responsibility attribution.
• Qualitative studies: In-depth analyses of how organizations engage with AI systems through interviews and observations, in order to uncover informal practices of oversight, delegation, and responsibility attribution.
• Design Science Research (DSR): Systematic development and evaluation of artifacts (e.g., governance dashboards, decision logs, audit mechanisms) that operationalize “accountability by design.”

Within IS research, DSR is understood as a research paradigm that generates knowledge through the construction and evaluation of innovative artifacts; such artifacts may take the form of constructs, models, methods, or implemented instantiations Hevner et al., 2004. For doctoral projects on algorithmic responsibility, this implies that a governance dashboard, an XAI explanation module, or an accountability framework should be conceived as explicitly theory-based artifacts whose design decisions can be traced back to behavioral IS theories (e.g., TAM/UTAUT, trust, risk, control) and normative requirements (e.g., meaningful human control, AI governance principles).

The design science research methodology proposed by Peffers et al. provides a procedural framework that systematically links problem identification, objective definition, design and development, demonstration, evaluation, and communication Peffers et al., 2007. A methodologically rigorous DSR study on algorithmic responsibility should therefore specify which type of artifact is being developed, which requirements are derived from behavioral IS theory and practice, and which evaluation forms—such as experiments, case studies, simulations, or analytical assessments—are used to judge the artifact’s quality Hevner et al., 2004; Peffers et al., 2007.

Mixed-methods approaches, which combine data science techniques (e.g., logfile analyses of actual usage patterns) with social science methods (surveys, experiments, qualitative analyses), are particularly promising for empirically capturing the gap between technical system logic and observed behavior in organizations Doshi-Velez & Kim, 2017.

Challenges and Implications for Research

A persistent core challenge is the “black box” nature of many AI models. Despite advances in XAI, the implementation of legally robust yet practically feasible transparency requirements remains limited, with direct consequences for trust, responsibility attribution, and usage patterns Rai, 2020; Doshi-Velez & Kim, 2017. At the same time, regulatory developments—most notably the EU AI Act—increase pressure on organizations to implement robust governance and documentation mechanisms European Union, 2024.

Against this backdrop, several key research questions arise that are closely aligned with behavioral IS and DSR:

• How do governance structures and XAI features shape trust, perceived control, and responsibility attribution across different user groups?
• Which artifacts (e.g., dashboards, explanation interfaces, audit trails) are suitable for making algorithmic responsibility operational in organizational contexts, and how can they be developed and evaluated in line with DSR principles?
• How can ethical principles and regulatory requirements be translated into behaviorally grounded design requirements and embedded in technically implementable decision logics?

Implications for Research Practice

Algorithmic responsibility is emerging as a core research area in information systems, in which behavioral IS and DSR offer complementary perspectives. Behavioral IS approaches enable theory-driven analyses and explanations of attitudes, perceptions, and usage patterns in relation to AI systems, while DSR advances the design and evaluation of artifacts that operationalize accountability, transparency, and control in practice Hevner et al., 2004; Peffers et al., 2007.

For doctoral researchers, this implies designing research projects that systematically link behavioral theories (e.g., TAM/UTAUT, trust, risk, control) with the development of governance and XAI artifacts. It is crucial not only to analyze algorithmic models themselves, but also to critically investigate their sociotechnical embedding and the associated behavioral and governance processes, and to actively shape these through the design and evaluation of artifacts Santoni de Sio & van den Hoven, 2018; European Union, 2024.

References

• Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340.
• Venkatesh, V., Thong, J. Y. L., & Xu, X. (2012). Consumer acceptance and use of information technology: Extending the unified theory of acceptance and use of technology. MIS Quarterly, 36(1), 157–178.
• Rai, A. (2020). Explainable AI: From black box to glass box. Journal of the Academy of Marketing Science, 48(1), 137–141.
• Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608.
• Santoni de Sio, F., & van den Hoven, J. (2018). Meaningful human control over autonomous systems: A philosophical account. Frontiers in Robotics and AI, 5, 15.
• Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design science in information systems research. MIS Quarterly, 28(1), 75–105.
• Peffers, K., Tuunanen, T., Rothenberger, M. A., & Chatterjee, S. (2007). A design science research methodology for information systems research. Journal of Management Information Systems, 24(3), 45–77.
• European Union. (2024). Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union.