For their research in the field of artificial intelligence (AI), Kevin Bauer, Oliver Hinz, and Moritz von Zahn have been awarded the Information Systems Research 2023 Best Paper Award. Their paper, titled Expl(AI)ned: The Impact of Explainable Artificial Intelligence on Users’ Information Processing, published in 2023, investigates the influence of feature-based explanations provided by AI systems on users’ cognitive processing and decision-making.
The authors conducted two empirical studies to explore this impact. The first study involved laypeople making decisions in an abstract investment game, while the second focused on real estate experts estimating apartment prices. The results demonstrated that providing explanations significantly alters how users process information, leading to adjustments in their mental models. However, these adjustments are subject to confirmation bias, which can result in persistent misconceptions, biased decisions, and even unintended spillover effects in other areas of decision-making.
This research emphasizes the potential risks associated with the widespread adoption of explainable AI, such as the manipulation of user behaviour and the reinforcement of discriminatory tendencies. These findings are essential for companies and regulators as they navigate the challenges of implementing and regulating AI systems.
Further information about the paper can be found here.