Thinking inside the box : the promise and boundaries of transparency in automated decision-making

Abstract

The normative attractivity of transparency is beyond compare. No wonder it is one of the main principles in the EU’s General Data Protection Regulation. It also features in a majority of AI ethics codes. Transparency is called for because it is assumed that it will solve the so-called ‘black box problem’ (uncertainty about how inputs translate into outputs in algorithmic systems) and by so doing legitimize automated decision-making (computer-based decisionmaking without human influence; ADM). In this paper, the legitimizing effect of transparency in ADM is discussed. I argue that transparency cannot deliver in its quest to resolve the black box problem. The main claim is that transparency is inherently performative in nature and cannot but be so. This performativity goes against the promise of unmediated visibility, vested in transparency. As demonstrated, when transparency is brought into the context of ADM, its hidden functioning logic becomes visible in a new way.