Preserving procedural fairness in the AI era : the role of courts before and after the AI Act

Abstract

In the past years, scholars and civil society organisations have raised concerns regarding the use of automated systems in public decision-making. The child benefit scandal, which led the Dutch government to resign in 2021, is just an example of poor human decision-making aided by flawed automated systems. In the same year, the European Commission proposed a Regulation for AI systems, the so-called “AI Act”, to address the challenges and concerns raised by the increasing use of AI while encouraging businesses to develop them in the EU single market. While the proposal is still under discussion in the European Parliament, AI systems have been used and challenged by individuals affected by their output. In the absence of a regulatory framework, national courts in Europe have been called upon to address claimants’ demands for fairness and legal protection. This post investigates the role of courts in preserving procedural fairness in the era of AI. I focus on six cases where automated systems were used to inform public decision-making to detect tax and social welfare fraud (eKasa case and Syri case), to grant permission for agricultural businesses (AERIUS case), and to recruit and transfer high school teachers (Altomare and C.P. cases) in Slovakia, Italy and in the Netherlands. I show how national courts have been activists in preserving individuals’ procedural rights by setting requirements for AI systems. Their role is, however, doomed to change when the AI Act enters into force.