Reclaiming transparency: contesting the logics of secrecy within the AI Act

Reclaiming transparency: contesting the logics of secrecy within the AI Act

Abstract

Transparency is widely acknowledged as a core value in the governance of artificial intelligence technologies. However, scholarship on AI technologies and their regulation often casts this need for transparency in terms of requirements for the explanation of algorithmic outputs and/or decisions produced with the involvement of opaque black-box AI systems. Our Article argues that this discourse has re-interpreted and reshaped transparency in fundamental ways away from its original meaning. The target of transparency - in most cases, the provider of AI software - determines and shapes what is made visible to the outside world, and there is no external check on the validity and accuracy of such mediated accounts and explanations, opening transparency up for manipulation. Through a theoretically informed and critical analysis of the transparency provisions in the European Union's AI Act proposal, the Article shows that the substitution of transparency with mediated explanations faces important technical constraints, creates opportunities and incentives for both providers and public-sector users of AI systems to adopt opaque practices, and reinforces secrecy requirements that gag accountability in practice. An approach to transparency as disclosure thus becomes necessary, even if not sufficient in and of itself, to ensure the accountable development and use of AI technologies in the European Union. Transparency needs to be reclaimed as a core concept, accountability tailored and reinforced and the necessity for secrecy re-examined and cordoned off.

One. Introduction

Transparency has been critical to artificial intelligence debates, not least because artificial intelligence algorithms come with significant transparency challenges. Due to their so-called black-box nature, AI algorithms raise unprecedented opacity challenges by virtue of their technical complexity (powerful AI algorithms such as neural networks, or deep learning, are highly opaque in their functioning) as well as due to the proprietary nature of many AI algorithmic systems deployed both in public and private domains. At the same time, transparency is crucial in view of the core areas where AI algorithms are increasingly relied upon and the high-stakes implications of their deployment. For instance, the reliance by tax authorities in the Netherlands on a 'learning algorithm' as decisional aid in childcare benefits assessments resulted in state-sanctioned discrimination. The AI algorithm disproportionately flagged citizens with a migration background, causing widespread harm. Among the repercussions faced by wrongly accused families were dire financial circumstances, bankruptcies, lasting mental health damage, broken families and thousands of children taken into foster care. Victims that attempted to challenge the system were told that officials could not access the algorithmic inputs, with public officials reportedly justifying decisions 'because the algorithm said so'. The scandal speaks acutely to the high-stakes implications of the reliance on AI systems in the absence of systemic transparency as to their functioning and adequate (institutional) guardrails and safeguards on their deployment, in line with administrative law imperatives.

Calls for more transparency are all around us, in the public sphere and for government actors, across almost all policy areas, but also increasingly in the private sphere too. Transparency is what can be called a 'floating signifier', a malleable concept that is empty of specific content but rather refers to form. That form, at its core, is a medium that is seen through, rather than looked at directly. Koivisto has described its original meaning as 'the promise of unmediated visibility'. As a normative metaphor, 'it promises legitimacy by making an object or behaviour visible'. Birchall describes transparency as the invisible medium 'through which content is brought to our attention, into the visible realm'.

Conceptually, transparency is closely linked to accountability, yet differs from it on important counts. Its ambitions are more modest as it does not proclaim - in and of itself - to justify, to explain, to control nor to hold power to account but rather to render visible that which is hidden, and in doing so, to open up possibilities for oversight that otherwise would not be there. As such, transparency is an indispensable first step (albeit not sufficient on its own) towards the realisation of other goals - a necessary condition for the functioning of accountability is the acquisition of accurate and reliable information by relevant forums. While transparency often gets unjustly 'maligned' for failing to realise these related (proximate) goals, in fact transparency or information disclosure is a vital first phase of accountability, but this information then needs to be next taken up, questioned, scrutinized, and prodded by forums, explained, debated, and justified as part of accountability processes, with the possibility for consequences to arise, should actor explanations and justifications fall short of expectations. If this fails to materialize and transparency is not taken up further to have meaningful effects, it is a failure of accountability as a check on power, and speaks to the need to bolster our accountability mechanisms and processes rather than to the limitations of transparency.

The value of transparency is thus instrumental to other goals and follows rights. If new rights are attributed, transparency may be obliged to ensure their enjoyment. If rules expand, such as free movement or the internal market, then so too does the scope of transparency.

In the new digital context of automated decision-making and the use of AI there is a vigorous debate on the meaning and reach of transparency, which has, in turn, impacted how transparency is being legislated in this area. A key argument advanced by this article is that transparency in this context (and relatedly, in recent legislative efforts) has shifted from its original meaning of visibility to the adoption of a completely different logic, namely that of communication, where the target of transparency determines, shapes, and influences the content of what is made visible to the outside world. Many authors - and policy-makers - now equate transparency as meaning (only) communication in the sense of explanation. Explainability (and related concepts such as explicability or understandability) dominate AI transparency and governance debates, with explainability advanced as a way to address transparency problems raised by opaque black-box models. With its emphasis on explainability, the discourse on transparency in relation to AI has re-interpreted and reshaped transparency in fundamental ways, away from this original and literal meaning. In these understandings, transparency is no longer about immediate visibility but rather about a significant re-framing occurring towards explanation. In the name of facilitating understanding, a form of heavily mediated, pre-digested information provision is being advanced, often to a limited group of users and notably, in the absence of any external check on the validity and accuracy of these mediated accounts and explanations, opening transparency up for manipulation.

This directly connects with the 'dark side' of transparency. The dark side of transparency is secrecy and secrecy has expanded. We argue that the conceptual substitution of transparency for explanation risks reinforcing secrecy in this context. In an institutional world where transparency is substituted with explainability, the AI 'black box' no longer needs to be opened and disclosure becomes seemingly redundant. The corollary is that secrecy and proprietary protections can then be kept, expanded, and legitimized, with transparency in its core meaning discarded as 'inadequate'. Black boxes (produced by technical and/or legal opacity), become the accepted norm. And this reframed logic, as we will see below, feeds directly into how AI transparency is to be legislated in practice in this context, weakening accountability of providers and use by public authorities with transparency becoming a rudimentary facsimile of its former self.

These dynamics are not new and closely map onto familiar debates from more traditional areas, outside and beyond the scope of AI. For example, part of the ecosystem of transparency within the European Union has always been the rules on secrecy, featuring alongside mentions of transparency in many and varied legislative instruments and policy areas. Now, more than two decades after the adoption of the transparency regulation on access to documents, a draft regulation is proposed that will regulate professional secrecy within the European Union in a legislative instrument that also impacts on the requirements of professional secrecy and trade secrecy within the AI Act. Such rules and practices throw up opaque filters that block visibility or only give it to specific audiences (and subject to confidentiality requirements) or in highly secluded ways, also for accountability forums such as courts or parliaments, with the risk of cordoning such areas from oversight.

With the adoption of regulation on AI, the European Union is seemingly the first political and legal system to define in legislation what transparency must be taken to mean in this context. It turns out that information governance under the AI Act is mainly about secrecy not about visibility, about concealment rather than about disclosure and if there is disclosure it is to be a limited and secluded one, with no participation or accountability envisaged. Our article explores the provisions of the draft AI Act on transparency, secrecy, and related aspects of governance to understand the implications in the context of the wider conceptualization of transparency and accountability, in the broader context of European governance. We do so as European lawyers, accountability scholars and a computer scientist/lawyer at a crucial moment in the debate on the regulation of AI in Europe and ultimately globally in a context where the language of transparency is misappropriated. It is borrowed and used to imply a promise of visibility, of public accountability and with a vista of participation, none of which are or can in fact be realized in practice as currently framed. We critically contest existing conceptualizations and call for some recalibration of the core concepts as applied in the AI context to ground the normative promise implied in the use of the term transparency. We argue that transparency needs to be reclaimed as a core concept, accountability tailored and reinforced and the necessity for secrecy re-examined and cordoned off.