Should AI autonomously negotiate a settlement? Recommend litigation strategy? Influence judicial decision-making?
Not long ago, these questions belonged in the realm of science fiction. Today, they are becoming governance questions. Yet the most important question may be neither technological nor regulatory; it may be jurisprudential. As increasingly capable AI systems acquire greater agency and influence over legal, regulatory, and dispute-resolution processes, are the safeguards developed to uphold the rule of law still sufficient? This article argues that the emergence of AI agents should not be viewed solely as a technological development. It should also be viewed as a test of the institutional mechanisms through which legal systems preserve accountability, transparency, fairness, and legitimacy.
The central challenge is whether institutions designed to govern human decision-makers remain adequate as agency becomes increasingly distributed between humans and machines. Although many current examples arise in legal workflows such as contract review, compliance management, and dispute resolution, the issue extends well beyond legal technology. As AI systems become more capable, they can perform increasingly complex tasks, operate over longer time horizons, interact with multiple actors and systems, and exercise greater delegated authority. The governance challenge is therefore not limited to legal practice. Instead, legal institutions offer a useful lens for examining a broader societal question: whether existing safeguards remain sufficient as increasingly capable AI systems take on more influential roles in decision-making.
The Rule-of-Law Framework
The rule of law remains one of the foundational ideas in legal and political theory. Although scholars have formulated it in different ways, the central insight is consistent: public power should be exercised through law, not at discretion.
Joseph Raz develops this insight in functional terms, arguing that law can govern conduct only where rules are clear, accessible, stable, and consistently applied. Building on this, Lord Bingham offers a particularly useful framework for evaluating AI agency. Lord Bingham’s formulation is well-suited to the challenges posed by AI agency. His eight principles capture the institutional conditions under which the rule of law operates in modern democratic societies. This article focuses on four of those principles: accessibility and the ability of law to guide conduct, equality before the law, accountability in the exercise of power, and fair dispute resolution. These four speak most directly to the question of whether existing legal safeguards remain adequate as AI systems assume more consequential roles in decision-making.
The remaining principles, including those on discretion, fundamental rights, and international legal obligations, raise important questions of their own and warrant separate analysis. More than an abstract theory of governance, these principles provide a framework for assessing whether existing institutional safeguards remain effective as AI systems become increasingly active participants in legal processes. That concern echoes the work of Woodrow Hartzog and Jessica Silbey, who warn that emerging technologies can weaken the institutions societies rely on for trust, accountability, and legitimacy. The deeper question, then, is whether the institutions charged with enforcing those rules remain adequate as AI systems acquire greater agency
AI Agency Is Already Here
An AI agent is a type of AI system that can generate information and also carry out tasks. It can follow workflows, make decisions, and take actions in pursuit of a defined objective, typically under some level of human oversight.
AI-powered legal agents are currently being used to review contracts, process data privacy requests, construct claim charts, identify legal risks, analyze termination provisions, and manage recurring legal workflows. Dispute resolution is evolving as well: the American Arbitration Association has introduced AI-enabled tools capable of analyzing claims, summarizing evidence, generating legal analysis, and supporting settlement exploration through its Resolution Simulator platform. Viewed together, these developments signal something consequential: AI is moving beyond legal assistance and increasingly participating in legal processes. The question, then, is what this shift means for the rule of law.
Bingham Principle: Law Must Be Accessible
This concern leads directly to Bingham’s first principle: that the law must be accessible and, so far as possible, intelligible and predictable. If individuals are to understand and respond to legal rules, they must also be able to make sense of the processes through which those rules are applied. That requirement becomes more difficult to satisfy when AI systems review contracts, evaluate claims, assess legal risk, or recommend settlement outcomes. At lower levels of agency, the problem may be mitigated by the continuing presence of lawyers who can review, explain, and take responsibility for the resulting work product. As AI systems move from assistants to workflow managers and advisors, however, transparency becomes harder to secure. Affected individuals may increasingly encounter recommendations, classifications, or assessments generated by systems whose reasoning is not readily visible. The relevant question, then, is whether legal institutions can still ensure meaningful accessibility when decision-making is mediated by increasingly capable AI systems.
This suggests that transparency obligations designed for human decision-makers will increasingly need to focus on the reviewability of AI-assisted decisions rather than the explainability of underlying models. From a rule-of-law perspective, the critical question is not whether every aspect of an AI system can be explained, but whether affected individuals can obtain a meaningful explanation of the reasons underpinning decisions that affect their rights or obligations.
Bingham Principle: Equality Before the Law
Bingham also identified equality before the law as a foundational requirement of the rule of law. The law must be applied equally to all persons “in like circumstances.”
AI systems have the potential to improve consistency and reduce certain forms of human error. Yet as AI systems become more influential, they also raise new questions about equal treatment. If similarly situated parties receive materially different recommendations from AI-driven systems, how should those differences be evaluated? If access to advanced AI-assisted legal services becomes concentrated among better-resourced litigants, what are the implications for equality of arms? As AI systems move from assistants to advisors and negotiators, these concerns become increasingly significant. The core question is whether legal institutions possess adequate mechanisms to identify, evaluate, and remedy unequal treatment when AI systems play a substantial role in shaping outcomes.
Traditionally, equality before the law has been operationalized through mechanisms that evaluate individual decisions and individual claims of unequal treatment. Appeals, judicial review, evidentiary challenges, and anti-discrimination doctrines all assume that disparities can be identified by examining particular decisions and particular decision-makers. AI-assisted systems may challenge that assumption. Unequal treatment may emerge not from a single decision but from patterns distributed across thousands of interactions, recommendations, or outcomes. Individual decisions may appear lawful when viewed in isolation, while broader disparities become visible only at the system level. From a rule-of-law perspective, the challenge is therefore not merely to prevent unequal treatment. It is to determine whether institutions designed to detect and remedy individual instances of inequality remain adequate when the relevant unit of analysis increasingly becomes the system rather than the individual decision.
Bingham Principle: Accountability in the Exercise of Power
Perhaps the most significant rule-of-law challenge concerns accountability. Legal systems have long relied on identifiable human actors to whom power can be attributed and against whom accountability can be enforced. Ministers and public officers at all levels are required to exercise their powers in good faith, fairly, for the purpose for which they were conferred, without exceeding the limits of those powers. Judges, lawyers, regulators, arbitrators, and public officials can be questioned, challenged, disciplined, or removed. AI systems cannot and the issue is already emerging in practice
In Mata v. Avianca, lawyers filed briefs containing fictitious AI-generated citations. No one, not the airline’s counsel, and not even the judge, could locate the decisions or quotations cited and summarized in the brief, because the AI tool had invented them. Counsel explained that he had relied on an artificial intelligence program for legal research, “a source that has revealed itself to be unreliable,” and was ultimately sanctioned by the court. The significance of Mata extends beyond hallucinated citations. It underscores the enduring importance of accountability when AI-generated work enters legal proceedings. As AI systems become more deeply integrated into legal workflows, responsibility may become distributed among developers, deployers, organizations, supervisors, and end users. Accountability will not disappear; the challenge is to determine whether existing accountability mechanisms remain effective and adequate when agency is distributed across human and machine actors.
Historically, accountability has depended on the ability to identify a responsible actor and subject that actor to review, discipline, or liability. AI-assisted systems complicate this model because responsibility becomes fragmented across developers, deployers, supervisors, institutions, and users. The institutional challenge is therefore about ensuring that existing accountability mechanisms remain capable of locating responsibility within increasingly distributed decision systems. In one sense, Mata v. Avianca demonstrates the resilience of existing accountability mechanisms. The court was able to identify responsible actors, apply established professional obligations, and impose sanctions. Accountability remained intact because the relevant human decision-makers were identifiable and subject to institutional oversight.
Viewed through a rule-of-law lens, however, the case raises a more difficult question. The accountability mechanisms employed in Mata were designed to address discrete instances of professional error. They are well-suited to situations in which responsibility can be traced to particular individuals acting in particular matters. AI systems may present a different challenge. If the same model contributes to similar errors, omissions, recommendations, or procedural distortions across thousands of matters simultaneously, individual professional accountability may remain necessary but become institutionally insufficient. The question then becomes whether accountability mechanisms designed to identify responsibility case by case remain adequate when the underlying sources of error or influence operate systematically and at scale. From a rule-of-law perspective, the challenge may be shifting towards accountability mechanisms remaining capable of locating and assigning responsibility within distributed and technologically mediated decision systems.
Bingham Principle: Fair Dispute Resolution
Bingham also emphasized the importance of fair and effective dispute resolution. This principle becomes particularly relevant as AI systems enter negotiation, mediation, arbitration, and adjudication processes. The emergence of AI-assisted negotiation tools illustrates both the promise and the challenge. Such systems may reduce costs, increase efficiency, and expand access to dispute resolution. At the same time, important questions arise. What level of transparency should parties receive regarding AI-generated recommendations? How should informed consent operate when settlement pathways are suggested by sophisticated non-human systems? What safeguards are necessary to ensure that efficiency does not come at the expense of procedural fairness? These questions become even more significant if AI systems move beyond recommendation and begin to influence or conduct negotiations directly. The core question is whether legal institutions can preserve procedural fairness as AI assumes a more active role in dispute resolution.
Historically, procedural fairness has focused on the conduct of parties, advocates, and adjudicators. AI-assisted dispute resolution introduces a different challenge: ensuring that procedural protections remain meaningful when recommendations, negotiation pathways, or outcome assessments are generated by systems that are neither parties nor decision-makers in the traditional sense. This may require renewed attention to disclosure requirements, review rights, and the circumstances under which parties can contest machine-generated recommendations.
Are Existing Safeguards Still Sufficient?
Legal systems already include safeguards designed to constrain power and promote accountability, such as judicial review, appeals, professional discipline, ethical obligations, due process protections, procedural fairness, and negligence doctrines. The challenge is that these mechanisms were largely designed for human actors. AI agents introduce characteristics that may strain those safeguards in important ways:
- They operate at scale.
- They act at speed.
- They may be opaque.
- They evolve over time.
- Responsibility may be distributed across multiple actors.
Increasingly, they may interact with other AI systems rather than directly with humans. The rule-of-law challenge posed by AI agents is not primarily that they may be inaccurate; legal systems have always dealt with human error. The deeper question is whether the institutional safeguards developed to ensure accountability, transparency, reviewability, and procedural fairness remain effective when AI systems participate in legal, regulatory, and dispute-resolution processes. Would the institutions responsible for constraining power continue to function effectively when agency becomes increasingly distributed between humans and machines?
The African Question
Much of the scholarship and regulatory activity surrounding AI governance originates in North America and Europe. Yet the rule-of-law implications of AI agency may be particularly significant in jurisdictions characterized by judicial backlogs, resource constraints, evolving regulatory capacity, and legal pluralism.
Legal pluralism presents a challenge that has received relatively little attention in contemporary AI governance debates. Many existing governance frameworks implicitly assume a relatively unified legal order in which legal obligations emerge from a coherent system of statutes, regulations, judicial decisions, and administrative institutions. In many African jurisdictions, however, legal authority may be distributed across constitutional, statutory, customary, religious, and local dispute-resolution systems operating simultaneously. This raises a distinct rule-of-law question. Joseph Raz argued that law must be capable of guiding conduct. Yet guidance becomes more complex when individuals navigate multiple sources of legal authority simultaneously. AI governance frameworks designed for more unified legal systems may therefore confront a different adequacy problem in plural legal orders: how should increasingly capable systems identify, interpret, and operationalize legal obligations when authority itself is distributed across multiple normative frameworks? This challenge is not unique to Africa, but it may be especially visible there.
The implication is that the governance of AI agents cannot be reduced to questions of technical capacity or regulatory adoption. It is also a question of institutional fit. Frameworks developed in one legal context may not automatically satisfy the rule-of-law requirements of another. For African states, the question goes beyond the adoption of existing AI governance frameworks. It is whether those frameworks remain adequate when applied within legal systems whose institutional structures, sources of authority, and pathways to legitimacy differ in important respects from those for which many current frameworks were designed.
Conclusion
This article began with a simple question: are existing rule-of-law safeguards sufficient to govern increasingly capable AI agents? The analysis suggests that the challenge is not the absence of safeguards per se. Legal systems already possess extensive mechanisms designed to promote transparency, accountability, equality before the law, and procedural fairness. Courts, regulators, professional bodies, and review processes have long served as the institutional means through which rule-of-law values are translated into practice.
Across each of the four dimensions examined in this article, however, a common pattern emerges. Existing safeguards appear most effective under the conditions for which they were originally designed: identifiable actors, discrete decisions, visible errors, traceable chains of responsibility, and relatively coherent institutional structures. Increasingly capable AI systems may strain those assumptions. They can operate at scale, influence large numbers of decisions simultaneously, distribute responsibility across multiple actors, obscure causal pathways, and function within legal and institutional environments that were not designed with machine agency in mind.
The principles examined in this article remain as important as ever. The more difficult question is whether the institutions and mechanisms through which those principles are operationalized remain adequate under new conditions of distributed agency.
The analysis also suggests that adequacy may not be a universal question with a universal answer. The challenges presented by AI agency are likely to manifest differently across legal systems, institutional environments, and governance traditions. As the discussion of legal pluralism illustrates, safeguards that appear adequate in one jurisdiction may prove less effective in another. Future work should therefore move beyond asking whether AI is compatible with the rule of law and focus more directly on the conditions under which existing institutions remain capable of preserving rule-of-law values. The most important governance question is fast becoming whether the institutions responsible for governing them remain fit for purpose. Viewed in this way, AI agents are not merely a new category of technology. They are a test of institutional adequacy. And the outcome of that test may shape not only the future of legal practice and dispute resolution, but the future evolution of the rule of law itself.