After trial judge cites AI-generated judgment, Supreme Court asks BCI to set up expert panel to examine issue

The Supreme Court on Tuesday asked the Bar Council of India (BCI) to constitute a committee of experts to examine the issue of judges and lawyers citing non-existent judgments generated via Artificial Intelligence (AI) tools [Gummadi Usha Rani v. Sure Mallikarjuna Rao]

A Bench of Justices PS Narasimha and Alok Aradhe was hearing a case arising from an order of a trial court in Andhra Pradesh, where non-existent judgments generated by AI were cited.

The Court did not pass any official direction but orally communicated the same to the BCI.

“We want the Bar Council of India to constitute a committee of experts, including outside experts, and file a report before us. Please communicate. We are not passing any order...,” the Court said.

The Bench added that a judge cannot simply rely on an AI-generated judgment and stressed the need for accountability and responsibility in judicial orders.

Pertinently, the Court also indicated the need to move towards creating a sovereign database and developing an India-based AI model.

The case arose from property dispute in which the trial court dismissed objections to an Advocate Commissioner’s report.

In doing so, it relied on certain case laws which were later found to be non-existent.

The defendants challenged the order, arguing that the citations were AI-generated and imaginary.

The matter eventually reached the Supreme Court.

The Supreme Court had earlier sought responses from the Attorney General (AG), the Solicitor General and the Bar Council of India and had also appointed Senior Advocate Shyam Divan as amicus curiae to assist it in the matter.

During the hearing today, Attorney General R Venkataramani said he had some suggestions but would first engage with the Ministry of Electronics and Information Technology (MeitY).

At this point, the Bench said,

"The AI committee is looking into it. It is almost at the verge of finalising the draft regulations. It will then be thrown open for consultation.”

Addressing Senior Advocate Shyam Divan, the Court said its concern was not limited to regulation but extended to the onus and responsibility of members of the Bar in placing synthetic or non-existent orders before courts.

“Research is also undertaken at the end of the judge, whether by mistake or otherwise, wherein a non-existent judgment is cited. Where then does the obligation and responsibility lie to ensure accountability, and ultimately, integrity in the conduct of proceedings?” the Bench asked.

The Court then sought to know from Divan what steps had been taken in this regard.

Divan said the Centre for Research and Planning has prepared a white paper on AI in the judiciary and the same contains recommendations and guidelines that could serve as a starting point for the Court.

The Bench observed that the problem is compounded by the absence of sovereign Large Language Models (LLMs), noting that reliance on open-source algorithms makes “hallucinations invariable”.

It indicated the need to move towards creating a sovereign database and developing an India-based AI model.

Thus, the Court asked the BCI to constitute a committee and file a report and listed the matter for further hearing in the last week of May.

[Read Live Coverage]