Supreme Court declares zero tolerance to citing fake AI-generated judgments, asks BCI to frame norms

The Supreme Court on Thursday ruled that courts must adopt a “zero-tolerance” approach towards producing, citing or using AI-generated precedents without verification [Pooja Ramesh Singh Vs J&K Bank].

A Bench of Justices PS Narasimha and Alok Aradhe made the observation while setting aside the NCLT and NCLAT orders in the Essel Infraprojects insolvency case. The Court found that the NCLT had relied on fake, non-existent and hallucinated AI-generated material as judicial precedents.

“It is necessary for Courts to adopt a zero-tolerance mode for producing, citing or using AI-generated precedents without verification,” the Court said.

The Court also held that it would amount to misconduct for an advocate to cite such judgments without verification.

“Equally, it is a serious lapse if a judge relies on such fake or hallucinated AI-generated material as precedents in support of the determination,” the Court added.

The Bench went on to say that any decision based on fake or hallucinated AI-generated material is “no decision in the eyes of the law.” It said such decisions must be set aside even if “an iota” of fake or hallucinated material enters the decision-making process.

The Court also directed the Bar Council of India (BCI) to constitute a committee to examine the issue. It asked the BCI to frame guiding principles and prescribe disciplinary action for lawyers who place fake or hallucinated material before courts as legal precedents.

The Bench warned that AI-generated fake precedents poses a grave threat to the justice system.

“For those in the province of adjudication and determination of disputes, this by-product of AI, i.e., the production of fake, non-existent, and hallucinated material and its utilisation as precedents in law, is like the release of methyl isocyanate in the province of law and justice,” the Court said.

In the present case, the Supreme Court noted that the NCLT had relied on several non-existent judgments and AI-generated extracts wrongly attributed to genuine Supreme Court decisions. On independent verification, the Court found that some citations were entirely fictitious, while others carried fabricated extracts.

The Court also recorded that Jammu and Kashmir Bank had filed an affidavit stating that the judgments were not cited by its counsel. The affidavit said the precedents relied on by the NCLT were obtained through its own research. The Supreme Court also noted that the fake precedents escaped scrutiny before the NCLAT.

The Court held that the judicial process stood tainted by reliance on unreal and fake precedents. It restored the Section 7 application to the NCLT and directed it to decide the matter afresh, preferably within two weeks. The parties were directed to maintain status quo till then.

Senior Advocate Madhavi Divan appeared for Pooja Ramesh Singh, assisted by advocate Vishesh Kalra.

Jammu and Kashmir Bank was represented by advocate Sumesh Dhawan, along with a team from Dua Associates led by advocate Sanjana Dua.

[Read Judgment]