The trial of Artificial Intelligence

Artificial Intelligence (AI) promises to be one of the most transformative technologies ever developed, capable of reshaping every aspect of human life. Its reach now extends from mining and agriculture to healthcare and space exploration. Yet, one of the most intriguing and complex frontiers it has begun to influence is the legal profession.

In the twenty-first century, the legal profession stands at the crossroads of tradition and technology. For centuries, law has been shaped by human judgment, reasoning, and empathy, traits that no machine could replicate. Yet, with the rise of AI, we find ourselves confronting an extraordinary paradox: a tool that can accelerate justice, but also distort it, a tool that can be beneficial yet carry the risk of being misused. As courts, lawyers, and policymakers begin to experiment with AI, the question is no longer whether we should use it, but rather how to use it responsibly.

In the legal industry, AI is increasingly recognised as a transformative force. The pace at which AI is reshaping legal work is unprecedented, redefining research, drafting, and client expectations almost overnight. AI-powered legal tools can help identify relevant landmark judgments, break down complex legal propositions, or to the extent even suggest cross-examination questions.

However, this comfort also brings multiple challenges and the convenience of using AI comes with an equally significant responsibility, i.e., the duty to verify, interpret, and ensure accuracy. AI tools are not authorities they are sophisticated search engines, programmed in a way that they quickly go through all the available data on the web and produce a relevant compilation or what they believe to be relevant. The final responsibility falls in the hands of the lawyers and the professionals to check, read, and understand the context correctly.

Unfortunately, there have been instances where reliance on AI without adequate verification has led to serious errors. Both in India and abroad, there have been stories of unverified citations of judgments that are fabricated by AI-generated tools with false citations or misquoted excerpts that never existed in official records. At times, these generated cases are just the court’s interim orders or final orders picked up online because the AI found a relevant extract, though a full reading would reveal a completely different meaning and context.

A notable example occurred before an Indian High Court, where a homebuyers’ association sought to withdraw its plea after senior counsel flagged the use of a generative AI chatbot in drafting the petition. The petitioners had inadvertently referred to fictitious quotes and cases. One such citation referenced paragraph 73 of a 1972 Supreme Court judgment, which in fact contained only 27 paragraphs. Such a paragraph as was quoted in the petition was entirely fictitious and did not exist.

Recently, Chief Justice of India, Justice BR Gavai, while addressing a judicial training and capacity building programme jointly organised by the Supreme Court of India and Kenya, cautioned that while AI can assist in legal processes, it must never replace human judgment. He aptly questioned: “Can a machine, lacking human emotions and moral reasoning, truly grasp the complexities and nuances of legal disputes?”

He emphasised that justice demands empathy, ethics, and contextual understanding, qualities that remain beyond the reach of algorithms.

The High Court of Kerala, through its memorandum dated July 19, 2025, issued a detailed policy on the responsible use of AI tools in the District Judiciary. The policy stresses that AI may only be used as an assistive tool and must never replace human judgment or decision-making.

While caution in the use of AI is warranted, to condemn it for the mistakes of its users would be to deny innovation its natural learning curve. The errors made by advocates citing fictitious cases reflect not the incapacity of technology but our premature dependence on it. Every advancement in legal practice, from e-filing to virtual hearings and paperless proceedings, once faced scepticism before it became indispensable. The key lies in responsible adoption. AI should be used as an assistant, not an oracle. It can prepare us for cases, but it should never plead them for us.

India’s overburdened judicial system, with millions of pending cases, stands to gain immensely from the judicious use of AI. The Supreme Court Vidhik Anuvaad Software (SUVAS) is a good example of the same. SUVAS is an AI-based application that translates legal documents and orders that have been written in English into ten vernacular languages. Another such example could be the Supreme Court Portal for assistance in Court’s Efficiency (SUPACE ), a tool that collects relevant facts and laws and makes them available to a judge. It yields outcomes customised to the specific requirements of the case, and however the judge requires.

Recently, it has also been seen that law firms, companies, and professionals have entered into partnerships or agreements with generative AI platforms. These AI platforms could help in more effectiveness by accelerating drafting and reviewing of documents, streamlining due diligence processes, enhancing legal research and predictive analysis, and even delivering sharper data-driven insight for both litigation and advisory services.

However, this rapid integration of AI also raises critical questions regarding the protection of attorney–client privilege, confidentiality of sensitive data, and the ethical duties of legal professionals. We might not realise that the use of AI platforms could easily expose the confidential attorney-client privilege, as many of these platforms store their data on cloud servers, which introduces the real possibility of unintended data exposure or misuse, potentially compromising client confidentiality and breaching professional codes of conduct.

The responsibility, therefore, lies squarely on lawyers and firms to exercise due diligence before using such tools. To mitigate risks, legal practitioners must ensure that all sensitive data is encrypted, access is restricted to authorized users, and only trusted, compliant AI vendors are engaged. Reviewing a platform’s privacy policy, verifying its adherence to data protection laws and opting for services that offer end-to-end encryption are essential safeguards.

In conclusion, the integration of AI into the judicial and legal ecosystem is neither a passing trend nor an inevitability to be feared, it is an evolution that demands careful calibration. As India’s justice system contends with staggering pendency and procedural delays, AI offers undeniable potential to streamline processes, assist in legal research, and enhance access to justice for litigants.

However, the deployment of AI in such a sensitive domain cannot be driven by convenience alone. The same algorithms that promise objectivity may inadvertently perpetuate bias. The same databases that simplify research may also compromise confidentiality. The challenge before the legal fraternity, therefore, is to strike a balance between innovation and integrity, to ensure that efficiency does not eclipse ethics.

As we stand on the threshold of a new era, the question is no longer about whether AI will change the legal system. It is already doing so. AI must remain an instrument of human reasoning, not its replacement. Its success in the legal profession, specially in litigation will depend on transparent governance, judicial oversight, and an unwavering respect for the sanctity of client–attorney privilege. For in a system founded upon trust, confidentiality and ethics, one must ask: can justice truly be served if the very data that fuels its delivery risks exposing those it seeks to protect?

About the authors: Ashish Kumar is a Partner and Anju Shree Nair is an Associate at SNG & Partners.

Ansh Jain, Assessment Intern, provided assistance.

Disclaimer: The opinions expressed in this article are those of the author(s). The opinions presented do not necessarily reflect the views of Bar & Bench.

If you would like your Deals, Columns, Press Releases to be published on Bar & Bench, please fill in the form available here.