India's Supreme Court has directed the Bar Council of India to draft rules regulating AI-generated legal material. The order follows the discovery of fake, AI-produced judgments used in an insolvency case involving Essel Infraprojects Ltd. The Court has emphasized that human oversight is essential to prevent judicial errors caused by AI 'hallucinations' in the justice system.
What Happened
The Supreme Court of India has issued a stern directive requiring the Bar Council of India to establish a comprehensive policy governing the use of artificial intelligence in legal research and practice. This action follows the discovery that fake, hallucinated judicial precedents were used as evidence in insolvency proceedings against Essel Infraprojects Ltd. The National Company Law Tribunal (NCLT) and the National Company Law Appellate Tribunal (NCLAT) reportedly relied on these non-existent judgments, leading the Supreme Court to highlight the urgent need for stricter oversight to maintain the integrity of the justice system.
Why It Matters for Legal Integrity
The growing use of artificial intelligence to draft legal arguments or conduct research carries the risk of "hallucination," where AI models generate plausible-sounding but completely fabricated facts or case laws. For companies and legal professionals involved in NCLT matters, this poses a major risk. If legal decisions are influenced by fake data, it can lead to unfair outcomes, significant delays, and damaged reputations. The Supreme Court's mandate for a clear policy signals that while technology is useful, the responsibility for verifying the accuracy of all submitted material remains strictly with human lawyers and judges.
Previous Warning Signs
This is not the first instance where AI errors have impacted the legal system. In past cases, the Delhi High Court saw a petition withdrawn after it emerged that cited authorities were non-existent. Former Chief Justice of India D.Y. Chandrachud had also cautioned in 2024 that relying on AI tools without rigorous verification could lead to a miscarriage of justice. The Supreme Court's recent order is a formal step to standardize how these tools are used to ensure that such errors do not become a systemic problem in Indian courts.
The New Policy Mandate
The Bar Council of India has been tasked with forming a committee to create specific guiding principles for AI use in law. This committee will outline preventive measures and define the disciplinary consequences for lawyers who submit fabricated AI-generated authorities. The Court has adopted a policy of "zero tolerance" for fake precedents, clarifying that advocates face charges of professional misconduct for such lapses, while judges who rely on unverified material face scrutiny for a "serious lapse" in duty.
Human Oversight Remains Mandatory
Despite the potential benefits of AI in assisting with legal research and adjudication, the Supreme Court has declared that "human in the loop" is non-negotiable. Technology is being positioned only as an aid to decision-making, not a replacement for human judgment. For all parties involved in legal disputes, this means that manual due diligence and the verification of every cited judgment remain a critical requirement before submitting documents to any tribunal or court.