Lawyers could face ‘severe’ penalties for fake AI-generated citations, UK court warns

The High Court of England and Wales says lawyers need to take stronger steps to prevent the misuse of artificial intelligence in their work.

In a ruling tying together two recent cases, Judge Victoria Sharp wrote that generative AI tools like ChatGPT “are not capable of conducting reliable legal research.”

“Such tools can produce apparently coherent and plausible responses to prompts, but those coherent and plausible responses may turn out to be entirely incorrect,” Judge Sharp wrote. “The responses may make confident assertions that are simply untrue.”

That doesn’t mean lawyers cannot use AI in their research, but she said they have a professional duty “to check the accuracy of such research by reference to authoritative sources, before using it in the course of their professional work.”

Judge Sharp suggested that the growing number of cases where lawyers (including, on the U.S. side, lawyers representing major AI platforms) have cited what appear to be AI-generated falsehoods suggests that “more needs to be done to ensure that the guidance is followed and lawyers comply with their duties to the court,” and she said her ruling will be forwarded to professional bodies including the Bar Council and the Law Society.

In one of the cases in question, a lawyer representing a man seeking damages against two banks submitted a filing with 45 citations — 18 of those cases did not exist, while many others “did not contain the quotations that were attributed to them, did not support the propositions for which they were cited, and did not have any relevance to the subject matter of the application,” Judge Sharp said.

In the other, a lawyer representing a man who had been evicted from his London home wrote a court filing citing five cases that did not appear to exist. (The lawyer denied using AI, though she said the citations may have come from AI-generated summaries that appeared in “Google or Safari.”) Judge Sharp said that while the court decided not to initiate contempt proceedings, that is “not a precedent.”

“Lawyers who do not comply with their professional obligations in this respect risk severe sanction,” she added.

Both lawyers were either referred or referred themselves to professional regulators. Judge Sharp noted that when lawyers do not meet their duties to the court, the court’s powers range from “public admonition” to the imposition of costs, contempt proceedings, or even “referral to the police.”