How terrifying is ChatGPT: It can destroy a 30-year legal career and is about to be brought to court.

How did ChatGPT ruin the career of a lawyer? It generated numerous summaries about non-existent events.

A man named Roberto Mata has just filed a lawsuit against Avianca airline after being hit by a baggage cart at John F. Kennedy International Airport in New York. When Avianca requested the federal judge in Manhattan to dismiss the lawsuit, Mata’s lawyer strongly objected and subsequently submitted a lengthy 10-page summary citing related lawsuits, such as one filed by a man named Martinez against Delta Air Lines, Zicherman against Korean Air Lines, and Varghese against China Southern Airlines.

However, later on, no one, including the airline’s lawyer and the presiding judge, could find any sources or citations for the mentioned information. It turned out that all the summarized documents were generated by ChatGPT.

The attorney, Steven A. Schwartz from the law firm Levidow, Levidow & Oberman, referenced the chatbot as a source and requested the court for forgiveness. He acknowledged his mistake in using an unreliable artificial intelligence program as evidence for the lawsuit.

According to The New York Times, Schwartz has been practicing law in New York for 30 years and had no intention of deceiving the court. He stated that he had never used ChatGPT before and was unaware of its potential inaccuracies, promising never to do so again.

Judge Castel expressed that he had never encountered a legal submission filled with “false rulings” like this before. He decided to schedule a hearing on June 8th to discuss the potential punishment for the attorney.

As AI permeates the online world, it has raised the prospect that one day machines will replace humans in the workforce. This prospect is particularly daunting for knowledge workers.

According to Stephen Gillers, a professor of legal ethics at New York University School of Law, this issue is particularly contentious among lawyers, who are already debating the value and dangers of AI.

“You cannot use ChatGPT, gather information, and then paste it into your court filings,” said Stephen Gillers.

Earlier, Avianca airline’s lawyers wrote a letter to Judge Castel, stating that they could not find the referenced cases in the summary provided by attorney Roberto Mata.

“These quoted phrases cannot be found, nor anything similar to them,” the airline’s lawyers said.

Bart Banino, Avianca’s lawyer from Condon & Forsyth, also stated that his firm could not find any evidence to support the authenticity of the cited references. He suspected that a chatbot might have been involved.

It is known that ChatGPT has been the subject of much controversy, such as how long it will take for this technology to operate without any human intervention. Even CEO Sam Altman of OpenAI has acknowledged the limitations of this technology.

“ChatGPT still has many shortcomings but makes people think it is flawless. We should not rely on this chatbot for important matters,” he shared.

A letter signed by prominent figures including Steve Wozniak, co-founder of Apple, and Elon Musk, along with over 1,000 business leaders, has called for a temporary halt and enhanced security measures in the development of AI models. Their concerns arise from the rapid advancement of artificial intelligence technology and the need to ensure trust and risk control.

The signatories propose that the development of powerful AI models, such as OpenAI’s GPT-4, should proceed only when there is a guarantee of earning trust and maintaining full control over potential risks. The idea behind slowing down the process is rooted in the belief that taking the time to refine and perfect AI will not cause any harm, as the ultimate goal will still be achieved.

James Grimmelmann, a distinguished professor of digital technology, expressed his support for the initiative, stating, “Slowing down the pace of developing new AI models is a commendable idea. If AI is truly beneficial, waiting for months or even years will not hinder our progress. In the end, we will ultimately attain our desired objectives.”