The Israeli authorities accepted this translation as accurate without even checking with Arabic-speaking people whether it was correct.
Automatic translation has also become an ineffective substitute in other critical situations, such as handling asylum applications. In this case, the problem to be solved is communication between people fleeing violence in their countries of origin and immigration officials. Automatic translation systems, which can work well when translating newspapers written in some standard varieties of a few dominant languages, can fail miserably when translating asylum applications written or spoken in minority languages or dialects.
In August 2020, thousands of British students who could not take their final secondary-school exams because of the COVID-19 pandemic were given grades calculated using an algorithm that took as an input, among other criteria, the grades obtained in previous years by other students at the same schools. After widespread protests, during which hundreds of students gathered outside the prime minister's residence at 10 Downing Street in London, chanting the slogan "Fuck the algorithm!", the grades were withdrawn and replaced with grades based on teachers' assessments of the students' work.
Promises of self-driving cars have brought death and destruction to our roads. A Tesla employee died after activating the so-called "Full Self-Driving" mode in his Tesla Model 3, which caused the car to leave the road. (The passenger survived the crash.) A few months later, during Thanksgiving 2022, Tesla CEO Elon Musk announced the availability of "Full Self-Driving" mode. A few hours later, a car being driven with this mode was involved in an eight-car collision on the San Francisco-Oakland Bay Bridge.
In 2023, attorney Steven A. Schwartz, who represented a plaintiff in a lawsuit against an airline, submitted a legal brief citing several legal precedents that the attorney had found by asking ChatGPT. When the airline's defense attorney reported that he had been unable to find some of the cited cases and the judge then asked Schwartz to produce them, Schwartz submitted excerpts instead of the full opinions, as was customary. In the end, Schwartz had to admit that he had trusted ChatGPT's output to be accurate, and both he and the attorney he worked with were sanctioned and fined by the court.
In November 2022, Meta released Galactica, a large language model trained on scientific texts, presenting it as capable of "summarizing academic papers, solving mathematical problems, generating Wiki articles, writing scientific code, annotating molecules and proteins, and much more". The demo version remained available for three days, while the global scientific community shared examples of how its outputs were completely fabricated, including false citations, and how the model could be induced to produce harmful content, presented in academic-style prose.