Meta’s New Large Language Model Galactica Pulled Down Three Days After Launch
The level of criticism Galactica or any other tool may receive is relative to the consequence of where its application lies.
Meta has pulled down Galactica, its latest large language model, just three days after it was introduced. According to MIT Technology Review, Galactica, designed to assist scientists with relevant scientific compositions, was taken down because it is “a mindless bot that cannot tell fact from fiction.”
On Tuesday last week, a demo of Meta’s much touted artificial intelligence model Galactica was unveiled to the public. Before the weekend, however, the tool was taken down following its tendency to generate what can only be termed gibberish and its inability to solve some rudimentary mathematical questions accurately.
Galactica is Meta’s initiative to streamline scientific research discovery and ease scientific text generation. In a paper, Meta termed it as a tool to organize science and disclosed that it is trained on 48 million papers, textbooks and lecture notes, millions of compounds and proteins, scientific websites, and encyclopedias totaling 120 billion parameters.
If the development of large language models has indicated anything, it is that they tend to go awry as soon as netizens get to play with them. Remember the GPT-3-based Blender Bot 3 conversational AI (also by Meta), Microsoft’s Tay, Scatter Lab’s Luda chatbot, and others?
The fact that Meta released the demo with a “NEVER FOLLOW ADVICE FROM A LANGUAGE MODEL WITHOUT VERIFICATION” message almost suggests that the company was half expecting the reaction it has garnered.
The scientific community was quite dismissive of the tool. Director at the Max Planck Institute for Intelligent Systems, Michael Black’s scathing criticism included the word ‘dangerous.’ “Why dangerous? Galactica generates text that’s grammatical and feels real. This text will slip into real scientific submissions. It will be realistic but wrong or biased. It will be hard to detect. It will influence how people think.”
“It offers authoritative-sounding science that isn’t grounded in the scientific method. It produces pseudo-science based on statistical properties of science *writing*. Grammatical science writing is not the same as doing science. But it will be hard to distinguish. This could usher in an era of deep scientific fakes.”
Brian Umberger, professor and chair of movement science at the University of Michigan, concurred. “It reads like someone who knows just enough terminology to sound convincing, but actually has many of the details wrong,” Umberger noted.
A user who took Galactica for a spin tweeted the results:
This is hilarious https://t.co/Vu23PJncY1 pic.twitter.com/9QdT3oDgBu
— Gryzor (@krakout) November 17, 2022
See More: Google Techie Suspended After Questioning Whether LaMDA Has Become Self-Aware
Based on the numerous other results generated by Galactica, it would seem that Meta’s latest tool to extract the actual meaning or value through the interpretation of the papers and data it was trained on is just another cargo-cutting effort.
The level of criticism Galactica is receiving is relative to the consequence of where its application lies. Establishing language-specific and grammatically correct combinations of words that make sense on paper based on established rules and patterns doesn’t necessarily mean the results should satisfy level-headed scientific tempers. That’s exactly what happened with the Galactica demo.
People found that it generated fake research and wiki articles that sounded right but were hokum. Sebastian Raschka, a machine learning and AI researcher, assessed that Galactica is more of a wiki article template generator than a wiki article generator.
Additionally, Meta’s claim that it solves math problems was also disputed by Carl Bergstrom, a biologist and a professor at the University of Washington in Seattle. Bergstrom went on to call Galactica a “random bullshit generator.”
However, not all were negative. Some professionals termed the model “good enough,” “very helpful,” “not bad,” “game changer,” “pretty impressive,” and “cool” but “not quite ready.”
Nevertheless, Meta’s chief AI scientist and New York University professor Yann LeCun expressed dismay that Galactica, even if not perfect, shouldn’t have garnered the ridicule and criticism it got. He added the online vitriol was the reason why the team who built Galactica became distraught and took down the demo.
“I figured out what bothers me so much about Facebook’s Galactica,” Bergstrom said. “It’s that it pretends to be a portal to knowledge. In their words, ‘new interface to access and manipulate what we know about the universe’.”
Let us know if you enjoyed reading this news on LinkedIn, Twitter, or Facebook. We would love to hear from you!
Image source: Shutterstock
MORE ON ARTIFICIAL INTELLIGENCE
