Meta has just unveiled a new AI model that is closer to the human brain than ChatGPT and others. The model, headed by French researcher Yann LeCun, aims to supplant generative AIs.
On the sidelines of the VivaTech show in Paris, Yann LeCun, the French researcher in charge of artificial intelligence at Meta, unveiled a new AI model called “JEPA” (Joint Embedding Predictive Architecture).
This model is intended to overcome key limitations of today’s most advanced AI systems, Meta said in a statement. During the presentation, Yann LeCun also insisted on distinguishing “JEPA” from other models, such as GPT, which powers OpenAI’s essential ChatGPT.
“Today’s AI and machine learning really suck. Humans have common sense while machines don’t,” says Yann LeCun. The researcher has always been very critical of language models on the market. Back in January, LeCun had already felt that ChatGPT was far from revolutionary . For him, “ChatGPT is not particularly innovative” on a technical level.
A new kind of AI model Determined to do better than its competitors, Meta offers an AI model that learns a little like human intelligence . The operation of the system is therefore very different from that of a model such as GPT or Google’s PaLM2. These operate by relying on a huge database. By drawing on this information, the AI models seek to guess the most logical sequence of words based on the user’s question. In fact, these AIs do not understand the texts they generate. They merely simulate reflection. Image generators, such as DALL-E, Adobe Firefly or Midjourney, work the same way.
This is not the case with the new Meta model. Instead of aggregating a mountain of human-created data, “JEPA” compares abstract representations of images or sounds. This approach forces the AI to build “an internal model of the outside world,” which makes it easier to learn.
In this way, the model mimics the functioning of a human brain. Every day, your mind analyzes a lot of data in an unconscious way in order to understand the world around it. Like the human intellect, the model can juggle abstract concepts rather than sequences of words. “Our work […] is based on the fact that humans learn a tremendous amount of basic knowledge about the world just by passively observing it ,” Meta says.
With this approach, the model truly understands the data entrusted to it. The AI is not forced to guess the answers to its users’ questions based on the most probable sequence of words. In theory, “JEPA” should therefore not “hallucinate”, ie generate factually false information. This is the main flaw of generative AIs. Their assertions are far from reliable, because they have not really understood what they are talking about. This is why the generators have a hard time designing certain elements, such as human hands or ears.
“If you train (a model) with 1,000 or 2,000 billion tokens, it seems able to understand. But he makes stupid, factual or logical errors , ”said Yann LeCun. In addition, “JEPA” stands out from other major language models for its high efficiency. According to the Meta press release, the AI does not require significant resources to operate, unlike more demanding systems like GPT.
The end of generative AI? For Meta, “ generative methods focus too much on details, instead of capturing big picture concepts,” as his new system does. According to Yann LeCun, generative AIs are also destined to disappear, because “we will have something better to replace them” : “Broad language models are still useful today, but in five years they won’t be used anymore . “
On paper, the model represents another step on the road to artificial general intelligence , the big goal of OpenAI and other tech titans. This type of AI, still far from current technologies, is able to evolve and adapt to the tasks entrusted to it, even if it is not programmed to carry them out. “Generative models are a thing of the past, we are going to abandon them for predictive architectures ,” says the researcher, who won the Alan Turing Prize in 2019.
True to form, Meta has made the entire “JEPA” code available to researchers. This open source strategy aims to stimulate research on AI by encouraging developers to appropriate this technology. The Menlo Park group has already done this with its previous innovations, such as MusicGen , the AI that generates music based on a short text, or LLaMA, for Large Language Model Meta AI . Source : Meta”
