AI Learns to Remember, Boosting Smarts and Efficiency

AI Learns to Remember, Boosting Smarts and Efficiency

Most advanced AI systems, like ChatGPT and Google’s Gemini, are surprisingly inefficient. When asked a simple question, such as “Who was Alexander the Great?”, they don’t just look up the answer. Instead, they go through complex, step-by-step calculations every single time. This is like asking a world-class chef to make a peanut butter sandwich and having them first grow peanuts, harvest them, make the peanut butter, and then make the bread – all from scratch for each request.

This process wastes a lot of computing power. The reason is that standard AI designs, called transformers, lack a simple way to quickly access stored information. They have to crunch numbers and reason through everything each time, a process that consumes significant energy and time.

DeepSeek AI Introduces “Engram” for Smarter AI

Researchers at DeepSeek AI have developed a new technology called “Engram.” This innovation aims to fix this fundamental problem by giving AI a kind of “pantry” or memory system. Instead of rebuilding everything from scratch, the AI can now grab pre-made “ingredients” – stored information – from this pantry.

This makes the AI much more efficient. But the surprise is that Engram doesn’t just speed things up; it also makes the AI smarter. When researchers experimented with Engram, they found that removing some of the AI’s complex reasoning parts and replacing them with this memory system actually led to better performance. The AI made fewer mistakes, showing a smart balance between actively thinking and recalling information.

A “Pantry” with Quality Control

DeepSeek AI also added a clever way for the AI to check the quality of its “ingredients” before using them. They created a system called a “context-aware gating mechanism.” Think of it like a chef checking if the ingredients are fresh before cooking. If the stored information (the “jar from the pantry”) doesn’t fit the current task (the “dish being cooked”), the system discards it. This prevents the AI from using outdated or incorrect information, ensuring accuracy.

Performance That Surprises

Typically, new AI techniques are better at some things and worse at others when compared to existing methods. However, Engram performed better across the board. On every single test, the Engram model outperformed previous techniques. This was an unexpected and significant improvement.

The core idea behind Engram is simple: automate the easy tasks and let the AI focus its complex processing power on harder challenges. Engram uses something called “n-gram embeddings” combined with “multi-head hashing.” In simpler terms, it’s like the AI can quickly scan a short phrase from a request and instantly know which shelf in its “pantry” holds the exact information it needs. This is essentially a highly efficient lookup table.

For example, when testing the Engram system, researchers found that if they turned off the Engram memory, the AI’s ability to answer trivia questions dropped by 70%. However, its reading comprehension remained high at 93%. This suggests the AI effectively split its “brain,” using the new Engram part specifically for storing facts, while its core reasoning abilities remained intact for understanding and processing new information.

Why This Matters

The development of Engram could lead to significantly cheaper and smarter AI systems. This could mean more AI tools that people can own and run locally, rather than relying on expensive cloud services. These systems would be faster and potentially free to use, making advanced AI more accessible to everyone. This approach of using efficient memory recall for factual data, while reserving complex computation for reasoning, is a promising direction for future AI development.

Limitations and Future Potential

While Engram shows great promise, it’s not perfect. One limitation is that if the Engram module is placed too deep within the AI’s processing layers, its accuracy can decrease. This is because the AI might have already spent time processing the information, making a lookup redundant. The “pantry” is most effective when checked early in the process, not after the “meal” has already been served.

Despite this, DeepSeek AI’s Engram represents a major step forward. It’s a publicly available research paper, meaning the knowledge is free for all to learn from and build upon. This open approach contrasts with proprietary systems that can be very expensive to operate. Tools like Lambda are mentioned as ways to run DeepSeek models privately, suggesting a growing interest in accessible and customizable AI solutions.

Source: DeepSeek Just Fixed One Of The Biggest Problems With AI (YouTube)