As the founder of LLM Watch, my goal is to curate the most relevant Large Language Model (LLM) research and make it accessible to a broad audience. 

I distill complex technical topics into summaries that respect your time. 

Before embarking on this journey, I led Deep Learning R&D at one of Europe's largest telecom companies. There, I spearheaded numerous AI projects with a focus on Natural Language Processing (NLP) and Automatic Speech Recognition (ASR). 

With more than 6 years of experience in AI, I was there before the hype. 

I try to offer a healthy balance between shiny new things and what really matters.

Pascal Biese

Self-Evolving Memory:

A new framework from OPPO AI and LV-NUS lab now introduces memory systems that evolve their own architecture.

The paper draws a compelling analogy to human learning. Most AI agents today are like students who know how to take notes, but always use the same study method regardless of the subject. MemEvolve creates "adaptive learners" that dynamically adjust their learning strategy based on what they're trying to accomplish - memorizing facts for one task, abstracting patterns for another.

For this, they developed a dual-evolution process. The inner loop accumulates experience within a fixed memory architecture. The outer loop does something more interesting: it evaluates how well that architecture is working and proposes structural modifications. 

The system decomposes memory into four modules - Encode, Store, Retrieve, Manage - and uses performance feedback to diagnose bottlenecks and redesign components accordingly. 

The results? MemEvolve improves agent frameworks like SmolAgent and Flash-Searcher by up to 17% on challenging benchmarks. And the memory architectures evolved on one task transfer effectively to unseen benchmarks and different backbone models, suggesting the framework discovers generalizable principles rather than task-specific heuristics. 

In addition to the paper, they also released EvolveLab which unifies twelve existing memory systems under a common interface, making this a practical foundation for future research on self-improving agents. 
↓ 
𝐖𝐚𝐧𝐭 𝐭𝐨 𝐤𝐞𝐞𝐩 𝐮𝐩? Join my newsletter with 50k+ readers and be the first to learn about the latest AI research: llmwatch.com 💡

AI Lead at PwC <​/​> Daily AI highlights for 80k+ experts 📲🤗

MemEvolve: Meta-Evolution of Agent Memory Systems

A DeepSeek moment for AI memory? 

DeepSeek AI just published research suggesting that modern language models are burning valuable compute simulating memory lookups through expensive neural operations.

The problem is fundamental: language modeling involves two very different tasks. One is compositional reasoning - connecting ideas, logical inference. The other is knowledge retrieval - recognizing "Diana, Princess of Wales" or common phrases. Current Transformers have no native mechanism to simply look up static patterns, so they're forced to reconstruct them through multiple layers of attention and feed-forward networks.

The proposed solution is called Engram, a "conditional memory" module based on N-gram embeddings that retrieves static patterns in O(1) time. The key insight is that it's about finding the optimal balance between conditional computation (MoE experts) and conditional memory. The researchers discovered a U-shaped scaling law: pure MoE isn't optimal. Reallocating roughly 20-25% of sparse parameters to memory yields superior results. 

While memory was expected to help knowledge-intensive tasks (MMLU +3.4), the largest gains appeared in reasoning (BBH +5.0) and code/math (HumanEval +3.0, MATH +2.4). Mechanistic analysis reveals why: by offloading pattern reconstruction, Engram effectively deepens the network - shallow Engram layers produce representations equivalent to much deeper MoE layers. 

Perhaps most practically significant: the deterministic addressing allows offloading 100B-parameter tables to host memory with less than 3% overhead. 

This suggests that the next generation of sparse models might not just be about smarter routing - but about recognizing that some knowledge simply doesn't need to be computed at all.
↓ 
𝐖𝐚𝐧𝐭 𝐭𝐨 𝐤𝐞𝐞𝐩 𝐮𝐩? Join my newsletter with 50k+ readers and be the first to learn about the latest AI research: llmwatch.com 💡

A New Axis of Sparsity for Large Language Models

Meta's Super Intelligence Lab just published their solution to the training data problem. 

Training search-augmented LLMs typically requires extensive human-annotated questions and answers. As high-quality data becomes scarcer and more expensive, this dependency creates a significant bottleneck for developing capable AI research agents.

Dr. Zero introduces a clever proposer-solver feedback loop. Both models start from the same base LLM, but play different roles: the proposer generates increasingly difficult questions, while the solver learns to answer them using external search. 

The key insight is that as the solver improves, easy questions stop providing useful training signal, which forces the proposer to generate harder ones. This creates an automated curriculum that pushes both models forward without any human data.

The computational challenge with such approaches is typically prohibitive. Standard reinforcement learning methods require sampling many questions and many answers per question, creating nested costs that explode for multi-turn reasoning. Dr. Zero solves this through Hop-Grouped Relative Policy Optimization (HRPO), which clusters questions by their structural complexity (number of reasoning "hops" required) and computes advantages within these groups. This reduces sampling overhead to roughly one-fourth of standard methods while maintaining performance. 

The data-free Dr. Zero matches or exceeds fully supervised search agents across seven QA benchmarks, with up to 14.1% improvements on complex multi-hop reasoning tasks. 

This suggests that complex reasoning and search capabilities can emerge purely through self-evolution. 
↓ 
𝐖𝐚𝐧𝐭 𝐭𝐨 𝐤𝐞𝐞𝐩 𝐮𝐩? Join my newsletter with 50k+ readers and be the first to learn about the latest AI research: llmwatch.com 💡

Self-Evolving Search Agents without Training Data

LLM-as-a-Judge -> Agent-as-a-Judge

Using LLMs to evaluate other LLMs seemed like a clever shortcut. But single-pass evaluation is hitting its limits: we need evaluators that can actually verify claims, not just assess how plausible an answer sounds. 

Traditional LLM judges are passive observers. They grade based on linguistic patterns without checking if the code runs, the math is correct, or the facts hold up. 

A new comprehensive survey traces the shift to Agent-as-a-Judge, where evaluation becomes an active, multi-step process. Instead of one model making a snap judgment, agentic judges decompose evaluation into subtasks, invoke external tools like code interpreters and search engines to verify claims, and use multi-agent debate to neutralize individual model biases. 

The researchers identify three developmental stages: procedural agents following fixed workflows, reactive agents adapting based on intermediate feedback, and self-evolving agents that can refine their own evaluation criteria on the fly. Applications already span code verification, fact-checking, medical assessment, legal judgment prediction, and financial research analysis. 

When AI systems are used to generate training signals for other AI systems, the precision of the judge determines everything downstream. Can we build evaluators that know when they need to verify rather than guess?
↓ 
𝐖𝐚𝐧𝐭 𝐭𝐨 𝐤𝐞𝐞𝐩 𝐮𝐩? Join my newsletter with 50k+ readers and be the first to learn about the latest AI research: llmwatch.com

A Survey on Agent-as-a-Judge

What if you could surgically edit an LLM's knowledge by swapping a single vector?

A new architecture from CMU and Meta AI makes this possible, while also solving some of the thorniest problems in scaling transformers.

Scaling language models has traditionally meant more compute. Mixture-of-Experts (MoE) promised a workaround by activating only a subset of parameters per token, but came with notorious training instability, load balancing headaches, and communication overhead that grows with expert count.

Instead of dynamic routing to experts, STEM (Scaling Transformers with Embedding Modules) replaces the FFN up-projection with a static, token-indexed embedding lookup. Each token has its own embedding per layer, while the gate and down projections remain shared. No routing decisions, no load balancing losses, no cross-device communication for expert selection. 

The results? 3-4% accuracy gains over dense baselines, with particularly strong improvements on knowledge-intensive benchmarks (9-10% on ARC-Challenge and OpenBookQA). Training remains stable where MoE would show loss spikes, and the architecture naturally activates more parameters as context length grows. 

Because each embedding maps to a specific token, you can literally swap "Spain" for "Germany" at the embedding level and watch the model output "Berlin" instead of "Madrid", without changing the input text at all. 

If interpretability and controllability matter as much as raw performance, STEM suggests we may have been overcomplicating sparse scaling all along.
↓
𝐖𝐚𝐧𝐭 𝐭𝐨 𝐤𝐞𝐞𝐩 𝐮𝐩? Join my newsletter with 50k+ readers and be the first to learn about the latest AI research: llmwatch.com

STEM: Scaling Transformers with Embedding Modules

The next big step in LLMs?

A new inference technique from Massachusetts Institute of Technology called Recursive Language Models (RLMs) is rethinking context windows entirely.

The core problem is well-known: even frontier models like GPT-5 suffer from "context rot" - performance degrades quickly as prompts get longer, regardless of the technical context limit. Summarization helps but loses critical details. Retrieval misses complex reasoning patterns. Have we been trying to make models see more, when perhaps the answer is to make them see differently?

RLMs treats the prompt not as direct neural network input, but as an external object the model interacts with programmatically. The prompt is loaded as a variable in a Python REPL environment, and the LLM writes code to peek into it, decompose it, and recursively call itself over smaller snippets. Same interface as a regular LLM, radically different execution.

On information-dense tasks where GPT-5 scores below 0.1%, RLMs achieve 58%. On multi-hop research questions spanning 6-11M tokens, RLMs hit 91% accuracy while costing less than feeding the full context would. Crucially, performance degrades far more gracefully as complexity scales - the approach handles inputs two orders of magnitude beyond native context windows.

This suggests that scaling context is not just an architecture problem but also an inference problem. RLMs demonstrate that letting models reason about their input symbolically rather than processing it neurally could be a promising new direction.

If this approach generalizes, we may be looking at a new axis for scaling language model capabilities entirely.
↓ 
𝐖𝐚𝐧𝐭 𝐭𝐨 𝐤𝐞𝐞𝐩 𝐮𝐩? Join my newsletter with 50k+ readers and be the first to learn about the latest AI research: llmwatch.com 💡

Recursive Language Models

The latest reasoning models fail on puzzles with just 1 unknown cell.

Yet they solve complex ones with dozens of steps.

New research reveals why: these models might be pattern matching, not actually reasoning.

Hierarchical Reasoning Models (HRMs) crush benchmarks, outperforming LLM-based reasoners by wide margins. But researchers dug into the mechanics and found something troubling.

Think of it like a student who memorizes exam answers. They ace questions they've seen before but stumble on trivially easy variations.

The researchers identified three failure modes:

- HRMs fail on extremely simple puzzles (single unknown cell) when patterns violate their training distribution
- Models lack the "fixed point property" - they can't recognize when a problem is already solved
- Performance doesn't correlate with puzzle difficulty the way true reasoning would

This matters because we're deploying these models assuming they generalize.

If your reasoning model solves a 50-step problem but fails a 2-step variant, it's not reasoning - it's sophisticated pattern matching.

Before trusting HRMs in production, test them on deliberately simple edge cases. The failures will tell you more than the successes.
↓
𝐖𝐚𝐧𝐭 𝐭𝐨 𝐤𝐞𝐞𝐩 𝐮𝐩? Join my newsletter with 50k+ readers and be the first to learn about the latest AI research: llmwatch.com 💡

Are Your Reasoning Models Reasoning or Guessing?

Machine Learning Engineer | Data Scientist | Gen AI | Women TechMaker @Google | ExploreML CrowdSource Facilitator @Google | GCP | AWS | Artificial Intelligence | GSSoC '22 |

Hey there! 👋 I’m Vedant Pandya (He/Him) - an AI and Generative AI Researcher, visionary team leader, and Generative AI Developer driven by innovation and impact. With over four years of expertise spanning Machine Learning, Generative Artificial Intelligence, Data Science, and Cloud Computing (AWS & GCP), I navigate the evolving landscape of Industry 4.0 with purpose and passion.

As a trailblazing Machine Learning Engineer and Data Scientist, I specialize in turning complex ideas into transformative solutions. My journey has encompassed consulting, tech, and industry domains, where I’ve spearheaded groundbreaking open-source projects and delivered measurable outcomes. A champion for continuous learning, I empower individuals and teams to exceed their potential through impactful mentorship and education.

Certified in Google Cloud Machine Learning and Data Science and a proud Google Women TechMaker, I also thrive as a content creator and blogger, simplifying cutting-edge tech concepts. My expertise spans AI model development, NLP, computer vision, multi-modal learning, and deploying next-generation AI solutions in real-world contexts.

Rooted in resilience and values from my upbringing as the child of a single parent, I embrace challenges with dignity and determination. Let’s connect to explore AI, Machine Learning, Quantum Computing, or the expansive universe of technology—and unlock the latent potential within. 🚀

===================================================================
Key Words: AI Researcher • Generative AI Innovator • AI Developer • Tech Leadership • Mentorship in Innovation • AI Model Architect • Cloud Computing Pioneer • NLP Virtuoso • Cutting-Edge ML Solutions • Industry 4.0 Leader • Generative Pre-Trained Transformers (GPTs) • Multi-Modal AI Expert • Quantum Computing Visionary • Cloud Native Architect • Data Science Mentor • Strategic Collaboration in AI

Vedant Pandya

𝐈𝐭 𝐭𝐨𝐨𝐤 𝐦𝐞 27 𝐝𝐚𝐲𝐬 𝐭𝐨 𝐜𝐨𝐦𝐩𝐥𝐞𝐭𝐞 𝐚𝐧𝐝 𝐭𝐫𝐮𝐥𝐲 𝐮𝐧𝐝𝐞𝐫𝐬𝐭𝐚𝐧𝐝 𝐭𝐡𝐞 𝐩𝐚𝐩𝐞𝐫 “𝐕𝐋-𝐉𝐄𝐏𝐀” 𝐛𝐲 Yann LeCun 𝐚𝐧𝐝 𝐭𝐡𝐞 AI at Meta Team along with New York University.

For almost a month, I kept rereading the same sections - not because the paper was written with complexity, but because it challenges a deeply ingrained assumption in modern AI:

👉 𝐓𝐇𝐀𝐓 𝐈𝐍𝐓𝐄𝐋𝐋𝐈𝐆𝐄𝐍𝐂𝐄 𝐌𝐔𝐒𝐓 𝐁𝐄 𝐋𝐄𝐀𝐑𝐍𝐄𝐃 𝐁𝐘 𝐆𝐄𝐍𝐄𝐑𝐀𝐓𝐈𝐍𝐆 𝐓𝐎𝐊𝐄𝐍𝐒.

Now, "VL-JEPA" breaks that assumption. Instead of teaching a model how to talk, it teaches the model what something means - directly in semantic space.

𝐓𝐇𝐀𝐓 𝐒𝐎𝐔𝐍𝐃𝐒 𝐒𝐈𝐌𝐏𝐋𝐄. 𝐁𝐔𝐓, 𝐈𝐓’𝐒 𝐍𝐎𝐓.

🧠 Understanding VL-JEPA required me to unlearn:
- Autoregressive decoding as a necessity
- Token-level loss as the only supervision
- Generation as the core of intelligence

The hardest part wasn’t the architecture - it was the shift in mindset:

𝐏𝐫𝐞𝐝𝐢𝐜𝐭 𝐦𝐞𝐚𝐧𝐢𝐧𝐠 𝐟𝐢𝐫𝐬𝐭. 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐢𝐬 𝐣𝐮𝐬𝐭 𝐚 𝐜𝐨𝐦𝐩𝐫𝐞𝐬𝐬𝐢𝐨𝐧 𝐟𝐨𝐫𝐦𝐚𝐭.
𝐓𝐡𝐞 𝐦𝐚𝐭𝐡 𝐥𝐢𝐯𝐞𝐬 𝐢𝐧 𝐞𝐦𝐛𝐞𝐝𝐝𝐢𝐧𝐠 𝐠𝐞𝐨𝐦𝐞𝐭𝐫𝐲, 𝐈𝐧𝐟𝐨𝐍𝐂𝐄 𝐚𝐥𝐢𝐠𝐧𝐦𝐞𝐧𝐭, 𝐜𝐨𝐥𝐥𝐚𝐩𝐬𝐞 𝐚𝐯𝐨𝐢𝐝𝐚𝐧𝐜𝐞, 𝐚𝐧𝐝 𝐥𝐚𝐭𝐞𝐧𝐭 𝐩𝐫𝐞𝐝𝐢𝐜𝐭𝐢𝐨𝐧 - 𝐧𝐨𝐭 𝐜𝐫𝐨𝐬𝐬-𝐞𝐧𝐭𝐫𝐨𝐩𝐲 𝐨𝐯𝐞𝐫 𝐯𝐨𝐜𝐚𝐛𝐮𝐥𝐚𝐫𝐲.

🤔 Why did it take me 27 days? Because this paper quietly proposes a different future for vision-language models which are:

1. Non-generative
2. Real-time
3. Sample-efficient
4. Semantics-first

- "VL-JEPA" shows that you can outperform large VLMs with half the parameters, decode 3× less often, and still handle captioning, retrieval, and VQA - using just one unified model.

𝐓𝐇𝐈𝐒 𝐈𝐒𝐍’𝐓 𝐉𝐔𝐒𝐓 𝐀𝐍 𝐎𝐏𝐓𝐈𝐌𝐈𝐙𝐀𝐓𝐈𝐎𝐍. 𝐈𝐓’𝐒 𝐀 𝐏𝐇𝐈𝐋𝐎𝐒𝐎𝐏𝐇𝐈𝐂𝐀𝐋 𝐒𝐇𝐈𝐅𝐓.

I now believe:
"𝐓𝐎𝐊𝐄𝐍𝐒 𝐀𝐑𝐄 𝐀𝐍 𝐈𝐍𝐓𝐄𝐑𝐅𝐀𝐂𝐄; 𝐍𝐎𝐓 𝐈𝐍𝐓𝐄𝐋𝐋𝐈𝐆𝐄𝐍𝐂𝐄."

And "𝐕𝐋-𝐉𝐄𝐏𝐀" might be the clearest step yet toward machines that understand before they speak.

If you’re working on multimodal AI, world models, robotics, or real-time systems - this paper is worth every difficult page.

#ArtificialIntelligence #MachineLearning #VisionLanguageModels #MultimodalAI #RepresentationLearning #SelfSupervisedLearning #DeepLearning #AIResearch #YannLeCun #MetaAI #WorldModels #VLJEPA #JEPA

📄 𝐏𝐚𝐩𝐞𝐫 𝐑𝐞𝐚𝐝 & 𝐍𝐨𝐭𝐞𝐬: 𝐦𝐇𝐂 - 𝐌𝐚𝐧𝐢𝐟𝐨𝐥𝐝-𝐂𝐨𝐧𝐬𝐭𝐫𝐚𝐢𝐧𝐞𝐝 𝐇𝐲𝐩𝐞𝐫-𝐂𝐨𝐧𝐧𝐞𝐜𝐭𝐢𝐨𝐧𝐬 (DeepSeek AI)

I recently read “𝐦𝐇𝐂: 𝐌𝐚𝐧𝐢𝐟𝐨𝐥𝐝-𝐂𝐨𝐧𝐬𝐭𝐫𝐚𝐢𝐧𝐞𝐝 𝐇𝐲𝐩𝐞𝐫-𝐂𝐨𝐧𝐧𝐞𝐜𝐭𝐢𝐨𝐧𝐬” and found it to be a strong example of how macro-architecture design + mathematical constraints can directly improve large-scale model training. 🔍🧠

🔹 𝐂𝐨𝐧𝐭𝐞𝐱𝐭
- Hyper-Connections (HC) extend residual connections by widening the residual stream into multiple parallel paths. This increases topological capacity without increasing FLOPs inside each layer - a very attractive idea for scaling.

⚠️ 𝐓𝐡𝐞 𝐩𝐫𝐨𝐛𝐥𝐞𝐦 𝐰𝐢𝐭𝐡 𝐇𝐂
- At scale, HC breaks a key property of residual learning: identity mapping.
 Unconstrained residual mixing causes:

1. Exploding / Vanishing signals 📉📈
2. Unstable gradients
3. Training collapse in deep or large models

🔧 𝐖𝐡𝐚𝐭 𝐦𝐇𝐂 𝐜𝐡𝐚𝐧𝐠𝐞𝐬
- mHC fixes this by constraining residual mixing onto a geometric manifold.
Instead of unconstrained matrices, residual mixing is projected (via Sinkhorn–Knopp) onto the Birkhoff polytope:

1. Non-negative entries
2. Rows sum to 1
3. Columns sum to 1

📐 𝐖𝐡𝐲 𝐭𝐡𝐢𝐬 𝐰𝐨𝐫𝐤𝐬 (𝐦𝐚𝐭𝐡 𝐢𝐧𝐭𝐮𝐢𝐭𝐢𝐨𝐧)
 ✔️ Residual updates become convex combinations
 ✔️ Signal mean is preserved
 ✔️ Composite residual mappings remain stable across depth
 ✔️ Identity mapping is generalized, not removed

In short: expressive multi-stream residuals without breaking optimization.

⚙️ Systems work (very important)
- This isn’t just theory - the paper includes serious systems engineering:
1. Fused kernels for normalization + projection
2. Recomputation to control memory
3. Pipeline-parallel scheduling to hide overhead

📊 Result: only ~6.7% training overhead at expansion rate n = 4.

📈 Empirical results
- HC shows gradient explosions at large scale ❌
- mHC remains stable and converges cleanly ✅
- Consistent gains over baseline Transformers and HC on reasoning benchmarks (BBH, DROP, GSM8K, MATH)
- Improvements persist from 3B → 27B models

🧩 Why this paper matters
- It reframes residual connections as operators living on a constrained manifold, not just skip connections.
- A great example of how geometry, optimization, and systems design should co-evolve in large-model architecture.
- Highly recommended if you’re thinking about:
 🔹 residual stream scaling
 🔹 training stability at depth
 🔹 architecture beyond attention/FFN tweaks

Happy to discuss or hear others’ takes! 💬

#AIResearch #DeepLearning #LargeLanguageModels #ModelArchitecture #ResidualConnections #TrainingStability #GeometricDeepLearning #MLSystems #Optimization #FoundationModels

mHC: Manifold-Constrained Hyper-Connections.

taplio

The Battle

Pascal Biese

Vedant Pandya

Followers

Posts

Engagements

Likes

Performance ScorePascal Biese's LinkedIn Performance

Performance ScoreVedant Pandya's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Pascal Biese

Vedant Pandya

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Pascal Biese's LinkedIn Performance

Performance ScoreVedant Pandya's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScorePascal Biese's LinkedIn Performance