Pascal Biese's Linkedin Analytics

Reinforcement learning is touted as the secret sauce for AI reasoning. But there might be an unexpected catch... Many practitioners fine‑tune language models with reinforcement learning (RL) hoping they’ll “figure things out” on their own. A new paper puts that claim under the microscope. They asked a simple question: if you let a vanilla model take lots of shots at a problem, can it already solve the very tasks an RL‑tuned model aces on the first try? How they tested it — in plain words: 1. Lots of retries. Instead of judging a model on one answer, they let it guess up to 256 times (“pass@k”). 2. Track the score. RL models do better on the first guess, but as retries pile up, the untuned model catches up—and eventually passes it. 3. Peek inside. The RL model’s reasoning steps already existed in the base model; RL just nudged the model to pick those tried‑and‑true paths more often. 4. Cross‑checks. Same story in math puzzles, coding challenges, and picture‑based questions. What this means: According to their results, RL is an efficiency booster - not a talent creator. It makes the first answer more likely to be right, but it narrows the range of ideas the model will explore. Distillation (learning from a stronger teacher) does broaden skills - so if you need new tricks, start there or rethink the training recipe altogether. Bottom line: I don't think the idea of RL leading to better - and even "unintended" - reasoning will drop in popularity after this paper. Even if their results could be replicated, as long as the numbers go up and users feel like there's an actual improvement... few will care. Afterwards, of course, if we hit a wall, everyone will point to work like this and say that it was obvious. Personally, I do think RL with Verifiable Rewards (RLVR) is part of the solution. But is it the whole solution? ↓ 𝐖𝐚𝐧𝐭 𝐭𝐨 𝐤𝐞𝐞𝐩 𝐮𝐩? Join my newsletter with 50k+ readers and be the first to learn about the latest AI research: llmwatch.com 💡

Will GenAI take over the workplace? Let's find out what IBM says. A new study from IBM Research reveals that knowledge workers' use of Large Language Models is evolving rapidly - but not in the ways many predicted. While early usage focused on text generation, workers are increasingly interested in integration with workflows and data analysis rather than just content creation. The researchers conducted two surveys of knowledge workers in 2023 and 2024, categorizing LLM usage into four types: creation (generating text and ideas), information (search, learning, summarizing), advice (improving content, guidance, validation), and automation. The longitudinal approach revealed a noteworthy shift: while content creation dominated early adoption, knowledge workers now prioritize information tasks and desire advanced analytics and automation capabilities that tap into their proprietary data. Perhaps most interesting is that despite widespread fears of AI replacing knowledge work, the research shows workers view LLMs as collaborative tools within larger workflows rather than as replacements. Workers primarily used LLMs for tasks they previously handled manually, but now want systems that can access their specialized knowledge, organizational context, and integrate with existing tools. In the not too distant future, their role in the workplace will likely shift from isolated assistants to integrated components of knowledge work ecosystems. Leaders should focus less on replacing workers and more on building systems that enhance human capabilities through meaningful collaboration with AI. ↓ 𝐖𝐚𝐧𝐭 𝐭𝐨 𝐤𝐞𝐞𝐩 𝐮𝐩? Join my newsletter with 50k+ readers and be the first to learn about the latest AI research: llmwatch.com 💡