I create content about deep learning | The DevRel and Deep Learning Dude

Hacking on LLMs, curating multimodal datasets, and writing a book about RAG.

Harpreet Sahota 🥑

i've literally been waiting for years to have an OPEN SOURCE model like qwen3-vl-embedding

i tested its multimodal retrieval capabilities on a corpus of 412 short video clips and the results literally blew my mind

here are the queries i tested:

1. a cartoon guy drinks merlot wine

i like this query because we see how it can retrieve based on semantics (a cartoon), text (the label merlot), and temporal action in the video (the cartoon guy drinks the wine mid-way through the video)

2. garfield runs out of the door

semantics are awesome here, the model knows i'm talking about the cartoon character...

3. a woman in a blue dress sleeping on a red bench

the skeptic in you might think that it's retrieving just based on the red color block in the 2/3 of the video...but notice the specific part of the query "a woman in a blue dress..." and this is only shown in 3s out of the full 10s

this is such a huge release 

and it's gonna open up SO much more for multimodal video retrieval this year

on my wishlist is natural language search of pcd datasets, who gonna ship that?

you can hack around with the model using the resources below

check out the docs: here: https://lnkd.in/gh7yfBNP

and the quickstart nbhere:  https://lnkd.in/gbinsx7p

🤖 Hacker-in-Residence @ Voxel51| 👨🏽‍💻 AI/ML Engineer | 👷🏽‍♀️ Technical Developer Advocate | Learn. Do. Write. Teach. Repeat.

for those who missed it, or were there in-person and know the trouble i was facing 😆, my ODSC webinar is now live on youtube!

in this session i discuss visual ai techniques for automotive damage detection. 

i used the cardd dataset, which is a public dataset built specifically for vehicle damage analysis and demonstrated how fiftyone helps you explore, visualize, and build models to detect dents, scratches, cracks, glass shatter, tire flats, and broken lamps.

i feel like its a super practical workshop with some insightful workflows for visual AI in automotive settings, but as the creator i might be biased.

only one way to find out for yourself, watch the webinar!

https://lnkd.in/ewfz5vZD

Open video: Mastering Visual AI with Vision-Language Models & Advanced Evaluation Techniques by Harpreet Sahota on youtube.com

Mastering Visual AI with Vision-Language Models & Advanced Evaluation Techniques by Harpreet Sahota

sometimes i look back at the shit i get done and i genuinely impress myself

how teams really build Visual AI in the wild? what's the he messy reality?  

i don't know...and maybe its because nobody’s asking practitioners and compiling their answers so we can find out the landscape actually looks like. 

so, we at Voxel51 are kickin off the...

wait for it...

 State of Visual AI survey...vey...vey

for those out ther building visual AI systems i want your hot takes

what’s the hardest part about making visual ai a reality? 

let us know in this survey

it'll take you 30 minutes and is a much better use of time than baby sitting your runs and staring at your loss curves

i can't offer much beyond my heartiest thanks, but you will get early access to the full report with benchmarks, common pain points, and where you stand within the landscape. 

here's a link: https://lnkd.in/gnsADACy

MedGemma 1.5 supports detection, but for best results, you'll need to fine-tune. 

there's a kaggle competition using the model, i created a starter notebook to give you a jump start on how to fine-tune it for detection

checkout the starter notebook and additonal resources 👇🏼

cc Omar Sanseviero Megan Risdal Logan Kilpatrick

in the asu area and wanna get started with open source contributions? then don't miss this hack

On March 21, 2026, we’re hosting Agents World: Visual AI Hackathon @Arizona State University, together with Pavan Turaga 🚀 

This is a hands-on day: meet people, build with AI, eat pizza, and leave with something you created after a short workshop and a full day of hacking. 
You’ll work with the FiftyOne ecosystem and real models used in practice, like Qwen, Gemini Vision, Sharp, and more. 

If you’re an ASU student who wants to build, learn by doing, and work with real Visual AI tools, this is for you.

Registration is open:
https://lnkd.in/gBCHW5Fg

Hope to see you at ASU. I’ll be there with the amazing Voxel51 team:  Sid Mehta, Ritchie Martori, Jimmy Guerrero ! 🙌

looking forward to the next meetup in boston!

Join our in-person meetup at the Microsoft Research Lab at the Massachusetts Institute of Technology in Boston, Massachusetts on Feb 18 to hear talks from experts on cutting-edge topics across AI, ML, and computer vision. Register here: https://lnkd.in/gi4kpSMB

Talks will include:

• SNAP: Towards Segmenting Anything in Any Point Cloud - Hanhui Wang at Northeastern University

• Culturally Adaptive AI - Anku R. at Massachusetts Institute of Technology

• Data Foundations for Vision-Language-Action Models - Harpreet Sahota 🥑 at Voxel51

• Better Together: Leveraging Unpaired Multimodal Data for Stronger Unimodal Models - Sharut G. at Massachusetts Institute of Technology

• Neural Radiance Fields for Image Verification - Sage Simhon at Pasteur Labs & ISI

#computervision #ai #artificialintelligence #machinevision #machinelearning #datascience

Syed Abdul Sami Rizvi, Matthew Ebisu and 118 other attendees

View event: Feb 18 - Boston AI, ML and Computer Vision Meetup

Feb 18 - Boston AI, ML and Computer Vision Meetup

Inspired by the amazing Hugging Face’s 2025 AI Timeline, we created one for the Fiftyone community. The FiftyOne Community Development Timeline highlights more than 131 contributions from 2025, including datasets, models, plugins, and integrations.

You can filter by type or modality and jump straight to each integration.
https://lnkd.in/ev2kpCGN

Big thanks to the entire Voxel51 community for building all of this together.
Harpreet Sahota 🥑, Daniel Gural, Paula Ramos, PhD, Jimmy Guerrero, Derek Van de Streek, Nicholas Lotz 🚀

Gonna be a great series of talks, lots of technical depth!

Chief Community Officer @ Voxel51 | Startup Advisor

Join us on Feb 18 for the Boston AI, ML and Computer Vision Meetup! Pre-registration is mandatory to clear building security - https://lnkd.in/gHtBuH49

Talks will include:

* SNAP: Towards Segmenting Anything in Any Point Cloud - Hanhui Wang at Northeastern University

* Culturally Adaptive AI - Anku R. at MIT

* Data Foundations for Vision-Language-Action Models - Harpreet Sahota 🥑 at Voxel51

* Better Together: Leveraging Unpaired Multimodal Data for Stronger Unimodal Models - Sharut G. at MIT

* Neural Radiance Fields for Image Verification - Sage Simhon at Pasteur Labs & ISI

***********

Want to build better computer vision models? FiftyOne is an open source toolkit from Voxel51 (our Meetup sponsor) that helps you curate datasets, evaluate model performance, visualize embeddings, catch annotation errors, and eliminate duplicate images—all in one place.

“pip install fiftyone” is all it takes to get started - https://docs.voxel51.com/

#computervision #ai #artificialintelligence #machinevision
#machinelearning #datascience #physicalai

Greg Coquillo

Product Leader @AWS | Startup Investor | 2X Linkedin Top Voice for AI, Data Science, Tech, and Innovation | Quantum Computing & Web 3.0 | I build software that scales AI/ML Network infrastructure

Product leader with a passion for solving business use cases such as Growth, Go-to-Market, and Price Optimization. I owned P&L for commodity-based products, and leveraged AI to increase the average customer contract values across business industries. As a Product Leader at Amazon, I own a roadmap that drives the adoption of AI products through the Media Automation charter, which supports Private Brands’ YOY growth. I work with Data Scientists, Software Engineers and Business Intelligence Engineers to leverage Machine Learning and Big Data Analytics to design predictive models that enhance business decision-making and performance to accelerate growth, reduce Time to Market of products, monitor cross-geo compliance and import requirements, and ensure customer safety.

I am always curious and eager to learn how new technology affects our daily lives, and I love to support communities on their quest to adopt Artificial Intelligence across industries. Feel free to contact me if you have Data Science projects that you’d like to be showcased on this platform for visibility.

🌐I am a Global Business Leader with 12 years of experience and skills including Product Management, P&L management, Global Networking, EBITDA Optimization, Pricing Management, Margin Management, Product Management, Program Management, Operations and Supply Chain Management, Lean Manufacturing, Six Sigma, and Continuous Improvement.

🧘🏽‍♂️Soft Skills include: Communication, Collaboration, Lateral Thinking, Adaptability, Inspiration, Innovation, Empathy, Listening, Diplomacy, Organization, Responsibility, Leadership...

🎬🎥🎞 I really enjoy watching movies, and if I could redo it all over, I’d be a Sci-Fi script writer so I can be bounded by the limits of imagination.

(Opinions are my own)

Product Management, Media Automation

Owner of the AI vision and roadmap for Private Brand’s Compliance Division. Driving adoption of new technology, allowing my customers to scale their processes and increase productivity. Core responsibilities include:

-Building cross-functional teams consisting of Data Scientists, Software Engineers, Data Engineers, Product Managers and Program Managers, to plan and implement sophisticated AI projects
-Manage the New Initiative Intake Process
-Gather Business requirement artifacts
-Communicate roadmap definition and prioritization
-Coordinate User Acceptance Testing
-Manage Launch Announcements
-Prepare and communicate annual planning artifacts

AI Agents are task-specific, autonomous systems that integrate large language models with structured tools, APIs, and real-time data sources. 

They operate across domains such as cybersecurity, supply chain logistics, and healthcare by executing workflows that traditionally required human-in-the-loop decision making. 

These agents leverage vector databases, retrieval-augmented generation, and fine-tuned embeddings to enable contextual reasoning and dynamic response generation. 

As orchestration frameworks mature, multi-agent systems are increasingly capable of handling end-to-end processes like demand forecasting, patient triage, and adaptive tutoring with minimal supervision.

The below chart shows just how broad their impact is:

1.🔹 IT & Security : Phishing filters, threat detection, patch suggestions
2.🔹Healthcare : Patient alerts, medical chatbots, symptom matching
3.🔹 Education : Flashcards, concept explainers, AI tutors
4.🔹 Sales & Marketing : Lead scoring, campaign ideas, email outreach
5.🔹Logistics : Fleet tracking, demand forecasting, inventory updates
6.🔹Manufacturing : Predictive maintenance, robotic control, energy monitoring
7.🔹 Research : Academic writing, data cleaning, topic expansion
8.🔹 Customer Support : FAQ bots, emotion detection, chat summaries
9.🔹 Smart Environments : Digital twins, voice commands, access control
10.🔹Ops Automation : Shift scheduling, system alerts, order tracking

What used require significant manual effort, now takes a few smart agents.

I believe it’s a great time to start exploring and experimenting in this space…

If you’re learning AI automation without a roadmap, you’re guaranteed to get overwhelmed.

People usually “learn AI automation” by jumping straight into tools… and then wonder why nothing works consistently.
Real automation requires structure - thinking, logic, testing, and a gradual build-up of skills.

This 18-day roadmap breaks down the exact sequence to go from zero → confidently building automations with AI, APIs, tools, and no-code platforms.

Here’s the full breakdown, day by day:

Day 1 - AI Automation Fundamentals
Learn what automation really means, how it differs from AI and agents, and see real examples.

Day 2 - Automation Thinking
Break work into steps, triggers, and outcomes - the mindset behind every good automation.

Day 3 - APIs & Webhooks Basics
Understand how apps communicate and how events trigger workflows.

Day 4 - No-Code Automation Platforms
Explore Zapier, Make, n8n - and how no-code tools actually run workflows.

Day 5 - Build Your First Automation
Create a simple trigger-action workflow and connect two apps.

Day 6 - Data Handling
Pass data between steps, map fields, and work with text, numbers, and dates.

Day 7 - Logic & Error Handling
Add filters, conditional logic, retries, and fallbacks to keep automations reliable.

Day 8 - AI Model Basics
Learn prompts vs system instructions, tokens, limits, and LLM behavior.

Day 9 - Using AI Inside Automations
Insert AI steps into workflows and parse structured AI outputs.

Day 10 - Prompt Design for Automation
Write consistent prompts and reduce hallucinations with JSON outputs.

Day 11 - Text-Based Task Automation
Automate email replies, summaries, CRM updates, and document tasks.

Day 12 - Knowledge Automation (RAG Basics)
Connect AI to internal documents and fetch accurate answers from real data.

Day 13 - AI Agents Basics
Understand agent planning, tools, and identify use cases for agents.

Day 14 - Business Use Case Automation
Automate lead qualification, ticket routing, and internal processes.

Day 15 - Sales & Marketing Automation
Personalize outreach, repurpose content, and automate follow-ups.

Day 16 - Operations Automation
Manage approvals, notifications, and repetitive operational tasks.

Day 17 - Monitoring & Optimization
Track workflow success, cut costs, and improve performance.

Day 18 - Build & Ship Your System
Design, test, document, and finalize a complete end-to-end automation.

You don’t master AI automation by learning tools, you master it by learning systems thinking, data flow, and structured execution.

Follow this roadmap, and you’ll build automations that are reliable, scalable, and business-ready.

Data ➝ AI ➝ Value.
That’s still what companies think AI Looks like

But in the real world…

AI inside companies looks nothing like that.

Because value doesn’t come from “building a model”.

Value comes from building a working system around the model.

This guide explains what AI actually looks like in companies 👇

It’s a full production chain, not a single step:

1) Data
Selection, sourcing, synthesis - getting usable data is already 50% of the work.

2) Data Science
Exploration → cleaning → normalization → feature engineering → scaling
Then modeling: selection → training → evaluation → tuning

3) Value (Operationalizing)
Real value starts only after:
registration → deployment → monitoring → retraining

So ML is not “AI work”.

It’s data engineering + modeling + MLOps, all together.

What GenAI-enabled AI actually looks like

GenAI doesn’t remove complexity.

It shifts complexity from “models” to workflows + control.

The real GenAI path is:

- Understand the decision process
What decisions are being made today? Where does judgment happen?

- Workflow experimentation
Iterate with prompts, APIs, tools, retrieval, automation, coding — until it works reliably.

- Guardrails
Policies, safety filters, access control, cost limits, prompt rules, grounding.

- Monitoring
Track quality, drift, failure modes, user behavior, data leakage, hallucinations.

- Human in the loop
Because the highest-value workflows still need supervision, approvals, and feedback.

The takeaway?

AI isn’t a “feature.”

AI is an operating system change.

And companies that understand this early…

ship faster, safer, and with real ROI.

If you're building enterprise AI, stop chasing “Data → AI → Value”.

Start building: Workflows → Guardrails → Monitoring → Value.

#AI

Knowing RAG is easy. Measuring it is the real skill.

Everyone can plug in a vector DB and call it a RAG system.
But only a few can actually evaluate whether the system retrieves the right knowledge, avoids hallucinations, and answers with confidence.

That’s where retrieval metrics matter, they turn guesswork into engineering.

Here’s a simple breakdown of the 20 metrics that actually show whether your RAG system is good… or just lucky.

1. Precision@K - Checks how many retrieved chunks are truly relevant.
2. Recall@K - Measures whether all important chunks were found.
3. Hit Rate - Confirms if at least one correct chunk appears.
4. MRR - Rewards results where the correct chunk ranks high.
5. NDCG@K - Evaluates ranking quality, not just correctness.
6. Top-1 Accuracy - Tests whether the first chunk is the best one.
7. Query Coverage - Checks if retrieval answers the entire question.
8. Evidence Sufficiency - Asks: “Is this enough to answer fully?”
9. Context Relevance Score - Measures alignment between query meaning and chunk content.
10. Chunk Redundancy Rate - Tracks repeated or duplicate chunks.
11. Chunk Diversity Score - Shows whether results come from varied, reliable sources.
12. Source Coverage - Checks if retrieval pulls from the right documents.
13. Citation Accuracy - Confirms the final answer cites real, correct evidence.
14. Groundedness Score - Tests how strongly the answer stays rooted in retrieved context.
15. Faithfulness - Ensures the model doesn’t distort the evidence.
16. Hallucination Rate - Measures unsupported claims the model invents.
17. Retrieval Latency - How fast chunks reach the model.
18. Token Efficiency - How much useful context fits into the token limit.
19. Freshness - Checks if the retrieved info is up-to-date.
20. Retrieval Failure Rate - Tracks how often retrieval returns irrelevant or misleading chunks.

Anyone can build RAG.
Only skilled teams can measure retrieval, and that’s what decides whether your system scales or breaks.

Which metric do you think is the most underrated right now?

Single agents are powerful.
But multi-agent systems?
That’s where AI starts behaving like a real team.

As AI moves from “answering questions” to doing actual work, the real unlock isn’t a bigger model, it’s how multiple agents collaborate.

Different patterns fit different types of tasks, and choosing the right one can be the difference between a smart system and a chaotic one.

Here are 8 multi-agent collaboration patterns every builder should understand:

1. Manager–Worker Pattern
One agent breaks down the task, others execute the pieces.
Great for large, structured work like research + writing + QA.

2. Specialist Agents (Role-Based Team)
Each agent has a defined role - researcher, analyzer, writer.
Best when accuracy and expertise matter.

3. Planner–Executor Pattern
A planner decides the steps; an executor runs tools and actions.
Ideal for automation tasks, workflows, and system integrations.

4. Critic / Reviewer Agent
One agent generates, another fixes mistakes.
Useful for RAG responses, code checks, compliance-sensitive tasks.

5. Debate Pattern
Agents challenge each other with alternative solutions until the best idea wins.
Good for strategy, architecture, and trade-off decisions.

6. Voting / Consensus Pattern
Multiple agents propose answers, the system picks the best one.
Great for open-ended, ambiguous, or high-risk queries.

7. Swarm / Parallel Exploration
Many agents explore multiple sources at once.
Perfect for fast research and time-sensitive retrieval.

8. Tool Router / Dispatcher Pattern
A router agent decides which tool or agent should handle the request.
Works best in enterprise setups with many databases, APIs, and systems.

AI isn’t becoming powerful because models are getting bigger.
It’s becoming powerful because agents are learning to work together.

This is what turns AI experiments into real products.

Teams get excited about the model… 
but the model is only one slice of the stack.

Real AI products work because every layer around it is engineered with intent.

This breakdown shows what a modern, production-grade AI product stack actually looks like: 

1. Application Layer
The user-facing shell - authentication, UI, APIs, monitoring, analytics. This is where reliability, speed, and trust are felt long before AI runs.

2. Agentic AI Layer
The intelligence engine - reasoning, retrieval, tool use, orchestration, and memory. This layer decides whether your system simply answers… or actually acts.

3. GenAI Ops Layer
The guardrails - tracing, evaluation, experiments, and quality checks. This is how you keep an AI system stable as it learns and evolves in production.

4. Tools & Services Layer
The essentials - model APIs, data warehouses, and external integrations. These services give your AI the inputs, structure, and capabilities it needs to function.

5. Infrastructure Layer
The backbone - compute, storage, GPUs, networking, and security. If this layer is weak, nothing above it scales safely.

AI products don’t succeed because of one great model.
They succeed because every layer, from UI to GPUs - works together as one system.

RAG is not just “upload PDFs + chat with them.”

But that’s like calling every vehicle a “car.”

Because in real AI systems, there are 3 completely different architectures hiding behind the same word:

Naive RAG vs Agentic RAG vs Agentic Memory.

And if you don’t know the difference, you’ll build the wrong system… and blame the LLM later.

Here’s what’s actually happening : 

1. Naive RAG (Basic Retrieval)
Naive RAG is the most common setup: documents get indexed once, stored in a vector database, and whenever a user asks a question, the system fetches the closest chunks and sends them to the LLM. It works well for simple document Q&A, but the accuracy completely depends on retrieval quality  - if the wrong chunk is retrieved, hallucinations become inevitable.

2. Agentic RAG (Tool-based Retrieval + Reasoning)
Agentic RAG is not a fixed pipeline - it’s a decision-making system. Instead of blindly retrieving from a single database, an agent dynamically decides which sources to search, which tools to use, whether the answer needs verification, and when to retry retrieval. This is what enables enterprise-grade search across documents, CRMs, knowledge bases, and even the web - but it also introduces more complexity and cost.

3. Agentic Memory (Long-term Learning Loop)
Agentic Memory is not about pulling knowledge from documents - it’s about building continuity. The system stores useful signals from past interactions (preferences, decisions, repeated tasks, user context) and retrieves them later to give better responses over time. This turns the assistant into something that evolves with usage but if memory isn’t governed properly, it can quickly become noisy, irrelevant, or even risky.

The easiest way to remember it:

Naive RAG = answers from stored docs
Agentic RAG = answers using tools + multiple sources
Agentic Memory = answers that improve over time

If you’re building agents in 2026…
understanding this difference is not optional, it’s architecture.

taplio

The Battle

Harpreet Sahota 🥑

Greg Coquillo

Followers

Posts

Engagements

Likes

Performance ScoreHarpreet Sahota 🥑's LinkedIn Performance

Performance ScoreGreg Coquillo's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Harpreet Sahota 🥑

Greg Coquillo

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Harpreet Sahota 🥑's LinkedIn Performance

Performance ScoreGreg Coquillo's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreHarpreet Sahota 🥑's LinkedIn Performance