Voxel51

• Explore, tinker with, and push the boundaries of the latest AI and machine learning technologies to showcase what's possible with open-source tools and frameworks

• Build inspiring prototypes, proof-of-concepts, and example applications that demonstrate creative and impactful uses of AI to the developer community

• Contribute meaningfully to the open source ecosystem by creating and maintaining libraries, tools, datasets, tutorials and guides that enable developers to more easily leverage AI capabilities

• Curate, clean, annotate and publish unique multimodal datasets spanning text, images, audio, video, 3D and other emerging data types to facilitate novel machine learning research and applications

• Create engaging technical blog posts, social media content, videos, and presentations that educate and inspire developers and data scientists to build with AI

•  Collaborate openly with our internal teams and the broader AI/ML community to cross-pollinate ideas, share knowledge, and collectively advance the state-of-the-art

• Evangelize our technology and vision by representing the company at hackathons, conferences, meetups, universities, and other developer-focused events

•  Influence the product roadmap by providing input on the developer experience, uncovering opportunities for new features and improvements based on community feedback and your work

• Continuously learn, experiment with, and provide feedback on beta versions of our latest models and capabilities to help shape the future of our platform

Hacker-in-Residence

LinkedIn

1) Introductory computer vision course for LinkedIn Learning

2) Prompt Engineering with LangChain (releasing March 2024)

3) RetrievalAugemnted Generation with LlamaIndex (releasing Q2 2024)

4) Fine-tuning LLMs with Hugging Face TRL (est release Q3 2024)

5) Text Generation with Hugging Face Transformers (est release Q3 2024)

LinkedIn Learning Instructor

The Artists of Data Science

The ONLY personal growth and development podcast for Data Scientists

Host

Deci AI

Developer Relations Manager

Pachyderm Inc.

Senior Developer Advocate

Comet

ML Developer Advocate

Data Science Dream Job

Principal Data Science Educator

Price Industries

Lead Data Scientist

Bold Commerce

Senior Data Scientist

Emergent BioSolutions

Biostatistician

I create content about deep learning | The DevRel and Deep Learning Dude

Hacking on LLMs, curating multimodal datasets, and writing a book about RAG.

Harpreet Sahota 🥑

sam3 is seriously a step change improvement over sam2

the model is seriously good, glad they brought back text prompts. the automatic segmentation isn't as good as i hoped it to be...but it still works quite well. 

i also got it to output image embeddings and that was cool to visualize in the app

learn more here: https://lnkd.in/gzqJzj6G

🤖 Hacker-in-Residence @ Voxel51| 👨🏽‍💻 AI/ML Engineer | 👷🏽‍♀️ Technical Developer Advocate | Learn. Do. Write. Teach. Repeat.

qwen3vl is literally the best model i've used for video usecases. blows my mind

hosting a virtual event tomorrow about document ai

you may or may not find it interesting, but i guarantee you'll learn something

join me here: https://lnkd.in/dFXPimVc

sony ai released a pretty cool dataset called the fairness human centric image benchmark, super high quality labels

instructions for downloading and parsing the dataset into fiftyone format can be found here: https://lnkd.in/g_VwxJx4

physical AI is top of mind as the next frontier for deep learning. but it'll only succeed if you have good datasets

language models scaled with web-scale text. 

vision models got ImageNet. 

now Voxel51 and NVIDIA are trying to do the same thing for physical AI 

autonomous vehicles, humanoid robots, anything that moves around atoms.

building these systems means millions of hours of testing in high-fidelity sims before you can actually deploy anything. 

petabytes of multimodal sensor data from various modalities

 LiDAR, cameras, radar, IMUs. 

but over 50% of simulations end up completely unusable because the input data is bad.

even small calibration or timing errors between sensors cascade through your neural reconstructions. 

you waste weeks and burn through millions in compute costs before you realize what went wrong.

the new FiftyOne Physical AI Workbench. it integrates with NVIDIA Omniverse and NVIDIA Cosmos and basically sits at the start of your sim pipeline to validate, clean up, and prep your multimodal data before it hits neural reconstruction and synthetic data generation.

what it does:

- automatically audits physical AI inputs across 75+ critical checkpoints

- transforms raw sensor streams into structured, searchable datasets

- triggers accurate neural reconstructions and generates synthetic scene variations

this means fewer wasted compute hours. 

iteration loops go from weeks to hours. 

you build safer, more reliable systems on data you can actually trust.

read more about what we've been building: https://lnkd.in/gC6_zn-X

i recently integrated 4 ocr models into fiftyone as remote zoo models these handle text extraction and document parsing. 

all available as remote zoo sources, you can get started with a few lines of code

different approaches for different needs:

1. mineru-2.5

1.2b params, two-stage strategy: global layout on downsampled image, then fine-grained recognition on native-resolution crops. 

handles headers, footers, lists, code blocks. strong on complex math formulas (mixed chinese-english) and tables (rotated, borderless, partial-border).

good for: documents with complex layouts and mathematical content

https://lnkd.in/dSwimdvp

deepseek-ocr

dual-encoder (sam + clip) for "contextual optical compression." 

outputs structured markdown with bounding boxes. has five resolution modes (tiny/small/base/large/gundam). gundam mode is the default - uses multi-view processing (1024×1024 global + 640×640 patches for details). 

supports custom prompts for specific extraction tasks.

good for: complex pdfs and multi-column layouts where you need structured output

https://lnkd.in/dhJn2h-v

olmocr-2

built on qwen2.5-vl, 7b params. outputs markdown with yaml front matter containing metadata (language, rotation, table/diagram detection). 

converts equations to latex, tables to html. labels figures with markdown syntax. reads documents like a human would.

good for: academic papers and technical documents with equations and structured data

https://lnkd.in/dNdTQYKg

kosmos-2.5

microsoft's 1.37b param multimodal model. two modes: ocr (text with bounding boxes) or markdown generation. automatically optimizes hardware usage (bfloat16 for ampere+, float16 for older gpus, float32 for cpu). handles diverse document types including handwritten text.

good for: general-purpose ocr when you need either coordinates or clean markdown

https://lnkd.in/dCTfGfV8


two modes typical across these models: detection (bounding boxes) and extraction (text output)

i also built/revamped the caption viewer plugin for better text visualization in the app: 

https://lnkd.in/dBebWdga

i've also got two events poppin off for document visual ai:

- nov 6 (tomorrow) with a stellar line up of speakers: https://lnkd.in/gkBEY4_r

- a deep dive into document visual ai with just me: 

https://lnkd.in/dFXPimVc

Open article: Document Visual AI with FiftyOne - November 14, 2025 by voxel51.com

Document Visual AI with FiftyOne - November 14, 2025

hype for this new integration, and proud of the team for shipping it at lightning speed. big ups to Daniel and AdonAI for goin beast mode on this one

🚀 Introducing Physical AI Workbench — integrated with NVIDIA Omniverse NuRec and NVIDIA Cosmos to solve the biggest bottleneck in physical AI.  

More than half of all physical AI simulations fail — not from bad models, but from bad input data. A timestamp off by milliseconds or a calibration drift of a few degrees can cascade through your neural reconstructions, leaving you with unusable results after weeks and millions in wasted compute. 

FiftyOne Physical AI Workbench sits at the start of your simulation pipeline and provides turnkey access to NVIDIA technology to ensure every simulation run begins with trustworthy data: 

✅ Audits 75+ critical checkpoints to ensure every reconstruction starts with validated inputs 
🎯 Enriches raw sensor streams into structured, searchable datasets 
🔗 Integrates directly with NVIDIA Omniverse and NVIDIA Cosmos for high-fidelity reconstruction and synthetic data generation. 

The result: an end-to-end workflow that prevents downstream failures — cutting iteration cycles from weeks to hours and eliminating GPU waste on failed reconstructions. We’re excited to see this workflow power autonomous vehicles NVIDIA DRIVE and robotics NVIDIA Robotics development teams move faster with higher accuracy. 

🔗 Read the full technical breakdown: https://lnkd.in/gTy5-TNF

Greg Coquillo

Product Leader @AWS | Startup Investor | 2X Linkedin Top Voice for AI, Data Science, Tech, and Innovation | Quantum Computing & Web 3.0 | I build software that scales AI/ML Network infrastructure

Product leader with a passion for solving business use cases such as Growth, Go-to-Market, and Price Optimization. I owned P&L for commodity-based products, and leveraged AI to increase the average customer contract values across business industries. As a Product Leader at Amazon, I own a roadmap that drives the adoption of AI products through the Media Automation charter, which supports Private Brands’ YOY growth. I work with Data Scientists, Software Engineers and Business Intelligence Engineers to leverage Machine Learning and Big Data Analytics to design predictive models that enhance business decision-making and performance to accelerate growth, reduce Time to Market of products, monitor cross-geo compliance and import requirements, and ensure customer safety.

I am always curious and eager to learn how new technology affects our daily lives, and I love to support communities on their quest to adopt Artificial Intelligence across industries. Feel free to contact me if you have Data Science projects that you’d like to be showcased on this platform for visibility.

🌐I am a Global Business Leader with 12 years of experience and skills including Product Management, P&L management, Global Networking, EBITDA Optimization, Pricing Management, Margin Management, Product Management, Program Management, Operations and Supply Chain Management, Lean Manufacturing, Six Sigma, and Continuous Improvement.

🧘🏽‍♂️Soft Skills include: Communication, Collaboration, Lateral Thinking, Adaptability, Inspiration, Innovation, Empathy, Listening, Diplomacy, Organization, Responsibility, Leadership...

🎬🎥🎞 I really enjoy watching movies, and if I could redo it all over, I’d be a Sci-Fi script writer so I can be bounded by the limits of imagination.

(Opinions are my own)

Product Management, Media Automation

Owner of the AI vision and roadmap for Private Brand’s Compliance Division. Driving adoption of new technology, allowing my customers to scale their processes and increase productivity. Core responsibilities include:

-Building cross-functional teams consisting of Data Scientists, Software Engineers, Data Engineers, Product Managers and Program Managers, to plan and implement sophisticated AI projects
-Manage the New Initiative Intake Process
-Gather Business requirement artifacts
-Communicate roadmap definition and prioritization
-Coordinate User Acceptance Testing
-Manage Launch Announcements
-Prepare and communicate annual planning artifacts

Building Agentic AI systems beyond connecting APIs or LLMs is complicated, but not impossible.

This architecture lays the foundation for how AI agents think, communicate, and improve, covering everything from testing and observability to deployment and memory management.

Here’s a breakdown of the key layers and components that make up a scalable Agentic AI Architecture :

1.🔸Decomposition
Break down complex systems by domain (e.g., Coding Agent, Data Agent), by cognitive capability (Reasoning, Planning, Execution), or by agent role (Planner, Executor, Memory Manager, Communicator).

2.🔸Communication
Enable message passing between agents using inter-agent protocols or A2A (Agent-to-Agent) orchestration.
Support both single-agent and multi-agent setups for small or distributed workflows.

3.🔸Deployment
Deploy agents in containerized or serverless environments using Docker or Modal.
Support orchestrators like CrewAI or AutoGen for collective intelligence in multi-agent workflows.

4.🔸Data & Discovery
Integrate knowledge bases (like vector databases for RAG), memory stores (FAISS, Redis, Pinecone), and APIs for dynamic data access.
Context is passed using Model Context Protocol (MCP) for structured and real-time reasoning.

5.🔸Testing & Observability
Validate workflows end-to-end, test reasoning logic, and evaluate performance under real conditions.
Monitor using Weights & Biases, LangFuse, and track metrics like latency and task success rate.

6.🔸UI & Style
Provide intuitive feedback loops through visualization layers, dashboards, and self-reflective modes.
Enable collaborative, proactive, and goal-driven reasoning among multiple agents.

7.🔸Security
Protect access with token-based authorization and data encryption.
Include Trust Layers for human-in-the-loop validation and Policy Enforcement for safe execution.

8.🔸Cross-Cutting Concerns
Handle configuration, secrets, and environment management.
Support flexible frameworks like LangChain, AutoGen, or CrewAI for runtime execution and modular design.

Agentic AI is the future of automation - where AI doesn’t just assist but collaborates and learns.

Save this post to understand the architecture that powers the next generation of AI systems

#AgenticAI

No alternative text description for this image

AI agents are great at working alone. However, they can also be effective when operating within networks of collaboration.

The real power of AI lies in how these agents communicate, coordinate, and complement each other’s abilities.

Orchestration patterns define this collaboration which turns isolated agents into intelligent, goal-driven systems capable of solving complex tasks together.

Here are 6 orchestration patterns that power the next generation of AI systems :

1.🔸Sequential Orchestration
Agents work one after another - each passing its result to the next.
Perfect for structured, step-by-step tasks like data pipelines or analysis chains.

2.🔸Concurrent Orchestration
Multiple agents work in parallel, handling different parts of a problem simultaneously.
This boosts speed, scalability, and overall task efficiency.

3.🔸Group Chat Orchestration
Agents communicate in a shared conversation, refining ideas together.
Useful for brainstorming, collaborative decision-making, or generating consensus.

4.🔸Maker–Checker Loops
One agent creates or proposes; another verifies, corrects, or approves.
Ideal for maintaining accuracy, compliance, and quality assurance in workflows.

5.🔸Handoff Orchestration
Agents transfer control dynamically based on context or expertise.
Best suited for adaptive workflows like support systems or multi-step automation.

6.🔸Magnet Orchestration
A central “manager” agent dynamically assigns tasks to the right specialists.
Enables fluid, context-aware collaboration across multiple intelligent agents.

AI Agents are evolving fast and orchestration is what makes them powerful.

Learning how these patterns can help you build smarter, more collaborative AI systems can be a huge advantage.

#AIAgent

AI Agent Orchestration Patterns

📚LLMs in the Enterprise are finally getting the playbook they deserve… LLMs in Enterprise by Ahmed Menshawy and Mahmoud Fahmy provides practical guides for building real AI systems that operate at scale.

Most teams talk about LLMs in theory. This book focuses on execution. It bridges foundational concepts with the hands-on design patterns that matter when you are integrating models into production environments.

Here are the insights that stood out👇

1.🔸Enterprise LLM integration is a data architecture problem
The book breaks down how to design pipelines, tune retrieval, and structure data so models operate with consistency and low latency in real workloads.

2.🔸Scaling LLMs requires pattern-level thinking
They go deep on architectural patterns that reduce complexity, improve efficiency, and streamline deployment. This includes RAG frameworks, fine-tuning strategies, segmentation techniques, and evaluation patterns that teams often overlook.

3.🔸Performance is not just about bigger models
The authors show how to optimize model behavior with advanced inferencing engines, contextual model customization, and monitoring systems that keep applications predictable.

4.🔸Real enterprise value comes from operational rigor
Security, fairness, transparency, and accountability are not afterthoughts. They are part of the design process, especially when LLMs touch business workflows and customer data.

5.🔸AI teams win by mastering both concepts and impact
The flow of the book reflects the real enterprise lifecycle: Concept → Customization → Impact
A clear, structured way to think about turning LLM capabilities into business outcomes.

If you are building production AI systems, leading an LLM program, or preparing for the next wave of enterprise adoption, you should definitely get a copy of this book.

Enterprise AI is evolving fast. Understanding these design patterns early puts at a career advantage which allows you to shape the next generation of intelligent applications.

#LLM

Agentic AI vs Traditional AI: The Shift from Supervision to Autonomy

For years, Traditional AI has been like a skilled worker. It has been efficient, but dependent on human supervision.

Agentic AI, however, takes a different approach. It is a decision-maker that thinks, plans, and adapts without constant direction.

Let’s dive into what truly separates them 👇

Agentic AI: Smarter. Adaptive. Independent.

- Dynamic Learning: Perceives its environment, plans actions, and evaluates outcomes continuously.

🔸Goal-Oriented Thinking: Sets goals, creates sub-goals, and adjusts strategies like a human problem solver.

🔸Reasoning & Reflection: Learns from mistakes, refines decisions, and improves performance over time.

🔸Collaborative Agents: Multiple agents share context, delegate tasks, and synchronize for collective intelligence.

Traditional AI: Powerful but Limited.

🔹Static Learning Models: Train once, deploy, and manually retrain, with no real-time adaptability.

🔹Human-Dependent Execution: Waits for prompts or tasks, performs inference, and stops.

🔹Narrow Focus: Optimized for specific use cases without flexibility.

🔹No Self-Improvement: Requires human supervision for updates and tuning.

In Short:

- Traditional AI executes.

- Agentic AI thinks, plans, and evolves.

It’s the difference between automation and autonomy, a shift that’s redefining how AI systems operate, learn, and collaborate in the real world.

What other diffenreces have you noticed?

#AI

taplio

The Battle

Harpreet Sahota 🥑

Greg Coquillo

Followers

Posts

Engagements

Likes

Performance ScoreHarpreet Sahota 🥑's LinkedIn Performance

Performance ScoreGreg Coquillo's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Harpreet Sahota 🥑

Greg Coquillo

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Harpreet Sahota 🥑's LinkedIn Performance

Performance ScoreGreg Coquillo's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreHarpreet Sahota 🥑's LinkedIn Performance