Voxel51

• Explore, tinker with, and push the boundaries of the latest AI and machine learning technologies to showcase what's possible with open-source tools and frameworks

• Build inspiring prototypes, proof-of-concepts, and example applications that demonstrate creative and impactful uses of AI to the developer community

• Contribute meaningfully to the open source ecosystem by creating and maintaining libraries, tools, datasets, tutorials and guides that enable developers to more easily leverage AI capabilities

• Curate, clean, annotate and publish unique multimodal datasets spanning text, images, audio, video, 3D and other emerging data types to facilitate novel machine learning research and applications

• Create engaging technical blog posts, social media content, videos, and presentations that educate and inspire developers and data scientists to build with AI

•  Collaborate openly with our internal teams and the broader AI/ML community to cross-pollinate ideas, share knowledge, and collectively advance the state-of-the-art

• Evangelize our technology and vision by representing the company at hackathons, conferences, meetups, universities, and other developer-focused events

•  Influence the product roadmap by providing input on the developer experience, uncovering opportunities for new features and improvements based on community feedback and your work

• Continuously learn, experiment with, and provide feedback on beta versions of our latest models and capabilities to help shape the future of our platform

Hacker-in-Residence

LinkedIn

1) Introductory computer vision course for LinkedIn Learning

2) Prompt Engineering with LangChain (releasing March 2024)

3) RetrievalAugemnted Generation with LlamaIndex (releasing Q2 2024)

4) Fine-tuning LLMs with Hugging Face TRL (est release Q3 2024)

5) Text Generation with Hugging Face Transformers (est release Q3 2024)

LinkedIn Learning Instructor

The Artists of Data Science

The ONLY personal growth and development podcast for Data Scientists

Host

Deci AI

Developer Relations Manager

Pachyderm Inc.

Senior Developer Advocate

Comet

ML Developer Advocate

Data Science Dream Job

Principal Data Science Educator

Price Industries

Lead Data Scientist

Bold Commerce

Senior Data Scientist

Emergent BioSolutions

Biostatistician

I create content about deep learning | The DevRel and Deep Learning Dude

Hacking on LLMs, curating multimodal datasets, and writing a book about RAG.

Harpreet Sahota 🥑

sam3 is seriously a step change improvement over sam2

the model is seriously good, glad they brought back text prompts. the automatic segmentation isn't as good as i hoped it to be...but it still works quite well. 

i also got it to output image embeddings and that was cool to visualize in the app

learn more here: https://lnkd.in/gzqJzj6G

🤖 Hacker-in-Residence @ Voxel51| 👨🏽‍💻 AI/ML Engineer | 👷🏽‍♀️ Technical Developer Advocate | Learn. Do. Write. Teach. Repeat.

qwen3vl is literally the best model i've used for video usecases. blows my mind

hosting a virtual event tomorrow about document ai

you may or may not find it interesting, but i guarantee you'll learn something

join me here: https://lnkd.in/dFXPimVc

sony ai released a pretty cool dataset called the fairness human centric image benchmark, super high quality labels

instructions for downloading and parsing the dataset into fiftyone format can be found here: https://lnkd.in/g_VwxJx4

physical AI is top of mind as the next frontier for deep learning. but it'll only succeed if you have good datasets

language models scaled with web-scale text. 

vision models got ImageNet. 

now Voxel51 and NVIDIA are trying to do the same thing for physical AI 

autonomous vehicles, humanoid robots, anything that moves around atoms.

building these systems means millions of hours of testing in high-fidelity sims before you can actually deploy anything. 

petabytes of multimodal sensor data from various modalities

 LiDAR, cameras, radar, IMUs. 

but over 50% of simulations end up completely unusable because the input data is bad.

even small calibration or timing errors between sensors cascade through your neural reconstructions. 

you waste weeks and burn through millions in compute costs before you realize what went wrong.

the new FiftyOne Physical AI Workbench. it integrates with NVIDIA Omniverse and NVIDIA Cosmos and basically sits at the start of your sim pipeline to validate, clean up, and prep your multimodal data before it hits neural reconstruction and synthetic data generation.

what it does:

- automatically audits physical AI inputs across 75+ critical checkpoints

- transforms raw sensor streams into structured, searchable datasets

- triggers accurate neural reconstructions and generates synthetic scene variations

this means fewer wasted compute hours. 

iteration loops go from weeks to hours. 

you build safer, more reliable systems on data you can actually trust.

read more about what we've been building: https://lnkd.in/gC6_zn-X

i recently integrated 4 ocr models into fiftyone as remote zoo models these handle text extraction and document parsing. 

all available as remote zoo sources, you can get started with a few lines of code

different approaches for different needs:

1. mineru-2.5

1.2b params, two-stage strategy: global layout on downsampled image, then fine-grained recognition on native-resolution crops. 

handles headers, footers, lists, code blocks. strong on complex math formulas (mixed chinese-english) and tables (rotated, borderless, partial-border).

good for: documents with complex layouts and mathematical content

https://lnkd.in/dSwimdvp

deepseek-ocr

dual-encoder (sam + clip) for "contextual optical compression." 

outputs structured markdown with bounding boxes. has five resolution modes (tiny/small/base/large/gundam). gundam mode is the default - uses multi-view processing (1024×1024 global + 640×640 patches for details). 

supports custom prompts for specific extraction tasks.

good for: complex pdfs and multi-column layouts where you need structured output

https://lnkd.in/dhJn2h-v

olmocr-2

built on qwen2.5-vl, 7b params. outputs markdown with yaml front matter containing metadata (language, rotation, table/diagram detection). 

converts equations to latex, tables to html. labels figures with markdown syntax. reads documents like a human would.

good for: academic papers and technical documents with equations and structured data

https://lnkd.in/dNdTQYKg

kosmos-2.5

microsoft's 1.37b param multimodal model. two modes: ocr (text with bounding boxes) or markdown generation. automatically optimizes hardware usage (bfloat16 for ampere+, float16 for older gpus, float32 for cpu). handles diverse document types including handwritten text.

good for: general-purpose ocr when you need either coordinates or clean markdown

https://lnkd.in/dCTfGfV8


two modes typical across these models: detection (bounding boxes) and extraction (text output)

i also built/revamped the caption viewer plugin for better text visualization in the app: 

https://lnkd.in/dBebWdga

i've also got two events poppin off for document visual ai:

- nov 6 (tomorrow) with a stellar line up of speakers: https://lnkd.in/gkBEY4_r

- a deep dive into document visual ai with just me: 

https://lnkd.in/dFXPimVc

Open article: Document Visual AI with FiftyOne - November 14, 2025 by voxel51.com

Document Visual AI with FiftyOne - November 14, 2025

hype for this new integration, and proud of the team for shipping it at lightning speed. big ups to Daniel and AdonAI for goin beast mode on this one

🚀 Introducing Physical AI Workbench — integrated with NVIDIA Omniverse NuRec and NVIDIA Cosmos to solve the biggest bottleneck in physical AI.  

More than half of all physical AI simulations fail — not from bad models, but from bad input data. A timestamp off by milliseconds or a calibration drift of a few degrees can cascade through your neural reconstructions, leaving you with unusable results after weeks and millions in wasted compute. 

FiftyOne Physical AI Workbench sits at the start of your simulation pipeline and provides turnkey access to NVIDIA technology to ensure every simulation run begins with trustworthy data: 

✅ Audits 75+ critical checkpoints to ensure every reconstruction starts with validated inputs 
🎯 Enriches raw sensor streams into structured, searchable datasets 
🔗 Integrates directly with NVIDIA Omniverse and NVIDIA Cosmos for high-fidelity reconstruction and synthetic data generation. 

The result: an end-to-end workflow that prevents downstream failures — cutting iteration cycles from weeks to hours and eliminating GPU waste on failed reconstructions. We’re excited to see this workflow power autonomous vehicles NVIDIA DRIVE and robotics NVIDIA Robotics development teams move faster with higher accuracy. 

🔗 Read the full technical breakdown: https://lnkd.in/gTy5-TNF

Jacob Marks

Building the future of data-centric AI!

✅ Large language models (LLMs) for computer vision | Prompt engineering
✅ Vector Search: Pinecone, Qdrant, LanceDB, Milvus, Chroma, Redis
✅ Object Detection (YOLO-NAS, YOLOv8)
✅ Segmentation (Segment Anything - SAM)
✅ Multimodal ML (text-to-image, visual question answering, captioning)

Senior Machine Learning Engineer and Researcher

Machine Learning Engineer and Developer Evangelist

Physics World

Student Contributor

Synphonic

• Recruited and led founding team.
• Took the company from idea to MVP and revenue.
• Created data pipeline and customer acquisition funnel.
• Secured funding.

Co-Founder and CEO

Samsung Research America (SRA)

• Designed, analyzed, and implemented efficient algorithms.
• Developed novel RNN-based machine learning architecture.

Algorithms Researcher, Think Tank Team (TTT)

X, the moonshot factory

• Designed and implemented classical energy based models (EBMs) 
      and their quantum counterparts, quantum Hamiltonian based models (QHBMs).
• Optimized machine learning algorithms.
• Ran large-scale simulations.

Ph.D. Resident

Wolfram

Led development of fast, symbolic code for the Wolfram language

Research Fellow

Algorithms R&D Intern

JointPoint

• Developed software in C# to assist surgeons performing minimally-invasive hip surgery.
• Employed computer vision techniques including segmentation, adaptive thresholding, and edge detection
• Company acquired by Johnson & Johnson subsidiary in 2019.

Imaging Engineer

Yale Quantum Institute

• Conducted analytic and computational research on quantum networks for undergraduate thesis under the guidance of Yale professor of physics Liang Jiang. 
• Proposed two-cavity nodes that could allow for multi-excitation quantum state transfer protocol.

Student Researcher

Institute for Quantum Computing

• Developed QTop, the first open-source simulation and visualization package for topological quantum error correcting codes. 
• Created first clustering decoder for qudit color codes.
• Published 1st-author peer-reviewed article in New Journal of Physics as an undergraduate.

Undergraduate Research Assistant

Reservoir Labs

• Implemented and optimized a scalable Combinatorial Multigrid (CMG) Solver in C for improving numerical simulations of Lattice Quantum Chromodynamics (LQCD). 
• Parallelized calculations for LQCD

Software Engineer Intern

🔍 Senior Machine Learning Engineer & Researcher at Voxel51 | 🎓 Ph.D. in Theoretical Physics from Stanford University

🔹 Focus Areas: Artificial Intelligence, Machine Learning, Computer Vision, Data Curation, Startups

🌐 Transforming Data Transparency & Clarity at Voxel51

At Voxel51, I work on AI-driven solutions, data visualization, and curation tools to make the world's data more transparent and actionable. Recent/ongoing efforts related to large language models, vector search, synthetic data, and MLOps.

🛠️ Previous Experience

• Google X: Applying probabilistic AI to quantum computing (70+ citations)
• Samsung Research: Machine learning and physics-based algorithms for secret projects (Patent-pending).
• Wolfram Research: Led development of Wolfram Quantum Framework
• CERN: Deep neural networks for identifying particles in collisions.
• Institute for Quantum Computing, Canada: Published peer-reviewed quantum computing research (1st author) as undergrad
• Stanford Institute for Theoretical Physics: Multiple 1st author publications and doctoral dissertation in symmetric protected topology (condensed matter theory)
• Synphonic:  Founded startup to help musicians monetize their creative process, going from idea to MVP and revenue.

📝 Writing

I've contributed to Physics World magazine and regularly write technical blog posts on topics like AI, ML, and data science on Medium (6,900+ followers).

✍️ Versatility

Whether it's Python code, academic papers, prose, or pitch decks, I bring the same level of expertise and enthusiasm. Don't put me in a box.

🤝 Let's Connect!

I'm always open to new opportunities and collaborations in AI, machine learning, computer vision, and data curation. Feel free to reach out!

taplio

The Battle

Harpreet Sahota 🥑

Jacob Marks

Followers

Posts

Engagements

Likes

Performance ScoreHarpreet Sahota 🥑's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Harpreet Sahota 🥑

Jacob Marks

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Harpreet Sahota 🥑's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreHarpreet Sahota 🥑's LinkedIn Performance