I’ve been working in AI for close to a decade, starting well before today’s LLM-driven wave.

My work is grounded in understanding AI across layers — from hardware, AI models, and data to system design, reliability, and scale. I approach AI as an end-to-end engineering problem: not just modeling, but how models interact with data, infrastructure, and real-world constraints.

Alongside this, I write The AI Merge, a technical newsletter on AI foundations and systems engineering, focused on clear thinking and practical system design rather than hype.

I’m currently building an AI Systems course, centered on end-to-end system design, for engineers who want a deeper, systems-first understanding of modern AI.

🗞️ Read The AI Merge: https://read.theaimerge.com

Alex Razvant

Read more code.

And fewer articles, tutorials, or surface-level books.

If you’re already a Software Engineer or AI / ML Engineer, one of the fastest ways to level up right now is to: 

Study how big AI teams write and ship code - in public. 

Many top companies and major contributors to the AI space have been open-sourcing their internal frameworks, kernels, or infrastructure libraries. 

You might feel overwhelmed by the rapid pace at which AI is going.
But this is a massive learning opportunity if you know where to look.

Learning doesn’t just happen from papers and tutorials, especially if you already build software.

It happens when you:
- Read real production code
- Study PRs or commit history
- See how teams structure repos, APIs, configs, tests

Check what gets reviewed, what gets rejected, and why.
This kind of learning will upgrade your engineering intuition. 

Here are some excellent codebases to learn from.

✅ From DeepSeek:
1. FlashMLA  - A library of high-performance attention kernels.
Link: https://buff.ly/XbFnPA7 

2. DeepEP - A Communication Library for high-throughput MoE workloads.
Link: https://buff.ly/3qxMc3S

3. 3FS - A Distributed Filesystem for AI Workloads, for AI training and inference.
Link: https://buff.ly/ovfW47A

✅ From NVIDIA:
1. Dynamo - A datacenter-scale Inference Serving Framework for GenAI workloads.
Link: https://buff.ly/WyAMzBz 

2. NeMo  -  A Scalable Generative AI Framework, for training and deploying LLMs and VLMs.
Link: https://buff.ly/L9KTuJp 

✅ From vLLM:
1. The vLLM Library  - One of the most popular LLM inference engines, designed for scale.
Link: https://buff.ly/GLCfoIM

✅ From SGLang:
1. The SGLang Engine - A high-performance serving engine for LLMs & multimodal models.
Link: https://buff.ly/m4HhsWQ 

✅ From llama.cpp
1. The llama.cpp Engine - A minimal C/C++ inference engine for LLMs with broad hardware support and quantization options.
Link: https://buff.ly/u9yaXwp

📌 Don't study everything, you won't be able to.

Instead, pay attention to:
⇾ Code structure & abstractions
How are core functionalities separated (core logic vs hardware-specific optimizations)?

⇾  PRs & commit history
What context do authors provide? How do reviewers push back?

⇾  Interfaces & public APIs
How are public APIs designed?

⇾  Tests & benchmarks
How do the test suites look? What CI pipelines do they have, profiling? 

Don’t just use these libraries. 
Read them.

That will build your intuition and help you get better at your craft, either as a curious learner or an engineer as part of a larger team at your job.

---
I distill my learnings with my audience on The AI Merge.
Check it here (subscribe if interested) https://buff.ly/2rwqlNL

Senior AI Engineer | Writing The AI Merge Newsletter | Helping Engineers Build AI Beyond Demos

Head of AI/ML (10 years) with a strong background in Machine Learning and MLOps across various domains including information retrieval (Search, Recommendations, RAG-systems & ChatBots), NLP, and Customer Analytics. Brand new: AI Agents!

I specialize in guiding cross-functional teams of ML Engineers, Software Engineers, etc. to build scalable, event-driven ML solutions with a focus on robust MLOps practices using Python/PySpark and cloud platforms like AWS, GCP, etc. - (GCP Certified ML Engineer)

Currently, at HeyJobs, I have been fortunate to initiate and lead multiple complex ML projects (Search, RecSys, NLP and AI Agent) that that have significantly outperformed previous benchmarks and generated substantial business value. Additionally, I also designed and lead the implementation of a new MLOps and monitoring framework that has been embraced across all Data Science teams within the company.

Additionally, I love learning the latest advancements in the ML space, always exploring new ideas to push boundaries and improve outcomes. I’m also dedicated to mentoring and sharing knowledge, focused on building systems that drive innovation and business outcome.

Shantanu Ladhwe

Many wouldn't like this:
If I’m hiring for an AI Engineer…
I’m not going to ask 5 RAG flows or 10 agent flows.

Here’s what I really want to know 👇

1. Can you design a system end-to-end? from ingestion to serving? what are the bottlenecks?
2. How would you estimate costs? how to reduce it?
3. How would you reduce latency? what is a good tradeoff of latency vs quality? 
4. Do you really need self-hosted LLMs? When is it needed? 
5. How would you fine-tune language models on user behavior? which framework? what about model serving?
6. How would you construct the dataset, what about loss function? what about MLOps? 
7. Which database would you use and why? Vector DB? SQL? NoSQL?
8. What metrics would you track? How? 
9. What about system monitoring? How would you debug failure cases?
10. What about feedback loop? how would we track and evaluate?
11. How would you make the system more deterministic?
12. How would you replace embedding models and backfill embeddings without any downtime?
13. What are the fallback mechanisms?

My favourite:
1️⃣ How would you solve this problem without LLMs or vector DBs?
2️⃣ Can you solve the problem with classical IR, rules, heuristics?
3️⃣ How would you make the system more deterministic?
4️⃣ Can you explain tokenization and embeddings from scratch?
5️⃣ What happens during fine-tuning? optimizer, learning rate, layer freezing? - not just “let's use LoRA”

These are just a few top-of-mind.

The reason:
I’ve seen engineers spend months building something flashy and complex that never made it to production.
I want engineers who understand the fundamentals, not just the hype.

I don’t want a flashy agent.
I want a resilient system.

Do you agree?

PS: This can vary by role, but I’d be very cautious hiring someone with shallow foundations.

--
♻️ Repost to share with your network :)
➕ Join 37.000+ AI/ML builders here: https://lnkd.in/ds_SzEUH

Head of AI/ML - AI Agents, RAG, NLP, Recommenders, Search & MLOps

If you are into AI/ML Engineering

This one video covers the production gap that kills 90% of ML projects.

"System Design Course" by freeCodeCamp (975K+ views)
Free. 1 hour. Changes everything.

The problem with ML education: 
They teach you to build models. 
They don't teach you to serve 1000 users.

As someone building RAG & AI Agents: 
Model/LLM matters 10%. 
Infrastructure matters 90%.

🔷 What this covers:
→ Caching & CDNs: Cut RAG costs 70% 
→ Load Balancers: Handle 10x traffic without crashing
→ Database Sharding: Scale vector storage properly
→ Computer Architecture: 10x faster training and inference
→ CI/CD Pipelines: Deploy confidently

📌 Video link - https://lnkd.in/ddSYvGpc

Must watch!
---
♻ Repost if to help others!
🔔 Follow me, Shantanu for production AI/ML/MLOps & careers
➕ Join 38.000+ AI/ML builders here: https://lnkd.in/ds_SzEUH

70% of AI Engineers don’t know how to structure a repo

If your FastAPI + GenAI project is starting to look messy… this post is for you 👇

Here’s how we structure production-grade FastAPI based GenAI systems:
🔹 𝘀𝗿𝗰/ - 𝗖𝗼𝗿𝗲 𝗔𝗽𝗽𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻 𝗟𝗼𝗴𝗶𝗰
We use a layered + domain-first architecture - the same structure that powers robust ML/AI systems in production.

Let’s break it down:
1️⃣ 𝗺𝗼𝗱𝗲𝗹𝘀/
→ SQLAlchemy ORM models
↳ Defines your database schema: tables, types, indexes
↳ E.g. academic paper metadata: title, authors, abstract

2️⃣ 𝘀𝗰𝗵𝗲𝗺𝗮𝘀/
→ Pydantic models for data validation
↳ Type-safe request/response structures
↳ Handles optional fields, default values, constraints

3️⃣ 𝗿𝗼𝘂𝘁𝗲𝗿𝘀/
→ FastAPI route definitions
↳ Logical grouping of endpoints (e.g., /search, /upload)
↳ Auto-generates clean Swagger/OpenAPI docs
✅ Exposes your system cleanly to devs, tools, and UIs.

4️⃣ 𝗿𝗲𝗽𝗼𝘀𝗶𝘁𝗼𝗿𝗶𝗲𝘀/
→ DB interaction layer
↳ Clean abstraction over SQL queries
↳ Makes business logic readable + testable
✅ No raw SQL scattered in your app. Easy to test and evolve.

5️⃣ 𝘀𝗲𝗿𝘃𝗶𝗰𝗲𝘀/ - The brain of your application
→ Logic organized by domain, not just CRUD
e.g.:
↳ opensearch/ → Hybrid retrieval logic 
✅ Modular by design: swap, extend, or evolve features without chaos.

6️⃣ 𝗱𝗯/
→ Database engine config
↳ Connection pooling, retry logic, async sessions
✅ Rock-solid DB handling. No “random connection errors” in prod.

7️⃣ 𝗺𝗮𝗶𝗻.𝗽𝘆
→ FastAPI entry point
↳ Imports routes, applies middlewares, sets up CORS + logging
✅ Single source of truth for app startup.

8️⃣ 𝗺𝗶𝗱𝗱𝗹𝗲𝘄𝗮𝗿𝗲𝘀.𝗽𝘆 / 𝗲𝘅𝗰𝗲𝗽𝘁𝗶𝗼𝗻𝘀.𝗽𝘆
→ Reliability and observability layer
↳ Logs tracebacks
↳ Converts random errors into structured API messages
✅ Debugging and monitoring become predictable.

🔹 𝗕𝗲𝘆𝗼𝗻𝗱 𝘀𝗿𝗰/: 𝗜𝗻𝗳𝗿𝗮 + 𝗧𝗼𝗼𝗹𝗶𝗻𝗴
1️⃣ 𝗗𝗼𝗰𝗸𝗲𝗿𝗳𝗶𝗹𝗲 + 𝗰𝗼𝗺𝗽𝗼𝘀𝗲.𝘆𝗺𝗹
→ One-line infra bootstrapping:
↳ Starts: FastAPI, PostgreSQL, OpenSearch, Airflow, Ollama
✅ Works on every machine. No setup nightmares.

2️⃣ 𝗠𝗮𝗸𝗲𝗳𝗶𝗹𝗲
→ Simplifies commands like:
↳ make test, make lint, make up, etc.
✅ Standardized developer workflows.

3️⃣ 𝗽𝗿𝗲-𝗰𝗼𝗺𝗺𝗶𝘁-𝗰𝗼𝗻𝗳𝗶𝗴 . 𝘆𝗮𝗺𝗹
→ Linting + formatting (Ruff, Black, isort)
→ Prevents bad code from being committed
✅ Clean code. Enforced automatically.

4️⃣ 𝘁𝗲𝘀𝘁𝘀/
→ Pytest structure
↳ Unit tests with fixtures and mocks
✅ You can confidently refactor without breaking things.

5️⃣ 𝗻𝗼𝘁𝗲𝗯𝗼𝗼𝗸𝘀/
→ Analysis or test notebooks

PS: As you put the repo into production there would more components added, e.g. CI/CD, infrastructure, etc.


--
♻️ Repost to share with others :)
🔔 Follow me, Shantanu for production AI/ML/MLOps & careers
➕ Join 36.500+ serious AI/ML builders here: https://lnkd.in/ds_SzEUH

I am offering a 50% new year discount so you can begin your journey with Production AI Systems today.
↳ Code: PDL4GLSMEM
↳ Link: https://lnkd.in/dzdsTWz9

taplio

The Battle

Alex Razvant

Shantanu Ladhwe

Followers

Posts

Engagements

Likes

Performance ScoreAlex Razvant's LinkedIn Performance

Performance ScoreShantanu Ladhwe's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Alex Razvant

Shantanu Ladhwe

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Alex Razvant's LinkedIn Performance

Performance ScoreShantanu Ladhwe's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreAlex Razvant's LinkedIn Performance