I’ve been working in AI for close to a decade, starting well before today’s LLM-driven wave.

My work is grounded in understanding AI across layers — from hardware, AI models, and data to system design, reliability, and scale. I approach AI as an end-to-end engineering problem: not just modeling, but how models interact with data, infrastructure, and real-world constraints.

Alongside this, I write The AI Merge, a technical newsletter on AI foundations and systems engineering, focused on clear thinking and practical system design rather than hype.

I’m currently building an AI Systems course, centered on end-to-end system design, for engineers who want a deeper, systems-first understanding of modern AI.

🗞️ Read The AI Merge: https://read.theaimerge.com

Alex Razvant

Read more code.

And fewer articles, tutorials, or surface-level books.

If you’re already a Software Engineer or AI / ML Engineer, one of the fastest ways to level up right now is to: 

Study how big AI teams write and ship code - in public. 

Many top companies and major contributors to the AI space have been open-sourcing their internal frameworks, kernels, or infrastructure libraries. 

You might feel overwhelmed by the rapid pace at which AI is going.
But this is a massive learning opportunity if you know where to look.

Learning doesn’t just happen from papers and tutorials, especially if you already build software.

It happens when you:
- Read real production code
- Study PRs or commit history
- See how teams structure repos, APIs, configs, tests

Check what gets reviewed, what gets rejected, and why.
This kind of learning will upgrade your engineering intuition. 

Here are some excellent codebases to learn from.

✅ From DeepSeek:
1. FlashMLA  - A library of high-performance attention kernels.
Link: https://buff.ly/XbFnPA7 

2. DeepEP - A Communication Library for high-throughput MoE workloads.
Link: https://buff.ly/3qxMc3S

3. 3FS - A Distributed Filesystem for AI Workloads, for AI training and inference.
Link: https://buff.ly/ovfW47A

✅ From NVIDIA:
1. Dynamo - A datacenter-scale Inference Serving Framework for GenAI workloads.
Link: https://buff.ly/WyAMzBz 

2. NeMo  -  A Scalable Generative AI Framework, for training and deploying LLMs and VLMs.
Link: https://buff.ly/L9KTuJp 

✅ From vLLM:
1. The vLLM Library  - One of the most popular LLM inference engines, designed for scale.
Link: https://buff.ly/GLCfoIM

✅ From SGLang:
1. The SGLang Engine - A high-performance serving engine for LLMs & multimodal models.
Link: https://buff.ly/m4HhsWQ 

✅ From llama.cpp
1. The llama.cpp Engine - A minimal C/C++ inference engine for LLMs with broad hardware support and quantization options.
Link: https://buff.ly/u9yaXwp

📌 Don't study everything, you won't be able to.

Instead, pay attention to:
⇾ Code structure & abstractions
How are core functionalities separated (core logic vs hardware-specific optimizations)?

⇾  PRs & commit history
What context do authors provide? How do reviewers push back?

⇾  Interfaces & public APIs
How are public APIs designed?

⇾  Tests & benchmarks
How do the test suites look? What CI pipelines do they have, profiling? 

Don’t just use these libraries. 
Read them.

That will build your intuition and help you get better at your craft, either as a curious learner or an engineer as part of a larger team at your job.

---
I distill my learnings with my audience on The AI Merge.
Check it here (subscribe if interested) https://buff.ly/2rwqlNL

Senior AI Engineer | Writing The AI Merge Newsletter | Helping Engineers Build AI Beyond Demos

I make learning Data Science effortless for you | Data Science @ Bosch

Hi 👋🏻
I'm Banias, AI Engineer at Netze BW.

Applying new technologies and methods in Data Science & AI is my passion.

Through my experience and continual learning, I've gained valuable insights and tips that I love sharing with Data Scientists, Machine Learning Engineers and builders like you.

If you're looking for practical advice, helpful resources, and articles on the latest trends or best practices in the field, you're in the right place.

Let's connect and stay updated on the exciting world of Data Science together. Feel free to reach out, read my articles, and let's grow our knowledge and network. :)


Note: My views are my own and don‘t represent my employer.

Banias Baabe

How do you hit 98.7% accuracy with RAG on complex documents?

You don't use vector search.

Traditional RAG often fails because semantic similarity isn't true relevance. For tough domains, you need a system that reasons.

This is where 𝗣𝗮𝗴𝗲𝗜𝗻𝗱𝗲𝘅 shines.

It's a new, vectorless RAG framework that mimics how a human expert analyzes a document.

Instead of chunking and embedding, 𝗣𝗮𝗴𝗲𝗜𝗻𝗱𝗲𝘅 builds a "table-of-contents" tree and uses tree search to find the most relevant information.

Why is this better?
✅ 𝗛𝘂𝗺𝗮𝗻-𝗹𝗶𝗸𝗲 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹: It simulates how experts navigate complex information.
✅ 𝗧𝗿𝗮𝗻𝘀𝗽𝗮𝗿𝗲𝗻𝘁 𝗣𝗿𝗼𝗰𝗲𝘀𝘀: The reasoning is traceable and interpretable. No more "vibe retrieval."
✅ 𝗡𝗼 𝗔𝗿𝗯𝗶𝘁𝗿𝗮𝗿𝘆 𝗖𝗵𝘂𝗻𝗸𝗶𝗻𝗴: It respects the document's natural structure.
✅ 𝗡𝗼 𝗩𝗲𝗰𝘁𝗼𝗿 𝗗𝗕: The system relies purely on LLM reasoning and document hierarchy.

This approach achieved state-of-the-art results on the FinanceBench benchmark.

🔗 Check it out: github(.)com/VectifyAI/PageIndex


---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

I teach AI Engineers the concepts and tools they need to stay ahead | AI Engineer @ Netze BW

Running a 70B LLM on a single 4GB GPU is possible with 𝗔𝗶𝗿𝗟𝗟𝗠.

𝗔𝗶𝗿𝗟𝗟𝗠 optimizes inference memory usage so you can run big models without quantization, distillation, or pruning.

It runs inference layer by layer, loading only the layer that is needed for the next compute step, then freeing that memory before moving on.​

To make that practical, it first preprocesses the original Hugging Face weights and re-shards them by transformer layer (instead of giant multi‑GB shards), so each load is focused and smaller.​

This matters because disk I/O becomes the bottleneck when you stream weights.

The main thing that still sits in GPU memory is the KV cache for attention, which is much smaller than full model weights for typical context lengths.​

𝗔𝗶𝗿𝗟𝗟𝗠 also adds prefetching so it can overlap loading the next chunk of weights with the current compute, which helps claw back some speed.

And if you want more throughput, it supports an optional “model compression” mode using block-wise quantization of weights to shrink what needs to be loaded from disk, with the project claiming up to ~3x speedup and minimal accuracy loss.

🔗 Link to repo: github(.)com/lyogavin/airllm


---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

Stop destroying your documents with 512-token chunks.

Traditional RAG is breaking your tables, splitting your diagrams, and mixing up your model specifications.

I just read a great article from Alex Vesa about a working data modeling approach for complex technical documents.

The problem? Most RAG systems treat a 500-page service manual like a blog post. They chunk blindly. A table that spans three pages gets shredded into meaningless fragments.

Here's what actually works:

✅ Pages, not chunks
Use full pages as your retrieval unit. That's how humans read technical docs anyway.
✅ Three-page sliding windows
Process each page with its neighbors (N-1, N, N+1). Now your system knows when content continues across boundaries.
✅ Treat tables like they matter
Extract structure. Generate metadata. Flatten for search. Keep all three representations.
✅ Multi-vector indexing
Dense embeddings for semantic search. Sparse vectors for exact matches. ColBERT for precision. Combine them.
✅ Domain-specific metadata
Model applicability, component types, cross-page relationships. Not just title and date.

Link ➙ https://lnkd.in/dJ9aJKAB

Thinking about using 𝗽𝗴𝘃𝗲𝗰𝘁𝗼𝗿 for your next project?

You might want to think again.

Here's the reality of using it in production:
1️⃣ 𝗜𝗩𝗙𝗙𝗹𝗮𝘁 𝗶𝗻𝗱𝗲𝘅𝗲𝘀 𝗻𝗲𝗲𝗱 𝗰𝗼𝗻𝘀𝘁𝗮𝗻𝘁 𝗿𝗲𝗯𝘂𝗶𝗹𝗱𝘀. New vectors throw off your cluster distribution over time.
2️⃣ 𝗛𝗡𝗦𝗪 𝗶𝗻𝗱𝗲𝘅𝗲𝘀 𝗮𝗿𝗲 𝗺𝗲𝗺𝗼𝗿𝘆 𝗵𝗼𝗴𝘀. We're talking 10GB of memory for hours just to build an index for millions of vectors.
3️⃣ 𝗜𝗻𝘀𝗲𝗿𝘁𝗶𝗻𝗴 𝗻𝗲𝘄 𝘃𝗲𝗰𝘁𝗼𝗿𝘀 𝗶𝘀 𝗮 𝗽𝗮𝗶𝗻. With IVFFlat, you have to reindex preiodically. With HNSW, you're looking at graph traversals and memory allocation for every new vector.
4️⃣ 𝗠𝗲𝘁𝗮𝗱𝗮𝘁𝗮 𝗳𝗶𝗹𝘁𝗲𝗿𝗶𝗻𝗴 𝗶𝘀𝗻'𝘁 𝗰𝗹𝗲𝗮𝗿-𝗰𝘂𝘁. Pre-filtering is great for selective filters, but post-filtering can give you incomplete results, and it only gets worse with more filters.
5️⃣ 𝗡𝗼 𝗯𝘂𝗶𝗹𝘁-𝗶𝗻 𝗵𝘆𝗯𝗿𝗶𝗱 𝘀𝗲𝗮𝗿𝗰𝗵. You'll have to figure out how to balance vector similarity with text relevance and normalize scores on your own.

A new extension, pgvectorscale, does fix some of these problems with more memory-efficient indexing and better filter performance. But it isn't supported out-of-the-box on Azure and AWS, so you'd have to manage your own Postgres instance.

𝗽𝗴𝘃𝗲𝗰𝘁𝗼𝗿 is a great tool, but it's not a silver bullet. Make sure you're ready for the operational overhead before you commit.

🔗 Link to article: https://lnkd.in/d6jSRy2y


---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

Tired of blocking I/O messing up your async Python apps?

Try 𝗮𝗶𝗼𝗳𝗶𝗹𝗲𝘀 - the game-changer for handling files in asyncio without the headaches.

It's an Apache-licensed library that wraps standard file ops into async magic.

Why you should use it:
• Mirrors Python's built-in file API, so it's super familiar.
• Supports buffered text, binary files, and even tempfiles asynchronously.
• Delegates to a thread pool to keep your event loop flying.


🔗 Link to repo: github(dot)com/Tinche/aiofiles

---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

Vector databases are bloated.

What if you could get the same power with 97% less space?

Try 𝗟𝗘𝗔𝗡𝗡, the world's smallest vector index.

It transforms your laptop into a private and powerful RAG system.

You can finally RAG on everything.
-> Your entire file system
-> Millions of local emails
-> 💻 Your entire codebase
-> 💬 All your chat histories

Why is 𝗟𝗘𝗔𝗡𝗡 interesting?
✅ 𝗜𝘁'𝘀 𝗶𝗻𝗰𝗿𝗲𝗱𝗶𝗯𝗹𝘆 𝗹𝗶𝗴𝗵𝘁𝘄𝗲𝗶𝗴𝗵𝘁: Index 60 million documents in just 6GB instead of 201GB.
✅ 𝗜𝘁'𝘀 𝟭𝟬𝟬% 𝗽𝗿𝗶𝘃𝗮𝘁𝗲: Your data never leaves your laptop. No cloud costs, period.
✅ 𝗧𝗵𝗲𝗿𝗲 𝗶𝘀 𝗻𝗼 𝗮𝗰𝗰𝘂𝗿𝗮𝗰𝘆 𝗹𝗼𝘀𝘀: Get the same search quality as heavyweight solutions.

🔗 Link to repo: github(dot)com/yichuan-w/LEANN


---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

Do you want to identify which lines use the most amount of memory in your Python script?

Try 𝗺𝗲𝗺𝗼𝗿𝘆_𝗽𝗿𝗼𝗳𝗶𝗹𝗲𝗿.

𝗺𝗲𝗺𝗼𝗿𝘆_𝗽𝗿𝗼𝗳𝗶𝗹𝗲𝗿 is a Python module to make a line-by-line analysis of memory consumption for Python functions.

Below you can see how to use memory_profiler within your Python script:

• Decorate the function you want to profile with @𝗽𝗿𝗼𝗳𝗶𝗹𝗲.
• Run the script by passing the option -𝗺 𝗺𝗲𝗺𝗼𝗿𝘆_𝗽𝗿𝗼𝗳𝗶𝗹𝗲𝗿 to load the 𝗺𝗲𝗺𝗼𝗿𝘆_𝗽𝗿𝗼𝗳𝗶𝗹𝗲𝗿 module.

Link to project: pypi(dot)org/project/memory-profiler


---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

What, if an API call in your program fails?

Because of, let’s say, an instable internet connection?

This is not so uncommon.

You usually should have some retry mechanism in your program.

With 𝐭𝐞𝐧𝐚𝐜𝐢𝐭𝐲 in Python, this isn’t a problem anymore.

𝐭𝐞𝐧𝐚𝐜𝐢𝐭𝐲 offers a retrying behavior with a decorator with powerful features like:

- Define Stop Conditions
- Define Wait Conditions
- Customize retrying on Exception
- Retry on Coroutines

One of my favorite libraries!

Check it out: github(dot)com/jd/tenacity


---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

OpenAI's Responses API isn't what they're telling you it is.

They are talking about how it's "faster, more flexible, easier" than the old chat completions API.

But here's what's really going on.

OpenAI hides their reasoning traces.

Unlike Claude, DeepSeek, or Qwen, you can't see GPT-5's chain of thought.

This creates a massive problem for developers using the stateless chat completions API.

Without access to reasoning traces, you can't pass them back into the conversation.

Result? GPT-5 performs worse than it should in your applications.

The Responses API is the workaround.

✅ It's stateful (OpenAI stores conversation history on their servers)
✅ They maintain the hidden reasoning traces in the backend
✅ They plug it into conversations automatically
✅ They strip it out before sending responses back to you

You get full model capability without ever seeing the reasoning.

Here's the uncomfortable truth:

The old stateless API is actually simpler.

Prefix caching works in both. Parallel tool calling works in both.

There's nothing inherently better about stateful architecture here.

The Responses API only exists because OpenAI won't show you what their models are thinking.

Meanwhile, Anthropic keeps their reasoning traces completely transparent.

If you're building with AI APIs in 2026, this matters.

The question is whether you're comfortable with your provider hiding critical information from you.

Link to article: https://lnkd.in/drZjD9Xj


---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

AI makes your team faster.

But it also puts your data at risk.

Files are moving to new tools constantly.

AI models are training on things they shouldn't see.

And old security tools just can't see what is happening.

They are blind to the context that actually matters.

That is why I'm watching Cyberhaven’s launch on February 3.

They are revealing a new platform that combines data security and protection into one tool.

It's built to handle how messy and fast data moves today.

Join the live event to see:
📍 A live demo of this new AI-native platform.
📍 How to spot the few dangerous actions hidden inside millions of normal ones.
📍 Real stories from leaders on why old rules don't work anymore.

Don't wait for a data leak to find the holes in your security.

Grab your spot here: https://lnkd.in/dC-aGjjM

Struggling with slow data processing for AI training?

Try 𝗹𝗶𝘁𝗱𝗮𝘁𝗮, the Python library that transforms and optimizes datasets at scale!

𝗹𝗶𝘁𝗱𝗮𝘁𝗮 offers:
🟢 Streaming large cloud datasets without local downloads
🟢 Accelerating training by 20x with optimized data
🟢 Parallelizing tasks across multiple machines effortlessly
🟢 Securely handling sensitive data with advanced encryption

Perfect for images, text, video, and more.

Check it out: github(dot)com/Lightning-AI/litdata


---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

taplio

The Battle

Alex Razvant

Banias Baabe

Followers

Posts

Engagements

Likes

Performance ScoreAlex Razvant's LinkedIn Performance

Performance ScoreBanias Baabe's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Alex Razvant

Banias Baabe

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Alex Razvant's LinkedIn Performance

Performance ScoreBanias Baabe's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreAlex Razvant's LinkedIn Performance