I build AI products from zero to one, and I teach others how to do the same.Over 12+ years, I've taken AI/ML products from proof-of-concept to launch at Amazon, Nike, T-Mobile, and Disney. The work I love most is early-stage: validating ideas, getting executive greenlight, and shipping MVPs that prove value before scaling. At Nike, that approach drove a $100M annual reduction in product returns.I also teach what I build. I co-created and lead the AI Product Management Certificate at the University of Washington, where experienced PMs ship real GenAI products complete with evaluation plans and GTM strategies. Previously, I trained 4 cohorts at Product School.What I bring: the ability to translate messy AI capabilities into clear product value, align technical and business stakeholders, and move fast without breaking trust. I write about AI PM craft at aipmguru.substack.com.I'm currently exploring opportunities to lead 0→1 AI product work at companies solving meaningful problems. I'm especially energized by AI in healthcare and education, though I'm open to any domain where the product can genuinely improve lives. 

Shaili Guru

Technical Product Manager | AI Enthusiast | Making Kubernetes Work Smarter (CKA, CKAD) | Figuring out AI to build better Products | AI PM | PLGC™️ | PSC™️

I build Products that power AI at Scale.

As a Group Product Manager, I lead strategy for GPU cloud platforms designed for LLM inference, training, and fine-tuning in real production environments.

My focus is on:
- Turning GPUs into reliable AI platforms
- Scaling inference efficiently and responsibly
- Lowering cost and complexity of AI infrastructure

I care deeply about making advanced AI usable beyond big labs, so startups and enterprises can build, deploy, and scale AI without friction.

Also:
- KubeCon Japan speaker on Responsible AI
- Podcast guest on LLM inference and AI systems

Shamsher Ansari

How Effective is GPU→CPU Memory Swap?

GPU memory can be swapped to CPU RAM to improve GPU utilization, especially for AI inference workloads.

Instead of keeping every model loaded in GPU memory, 
inactive models can stay in CPU RAM. 

When needed, they are quickly moved back to the GPU.


This approach helps in scenarios like:
 • Running multiple AI models on the same GPU
 • Multi-model inference platforms
 • Interactive notebooks and experiments

Some benchmarks show time-to-first-token (TTFT) improving from 75–180 seconds to just 1–4 seconds compared to scaling GPUs from zero.


The trade-off?
You may add a little latency, 
but you gain better GPU utilization and lower infrastructure cost.

Important points:
 • Works best for inference workloads
 • Not ideal for continuous high-throughput training
 • Requires enough CPU RAM to store swapped models


In short, GPU → CPU memory swap is a powerful technique to run more models on fewer GPUs.


---------------------------------------------------
#ai #genai #gpu #Nvidia #utilization #aipm #inference #training #productmanagers #data #llm #vllm #LLMInference #kernel #kubernetes #infrastructure #vllm  #kubernetes #infrastructure

Group Product Manager | LLM Inference | GPU Cloud | AI Infrastructure @ NeevCloud

How AI Workloads Run on GPUs ?

The layers that connect AI software to physical GPU hardware.

1️⃣ User Space – “What to do”
This is where developers write code.
→ Apps and frameworks like PyTorch or vLLM
→ CUDA/cuDNN translate Python code into GPU instructions
→ Understands the intent: model structure, speed, SLOs


2️⃣ Kernel Driver – “The bridge”
Lives inside the OS.
→ Manages GPU memory
→ Handles communication between software and hardware
→ Schedules work so multiple apps can share the GPU safely


3️⃣ GPU Device – “How it runs”
The actual hardware.
→ Thousands of parallel compute units (SMs) do the math
→ Ultra-fast GPU memory (HBM, cache)
→ On-chip schedulers decide what runs and when


Which means
→ User Space decides what to compute
→ Kernel Driver manages access and fairness
→ GPU Hardware does the heavy lifting


Understanding this stack helps with GPU efficiency, performance tuning, and cost optimization.

Credit: https://lnkd.in/d-PubtgR
---------------------------------------------------
#ai #genai #gpu #Nvidia #utilization #aipm #inference #training #productmanagers #data #llm #kernel #kubernetes #infrastructure #vllm  #kubernetes #infrastructure

Episode 2: How vLLM Is Making LLM More Efficient

LLM efficiency is not just a performance metric. 

It is a business necessity.

In this sharp and practical conversation, 
we explore how to run LLMs smarter in production:

• Why efficiency truly matters at scale
• How vLLM improves throughput
• What this means for teams deploying LLMs in real-world environments


Featuring
Prasad Mukhedkar
AI Specialist Solution Architect, Red Hat

Hosted by
Shamsher Ansari
Group Product Manager, Technical, NeevCloud®


If you are building, deploying, or scaling LLMs, this episode is for you.

Subscribe to NeevCloud® YouTube channel to watch the episode
https://lnkd.in/d99mFex8

We are also hosting vLLM Inference Meetup in Pune next month, keep an eye.


---------------------------------------------------
#ai #genai #gpu #Nvidia #utilization #aipm #inference #training #productmanagers #data #llm #kubernetes #infrastructure #vllm  #IndiaAIImpactSummit2026 #kubernetes #infrastructure

What If your GPU could have almost unlimited Memory?

LLMs and datasets are growing so fast that they often exceed the HBM/VRAM available on a single GPU. 

Increasing GPU memory is expensive and technically complex.

This is where CXL GPUs come into the picture.

Compute Express Link (CXL) allows GPUs to use PCIe-attached memory like system DRAM,

And in some designs even SSDs as an extension of their own memory.

Instead of moving data back and forth between CPU and GPU memory, 

the GPU can directly access this external memory with much lower latency (100-200 ns)

This effectively extends GPU memory capacity 
without needing more GPUs or expensive HBM upgrades.

One interesting example is Panmnesia’s CXL GPU architecture, 

which connects GPUs to system memory using a CXL over PCIe. 

This allows larger models and datasets to run on the same GPU.

We are still early in this transition. 

Major GPU vendors like Nvidia, AMD, and Intel will need deeper native CXL support to unlock the full potential.


But if the ecosystem evolves, CXL could help break the AI memory wall by combining:
• Ultra-fast HBM on the GPU
• Flexible pooled memory through CXL


This could fundamentally change how we scale AI workloads.

---------------------------------------------------
#ai #genai #gpu #Nvidia #utilization #aipm #inference #productmanagers #data #llm #vllm #LLMInference  #kubernetes #memory #hbm #infrastructure #vllm #CXL

vLLM Inference Meetup — Pune | 14 March 2026

Excited to announce the vLLM Inference Meetup hosted by Red Hat AI, NeevCloud®, and Hewlett Packard Enterprise at the HPE office in Kharadi, Pune.


What to expect
- Deep dives on vLLM, Semantic Routing, and Project Sardeenz
- Practitioner sessions from HPE & NeevCloud®
- A hands-on workshop with GPU instances provided by the organizers


If you’re building, deploying, or scaling LLM inference, this is a great room to learn, share, and network with folks doing it in production.


🗓 Date: Saturday, 14 March 2026
⏰ Time: 1:00 PM – 6:00 PM
📍 Venue: HPE, International Tech Park, Kharadi, Pune


⚠️ Limited seats | Approval-based registration

👉 Register here: https://luma.com/thzzwjja


cc: Prasad Mukhedkar

---------------------------------------------------
#ai #genai #gpu #Nvidia #utilization #aipm #inference #training #productmanagers #data #llm #vllm #LLMInference #kernel #kubernetes #infrastructure #vllm  #kubernetes #infrastructure #PuneMeetup

The first vLLM Inference Meetup of 2026 in Pune was fantastic.

It was great to finally share the stage with some of the brightest minds working on AI inference today.

In my session, I talked about the current challenges in LLM inference and how vLLM addresses them using PagedAttention and Continuous Batching to improve GPU utilization and efficiency.

It was also great hearing from Steve Shirkey, who shared insights on where AI inference infrastructure is heading.

Rahul Tuli gave an interesting talk on LLM Compressor and showed how quantization can be done simply and efficiently with minimal accuracy loss.

Ritesh Shah delivered an excellent deep dive into the vLLM Semantic Router and how it helps route requests intelligently.

A big thanks to Prasad Mukhedkar for the opportunity to speak and for organizing a great event in collaboration with Hewlett Packard Enterprise team.

It was amazing to see the AI community in Pune coming together to learn and share ideas.

Looking forward to more such meetups.


Feel free to try vLLM on NeevCloud®
https://lnkd.in/dHkD_NnQ


NeevCloud® , Red Hat , Hewlett Packard Enterprise 


#vLLM #LLMInference #GenAI #AIInfrastructure #GPU #PuneTech #AICommunity

We shipped a lean version of our product.

On purpose.


Because our goal was to ship early and learn, 

Instead of waiting to build everything upfront.

We believe it’s better to get a working product into users’ hands first, 

And then add features based on real feedback.

There’s a well-known principle that around 85% of product features are rarely or never used. 

Because of this, 

we chose to focus only on the 15% that truly matters.


So instead of building everything, we concentrated on:

→ Core features that actually solve the main problem
→ Simple and fast onboarding, so users can get value immediately
→ Free credits, so users can start experimenting without friction.


We’ll continue adding features, 

but only when we’re confident they create real value for our users.


What's your strategy?

---------------------------------------------------
#ai #genai #gpu #Nvidia #utilization #aipm #inference #training #productmanagers #data #llm #kubernetes #infrastructure #vllm  #IndiaAIImpactSummit2026 #kubernetes #infrastructure

Most GenAI platforms scale #LLMs like regular Microservices.

Run a few vLLM / TGI pods, put them behind a load balancer, 
enable autoscaling, done.

Sounds simple.

But this pattern quietly wastes GPUs.

What goes wrong?

KV cache locality is lost
→ Round-robin routing sends requests to different replicas 
→ Cache benefits disappear.

Prefill and Decode compete for GPUs
→ Prompt processing and token generation run on the same GPUs 
→ Imbalance and high p95/p99 latency.

Load balancer is blind
→ It doesn’t see cache, queue depth, or SLAs 
→ Teams end up over-provisioning GPUs.


A better approach?
Treat LLM inference as a distributed systems problem, not just a runtime problem.


KServe + llm-d is an interesting architecture for this.

KServe
 • OpenAI-compatible inference API on Kubernetes
 • Handles autoscaling, routing, and model lifecycle
 • Works with runtimes like vLLM / TGI

llm-d
 • Adds cluster-level intelligence
 • Cache-aware routing
 • Prefill / decode separation across GPU pools
 • SLA-aware scheduling


The result:
→ Better KV cache reuse
→ Higher GPU utilization
→ Lower tail latency
→ Lower cost per token


Curious, how are you handling?
KV cache locality, prefill/decode split, and GPU scheduling in your stack?


Excellent post by Yuan Tang and Ran Pollak
https://lnkd.in/dCWStied


---------------------------------------------------
#ai #genai #gpu #Nvidia #utilization #aipm #inference #training #productmanagers #data #llm #vllm #llmd #LLMInference #kernel #kubernetes #infrastructure #vllm  #kubernetes #infrastructure #PuneMeetup

Will the LLM fit on my hardware? 🤔

I run most of my local LLM experiments on my M4 Mac Mini.

My setup
→ M4 Mac mini with 10-core GPU
→ 32 GB RAM
→ ~27 GB VRAM

One question I always had was:
Which model will actually run well on my machine, so that I don't have to download each large LLMs?

There are hundreds of models available now, 
and figuring out the right one for your hardware is not easy.

Recently, I found an interesting library called "llmfit" that solves this problem.

With one command, it checks your system’s RAM, CPU, and GPU, and tells you which LLM models will actually fit and run efficiently on your hardware.

It scores models based on:
→ Quality
→ Speed
→ Hardware fit
→ Context length

It also supports:
→ Multi-GPU setups
→ MoE models
→ Dynamic quantization selection
→ Speed estimation
→ Local runtimes like Ollama, llama.cpp, and MLX

It even comes with an interactive terminal UI, which makes exploring models very easy.

If you experiment with local LLMs, this tool can save a lot of trial and error.

Great project by Alex Jones 🫡 
----------------------------------------------------------------------
#ai #aiml #cpu #gpu #llm #localai #vllm #llama #token #model #kvcache #inference #serving #quantization #context  #m4mac  #mlx #moe #OnDeviceAI #AppleSilicon  #GenAI  #M4MacMini #kubernetes

vLLM Inference Meetup — Pune | 14 March 2026

If you’re building, deploying, or scaling LLM inference, this is a great room to learn, share, and network with folks doing it in production.

Registration Link:
https://lnkd.in/d5b43ykt


#vLLM #LLM #Inference

𝐑𝐮𝐧𝐧𝐢𝐧𝐠 𝐋𝐋𝐌𝐬 𝐢𝐧 𝐩𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐨𝐧 𝐢𝐬 𝐧𝐨𝐭 𝐭𝐡𝐞 𝐡𝐚𝐫𝐝 𝐩𝐚𝐫𝐭 𝐚𝐧𝐲𝐦𝐨𝐫𝐞.
Running them efficiently at scale is.

From 𝐥𝐚𝐭𝐞𝐧𝐜𝐲 𝐛𝐨𝐭𝐭𝐥𝐞𝐧𝐞𝐜𝐤𝐬 𝐭𝐨 𝐆𝐏𝐔 𝐮𝐭𝐢𝐥𝐢𝐳𝐚𝐭𝐢𝐨𝐧 and 𝐢𝐧𝐟𝐫𝐚𝐬𝐭𝐫𝐮𝐜𝐭𝐮𝐫𝐞 𝐨𝐩𝐭𝐢𝐦𝐢𝐳𝐚𝐭𝐢𝐨𝐧, LLM inference has become one of the biggest challenges for AI builders today.

At the 𝐯𝐋𝐋𝐌 𝐈𝐧𝐟𝐞𝐫𝐞𝐧𝐜𝐞 𝐌𝐞𝐞𝐭𝐮𝐩, we’ll dive into how modern AI systems are solving these challenges and what it takes to run large models efficiently in real-world environments.

𝐒𝐩𝐞𝐚𝐤𝐞𝐫
Shamsher Ansari  
𝐆𝐫𝐨𝐮𝐩 𝐏𝐫𝐨𝐝𝐮𝐜𝐭 𝐌𝐚𝐧𝐚𝐠𝐞𝐫 – 𝐓𝐞𝐜𝐡𝐧𝐢𝐜𝐚𝐥

📅 𝐒𝐚𝐭𝐮𝐫𝐝𝐚𝐲, 𝟏𝟒 𝐌𝐚𝐫𝐜𝐡 𝟐𝟎𝟐𝟔
⏰ 𝟏:𝟎𝟎 𝐏𝐌 - 𝟔:𝟎𝟎 𝐏𝐌
📍 𝐇𝐏𝐄, 𝐈𝐧𝐭𝐞𝐫𝐧𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐓𝐞𝐜𝐡 𝐏𝐚𝐫𝐤, 𝐏𝐮𝐧𝐞

If you’re a developer, ML engineer, or AI practitioner working with LLM inference, this meetup will bring together the community to exchange ideas, insights, and practical approaches.

🔗 𝐑𝐞𝐠𝐢𝐬𝐭𝐞𝐫 𝐡𝐞𝐫𝐞:
https://lnkd.in/eWwEukPJ

#LLM #vLLM #AIInfrastructure #Inference #MLOps #AICommunity #NeevCloud

taplio

The Battle

Shaili Guru

Shamsher Ansari

Followers

Posts

Engagements

Likes

Performance ScoreShamsher Ansari's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Shaili Guru

Shamsher Ansari

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Shamsher Ansari's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreShamsher Ansari's LinkedIn Performance