Thoughtworks

Global Service Portfolio Lead for Data & AI Strategy services, including strategic enablement for Data & AI Governance, Data Modernization, AI readiness and other related client objectives.

Director, Data & AI Strategy

Leading our data strategy and data governance services, shaping the offering and approaches to create value-driven solutions for our clients in Europe

Principal Data & AI Strategist

DataQG

DataQG is the leading Data Enablement and Community Platform designed to help organizations upskill their workforce in data literacy.

Education Advisory Board Member

DAMA Germany

Lead marketing planning, execution and monitoring to grow and engage the community and to establish thought leadership and knowledge exchange in all things Data Management & Data Governance in Germany

Head of Marketing

Technics Publications

Author - "Humanizing AI Strategy - Leading AI with Sense and Soul"

Author - "Humanizing Data Strategy - Leading Data with the Head and the Heart"

Ambassador

adidas

Ensuring product data is consistent, accurate and available and making sure it’s used in a secure and meaningful way. We govern product data end to end, from data creation through data processing all the way to data usage and enrichment.

Senior Director, Product Data Governance

Lead in Social, PR and Voice of Consumer Analytics to integrate Consumer Insights from Social Media, PR, Search Behavior, Product Reviews and Experience NPS feedback into any decision-making in brand strategy, campaign planning and in-activation optimization.

Director, Digital Analytics - Social, PR & Voice of the Consumer

Lead in Voice of the Consumer Analytics with focus on integrated insights based on consumer feedback across digital channels (e.g. Digital Brand Perception on Social Media, Product Ratings / Reviews on Website, Search Behaviour on Google)

Director, Digital Analytics - Voice of the Consumer

Creation of Digital Marketing Performance KPI Framework to measure success within branding activities and consumer perception on strategic and operational level

Lead standardisation and alignment of eCom KPI definitions and reporting methodology across Digital Analytics, BI/Finance, Markets and Business Units

Governance of holistic eCom business performance KPI framework across digital and financial teams

Senior Manager, Digital Analytics

TD Reply

Project Manager of projects in Digital Strategy & Performance Optimization

adidas: On-site Project Manager and Product Owner of KPI Frameworks and Dashboard Visualizations at Digital Brand Commerce

Miele: Account Management of International Brand Marketing and coordination with over 10 markets

Support in Digital Performance Management projects of Coca Cola, Volkswagen and Döhler

Knowledge Management & Advisory between German and Chinese Office

Senior Consultant

Lead of projects in Digital Strategy & Market Research

Key account manager of VW brand marketing

Development of Social Media strategies

Setting up processes and guidelines for crisis management

Consultant

Junior Consultant / Master Thesis

Student Worker

Berlin Institute of Technology (Technische Universität Berlin)

Analysis of business models and market analysis of various industries

Research Assistant - Chair of Strategic Leadership & Global Management

I’m a Data & AI leader by day, a musician at night, and an optimist at heart. 

After 12+ years in data analytics, data governance, consumer insights, digital transformation and AI innovation, I grew fond of the human aspect - how to communicate, collaborate and get creative around data and AI.

My passion lies in making data and AI more approachable and fun, and I often use unconventional methods to do so - with music, humor or other creative outlets. 

Nice to meet you! 
Let’s have a (virtual) coffee?

Tiankai Feng

Data quality has always been a little bit of a catch-all for all kinds of data complaints. Whenever data looks a little different than expected, it immediately must be a data quality problem. 

The rapid increase of AI-driven use cases for data now puts even more attention to the concept of data quality, because AI use cases often require expanded historical data, higher granularity, higher speed and a different focus - like unstructured data for context. 

One thing has not changed: the only way to manage data quality in the right way is to move away from generic complaints about it to detailed conversations about what "high quality" data actually means, and what currently is missing to get there - to then improve data quality together towards a defined state. 

When in doubt, dad jokes can also help - to not only break the ice, but to also exercise authority by positioning yourself as someone's dad. 😁 

Let me know if you need any help.

#dataquality #datagovernance #tiankaistuff

Data & AI Strategy Director @ Thoughtworks | Author of “Humanizing AI Strategy” | TEDx Speaker | Data Musician

This is an appreciation post for everyone working in data governance.

While most data governance people I talk to are really driven and passionate about making data better for their organization, they are also the most modest and self critical - believing they always can do more things better. 

Facing resistance and misunderstandings around what data governance even is trying to do, being a great empath and communicator became a necessity - making them great collaborators and people to hang out with (if you give them a chance).

Since data governance conferences very quickly turn into group therapy, I know we are not getting the attention and acknowledgement we deserve - and although the hype around AI has finally put us in the spotlight, we are now spending even more time and efforts clarifying requirements and objectives, and setting the right expectations.

So, to everyone working in data governance: you are doing your best and it’s awesome - just a little longer and you can recharge during the break. There’s a lot of exciting work to do in 2026.

Data governance rules. 

#datagovernance #appreciation #tiankaistuff

Fun with data literacy in Kindergarten?! 

Yesterday I was invited to read “The Data Detective at the carnival” to the kids at my son’s Kindergarten group - a book that I translated for the German community and donated to the Kindergarten before. 

I was first a little worried that talking to kids between 4 and 6 years old about qualitative and quantitative data would be too difficult - oh how wrong I was! 

We had the most fun letting the kids count and describe things not only in the book but also in the room and in their lives - and everyone was excited to get a “data detective certificate” afterwards! 

But the most rewarding thing of all is how proud my son was that his dad came to Kindergarten to read a book to everyone - showing up as his cool dad is probably my favorite speaking engagement ever. 

Thanks to Chandra Donelson and Jordan Morrow for writing such a wonderful book! 

#dataliteracy #education #tiankaistuff

No alternative text description for this image

In a desperate attempt to make data jobs cool for the next generation, Fortnite launches the new “data squad” skins! 

They even come with their own signature powers: 

👉 the data engineer can build pipelines to move faster but everytime a pipeline breaks the player is teleported next to the pipeline to fix it for an hour

👉 the data scientist can predict other players’ movements but needs to clean data in a spreadsheet for 30 minutes first 

👉 the data analyst puts random dashboards all across the map to confuse other players with KPIs and metrics 

👉 the data product owner can build powerful products but with every minute without a RoI the player loses power 

👉 the data governor can change the rules of the game entirely but other players only have to comply optionally 

Can’t wait for the AI roles skins to drop with equally as much realism! 

P.S. this is satire and not real. 

#data #humor #tiankaistuff

Receiving this kind of feedback after my talk and musical performance at the Forward Data Conference is truly so rewarding.

"People will forget what you said. People will forget what you did. But people will never forget how you made them feel”, the great Maya Angelou once said - that’s why sharing positivity and joy with others is for me equally as important as exchanging knowledge. 

After all, there are two things I love to spread: Happiness and germs. 

Just kidding, it’s Nutella. And germs. 😁

How we make people feel is our real legacy, y’all. Have a great weekend everyone!

#conference #mindset #tiankaistuff

Here are five reasons why you should buy my book "Humanizing Data Strategy":

👉 My mom calls it "better than expected"

👉 Critics call it "marxism for data" and "DEI Cr*p"

👉 It looks great on a book shelf

👉 That's why your kids will love to draw stuff onto the cover

👉 It's currently on Black Friday sales (but apparently only in Germany)

It's also about the human side of data and introduces the framework of the 5Cs (competence, collaboration, communication, creativity & conscience) to make data strategy truly human-centric, but those are just minor details. 😁 

Maybe those that read this book already can leave more reasons to buy it in the comments. Thanks in advance!

#datastrategy #author #tiankaistuff

“You’re too sensitive.” 

One of the worst things to say to someone who’s upset - and it never has made anyone less sensitive, ever. 

There’s always a reason why we are triggered by certain things based on our unique experiences, and this kind of statement shows ignorance to that. 

We can say many other things instead, like: 

“I’m sorry you’re upset - could you help me understand why this bothers you so much?”

“I think we can look at this from a different angle, what about…” 

“What can I do right now to make this situation better? I want to help make this right.”

It’s always easy to judge someone else, but trying to put yourself in someone else’s shoes is where real human connection starts. 

Be human, y’all! 

#empathy #collaboration #tiankaistuff

I’m really happy about anyone who voluntarily listens to my data music, but almost a thousand listeners from 51 countries is crazy! 

2025 was also very exciting because I’ve never performed music live so much as this year - I even got to play on a real concert piano thanks to the Inspari A/S data & AI day, and I got to write and perform a new original conference song for the Forward Data Conference! 

I know that making music about Data & AI is not everyone’s cup of tea - but as someone who’s passionate about the human side of data & AI, isn’t that subjectivity what truly shows our humanity? 

As long as my music brings a net positive to the data and AI community, I’ll keep on doing it - thank you all. 

P.S. I have some very cool new music coming before the year ends, stay tuned! 

#music #personalbranding #tiankaistuff

What do virtual meetings and Disney movies have in common? 

Not a lot, but I created this musical parody anyway. You’re welcome. 

And now everyone: 
🎶 can you see my screen right now? 🎶

YouTube link in comments. 

#meetings #humor #tiankaistuff

Forward Data Conference, here I come! 

This year’s final station of what for me felt like the “humanizing data & AI world tour” will be in Paris - and it will be a doozy! 

I’ll be giving a talk about the key success criteria of truly scaling AI, and I can’t wait to meet data friends like Joe Reis, Eevamaija Virtanen, Jessica Talisman, Andrea Gioia and Gaëlle SERET! 

The forward data conference was an excellent experience already last year, and from what I’ve seen so far Yoann Benoit, François Laurain and the team took it to really the next level this year.

Oh yeah, the scarf indoors is protecting my voice because there may or may not be some never before seen or heard music by yours truly tomorrow… 🎶

à demain!

#conference #data #tiankaistuff

Congratulations to Greg Freeman on publishing The Data & AI Literacy Bible - a book that’s timely, important and truly insightful! 

I had the honor to review and endorse this book, and what I really love about it is the focus on human confidence, growth and actions - aspects that are often forgotten in conceptual and abstract conversations around literacy. 

This having said, I’ve had the chance to meet various Data Literacy Academy folks and they’re all awesome - you can talk to them and find out yourself 🙂

The book is now out on amazon, link in the comments! 

#dataliteracy #book #tiankaistuff

There are many reasons why AI governance is important right now, but the biggest undeniable reason for me is that we need to admit that human-beings aren't perfect and make mistakes, and that we should do everything we can do prevent machines from making the same ones - faster and on bigger scale. 

I was asked to contribute to the "Document automation trends in 2026" report by Rossum and focused on the importance and challenges of AI governance, and there are many other interesting trends to read about that are not only specific to document automation but to AI use cases in general.

On a side note, I love how we are all trying our best to predict the next year, when AI specifically this year showed us that many things are truly unpredictable. I guess we all want some certainty for comfort, and I'm not blaming anyone.

Link to check out the full report in the comments!

Huge thanks to Julie BIASION ✨ for inviting me to contribute!  

#aigovernance #trends #tiankaistuff

🎶 EXCITING NEWS!🎶  

As a token of gratitude for all the opportunities and encouragement I got this year for my data & AI music, I want to give back with something very special: MY FIRST EVER (VIRTUAL) LIVE CONCERT!

Join me in an exclusive livestream where I'll perform my most iconic songs about, data & AI and so much more - and since it's live, of course I'll engage with all of your questions and comments. 

My only question for you is: Which of my songs do you want to see me perform live? The setlist is not completely done yet so I can still make changes 😎 

See you there!

#data #music #tiankaistuff Thoughtworks

TIANKAI LIVE IN CONCERT - a musical end of year celebration

How do I make my AI strategy human-centric, and why does it matter?! 

I’m so excited to return to DataQG to give a community webinar exactly on that topic tomorrow. 

Everyone can join, and there will be a live Q&A - so if you want to visit one of the best data governance communities and watch me improvise answers to your trick questions, join us! 

Thanks to my friend Philip Black for the invitation!

Link to sign up in the comments. 

#aistrategy #webinar #tiankaistuff

I often think about Formula 1 races as best practice environment where actual data-driven decisions are being made - a lot of them at an incredible speed! 

Being able to do that requires not only a well connected ecosystem of machines, tools, and systems - but also an incredibly well-oiled (get it?) team that understands how to work together in a high stakes environment. 

This is why I’m very excited about this upcoming Alteryx webinar with the Lisa Jarman, PhD from the McLaren Group Formula Team to exactly discuss how data-driven decisions are being made there, and I can’t wait to learn and get inspired!

See you at the webinar?

Link to register here: https://lnkd.in/dJi4dKKi

#webinar #dataanalytics #tiankaistuff

Do the Data

Founder

Deloitte

Data Analytics Manager

Novartis

Lead a team of data analysts and oversee the design and implementation of healthcare data analytics projects

Develop and maintain healthcare data management systems, ensuring data accuracy, completeness, and security

Develop and implement data analysis plans to support clinical and operational decision-making processes

Collaborate with stakeholders across the organization to identify and prioritize business intelligence needs, and ensure that data analytics projects align with the overall strategic goals of the organization
Analyze large datasets using statistical and data visualization tools to identify trends, patterns, and insights that can be used to drive business performance

Ensure compliance with data privacy and security regulations such as HIPAA, and work with legal and compliance teams to address any data-related issues

Develop and deliver presentations to communicate complex data analysis to non-technical stakeholders, including executive leadership, and make recommendations for process improvements based on data-driven insights

Stay up-to-date with the latest trends and technologies in healthcare data analytics and provide guidance on how the organization can leverage them to improve performance and patient outcomes

Data Analytics - Turning Coffee into Insights, One Caffeine-Fueled Query at a Time! | Healthcare Data | Financial Expert | Driving Business Growth | Data Science Consultant | Data Strategy

As a data analytics professional, I have a passion for turning raw data into meaningful insights that drive business decisions. With my experience in data visualization, statistical analysis, and data modeling, I can help your organization harness the power of data to optimize operations, improve customer experiences, and achieve your strategic goals.

🔍 Discovering Insights

My expertise lies in exploring complex data sets, uncovering hidden trends and patterns, and providing actionable insights that support informed decision-making. I can help you answer questions like "What are our customer's buying habits?" or "What factors are impacting our sales performance?", and use data to guide your business strategy.

🎨 Data Storytelling

I believe that data storytelling is an art form, and I love using creative and engaging data visualizations to communicate complex information in a way that's easy to understand. Whether it's through interactive dashboards or custom reports, I can help you see your data in a new light, and use it to tell the story of your business.

🧠 Continuous Learning

I'm a lifelong learner, and I'm always exploring new data tools and techniques to stay at the forefront of the industry. Whether it's machine learning, artificial intelligence, or predictive analytics, I'm excited to apply the latest advancements to help you gain a competitive edge.

🤝 Let's Connect

If you're looking for a data analytics professional who can help you unlock the power of your data, let's connect! Whether you're a startup or a large corporation, I'm excited to collaborate with you and help you achieve your business goals. Feel free to reach out at Ravenaravs@gmail.com

Ravena O

We throw around the term “AI agents” a lot, but this visual breaks down how we actually got here and where we are going.

Here is how I read stages in the image.

- LLM processing flow
This is the basic setup most people started with.
You type a prompt, the large language model processes it, and you get a text answer.
Helpful, but it has no real view of your world, your data, or your systems.

- LLM with document processing
Next, we plug in longer documents.
The model can now read PDFs, reports, knowledge bases and respond using that content.
This is where many “chat with your documents” tools live today.
Still, it is mostly a single shot question and answer pattern.

- LLM with RAGs and tools
Now we add retrieval and tools.
Retrieval means the agent can search your own data sources in real time instead of relying only on training data.
Tools mean it can call APIs, run code, hit a database or a SaaS application.
This is where the system stops being just a chat box and starts doing work.

- Multi modal LLM workflow
Here the model works across different input types, not only text.
It can look at images, screenshots, maybe audio or video, and combine that with text.
You also see memory and a more complete workflow.
The system can keep track of previous steps and use them to answer better next time.

Advanced AI agent architecture
In this stage, the agent does more than respond.
It can:

remember past conversations and actions

break a task into smaller steps

choose when to use tools, when to ask for more info, and when to stop

produce structured outputs that plug into other systems
This is closer to a digital teammate that can own a small business workflow end to end.

- Future AI agent architecture
The last box is where things get really interesting.
You do not have a single agent. You have many agents working together behind the scenes.
On top of them, there is an output layer that checks for safety, control, responsibility and interpretability.
This layer makes sure answers are not only useful, but also explainable, auditable and aligned with company rules.

For me, this evolution is a useful lens to judge any “AI agent” product. I ask myself where it sits on this path and what is missing for it to become a reliable, safe part of real business workflows.

CC: Brij Kishore Pandey

Ravena  O

How do we move from playing with AI agents to actually running them in production on AWS. The new AWS Marketplace and Agentic AI updates are basically trying to solve that gap. I am blown with 2300+ AI Agents and Tools listing that I saw in the Keynote announcements

You can even learn more about it here - https://lnkd.in/dkmvXM9m

What does this mean? Let me break it down

- Marketplace now has a clear area for AI agents and tools. Not just random listings. You see agents, products with agents inside, tools, and services in one place, sorted by what they help you automate

- Discovery feels more like “tell me the problem” and less like “guess the right keyword.” With things like agent mode, you describe what you want and get a few real options instead of pages of noise

- For a growing set of partners, you can land all this on Amazon Bedrock AgentCore, so you are not rebuilding identity, memory, actions, and monitoring from scratch every time

On top of that, AWS has added new Agentic AI categories in the AWS AI Competency. One for applications, one for tools, one for consulting.

If you are here at re:Invent and trying to figure out where to start with agents on AWS, I would use those two signals as a filter.

Check who is in the Agentic AI categories and how they show up in Marketplace, then go talk to those teams first.

#awsreinvent #aws AWS Partners Amazon Web Services (AWS) #awspartners #ravenaondata

Agentic AI just got a scoreboard at Amazon Web Services (AWS). At re: Invent AWS has formally recognized 60 partners in new Agentic AI categories inside its AI Competency. In a crowded AI market, this is one of the first clear signals of who can actually deliver working agent systems on AWS, not just demos. Check out more details here – https://lnkd.in/dk34XpGK

At a high level, AWS is doing three things here:

- Setting a higher bar for what “agent ready” means
- Sorting partners into clear categories that match how customers actually buy
- Connecting this to real demand for AI agents in AWS Marketplace

The competency breaks partners into three groups:
- Agentic AI applications
Vendors who ship products where agents are already solving real business problems end to end

- Agentic AI tools
Building blocks that help teams design, orchestrate, and monitor agents on AWS.

- Agentic AI consulting services
Partners who know how to design architectures, handle data access, and take these systems into production.

The interesting part is how this lines up with what we are already seeing in AWS Marketplace.

AI agents on Marketplace have grown far faster than expected, with adoption numbers reported at more than 40 times the internal targets. That tells you customers are not just experimenting. They are actively looking for packaged ways to bring agents into their environments.

On top of that, Marketplace has been evolving beyond single products:
Solution based offers where customers can buy a whole stack in one go

Bundles that combine software, services, and integration work

New “agent mode” style experiences that help teams discover and deploy agents faster

Put it together and a picture emerges:

- Marketplace becomes the front door for discovering and trying agentic solutions
- The Agentic AI competency tells you which partners have already hit a certain technical and customer bar
- The three categories make it easier to plan your mix of applications, tools, and services

Why does this matter?

Because most enterprises are now asking very practical questions:
- Where can we use agents safely in production today
- What should we buy off the shelf versus build
- Which partners have already solved problems similar to ours

The 60 partners in these new Agentic AI categories now sit at the intersection of those questions. 

This is not the end state, but it is a useful early map of who is doing serious work in agentic AI on AWS, and how buyers can navigate that ecosystem without getting lost in the noise.

#awsreinvent #aws AWS Partners Amazon Web Services (AWS) #awspartners #ravenaondata

RAG was introduced with a clear promise:
 to make LLMs smarter, give them memory, and anchor their responses in verifiable facts.
But there’s an important reality that often goes unspoken.
Most RAG implementations function like enhanced search engines.
 They retrieve documents, insert them into context, and expect the LLM to make sense of it.
That isn’t true intelligence — it’s structured copy-and-paste.
What’s emerging now represents a meaningful shift:
 agents are beginning to manage the entire retrieval and reasoning workflow.
Platforms like Glean, Perplexity, and Harvey are not simply retrieving documents.
 They are reasoning before retrieval, after retrieval, and at times choosing not to retrieve at all.

This changes the entire paradigm:

🔴 Instead of embedding every query by default, an agent evaluates:
 “What information is actually required here?”
🔴 Instead of flooding context with irrelevant chunks, it determines:
 “Which sources matter for this specific question?”
🔴 Instead of generating a single response, it reflects:
 “Did this answer fully address the user’s intent?”
🔴 Memory becomes functional:
 short-term for the current interaction, long-term for patterns across sessions.
🔴 A broader toolset becomes accessible:
 search engines, APIs, databases — the agent selects the right tool for the task.
🔴 The LLM stops being an isolated generator and becomes an integral component in a coordinated reasoning system.

This is Agentic RAG.
Not an incremental improvement in retrieval —
 but a fundamentally different architecture for enterprise intelligence.
And once you see it working inside real, complex workflows,
 traditional RAG begins to feel… noticeably incomplete.

CC: Om Nalinde

If your RAG system isn’t performing well, the problem is usually not the model — it’s the chunking strategy.

Chunking decides what your system actually “sees,” retrieves, and reasons over.
 Get it wrong, and even the best LLM will hallucinate.
 Get it right, and your RAG pipeline becomes dramatically more accurate.

Here’s a quick breakdown of the 10 major chunking techniques every RAG practitioner should know:

 ➡️ Fixed-length Chunking – Simple, consistent, great for baseline setups.
 ➡️ Semantic Chunking – Splits content based on meaning, not size.
 ➡️ Overlapping Chunking – Keeps context intact across boundaries.
 ➡️ Sliding Window Chunking – Keeps a rolling view of dynamic context.
 ➡️ Hierarchical Chunking – Multi-level and structured for complex docs.
 ➡️ Dynamic Chunking – Adapts chunk size to content length and density.
 ➡️ Sentence-based Chunking – Clean, natural breaks for readability.
 ➡️ Paragraph-based Chunking – Ideal for cohesive topics and narratives.
 ➡️ Token-based Chunking – Precise, model-friendly token-level control.
 ➡️ Contextual Chunking – Adjusts based on the specific retrieval goal.
Great RAG isn’t magic — it’s design.

 Master chunking, and the rest of the pipeline becomes significantly easier.

CC: Harshit Ahluwalia

#RAG #LLM #AIEngineering #VectorSearch

Chunking

You can build a RAG demo in an hour.
But building production-grade RAG is what interviews really test.

RAG is not just embed → retrieve → generate.
It’s a full engineering stack spanning parsing, chunking, embeddings, vector search, reranking, guardrails, latency, and evaluation.

To make your prep easier, I’ve compiled 50 real RAG interview questions—from fundamentals to deep production challenges.

Here’s what’s inside the cheat sheet 👇

🔴 1. Parsing & Data Prep
	•	PDF parsing failures
	•	OCR vs. text extraction
	•	Cleaning, deduping, noise detection

🔴 2. Chunking Strategy
	•	Fixed, semantic, recursive, hierarchical
	•	Overlap logic & boundary effects
	•	Chunking code vs. text

🔴 3. Embeddings & Vector Stores
	•	Model selection
	•	Drift, noise, normalization
	•	Index types: Flat, HNSW, IVF

🔴 4. Retrieval & Reranking
	•	Hybrid search
	•	ANN recall tuning
	•	Cross-encoders, MMR, HyDE
	•	Multi-hop retrieval

🔴 5. Latency, Cost & Scaling
	•	Query caching
	•	Parallel retrieval
	•	Token budgeting
	•	Dimensionality trade-offs

🔴 6. RAG Guardrails
	•	Hallucination control
	•	Citation enforcement
	•	Retrieval poisoning defense
	•	Jailbreak detection

🔴 7. Memory & Agentic Extensions
	•	Short-term vs long-term memory
	•	Chunk routing
	•	Agentic RAG workflows

👉 If you’re preparing for RAG-heavy roles or building real-world retrieval systems, this cheat sheet helps you understand the why behind every design choice.

🔖 Save for interview prep.
♻️ Repost to help others level up their RAG skills.

CC: Naresh Edagotti

50 RAG interview Questions 

Why Vector Databases Are Now Core to Modern AI

Vector databases quietly power the AI experiences we use every day. Behind every smart search, every RAG pipeline, and every reliable chatbot—embeddings and similarity search do the heavy lifting.

🔴 What They Do
Store and retrieve high-dimensional embeddings so systems can search by meaning, not keywords.

🔴 Why AI Uses Vectors
Embeddings capture intent, context, tone, and relationships—enabling semantic understanding.

🔴 How They Work
Embed → Index → Similarity Search → Rank → Reason.
The backbone of RAG and intelligent search.

🔴 Why They Matter
They reduce hallucinations, provide long-term memory, and enable stable, enterprise-grade reasoning.

🔴 Where They’re Used
	•	Chatbots (context + domain knowledge)
	•	Search engines (semantic + multimodal search)
	•	Recommendation systems (personalized, context-aware suggestions)

🔴 Popular Vector DBs (2025)
Pinecone, Weaviate, ChromaDB, FAISS, Milvus, Qdrant.

🔴 Key Features to Know
ANN search, hybrid retrieval, distributed indexing, sharding, real-time embedding updates, LLM re-ranking.

Vector databases are no longer optional—they’re a foundational layer of the AI stack. Understanding embeddings and similarity search is now a real competitive advantage.

CC: Greg Coquillo

Understanding Vector Database

If Data Engineering feels overwhelming, it’s rarely the tools — it’s the terminology.

Master these core concepts, and the entire data ecosystem becomes clear.
Here’s a concise breakdown of 30 fundamentals every DE, analyst, and ML engineer should know:

🟣 DATA MOVEMENT
 🔴 ETL vs ELT: ETL transforms before loading; ELT loads raw and transforms  later.
 🔴 Stream vs Batch: Stream = real-time; Batch = scheduled processing.
 🔴 Data Pipeline: Automated flow from source to destination.

🟣 DATA STORAGE
 🔴 Data Warehouse: Structured, analytics-ready (Snowflake, BigQuery).
 🔴 Data Lake: Stores raw, unprocessed data.
 🔴 Data Lakehouse: Lake flexibility + warehouse reliability (Delta, Iceberg).

🟣 DATA STRUCTURE
 🔴 Star Schema: Simple, fast for analytics.
 🔴 Snowflake Schema: Normalized variant with more joins.
 🔴 Data Modeling: Designing relationships for clean, efficient queries.

🟣 DATA PROCESSING
 🔴 OLTP: Real-time transactions.
 🔴 OLAP: Analytical workloads.
 🔴 Partitioning: Slice data to reduce full scans.
 🔴 Sharding: Distribute data across servers for scale.

🟣 DATA DISCOVERY
 🔴 Data Catalog: Metadata and dataset discovery.
 🔴 Data Lineage: Track data end-to-end.
 🔴 Indexing: Faster lookups on large tables.

🟣 QUALITY & GOVERNANCE
 🔴 Data Quality: Accuracy, completeness, consistency.
 🔴 Data Governance: Access control, security, compliance.
 🔴 Caching: Quick access to frequently used data.

🟣 SYSTEM DESIGN
 🔴 Distributed Systems: Many machines acting as one.
 🔴 Message Queue: Async communication (Kafka, RabbitMQ).
 🔴 Orchestration: Workflow control (Airflow, Prefect).
 🔴 Fault Tolerance: Operate even during failures.
 🔴 Scalability: Grow without re-architecting.

💡 Truth: Tools change constantly. These concepts don’t.
 Master them, and you’re already ahead of most professionals in the field.

Image Credits: Shalini Goyal

What happens when your AI can’t reason, can’t coordinate tools, and can’t operate safely at scale?

You don’t have an agent — you have a chatbot.
Agentic AI is evolving into a full technology stack.
Not just models, but an entire ecosystem designed for autonomous, tool-using, decision-making systems.

Here’s the stack broken down into seven essential layers:

🟡 Layer 1 — Foundation Models & Infrastructure
 LLMs, vector databases, cloud compute (GPU/TPU), data pipelines, and hosting.
 This layer provides the raw reasoning and generative power.

🟡 Layer 2 — Agent Runtime & Execution Infrastructure
 Docker, Kubernetes, workflow engines, sandboxed runtimes.
 Secure, scalable environments where agents operate autonomously.

🟡 Layer 3 — Protocols & Interoperability
 MCP, AGP, function-calling APIs, tool invocation standards, agent-to-agent communication.
 The language that allows systems to work together.

🟡 Layer 4 — Orchestration
 LangGraph, CrewAI, Semantic Kernel, AutoGPT, Camel AI, PyDanticAI.
 The frameworks that manage multi-step workflows, memory, and reasoning loops.

🟡 Layer 5 — Tooling & Enrichment
 Retrieval systems, vector stores, browser automation, APIs, scraping engines, memory layers.
 Everything that expands an agent’s capabilities and context.

🟡 Layer 6 — Applications
 Copilots, autonomous teammates, AI operations assistants, research agents, domain-specific apps.
 Where real business value emerges.

🟡 Layer 7 — Observability & Governance
 Guardrails, Lakera, LangSmith, Arize, Helicone, Langfuse.
 The safety, monitoring, and reliability layer that makes agentic systems enterprise-ready.

Agentic AI isn’t a feature or a product —
 it’s a layered architecture.

When all seven layers work together, AI gains the ability to reason, plan, act, coordinate tools, collaborate with humans, and operate safely at scale.
This is the blueprint for the next generation of intelligent enterprise systems.

Data & AI Governance is one of the most misunderstood capabilities in the enterprise.

Some see it as control. Others see it as compliance.
In reality, governance is the discipline that creates trust, alignment, and accountability—especially as intelligence becomes distributed across systems, teams, and AI agents.

Its role today is simple:
Governance ensures data doesn’t just move—it moves with meaning, securely, ethically, and with intent.

🔴 Why Governance Matters More Than Ever

🔴 Distributed intelligence needs shared accountability.
As humans, systems, and AI agents co-execute decisions, accountability can’t sit in one place. Governance is the only way to preserve intent across autonomous actors that adapt and learn independently.

🔴 Business operations now are data products.
Modern operations run on APIs, pipelines, and platform logic—not org charts. Governance ensures these data-driven processes remain intelligible, traceable, and aligned to strategic goals.

🔴 Federated models require a shared grammar.
Decentralization works only when every domain speaks a compatible language. Governance isn’t control—it’s choreography. It allows federated teams to scale without fragmenting.

🔴 Democratization requires responsibility, not just access.
Giving teams and agents more data power is cultural and ethical, not technical. Governance turns access into responsible action, ensuring coherence as capabilities grow.

The mission of Data Governance:
When intelligence scales, integrity must scale with it.
The organizations that win will be the ones where human and AI intelligence act with coherence—not in isolation.

Where does your organization stand on this spectrum?

CC: Respective Owner

taplio

The Battle

Tiankai Feng

Ravena O

Followers

Posts

Engagements

Likes

Performance ScoreTiankai Feng's LinkedIn Performance

Performance ScoreRavena O's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Tiankai Feng

Ravena O

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Tiankai Feng's LinkedIn Performance

Performance ScoreRavena O's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreTiankai Feng's LinkedIn Performance