I love to teach. I post about #dataengineering and #mentalhealth daily. 

Many years of data engineering experience doing growth analytics at Facebook, security infrastructure at Netflix, and pricing and availability infrastructure at Airbnb. 

You can find me by looking up EcZachly on all social networks 

Join my Data Learning Platform https://www.dataexpert.io

Zach Wilson

Databricks is going to win “data” the same way that Meta won “social media” 

They are uniquely positioned because of the convergence of a few trends:

- the death of the data analyst
We are moving away from analytical architectures and towards action-oriented architectures. Instead of “the data engineer does ETL so analysts can look at it” it will be “the data engineer creates context for AI agents to do something” Actions will reduce timelines and speed up business velocity. Humans will no longer be the analytical bottleneck. 

- the crushing of data siloes 
Open table formats like delta and iceberg will make business able to do things that were previously though impossible! We’ll finally have all the context for our AI in one place. Entire business context + Agents = a waterfall of business value! 


I’m launching a paid Databricks boot camp on February 9th. Send me a DM with the word Databricks and I’ll get you the details!

Founder @ DataExpert.io | ADHD | 500k+ followers | Angel Investor

No alternative text description for this image

The first speaker lineup for the analytics engineering boot camp is going to be so amazing! 

These sessions are what truly separates my boot camp from “just another data course”

Data modeling is far from dead! It’s actually more relevant than ever. 

There’s been an interesting shift in the seas with AI. Some people saying we don’t need to do facts and dimensions anymore. This is a wild take because product analytics don’t suddenly disappear because LLM has arrived.

It seems like to me that multi-modal LLM is bringing together the three types of data:

- structured

- semi-structured

- unstructured

Dimensional modeling is still very relevant but will need to be augmented to include semi-structured outputs from the parsing of text and image data.

The necessity for complex types like VARIANT and STRUCT seems to be rising. Which is increasing the need for data modeling not decreasing it.

It feels like some company leaders now believe you can just point an LLM at a Kafka queue and have a perfect data warehouse which is still SO far from the actual reality of where data engineering sits today

Am I missing something or is the hype train just really loud right now?

I'm doing a live with Karan Dwivedi to talk about a cybersecurity boot camp that is launching on January 9th! 

This boot camp has an amazing capstone project where you do all of these on a company server!
- threat modeling
- detection engineering
- preventative controls 

Don't miss this webinar tomorrow!

January 9th Cybersecurity Boot Camp Webinar

Open to Collaborate | Storyteller | Lead Data Engineer@Wavicle| Linkedin Top Voice 2024 | Globant | Linkedin Learning Instructor | 2xGCP & AWS Certified | LICAP'2022

Data Engineer with 7+ years of experience working on data engineering, analysis, and modelling projects for various domains, such as banking, healthcare, sales, and e-commerce. I leverage my strong statistical and analytical ability to explore business ideas and provide innovative solutions. 

Skilled with technologies and strong intellect with enthusiasm as an avid learner with challenging opportunities. I’m working as a LinkedIn Learning Instructor to bring change as a data evangelist.

Post Graduated in Master of Technology in Data Science and Engineering from Birla Institute of Technology and Science, Pilani in 2023, where I learned advanced concepts and techniques in data science and engineering, machine learning, cloud computing, and visualization. 

I hold multiple certifications in GCP, AWS, and Azure, and I enjoy working with cloud technologies and big data platforms.

I am passionate about data science and women empowerment, and I have been recognized as the 
- Linkedin Top Voice 2025, 2024
- Data Science and Women Empowerment Thought Leader by Global AI Hub. 

I am also a proud participant of the LinkedIn Creator Accelerator Program 2022, where I share my insights and experiences with the data science community. 

I am open to collaborate and share ideas, or explore various opportunities in the data field.

Pooja Jain

Learn, Earn or Leave

Loyalty is beautiful… until you realize  
it was never mutual.

Your company will replace you faster than you can say "team player"

The company?  
It’s just business.  
Treat it accordingly. 🛡️

Agree?

Data pipelines are evolving from the "copy-paste" era to the "access-anywhere" era. Are you evolving with them?

At its core, data pipelines answers one thing -
How does data moves from "happened" to "decision made"?

Here's how the evolution looks like:
𝗘𝗧𝗟 (The Old Way) Extract → Transform → Load
 • Like preprocessing your groceries in the parking lot before bringing them inside
 • Great for strict governance, legacy systems
 • Slow. Rigid. Breaks on every schema change
 • Tools: Glue, Talend, NiFi

𝗘𝗟𝗧 (The Cloud Era) Load raw → Transform in warehouse
 • Load raw, transform in warehouse. Like, Dump groceries in fridge, prep later
 • Scales with cloud compute
 • Better, but still copying data everywhere
 • Tools: dbt, Snowflake, BigQuery

𝗖𝗗𝗖 (Real-Time Tax) Change Data Capture
 • Stream every database change via transaction logs
 • Captures inserts/updates/deletes from source. Near real‑time, low overhead
 • Perfect for fraud detection. Expensive for everything else
 • Tools: Debezium, AWS DMS

𝗦𝘁𝗿𝗲𝗮𝗺𝗶𝗻𝗴 (Over-Engineering Champion) 
 • Real‑time processing, no waiting. Just like Food truck serving as you arrive
 • Necessary for IoT, trading, Fraud and live dashboards
 • Overkill for your Monday morning dashboard
 • Tools: Kafka, Flink, Kinesis.

Enter 𝗭𝗲𝗿𝗼 𝗘𝗧𝗟: The "Why Are We Doing This?" Moment
The modern lakehouse game‑changer
One city library for everyone – no need to photocopy books.
Data lives in one place (lakehouse).
Query it directly with any engine. No copying.

🤔 How Modern Lakehouses Make This Real?

𝗢𝗽𝗲𝗻 𝗧𝗮𝗯𝗹𝗲 𝗙𝗼𝗿𝗺𝗮𝘁𝘀 (Iceberg, Hudi, Delta)
Think: Git for data. ACID transactions, time travel, schema evolution—on cheap object storage.

𝗦𝘁𝗼𝗿𝗮𝗴𝗲 ≠ 𝗖𝗼𝗺𝗽𝘂𝘁𝗲
Data sits in S3. Spark reads it. Snowflake queries it. Trino analyzes it. Same files. No copies. No vendor lock-in.

𝗡𝗮𝘁𝗶𝘃𝗲 𝗖𝗼𝗻𝗻𝗲𝗰𝘁𝗼𝗿𝘀
AWS Aurora → S3? Managed replication.
No custom Glue jobs. No 3 AM debugging.

What Changes for You?

You stop:
- Writing extraction code for the 47th time
- Maintaining Airflow DAGs that "mysteriously fail on Wednesdays"
- Explaining why reports are 6 hours delayed
- Paying for storage in four different systems

You start:
- Designing smart partitioning strategies
- Building dbt models where transformation actually matters
- Focusing on data quality, not data plumbing
- Solving business problems instead of pipeline problems

The Real Talk?

Zero ETL isn't "no data engineering." 
It's engineering that matters.
Traditional ETL won't vanish overnight—too much legacy infrastructure. 

But for new projects? Start lakehouse-first:
- Store once in open formats
- Let compute engines come to data
- Create copies only when necessary

Think of it this way: 
You wouldn't photocopy a book every time someone wants to read a different chapter. 
So why copy your data every time a new team needs access?

Without governance, your data is a 10-lane highway with no traffic lights.

Everyone's driving. Nobody knows the rules. Accidents are inevitable.

Why does it Matters?
Think traffic system for organizational data.
 • Data flows smoothly, safely, smartly
 • Everyone knows their lane
 • Decisions made with confidence
 • Teams drive results, not fix messes

Without traffic lights, you don't get speed; you get gridlock and crashes.

🔑 Your Role as a Leader
Strategy & Oversight:
 → Champion governance (sponsorship + resources)
 → Set the tone for responsible data use
 → Hold leaders accountable for data quality
 → Not your job: Managing day-to-day data operations

👥 Who Does What ?(Simplifying the roles)
 • Chief Data Officer: Sets data strategy
 • Data Security Officer: Shields from risk
 • Head of Governance: Enforces data rules
 • Data Quality Manager: Validates accuracy

 → Data Owners: Accountable for specific datasets
 → Data Stewards: Maintain quality and access
 → Product Owners: Align data with user needs
 → Data Custodians: Handle storage and security

Here's to quick start and win early -
Reflect: Where do you rely on data today? Where do you trust it?
Clarify: Which outcomes matter most?
Identify: Who owns critical data decisions?

But, Why Now?
AI is scaling. Without governance, pilots stall and ROI evaporates.
Competition moves fast. Trusted data = speed advantage.
Regulations tighten. Fines and reputation hits are expensive.

𝘎𝘰𝘰𝘥 𝘨𝘰𝘷𝘦𝘳𝘯𝘢𝘯𝘤𝘦 𝘪𝘴𝘯'𝘵 𝘣𝘶𝘳𝘦𝘢𝘶𝘤𝘳𝘢𝘤𝘺—𝘪𝘵'𝘴 𝘵𝘩𝘦 𝘣𝘭𝘶𝘦𝘱𝘳𝘪𝘯𝘵 𝘧𝘰𝘳 𝘈𝘐 𝘵𝘩𝘢𝘵 𝘢𝘤𝘵𝘶𝘢𝘭𝘭𝘺 𝘥𝘦𝘭𝘪𝘷𝘦𝘳𝘴.

My final thoughts:
If 𝗱𝗮𝘁𝗮 is the fuel and 𝗔𝗜 is the engine,
𝗚𝗼𝘃𝗲𝗿𝗻𝗮𝗻𝗰𝗲 is the steering wheel.
Without it, speed means nothing.

Refer this to explore what Atlan says about Data Governance pillars in 2026 -https://lnkd.in/dAMTYJGy

taplio

The Battle

Zach Wilson

Pooja Jain

Followers

Posts

Engagements

Likes

Performance ScoreZach Wilson's LinkedIn Performance

Performance ScorePooja Jain's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Zach Wilson

Pooja Jain

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Zach Wilson's LinkedIn Performance

Performance ScorePooja Jain's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreZach Wilson's LinkedIn Performance