DataExpert.io

Founder

Ardent AI

I think AI will disrupt data engineering in a significant way. I’m investing in Ardent to accelerate that disruption!

Angel Investor

Peony Health

I invested in Peony Health because I believe habits are greatly impacted by neurochemistry and Susie Chen is a technical founder with business chops who will deliver on their mission

Illuminate AI

I'm one of the new Bay Area chapter leads for Illuminate AI! 

I help make events better and build this community to the next level!

Chapter Lead

Mage

I'm advising Mage on how to make a data engineering friendly Airflow competitor that will be the next generation of job orchestration!

Advisor

A House in Utah

Former Child

SylphAI

I’ve invested in SylphAI because I’m passionate about building community in AI that helps everyone meet their growth goals and fulfill their potential

Airbnb

Changed Airbnb's availability definition from "did a host set a rule that blocked this night" to "can a trip be booked that contains this night" 

Consolidate the availability definition across the company. Saving >500 TBs in the cloud and shrinking the data set to 3% its old size.


Decreased the landing time of Airbnb's unit economics dataset from 3 days to 1 1/2 days. Built a long-term roadmap to further increase the quality of this critical dataset. 

Improved the quality of online systems at Airbnb to be easily compilable by Spark pipelines to compute metrics offline. 

Lead a team of 6 engineers to deliver on the MIDAS initiative to increase the data quality of the Commercial Products org. 

Upleveled smart pricing at Airbnb by improving the feature engineering and latency of the data used to train the smart pricing model.

Staff Data Engineer - TC $600k ($220k Salary $320k Stock $60k bonus)

Netflix

Built a machine learning feedback system that allowed security engineers to label corporate user behavior as risky or not risky.

Built Asset Inventory - a graph database solution that is a map of all of Netflix's cloud infrastructure.

Senior Software Engineer - TC ~$575k ($550k salary + 5% options)

I built a pipeline that measures the cloud infrastructure impact on AB tests, saving Netflix millions by allowing them to make smarter AB test rollout decisions.

Senior Data Engineer - TC ~$365k ($365k salary + 5% options)

Facebook

- Managed a 10 PB+ Hive data warehouse
- Consolidated and conformed company-wide growth metrics (across WhatsApp, Instagram, Messenger, and Facebook) into a single, company-wide view. 
- Optimized machine learning feature set generation pipelines (200+ TB/day) from having a 4 day latency to having a 1 day latency. While also dropping compute costs for those pipelines 4x. 
- Reduced core notification data set latencies from 36 hours to < 8 hours. 
- Migrated 50% of notifications pipelines from using Hive to use Spark, Presto, or real-time streaming.
- Cut compute cost from notifications pipelines by 40% over the course of 9 months.

Data Engineer - TC $185k ($135k salary, $50k stock)

I love to teach. I post about #dataengineering and #mentalhealth daily. 

Many years of data engineering experience doing growth analytics at Facebook, security infrastructure at Netflix, and pricing and availability infrastructure at Airbnb. 

You can find me by looking up EcZachly on all social networks 

Join my Data Learning Platform https://www.dataexpert.io

Zach Wilson

I’m happy to share that I’m starting a new position as Strategic Advisor at omi!

I believe in Nik Shevchenko and I want to start having a much more data rich life!

Founder @ DataExpert.io, use code BF for 40% off!

Hitting 500k followers here on LinkedIn feels like a dream come true!

I started posting every day back on December 12th, 2020 and I had no idea it would impact my life in such a profound way

I met so many amazing creators along this journey like Carly, Aishwarya, Joe, Chip, Delia, Li, Annie, Jhillika and many more! 

To celebrate 500k followers, You can use code 500k at DataExpert.io/500k to get 50% off! This deal ends today at midnight pacific time! 

How long until I hit 1 million?

We're making some important updates to the Snowflake + dbt boot camp on January 6th! 

- We are limiting seats to 60 students to keep quality of instruction high (we are already halfway booked up)
- We are adding Cortex lessons for both vector search and document parsing
- We are adding a lesson change data capture 
- We are adding an Iceberg and Snowflake module for those interested in the open data format! 

If you're interested in some early Black Friday deals on this boot camp, send me a DM!

I'm launching a free data science interview boot camp with Vaishali Macwan tomorrow!

For the next four weekends, Vaishali will go over the most important things to land a product data scientist role!

- Week 1 Getting Noticed
You can't land a job without landing an interview. Master the job search skills quickly!

- Week 2: The fundamentals and interview formats
SQL is absolutely essential for data scientists. We'll be covering the different interview formats this week too

- Week 3: How to pass the product sense and company fit rounds
Product sense is one of the most difficult interview rounds for data scientists. Vaishali has mastered this round and will pull back the curtain!

- Week 4: The behavioral round and negotiation
These two rounds are critical to sealing the deal for your new data science job and getting paid the most you can!

You can join the free boot camp here: https://lnkd.in/gj7xxGU3

Wavicle Data Solutions

Lead Data Engineer

LinkedIn

LinkedIn Learning Instructor

Globant

Senior Data Engineer

CRISIL Limited

Senior Analyst - Data Engineer

Quantiphi

Data Engineer

Tata Consultancy Services

System Engineer

Assistant System Engineer

Assistant System Engineer - Trainee

InsideSherpa

KPMG Data Analytics Consulting Virtual Internship

Monisha Enterprises

•	Data mining and Extraction Transformation Loading using R programming.
•	Data cleaning and processing for Data analysis done using Python using NumPy, pandas, SciPy, Scikit. Data modelling and prediction was done using algorithms – Logistic Regression, Time Series Forecasting, Clustering and Classification.
•	Data Visualization done using Tableau for generating reports to enhance the productivity of the products.
•	Built a Data Warehouse on MSSQL Server for ERP System of the enterprise to organize the data for future processing.

Data Analyst Intern

Open to Collaborate | Storyteller | Lead Data Engineer@Wavicle| Linkedin Top Voice 2024 | Globant | Linkedin Learning Instructor | 2xGCP & AWS Certified | LICAP'2022

Data Engineer with 7+ years of experience working on data engineering, analysis, and modelling projects for various domains, such as banking, healthcare, sales, and e-commerce. I leverage my strong statistical and analytical ability to explore business ideas and provide innovative solutions. 

Skilled with technologies and strong intellect with enthusiasm as an avid learner with challenging opportunities. I’m working as a LinkedIn Learning Instructor to bring change as a data evangelist.

Post Graduated in Master of Technology in Data Science and Engineering from Birla Institute of Technology and Science, Pilani in 2023, where I learned advanced concepts and techniques in data science and engineering, machine learning, cloud computing, and visualization. 

I hold multiple certifications in GCP, AWS, and Azure, and I enjoy working with cloud technologies and big data platforms.

I am passionate about data science and women empowerment, and I have been recognized as the 
- Linkedin Top Voice 2024 
- Data Science and Women Empowerment Thought Leader by Global AI Hub. 

I am also a proud participant of the LinkedIn Creator Accelerator Program 2022, where I share my insights and experiences with the data science community. 

I am open to collaborate and share ideas, or explore various opportunities in the data field.

Pooja Jain

73% of data pipeline failures happen because of dependency issues, not bad code.
Cron jobs don't care if yesterday's data never arrived. They'll happily run on empty tables and produce garbage.

Airflow? It checks, retries, alerts, and refuses to cascade your failure downstream.

→ Beyond scheduling Airflow gives an important feature of 𝗗𝗔𝗚𝘀(𝗗𝗶𝗿𝗲𝗰𝘁 𝗔𝗰𝘆𝗰𝗹𝗶𝗰 𝗚𝗿𝗮𝗽𝗵𝘀), that helps to monitor, retry and gives flexibility to organize pipelines with minimum/less code.

🤔 Data Engineers often misinterpret Airflow just as an orchestration tool.

Apache Airflow is an open-source platform designed to programmatically author, schedule, retry and monitor workflows. 

🎼Think of it as the "conductor of an orchestra", where each musician (your data tasks, like data extraction, transformation, loading, or model training) knows when to play, what to play, and what happens if they make a mistake.

Most data engineers automate the wrong things at the wrong time.

Here's the framework I use after 8 years of building production systems:
✅ AUTOMATE WHEN: 
→ Task runs daily/weekly 
→ Human errors cause outages 
→ Work blocks other priorities
→ Team growth = more manual work
Examples: Reports, schema checks, alerts

❌ DON'T AUTOMATE WHEN: 
→ Task happens quarterly 
→ Requirements change weekly 
→ Process isn't understood yet 
→ Manual steps reveal insights

My rule: If it’s done 3+ times, script it; 10+ times, automate it; fails 5+ times, redesign it.
Automate what matters, when it matters—not everything!

Airflow isn’t just about running tasks.

🕑 It’s about building trustworthy, scalable, and observable pipelines that run reliably at 2am — even when you’re not watching.

Airflow gives structure to your chaos.
DAGs give clarity to your pipelines. 
That’s why DEs love them ❤️

Real Example: 
Instead of: 
❌ Cron jobs that fail silently 
❌ Manual dependency management
❌ No visibility into failures
You get: 
✅ Visual workflow monitoring 
✅ Automatic failure notifications
✅ Smart task scheduling 
✅ Easy debugging & restarting

Here's an amazing Airflow cheatsheet with some fundamentals concepts and interview questions to prepare for data engineers. 👇

Carousel Credits -  Futurense.com

Willing to learn more, here's the link to Astronomer free learning path - https://lnkd.in/g8-EWBda

Also, level up with Marc Lamberti's Udemy course on Airflow 3 - https://lnkd.in/gwArwaME

taplio

The Battle

Zach Wilson

Pooja Jain

Followers

Posts

Engagements

Likes

Performance ScoreZach Wilson's LinkedIn Performance

Performance ScorePooja Jain's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Zach Wilson

Pooja Jain

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Zach Wilson's LinkedIn Performance

Performance ScorePooja Jain's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreZach Wilson's LinkedIn Performance