Wavicle Data Solutions

Lead Data Engineer

LinkedIn

LinkedIn Learning Instructor

Globant

Senior Data Engineer

CRISIL Limited

Senior Analyst - Data Engineer

Quantiphi

Data Engineer

Tata Consultancy Services

System Engineer

Assistant System Engineer

Assistant System Engineer - Trainee

InsideSherpa

KPMG Data Analytics Consulting Virtual Internship

Monisha Enterprises

•	Data mining and Extraction Transformation Loading using R programming.
•	Data cleaning and processing for Data analysis done using Python using NumPy, pandas, SciPy, Scikit. Data modelling and prediction was done using algorithms – Logistic Regression, Time Series Forecasting, Clustering and Classification.
•	Data Visualization done using Tableau for generating reports to enhance the productivity of the products.
•	Built a Data Warehouse on MSSQL Server for ERP System of the enterprise to organize the data for future processing.

Data Analyst Intern

Open to Collaborate | Storyteller | Lead Data Engineer@Wavicle| Linkedin Top Voice 2024 | Globant | Linkedin Learning Instructor | 2xGCP & AWS Certified | LICAP'2022

Data Engineer with 7+ years of experience working on data engineering, analysis, and modelling projects for various domains, such as banking, healthcare, sales, and e-commerce. I leverage my strong statistical and analytical ability to explore business ideas and provide innovative solutions. 

Skilled with technologies and strong intellect with enthusiasm as an avid learner with challenging opportunities. I’m working as a LinkedIn Learning Instructor to bring change as a data evangelist.

Post Graduated in Master of Technology in Data Science and Engineering from Birla Institute of Technology and Science, Pilani in 2023, where I learned advanced concepts and techniques in data science and engineering, machine learning, cloud computing, and visualization. 

I hold multiple certifications in GCP, AWS, and Azure, and I enjoy working with cloud technologies and big data platforms.

I am passionate about data science and women empowerment, and I have been recognized as the 
- Linkedin Top Voice 2024 
- Data Science and Women Empowerment Thought Leader by Global AI Hub. 

I am also a proud participant of the LinkedIn Creator Accelerator Program 2022, where I share my insights and experiences with the data science community. 

I am open to collaborate and share ideas, or explore various opportunities in the data field.

Pooja Jain

73% of data pipeline failures happen because of dependency issues, not bad code.
Cron jobs don't care if yesterday's data never arrived. They'll happily run on empty tables and produce garbage.

Airflow? It checks, retries, alerts, and refuses to cascade your failure downstream.

→ Beyond scheduling Airflow gives an important feature of 𝗗𝗔𝗚𝘀(𝗗𝗶𝗿𝗲𝗰𝘁 𝗔𝗰𝘆𝗰𝗹𝗶𝗰 𝗚𝗿𝗮𝗽𝗵𝘀), that helps to monitor, retry and gives flexibility to organize pipelines with minimum/less code.

🤔 Data Engineers often misinterpret Airflow just as an orchestration tool.

Apache Airflow is an open-source platform designed to programmatically author, schedule, retry and monitor workflows. 

🎼Think of it as the "conductor of an orchestra", where each musician (your data tasks, like data extraction, transformation, loading, or model training) knows when to play, what to play, and what happens if they make a mistake.

Most data engineers automate the wrong things at the wrong time.

Here's the framework I use after 8 years of building production systems:
✅ AUTOMATE WHEN: 
→ Task runs daily/weekly 
→ Human errors cause outages 
→ Work blocks other priorities
→ Team growth = more manual work
Examples: Reports, schema checks, alerts

❌ DON'T AUTOMATE WHEN: 
→ Task happens quarterly 
→ Requirements change weekly 
→ Process isn't understood yet 
→ Manual steps reveal insights

My rule: If it’s done 3+ times, script it; 10+ times, automate it; fails 5+ times, redesign it.
Automate what matters, when it matters—not everything!

Airflow isn’t just about running tasks.

🕑 It’s about building trustworthy, scalable, and observable pipelines that run reliably at 2am — even when you’re not watching.

Airflow gives structure to your chaos.
DAGs give clarity to your pipelines. 
That’s why DEs love them ❤️

Real Example: 
Instead of: 
❌ Cron jobs that fail silently 
❌ Manual dependency management
❌ No visibility into failures
You get: 
✅ Visual workflow monitoring 
✅ Automatic failure notifications
✅ Smart task scheduling 
✅ Easy debugging & restarting

Here's an amazing Airflow cheatsheet with some fundamentals concepts and interview questions to prepare for data engineers. 👇

Carousel Credits -  Futurense.com

Willing to learn more, here's the link to Astronomer free learning path - https://lnkd.in/g8-EWBda

Also, level up with Marc Lamberti's Udemy course on Airflow 3 - https://lnkd.in/gwArwaME

Airflow

Ford Credit

● Led data modernization project with GCP for Europe region at Ford Credit, ensuring alignment with regulatory restrictions.
● Developed data ingestion, analytical domain, and data mart for presentation layer, enhancing data accessibility and insights.
● Implemented solutions to optimize data processing and storage, improving efficiency and scalability.

Just Eat Takeaway.com

● Developed and implemented a Security Data Insights Project on GCP, facilitating real-time queries and resulting in a 30% reduction in response time for security threat mitigation.
● Collaborated cross-functionally with InfoSec teams to architect a scalable data infrastructure, leading to a 50% increase in data processing efficiency.
● Orchestrated the delivery of a dynamic dashboard and automated ETL pipeline, optimising vulnerability management processes and achieving a 40% improvement in data accuracy

bigspark

● Led a team of 5 to create an MVP for a Business-Screening Project, resulting in a 20% increase in client satisfaction and a 30% reduction in time-to-market.
● Managed a team of 6 and collaborated with 20 pipeline owner teams to migrate 140 pipelines from Kafka Cloudera to Kafka Confluent, achieving a 50% reduction in data processing costs.
● Designed and constructed an end-to-end ingestion and analytics platform in Microsoft Azure for a Real Estate client, resulting in a 25% increase in data accessibility and a 15% improvement in data analysis speed.

PayU

● Automated and fine-tuned a critical machine learning model using Python, AWS EMR, Apache Spark, and Apache Airflow, resulting in a 30% improvement in model accuracy and a 20% reduction in model training time.
● Streamlined data processing by developing efficient data loading and aggregation frameworks using Apache Airflow, leading to a 40% decrease in data processing errors.
● Revitalised data and pipeline architecture, resulting in a 60% increase in business productivity and a 25% reduction in infrastructure costs.

● Collaborated with clients to identify process improvements for consumer payment options, resulting in a 15% increase in transaction success rates and a $10,000 reduction in payment processing costs.
● Reduced payment processing failure rates, saving $10,000 in costs.
● Devised and executed solutions that streamlined workflows and maximised resource utilisation, resulting in a 20% increase in team efficiency.

Senior Data Analyst

● Achieved up to a 23% improvement in the success rate of statistical analysis models on large data sets, leading to more accurate business insights.
● Revamped reporting dashboards and planning tools, reducing data processing time by 95% and enabling faster decision-making.
● Systematized data pipelines, reducing processing time by 40% and enhancing data accessibility for stakeholders.

Data Analyst

SAMRAT FORGINGS LIMITED

● Devised and deployed a modernised payroll management system utilising SQL and PHP, resulting in a 20% reduction in processing time for the HR team.

Intern

Security Data Engineer @Just Eat Takeaway| Ex-bigspark, PayU| Mentor @ Learnbay| Women Health | Mental Health

As the Senior Data Engineer at Ford Credit, my role encompasses leveraging advanced cloud technologies to drive innovation and efficiency in data management. Harnessing a solid foundation from Dr. A.P.J. Abdul Kalam Technical University in Computer Science, I've translated academic concepts into tangible results, such as implementing a Security Data Insights Project that significantly reduced threat response times. 

My commitment extends beyond technical skills to championing diversity, equity, and inclusion in the tech industry. As a mentor and advocate for Women in Tech, we work towards creating inclusive environments where every voice is heard and valued. This holistic approach to technology and society underscores my role as a leader and change-maker in the realm of data engineering and FinTech.

I actively engage in thought leadership and speaking opportunities, focusing on how inclusive practices can drive innovation in tech. Whether through keynote speeches, panel discussions, or collaborative projects, I aim to forge impactful partnerships that align with my commitment to innovation, DEI, and wellness.

I am eager to collaborate with forward-thinking organizations that share my vision for a more inclusive tech industry. Let’s connect to explore how we can work together to foster meaningful change and drive success through collaborative efforts.

Skills & Expertise:
Technical Skills: Data Engineering, Information Security, Cloud Data Architecture (Azure, AWS, GCP), Kafka, Apache Airflow, Python, SQL
Advocacy & Leadership: DEI Advocacy, Women’s Health, Mental Health, Public Speaking, Mentorship, Technical Project Leadership

Disha Mukherjee

Most people trying to become data engineers spend months watching tutorials but never build anything end to end.

The truth is simple. 
You can’t learn data engineering by memorising tools. You learn it by shipping real pipelines.

If you want to grow fast in this field, pick projects that force you to deal with actual engineering problems: 
Broken data, 
Schema changes, 
Orchestration issues, 
Cloud setups,
Streaming delays, and scaling bottlenecks.

Here are seven project ideas that give you the closest experience to what real teams face every day:

If you complete even three of these, you won’t just understand data engineering. You will have a portfolio that proves you can solve real problems, not just write code.

The fastest way to level up in this field is simple:
Build systems, not tutorials.

Follow Disha Mukherjee for more!

7 Projects to Master Data Engineering

Most engineers learn ACID in theory, 
But its importance becomes clear only when something breaks in production. 

Here is a simple reading-friendly explanation of each property.
1. Atomicity
A transaction is all or nothing. 
No partial writes. 
No half-completed updates. 
If one step fails, the entire transaction is rolled back and the system returns to a safe state.

2. Consistency
Every transaction must move the database from one valid state to another valid state. 
All rules, constraints and relationships must remain intact. 
This is different from the CAP theorem definition. 
Here, consistency is about preserving data integrity at every step.

3. Isolation
Multiple transactions often run at the same time. Isolation ensures they do not interfere with each other. 
The ideal model is serializability, where each transaction behaves as if it were running alone. 
In practice, databases use weaker isolation levels for performance, but the principle remains: data should not be confused about what happened first.

4. Durability
Once a transaction commits, it must remain stored. 
Even if the system crashes or a node fails. 
Modern systems achieve this through persistent logs, replication and stable storage.

Why ACID matters
ACID is not an academic concept. 
It is the reason money does not disappear mid-transfer. 
It is the reason orders are not duplicated. 
It is the reason data systems remain predictable and trustworthy at scale.

Image credit to ByteByteGo

Most people think learning 
Data engineering is about collecting tools.
 SQL. Airflow. Spark. Snowflake.

But the truth is, 
The tools are the easiest part.
What really shapes a great data engineer is how they think.

When I look back at my own journey, 
The biggest breakthroughs didn’t come from adding another technology to my stack. 

They came from understanding how data moves, 
Where it breaks, and why it matters to the business in the first place.

The first mindset shift is context. 
Before touching a line of code, ask why the data exists. 

Is it powering decisions, insights, or machine learning? 
That single question changes how you design everything downstream.

The second is systems thinking. 
Anyone can build an ETL job, but few can visualize how each component connects to deliver trust and consistency end-to-end. 
Think in flows, not fragments.

The third is adaptability. 
Data systems live in motion. 
What works today might not scale tomorrow. 

Design with change in mind, with observability, versioning, and lineage built in from day one.

And finally, communication. 
The most underrated skill in engineering. 
The ability to explain complex systems simply, to translate between technical and business worlds, is what turns good engineers into leaders.

If I were starting out today, 
I wouldn’t start with a list of tools.
I’d start with a mindset.

Because frameworks evolve, but fundamentals compound.
What’s the one mindset that shaped your growth the most as an engineer?

#DataEngineering #TechLeadership #DataEvangelism #EngineeringMindset #WomenInTech #CareerInData

The 4 Mindsets of a Great Data Engineers have.

Today is International Mens Day.
A day that often gets far less conversation than it deserves.

We talk a lot about creating supportive workplaces, stronger communities, and healthier teams. But we rarely talk about the pressures many men carry silently while trying to meet those expectations.

In tech especially, I have seen incredibly talented men struggle quietly with:
 • performance pressure
 • fear of being judged for vulnerability
 • imposter syndrome in fast moving environments
 • the expectation to always appear strong or certain
 • work life imbalance they feel they "should not" talk about

The reality is simple.
Support is not gender specific.
Respect is not gender specific.
And growth definitely is not gender specific.

International Mens Day is a reminder to create spaces where everyone feels comfortable asking for help, admitting they are overwhelmed, or simply being human.

Celebrating the men who show up every day.
Who lead with integrity.
Who mentor, contribute, and hold things together even when it’s not easy.

Let’s build a culture where strength is not defined by silence, but by honesty and collaboration.

Everyone loves to talk about data science.
Few talk about what makes it possible in the first place.

Data science is the headline.
Data engineering is the story beneath it.
One builds models that predict the future.

The other builds systems that make those predictions possible.
I’ve often seen these roles confused or even compared, as if one is “smarter” or “more technical.”

But they serve different purposes, 
And the best teams know how to make them complement each other.
Data scientists explore, experiment, and extract meaning.

They ask why things happen and what might happen next.
Data engineers build reliable, scalable foundations.

They make sure data is accessible, accurate, 
And ready for those experiments to even exist.

In a healthy data ecosystem, science drives curiosity and engineering drives consistency.

You can’t move fast without both.
So instead of asking “Which is better?”
Ask “How do we make them work better together?”

Because great insights need great infrastructure.
And in the end, that’s what turns data from an experiment into impact.

What have you seen, more tension or teamwork between data engineers and data scientists in your org?

Follow Disha Mukherjee for more!

#DataEngineering #DataScience #DataEvangelism #TechLeadership #Analytics #WomenInTech

Data Engineering vs Data Science

Everyone wants to break into Data Engineering.
Few know where to actually start.

When I started, I made the same mistake most beginners do.
I tried to learn everything at once.
Hadoop, Spark, Kafka, Airflow, Snowflake, AWS, Terraform.

By week two, 
I had more browser tabs than actual understanding.

The truth?
You don’t need to learn all the tools.
You need to learn how data moves.

Start small.
Start simple.
Start right.

𝐇𝐞𝐫𝐞’𝐬 𝐚 𝐫𝐨𝐚𝐝𝐦𝐚𝐩 𝐈 𝐰𝐢𝐬𝐡 𝐈 𝐡𝐚𝐝 𝐰𝐡𝐞𝐧 𝐈 𝐛𝐞𝐠𝐚𝐧 👇

↳𝐋𝐞𝐚𝐫𝐧 𝐡𝐨𝐰 𝐝𝐚𝐭𝐚 𝐟𝐥𝐨𝐰𝐬
Understand how data is collected, stored, transformed, and used.
Concepts matter more than tools.
If you can explain a pipeline on a whiteboard, the tools will follow easily.

↳𝐌𝐚𝐬𝐭𝐞𝐫 𝐨𝐧𝐞 𝐥𝐚𝐧𝐠𝐮𝐚𝐠𝐞
Pick Python or SQL.
Don’t chase syntax. Learn how to think in queries and scripts.

↳𝐆𝐞𝐭 𝐡𝐚𝐧𝐝𝐬-𝐨𝐧
You don’t need a full-time job to start building.
Try projects like:
 • Building a mini ETL pipeline
 • Cleaning a dataset and publishing insights
 • Automating a small reporting workflow

↳𝐋𝐞𝐚𝐫𝐧 𝐭𝐨 𝐭𝐞𝐥𝐥 𝐭𝐡𝐞 𝐬𝐭𝐨𝐫𝐲
Data engineers who communicate well grow faster.
Document what you build. Write about it. Teach someone.
That’s how you go from engineer to trusted problem-solver.

↳𝐅𝐨𝐜𝐮𝐬 𝐨𝐧 𝐟𝐮𝐧𝐝𝐚𝐦𝐞𝐧𝐭𝐚𝐥𝐬, 𝐧𝐨𝐭 𝐟𝐫𝐚𝐦𝐞𝐰𝐨𝐫𝐤𝐬
Tech stacks change. 
Foundations don’t.
If you understand architecture, performance, and scalability, you’ll adapt to any tool in months.

You don’t get into data engineering.
You grow into it, one pipeline, one problem, one insight at a time.

If you’re starting your data engineering journey, what’s the one topic or challenge you’re stuck on right now?

Let’s talk in the comments.

Follow Disha Mukherjee fore Data Made Easy!

#DataEngineering #CareerGrowth #DataLearning #ETL #Python #SQL #WomenInTech

Ever opened a Data Engineer interview question sheet and thought-
 “Wait… they’re not just testing my tech skills, they’re testing how I think?”
You’re absolutely right. 

Here’s the truth:
Most data engineering interviews don’t just evaluate whether you can write SQL or build a data pipeline.

 They assess:
→ How you connect business problems to data systems
→ How you make design trade-offs
→ How you communicate your process

I’ve broken down the most common Data Engineer Interview Questions- not just with answers, but with the why behind them.

You’ll find:
 ↳ “General” questions that test your story and clarity
 ↳ “Process” questions that reveal your decision-making
 ↳ “Technical” questions that test depth, not just recall

Some highlights 
→ How to explain your “path to data engineering” without sounding like a
    CV
→ The right way to talk through a project from start to finish
→ What to say when they ask about schemas, unstructured data, and
   Hadoop (without jargon dumping)
→And the golden closer: how to ask them the right questions back.
Whether you’re just preparing for your first interview or aiming for a senior role, this guide will help you sound confident, structured, and human.

I’ve compiled the full breakdown (with examples, frameworks, and prep strategy) into a handy PDF:
 “The Ultimate Data Engineer Interview Prep Guide.”

#DataEngineering #CareerInData #TechInterviews #DataEngineer #SQL #WomenInTech #CareerGrowth

DE Interview Prep Guide.

Every modern data system begins with one simple question:
How will your data get in?

That’s where data ingestion comes in; 
The backbone of every pipeline.

It’s not glamorous, but it’s where scalability, reliability, and cost are decided long before any dashboard comes to life.
Most teams think ingestion is “just about connecting sources.”

In reality, it’s about handling variety, velocity, and volume; all at once.

Some of the tools leading this space:
→ Azure Event Hubs for high-speed event collection and real-time analytics
→ Google Cloud Pub/Sub for asynchronous messaging and streaming ingestion
→ Apache Kafka for large-scale, low-latency event pipelines
→ Apache Flume for distributed log and event collection

Each serves the same purpose; moving data, but each does it differently, depending on your architecture and scale.

Here’s the real challenge:
Picking the right tool is not about brand preference.
It’s about understanding your data flow, latency needs, and consumption pattern.

Because when ingestion goes wrong, everything downstream suffers; from processing to analytics to trust.

If you’re building or optimizing data pipelines right now, which ingestion tool have you found most reliable in production?

Follow Disha Mukherjeefor Data Made Easy!

#DataEngineering #StreamingData #SystemDesign #Kafka #CloudComputing #ETL #DataPipeline

Image Credit Simform.

Good data products don’t start with SQL. 
They start with questions.

I’ve seen teams jump straight into models, schemas, and pipelines because the requirement looked “simple.”

 A new metric.
 A new dashboard.
 A new table.

But every time a data project struggles, the root cause is almost always the same:
No one slowed down to ask the right questions.

Before a single line of SQL is written, great data teams ask:
→What problem are we trying to solve?
 Not “what table should we build,” but “what decision will this enable?”

→ Who will use this?
 A data scientist, a product manager, a finance analyst — each needs a different kind of truth.

→ What does ‘correct’ look like?
 If the business doesn’t know the definition, the data team will guess, and that guess becomes debt.

→ What could go wrong later?
 A quiet discussion today prevents a loud incident tomorrow.

Once the questions are clear, SQL becomes the easy part.
The complexity drops.
The ownership becomes obvious.
The data product actually solves something.
Good data engineering is not about writing queries faster.
It’s about creating clarity before creating tables.

Takeaway:
 If you want better data products, start with alignment. The code follows naturally.

Follow Disha Mukherjee fore data made easy!

I am excited to announce that 
I have become a Dell Pro Max Ambassador.

As someone who has spent more than a decade building data systems and, more recently, working toward full-stack AI development, access to reliable high-performance compute has become a critical part of my workflow. 

Training models, running iterative experiments, and evaluating data-intensive pipelines require both speed and stability at the hardware and software level.

I have received the Dell Pro Max Plus 16, a system designed for professional-grade workloads. It is equipped with the NVIDIA RTX 5000 GPU built on the Blackwell architecture with 24 GB of GPU memory. 

This GPU is part of the NVIDIA AI Platform, which combines accelerated hardware with the NVIDIA AI software stack to support modern AI, simulation, and data workflows end to end.

Along with the system, 
I will be using the Dell UltraSharp Webcam, 
the Dell Pro Plus Compact Keyboard and Mouse (KM7120W), 
and Dell headphones to create detailed, workflow-driven content for the community.

Over the next few weeks, I will share practical, hands-on experiences around:
• Building and optimizing data pipelines
 • Training and fine-tuning models using local accelerated compute
 • Evaluating performance gains and iteration speed
 • Demonstrating how the NVIDIA AI Platform supports real AI development work
 • Offering guidance that other engineers and aspiring practitioners can apply directly

My goal is to produce content that is honest, useful, and grounded in real engineering practice, not theory.

Thank you to Dell Technologies for the opportunity to contribute to the program. 

More updates soon.

Logan Lawler Cindy Olivo Crystal Cook

 #DellProMax #DellTech #NVIDIA

If you want to build a strong career in tech, especially Abroad, 
Focus on three things:
→ Personal branding.
→ Networking.
→ And asking the right questions.

Your personal brand helps people find you.
Your network helps people trust you.
And your questions help people remember you.

Too many skilled engineers stay invisible because they never show what they know.
Too many ambitious professionals connect without context, adding people, not building relationships.

Start small.
Share your perspective on the problems you solve.
Engage with others’ posts thoughtfully.

Reach out with intent- not to ask for a job, but to learn.
One thoughtful conversation can open more doors than ten cold applications.

And one consistent voice online can build more credibility than years of quiet expertise.
Your skills are your foundation.

Your brand and network are your amplifier.

#CareerInTech #DataEngineering #PersonalBrand #Networking #TechLeadership #UKTech #DataEvangelism

The cost of not deleting anything: data debt

Most data teams think their problems come from complex pipelines or slow queries.
But a huge part of the struggle comes from something much simpler.
Not deleting things.

Old tables that no one owns.
Metrics created for projects that ended long back.
Pipelines that still run even though no one knows why.
Dashboards that have not been opened in months.

All of this becomes data debt.
And data debt grows quietly.

You feel it during migrations.
Every change takes longer because you need to check what might break.
You feel it during outages.
Debugging becomes harder because no one knows which assets matter.
You feel it during onboarding.
New engineers spend more time decoding history than creating impact.

Most data debt starts with good intentions.
A quick table.
A temporary metric.
A one time pipeline.
The team moves fast.
But nothing gets retired.

The fix is not a one time cleanup.
The fix is a habit.
Review. Remove. Archive. Repeat.

If your data stack feels slow, cluttered, or risky, you may not need more tools.
You may need fewer unused assets.

Delete what no longer serves the work.
Your future team will thank you.

Most people prepare for Data Engineering interviews 
By memorizing answers.
But interviews aren’t about reciting definitions.

They’re about showing how you think, how you solve problems, and how well you understand the flow of data.

If you’re preparing for DE roles in 2025, 
There are a few patterns you’ll notice.

Companies want engineers who can design systems, 
Reason about trade-offs, and communicate clearly.

Tools matter, 
But clarity matters more.

I spent some time reviewing the most common interview questions asked across companies.

The themes are consistent: data modeling, pipeline design, orchestration, Python, SQL, scaling decisions, and how you collaborate across teams.

 And almost every interview includes one deeper question:
 “Walk me through a data project you designed end to end.”
Most candidates fail here because they talk about tools instead of the problem they solved.

If you want to stand out, focus on three things:
→Show how you think.
→Break down a problem. 
→Explain the constraints. 
→Share why you chose one approach over another.
→Show how data moves.

Interviewers want to see how you design reliable pipelines, ensure data quality, and handle failure scenarios.

Show your communication muscles.
A lot of data engineering is working with analysts, PMs, ML teams and explaining trade-offs in plain English.

To make preparation easier, Use interview prep guide covering the most asked DE questions:
HR screen, technical deep dives, SQL, Python, orchestration, big data, and system design.

What interview topic do you find the most challenging right now? 
Let’s discuss in the comments.

Follow Disha Mukherjee for Data Made Easy!

Data Engineering Interview Prep.

When I first started using dbt Labs, it felt like just another transformation tool.
Write SQL, run models, build tables- simple enough.

But over time, I realized something deeper.
dbt wasn’t just about transformations.
It was teaching data teams how to think like product engineers.

Here’s how:
1. 𝐕𝐞𝐫𝐬𝐢𝐨𝐧 𝐂𝐨𝐧𝐭𝐫𝐨𝐥 𝐚𝐬 𝐚 𝐌𝐢𝐧𝐝𝐬𝐞𝐭
 Every model lives in Git. Every change is reviewed.
 That single shift turns data from a “shared folder of tables” into a product
 with lifecycle management.

2. 𝐓𝐞𝐬𝐭𝐢𝐧𝐠 𝐚𝐬 𝐚 𝐅𝐞𝐚𝐭𝐮𝐫𝐞, 𝐍𝐨𝐭 𝐚𝐧 𝐀𝐟𝐭𝐞𝐫𝐭𝐡𝐨𝐮𝐠𝐡𝐭
 In dbt, tests aren’t optional.
 They’re part of the build.
 Just like a product, your data ships only if it passes quality gates.

3. 𝐃𝐨𝐜𝐮𝐦𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 𝐚𝐬 𝐚 𝐅𝐢𝐫𝐬𝐭-𝐂𝐥𝐚𝐬𝐬 𝐂𝐢𝐭𝐢𝐳𝐞𝐧
 Auto-generated docs may sound simple, but they redefine ownership.
 When every model is documented in context, data stops being tribal 
 knowledge.

4. 𝐓𝐫𝐚𝐧𝐬𝐩𝐚𝐫𝐞𝐧𝐜𝐲 𝐚𝐬 𝐭𝐡𝐞 𝐃𝐞𝐟𝐚𝐮𝐥𝐭
 With lineage, every dependency is visible.
 Suddenly, consumers can see how data is built, not just trust that it works.

This is what “data as a product” looks like in action:
↳Ownership
↳Quality
↳Discoverability
↳Feedback

Tools like dbt Labs don’t just change how we engineer data.
They change how we think about data.
And that mindset shift is what modern data teams are built on.

Takeaway:
The best data tools don’t just solve technical problems, they shape better engineering behavior.

#DataEngineering #dbt #DataAsAProduct #ModernDataStack #DataCulture #DataEvangelism

When a dashboard breaks, the first question people ask is,
 “Who wrote this?”

But the better question is,
 “Who owns this?”

I’ve seen technically brilliant data projects collapse, 
not because the pipelines were fragile,
but because no one owned the truth behind the data.

Ownership in data isn’t about hierarchy.
It’s about accountability.
↳Who decides when a table is deprecated?
↳Who defines what “active user” means?
↳Who ensures the data still makes sense a month later?

When everyone assumes “someone else” owns it, the data slowly loses trust.
And once trust is gone, even perfect code can’t save the project.

The best data teams I’ve worked with treat ownership like a design principle
clearly defined, versioned, and communicated.

Because good data doesn’t just flow through systems.
It flows through people who take responsibility for it.

𝐓𝐚𝐤𝐞𝐚𝐰𝐚𝐲:
You can’t automate ownership.
You have to design it into your data culture.


Follow Disha Mukherjee for Data Made Easy!

#DataEngineering #DataLeadership #DataGovernance #DataCulture #ModernDataStack #DataEvangelism

For years, I thought the only way to stand out in tech was to be louder, tougher, faster.
Turns out, the real power was in being clearer.

When I started in data engineering, 
I walked into rooms where I was often the only woman.

And in those moments, I used to think-
 “Maybe if I just work harder, they’ll see I belong here.”
So I built.
I delivered.
I stayed quiet and let my work speak for me.

But the truth?
Work doesn’t always speak.
People do.

It took me years to realize that being good at what you do isn’t enough.
You also need to be visible for what you stand for.

That shift changed everything.
I stopped trying to fit the “serious engineer” mold.
I started sharing what I learned- the wins, the mistakes, the messy middle.

And something incredible happened:
More women reached out.
More voices joined the conversation.
More teams started listening.

Because representation isn’t about perfection.
It’s about presence.

Today, I still build scalable systems and write production-grade code 
but I also make space for others to step in, speak up, and be seen.

Because the goal isn’t to be the only woman in the room.
It’s to make sure you’re not the last.

We don’t need more women proving they can code.
We need more women leading the conversation about why they do.

If you’re a woman in tech- what’s one thing you’d tell your younger self just starting out?

Or, if you lead a team, what's one way you create space for more voices at the table?

Follow Disha Mukherjee for more.

#WomenInTech #DataEngineering #Leadership #TechCommunity #DiversityInTech

Most businesses don’t collapse because of bad ideas.
They collapse because they stop paying attention to what their data is trying to say.

Every team leaves behind traces user clicks, transactions, delays, cancellations, costs. Together, they paint a story. But most leaders only look at that story when things start breaking.

By then, it’s usually too late.
Data isn’t just a report to read. It’s feedback in real time. It’s how your business tells you what’s healthy, what’s at risk, and what needs attention.

Think of it like your own health. Your body gives you signals long before something goes wrong. The same way, your business gives you signals long before revenue dips or customers churn.

The real problem isn’t a lack of data.
 It’s the habit of ignoring it until it hurts.
The companies that win are the ones that listen early and act fast.

Because in data and in business, precaution is always better than cure.
How to stay ahead:
→Build observability into your data systems not as an afterthought.
→Set early warning metrics for business health, not just performance dashboards.
→Make data reviews a part of decision-making, not a post-mortem activity.
→Create a culture where teams talk about what the data means, not just what it shows.

Your data always knows before you do. 
The question is are you listening?

Follow Disha Mukherjee for more! 

#DataEngineering #DataDrivenCulture #DataStrategy #Analytics #Leadership #DataObservability

Most people assume everything in AI has to run in the cloud.
But anyone who has worked with large datasets knows the cloud is only part of the workflow.
When we are preparing data for AI systems, the work is rarely linear.

You iterate.
You validate.
You experiment.

And every cycle depends on how quickly your environment responds.
Over the past decade in data engineering, I’ve seen teams move from on-prem clusters to cloud-native ecosystems. 

Both have their strengths, but neither eliminates the need for fast, predictable local compute. 
Especially when we are engineering features, exploring data at scale, or testing preprocessing strategies that directly influence model behaviour.

Cloud supports scale.
Local systems support understanding.

When we are preparing large datasets, a predictable environment becomes critical. 
You need to run transformations repeatedly and reliably. 
You need consistency when debugging subtle issues. 
And you need the freedom to iterate without waiting for resources or watching budgets fluctuate.

The Dell Pro Max Plus 16, equipped with the NVIDIA RTX PRO 5000 Blackwell GPU with 24GB of VRAM, brings that consistency back. 
It allows me to run preprocessing pipelines, feature engineering workloads, and training experiments without delays, queues, or variable performance.
For AI teams, iteration speed shapes outcomes.

The faster we can prepare, test, and refine data, the better our models become.
Quality in AI starts with the quality and reliability of the data workflow.
Local accelerated compute removes friction.

It keeps the entire process grounded and controlled.
And it allows deeper technical work to happen without interruption.

In the next posts, I will break down how this system supports each part of the data-to-model lifecycle, and where it makes the biggest difference.

What part of your workflow slows down your iteration speed the most?

Logan Lawler Cindy Olivo Crystal Cook

#DellProMax #DellTech #NVIDIA

taplio

The Battle

Pooja Jain

Disha Mukherjee

Followers

Posts

Engagements

Likes

Performance ScorePooja Jain's LinkedIn Performance

Performance ScoreDisha Mukherjee's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Pooja Jain

Disha Mukherjee

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Pooja Jain's LinkedIn Performance

Performance ScoreDisha Mukherjee's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScorePooja Jain's LinkedIn Performance