David Regalado

AI Time Journal

AI Time Journal is a publishing platform that strives to aware users of and around the world of Artificial Intelligence, growing technology trends, and other industry news. Its final aim is to contribute to steering the development of AI for the benefit of humanity.

Ambassador at AI Time Journal

Data Engineering Latam

Data Engineering Latam is the largest, fastest growing, and coolest data community in Latin America whose mission is to promote the region's talent through the dissemination of talks, workshops, study groups, collaborative help and the creation of relevant content.

Founder

Stealth Startup

I spearhead the engineering and infrastructure area, leading the design, development, and maintenance of advanced data processing systems. My focus is on ensuring the robustness, scalability, maintainability, security, and efficiency of technological solutions, anticipating future challenges to address them proactively.

I foster a culture of collaboration and technical excellence among the data science and engineering teams, promoting best practices that drive innovation and a positive work environment.

I report directly to the company founders, explaining the work we do in simple terms.

Key achievements:

- I reduced costs by 90% by moving workloads to a combination of Dataform and Dataflow jobs.
- I introduced modern techniques such as RAG and semantic search to the company.
- Designed and implemented a cloud architecture tailored to the startup's needs.
- Parameterized the product code, enabling efficient customization for multiple clients.
- Spearheaded the adoption of key practices such as code versioning, infrastructure as code (IaC), and disaster recovery procedures, strengthening the organization's resilience and technological responsiveness.
- I delivered a custom industry data model tailored for retailers.

VP of Engineering

Nebula.io

Nebula is a well funded startup that develops cutting edge technology that combines proprietary algorithms with deep learning neural networks, natural language processing, and generative AI to connect talent with opportunity. 

I leveraged Apache Beam on Google Cloud Dataflow for building cost effective streaming and batch data pipelines. Other technologies include MongoDB, Elasticsearch, PubSub, BigQuery, and other GCP services.

Senior Data Engineer

University of Chicago

The University of Chicago is a top 10 U.S. University, with more than 100 associated Nobel Prize winners. 

I taught the Python for Data Science program. Some topics covered: data manipulation and analysis, training, evaluation and deployment of Machine Learning (ML) models. 

4 cohorts.

Data Scientist - Teaching Assistant

Anomali

Anomali is the leading cybersecurity company that develops and provides intelligence-driven cybersecurity solutions to enhance threat visibility, automate threat processing and detection, and accelerate threat remediation. 

I built data pipelines using Scala to serve high concurrency requests that enrich data about suspicious traffic sources in order to classify them as trustworthy or not.

Clevertech

Projects:

• MadHive – Responsible for the implementation of pipelines involving Google Cloud Platform to retrieve terabyte-scale data in sub-second response time. This enabled data scientist to be more efficient when doing exploratory data analysis with data across the country (U.S.).

 • DevOps — I spearheaded the implementation of modules in Terraform for the use of infrastructure as code on the GCP cloud.

In charge of a team of 2.

Senior GCP Data Engineer

Scotiabank

Projects:

• I spearheaded the Scotiabank's Data Hub. I did the architecture and design. Some features: ETL Automation, Data Observability, alerts, monitoring. In charge of a team of 5.

Data Architect

Indra

Projects 

• Responsible for Big Data projects for the IT area. of Telefónica del Perú. In charge of a team of 5.
• Responsible for the B.I. Factory for the I.T. area. of Telefónica del Perú. In charge of a team of 20.

Data Engineer Tech Lead

DWConsulware 

• Construction of campaign variables, risks, financial products, etc. to be used by the analytical models of data scientists.
• Preparation of reports in Excel with macros
• Monitoring and resolution of incidents in the production network

Data Engineer

Projects:	

●	Implementation of the Interbank Data Warehouse
●	Implementation of the Universidad Peruana de Ciencias Aplicadas Data Warehouse
●	Implementation of the Sura Data Warehouse

Senior Business Intelligence Consultant

Hi there!

Thank you for visiting my profile. I am passionate about data & AI. Throughout my 15 years of professional experience I have seen projects of various types, sizes and industries. 

Summary

In my country, I've worked in the biggest companies like:

- Financial Services: Scotiabank, Interbank
- Telecommunications: Telefónica del Perú, Claro
- Consulting: Teradata Perú, Indra, Encora
- Consumer Credit Reporting Agencies: Equifax
- Ad-Tech: Manya.pe

In those companies, I did the whole data lifecycle: data extraction, ingestion, data manipulation, data modeling (normalization, denormalization, etc), data analysis, data quality assessments. In short, the ETL part (data pipelines) + data analysis. 

Achievements: The Data Warehouses/Lakes and dashboards were delivered to my clients, so they could start making data-driven decisions.

These recent years I've been working for U.S. clients as a contractor. Specifically:

- Academics: University of Chicago
- Cyber Security: Anomali
- Ad-Tech: Clevertech
- Startup: Nebula.io

Achievements: Data pipelines delivered and students gratuated from the University program.

Highlight

- Founder at Data Engineering Latam, the largest, fastest-growing data community in Latin America: https://www.linkedin.com/company/data-engineering-latam
- I taught Python for Data Science in the University of Chicago.
- Recognized as Champion Innovator by Google Cloud
- I led teams of 20 people in 3 different occasions.
- Recurrent speaker and jury in Big Data and Cloud Computing events.

Studies

- Systems Engineer from the Technological University of Peru (UTP), tenth superior.
- With a specialization in Data Science from Johns Hopkins University.
- With a specialization in Data Warehousing for Business Intelligence from Colorado University.
- With a specialization in Data Science and Big Data Analytics from MIT.
- With 4 certifications, 15 specializations and 46 Google Cloud Platform digital badges.
- With studies in Artificial Intelligence at UBA-IALAB

Tech stack

Cloud Computing: Google Cloud (GCP), AWS, Azure
Data Engineering: Apache Beam, Airflow, Spark, Databricks, Snowflake, dbt, Hadoop, Hive, Luigi
Databases: BigQuery, Teradata, MongoDB, Elasticsearch, Oracle, SQL Server, PostgreSQL, Mysql
Data Visualization: Power BI, Tableau, Looker, Data Studio
Programming languages: Python, Scala, R
DevOps & CI/CD: Terraform, Docker, Kubernetes, Git, GitHub

Roles 

CTO, CDO, CDAO, Data Science Manager, Data Engineering Manager, Head of Data, Tech Leader, Data Architect, Data Engineer, Data Scientist, BI Consultant.

This was four years ago. Young David studied as if there were no tomorrow.

At the time, I shared this achievement along with this message:

"If you hear a voice within you saying 'you cannot paint,' then paint, and that voice will be silenced."

—Van Gogh

It remains relevant.

𝐖𝐡𝐚𝐭 𝐢𝐬 𝐚𝐧𝐭𝐡𝐨𝐬?

Anthos is a hybrid and multi-cloud application platform from Google Cloud that allows you to modernize and manage applications across different environments, including on-premises servers and multiple public clouds like AWS and Azure. It provides a consistent development and operations model by extending Google Cloud's services and engineering practices to these diverse infrastructures, enabling consistent security, governance, and management. Built on Kubernetes, Anthos helps organizations build new apps, modernize existing ones, and run them securely anywhere. 

𝐊𝐞𝐲 𝐟𝐞𝐚𝐭𝐮𝐫𝐞𝐬 𝐚𝐧𝐝 𝐜𝐚𝐩𝐚𝐛𝐢𝐥𝐢𝐭𝐢𝐞𝐬

✅ Hybrid and multi-cloud consistency: Provides a unified platform for managing applications across different environments, reducing inconsistencies.
✅ Application modernization: Helps you modernize existing applications and build new cloud-native ones more quickly.
✅ Security and governance: Offers robust security features and allows for the implementation of security guardrails and policy enforcement across all your environments.
✅ Unified management: Enables you to view and manage multiple Kubernetes clusters from different locations through a single interface.
✅ Streamlined development: Supports consistent development and operations models, which can speed up deployment and improve quality through features like centralized logging and tracing.
✅ Infrastructure and service management: Provides tools for provisioning infrastructure, managing containers with Kubernetes, and managing the communication between services. 

--
☁️🧑‍💻

👍 Like, 🔗 share, 💬 comment, 👉 follow

I post about Data & AI. I also share thoughts and resources along the way. Follow David Regalado and smash the 🛎 in my profile if you are interested in these topics. Let's connect on other social as well:

Medium: https://lnkd.in/eNhy6CBB
YouTube: https://lnkd.in/d3Kc6PMc
TikTok: https://lnkd.in/d8RvZ6p5
Threads: https://lnkd.in/e285fJdt
Topmate: https://lnkd.in/eQjH3RWP 
Sessionize: https://lnkd.in/dipdM7Zs
X: https://lnkd.in/ezjm873T
Instagram: https://lnkd.in/eejcaqzE

💸📈Unlocking Business Potential with Data & Generative AI ╏ Startup Advisor ╏ Mentor Featured on Times Square ╏ International Speaker ╏ Google Developer Expert

3d isometric version of Lima!!!

(credits: Lesly Zerna & r/aicuriosity for the prompt idea!)

Try Image Prompt 👇

Present a clear, 45° top-down isometric miniature 3D cartoon scene of [CITY], featuring its most iconic landmarks and architectural elements. Use soft, refined textures with realistic PBR materials and gentle, lifelike lighting and shadows. Integrate the current weather conditions directly into the city environment to create an immersive atmospheric mood. Use a clean, minimalistic composition with a soft, solid-colored background. At the top-center, place the title “[CITY]” in large bold text, a prominent weather icon beneath it, then the date (small text) and temperature (medium text). All text must be centered with consistent spacing, and may subtly overlap the tops of the buildings. Square 1080x1080 dimension.

--
🇵🇪🧑‍💻

I post about Data & AI. I also share thoughts and resources along the way. Follow David Regalado and smash the 🛎 in my profile if you are interested in these topics.

👍 Like, 🔗 share, 💬 comment, 👉 follow

David Regalado @thecodemancer_ "Google Gemini Nano Banana Pro" Lima Peru

🤯 This is really an improvement!

I wrote about an earlier version of this image generation model here:
https://lnkd.in/etub9QZ2

Now I'm looking forward writing about this new one.

--
🧑‍💻

👍 Like, 🔗 share, 💬 comment, 👉 follow

I post about Data & AI. I also share thoughts and resources along the way. Follow David Regalado and smash the 🛎 in my profile if you are interested in these topics. Let's connect on other social as well:

Medium: https://lnkd.in/eNhy6CBB
YouTube: https://lnkd.in/d3Kc6PMc
TikTok: https://lnkd.in/d8RvZ6p5
Threads: https://lnkd.in/e285fJdt
Topmate: https://lnkd.in/eQjH3RWP 
Sessionize: https://lnkd.in/dipdM7Zs
X: https://lnkd.in/ezjm873T
Instagram: https://lnkd.in/eejcaqzE

There is a quiet part of data engineering that we rarely say out loud: The more maintenance a data architecture needs, the more it favors the external firm that built it.

👇

I've seen too many "Vendor Agnostic" architectures that are really just "Job Security" architectures. They really are Rube Goldberg machines.

I wrote my latest piece, "The Art of KISSing in Data Architecture," to push back against this entropy.

It's about the DIY Trap: Why serverless usually beats "managing it yourself" (unless you love waking up at 3 AM).

I break down the principles (and the traps) here:

https://lnkd.in/eiJQkHiv

💬 Do you trust your vendors? How can you make sure they're doing what's best for your business rather than their own? Share your thoughts in the comments.

--
🧑‍💻

👍 Like, 🔗 share, 💬 comment, 👉 follow

I post about Data & AI. I also share thoughts and resources along the way. Follow David Regalado and smash the 🛎 in my profile if you are interested in these topics. Let's connect on other social as well:

Medium: https://lnkd.in/eNhy6CBB
YouTube: https://lnkd.in/d3Kc6PMc
TikTok: https://lnkd.in/d8RvZ6p5
Threads: https://lnkd.in/e285fJdt
Topmate: https://lnkd.in/eQjH3RWP 
Sessionize: https://lnkd.in/dipdM7Zs
X: https://lnkd.in/ezjm873T
Instagram: https://lnkd.in/eejcaqzE

Young David treated his homework like a quiet way to beat yesterday's version of him. This was four years ago. At the time, I shared this achievement along with this message:

"🛫 Fly high, not so the world can see you, but so you can see the whole world! 🌍"

𝐀𝐧𝐭𝐡𝐨𝐬 𝐰𝐡𝐚𝐭?

Anthos functions as a management platform that extends Google Cloud services to run applications consistently across Google Cloud, AWS, Azure, and on-premises data centers, providing a unified management and operational experience. Anthos is explicitly designed as a strong fit for multicloud environments.

𝐊𝐞𝐲 𝐑𝐞𝐚𝐬𝐨𝐧𝐬 𝐀𝐧𝐭𝐡𝐨𝐬 𝐢𝐬 𝐚 𝐆𝐨𝐨𝐝 𝐅𝐢𝐭 𝐟𝐨𝐫 𝐌𝐮𝐥𝐭𝐢𝐜𝐥𝐨𝐮𝐝

✅ Unified Management Console: Anthos provides a "single pane of glass" in the Google Cloud Console to view, monitor, and manage all your Kubernetes clusters and workloads, regardless of where they are deployed (e.g., AWS, Azure, or on-prem). This significantly reduces operational complexity that typically comes with managing disparate cloud platforms.

✅ Operational Consistency: By standardizing on open-source technologies, primarily Kubernetes (GKE), Istio, and GitOps, Anthos provides a consistent application platform and developer experience across all environments. This means application teams can build and deploy applications once ("build once, run anywhere") without worrying about platform-specific details.

✅ Centralized Security and Policy Enforcement: Security policies and configurations (like role-based access controls and resource quotas) can be automated and enforced uniformly across all clusters using Anthos Config Management. This is crucial for maintaining compliance and a strong security posture in a fragmented multi-cloud landscape.

✅ Avoids Vendor Lock-in: Anthos is built on open standards, which gives organizations the flexibility to choose where to deploy their applications based on business needs (e.g., cost, performance, or data residency requirements) and move workloads between platforms if necessary.

✅ Application Modernization: It helps organizations modernize legacy applications by offering tools like Anthos Migrate, which can automatically transform VM-based workloads into containerized applications that can then run on any supported environment.

✅ Service Mesh for Cross-Cloud Communication: Anthos Service Mesh (based on Istio) helps manage, observe, and secure communication between microservices across different clouds, enabling robust, resilient distributed applications. 

𝐂𝐨𝐧𝐬𝐢𝐝𝐞𝐫𝐚𝐭𝐢𝐨𝐧𝐬

Anthos is designed specifically to address the core challenges of managing multi-cloud environments, making it a very strong and suitable solution for organizations seeking unified operations, security, and flexibility across their cloud landscape. 

Anthos involves a learning curve and significant enterprise investment, so it's typically best suited for large organizations with complex, distributed environments.

We tend to overcomplicate data stacks for the wrong reasons:

❌ Because it looks good on a resume.
❌ Because a consultant sold a "bespoke" 6-month project.
❌ Because we are afraid of "Vendor Lock-in" so we create "Complexity Lock-in" instead.

I just published "KISSing the Data Architecture" to explore why the simplest solution is usually the most robust one.

I dive into a specific architectural showdown: The Rube Goldberg Machine vs. The Modern Semantic Layer. One requires a team of engineers to keep the lights on; the other just runs.

The goal isn't to be clever. The goal is to build systems that don't give you a nervous breakdown.

Check it out below 👇

https://lnkd.in/eyPsuHcA

--
🧑‍💻

👍 Like, 🔗 share, 💬 comment, 👉 follow

I post about Data & AI. I also share thoughts and resources along the way. Follow David Regalado and smash the 🛎 in my profile if you are interested in these topics. Let's connect on other social as well:

Medium: https://lnkd.in/eNhy6CBB
YouTube: https://lnkd.in/d3Kc6PMc
TikTok: https://lnkd.in/d8RvZ6p5
Threads: https://lnkd.in/e285fJdt
Topmate: https://lnkd.in/eQjH3RWP 
Sessionize: https://lnkd.in/dipdM7Zs
X: https://lnkd.in/ezjm873T
Instagram: https://lnkd.in/eejcaqzE

David Regalado KISSing the Data Architecture @thecodemancer_

Wavicle Data Solutions

Lead Data Engineer

LinkedIn

LinkedIn Learning Instructor

Globant

CRISIL Limited

Senior Analyst - Data Engineer

Quantiphi

Tata Consultancy Services

System Engineer

Assistant System Engineer

Assistant System Engineer - Trainee

InsideSherpa

KPMG Data Analytics Consulting Virtual Internship

Monisha Enterprises

•	Data mining and Extraction Transformation Loading using R programming.
•	Data cleaning and processing for Data analysis done using Python using NumPy, pandas, SciPy, Scikit. Data modelling and prediction was done using algorithms – Logistic Regression, Time Series Forecasting, Clustering and Classification.
•	Data Visualization done using Tableau for generating reports to enhance the productivity of the products.
•	Built a Data Warehouse on MSSQL Server for ERP System of the enterprise to organize the data for future processing.

Data Analyst Intern

Open to Collaborate | Storyteller | Lead Data Engineer@Wavicle| Linkedin Top Voice 2024 | Globant | Linkedin Learning Instructor | 2xGCP & AWS Certified | LICAP'2022

Data Engineer with 7+ years of experience working on data engineering, analysis, and modelling projects for various domains, such as banking, healthcare, sales, and e-commerce. I leverage my strong statistical and analytical ability to explore business ideas and provide innovative solutions. 

Skilled with technologies and strong intellect with enthusiasm as an avid learner with challenging opportunities. I’m working as a LinkedIn Learning Instructor to bring change as a data evangelist.

Post Graduated in Master of Technology in Data Science and Engineering from Birla Institute of Technology and Science, Pilani in 2023, where I learned advanced concepts and techniques in data science and engineering, machine learning, cloud computing, and visualization. 

I hold multiple certifications in GCP, AWS, and Azure, and I enjoy working with cloud technologies and big data platforms.

I am passionate about data science and women empowerment, and I have been recognized as the 
- Linkedin Top Voice 2024 
- Data Science and Women Empowerment Thought Leader by Global AI Hub. 

I am also a proud participant of the LinkedIn Creator Accelerator Program 2022, where I share my insights and experiences with the data science community. 

I am open to collaborate and share ideas, or explore various opportunities in the data field.

Pooja Jain

73% of data pipeline failures happen because of dependency issues, not bad code.
Cron jobs don't care if yesterday's data never arrived. They'll happily run on empty tables and produce garbage.

Airflow? It checks, retries, alerts, and refuses to cascade your failure downstream.

→ Beyond scheduling Airflow gives an important feature of 𝗗𝗔𝗚𝘀(𝗗𝗶𝗿𝗲𝗰𝘁 𝗔𝗰𝘆𝗰𝗹𝗶𝗰 𝗚𝗿𝗮𝗽𝗵𝘀), that helps to monitor, retry and gives flexibility to organize pipelines with minimum/less code.

🤔 Data Engineers often misinterpret Airflow just as an orchestration tool.

Apache Airflow is an open-source platform designed to programmatically author, schedule, retry and monitor workflows. 

🎼Think of it as the "conductor of an orchestra", where each musician (your data tasks, like data extraction, transformation, loading, or model training) knows when to play, what to play, and what happens if they make a mistake.

Most data engineers automate the wrong things at the wrong time.

Here's the framework I use after 8 years of building production systems:
✅ AUTOMATE WHEN: 
→ Task runs daily/weekly 
→ Human errors cause outages 
→ Work blocks other priorities
→ Team growth = more manual work
Examples: Reports, schema checks, alerts

❌ DON'T AUTOMATE WHEN: 
→ Task happens quarterly 
→ Requirements change weekly 
→ Process isn't understood yet 
→ Manual steps reveal insights

My rule: If it’s done 3+ times, script it; 10+ times, automate it; fails 5+ times, redesign it.
Automate what matters, when it matters—not everything!

Airflow isn’t just about running tasks.

🕑 It’s about building trustworthy, scalable, and observable pipelines that run reliably at 2am — even when you’re not watching.

Airflow gives structure to your chaos.
DAGs give clarity to your pipelines. 
That’s why DEs love them ❤️

Real Example: 
Instead of: 
❌ Cron jobs that fail silently 
❌ Manual dependency management
❌ No visibility into failures
You get: 
✅ Visual workflow monitoring 
✅ Automatic failure notifications
✅ Smart task scheduling 
✅ Easy debugging & restarting

Here's an amazing Airflow cheatsheet with some fundamentals concepts and interview questions to prepare for data engineers. 👇

Carousel Credits -  Futurense.com

Willing to learn more, here's the link to Astronomer free learning path - https://lnkd.in/g8-EWBda

Also, level up with Marc Lamberti's Udemy course on Airflow 3 - https://lnkd.in/gwArwaME

taplio

The Battle

David Regalado

Pooja Jain

Followers

Posts

Engagements

Likes

Performance ScoreDavid Regalado's LinkedIn Performance

Performance ScorePooja Jain's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

David Regalado

Pooja Jain

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}David Regalado's LinkedIn Performance

Performance ScorePooja Jain's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreDavid Regalado's LinkedIn Performance