Onyx Data

Built and scaled a Microsoft Data & AI consultancy serving enterprise and public sector clients across the UK, UAE, and US.

Highlights:
Microsoft Fabric Featured Partner & Microsoft MVP (5x)
Delivered Azure AI-powered solutions across finance, education, and government
Launched FabOps – a Microsoft co-sell solution for Fabric observability
Enabled 3 enterprise clients to scale Fabric & Copilot across departments
Regularly featured at Microsoft Build, FabCon, and global MVP events

Founder / Chief Executive Officer

Forbes Technology Council

Thought Leader

LinkedIn

As a LinkedIn Learning Instructor specialising in Data and Artificial Intelligence (AI), I am dedicated to creating and delivering high-quality educational content that empowers learners to advance their careers in data and AI. My role involves developing comprehensive courses, engaging with the learner community, and continuously updating course materials to reflect the latest industry trends.

Key Achievements:

- Expert Content Development: Created and delivered high-impact courses that received positive feedback and high engagement from learners.

- Curriculum Innovation: Developed innovative and comprehensive curriculam that facilitated structured learning paths for students, enhancing their understanding and application of data and AI concepts.

- Effective Knowledge Transfer: Simplified complex technical concepts, making them accessible and engaging for a diverse learner base, resulting in improved learner outcomes.

- Community Building: Fostered an active and supportive learning community through regular engagement in user groups, helping learners overcome challenges and succeed in their educational journeys.

- Continual Improvement: Actively sought and integrated learner feedback, leading to continuous improvement of course content and instructional methods.

LinkedIn Learning Instructor

The AI Journal

Contributor

AI4C

The key aim of the AI4C is to help charities embarking on AI/ML deployments with pro bono insights/advice - all in a confidential, collaborative sales-free, vendor neutral environment. 

Over 30 charities have joined and gained value with their strategic thinking, user stories, design, deployment and operations. 

AI 4 Good / Tech 4 Good 

If you are a charity leader looking to get the most for the people you serve, feel free to contact the AI4C Chair Paul Excell paul.excell@member.wcit.org.uk

Member

Global AI Hub

Brainz Magazine Official

Executive Contributor

Pomerol Partners

Partner

Reassured

Head Of Data And Insight (Freelance)

Redkite

Principal Power BI Consultant (Freelance) 

Microsoft MVP | AI & Fabric Specialist for Business Transformation | Helping Enterprises Unlock ROI with Azure, Copilot & Power BI | Forbes Tech | Keynote Speaker | CEO, Onyx Data

Your data shouldn't just inform decisions, it should transform your business.

I’m Leon Gordon a Microsoft MVP, keynote speaker, and founder of Onyx Data, where we help enterprises turn Microsoft Fabric, Azure AI, Power BI and Copilot into measurable ROI.

I’ve spent the past 13 years leading enterprise data and AI transformations. From being a hands-on technologist to now heading up a global team, I’ve seen how the right data strategy, aligned with business goals, drives massive ROI.

Clients turn to Onyx Data when they need:
• End-to-end Microsoft Fabric & Copilot implementations
• AI strategy and deployment with Azure OpenAI
• Power BI governance, optimisation & adoption frameworks
• Executive training to build data culture from the top down

Real-world outcomes we’ve delivered:
• £3M in new revenue from insight-driven selling
• 187.1% reduction in data latency
• 35% increase in data accuracy
• 50%+ boost in data literacy
• Featured by Microsoft at Build, FabCon, and global case studies

Recognitions:
• 5x Microsoft MVP
• Microsoft Data & AI Solutions Partner
• Microsoft Digital & App Innovation Partner
• Microsoft Fabric & Real-Time Intelligence Featured Partner
• Microsoft Training Services Partner
• LinkedIn Top Voice
• Founder of #DataDNA – global monthly analytics challenge

Featured by Microsoft at Build, FabCon, and global case studies. Speaker at top industry events across the UK, US, and Middle East.

Latest launch - FabOps – our co-sell-ready Fabric observability solution designed for enterprise-scale efficiency and impact → https://fabops.ai

👉 Ready to move beyond insights and into action?
→ Book a strategy call → https://bit.ly/46MgPHM
→ Try FabOps free → www.fabops.ai
→ Join 28,000+ global pros in #DataDNA → https://www.linkedin.com/groups/9075495/
→ Subscribe to my weekly AI and Data insights → https://www.linkedin.com/newsletters/ai-and-data-tech-trends-6933659594284023808?lipi=urn%3Ali%3Apage%3Ad_flagship3_profile_view_base%3BxNISbSHFTkGWI32ogCOB4Q%3D%3D

Leon Gordon

For years, our data architecture looked like a complex web of disconnected services. We spent more time moving data than actually using it. 

This "engineering teardown" of Microsoft Fabric brilliantly visualises the end of that era.

Look at the foundation (Block #1). That’s OneLake.

Notice how the different compute engines, Data Engineering (#2), Data Science (#3), Warehousing (#4), and Real-Time (#5), aren't siloed in separate buildings? They are distinct modules sitting on the same foundation.

No copying. No movement. Just different lenses on the same data, governed by Purview (#8) and served up via Power BI (#7).

This isn't just a platform update, it's a fundamental shift in how we build data estates.

Is your organisation ready for the OneLake approach, or are you still managing silos?

#MicrosoftFabric #DataEngineering #OneLake #DataArchitecture #Synapse #PowerBI

Turn Microsoft Fabric, Copilot & Azure AI into safe, real‑time business value in weeks, not months | Data & AI Transformation Leader @ Onyx Data | Microsoft MVP | Global Speaker | Forbes Tech Council

No alternative text description for this image

The client: "Can you just quickly throw together a report by EOD?"

The reality beneath the surface: 🛠️⚙️👇

It’s easy to forget that a seamless Power BI report isn't just drag-and-drop magic. It's a finely tuned engine of interconnected components.

This "engineering teardown" perfectly visualises the ecosystem we build daily.

1️⃣ It starts with raw materials (Connectors)
2️⃣ Gets processed in the factory (Power Query)
3️⃣ Is structured into a robust foundation (Data Model)
4️⃣ And finally, shipped out for consumption (Service, Mobile, Embedded)

If your "quick report" is slow, chances are the engine block (The Data Model) needs a tune-up.

Which part of this machine do you spend the most time wrenching on? Let me know in the comments. 👇

#PowerBI #DataEngineering #Analytics #Microsoft #BusinessIntelligence #DataViz

Hardcoding Secrets? Stop It. 🛑

It is the #1 rule of secure data engineering... 𝗡𝗲𝘃𝗲𝗿 𝗵𝗮𝗿𝗱𝗰𝗼𝗱𝗲 𝘆𝗼𝘂𝗿 𝗰𝗼𝗻𝗳𝗶𝗴𝘂𝗿𝗮𝘁𝗶𝗼𝗻.

If your Fabric notebooks or functions contain lines like schema = "dev_db" or exposed API keys, you are creating a security risk and a deployment nightmare.

Microsoft Fabric just released Variable Libraries for User Data Functions, and it is the fix we've been waiting for.
- 𝗗𝗲𝗰𝗼𝘂𝗽𝗹𝗲 𝗖𝗼𝗱𝗲 𝗳𝗿𝗼𝗺 𝗖𝗼𝗻𝗳𝗶𝗴: Keep your logic clean and environment-agnostic.
- 𝗘𝗻𝗵𝗮𝗻𝗰𝗲 𝗦𝗲𝗰𝘂𝗿𝗶𝘁𝘆: Reference secrets securely without exposing them in source control.
- 𝗦𝘁𝗿𝗲𝗮𝗺𝗹𝗶𝗻𝗲 𝗖𝗜/𝗖𝗗: Swap values automatically as you deploy from Dev -> Test -> Prod.
- It’s time to mature your pipelines.

𝗖𝗹𝗶𝗰𝗸 𝘁𝗼 𝗿𝗲𝗮𝗱 𝘁𝗵𝗲 𝗳𝘂𝗹𝗹 𝗻𝗲𝘄𝘀𝗹𝗲𝘁𝘁𝗲𝗿 𝗼𝗻 𝗵𝗼𝘄 𝘁𝗼 𝗶𝗺𝗽𝗹𝗲𝗺𝗲𝗻𝘁 𝗩𝗮𝗿𝗶𝗮𝗯𝗹𝗲 𝗟𝗶𝗯𝗿𝗮𝗿𝗶𝗲𝘀 𝘁𝗼𝗱𝗮𝘆.

#MicrosoftFabric #DataEngineering #DevOps #DataSecurity #Python

No more hardcoded secrets in your Fabric code. Preheader: Manage environment configurations securely with the new Variable Libraries for User Data Fun

Open article: Goodbye, Hardcoded Secrets - Variable Libraries for Fabric User Data Functions by Leon Gordon • 2 min read

Goodbye, Hardcoded Secrets - Variable Libraries for Fabric User Data Functions

𝗠𝗶𝗰𝗿𝗼𝘀𝗼𝗳𝘁 𝗖𝗼𝗽𝗶𝗹𝗼𝘁 𝗜𝗴𝗻𝗶𝘁𝗲 𝗔𝗻𝗻𝗼𝘂𝗻𝗰𝗲𝗺𝗲𝗻𝘁𝘀 𝗰𝗮𝗻 𝗯𝗲 𝗰𝗼𝗻𝗳𝘂𝘀𝗶𝗻𝗴.

There are so many features to focus on.

- Agent mode
- App builders
- AI video creation
- Cross-app workflows
- Pricing tiers

There are definitely some features that matter more than others.

Here are my 5 key Copilot Microsoft Ignite announcements so far this week.

1. Agent capabilities that replace manual work - Excel agents now build full dashboards with one prompt. No sidebar tools needed. Charts, formulas, and insights appear automatically.

2. Seamless data flow between applications - Excel data becomes Word reports, then PowerPoint presentations. Three apps working as one workflow without exports or copying.

3. No-code app building that actually works - App Builder created a complete team ideas app in minutes. Changes happen through simple requests, not coding.

4. AI video generation with transparency - Sora 2 creates professional videos in seconds. Every output shows clear AI labels, keeping content honest.

5. Accessible pricing for small teams - At $21 per month for teams under 300, Copilot is finally within reach for businesses that couldn't afford the $30 tier.

And it is working for me.

The future of productivity isn't about learning every feature.

It's about knowing which ones actually change how you work.

If you’re planning how to roll this out responsibly, I’ve built a 2-week Copilot adoption playbook, complete with use cases, guardrails, and adoption steps.

👉 Comment “Copilot” and I’ll send it straight over.

Have you been keeping up with the Microsoft Fabric November updates?

Ignite happened this week and honestly, the pace of innovation is incredible.

I've just watched the latest update and there's a wave of features rolling out across the platform, from data engineering to Power BI and Copilot.

Here's what caught my attention:

1. Mirroring support for SQL Server, PostgreSQL, and Cosmos DB - bringing all your data straight into OneLake
2. Warehouse snapshots - giving analysts a consistent source of truth while ETL jobs are running
3. Data clustering in Fabric Data Warehouse - cutting query times from 40 seconds down to just 1 second
4. Read/write permissions in OneLake - enabling real collaboration without compromising governance
5. Copilot getting smarter - now identifying anti-patterns in your code and suggesting optimisations

The warehouse snapshots feature is particularly interesting. Imagine running hourly ETL jobs while your analysts query data without worrying about inconsistencies. The data engineer creates a snapshot, analysts query against it, and once ETL completes, the snapshot rolls forward seamlessly.

And the data clustering demo was impressive. same query, same data, but 97% less data scanned and a fraction of the CPU time used.

What strikes me most is how Microsoft is making Fabric more accessible without sacrificing governance or performance. Features like customer-managed keys at the workspace level and enhanced Copilot capabilities show they're thinking about both security and developer experience.

If you're working with data at scale, these updates are worth exploring.

Question - which of these November features are you most excited to try out?

Join our Microsoft Fabric - UK User Group for our latest webinar.

This session we are excited to announce our host Microsoft MVP Leon Gordon and co-host Microsoft MVP Pragati Jain will be introducing Microsoft MVP Andy Cutler.

Join the Microsoft Fabric UK - User Group on -

Microsoft Community - https://lnkd.in/dtrfGvy

LinkedIn - https://lnkd.in/dX_Ziaz

MeetUp - https://lnkd.in/d6wteZBn

Session Abstract -

Materialized Lake Views (MLVs) in Microsoft Fabric Lakehouses offer a declarative approach to building data loading and data materialization pipelines. This session introduces how MLVs simplify data loading piplines, automate refreshes, and visualize data loading lineage across the process. Attendees will learn how MLVs support integrated data quality rules, enabling trend analysis and alerting for violations, all without manual pipeline orchestration. We'll also cover current limitations and the roadmap for new feature support.

Biography

Andy Cutler is a Microsoft Azure Cloud & Fabric consultant, Data Platform MVP, and runs a UK consultancy Datahai. With over 20 years experience in Microsoft data technologies, Andy has worked with SQL Server, Azure Data Platform, Synapse Analytics Power BI, and Fabric.

Andy’s consultancy Datahai offers a range of services including assessments and reviews, architecture guidance, end-to-end data platform solutions, bespoke projects, training, & helps clients leverage existing technology investments, optimize costs and maximize the value of Azure, Fabric, & Power BI.

Social Media / Website Links

https://lnkd.in/eW69uSs9

Can Fabric Materialized Lake Views Deliver Low Effort Data Engineering?

Most data teams aren’t struggling because of skill.

They’re struggling because of complexity.

Pipelines with too many moving parts.
Manual fixes that eat entire mornings.
That constant feeling of: “We’re one error away from another fire drill.”

If you’re leading a data team, you’ve lived this:

- Late-night messages when a job fails
- “Quick requests” that are anything but
- Pressure to deliver faster with zero tolerance for mistakes

Here’s the part we rarely say out loud:

It’s not the work that’s breaking teams.
It’s the invisible steps, dependencies, and rework behind the scenes.

That’s where Copilot for Data Factory actually matters.

Not because it magically “automates everything”…
but because it strips away the friction that slows everyone down.

When complexity drops, your team finally gets space to:

- Think instead of react
- Design instead of patch
- Deliver outcomes instead of chasing errors

This isn’t about replacing people.
It’s about giving them their bandwidth back.

If your team could use more clarity in how they build and run pipelines, comment “CLARITY” — I’ll share examples, frameworks, and real-world Copilot for Data Factory workflows that actually help.

#DataLeadership #DataTeams #AIinData #MicrosoftCopilot #DataFactory #DataEngineering

Your data team isn’t overwhelmed…

Wavicle Data Solutions

Lead Data Engineer

Globant

Senior Data Engineer

CRISIL Limited

Senior Analyst - Data Engineer

Quantiphi

Data Engineer

Tata Consultancy Services

System Engineer

Assistant System Engineer

Assistant System Engineer - Trainee

InsideSherpa

KPMG Data Analytics Consulting Virtual Internship

Monisha Enterprises

•	Data mining and Extraction Transformation Loading using R programming.
•	Data cleaning and processing for Data analysis done using Python using NumPy, pandas, SciPy, Scikit. Data modelling and prediction was done using algorithms – Logistic Regression, Time Series Forecasting, Clustering and Classification.
•	Data Visualization done using Tableau for generating reports to enhance the productivity of the products.
•	Built a Data Warehouse on MSSQL Server for ERP System of the enterprise to organize the data for future processing.

Data Analyst Intern

Open to Collaborate | Storyteller | Lead Data Engineer@Wavicle| Linkedin Top Voice 2024 | Globant | Linkedin Learning Instructor | 2xGCP & AWS Certified | LICAP'2022

Data Engineer with 7+ years of experience working on data engineering, analysis, and modelling projects for various domains, such as banking, healthcare, sales, and e-commerce. I leverage my strong statistical and analytical ability to explore business ideas and provide innovative solutions. 

Skilled with technologies and strong intellect with enthusiasm as an avid learner with challenging opportunities. I’m working as a LinkedIn Learning Instructor to bring change as a data evangelist.

Post Graduated in Master of Technology in Data Science and Engineering from Birla Institute of Technology and Science, Pilani in 2023, where I learned advanced concepts and techniques in data science and engineering, machine learning, cloud computing, and visualization. 

I hold multiple certifications in GCP, AWS, and Azure, and I enjoy working with cloud technologies and big data platforms.

I am passionate about data science and women empowerment, and I have been recognized as the 
- Linkedin Top Voice 2024 
- Data Science and Women Empowerment Thought Leader by Global AI Hub. 

I am also a proud participant of the LinkedIn Creator Accelerator Program 2022, where I share my insights and experiences with the data science community. 

I am open to collaborate and share ideas, or explore various opportunities in the data field.

Pooja Jain

73% of data pipeline failures happen because of dependency issues, not bad code.
Cron jobs don't care if yesterday's data never arrived. They'll happily run on empty tables and produce garbage.

Airflow? It checks, retries, alerts, and refuses to cascade your failure downstream.

→ Beyond scheduling Airflow gives an important feature of 𝗗𝗔𝗚𝘀(𝗗𝗶𝗿𝗲𝗰𝘁 𝗔𝗰𝘆𝗰𝗹𝗶𝗰 𝗚𝗿𝗮𝗽𝗵𝘀), that helps to monitor, retry and gives flexibility to organize pipelines with minimum/less code.

🤔 Data Engineers often misinterpret Airflow just as an orchestration tool.

Apache Airflow is an open-source platform designed to programmatically author, schedule, retry and monitor workflows. 

🎼Think of it as the "conductor of an orchestra", where each musician (your data tasks, like data extraction, transformation, loading, or model training) knows when to play, what to play, and what happens if they make a mistake.

Most data engineers automate the wrong things at the wrong time.

Here's the framework I use after 8 years of building production systems:
✅ AUTOMATE WHEN: 
→ Task runs daily/weekly 
→ Human errors cause outages 
→ Work blocks other priorities
→ Team growth = more manual work
Examples: Reports, schema checks, alerts

❌ DON'T AUTOMATE WHEN: 
→ Task happens quarterly 
→ Requirements change weekly 
→ Process isn't understood yet 
→ Manual steps reveal insights

My rule: If it’s done 3+ times, script it; 10+ times, automate it; fails 5+ times, redesign it.
Automate what matters, when it matters—not everything!

Airflow isn’t just about running tasks.

🕑 It’s about building trustworthy, scalable, and observable pipelines that run reliably at 2am — even when you’re not watching.

Airflow gives structure to your chaos.
DAGs give clarity to your pipelines. 
That’s why DEs love them ❤️

Real Example: 
Instead of: 
❌ Cron jobs that fail silently 
❌ Manual dependency management
❌ No visibility into failures
You get: 
✅ Visual workflow monitoring 
✅ Automatic failure notifications
✅ Smart task scheduling 
✅ Easy debugging & restarting

Here's an amazing Airflow cheatsheet with some fundamentals concepts and interview questions to prepare for data engineers. 👇

Carousel Credits -  Futurense.com

Willing to learn more, here's the link to Astronomer free learning path - https://lnkd.in/g8-EWBda

Also, level up with Marc Lamberti's Udemy course on Airflow 3 - https://lnkd.in/gwArwaME

taplio

The Battle

Leon Gordon

Pooja Jain

Followers

Posts

Engagements

Likes

Performance ScoreLeon Gordon's LinkedIn Performance

Performance ScorePooja Jain's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Leon Gordon

Pooja Jain

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Leon Gordon's LinkedIn Performance

Performance ScorePooja Jain's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreLeon Gordon's LinkedIn Performance