DataOps.live

I help customers deploy outstanding #dataops solutions powered by snowflake.

Senior Solutions Architect

EVERYTHING BUT THE HOUSE (EBTH)

Senior Data Scientist

Firstlight Media

I defined the Data Strategy for bringing data together from multiple applications to a consolidated enrichment platform that produces data visualizations. I designed and implemented the tools and techniques for producing consumer and content-based recommendations for our media player using graph technologies.

Data Strategist

KORE Wireless

I modernized our SQL Server environment to apply best practices in database use. 

I act as a Data Science Mentor, and Data Architect as we design new applications and use-cases for our data products. 

I implemented consolidated management of our SQL Server infrastructure using Idera tools. Upgraded our environment to current versions of SQL Server. 

I built an ETL team for managing the integrations that tie our business systems Netsuite and Salesforce to our internally developed applications. 

Using Signal analysis, I was able to demonstrate capabilities that could be implemented for any kind of anomaly detection as we monitor devices for our customers.

Senior Manager Data Services.

Cengage Learning

I created an ensemble predictive model using SQL rules, and R decision tree's to accurately (Precision/Recall both above 80%) predict the interactions with our platforms that at risk students exhibit. I created a Semantic analysis process for survey data to quickly parse through hundreds of feedback comments to focus customers on the issues and sentiment most important to respondents.

Data Scientist

Illumination Works

I managed our Data Science lab. Our Data Science Lab using both Map Reduce and Spark is based on CDH 5 from Cloudera. It is built using Cloudera Director and Cloudera Manager in AWS. Presentations built using our Data Science Lab were presented in December 2014 and January 2015.

Illumination Works is a solution enabling company. We provide professional consulting, design, and development services that deliver core information and insight capabilities. We are committed to solving problems in a way that exceeds expectations and delivers value-based capabilities for our customers.

Senior Consultant - Enterprise Data Systems

clear measures

I am a Principal Consultant with expertise in Data Warehouse building, Data Vault Methodology, Data Science, Business Analytics, Hadoop infrastructure, and Reporting.

I worked on multiple architecture projects, designing Data Vault Data Warehouse systems. Integrating Hadoop into an overall Logical Data Warehouse strategy for a major Retailer. Teaching internal sessions on Hadoop and Data Vault. I also built a Hadoop Data Science Lab for internal research and training.

Principal Consultant - Enterprise Architecture

3M Health Care

I engineered the cloud based infrastructure for the management of health care data for 3M customers. Using Rightscale to manage our clouds (Rackspace,Azure,AWS), and direct configuration tools for our hosted data centers. Our data platforms are diverse, DataStax Cassandra Enterprise, SQL Server,  Cloudera Hadoop, and Percona MySQL. I do data server configuration, capacity planning, data modeling, Pentaho Deployment, Performance tuning, support of our Data Science Lab and on-call support as the need arises. 
 
The typical duties of your average data-guy.

Lead Data Engineer

Rent The Runway

This was a research project to determine how Pentaho Business Intelligence platform could assist in the Data Science efforts of the analytics group. I deployed the platform, and we evaluated it based on comparison with the internally developed tools the analytics team had already built.

Data Management Consultant

Sunrise Senior Living

At Sunrise Senior living I show others how best to apply industry best practices of data management to our projects. There are times when the schedule dictates getting things done faster than what some best practices would allow. Knowing the best practices to follow for Data Mart design allows us to determine where we can speed up the process in order to meet deadlines.

Once we had pushed our Financial ETL processing to the limits of what we could do with software optimizations we did performance testing on new hardware from HP. We achieved significant performance improvements with the new hardware. The new hardware also allowed us to make further software modifications to reduce the run times of our financial reporting system from 14 hours every weekend and 48 hours every quarter to a nightly run of 2 hours. This increase in frequency of reporting allowed faster insight into our financial reporting system with a lag time from production of only a single day, instead of a week or more.

Data Management

► I am a data guy. How can I help you make your data more useful? 

► Author of The Enrichment Game: https://technicspub.com/enrichment/
Making Data more powerful

► Author of Data Structure Graphs : http://www.amazon.com/gp/product/B00XETTPCI
Social Network Analysis applied to Data Architecture!

► Snowflake, Clickhouse, Oracle, MySQL, SQL Server. Where is your data? What does your data say? How does your data flow? 

► My github: https://github.com/dougneedham

► My random thoughts, and sometimes useful notes are blogged here: http://the-data-guy.blogspot.com/

Congratulations Dan Linstedt on 10 years of WWFVC. Kent Graziano and I were the only speakers from the inaugural event that made it to 2023



Doug Needham

Invest in your Data People. Invest in your Data, people. Asking the right question is more meaningful than giving the right answer to the wrong question.

Small towns have secrets. Stowe, Vermont? It's hiding the brainiest of the Data Vault world this week. Picture this: the elite of the elite, swapping tales not of fish they caught, but of data they've wrangled. It's like summer camp, but with more algorithms and less s'mores. 
 
On one side, you've got the traditionalists. Their methods are tried and tested, the equivalent of a firm handshake or a well-aged whiskey. Predictable. Reliable. A little...boring? 
 
Then there's the innovators. These folks see data like a wild river, and they're ready to ride the rapids. New techniques, cutting-edge tools, and practices that might just be genius—or madness. 
 
The question is, in the ever-evolving world of Data Vault, are you playing it safe or diving headfirst into the unknown? 
 
Let's stir the pot. Drop a comment with your bet: Old School Stability or New Age Nerve? Who will lead the charge into the future of data management? 
 
Engage below and let's see who's leading the data revolution from the snowy confines of Stowe.

Work hard, play hard? How quaint. Welcome to the era of Rest Hard. 
 
Let's break it down. On one side, we have the age-old mantra: Work hard, play hard. It's the battle cry of the overachievers, the hustlers. You grind all day, then party all night. Sounds exhilarating, right? Until it's not. Burnout enters the chat, uninvited. 
 
Enter the challenger: Rest Hard. It's not just sleep; it's an art. It's understanding that to truly excel, your mind and body need downtime. Think of it as a radical act in a world that can't stop boasting about 80-hour work weeks. 
 
Scenario A: You're juggling tasks like a circus performer, fueled by caffeine and ambition. By day's end, you're too wired to rest, planning how to conquer the world from your pillow. 
 
Scenario B: You work diligently, then step back. You indulge in rest as fiercely as you pursue your goals. Your off-hours are sacred, a time to recharge and rejuvenate. 
 
One leads to a cycle of endless pursuit. The other? Sustainable success. 
 
The choice seems pretty clear unless you enjoy bragging about stress levels like they're trophies. 
 
So, what's it going to be? Continue on the hamster wheel of perpetual motion or embrace the revolutionary act of resting hard? 
 
Drop a comment below. It's time we redefine what it means to truly excel in life and work. Let's champion a culture that values rest as much as achievement.

One of the most exciting parts of being at #wwdvc is being able to hear people like Chris Date speak. Good portions of the research I have done on how best to put together good data models is based on his writings. Getting to hear him speak and explain concepts is a value that cannot be expressed. 

#data 

Work hard, play hard. We've all heard it, right? But let's introduce the unsung hero of success: Rest hard.  
 
Imagine two colleagues: One sprints through 12-hour days fueled by caffeine and ambition. The other? Masters the art of productivity, then unwinds with quality downtime. Fast forward. The sprinter is burnt out, while our balanced friend is thriving, innovating, and... still enjoying their weekends.  
 
We glorify the grind, but what about glorifying the recharge? Science backs it up: Rest isn't just good; it's essential for peak performance. Think of it as the secret ingredient in the success recipe.  
 
So, next time you're tempted to pull an all-nighter, remember: Resting isn't laziness. It's the smartest strategy for long-term achievement.  
 
Who's ready to join the rest revolution? Drop a '🛌' if you're with me!

Invest in your Data People. Invest in your Data, people.

🚀 Revolutionize your data game with Snowflake! 🌨️ 
 
Unlock the power of auto-scaling, workload separation, and time-travel for seamless data recovery. 💡 Query JSON data in variant columns effortlessly and accelerate your analysis using tools like Kafka and Dbezium. 📊 
 
With the Snowflake Kafka Connector, capturing change events and storing them in staging tables becomes a breeze. Say goodbye to complex ETL jobs and hello to quick data enrichment from multiple sources for insightful reporting. 🌟 
 
Let's shift the focus from tangled data ecosystems to leveraging data for problem-solving. 🌐 Are you ready to elevate your data strategies? #Snowflake #DataAnalytics #Innovation

Ever heard of a VIP club for data wizards? Picture this: the elite brains of the Data Vault community have turned Stowe, Vermont, into their own Hogwarts. But instead of wands, they're wielding groundbreaking ideas and best practices like never before. 
 
On one side, you have the traditionalists. Picture them in velvet robes, guarding their age-old secrets and nodding at each other in solemn agreement over ancient scrolls (or, you know, spreadsheets). 
 
On the flip side, the innovators. They're the ones with LED-lit wands (fine, laptops), brewing potions of cutting-edge implementations and daring each other to push the boundaries of what's possible with data. 
 
Here's where it gets interesting. They're not just there to bask in their own glory. Nope. They're exchanging ideas, challenging norms, and, most importantly, educating each other. 
 
Why should you care? 
 
Because this isn't just about being part of a secret society. It's about choosing your path in the ever-evolving world of data. Will you stick with the traditionalists, mastering the ancient arts? Or will you join the innovators, pushing the envelope and redefining what's possible? 
 
The choice is yours. But remember, in the realm of data, standing still is not an option. 
 
Ready to pick a side and revolutionize your approach? Let's chat. Who knows, maybe next year, you'll be the one sharing wisdom in Stowe.

Ever paid for a gym membership and never went? That's your failover strategy gathering dust. Tools may dent your wallet, but ignoring them? That's like buying a first-class ticket on the Titanic for its Wi-Fi. 
 
Picture this: Two businesses. One tests their failover like clockwork, the other thinks 'It won't happen to me.' Spoiler alert: When disaster strikes, one's sailing smooth, the other's in tech Titanic. 
 
No one likes unexpected swims in the icy waters of downtime. It's not just about having tools; it's the difference between thriving and becoming a cautionary tale.  
 
So, when was the last time you tested your failover? If you're scratching your head, it's been too long. Let's not wait for an iceberg to remind us what we should've done. 
 
Don't be a blockbuster story in a Netflix world. Test your failover. Today. Your future self will thank you. 
 
Ready to dive deeper? Let's chat about making your failover foolproof.

Data is like soil. Rich, yet often underappreciated. 
 
Imagine two gardeners: one who reuses and enriches the soil, and the other who tosses it aside after one use, always seeking new ground. The first flourishes, yielding bountiful harvests; the second struggles, his garden a testament to wasted potential. 
 
In the digital realm, we face a similar choice. Reuse and cultivate your data, turning it into a fertile ground from which insights and innovation can grow. Or, dismiss it after a single use, forever chasing fresh data sets, never realizing the full potential of what you already have. 
 
The irony? Both soil and data become richer with use. The more you work with them, the more value they provide. Yet, while gardeners have known this secret for centuries, many businesses today still overlook the treasure trove at their fingertips. 
 
Don't be the gardener who seeks new land. Be the one who nurtures and enriches the soil they already have. Dive deep into your data. Reuse, recycle, and reap the rewards. 
 
Ready to turn your data into a flourishing garden of insights? Let’s discuss how you can cultivate your digital soil to its fullest potential.

🚀🤖 Exciting times ahead in the stock market! 📈💡 The rise of generative artificial intelligence (AI) is creating a buzz among investors, propelling stocks like Nvidia to new heights. 🌟 While some worry about a possible AI bubble, the consensus is clear: the power of AI to transform industries is simply unmatched. 💥 Unlike past speculative bubbles, today's AI surge is backed by solid fundamentals from established tech giants. 🏦💼 These mega-cap firms are not just investing in AI but shaping its future through lobbying and venture capital. 💰💡 Investors, stay cautious yet optimistic as the AI revolution unfolds! 🚀🌐 #AI #Investing #TechRevolution

Ever tried learning something new every day? It's like choosing between a rollercoaster and a merry-go-round.  
 
One, a thrilling ride of unexpected twists, turns, and knowledge bombs that leave your brain buzzing. The other, a comfortable, predictable loop of the same old skills and ideas.  
 
Imagine stuffing your brain with random facts, skills, or maybe even a new language. Sounds exhilarating, right? But here's the kicker: it's not just about the thrill. It's about expanding your horizons and staying relevant in a world that changes faster than your social media feeds.  
 
On the flip side, sticking to what you know is like choosing safety over adventure. Sure, it's comfortable, but where's the fun in that? Where's the growth?  
 
So, what's it going to be? Will you step up to the challenge and join the ranks of lifelong learners, or will you stick to the merry-go-round and watch the world evolve without you?  
 
Let's make learning an everyday adventure. Share your thoughts or tag someone who's up for the challenge!

Ever heard the saying, 'Garbage in, garbage out'? Well, the world of questions isn't far off. Imagine spending hours crafting the 'perfect' answer to a question that, frankly, no one asked. Hilarious, isn't it? 
 
Here's a wild idea: What if we've been playing the game all wrong? Instead of priding ourselves on having all the answers, maybe, just maybe, we should master the art of asking the right questions. Sounds revolutionary, doesn't it? 
 
Think about it. The right question opens doors, peels layers, and uncovers truths. It's like choosing between a key or a battering ram to open a door. Both can get you through, but one certainly leaves the door in a better shape for the next person. 
 
Now, I'm not saying throw all your knowledge out the window. But, what if your arsenal of answers is ready for the wrong battle? It's like bringing a knife to a pillow fight – overly prepared and slightly embarrassed. 
 
So, here's a thought: Next time you're tempted to showcase that brilliant answer, pause. Ask a question instead. Not just any question, but the right one. It could change the game.  
 
Curious? Want to dive deeper into the art of question-asking? Let's start a conversation. Share your thoughts or, better yet, your questions below. Let's flip the script together.

Work trips: A prison sentence or a golden ticket? 
 
Imagine two travelers: One chained to their laptop, eyes glazed over in airport lounges and hotel rooms. The other? Sneaking moments of adventure between meetings, discovering the world's wonders—big and small. 
 
The first sees airports as transit zones. The second? Gateways to the new and unknown. 
 
Yes, work sends us places. But why travel with the mindset of a clock-watcher? Why not transform obligation into exploration?  
 
Think of it as a challenge: Next trip, find one thing that’s uniquely 'there'. A meal, a view, a street artist. Anything that says, 'I wasn’t just there; I lived it.' 
 
So, what’s it going to be? The same old grind or a mini-adventure? 
 
Drop a comment with the most memorable thing you’ve discovered on a work trip. Let’s inspire each other to turn business travel into an exploration!

Where is the Data that supports your mission statement?

This is something everyone should remember.

Ever felt the thrill of adding a new title to your reading queue? It's like unlocking a new world of possibilities. Recently, a friend recommended a book that reshaped his thinking. Naturally, it soared to the top of my list.  
 
Think about it - our reading queues are more than lists; they're roadmaps to our future selves. Every recommendation holds the power to alter our perspectives, challenge our thoughts, and inspire change.  
 
But here's the twist: How often do we share these transformative reads with others?  
 
Now, I challenge you. Look at your reading queue. Pick one book that changed you. Share it with someone.  
 
Let's not just consume information; let's be the catalysts for change.  
 
What's in your reading queue? More importantly, what book will you share today?

Invest in your Data people. Invest in your Data, people!

Data isn't just numbers, it's the new gold.  
 
Imagine a world where your financial assets are not just figures in your account but keys to unlocking immense value. That world is now. Data is not only a valuable asset in its own right; it tells the story of your wealth. 
 
Before, we viewed data as mere byproducts of transactions. Now, we see data as the backbone of strategic decision-making, offering insights that can catapult your financial growth to new heights. 
 
Here's the twist: while we guard our tangible assets with utmost care, our data often remains underutilized, sometimes even neglected. It's time to shift gears. By harnessing the power of your financial data, you transform it into a tool that not only safeguards your assets but also multiplies them. 
 
Start treating your data with the same reverence you do your tangible assets. Dive deep. Analyze trends. Make informed decisions. The treasure trove of insights you'll uncover can redefine your financial strategy, propelling you toward unparalleled growth. 
 
Ready to unlock the true potential of your financial data? Let’s turn your data into your most powerful asset yet. Engage below and let’s chart the course to a wealthier future together.

Garbage in, garbage out. Ever heard of it?  
 
In the world of machine learning, this isn't just a cheeky saying—it's the cold, hard truth. The battle of the century isn't between algorithms. It's between pristine data and the mess we often settle for. 
 
On one side, we have High-Quality Data. It's like the secret ingredient in a Michelin-star recipe. The outcome? Flawless, efficient, and almost clairvoyant machine learning models that predict, analyze, and innovate like a dream. 
 
Then there's the other side. The Dark Side, if you will: Poor Quality Data. It's the equivalent of trying to cook a gourmet meal with expired ingredients. The result? Models that stumble, mumble, and fumble in the dark—producing results as accurate as a weather forecast. 
 
Here's the kicker: the choice is yours.  
 
Do you want a machine learning model that leads the pack? Or one that's chasing its own tail? Investing in data quality isn't just a good idea; it's a game-changer. 
 
Swipe right on quality data. Your algorithms will thank you. 
 
Let's start a conversation below. Share your 'data quality vs. data quantity' horror stories or successes. Let's learn from each other!

🔍 Delve into the world of Data Structure Graphs! 📊 
 
Ever wondered how applications, database tables, and documents are all interconnected within an organization? 🤯 Enter Data Structure Graphs – the key to unlocking these intricate relationships! 🗝️ Visualized through tools like Gephi and NetworkX, these graphs offer a deep dive into the structure of enterprise applications, paving the way for enhanced decision-making. 💡 Excited to learn more about this fascinating topic as the author dives deeper into Data Structure Graphs! 🚀 #DataStructure #GraphAnalysis #EnterpriseInsights 🌐

Ever thought of becoming a mentor? You have two paths: The Traditionalist vs. The Trailblazer.  
 
The Traditionalist loves the sound of their own voice, sharing wisdom like it's going out of fashion, often uninvited. They thrive on the 'been there, done that' mantra, pouring out advice faster than you can Google it. 
 
Then, there’s The Trailblazer. They listen. They challenge. They throw the map out the window and dance in the ambiguity of uncharted territories. They don’t just pass down wisdom; they co-create it.  
 
The Traditionalist will give you a fish and then tell the tale of how they caught it, embellishing each time. The Trailblazer? They won’t just teach you to fish; they’ll jump into the water with you, laughing as you both figure it out. 
 
In a world saturated with mentors, be the one who stands out. Be the mentor who doesn’t just lead the way but explores new paths together with their mentee. 
 
Ready to ditch the podium for a pair of hiking boots? Let’s start a conversation on how we can redefine mentorship together. Drop a comment or send a message. Let’s blaze new trails.

Ever feel like your job title should be 'Professional Problem Solver'? Because, let's face it, half the time we're either putting out fires or preventing them. 
 
Now, there are two camps when it comes to tackling problems: 
 
1. The 'Traditional Troubleshooters' - Armed with a handbook of SOPs, they attack every problem with a tried and tested method. Predictable? Yes. Effective? Sometimes. 
 
2. The 'Creative Mavericks' - They see a problem and say, 'Rules? What rules?' They're the ones drawing outside the lines, finding solutions you didn't even know existed. 
 
Here's the kicker: Problems don't care about your approach. They just sit there, waiting to be solved. But the way you solve them? That's what sets you apart. 
 
Imagine turning every challenge into a story of innovation and creativity. Where others see a roadblock, you see a detour sign pointing towards uncharted territories. 
 
So, what's it going to be? Are you sticking with the old school, or are you ready to join the ranks of problem-solving legends? 
 
Let's not just solve problems. Let's redefine them. 
 
Think you've got a story of creative problem-solving that can inspire? Drop it in the comments. Let's show the world how it's done.

Taplio

Doug Needham's Linkedin Analytics

Doug Needham

What is Doug talking about?

Doug Needham's Best Posts (last 30 days)

Want to drive more opportunities from LinkedIn?

Taplio

.css-1jgreyo{background-image:linear-gradient(284deg, rgba(39,43,72,1) 0%, rgba(33,47,183,1) 100%);color:transparent;-webkit-background-clip:text;background-clip:text;}Doug Needham's Linkedin Analytics

Doug Needham

What is Doug talking about?

Doug Needham's Best Posts .css-qqfgvy{font-size:var(--chakra-fontSizes-md);}(last 30 days)

Want to drive more opportunities from LinkedIn?

Doug Needham's Linkedin Analytics

Doug Needham's Best Posts (last 30 days)