The life of a failed data science project

Step 1: manager gets excited about potential
Step 2: manager proposes a new team
Step 3: company rejects; too big
Step 4: manager refines
Step 5: company compromises to one data scientist
Step 6: manager hires without really knowing the space
Step 7: eager person takes a low-ball offer to get their first DS role
Step 8: enthusiasm, joy
Step 9: vague project ideas discussed
Step 10: DS struggles to find data
Step 11: DS spends months smooshing datasets together
Step 12: DS spends months trying models in Jupyter
Step 13: DS discovers there's no one to help them get this to production
Step 14: manager scolds DS for lack of progress after months
Step 15: DS quits
Step 16: company scolds manager
Step 17: manager quits
Step 18: company vows never to do this again

#DataScience #Leadership #DataScientist

Dr Adam Sroka, Director of Hypercube Consulting, is an experienced data and AI leader helping organisations unlock value from data by delivering enterprise-scale solutions and building high-performing data and analytics teams from the ground up. Adam shares his thoughts and ideas through public speaking, tech community events, on his blog, and in his podcast.

Many organisations aren't getting the most out of their data and many data professionals struggle to communicate their results or the complexity and value of their work in a way that business stakeholders can relate to. Being able to understand both the technology and how it translates to real benefits is key.

Simply hiring the most capable people often isn’t enough. The solution is a mix of clear and explicit communication, strong fundamentals and engineering discipline, and an appetite to experiment and iterate to success quickly. 

If this is something you’re struggling with - either as an organisation finding its feet with data and AI or as a data professional - the approaches and systems Adam has developed over his career will be able to help so please reach out.

Cutting-edge data technologies are redefining every industry and adopting these new ways of working can be difficult and frustrating. One day, there will be best practices and playbooks for how to maximise the value from your data and teams, but until then Adam is eager to share his experiences in both business and data and shed some light on what works.

Data is indeed magnificent!

Check out this captivating 3D representation, illuminating population density disparities across countries.

Countries such as Denmark and France exhibit pronounced urbanization, particularly in their capital cities. Meanwhile, Italy appears to embrace a more dispersed population structure.

Hat tip to Terence Fosstodon for this insight!

#DataVisualization #Demographics #UrbanPlanning #Geography #DataIsBeautiful

No alternative text description for this image

💲💲💲 I reduced the cost of our Snowflake data warehouse by 80% using these simple steps!

1️⃣ Reduce the sync frequency of your data ingestion tool. Evaluate whether having fresh data every hour is essential, as updating the data every 6 hours might be enough.

2️⃣ Schedule all data loading and transformation tasks to run simultaneously in the warehouse, taking advantage of parallelization. Instead of paying for 10 loads at 2 minutes each (totaling 20 minutes), you'll only pay for 2 minutes!

3️⃣ Explore incremental materializations and ordered tables to optimize performance and efficiency in your data warehouse, especially for complex models and large tables.

4️⃣ Use data extracts in your visualization tool to avoid querying the data warehouse with every visit. This will result in cost savings and a smoother user experience, with lightning-fast reports and dashboards.

👉 Let me know in the comments if you have any other simple techniques to save costs on your data warehouse!

#analyticsengineering #dataengineering #growth

Data & Analytics Engineer | Follow for data-driven growth systems

Dive into the inner workings of SQL queries in our latest video. Explore how query execution plans can improve database performance and learn more about the execution order of SQL operators.

Watch here: https://lnkd.in/g7kp-U5b

I build large-scale systems | 400K YouTube channel | 400K Newsletter | Best-selling books | Eng at Discord

Our supposed “Baselines” are outdated.

Today's unprecedented expansion of transparency and data have informed us how they're expiring faster than ever.

These conclusions become irrelevant very fast without serious, frequent and tech-assisted analysis..

..and that's a problem if you want to optimize conversions in your business.

Aka - you’re likely complacent operating with old info, leaving money on the table.

*Frequent, enhanced analysis of your 1st-party data is CRUCIAL; but doing it right is much more intensive (and fruitful) than ever before, deserving of a dedicated post*

That’s why I argue that when planning for conversion optimization throughout your business (see the roadmap here: https://lnkd.in/gEf__v9e)..

...Instead, you should look at the industry landscape.

We need to reexamine who "The Jones'" are in this AI-chaperoned reality.

--

The industry landscape bar is also constantly rising as AI blitzes.
 
We could go as far to say that goalposts are shifting - if not unfairly, certainly at a scale that’s hard to keep up with using old tricks.

So, here's a spotlight on "industry standard espionage."

AKA supercharged spying on competitors.
 
Get a grip, find the gap where there isn’t focus and THEN you can start baselining yourself.

--

First, know who your competitors are. You can start by simply asking Chat GPT using one key example you already know.

Next, reverse engineer - analyze and dissect what they're doing, so you can plan your strategy for what YOU should be doing.

You can use Chat GPT to ask about some these parameters straight up (i.e. “what is the most popular content on (competitor.com)?"

But tools like SEM Rush are powered by AI, and make it their bread and butter.

--

What to tag:

- How they generate traffic: Uncover traffic share and marketing mix, and what sites people visit before and after they hit

- Top offers and their cvr on product pages: Note their UI and messaging. 

- Trends and audience data: Spot seasonality, identify in-market audience and interests to build a better audience fit.

- Assess organic strategy: What keywords are they targeting that bring the bacon? What pages rank in Google’s SERPS?

- Examine advertising strategies: What ads are live now, for what products?

- Content and socials: What content is popular on x competitors site? Measure “share of voice” to compare your position to competitors. Track what competitor content drives the most people onto site, branded hashtags and keywords, topics that generate attention and backlinks, and even complaints.

- Partner channel marketing: Who are their affiliates, dealers and distributors? 

--

SEM Rush is a super rich tool for these insights, but alternatives include Mentionlytics, Hootsuite, Brandwatch and Keyhole. 

Mix and match with Chat GPT to give you your industry barings!

#competitiveresearch #competitiveanalysis #industrylandscape


I build companies & teams that turn BIG, BOLD ideas into actionable plans resulting in profitable businesses. SaaS Enthusiast. Startup Addict.

Friendly reminder that if you haven't landed a Data Analyst role

You need to pivot your strategy

Your strategy is unique to YOU

That's why it is a good idea to take time to reflect on what is working and what is not specifically for you

Not getting interviews? Your resume needs some edits
Not getting passed round 1? Work on communication skills
Not getting passed the technical? Practice more of your tech skills
Not getting passed panels? Practice more STAR questions

Adapt how you work and double down on what is working

#data #tech #dataanalytics #dataanalyst 

I'll help you land a career in Data Analytics | Datadad.io | Data Analyst

🎁Hey, Data Analysts - I have a FREE gift for you!🎁

Choosing the right statistical test can make or break your analysis. But with so many tests and complex data types, knowing which one to use and when can be overwhelming. 

That's where "WHICH TEST WHEN?" comes in! 🎯💡

⬇️Download this comprehensive guide and unlock the secrets to understanding data types, distributions, and statistical tests. 📈

Why should you care about data types and distributions? Because they hold the key to accurate analysis and informed decision-making! 🗝️

By mastering these fundamental concepts, you'll be able to:

✅ Select the right test for your data type, ensuring accurate and reliable results

✅ Gain confidence in your analytical skills and make impactful data-driven recommendations

✅ Propel your career as a trusted data analyst, valued for your expertise in statistical analysis

Download "WHICH TEST WHEN?" today and take your data analysis skills to new heights. 🚀💼

Want to learn more? Check out
StatQuest from Joshua Starmer PhD
Complete Statistics Course YouTube from freeCodeCamp
Khan Academy
and Brandon Foltz on YouTube

Happy Learning!

#DataAnalysis #Data #DataAnalytics

Data Leader | Data Science Professor | Your Guide to Data Analytics

WHICH TEST WHEN? Your Guide to Choosing the Right Test

Stop creating dashboards people ask for.

Start investigating what your end-users actually need.

#data #analytics #datascience

📍 𝟭𝟬 𝘆𝗲𝗮𝗿𝘀 𝗮𝗴𝗼, 𝗜 𝘄𝗮𝘀 𝗮 𝗽𝗿𝗼𝗳𝗲𝘀𝘀𝗶𝗼𝗻𝗮𝗹 𝗽𝗼𝗸𝗲𝗿 𝗽𝗹𝗮𝘆𝗲𝗿. Every day, I optimized my decisions based on data and human behavior. I don’t play poker anymore. 

But I'm still passionate about numbers and psychology. Because in my Data & Analytics career I experienced that you need more than just technical skills. 

Many times, people in the business did not listen to my insights. How frustrating. Isn't data the truth? 

Now I know it was not them. It was me. Many people in Data & Analytics face the same challenge. That's why I founded the company MindSpeaking: communication skills training for data professionals. 


📙    𝗕𝗢𝗢𝗞
People Skills for Analytical Thinkers: how to communicate as a Data Analyst or Scientist.
#1 bestseller in 3 categories. Order now or read a free preview:
▪️ MindSpeaking.com/book

🌱   𝗙𝗥𝗘𝗘 𝗦𝗘𝗟𝗙-𝗧𝗘𝗦𝗧
How skilled are you at communicating Data?
Take the free self-assessment (5 min) and grow to the next level:
▪️ MindSpeaking.com/maturity-model

 🚀   𝗖𝗢𝗠𝗣𝗔𝗡𝗬 𝗧𝗥𝗔𝗜𝗡𝗜𝗡𝗚 
MindSpeaking: Communication skills training for Data Analysts & Scientists. Have your team learn Data Storytelling, increase your business impact, and retain your data talent:
▪️ MindSpeaking.com

📈     𝗢𝗡𝗟𝗜𝗡𝗘 𝗖𝗢𝗨𝗥𝗦𝗘: 𝗗𝗔𝗧𝗔 𝗦𝗧𝗢𝗥𝗬𝗧𝗘𝗟𝗟𝗜𝗡𝗚
Are people not listening to your insights? 
Learn how to create a Data Story and present with confidence:
▪️ MindSpeaking.com/onlinecourse

Gilbert Eijkelenboom

My recent change to a machine learning engineer has led me to dive deeply into what it means to have a machine learning platform. 
 
Every machine learning platform has these 8 parts to them. 
 
Data Cleaning 🧹 
 
An essential step in machine learning pipelines. While data engineers do an initial cleaning, data scientists take it further and build for usefulness.  
 
Think: date formats, limiting outliers, and pattern-spotting! 
 
Feature engineering 🔧 
 
Select, extract, and store relevant features for better model performance. 
 
Feature Store 🏬 
 
Tables that are versioned and created for each model focus area 
 
Model training🏋️‍♂️ 
 
Pick an algorithm, train it on engineered data, and iterate until you achieve peak performance. 
 
Model evaluation🧐 
 
Test the model, evaluate its performance, and save it to a model registry once it's ready for deployment. 
 
Model Registry 🗄️ 
 
A central hub tracking models, performance metrics, and vital metadata for zero confusion. This information is critical to ensure models can be reproduced and tracked throughout their lifecycle. 
 
Model deployment 🚀 
 
Time to integrate the trained model into a production system and make predictions on new data. 
 
Model monitoring 📊 
 
Keep an eye on evaluation metrics to see how our models are performing.  
 
#ml #datascience

Senior Data & ML Platform Engineer | dutchengineer.substack.com | Host of Ask A Data Mentor Podcast

In recent years, many companies have been looking for data scientists who can collect, filter, and convert transactional data from an organization into valuable insights, helping them with important business and customer decisions.

In this new era of generative AI, what I call "content scientists" will become the new rockstars. They are the ones that will be dealing with all the information that is scattered throughout the company, in various forms and sources: from Word documents to PDFs and PowerPoints, stored on servers, Google Drives or SharePoints. Companies that can feed all that content in a coherent way into generative AI, will be able to make a huge efficiency gain. 

Read my piece about content scientists here:

Open article: Why content scientists are the new rockstars | Peter Hinssen by peterhinssen.com

Why content scientists are the new rockstars | Peter Hinssen

How do you know if you are onto something? 

When someone validates you, your vision, and your execution.

What's the best way to validate something?

You invest in it and put your 💰 where your 👄 is!

I felt the buzz at Catalyst Software right away, and I'm excited to share that Databricks has made a strategic investment in Catalyst.

More info here →

Are your customers your growth engine? They should be.

Open article: Databricks invests in Catalyst, targeting the elusive customer intelligence category by venturebeat.com

Databricks invests in Catalyst, targeting the elusive customer intelligence category

20 Power BI interview questions (broken down by experience level ⬇️)

Whether you’re a beginner, intermediate or expert in Power BI, many interviewers are looking for the same things.

Here are 20 of the most commonly used questions to help you prepare for your next #PowerBI interview, courtesy of DataCamp.

💡 Beginner:

- What are the benefits of self-service BI platforms like Power BI?
- Do you know of other BI platforms? How does Power BI compare to them?
- What are some of the disadvantages or risks that we should look out for as we use Power BI?
- What is Power Query, and what is it used for in Power BI?
- What are the different types of filters in Power BI reports?
- What's the difference between a report and a dashboard in Power BI?
- How can we refresh the data in a report published to Power BI Service?

📈 Intermediate:

- What is the difference between DAX functions, expressions, and variables?
- What are circular dependencies? How can you avoid creating circular dependencies in your DAX expressions?
- Describe what a star schema is and how it works.
- Explain the difference between single and bi-directional relationships
- What is cardinality? Describe a common issue with using many-to-many cardinality in a relationship
- We currently have a Power BI report that imports a large volume of data from our warehouse. How would you optimize the performance of a Power BI report for big data?

🔝 Advanced:

- What workarounds exist to avoid using bi-directional filters in relationships?
- Discuss a challenging Power BI project you worked on in a past organisation.
- Which Power BI project are you most proud of having accomplished?
- How can you reduce the risk of data leakage?
- Please share your thoughts on data governance and security policies around user permissions and access.
- Please share your thoughts on data governance and decision guidelines around workspaces, gateways, sharing, administration or privacy.
- How do you think we can best optimise or monitor performance? What about performing a dataset refresh?

⚡ Don’t forget to like this resource to keep it for future reference, and consider resharing to your network or tagging others who may find it helpful!

#data #datainterview #businessintelligence #bianalyst #techinterviews

I help businesses leverage the power of data to enhance their performance. 

At Onyx Data, we work with organisations to use data more intelligently and gain effective insights from data analytics. We also help understand the implications of new technologies such as artificial intelligence and big data to support business growth.

Outside of my work with Onyx Data, I contribute to various publications such as Brainz Magazine, the AI4C and the AI Journal. I’m also a Thought Leader at the Forbes Technical Council and the Global AI Hub.

As a Microsoft Data Platform MVP, I work closely with the entire Microsoft Data Platform suite of products, including Azure Synapse, Azure Data Factory, Azure Data Lake, Data Bricks, Azure Machine Learning, SQL Server, SSIS, SSAS, SSRS, Azure and Power BI from prototyping to deployment. I’m also chair for the Microsoft Power BI – UK community group.

Mentorship and giving back to the data community are incredibly important to me, and so I run monthly data visualisation challenges through #DataDNA. I communicate with individuals from a range of backgrounds as an international speaker and advisor, too.

I have a proven track record in implementing Business Intelligence (BI) on Big Data, Data Warehouse (DWH), Data Lakehouses, Tabular data modeling and Data Analytics-related consulting projects. As well as this, I possess experience in delivering and developing valuable on-premise and cloud-based AI and Data solutions to drive data security, accuracy and process efficiency.

My programming and data knowledge extends to implementing advanced Machine Learning solutions harnessing Jupyter Notebooks and Visual Studio Code, alongside frameworks like PyTorch on Azure, TensorFlow, and Scikit-Learn. 

Please get in touch with me through direct message to see how I can simplify data to help your business grow.

(Please note all posts are my own and do not reflect the opinions of my current, past, or future clients)

It's time to reveal something big that may make me some enemies 👇

We have been scammed by the so-called "data-oriented" hype.

In 99% of companies, data has a negative ROI.

Many people consider me to be a "data guy".

I've worked for 5 years in M&A, on spreadsheets all day long.

Quite comfortable using data, from Excel to SQL.

I don’t get tricked too often.

But recently, I had an epiphany: 99% of data is bullshit!

Data, in the way it's used most of the time, leads to poor decision-making.

→ It rarely happens that data leads you to a decision that good common sense wouldn't have guided you towards.

→ Most people lack an understanding of scales of magnitudes and “cross product,” which usually solve almost problems

→ The amount of time & money invested in creating & maintaining data throughout your company is HUGE and doesn’t stop growing.

→ Paralysis by analysis is real, but ultimately, most decisions come from a deep understanding of customers & market dynamics.

→ Most of the data is reporting - feels safe, but it does not contribute to improvement or progress.

Can you lose weight sitting on the scale? No

You lose weight through discipline.

Now the question is: should we ditch all data usage?

I would say NO, BUT:

→ We should limit ourselves to very high-level KPIs

→ Data teams should be limited in size with only top profiles with strong business acumen

→ And never, ever accept to go through an attribution model

If you want to know the ROI of your ads, just cut your budget for 1 month, and you’ll know.

Please give me 1 example to prove me wrong: what strategic decision you made using data - that you couldn’t have made without it?

#data #tech #startup 

🧑‍💻 Growing Tech companies from M$ to 10´s M$? Follow me and hit 🔔 - I share my learnings doing this multiple times | GTM Strategy and Execution (Sales, Marketing, Growth), Product, Hiring, HR and Leadership

Ready to elevate your data analysis skills to new heights? 🚀 

Join our FREE webinar on PowerBI and unlock the full potential of this extraordinary tool!

🔎 Whether you're a beginner or a seasoned pro, our comprehensive Power BI overview will empower you with the knowledge and skills needed to excel in data analysis.

👉 Calling all business owners, professionals, data analysts, and curious learners! 

This webinar is the perfect opportunity to dive into the power of data analysis.

✨ Seize the moment and reserve your spot now! Take the first step towards becoming a Power BI master. 

Don't miss out on this incredible opportunity – sign up today!

#dataanalysis #powerbi #webinar #datascience 

𝐀𝐫𝐞 𝐲𝐨𝐮 𝐬𝐭𝐫𝐮𝐠𝐠𝐥𝐢𝐧𝐠 𝐭𝐨 𝐠𝐞𝐧𝐞𝐫𝐚𝐭𝐞 𝐥𝐞𝐚𝐝𝐬 𝐚𝐧𝐝 𝐜𝐥𝐨𝐬𝐞 𝐝𝐞𝐚𝐥𝐬 𝐨𝐧 𝐋𝐢𝐧𝐤𝐞𝐝𝐈𝐧?

Have you been sending out messages but getting little to no response? 

Do you want to turn your LinkedIn profile into a powerful sales tool?

If you answered yes to any of these questions, then keep reading.

𝟖𝟎% 𝐨𝐟 𝐬𝐨𝐜𝐢𝐚𝐥 𝐦𝐞𝐝𝐢𝐚 𝐥𝐞𝐚𝐝𝐬 𝐜𝐨𝐦𝐞 𝐟𝐫𝐨𝐦 𝐋𝐢𝐧𝐤𝐞𝐝𝐈𝐧, 𝐲𝐞𝐭 𝐨𝐧𝐥𝐲 𝟐𝟎% 𝐨𝐟 𝐬𝐚𝐥𝐞𝐬 𝐩𝐫𝐨𝐟𝐞𝐬𝐬𝐢𝐨𝐧𝐚𝐥𝐬 𝐮𝐬𝐞 𝐢𝐭 𝐭𝐨 𝐬𝐞𝐥𝐥  

𝐖𝐡𝐚𝐭 𝐢𝐟 𝐈 𝐭𝐨𝐥𝐝 𝐲𝐨𝐮 𝐭𝐡𝐚𝐭 𝐲𝐨𝐮 𝐜𝐨𝐮𝐥𝐝 𝐢𝐧𝐜𝐫𝐞𝐚𝐬𝐞 𝐲𝐨𝐮𝐫 𝐬𝐚𝐥𝐞𝐬 𝐨𝐧 𝐋𝐢𝐧𝐤𝐞𝐝𝐈𝐧 𝐛𝐲 𝐮𝐩 𝐭𝐨 𝟑𝟎𝟎% 𝐢𝐧 𝐣𝐮𝐬𝐭 𝟗𝟎 𝐝𝐚𝐲𝐬?

My LinkedIn Social Selling Coaching program uses a proven system that combines the power of personal branding, lead generation, and social selling strategies.

You'll learn the strategies and tactics used by the top LinkedIn sales professionals, including how to create an effective LinkedIn profile, how to build your network, and how to leverage LinkedIn's features to generate leads and close deals.

With our program, you'll gain a competitive edge in the marketplace, increasing your sales and revenue. 

📌📌 I’ve been in the Freelancing/Consulting industry from over 6 years, and I’ve helped 1700+ freelancers & entrepreneurs earn 1000$$ by leveraging the power of LinkedIn. I know what works and what does not, and I've developed a system that I'm confident will work for you too.

I also have trained over 13000 freelancers & business owners through my Udemy courses.

Don't just take our word for it, hear from some of our satisfied customers (Check my Recommendations)

Imagine what it would be like to have a steady stream of leads coming in every week, to have decision-makers reaching out to you instead of the other way around, and to close more deals than ever before.

If you're ready to take your sales on LinkedIn to the next level, then I invite you to book a free discovery call with me. 

During this call, we'll discuss your sales goals, your current LinkedIn strategy, and how my coaching program can help you achieve your goals.

Don't wait any longer to start seeing results on LinkedIn. Book your free discovery call today and let's get started.

DM me to Book a Free Discovery Call

𝐑𝐞𝐚𝐜𝐡 𝐔𝐬
Gmail: careertribe22@gmail.com
WhatsApp: +92-345-9858-188

𝐂𝐚𝐥𝐞𝐧𝐝𝐥𝐲 𝐂𝐨𝐧𝐬𝐮𝐥𝐭𝐢𝐧𝐠 𝐂𝐚𝐥𝐥: https://calendly.com/ishtiaqmarwat/30min

Disclaimer: Any views published on my profile are my personal opinions and do not necessarily represent PTCL Group's views

Muhammad Ishtiaq Khan

A post about why we have shadow IT in Data and how to reduce it. 🥷🏿 
 
Shadow IT typically happens when the business has to move faster than the speed at which IT (or, in this case, data teams) can deliver. 
 
People cannot wait for a centralised function to deliver what they need, so they take matters into their own hands and MacGyver a solution. 
 
In Data, this happens because: 
 
❌ It’s hard to get to the right data. There's a centralised team responsible for data, reporting and analytics, but they can only do so much. If they don't work on it, it doesn't get done. (Or rather it does - by shadow IT.) 
That is because our data strategy does not focus on building foundations centrally and end-user capabilities distributedly but instead aims to do end-to-end delivery centrally. 
 
❌ People don’t trust the data. Even if they could get to it, it's hard to use it if they don’t trust it. In my experience, if the data others give me looks different to mine, I won’t trust it.  
I know my data and I trust it despite its limitations because I understand it and I know why it looks the way it does. If others can't leave me with the same confidence about their data, I won't use it. This tends to take a lot of time. 
 
How do we avoid the above 2 problems and help cut down on Shadow IT? Here are a few tips: 
 
👉 Focus the centralised data team on only doing two things: 
1️⃣ Delivering high-quality core data sets that stop just before the KPIs / metrics layer. Only integrate data from different systems if it doesn't mean applying logic that is specific to a team (and can differ for another team). 
2️⃣ Automating and enforcing data governance standards such as documentation, lineage, testing and ownership. 
This can be done by putting together the right tooling that helps with self-service and automated release processes. These make it possible for others to create and release datasets (KPIs, metrics, etc.) so long as the governance requirements are met. Reach out to me if you need help here. 
 
👉 Once you have made significant progress on the above, start putting some of your data people on sabbaticals within the business teams. Or upskill business power users. You will have less and less to do on the "core platform" if you automated it well.  
Good self-service tools help tremendously. The idea is to define and implement business logic within business teams to a point where it gets deployed into production. All the while not needing the centralised data team but using the core datasets, tooling and standards they provide. 
 
This helps the business build confidence in the data they see. And move at speed as they don't have to wait for anyone else. We transform shadow IT into proper IT. 


CEO @ Infinite Lambda - Data Academy & Consultancy

Taplio

Top trending Linkedin posts about data

Explore this list of the top LinkedIn posts related to data. Check them out to know what's hot right now.

Top trending Linkedin posts about "data"

Want to drive more opportunities from Linkedin?

Check out other trends: