👋 Hey! I'm a Marketing nerd and spent most of my career at Uber building Marketing Science teams across Brand, Perf, and Lifecycle.

During that time, I partnered with world-class marketing, data science, and martech leaders to help scale and measure the ROI of over $1Bn in Marketing Spend driving $200M+ in incremental revenue.  

After Uber, I worked at early stage consumer startups and scaleups helping founders and marketing leaders make better Marketing decisions with data and experimentation. Across my career, I've been at every stage of companies from seed to Enterprise so there's no Marketing Science challenge I haven't faced. 

Now I share those same playbooks we used to scale and measure ROI with 2,600+ Marketers from companies like Netflix, Spotify, Uber, and Google → https://experimental.beehiiv.com/subscribe

I also consult and advise for CMOs and Growth leaders helping you with complex Marketing Science challenges. 
⮑ For a quick jam session, grab some time here → https://app.hubble.social/sundarswaminathan
⮑ For longer engagements, DM me (no need to connect) to start up a conversation

Lastly, I love spending time with my 2 beautiful kids and my incredible wife. We used to love traveling and eating out but you know... kids ...  so now I love cooking and exploring all the fun things little kids can do!

Thanks for reading and for your time 😊

Sundar Swaminathan

CFO: Our CAC is too high.

New CMO: Which CAC?

CFO: …what do you mean?

CMO: Are we talking blended CAC? Paid CAC? Marginal CAC? Incremental CAC? 

CFO: Isn’t CAC just CAC?

CMO: Nope, there isn’t just one CAC. We should align because if we don't then we'll make the wrong decisions. 

CFO: Go on.

CMO: If we only optimize for the lowest CAC which is blended then we’ll keep pouring money into retargeting, brand searches, and discounts. It looks great on paper, but it doesn't necessarily grow the business. 

CFO: So which CAC should we look at?

CMO: All of them

CFO: What do you mean?

CMO: Blended to understand holistic business performance. We need to compare it to LTV to understand if our business as a whole is reasonable. We can combine that with payback period too. 

I'd ignore Paid CAC because it's based on attribution which we know is wrong.

Incremental CAC by running incrementality tests to cut inefficient spend on channels where we're not actually getting wins.

Marginal CAC to understand where the efficiency frontier is. 

CFO: Okay great. Marginal / Incremental CAC is what I care about most. 

CMO: Exactly. I figured. Let's work together to be more in sync!

CFO: Brilliant. Can't wait.

CMO: Why are we having this conversation at 12:05 AM?

CFO: Money never sleeps.

–
👋 I'm a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing 

🔔 Follow for Marketing Strategy & Science 
♻️ Repost because sharing is caring 
⬇️ Save to send to the C-suite later

CEO: CAC is up 25%. What are we doing?

CMO: I'm asking the same thing. What are WE doing?

CEO: What do you mean? You own CAC.

CMO: Yes, I do but CAC is a cross functional output metric. What are other teams doing about it?

CEO: What other teams?

CMO: My team launches 100 creatives a week, runs experiments, and is on top of budget allocation, but we're blocked. It's now on Eng to help. 

CEO: What can they do to fix CAC?

CMO: 3 things off the top of my head:
1. Product feed is broken or behind. Ads show items that are out of stock. CAC takes a hit.

2. Server-side tracking is broken. We're losing 30% of conversions in attribution. We can't feed the right data back. CAC takes a hit. 

3. Some landing pages load in 4 seconds. Customers are churning before they see a page. CAC takes a hit.

CEO: Okay, I see this. What are they prioritizing?

CMO: When I ask they always say new features or other fixes.

CEO: Why is this the first I'm hearing about this?

CMO: I've flagged it for 6 months. Product team says these aren't revenue-driving features, but they're costing us $125K/month in wasted ad spend. 

I’ve always hit CAC targets but I no longer can.

CEO: SH*T! $125K a month?! Let's fix this now. 

–
👋 I’m a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save to send to your C-suite later

CEO: I need profitable growth. Let's stop brand spend

New CMO: That's going to do the exact opposite. 

CEO: Really? How so?

CMO: It makes performance marketing more expensive short term and hurts long term growth.

CEO: How? Brand is the expensive stuff. Performance drives direct revenue.

CMO: Our MMM + testing shows brand is making performance cheaper. When brand spend is up, our CPCs drop and conversion rates improve.

CEO: Why does that happen?

CMO: Because brand builds awareness. When people already know us, they're more likely to click our ads and convert. Performance capitalizes on that equity.

CEO: So cutting brand hurts performance immediately?

CMO: Within a month to three months, yes. Our performance CAC will climb because we're competing for attention with a cold audience that hasn't been introduced to us. 

CEO: And long-term?

CMO: Six to twelve months out, you'll see volume drop too. Fewer people be searching for us and when they do look for a solution, we won't be top of mind. 

CEO: So we're stuck spending on brand?

CMO: We're not stuck. We're building a Marketing strategy based on your goals. This is the best path to reach it. 

CEO: Hmmmm true. Still a little nervous though.

CMO: Fair point. Shall we do a geo test to show you impact? We had it planned for next quarter but can move it up.

–
👋 I'm a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science
♻️ Repost because sharing is caring
⬇️ Save to send to the C-suite later

Podcast

Listen here: https://www.buzzsprout.com/2508383

Watch here: https://www.youtube.com/@TheExperiMENTALPodcast

Former Head of Marketing Science at Uber who optimized $1Bn+ across Brand, Performance, and Lifecycle → Sharing weekly playbooks used by top Marketers to measure and scale ROI.

Sumitha N.

CEO: Isn't Last Click always wrong? Why use it?

New CMO: Do you have an alternative? 

CEO: What do you mean?

CMO: Do you think I should be using MTA or First Click or some other attribution? I prefer Last Click because it's consistently wrong which is an advantage.

CEO: Consistently wrong doesn't sound great.

CMO: Agree, but focus on the "consistent" part.

CEO: That's philosophical . Break it down for me. 

CMO: Of course. First, let's look at the pros of LC:
1. It's easy to set up
2. It's easy to explain

I prefer LC over Multi Touch attribution any day.

But, my marketing team is smart and knows it's not the truth.

Here's how we use it:
1. Directional signals
2. Compare within platform 
3. Use it to prioritize where to dig.

CEO: Makes sense. 

CMO: Week to week it's over / under estimating to the same degree and so I can tell at a quick glance if something we did moved the numbers in the way we expected. Also, we can use ratios from our incrementality tests to convert LC numbers to actual ones.

CEO: Got an example?

CMO: Yeah. In Google, we were testing increasing bids. 

On a LC basis we expected Google Search to go up for obvious reasons. But, it only went up X% when we were expecting Y%. That's a signal for us that we can dig deeper into.

CEO: Ohhhhh. So using last click isn't an issue?

CMO: No. Believing it's the truth is. We're too smart for that.

–
👋 I'm a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing 

🔔 Follow for Marketing Strategy & Science 
♻️ Repost because sharing is caring 
⬇️ Save to send to the C-suite later

CEO: Woah! We're cutting Google Ad spend?! 

New CMO: Yup. By 65%. It's as I feared...

CEO: Um but but but... the dashboards are showing 6X ROAS. Aren't we shooting ourselves in the foot? It's the best channel we have!

CMO: Ahhhh, welcome to the illusion of attribution. So yeah it does look good "on paper" but that's why we run tests.

CEO: Can you share more?

CMO: Yup! So we ran a geo test where we lowered spend in some markets while keeping others the same. Think of it like an A/B test but with locations instead of randomizing by users. Here's what happened:

1. Revenue was the same
2. New customers were the same
3. Profit was wayyyyy up

In summary, 65% of our spend on Google wasn't incremental. 

CEO: So we were just paying for users that would've converted?

CMO: Yes 😢 

CEO: Why do you think that was happening?

CMO: Oh the classic... we were bidding on branded search which attracts people that already know about us. This makes ROAS look great because:
1. You're bidding on less competitive Brand terms
2. People know us so conversion looks great

⬆️ Conversion ⬇️ Ad spend = 📈 ROAS

CEO: Woah. This is nuts. What's next?

CMO: Well I want to validate the incrementality of Meta. My hunch is that Meta is getting less credit than it deserves. It's a classic dynamic where Meta shows the ads to get people interested then Google converts.

CEO: Just 🤯 . So do we need to redo our attribution too?!

CMO: NOOOOOO! We need to learn to stop being so addicted to our attribution dashboards. 

CEO: Wow, I guess Marketing is more complicated than I thought...

CMO: 😵 

–
👋 I'm a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science
♻️ Repost because sharing is caring
⬇️ Save to send to the C-suite later

CEO: Are we ready for an MMM?

New CMO: Are you ready to be wrong?

CEO: Wrong about what?

CMO: Wrong about "best performing" channels?

CEO: I don't know. I'm lost. Why are you asking?

CMO: So, the worst thing I can do is use $X of my budget on an MMM, run it, see that our "best performing" channel might not be incremental, propose we run a test, and then guess what happens?

CEO: We run the test and growth drops?

CMO: No! My biggest concern is that you'd block us from running the test because it might hurt growth. 

If growth falls, that's a great thing! It's a sign. It's data. It's progress.

CEO: Oooof fair. I feel like I'd be okay with that, but idk! What else do we need to do to be ready for an MMM?

CMO: Are we ready to move budgets up and down to introduce volatility so the MMM can pick it up? 

Are we ready to stop making so many decisions using Last Click? 

Are we ready for Last Click and MMM to have conflicting signals?

CEO: Idk! I've never thought of this. What do we do?

CMO: Let's first make sure we have the culture to support experimentation and more data driven decision making. What if I ramp down spend by 15% on Google Search because my hunch says it's not incremental?

CEO: I'm down to try but just need you to keep in the loop because I'm definitely nervous.

CMO: Good. Nervousness = growth.

CEO: You sound like my high school coach...

–
👋 I'm a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science
♻️ Repost because sharing is caring
⬇️ Save to send to the C-suite later

CEO: What's the diff between Performance vs Brand?

CMO: Sooo glad you asked. Here it is...

CEO: Ooooh I got my notebook out...

CMO: Ready?! Drumroll please...

CEO: Omg, I'm so excited to finally understand!

CMO: THERE. IS. NONE. 

CEO: WHAT?! Why do I keep hearing about it?

CMO: I have no idea but there is no real difference especially for our customers. 

CEO: What do you mean?

CMO: When you look at ads, do you go "Woah! This is a Brand Marketing ad... now I have more mental availability" or do you go "Woah! This is a Perf Marketing ad... I need to click it!"?

CEO: Hahahah no not at all.

CMO: Exactly, but also think of it another way. You can see a "Brand" ad and then decide to purchase because you were in a buying moment. You can also see a "Perf" ad and then log it in your mind for later. 

CEO: So, how should we be thinking about this?

CMO: We have channels and we have objectives. Our goal is to deploy budget on the channels that are the best at driving the objectives we want. 

CEO: Got an example?

CMO: Youtube looks terrible on a CAC basis but the objective of Youtube is usually to build awareness and consideration at a large scale. 

But if we didn't have Youtube then our Meta lower funnel CAC would spike. 

Channel → Objective
Youtube → Increase consideration
Meta → Convert when ready 

They work together.

CEO: That is SO much simpler than we make it out to be.

CMO: Yeah, not sure why us marketers keep shooting ourseles in the foot. 

–
👋 I'm a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing 

🔔 Follow for Marketing Strategy & Science 
♻️ Repost because sharing is caring 
⬇️ Save to send to the C-suite later

Dr Dre and Eminem - but about Marketing Science. 

We're making things too complicated.

Talkin' about use this metric and that metric.

Shoulda kept it simple.

Y'all forgot about M.R.E.

The CAC Album dropping July '26. 

Inspired by Henry Hayes and Tycho Luijten who are the real OGs. Still got a long way to be even close to your level but I'll get there.

Also Cian Weeresinghe, I've never met you but I think you'll appreciate this?

CFO: Google Ads ROAS is 5X! More budget there?

New CMO: How sure are you that it's 5X?

CFO: What do you mean?! I thought ROAS was 5X?

CMO: No! You've been bamboozled! 

CFO: Bamboozled?! How?

CMO: Platform attribution is misleading. Google Ads is notorious for looking better than it really is so we paused it just to see what happened.

CFO: ... and what happened?! 

CMO: We picked up 3 signals that confirmed our suspicions. 

1. The “pause bump”
We paused it and all the other channels picked up the credit. Google Ads is notorious for being over attributed. 

2. The “too good to be true ROAS”
In-platform ROAS looked too good. It felt fishy and we confirmed our suspicion that it’s not as incremental as we'd expected. 

3. The “flatline paradox”
Our total business KPIs stayed flat even as channel performance skyrockets. That only happens if our channel was non incremental. We were confusing activity for impact.

CFO: So what do we trust instead?

CMO: Nothing blindly. We triangulate. Blended CAC and ROAS for business sanity, incrementality tests for truth, MMM for calibration, and platform numbers for steering. But, we don't confuse platform numbers for truth. 

CFO: So attribution isn’t reality?! It’s just one opinion.

CMO: Yes. And like most opinions, it gets louder the more wrong it is. Sound like someone you know? 👀

–
👋 I'm a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing 

🔔 Follow for Marketing Strategy & Science 
♻️ Repost because sharing is caring 
⬇️ Save to send to the C-suite later

CEO: Is the brand marketing campaign working yet?

CMO: What do you mean by working?

CEO: Is it driving business metrics? ROI?

CMO: No, but that doesn't mean it's not working. 

CEO: What do you mean?

CMO: Well we're in month 1 of the campaign and I told you I didn't expect to see business impact until month 3. 

CEO: Why is that? 

CMO: Because of how our customers purchase. We have a product that people really only buy once a quarter. So when we launched our brand campaign we started to introduce our product to users but most of them will only be ready to buy in the next few months.

CEO: But, I thought it was demand "Generation" 

CMO: Ugh. That is such a stupid term. Here's what's actually happening:

1. A customer wants to buy a thingamabob
2. Before, they didn't know about us or that we sold them
3. Now, they've seen us and hear about us and know we sell thingamabobs.
4. When they're ready, they'll think of us and hopefully buy thingamabobs from us. 

We can't magically make them want a thingamabob.

CEO: Oh my god. That makes total sense. So it's not really demand generation?! 

CMO: No, it's Staytopofmindforwhentheyreready-tion

CEO: We need to find a better term for that... 

CMO: Yeah, it's called Mental Availability. 

–
👋 I'm a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science
♻️ Repost because sharing is caring
⬇️ Save to send to the C-suite later

CEO: What's your 90 day Marketing plan?

CMO: Become BFFs with Product and Finance.

CEO: Hmmm... not what I was expecting but explain?

CMO: Sure. Let's be honest here. The CMO role is in the hotseat from day 1 and there's two reasons for that:

1. It doesn't look like we're having an impact
2. It looks like we're too expensive

Sound accurate?

CEO: Yeah, I guess. So how does aligning with Product and Finance change that?

CMO: I can only Market what Product is building / selling. I need to undertand our vision, roadmap, and customers. I also need to know who owns pricing and likely will have some product asks so I want to build a great relationship with Product from day one. 

Finance is an obvious one but we need to build a collective understanding of how to measure impact of Marketing. I also need to sync regularly with them so we make investment decisions and keep them in loop so they're not surprised .

CEO: Okay seems fair, but I was kind of expecting a "quick" win. 

CMO: Do you care more about actions or impact? I have no doubt that I'll be able to identify a quick win but how often are the most impactful problems easily solvable? 

CEO: Basically never... 

CMO: Right... so it's no different in Marketing. I can't just "tweak" positioning here or copy there and expect huge wins. So, for me to last in this role, have the biggest impact, and deliver on what I believe I can , I need to align with Product and Finance.

–
👋 I'm a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science
♻️ Repost because sharing is caring
⬇️ Save to send to the C-suite later

CEO: Can you cut our CAC by 30%? 

CMO: I love a fun challenge. By when? 

CEO: …. by next month. 

CMO: Oof I can do it but it’ll hurt growth. 

CEO: No growth must go up.

CMO: Lol what? Okay Eng can ship some asks. 

CEO: No their backlog is full. 

CMO: Okay… We’ll try a lot of new creatives.

CEO: Need to let designer go. Budget cuts. 

CMO: Okay. Will run incrementality test on Google

CEO: No! That’s our Golden Goose!!!!!

CMO: Wtf do you want me to do? 

CEO: Idk. That’s your job to figure it out. 

–
👋 I'm a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing 

🔔 Follow for Marketing Strategy & Science 
♻️ Repost because sharing is caring 
⬇️ Save to send to the C-suite later

CEO: I want us to gain 10% market share this year!

CMO: I want to be excited, but I know there's a but...

CEO: But... we are cutting your budget 50%.

CMO: Say what? 

CEO: And your headcount by 75%.

CMO: Huh?!

CEO: And I don’t like TikTok or IG. So don't advertise there. 

CMO: Okay…

CEO: Also, I will be blindfolding you. 

CMO: WTF. Why? 

CEO: When you lose one sense your others get stronger. 

CMO: This is madness…

CEO: THIS. IS. MARKETING!

–
👋 Hey! I'm a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Perf, and Lifecycle Marketing

🔔 Follow for Marketing Strategy and Science
♻️ Repost because sharing is caring
⬇️ Save to send to the C-suite later

CMO: I'm struggling with our Data Science team

New CDO: What are you struggling with?

CMO: Idk, I can't get what I want out of them.

CDO: Sorry, but it's likely you.

CMO: Why do you say that?

CDO: It's usually strategy, comms, or skillset. 

CMO: That's pretty broad. Elaborate?

CDO: Of course. Let's start with Strategy.

Have you shared your Marketing Strategy? 

What bets are you making?
What audience are you going after? 
What audience insights are you using? 

Without context, data is just a jumble of numbers.

The team performs better when knowing what and where to look. 

CMO: Fair point. Communication?

CDO: How often do they talk to your team?

How often are they in your leadership meetings? 

If you meet 1x a month, it's weeks too late. They're operating on stale information. You need to meet with them weekly AND treat them like a part of your team. You're their partner not stakeholder.

CMO: Woah. Okay. And Skillset?

CDO: Generalist rarely succeed because there's too much business and channel knowledge required.  

Also, don't do pooled resourcing. 

Analysts should have clear lanes of scope. 

CMO: Shoot, might be a combo of all 3. What should I fix first? 

CDO: Fix comms + strategy. Then skillset. Even the best data scientist are going to stumble if they don't know where your heads at. 

CMO: Okay, this sounds promising. 

–
👋 Hey! I'm a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Perf, and Lifecycle Marketing

🔔 Follow for Marketing Strategy and Science
♻️ Repost because sharing is caring
⬇️ Save to send to the C-suite later

CEO: We need to fix our Marketing!

New CMO: Why do you think it's not working?

CEO: We're spending too much. Our CAC is too expensive.

CMO: I hear you. What do you think we should do?

CEO: Definitely cut our Brand Marketing spend.

CMO: Got ya. What's our growth rate?

CEO: It's fantastic! 50% YoY.

CMO: Nice. And what was it before we started Brand Marketing? 

CEO: Like 40%

CMO: So we're growing more? 

CEO: Yeah but so much more expensively. 

CMO: How long has the Brand Marketing campaign been running?

CEO: Idk. Probably 1-2 months. 

CMO: And our customers buy 1x every 3 monthish?

CEO: Yeah, usually! 

CMO: Okay great. So I don't think we should change a thing yet.

CEO: What?! 

CMO: We need Brand Marketing to do it's thing. I'll dig in more but high level the impact of Brand Marketing hasn't start to hit. Our growth rates will go up even more and likely our CACs will come even further down than before. Here's what it'll look like compared to last year:

✅ Lower CAC
✅ Higher YOY growth
✅ Higher acquisition volume

CEO: So we've been doing the right thing?

CMO: Yeah. I believe so. Who was leading these efforts before me?

CEO: Jane Smith. Why?

CMO: I'd like to give her a raise.

–
👋 I’m a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save to send to your CEO later

My next guest is who the world's biggest CMOs call on when they need to figure out their AI Marketing strategy. 

Nuno Leal, EY Studio+ CMO Practice Leader for the Americas, joined me to break down how the best CMOs are deploying AI and why marketing orgs are stuck in AI purgatory and.

We chat about:
⮑ Why the future of marketing needs 2 CMOs
⮑ How to show Gen AI ROI in months, not years
⮑ Why companies aren't ready for pattern changes

My favorite discussions:
⭐ The smartest CMOs are using the AI hype cycle to finally get CFO buy-in for the data infrastructure they've always needed. 

⭐ Gen AI is production-ready today in text based communications so that's where the quickest ROI is. 

Example: One of Nuno's clients saw 70–80% traffic growth from content they already had with zero ad spend by building an AI content system.

⭐ As agents take over transactions, consumers will spend more time evaluating brands emotionally. 

If you're not investing in brand, you're going to be invisible when it matters most.

If you're a CMO or growth leader trying to figure out where AI actually fits in your roadmap ( and how to prove ROI fast), this conversation is for you!

Nuno brings a rare 30,000-foot view of what's actually working across the world's largest marketing organizations right now. 

Tune in wherever you love listening to podcasts ( links in the comment).

Founder: Why do you say platform # are wrong? 

Me: Do a quick math exercise for me? 

Founder: For sure. What’s up? 

Me: Add up the acquisitions from Meta and Google. What is it? 

Founder: 350.

Me: And what does Stripe say? 

Founder: 308! Wtf?! 

Me: Haha don’t worry it’s not that bad. Meta and Google don’t always see the other channel's journey. They’re just giving credit to what they believe they drove. 

Founder: So what should we do?! 

Me: First, don’t trust any attribution blindly whether it’s platform or internal. 

Second, look at overall numbers. Are you happy with your LTV, CAC, and growth rates? 

Founder: Yes for the most part. 

Me: Okay so keep going for now. At some point you’ll want to care about incrementality but you don’t need to yet. You’re still likely incremental on these channels. 

Then, make sure you’re not making decisions purely on attribution. Use in platform numbers to optimize the platform and then overall blended numbers to steer the business.

Founder: Give me an example? 

Me: Yeah. Right now you’re spending $100 on Meta. Double it tomorrow. 

Give it a week and see what happens to your blended CAC. You can make some inferences there. Don’t purely trust platform . 

Founder: Ohhhh okay ! That makes sense. So platform for directional and blended for truth. 

Me: Yes but just remember blended doesn’t mean incremental but we’ll get there eventually.

–
👋 I’m a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science 
♻️ Repost because sharing is caring
⬇️ Save to send to the C-suite later

CEO: Got website feedback from a Marketing friend.

New CMO: Nice. Are they our target customer?

CEO: Hm... no not really. They've just got a lot of experience.

CMO: Do they work in or advise in our industry?

CEO: Well, not exactly, but broadly speaking kind of.

CMO: Have they worked with us before?

CEO: No. We were just chatting at dinner and then I asked about their thoughts so they took a quick look.

CMO: Okay. I really appreciate you sharing, but that "quick look" is not super helpful and actually it's going to hurt us.

CEO: Oh no. How so?

CMO: First, it makes me look defensive because 99% of the time I'll probably say no we're not going to do that. 

Second, our problems don't need "quick looks" . They're deeper rooted and holistic and it's my job to work through that. 

Third, everyone things they can do Marketing. Even Marketers. But there's too much context and understanding of our customers required and I need the space to just think and execute.

CEO: Oh damn. This all makes sense.

CMO: I know it's well intentioned and I'll create the space for us to have these chats and for you to share your opinions / ideas, but it's too early for that.

–
👋 I'm a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science
♻️ Repost because sharing is caring
⬇️ Save to send to the C-suite later

CEO: You made it go viral! Let's do more.

CMO: It was awesome! But I didn't "make it go viral" 

CEO: What do you mean? We hit 3M impressions!

CMO: I know. I'm saying I specifically can't do it again.

CEO: Why? Can't you just replicate that success? 

CMO: That's the problem. You can't replicate virality.

CEO: Really? Why? We know what worked. Same creative style, same platform, same targeting.

CMO: Well, virality isn't a specific formula. It often feels like It's lightning in a bottle. I've had so many times where I thought it would go viral and it flopped and countless times when it was the opposite.

CEO: So we just got lucky?

CMO: Partially, yes, but don't make it sound like that. We can't always control timing, cultural moment, algorithm quirks. 

CEO: But we still made good creative. That has to count for something.

CMO: It does. Good creative is necessary but not sufficient for viral. You also need the perfect storm of shareability and timing.

CEO: What if we just make ten campaigns and hope one goes viral?

CMO: Then you're gambling, not marketing. And the other nine campaigns that don't go viral still cost money.

CEO: So viral is worthless?

CMO: No, it's amazing when it happens. But you can't build a marketing strategy around hoping for it.

CEO: What should we build strategy around?

CMO: A consistent content system. We study formats and ideas that have made others go viral, then show up regularly with quality content. Some will hit bigger than others. That's exactly what happened this time. 

CEO: So we're not chasing viral, we're just showing up consistently?

CMO: Exactly. Viral is the byproduct of doing good work consistently, not the goal. The companies that obsess over viral end up with spiky, unpredictable results.

CEO: That makes sense. Consistency over home runs.

CMO: Right. And when something does take off, we'll know how to capitalize on it because we have a system in place.

–
👋 I'm a former Uber Head of Marketing Science where I optimized $1Bn+ across Brand, Performance, and Lifecycle Marketing

🔔 Follow for Marketing Strategy & Science
♻️ Repost because sharing is caring
⬇️ Save to send to the C-suite later

By 30, I was a multimillionaire .

Here's the exact blueprint:

Step 1: Inherit $2.9 Billion dollars.

Step 2. Launch a 4 sided consumer marketplace.

Easy business model. Plus 4 sides is more upside.

Pun intended.

Step 3: Hire a CMO pre PMF. 

They'll help me find it.

Step 4: Fire CMO even though they said they were underresourced, under listened to, and I kept overruling their decisions . 

All I heard was excuses. 

Step 5: Set up Last Click attribution for all sides of the marketplace.

Clicks over d*cks. 

Step 6: Steer based on platform attribution because internal attribution was broken. 

OOH campaigns weren't showing up. 

Plus, Meta and Google only share numbers in the best interest of their customers. 

I know Mark, Sergey, and Larry personally.

They'd never lie to me. 

Step 7: Bid aggressively on Branded Search. ROAS was INSANE. Like 10x+.

Gotta spend money to make money. 

Step 8: Use revenue to model out LTV and focus only on 5 year LTV because that's where LTV / CAC was 3x. 

Margins are for the marginalized.

Pay back? You need to stay back. 

Step 9: Invest more money when low on runway because LTV / CAC was too good so I just needed to wait it out.

Step 10: Focus religiously on Perf Marketing. Brand isn't measurable. Perf is. We knew EXACTLY where every dollar was going. 

Step 11: Declare Bankruptcy after investing $2 Billion Dollars + at peak 2021 valuations because somehow I was never able to become profitable even though my attribution, LTV / CAC, and ROAS were all saying I was.

Still not sure what went wrong, but who cares?

I'm about to launch my AI based crypto web 3 trader on openclaw.

I'll let you now how it goes.

I help companies scale high-impact experimentation programs that actually drive growth (not just more tests or prettier dashboards).

Over the past decade, I’ve led data and experimentation teams at some of the world’s most product-led companies including Meta, Canva, and Booking.com. I’ve helped teams design better experiments, make smarter decisions, and build the platforms and cultures that support real velocity.

These days, I work as an independent consultant out of Sydney, helping mid-to-large APAC companies turn experimentation into a growth engine. I bring a mix of hands-on support, strategic guidance, and proven frameworks developed from my experience like NAV (a practical lens to measure and improve the Number, Accuracy, and Value of your experiments).

My services include:

🧠 Experimentation strategy sprints
🧪 Fractional leadership and advisory
📈 Workshops & other custom projects

If you're trying to scale experimentation and want an experienced partner who's done it at the highest level, let’s talk.

➡️ Learn more at drsimonj.com

Simon Jackson

Many amazing people across the Australian tech community were impacted by recent layoffs at Atlassian (and earlier at Block).

That’s tough news for the people directly affected and, honestly, not great for the Australian tech ecosystem either.

We don’t have many companies here operating at that global scale. I certainly felt that moving back to Australia after working in big tech in the US and EU.

If you’re one of the people affected, I’m sorry you’re going through this.

If it’s helpful, I’m always happy to:
 → chat about what you’re looking for next
 → connect you with people in my network

My DMs are open.

I’m also building something new and growing a founding engineering team here in Australia.

If that’s of interest, or you just want to connect, feel free to reach out.

Dr Simon Jackson

Scaling Experimentation 🚀 Ex-Meta, Canva, Booking.com

Most explanations of statistical significance suck.

So here's how I teach it 👇

⬇️
Imagine I randomly split people into two groups and measure their average height.

Would the averages be identical?

Of course not.

Random groups naturally end up a little different.

Sometimes it’s just chance.

Sometimes it’s because a few NBA players happened to land in one group.

Either way, the average height looks different.

That’s not a meaningful SIGNAL.
It’s just random NOISE.

⬇️
Experiments work the same way.

Even if your change has zero real impact, the two groups will still show slightly different results.

So the real question isn’t:

“Are the groups different?”

It’s:

“Are they SO different that it’s unlikely to be random noise?”

⬇️
That’s what statistical significance answers.

It:
→ calculates the probability the difference is just noise  
→ compares it to a threshold (often < 5%)

If the probability is lower than the threshold, we call the result statistically significant!

In other words:

The difference is too big to blame on random chance or a few NBA players landing in one group.

⬇️
This concept is critical in experimentation, but it trips up a lot of teams.

So I ran a webinar explaining experimentation statistics in plain English.

You can still grab the recording if it’s useful (link in the comments).

A lot of people ask if getting my PhD was useful.

The honest answer: yes… but not why you think.

People assume a PhD gives you more techniques.

It does.

But the real value is learning how those tools work:
their strengths, their limits, and when they stop being the right fit.

👉 In other words: how to think when the standard methods stop working.

That gap shows up quickly when problems get hard. 

⬇️
Early in my career, I joined an experimentation team where small samples and skewed data made everything harder.

Standard A/B testing wasn’t working.

The solution was CUPED: a variance reduction technique that uses historical data to boost sensitivity without needing more traffic.

Same experiment.
More signal.

⬇️
At another job: different problem, same pattern.

How do you run a pricing experiment in a dynamic, three-sided marketplace?

You don’t split users.

You run switchback experiments.

Instead of assigning different users to treatments, you alternate the change across time windows: on, off, on again.

Same population. Different moments.

A cleaner way to isolate the effect.

⬇️
Neither solution was improvisation.

Both came from recognising the shape of the problem and knowing which research tool actually fits.

That’s the real value of academic training.

It doesn’t just give you more tools.

It changes how you see the problem in the first place.

Seeing a 40% conversion uplift is not exciting.

It’s suspicious 👇

⬇️
A client was scaling their experimentation program.

They accelerated.
Then suddenly, a huge win 🎉

On paper, everything looked perfect.

But the more surprising the result, the more cautious you should be.

⬇️
There’s a principle in stats called Twyman’s Law:

👉 If a result looks too good to be true, it probably is.

So we pulled the full dataset.

There it was: a Sample Ratio Mismatch.
The traffic split was off and a tracking issue had inflated the uplift.

Once corrected, the “40% win” disappeared.
The true result? Neutral.

⬇️
This is why data validation and guardrails matter.

Before you celebrate uplift, check:
→ Is the sample split correct?
→ Are events firing as expected?
→ Does the result make behavioural sense?

Big wins happen.
Broken data happens more.

⬇️
I’ve seen this enough times that I built a simple, free SRM checker alongside my power calculator.
Link in comments if you’d like to use it.

When a result looks too good (or too bad) pause before you ship.

Check the plumbing.

Most teams plan every A/B test in the same way but they don't need to.

It really depends what's at stake...

⬇️
High stakes: be rigorous

I was part of a team deciding whether to remove an entire product category from their platform.

That decision would affect margins, cannibalisation risk, and years of prioritisation.

At a multi-billion-dollar scale.

We built an extremely rigorous experiment.
The insight we got shaped the business for years.
Worth every bit of the extra time.

⬇️
Low stakes: move fast

Now compare that to testing the order of page links in your site footer.

Low traffic.
Low consequence.
Even if you're wrong, nothing critical changes.

Run a quick A/B test. Learn something. Check you’re not breaking anything. Move on.

⬇️
Treating every experiment the same way means you're either wasting time or taking unnecessary risks.

Before you debate methodology, ask one question:

"What's the cost of being wrong?"
Or, I like to flip it and ask:
"How much would you pay to know the answer?"

Set your rigour accordingly.

I was hired by a team drowning in just a few tests at a time. They're now smashing out 50+ every month. All it took were two main changes 👇

⬇️
Their structure was a mess.

A central service team supporting everyone, which sounds collaborative until you see what it creates:
→ Waiting on approvals from multiple stakeholders
→ Context-switching between unrelated problems
→ No-one with authority to break prioritisation ties
→ Attention spread too thin to go deep on anything

No autonomy = slow decisions
No focus = shallow work

We made 2 changes with a big impact.

1️⃣
Change 1: Cross-functional teams

We set up proper cross-functional teams, each owning the full cycle (analysis → ideation → execution → rollout) without any external dependencies).

2️⃣
Change 2: Clear focus

Each team got one primary objective (e.g. customer acquisition) and up to three specific problem areas to own within it (e.g. onboarding friction, pricing clarity, ease of use).

Suddenly, every experiment had a home.

⬇️
In just a few months, they were shipping experiments 5x faster.

Autonomy and focus aren't nice-to-haves when you scale experimentation. They're the entire game.

If I could steal one thing from the experimentation cultures at Booking, Meta, and Canva, it would be this 👇

I’ve been lucky enough to help grow experimentation at all three.
Each excels in its own way.

⬇️
*Booking taught me: Autonomy*

You can take an idea, build an experiment, and ship it.

No approvals.
No gatekeeping.

Leadership trusts you to do your thing.
If you fail, you own it.

Huge trust = huge pace of learning.

⬇️
*Meta taught me: Infrastructure*

The infrastructure removes friction at scale.

Everything is built to move fast.
Not just individual teams, but the whole organisation.

World-class internal tooling.
Knowledge shared widely.

You’re handed a rocket ship and told to go fast.

⬇️
*Canva taught me: Storytelling*

Canva is a visual communication company.
The experimentation culture reflects that.

The storytelling mindset is everywhere.

A well-communicated result travels faster and further than a technically perfect one.

⬇️
If you have:
• The freedom to try ideas
• The infrastructure to test them
• The ability to communicate results clearly

You’ve got the whole game.

"Just roll out to 10%" isn't actually a safe way to run an experiment.

It just slows you down 😬

⬇️
Most teams treat experiment exposure like a risk dial.

Turn it down to 10%? You're being cautious.
Crank it to 50%? You're being reckless.

But this framing collapses 2 inputs into 1 number.

⬇️
Input 1: Population included (0–100%)
Input 2: Control vs test split (100% control to 100% test)

When you separate these, the "10% debate" looks completely different.

⬇️
Most teams get this wrong:
They think they're choosing between safe and risky.

In reality, they're choosing where to sit on a tradeoff surface between learning speed and business impact.

The best choice?

⬇️
I break it down in this week's newsletter (with visuals to explore your own setups).

Link in comments 👇

I watched a 2-week test turn into a 6-week debate.

Here's where they went wrong 👇

⬇️
Most teams I work with don't have a decision-making problem.

They have a pre-decision problem.

When results come in, everyone starts debating what the data means instead of what to do with it. Leadership gets looped in. Adjacent teams weigh in. Weeks pass.

The fix happens before the experiment starts.

⬇️
Pre-register three types of metrics:

→ Primary: the one number driving your decision
→ Secondary: the 2-3 numbers you expect to move in a specific direction
→ Guardrails: what can't break (think performance, fraud, core UX)

Then write the decision rule upfront:

"If primary improves, secondaries move as expected, and guardrails hold, we ship. Otherwise, we iterate or stop."

When results come in, you follow the rule. That's it.

⬇️
The hard work isn't analysing the data.

Agreeing on what success looks like before you start?
That’s the hard part.

Everyone celebrates scaling to 100 tests per year.

No one talks about what breaks along the way 👇

⬇️
Most teams scale up experimentation, but don't scale the systems around it.

The result? Everything starts breaking down.

Here are the big three I see repeatedly.

1️⃣
Territorial pushback

When your experimentation speed jumps, everyone notices... including senior leaders in other parts of the business.

"Why are you testing in our area?!"

The fix:
→ Demonstrate wins early and often
→ Show you have guardrails to protect their concerns
→ Pull leadership into conversations before they push back

2️⃣
Tracking chaos

Remembering 2 experiments? Easy.
Remembering 20? Or 100? Impossible.

Without structure, teams lose track of what's live, what won, what lost, and what actually shipped.

The fix:
→ Don't wait until you're drowning to build the system
→ Set up your tracking structure from day one (your future self will thank you)

3️⃣
Production bottlenecks

You've got five wins ready to ship.
But engineering can't keep up.
All that great work, stuck in a queue.

The fix:
→ Sync capacity with whoever owns production early
→ Get an engineer on the team part-time if needed

⬇️
The irony is real: the better your experimentation program performs, the harder it gets to manage.

taplio

The Battle

Sundar Swaminathan

Simon Jackson

Followers

Posts

Engagements

Likes

Performance ScoreSundar Swaminathan's LinkedIn Performance

Performance ScoreSimon Jackson's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Sundar Swaminathan

Simon Jackson

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Sundar Swaminathan's LinkedIn Performance

Performance ScoreSimon Jackson's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreSundar Swaminathan's LinkedIn Performance