Ben Labay

Speero

CXL, and by extension Speero, is an inspired workplace, constantly pushing and questioning services and products and what works and what doesn't. The agency pushes on experimentation and customer experience research for SaaS, Ecommerce, and Lead Gen clients around the world. 

These organizations and teams have business and customer strategies, we test them. Ben works within Speero to scope research and test program strategy for companies like Proctor & Gamble, ADP, Codecademy, MongoDB, and Toast. https://speero.com

Siglo Group

Siglo Group is an environmental design consultancy dedicated to linking conservation into land planning and design practices. The firm's work has contributed to land being set aside in perpetuity for conservation, the restoration of public and private properties, and the creation of sustainable land use policy.

Associate and Consultant

The University of Texas at Austin

Conservation science work for the Texas Biodiversity Center, focused on research and data modeling to assess and plan for imperiled species and natural resource conservation initiatives. See research work here https://www.researchgate.net/profile/Ben_Labay

Research Scientist - consultant

Zara Environmental LLC

Researcher

CEO at Speero | Experimentation & Research Program Leader

I'm Ben, the CEO of Speero, a global experimentation and customer experience agency that helps SaaS, Ecommerce, and Lead Gen clients grow their businesses. I have over 8 years of experience in leading and managing research and testing programs for companies like Miro, Decathlon, Proctor & Gamble, ADP, Codecademy, MongoDB, and Toast.

As an experimentation consultant and recovering research scientist/academic, I combine data and stats principles, UX and usability research, and digital marketing to create and execute evidence-based strategies that optimize customer journeys, increase conversions, and generate insights. 

You can follow me on Linkedin, Twitter @benjlabay, Researchgate @Ben_Labay, or email me at ben@speero.com.

Wow Airtable well done. The future is now with AI Experimentation Ops Systems (XOS). 

Check out these prompts in the clip here for our XOS sandbox template (watch to the end with the dashboard creation!): 

• What tests have we run on the checkout page?
• Based on insight field, what are the most successful tests we have run?
• Is there a theme to our successful tests, what approach should we take when developing new ideas?
• Add a new field to this view that provides a one sentence snapshot of the current status that should auto populate with any pipeline updates and any blockers.
• Create an interface that provides month-over-month reporting and covers all aspects of the program. 

The key to making this work, an org process to produce quality structured data. It's not sexy until it is.

Next step, we're integrating an n8n automation to have our test analysis debate club (two LLMs debating, a third LLM as judge) feed the analysis into AT. h/t Vasisth S 

Our blocker atm is that so many of our enterprise clients can't even themselves touch AI... we're not allowed to roll this out for half our client bases. 

AI orchestration and integration into workflows needs to start someplace...I see now better place than at the heart of the learning management system. From there, can connect to analytics, testing, CS, sales databases. 

Airtable AI doesn't even have to power this. We've been playing with locally hosted versions of this that do the same thing (not as slick and plug&play as AT native tho...yet). I'm sure Shawn David would help here too.

We're an enterprise service partner with Airtable, so we're heavy users, but you can do the same setup with other 'knowledge management' tools that would have relational databases. I think task and workflow management tools (e.g., Jira, Asana) are next in line with integrations. Eventually from one console (e.g., curser, lovable) a knowledge worker can use english to build, update, report across all tools?? So fun. 

This will be a big topic at EXL2025, the Austin conference in Oct. Lots of talks on AI and Ops and their connection to experimentation. 

and props to Jason Lively for pushing on this!

CEO @ Speero | Experimentation for growing SaaS, Ecommerce, Lead Gen

Boom, ungated Experimentation Operating System (XOS) dashboard -> program velocity and outcome FOCUSED:

Check it here (if you do you must comment below your thoughts!....its only fair!) https://lnkd.in/gXGTJ5bm

Dashboards can be unsexy and unused, yes. But we still must try...this one, I'd argue helps: 

• 𝗕𝗼𝗼𝘀𝘁 𝗣𝗿𝗼𝗴𝗿𝗮𝗺 𝗩𝗲𝗹𝗼𝗰𝗶𝘁𝘆: XOS helps you measure and accelerate your experimentation program's speed, tracking key metrics like ideas submitted, tests launched, and wins by month.

• 𝗗𝗲𝗺𝗼𝗰𝗿𝗮𝘁𝗶𝘇𝗲 𝗥𝗲𝘀𝗲𝗮𝗿𝗰𝗵 & 𝗜𝗻𝘀𝗶𝗴𝗵𝘁𝘀: XOS systematizes customer insights and connects research to experiments, making data accessible across your business and fostering a learning culture.

• 𝗘𝗻𝘀𝘂𝗿𝗲 𝗖𝗿𝗼𝘀𝘀-𝗧𝗲𝗮𝗺 𝗔𝗹𝗶𝗴𝗻𝗺𝗲𝗻𝘁: It provides a blueprint for standardized processes and a common language for experimentation, promoting collaboration and reducing gaps between teams.

• 𝗖𝗼𝗻𝗻𝗲𝗰𝘁 𝘁𝗼 𝗕𝘂𝘀𝗶𝗻𝗲𝘀𝘀 𝗚𝗼𝗮𝗹𝘀: XOS links your experimentation efforts directly to overarching business objectives, ensuring your tests drive meaningful impact and support company goals.

• 𝗟𝗲𝘃𝗲𝗿𝗮𝗴𝗲 𝗔𝗜 𝗳𝗼𝗿 𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆: XOS is driven by AI to power workflows, dashboards, and learnings, helping automate and accelerate your experimentation process.

• 𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺 𝗜𝘀𝗼𝗹𝗮𝘁𝗲𝗱 𝗧𝗲𝘀𝘁𝗶𝗻𝗴 𝗶𝗻𝘁𝗼 𝗦𝗰𝗮𝗹𝗮𝗯𝗹𝗲 𝗚𝗿𝗼𝘄𝘁𝗵: XOS moves organizations from "ship and hope" to an iterative, learning cycle, turning individual tests into a compounding growth system.

• 𝗚𝗮𝗶𝗻 𝗗𝗲𝗲𝗽𝗲𝗿 𝗨𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱𝗶𝗻𝗴 𝗼𝗳 𝗥𝗲𝘀𝘂𝗹𝘁𝘀: The XOS dashboard provides insights into the financial impact of your experiments, including potential revenue found or saved, and helps you understand the portfolio of bets you're taking.

• 𝗙𝗮𝗰𝗶𝗹𝗶𝘁𝗮𝘁𝗲 𝗖𝗼𝗻𝘁𝗶𝗻𝘂𝗼𝘂𝘀 𝗜𝗺𝗽𝗿𝗼𝘃𝗲𝗺𝗲𝗻𝘁: XOS supports ongoing retrospectives by tracking velocity and results, allowing teams to analyze the ebb and flow of their experimentation program.

Building AI into our XOS (Experimentation Operating System) for clients. 

(Template and how-to blog post coming out soon)

We're coming in from two angles, 

1. integrating native capabilities in Airtable, we're working with them as a service partner on this. (the gif below)
2. a standalone LLM app that APIs into Airtable but also any other datasource. This makes it more flexible for federated teams especially. Can also send data into the table this way ;). This is the ultimate direction as I don't want to be locked down with a tool (Airtable), as a consultant firm we need to adapt to client stacks...as much as possible not force 'another tool' onto them. 

I'm not really wanting to get into the software biz, but the XOS is the one area I'm especially excited about. And this really isn't a new SaaS, rather a service-enabled product (for a while it would be custom setup for every application). Exciting stuff here. 

Want to explore how we can integrate AI into your experimentation workflows or adapt XOS to your team’s stack? Feel free to reach out—I’d love to chat about tailoring this to your needs

You have a testing tool, you have analytics....

but do you have an "experimentation management and analysis platform"? 

Here's my POV on how to set one up: 

(btw, no, it's not just 'Airtable' nor is it your testing tool + BigQuery)

Step 1. Benchmark the program maturity and capabilities across teams. 

- mainly needs to have A to Z questions, to identify gaps and critical drivers. (e.g., where are you now vs where you want to be...). This is the heart of usefulness of Speero's Program Maturity Benchmark Audit.
- capabilities are the key here. Esp ones that speed the flywheel of feedback and insight generation. (e.g., Leadership Buy-In, Evangelism, Collaboration, Personalization, Targeting, Strategy, Test Complexity, Tool Investment, Tracking and KPIs, Research Knowledge base, Hypothesis Generation, Prioritization Framework, Roadmapping, Quality Assurance, Pre-Test Analysis, Communication, etc., etc, etc)

Step 2. Plan against a north star for the program

- yes have your goal tree map, and Goal vs Driver vs Guardrail metrics (input/output/outcome metrics too)
- but also have a 'remit' for your org on what the experimentation program means to you and the team. Purpose & Motivation, Mission & Tenants, Mechanisms to invest in (tied to the capabilities audit above). 

Step 3. Document language (the heart of standardization)

- Nouns and verbs, types and categories... a data dictionary or semantic library for your org.
- The MVP is two columns of data, no joke. And it's what's lacking at 90% of orgs and is the 80/20 of good uses of AI btw.  
- This is the connective tissue across the teams and touchpoints and such that the testing program touches. It's what creates the leverage points for automations and AI to grab hold of. 
- This is also what Chad Sanderson won't stop talking about, for good reason. You can't align people if you don't have a shared language. 

Step 4. Standardize artifacts and rituals

- If the data language documentation are the notes, the rituals and artifacts (tools) you use are the music that you play. 
- This is the 'drumbeat' of your org. How your team finds a rhythm.
- Speero blueprints are ALL about this step 4. 
- Not all these tools and cultural practices will make it into the 'platform', but you *shouldn't* build that platform without having this figured out. 

Step 4. Standardize platform architecture

- how the data flows
- who sees what, when, and how
- what happens, when, and why
- tables, Interfaces, and Actions
- This step is the actual building of the bespoke 'platform' that is your testing program. 

Step 5. Test&Learn, with Experimentation Consultants to advise 

- Like an annual medical check, vitals being velocity and impact, and ultimately your organization's ability to adapt.  
- this is where you bring in me, Speero to check your work
- this is where you bring in consultants like Lukas Vermeer, Craig Sullivan, Iqbal Ali, Simon Jackson, Liam Furman, Roderik Koenders to check your work
- do it ;)

Product Ops perspectives are all the rage atm. It's the missing link. It's what most orgs are lacking right now.

There are new tools to try to solve this. Tools like Jira's 'Product Discovery' and Airtable's 'Product Central' and even dedicated solutions like Dotwork and Doubleloop. 

We have task management (Jira/Monday), we have wiki's and docs... but what is the tool that allows a team or program to do resource and portfolio management connected to a set of cascaded goals and org-level strategic initiatives? 

At it's simplest, it can look like the pic below here. 

"Testing Learnings," here is a program dash. Do you have a picture like this for your program/team? A perspective across the portfolio of bets that the team is doing? 

This shows a dashboard with several metrics and charts.

> Includes a summary of test results: 9 "Win," 11 "Flat," 5 "Loss," and 0 "Error". 
> displays the monthly relative potential in dollars for discovered wins, discovered saves, and the total. > > Charts showing the breakdown of test results, recommended actions (like abandon, implement, and iterate), and optimization pillars (conversion, frequency, merchandising, and spend)
> Can have charts that link to yearly/quarterly strategic goals as well.... not only the always on stuff but the bigger investments and bets. 

This isn't about a sheet of AB tests that are being worked on. It's a tactical, tangible mechanism that represents and aligns strategy (activities) and outcomes(!) for a team. Boom. It's what helps make a team accountable but also not stuck in an optimization rut.

If enterprise is ever to get good use out of AI, then context is king. 

Data can't live in decks and docs. 

In a post by Speero strat Gertrud Vahtra, she shares how to build a research system that:

1. Centralizes (& Databases) Insights

The first step is to centralize insights into a single repository, which creates a searchable library of insights that can be used to generate test ideas. Use an "𝗮𝘁𝗼𝗺𝗶𝗰 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵" approach, which breaks down research into three connected parts: 𝗼𝗯𝘀𝗲𝗿𝘃𝗮𝘁𝗶𝗼𝗻𝘀, 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀, and 𝗮𝗰𝘁𝗶𝗼𝗻𝘀.

2. Synthesizes Insights by Theme

The second step is to synthesize insights by theme using 𝘁𝗵𝗲𝗺𝗮𝘁𝗶𝗰 𝗮𝗻𝗮𝗹𝘆𝘀𝗶𝘀. This involves clustering findings into recurring themes, which helps teams identify major user pain points and prioritize them for testing.

3. Turns Themes into Test Roadmaps

The third step is to turn themes into test ideas using a structured format. Use a "𝗣𝗿𝗼𝗯𝗹𝗲𝗺-𝗛𝘆𝗽𝗼𝘁𝗵𝗲𝘀𝗶𝘀-𝗠𝗲𝘁𝗿𝗶𝗰" 𝗳𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 to create a clear and testable hypothesis for each experiment. 

4. Helps Make Research a Habit

The fourth step is to make research a habit by embedding it into the team's rituals. This includes including a research link in every test brief, reviewing research insights in test planning meetings, and standardizing the research request process.

Read the full post here: 𝗵𝘁𝘁𝗽𝘀://𝘀𝗽𝗲𝗲𝗿𝗼.𝗰𝗼𝗺/𝗽𝗼𝘀𝘁/𝗵𝗼𝘄-𝘁𝗼-𝗯𝘂𝗶𝗹𝗱-𝗮-𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵-𝘀𝘆𝘀𝘁𝗲𝗺-𝘁𝗵𝗮𝘁-𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆-𝗱𝗿𝗶𝘃𝗲𝘀-𝘁𝗲𝘀𝘁𝘀

taplio

Ben Labay's Linkedin Analytics

Ben Labay

Performance ScoreBen Labay's LinkedIn Performance

What is Ben talking about?

Ben Labay's Best Posts (last 30 days)

Ready to grow your LinkedIn presence?

taplio

.css-6ytdur{background-image:linear-gradient(284deg, rgba(39, 43, 72, 1) 0%, rgba(33, 47, 183, 1) 100%);color:transparent;-webkit-background-clip:text;background-clip:text;}Ben Labay's Linkedin Analytics

Ben Labay

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Ben Labay's LinkedIn Performance

What is Ben talking about?

Ben Labay's Best Posts .css-1emoo84{font-size:var(--chakra-fontSizes-md);font-weight:400;color:var(--chakra-colors-gray-600);}(last 30 days)

Ready to grow your LinkedIn presence?

Ben Labay's Linkedin Analytics

Performance ScoreBen Labay's LinkedIn Performance

Ben Labay's Best Posts (last 30 days)