Experimentation / CRO

Experimentation and Testing Programs acknowledge that the future is uncertain. These programs focus on getting better data to product and marketing teams to make better decisions.

Research & Strategy

We believe that research is an integral part of experimentation. Our research projects aim to identify optimization opportunities by uncovering what really matters to your website users and customers.

Data and Analytics

90% of the analytics setups we’ve seen are critically flawed. Our data analytics audit services give you the confidence to make better decisions with data you can trust.

How Comcast Aligned Teams and Found 50 New Ideas Through a Single DTA Workshop

How can I discover hard-hitting ideas? And get a green light for ideas that die in the normal budgeting process? How do I align teams and give space in ideation to team members who don't get a spotlight during live meetings? These were the challenges Comcast’s Xfinity experimentation team faced when they partnered with Speero for the Data to Action workshop.

By the end of it, siloed teams saw how their work fits into the bigger picture, 40-50 new ideas were generated in just one session, and one of those ideas turned into a test generating a substantial revenue uplift, paying for the session 100 times over. Here's the story of it:"

The Challenge

Comcast’s experimentation program was solid—they had the processes, the infrastructure, and a steady flow of tests. But something was missing:

  • Ideas weren’t hitting hard enough. Lots of optimizations, but not enough ideas that moved the needle.
  • Out-of-the-box ideas died in budgeting. Innovation was getting squeezed out by business-as-usual priorities.
  • Teams weren’t aligned. Comcast operates in silos, with teams handing off work in a chain instead of collaborating deeply.

"We've put down a lot of the great baseline processes you need for experimentation," shared Lee Carson, Lead of Digital Experimentation at Xfinity, "but we were looking for ways to create ideas more likely to provide benefit to the organization—especially those that don’t make it into the usual budget or prioritization process."

Lee Carson, Sr. Director of Digital Experimentation, Comcast

Comcast needed a structured approach to ideation that was repeatable, scalable, and inclusive, ensuring all types of thinkers could contribute meaningfully. They also wanted to incorporate this process after one or two workshops, not build it on their own after five. That’s when they brought in Speero.

The Solution

"A lot of times during ideation sessions you're shooting from the hip trying to think of all the random things you can do. That has benefits, but also plays into a certain type of person who's going to be able to generate ideas that way. We wanted to find ways to open ideation to every kind of thinker. A lot of our thinkers work fast offline and then present ideas in real-time once they've thought about it. So we really liked your approach since we were moving in that direction."

Lee Carson, Sr. Director of Digital Experimentation, Comcast

Speero’s Data to Action workshop allowed both offline and online thinkers from Comcast to generate ideas. This produced a huge number of ideas, with a caveat: they had to focus on making an impact on Comcast's real business problems, not just random optimizations. Key elements of the workshop included:

  • Pre-work & Offline Ideation: Instead of “shooting from the hip” in a room, ideas were developed offline first. This gave people who like to think in silence time to contribute and avoid letting the loudest voices dominate.
  • “Quantity Breeds Quality” Approach: Generate a high volume of ideas first, then refine and filter for impact.
  • Constraints Drive Creativity: Instead of focusing on any idea that could work, we anchored ideas to real business problems. Tight constraints stretch ideation participants to find unusual ideas that can be game changers. Without constraints, you end up with a ton of low-quality ideas.
  • Cross-Functional Collaboration: Bringing together teams that usually work in silos to foster deeper collaboration.  Instead of siloed teams handing off work, cross-functional groups tackled problems together

The Results

The impact of the workshop extended beyond just generating ideas—it transformed how Comcast’s teams worked together and approached experimentation.

Key Outcomes:

Stronger Collaboration – For the first time, siloed teams saw how their work fits into the bigger picture. Instead of fighting for their own solutions, teams rallied around shared KPIs.

Actionable Roadmaps – 40-50 new ideas were generated in just one session, with 5 already in testing and more in the pipeline. Another session generated 80 ideas for a much tighter problem. 10 were testable instantly.

Financial Impact – One idea led to a test generating a substantial revenue uplift, paying for the session 100 times over.

"Speero's approach to ideation helped Comcast in one key way: it gave full ideation access to teams that are connected to these projects but don't normally contribute. This brought a sense of collaboration in real-time where everyone understood the impact."

Lee Carson, Sr. Director of Digital Experimentation, Comcast

Key Takeaways

1

Create a golden thread

Data-to-action workshops combine research, ideation, experimentation, and results which creates a golden thread of experimentation and helps attribute A/B test success or failure back to research and ideation, giving the team feedback and helping them improve their exp programs. See XOS blueprint and ROI blueprint for more info.

2

Refine problem statements

Take a look at Problem statement focused hypothesis blueprint. You need to be able to verbalize a solid problem statement. For this you need research.

3

Run your own D2A workshops

Grab this Miro template for free or see our workshop options and we'll help you with your (first) session(s).

Let’s kick things off.
Tell us what you’re looking for

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.