EXPERIMENTATION
LIVE + UNITE 2024

BY SPEERO &  KAMELEOON

This event is different

We will tackle advanced topics and conversations like no other CRO/Experimentation conference out there.

The workshop, speaker, and moderator lists below should tell you why. In-house testing and data leaders at large testing programs, scaling experimentation across teams and orgs.

Half conference & half networking event, it's all about learning what works today and building lasting relationships with industry peers.

October (9)10-11, 2024


Austin, TX

GET TICKETS

this year it’s bigger.

We’re doubling down on the networking format with an extra day, and a special agenda theme focused on UNITING experimentation across teams.

People from these orgs are also attending

EXL = a practitioner focused agenda. Top practitioners in CRO and experimentation come together to hear what is working now, what’s the latest and greatest. The focus here will be on industry-specific applications of CX testing, AI, personalization, and data-privacy implications.

Unite = a program focused agenda. Kameleoon teamed up with EXL to invite product leaders, engineering, data, and the C-Suite to to reveal how all teams work together on experimentation-led growth (yes, you read that right, ELG).

Principles that make This Event format extraordinary

Half conference and half networking event. This event we make sure you meet the people you should meet.

1.
Relationships over powerpoints.

2.
Best possible practitioners teach their craft.

3.
It feels like a big dinner party.

We designed the event to ensure the people you should meet, you meet. We hand-curate discussion table sessions and facilitate connections.

You have very specific problems and goals. There are people who have the exact same goals, and have already figured it out. You should talk!

Shared experiences unite people – making the relationships you form here strong.

Learn from the absolute best in the field. All speakers are hand-picked practitioners. No motivational book-tour people. No sales pitches.

It’s a lot of how-to, some geek, with a sprinkle of inspiration. Model what the best are doing, and reap the rewards.

Speakers stick around for the full event, so you can pin them down for a 1:1 session. They're be in the table sessions with you.

Fun is a serious business at EXL. All-you-can-eat and drink all day long. Music and entertainment.

No to stuffy conference rooms. This is an outdoor-first event. We've rented out a beautiful brewery for the occasion. We're gonna be mostly outdoors.

You’re leaving your home, it’s gotta be worth it. You’ll have fun, we promise.

Topics you want to discuss. Hand-curated table sessions

Half of the event is designed for meeting people. We hand-curate whom you should meet.

You have very specific problems and goals. There are people who have the exact same goals, and have already figured it out. You should talk!

EXL + UNITE focuses on building authentic relationships and spending quality time with other like-minded people. Conferences can be great, but most people don’t go for the Powerpoint presentations.

And shared experiences unite people – making these relationships much stronger than a random encounter.

How we match you up with the right people?

We start by surveying every attendee about what they’re working on, key goals, level of experience, and obstacles. Based on what you tell us, we:

Create many rounds of dedicated, moderated discussion table sessions, so people working on similar things get together and learn from each other in a facilitated way.

This way you get to actually learn how something is done while becoming fast friends.

It’s not recorded. won’t end up on YouTube

You can pretend a lot of things, but you can’t pretend showing up. Events are not about the content. People forget what one or another speaker said. Most content is available on YouTube anyway, or might be better as a podcast.

What virtual events and online content can’t deliver is human relationships. Conferences are made for lasting professional relationships. That’s the whole point.

SPEAKERS

Lukas
Vermeer

Vista
Director of Experimentation

Luke
Mortensen

Capital One
Sr Director, Experimentation

Stewart
EHOFF

RS Group
Head of Experimentation

CHAD 
SANDERSON

Gable
CEO

Melissa
Weiss

Eventbrite
Director of Product

Shanelle
MULLIN

Shopify
Experimentation & Analysis

Phil
Burrows

Verizon
Head of Consumer Digital Sales & Experience

Moderators

Diana
Jerman

Disney
Director, Product - Experimentation

Sidhant
Bakhru

Hulu
Director, Growth Product/Marketing & Experimentation

Aaron
Shaffer

General Motors (GM)
Executive Director, Analytics Innovation

Barry
JAMES

YNAB
Lifecycle Marketing Manager

Sally
HALL

Auctane
Senior Manager and Global CRO

Dylan
Lewis

Atlassian
Experimentation Leader

Florent
Buisson

Cars Commerce
Experimentation Principal

Catherine
Crim

JCPenney
Senior Site Optimization

Erika
Sentz

WP Engine
Digital Marketing Manager, Lead

Peep
Laja

CXL, Wynter, and Speero
Founder

Ana
Catarina

Dell.com
Web Analytics & Optimization Product Manager

Priscilla
Leake

ADP
Director, Digital Experience

Sam
McAlear

MongoDB
Director, Growth Experimentation

Kendra
Bones

YETI
Digital Product

Sam
Barber

Suncorp
Senior Experimentation Specialist

Matt
Leonard

Purple Carrot
CMO

Kalki
Gillespie

NerdWallet
Experimentation and Growth

Kevin
Washington

Best Buy
Director of Product Management

Liona
To

lululemon
Experimentation & CRO

Travis
Brooks

Netflix
Product Manager - Experimentation Platform

Jocelyn
Czerwinski

U.S. Bank
VP, Digital Experimentation Team Lead

Robbie
Williams

Dr. Squatch
Product Management

Cory
Peterson

LED Lighting Supply
VP Marketing

Soumya
Gujjar

Tipalti
Conversion Optimization Manager

Marc
MILLER

Direct Energy (NRG)
Digital Sales Conversion

Alex
Rolleigh

rescue.org
CRO and Digital Optimization

Mike
De Boer

Xfinity.com
Digital Experimentation

Ben
Labay

Speero
CEO

Carlos
Trujillo

Speero
Sr. Experimentation Strategist

MORE Moderators
announced soon
Become a moderator yourself and get a ticket plus a VIP dinner party with all the moderators and speakers (in-house roles only).
Apply here

Workshop Day
(Optional)

A day before the conference (October 9th) we host a special Masterclass Day. Every attendee can choose to attend 2 workshops, one set in the morning, and the second set in the afternoon.

Workshops are a good match for you if you're really serious about skill building - they're much longer and go more in-depth than individual sessions at the conference.

Workshop 1: Ronny Kohavi – Advanced topics in Practical A/B Testing

The course covers topics not usually covered in introductory courses, including Ronny's Maven class on A/B Testing and his book. These advanced topics include:

- P-values misconceptions,
- Power, and False Positive Risk
- Why you should always give positive results a haircut 
- Success rate estimation, and success of ideas vs. experiments
- The cost of false positives
- TOST and early stopping – when and how to correctly peek
- Bayesian and Frequentist approaches
- CUPED – variance reduction
- Risks of unequal variants and rare events
- Long-term holdouts
- Simpson’s paradox
- Randomization strategies: front door or at trigger point, ramp-up
- Cultural issues: can I test that?  OFAT or major redesigns

When you register for the conference, you will get a $500 off coupon code for Ronny's Maven class: Accelerating Innovation with A/B Testing (https://bit.ly/MavenABClassEXL), which you can take before this workshop.

Workshop 2: Applications of AI in experimentation

This afternoon workshop will feature 10-15 minute presentations from various practitioners on their applications of AI within the practice and flywheels of AB testing, customer research, and experimentation.

Exact presenters and topics TBA, stay tuned.

We do know that the OG himself Craig Sullivan will MC the AI workshop session along with Iqbal Ali.

They've been working on AI 'playbooks' for experimentation and will lead some hands on sessions on these topics.

- discovery,
- problem exploration,
- problem statement,
- solution statement,
- ideation,
- hypothesis,
- hyp/problem checker

Additionally Andy McNaught from Onxmaps.com has been doing some fantasic applications of LLM models and testing, he'll be attending as part of this showcase.  

Duration: 8 hrs (9am to 5pm).
Lunch included
Added cost: $500

Conference AGENDA

DAY 0

WEDNESDAY, OCT 9th  (OPTIONAL)

9:00 am - 12:00 pm

Experimentation Workshop 1: Ronny Kohavi

The course covers topics not usually covered in introductory courses, including Ronny's Maven class on A/B Testing and his book. These advanced topics include:
- P-values misconceptions,
- Power, and False Positive Risk
- Why you should always give positive results a haircut
- Success rate estimation, and success of ideas vs. experiments
- The cost of false positives
- TOST and early stopping – when and how to correctly peek
- Bayesian and Frequentist approaches
- CUPED – variance reduction
- Risks of unequal variants and rare events
- Long-term holdouts
- Simpson’s paradox
- Randomization strategies: front door or at trigger point, ramp-up
- Cultural issues: can I test that? OFAT or major redesigns

1:00 pm - 5:00pm

Experimentation Workshop 2: Various, MC'd by Craig Sullivan

This afternoon workshop will feature various short presentations from practitioners on their applications of AI within the practice and flywheels of AB testing, customer research, and experimentation.

Exact presenters and topics TBA, stay tuned.

We do know that the OG himself Craig Sullivan will MC the AI workshop session along with Iqbal Ali.

They've been working on AI 'playbooks' for experimentation and will lead some hands on sessions on these topics.
- discovery,
- problem exploration,
- problem statement,
- solution statement,
- ideation,- hypothesis,
- hyp/problem checker

Additionally Andy McNaught from Onxmaps.com has been doing some fantasic applications of LLM models and testing, he'll be attending as part of this showcase.

5:00pm

Conference pre-party at TBD

DAY 1

THURSDAY, OCT 10th

10 am

Ben Labay & Collin Crowell: Opening

10:20 am

Lukas Vermeer: Bridging the gap: Integrating the Product Operating Model and the Experimentation Growth Model

In 'Bridging the Gap,' we compare the Product Operating Model (POM) and the Experimentation Growth Model (EGM), revealing their shared focus on customer-centricity, iterative development, and data-driven decision-making. Join us as we explore how integrating these models can drive sustainable success in today's digital landscape.

10:40 am

Special moderator panel session on scaling experimentation programs via org transformation

Ben Labay will ‘moderate’ a group of moderators who have specific experience in scaling experimentation via changing organizational structure. This will set the stage for our first break out session, and it will provide tips on how to get the best out of the moderated table sessions

11:00 am

Break

11:30 am

Moderated table session #1

Topic: Scaling Experimentation; velocity, impact, teams, etc
People are grouped to tables of 8-10 by company size and role. You will learn how your peers from similar-sized companies drive experimentation culture and practice.

12:30 am

Lunch and networking

1:30 pm

Moderated table session #2

Topic specific to each table
People are grouped to tables of 8-10 by company size and role. You will learn how your peers from similar-sized companies drive experimentation culture and practice.

2:30 pm

Break

3:00 pm

Stewart Ehoff: The Golden Thread - Connecting research, experimentation & business ROI

Speak to a researcher in any large organization and ask them their top frustration. Most will tell you it's action and impact as a result of the work they do. This problem goes far wider and deeper than just individual researchers getting their work stuffed into a drawer, it's a systemic problem with the way companies decide to solve problems, and validate that they've been solved. Join me in this talk to explore my thoughts on how to solve this big hairy business problem, by making research actionable, connected to your experimentation program, and most importantly, to the $$$ outcomes it drives.

3:20 pm

Chad Sanderson: Federated data management

Chad will tackle why you need modern data management in the age of hyper iteration, experimentation, and AI. Explore why traditional data management practices fail and how the cloud has fundamentally changed data development. Learn about the modern application of data management best practices, including data change detection, data contracts, observability, and CI/CD tests, and outline the roles of data producers and consumers. Get a clear understanding of modern data management's components and how to leverage them for better data handling and decision-making.

4:00 pm

Moderated table session #3

Topic specific to each table
You will be hand-curated into groups by interests, experience, etc. Topics and prompts will vary.

5:00 pm

Break + Games

6:00 pm

DINNER, GAMES, DRINKS AND CONNECTIONS UNTIL MIDNIGHT

Pure, unscripted fun. This is where the magic happens.

DAY 2

FRIDAY, OCT 11th

10:00 am

Shanelle Mullin: Learning from experimentation - How to build knowledge and extract insights

We experiment to measure, reduce risk, optimize, forecast, and most importantly—to learn. When a single experiment wins, we know what worked for one audience on one page at one point in time. But the real value of experimentation comes from learning at scale. With the right analysis, your experimentation program can become greater than the sum of its parts.

10:20 am

Luke Mortensen: Experimentation 2.0: Navigating the Next Wave of CX Innovation

We are a group of passionate practitioners energized by experimentation but not every individual, team, or organization will embrace the craft as we have. Resistance happens when one’s product development ideals conflict with misconceptions or misapplications of experimentation. Changing perceptions is crucial to fostering a successful practice. I’ll share examples of navigating resistance at the team and org level.

10:40 am

Break

11:00 am

Moderated table session #4

Topic: What's working for you right now?
People are grouped to tables of 8-10 by company size and role. You will learn how your peers from similar-sized companies are driving results.

12:00 am

Lunch and networking

1:00 pm

Moderated table session #5

Topic specific to each table
You will be hand-curated into groups by interests, experience, etc. Topics and prompts will vary.

2:30 pm

Break

3:00 pm

Melissa Weiss: When experimentation isn’t love at first test

We are a group of passionate practitioners energized by experimentation but not every individual, team, or organization will embrace the craft as we have. Resistance arises when one’s product development ideals conflict with misconceptions or misapplications of experimentation. Changing perceptions is crucial to fostering a successful practice. I’ll share examples of navigating reluctance at the team and org level.

3:20 pm

PHIL BURROWS: When moving fast becomes too slow: Operating frameworks for iteration, impact & scale

As talent continues to get sharper & technology becomes more advanced, it’s become a testing, optimization & AI/ML arms race over the last 2 to 3 years in an effort to reach consumers on a 1-1 level. While this new standard being set by consumers & companies continuously reaching to set a new bar has limitless opportunity, the process as well as organizational strain & bi-product of  these new models make it extremely difficult for organizations to adopt at scale. This interactive discussion will help you identify where challenges exist within your company's ecosystems as well as organizational strategies & frameworks that enable long term sustainable scale.

4:00 pm

Closing

6:00 pm

After party, location TBD

Did we say 2 parties?

Have fun and make friends. Everybody there will be in the same boat as you, so lots to talk about. A 30-minute conversation at an event can make a life-long connection.

Pre-party on October 9th.The main party is on October 10th. On 11th we'll have an unofficial party someplace somehow...

Location

Central
Machine
Works

4824 E Cesar
Chavez St,
Austin, TX 78702

Get your ticket

Individual tickets

Team ticket (3+ tickets)

Vendor,
Agency, or Consultant Ticket

You work for an agency, tool vendor, consultants, etc.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

$2400

Buy

Discounted inhouse ticket

You work for a brand, growing a product and program of experimentation.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

$900

Buy

TEAM Discount
on both
ticket types:

Bring 3+
people and get
20% off.

Build the best tech stack for experimentation and personalization

Sponsors

Want to sponsor? reach out to ben@speero.com for a sponsor prospectus and more information

Brought to you by