EXPERIMENTATION
LIVE + UNITE 2024

BY SPEERO &  KAMELEOON

This event is different

We will tackle advanced topics and conversations like no other CRO/Experimentation conference out there.

The workshop, speaker, and moderator lists below should tell you why. In-house testing and data leaders at large testing programs, scaling experimentation across teams and orgs.

Half conference & half networking event, it's all about learning what works today and building lasting relationships with industry peers.

October (9)10-11, 2024


Austin, TX

GET TICKETS SOLD OUT!

this year it’s bigger.

We’re doubling down on the networking format with an extra day, and a special agenda theme focused on UNITING experimentation across teams.

People from these orgs are also attending

EXL = a practitioner focused agenda. Top practitioners in CRO and experimentation come together to hear what is working now, what’s the latest and greatest. The focus here will be on industry-specific applications of CX testing, AI, personalization, and data-privacy implications.

Unite = a program focused agenda. Kameleoon teamed up with EXL to invite product leaders, engineering, data, and the C-Suite to to reveal how all teams work together on experimentation-led growth (yes, you read that right, ELG).

Principles that make This Event format extraordinary

Half conference and half networking event. This event we make sure you meet the people you should meet.

1.
Relationships over powerpoints.

2.
Best possible practitioners teach their craft.

3.
It feels like a big dinner party.

We designed the event to ensure the people you should meet, you meet. We hand-curate discussion table sessions and facilitate connections.

You have very specific problems and goals. There are people who have the exact same goals, and have already figured it out. You should talk!

Shared experiences unite people – making the relationships you form here strong.

Learn from the absolute best in the field. All speakers are hand-picked practitioners. No motivational book-tour people. No sales pitches.

It’s a lot of how-to, some geek, with a sprinkle of inspiration. Model what the best are doing, and reap the rewards.

Speakers stick around for the full event, so you can pin them down for a 1:1 session. They're be in the table sessions with you.

Fun is a serious business at EXL. All-you-can-eat and drink all day long. Music and entertainment.

No to stuffy conference rooms. This is an outdoor-first event. We've rented out a beautiful brewery for the occasion. We're gonna be mostly outdoors.

You’re leaving your home, it’s gotta be worth it. You’ll have fun, we promise.

Topics you want to discuss. Hand-curated table sessions

Half of the event is designed for meeting people. We hand-curate whom you should meet.

You have very specific problems and goals. There are people who have the exact same goals, and have already figured it out. You should talk!

EXL + UNITE focuses on building authentic relationships and spending quality time with other like-minded people. Conferences can be great, but most people don’t go for the Powerpoint presentations.

And shared experiences unite people – making these relationships much stronger than a random encounter.

How we match you up with the right people?

We start by surveying every attendee about what they’re working on, key goals, level of experience, and obstacles. Based on what you tell us, we:

Create many rounds of dedicated, moderated discussion table sessions, so people working on similar things get together and learn from each other in a facilitated way.

This way you get to actually learn how something is done while becoming fast friends.

It’s not recorded. won’t end up on YouTube

You can pretend a lot of things, but you can’t pretend showing up. Events are not about the content. People forget what one or another speaker said. Most content is available on YouTube anyway, or might be better as a podcast.

What virtual events and online content can’t deliver is human relationships. Conferences are made for lasting professional relationships. That’s the whole point.

SPEAKERS

Lukas
Vermeer

Vista
Director of Experimentation

Luke
Mortensen

Capital One
Sr Director, Experimentation

Stewart
EHOFF

RS Group
Head of Experimentation

CHAD 
SANDERSON

Gable
CEO

Melissa
Weiss

Eventbrite
Director of Product

Shanelle
MULLIN

Shopify
Experimentation & Analysis

Phil
Burrows

Verizon
Head of Consumer Digital Sales & Experience

Carole
Robin

Leaders In Tech
Co-Founder

Moderators

Diana
Jerman

Disney
Director, Product - Experimentation

Sidhant
Bakhru

Hulu
Director, Growth Product/Marketing & Experimentation

Aaron
Shaffer

General Motors (GM)
Executive Director, Analytics Innovation

Barry
JAMES

YNAB
Lifecycle Marketing Manager

Sally
HALL

Auctane
Senior Manager and Global CRO

Dylan
Lewis

Atlassian
Experimentation Leader

Florent
Buisson

Cars Commerce
Experimentation Principal

Catherine
Crim

JCPenney
Senior Site Optimization

Erika
Sentz

WP Engine
Digital Marketing Manager, Lead

Peep
Laja

CXL, Wynter, and Speero
Founder

Ana
Catarina

Dell.com
Web Analytics & Optimization Product Manager

Priscilla
Leake

ADP
Director, Digital Experience

Sam
McAlear

MongoDB
Director, Growth Experimentation

Kendra
Bones

YETI
Digital Product

Sam
Barber

Suncorp
Senior Experimentation Specialist

Matt
Leonard

Purple Carrot
CMO

Allyson
Larson

dbt Labs
Senior Web Growth Specialist

Kevin
Washington

Best Buy
Director of Product Management

Liona
To

lululemon
Experimentation & CRO

Travis
Brooks

Netflix
Product Manager - Experimentation Platform

Jocelyn
Czerwinski

U.S. Bank
VP, Digital Experimentation Team Lead

Robbie
Williams

Dr. Squatch
Product Management

Cory
Peterson

LED Lighting Supply
VP Marketing

Soumya
Gujjar

Tipalti
Conversion Optimization Manager

Marc
MILLER

Direct Energy (NRG)
Digital Sales Conversion

Frederic De
Todaro

Kameleoon
Chief Product Officer

Mike
De Boer

Xfinity.com
Digital Experimentation

Ben
Labay

Speero
CEO

Carlos
Trujillo

Speero
Sr. Experimentation Strategist

Aistė
Norkutė

NordVPN
Growth Conversion Manager

Leah
Wilson - Duff

ZoomInfo
Senior Product Manager

Nils
Stotz

Contentful
Head of Product - Growth

MORE Moderators
announced soon
Become a moderator yourself and get a ticket plus a VIP dinner party with all the moderators and speakers (in-house roles only).
Apply here

Henry
Jewkes

Nextdoor
Experimentation

Vijay
Krishnan

Apple
Data Scientist Manager

Collin
Crowell

Kameleoon
Vice President, North America

MORE Moderators
announced soon
Become a moderator yourself and get a ticket plus a VIP dinner party with all the moderators and speakers (in-house roles only).
Apply here

Workshop Day
(Optional)

A day before the conference (October 9th) we host a special Masterclass Day. Every attendee can choose to attend 2 workshops, one set in the morning, and the second set in the afternoon.

Workshops are a good match for you if you're really serious about skill building - they're much longer and go more in-depth than individual sessions at the conference.

Workshop 1: Ronny Kohavi – Advanced topics in Practical A/B Testing

The course covers topics not usually covered in introductory courses, including Ronny's Maven class on A/B Testing and his book. These advanced topics include:

- P-values misconceptions,
- Power, and False Positive Risk
- Why you should always give positive results a haircut 
- Success rate estimation, and success of ideas vs. experiments
- The cost of false positives
- TOST and early stopping – when and how to correctly peek
- Bayesian and Frequentist approaches
- CUPED – variance reduction
- Risks of unequal variants and rare events
- Long-term holdouts
- Simpson’s paradox
- Randomization strategies: front door or at trigger point, ramp-up
- Cultural issues: can I test that?  OFAT or major redesigns

When you register for the conference, you will get a $500 off coupon code for Ronny's Maven class: Accelerating Innovation with A/B Testing (https://bit.ly/MavenABClassEXL), which you can take before this workshop.

Workshop 2: Applications of AI in experimentation

This afternoon workshop will feature 10-15 minute presentations from various practitioners on their applications of AI within the practice and flywheels of AB testing, customer research, and experimentation.

Exact presenters and topics TBA, stay tuned.

We do know that the OG himself Craig Sullivan will MC the AI workshop session along with Iqbal Ali.

They've been working on AI 'playbooks' for experimentation and will lead some hands on sessions on these topics.

- discovery,
- problem exploration,
- problem statement,
- solution statement,
- ideation,
- hypothesis,
- hyp/problem checker

Additionally Andy McNaught from Onxmaps.com has been doing some fantasic applications of LLM models and testing, he'll be attending as part of this showcase.  

Duration: 8 hrs (9am to 5pm).
Lunch included
Added cost: $500

Conference AGENDA

DAY 0

WEDNESDAY, OCT 9th  (OPTIONAL)

9:00 am - 12:00 pm

Advanced topics in Practical A/B Testing Workshop 1: Ronny Kohavi

The course covers topics not usually covered in introductory courses, including Ronny's Maven class on A/B Testing and his book. These advanced topics include:
- P-values misconceptions,
- Power, and False Positive Risk
- Why you should always give positive results a haircut
- Success rate estimation, and success of ideas vs. experiments
- The cost of false positives
- TOST and early stopping – when and how to correctly peek
- Bayesian and Frequentist approaches
- CUPED – variance reduction
- Risks of unequal variants and rare events
- Long-term holdouts
- Simpson’s paradox
- Randomization strategies: front door or at trigger point, ramp-up
- Cultural issues: can I test that? OFAT or major redesigns

1:00 pm - 5:00pm

Experimentation Workshop 2: Various, MC'd by Craig Sullivan

This afternoon workshop will feature various short presentations from practitioners on their applications of AI within the practice and flywheels of AB testing, customer research, and experimentation.

Exact presenters and topics TBA, stay tuned.

We do know that the OG himself Craig Sullivan will MC the AI workshop session along with Iqbal Ali.

They've been working on AI 'playbooks' for experimentation and will lead some hands on sessions on these topics.
- discovery,
- problem exploration,
- problem statement,
- solution statement,
- ideation,- hypothesis,
- hyp/problem checker

Additionally Andy McNaught from Onxmaps.com has been doing some fantasic applications of LLM models and testing, he'll be attending as part of this showcase.

5:00pm

Conference pre-party at Zilker Brewing Company and Taproom

Address: 1701 E 6th St, Austin, TX 78702

DAY 1

THURSDAY, OCT 10th

10 am

Ben Labay & Collin Crowell: Opening

10:20 am

Melissa Weiss: When experimentation isn’t love at first test

We are a group of passionate practitioners energized by experimentation but not every individual, team, or organization will embrace the craft as we have. Resistance arises when one’s product development ideals conflict with misconceptions or misapplications of experimentation. Changing perceptions is crucial to fostering a successful practice. I’ll share examples of navigating reluctance at the team and org level.

10:40 am

Carole Robin “Connect” Workshop: Grow Impact by Building Stronger, More Robust Relationships

The premise of this session is that people do business with people -- not products, strategies, or money -- and interpersonal competence is a proven determinant of professional success. In this interactive experiential workshop (talk then table session), participants will increase their interpersonal competence and learn how to more effectively influence and lead.

11:30 am

‘Connect’ Moderated table session #1

Topic: Connect session
A special moderated table session that extends Carole Robin’s content to allow hands on application of principles. Small groups will perform some ‘connect’ exercises, prompts coming from Carole.

12:30 pm

‘Connect’ Q&A with Carole

1:00 pm

Lunch and Networking

2:00 pm

Moderated table session #2

Topic specific to each table
You will be hand-curated into groups by interests, experience, etc. Topics and prompts will vary.

3:00 pm

Break

3:20 pm

Lukas Vermeer: Bridging the gap: Integrating the Product Operating Model and the Experimentation Growth Model

In 'Bridging the Gap,' we compare the Product Operating Model (POM) and the Experimentation Growth Model (EGM), revealing their shared focus on customer-centricity, iterative development, and data-driven decision-making. Join us as we explore how integrating these models can drive sustainable success in today's digital landscape.

3:40 pm

Stewart Ehoff: The Golden Thread - Connecting research, experimentation & business ROI

Speak to a researcher in any large organization and ask them their top frustration. Most will tell you it's action and impact as a result of the work they do. This problem goes far wider and deeper than just individual researchers getting their work stuffed into a drawer, it's a systemic problem with the way companies decide to solve problems, and validate that they've been solved. Join me in this talk to explore my thoughts on how to solve this big hairy business problem, by making research actionable, connected to your experimentation program, and most importantly, to the $$$ outcomes it drives.

4:00 pm

Moderated table session #3

Topic specific to each table
You will be hand-curated into groups by interests, experience, etc. Topics and prompts will vary.

5:00 pm

Break + Games

6:00 pm

DINNER, GAMES, DRINKS AND CONNECTIONS UNTIL MIDNIGHT

Pure, unscripted fun. This is where the magic happens.

DAY 2

FRIDAY, OCT 11th

10:00 am

Chad Sanderson: Federated data management

Chad will tackle why you need modern data management in the age of hyper iteration, experimentation, and AI. Explore why traditional data management practices fail and how the cloud has fundamentally changed data development. Learn about the modern application of data management best practices, including data change detection, data contracts, observability, and CI/CD tests, and outline the roles of data producers and consumers. Get a clear understanding of modern data management's components and how to leverage them for better data handling and decision-making.

10:20 am

Luke Mortensen: Experimentation 2.0: Navigating the Next Wave of CX Innovation

Join Luke as he explores a broader definition of experimentation, focusing on upcoming avenues of product and CX innovation. Traditionally, most CX experimentation has centered on creative, design, copy, format, and product flow testing. With Gen-AI tools making these variations less expensive to release, we'll delve into new experimentation methods like model-variation testing and explore/exploit optimization. Discover less common experimentation types today that will become key to performance improvement in the near future.

10:40 am

Break

11:00 am

Moderated table session #4

Topic: What's working for you right now?
People are grouped to tables of 8-10 by company size and role. You will learn how your peers from similar-sized companies are driving results.

12:00 am

Lunch and networking

1:00 pm

Moderated table session #5

Topic specific to each table
You will be hand-curated into groups by interests, experience, etc. Topics and prompts will vary.

2:30 pm

Break

3:00 pm

Shanelle Mullin: Learning from experimentation - How to build knowledge and extract insights

We experiment to measure, reduce risk, optimize, forecast, and most importantly—to learn. When a single experiment wins, we know what worked for one audience on one page at one point in time. But the real value of experimentation comes from learning at scale. With the right analysis, your experimentation program can become greater than the sum of its parts.

3:20 pm

PHIL BURROWS: When moving fast becomes too slow: Operating frameworks for iteration, impact & scale

As talent continues to get sharper & technology becomes more advanced, it’s become a testing, optimization & AI/ML arms race over the last 2 to 3 years in an effort to reach consumers on a 1-1 level. While this new standard being set by consumers & companies continuously reaching to set a new bar has limitless opportunity, the process as well as organizational strain & bi-product of  these new models make it extremely difficult for organizations to adopt at scale. This interactive discussion will help you identify where challenges exist within your company's ecosystems as well as organizational strategies & frameworks that enable long term sustainable scale.

4:00 pm

Closing

6:00 pm

After party, location TBD

Did we say 2 parties?

Have fun and make friends. Everybody there will be in the same boat as you, so lots to talk about. A 30-minute conversation at an event can make a life-long connection.

Pre-party on October 9th.The main party is on October 10th. On 11th we'll have an unofficial party someplace somehow...

Location

Central
Machine
Works

4824 E Cesar
Chavez St,
Austin, TX 78702

Tickets SOLD OUT!

Contact ben@speero.com (or via linkedin) for wait list options.

Individual tickets

Team ticket (3+ tickets)

Vendor,
Agency, or Consultant Ticket

You work for an agency, tool vendor, consultants, etc.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

$2400 (USD)

Buy

Discounted inhouse ticket

You work for a brand, growing a product and program of experimentation.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

$900 (USD)

Buy

TEAM Discount
on both
ticket types:

Bring 3+
people and get
20% off.

Build the best tech stack for experimentation and personalization

Sponsors

Want to sponsor? reach out to ben@speero.com for a sponsor prospectus and more information

Brought to you by