Center of Excellence: Change Starts Everywhere

Most people think change management is something execs do from the top down—reorgs, roadmaps, strategy decks. But anyone pushing experimentation knows that’s only half the story. 

Real change? It happens everywhere in the org. From CRO specialists nudging their team to try one more test, to CoE consultants restructuring how an entire org learns and moves.

This post breaks down how change management works at both ends of the spectrum, what a Center of Excellence actually looks like inside different orgs, and how to make experimentation 

easier, faster, and more collaborative—without forcing people to learn 10 new tools on day one.

This blog post was written by Martin Pavlovic, based on the CoE interview between Ruben de Boer and Ben Labay.

Here’s what it really takes to build a culture of experimentation that sticks.

Change Management is a Two-way Road

Change management is anything related to change, which is twofold:

• Higher management leads how the org changes and how that change happens.
• At the CRO specialist level, you have change management tactics: aligning and convincing colleagues, coping with resistance, and motivating colleagues for experimentation.

While change management is twofold, it all depends on your place in the organization, the size and audacity of the change management tactics, and the way you make it thrive. 

I was a CRO specialist. During that time, I mostly aligned with my team and direct colleagues to get more support for my work, better understand customer needs, and help those teams achieve their goals with my experiments.

Now, while working at the Center of Excellence, I am entering organizations at a higher level (as a consultant). This means I work more on change management from a higher perspective: changing the organization, its structure, way of working, and mindsets.

Types of CoE Structures

I’ve coached and been in several CoEs by now. Most generally have:

• A veteran lead with a background in CRO
• One or two (technical) data analysts
• A UX researcher and/or UX designer

Sometimes, this team works with an outside agency containing other specialists ready to jump in. For instance, if we need to automate something, we’ll use a tech web analyst and developer to set up API connections from a testing tool or BigQuery to Airtable.

Of course, other situations are possible. I have a client where we’re guiding and coaching their CRO specialists in various product teams while an experienced team acts as a Center of Excellence.

We don’t design, analyze, or generate hypos. We teach, coach, guide, organize workshops, and help with morale. The point here is to make it easy and fun for the specialists working in different product teams to experiment.

With other clients, we have mixed structures. We mostly guide and help out CRO specialists in product teams. Sometimes, we do advanced research or analytics if the CRO specialist or product team doesn’t have the knowledge or time to run it themselves. The CoE could also operate as a traveling circus, teaching team by team how to do high-quality and high-velocity experimentation.

Product Teams Structures

Regarding structure, I generally see a team made of:

• A product owner
• Front-end developer
• Back-end developer (depending on the product)
• The designers (obviously)
• CRO specialist (sometimes)

Other times, an analyst plus a part of the product team run experiments. For instance, the analyst and the UX designer are responsible for experimentation in those teams. But I’ve also seen product owners responsible for experimentation and fully embedded teams. 

So again, I've seen many different situations, but generally, there’s a product owner, back-end dev, front-end dev, and a designer.t

Accountability and KPIs

Fortunately, I see fewer and fewer output-driven product teams (‘Release 10 features and everyone will applaud you!’). What I see more is maturity and strategy:

1. Continuous Discovery Habits book everywhere
2. The double diamond framework
3. The triple diamond framework

This is absolutely perfect. As an experimentation specialist, I can really help and align with those frameworks and embed the experimentation process inside product teams. Even when I’m just guiding and training them, outcome-driven product teams make it easier than any other structure.

In feature-output-driven orgs, it's really tough to get experimentation out of product teams since they get applause when they release 10 new features. If they release features and we say, ‘Sorry, only 25% of those features actually had a positive impact on KPIs we’re chasing, 75% showed nothing,’ they’d miss their end-of-year bonus.

CoE Isn’t the Holy Grail

CoE isn’t the holy grail of maturity but just another step toward it. Some do see Coe as the final maturity step, but you often CoE to fix other maturity levers like:

1. Tools
2. Teams
3. Processes
4. Culture 

The Center of Excellence can move lots of levers: improve tools, teams, structures, and testing itself, but it’s not the end goal. You aren’t experimenting to build a CoE.

Automate Early 

Automate recurring tasks so you find experimentation as easy and efficient as possible, especially in the beginning when product teams need to start experimenting. 

They’ll need to learn and do a lot. Better make it simple for them. Who finds it fun to type data from BigQuery into Airtable? Or make Google slides? Just automate that stuff.

Don’t Force Teams to Learn New Tools

I don’t like to make others switch tools. I’m not gonna force the product team that only recently started experimenting to learn Airtable. I just set up a Miro board and sync it to AirTable, where the experimentation specialist can then link the solutions and tests and see what gets proven.

Carefully integrate new tools and processes into existing workflows. Make the change gradual so you don’t meet so much resistance. Don’t be the guy who comes in and says: ‘Today, we’re starting to experiment. Just learn these 10 new tools first.’

Help Them Experiment Together

Your first product teams will have one or two people responsible for experimentation. You want those responsible to form a team as well, even though they may have different managers or positions inside the org. So set up:

1. Tools that let them easily give each other feedback (like a Slack or Teams channel)
2. Recurring meetings to celebrate successes and failures together

In other words, create a safe environment for them to learn and experiment together.