This is part 5 of the interview series between Ben Labay, Speero’s CEO, and various leaders of experimentation programs across brands like Hulu, Disney, Vista, Booking, AMEX, and more.
His goal is to explore how top tech companies structure and scale their experimentation efforts, especially when it comes to the Center of Excellence. Take a look at all the interviews and their resources in this free, ungated Miro board.
Or read any blog post/interview of the series:
- How to build the right CoE with the Right Tech, Process, and Contract
- Three Key Steps to Launching CoE
- CoE: Change Starts Everywhere
- Why One Structure Doesn't Fit All Companies
- How CoE can work like a Charm in a Brand with 6K Employees
This time, Ben chatted with Luis Trindade from Farfetch about how the brand scaled experimentation with Center-of-excellence, even though it has over 6,000 employees across multiple countries, with 3,000 of them in engineering or technical roles.
Making experimentation truly work across a big organization is tough. Every company has its own way of doing things, and just shoving this new, complex function in can be a real headache. Here’s why sometimes the structure is about teaching, not locating.
The CoE Model: Teach People How to Fish
Luis explained that a CoE isn't about having a single, central team of experts who run all the experiments. Instead, it's about "teaching how to fish and not necessarily to give the fish to everybody." This means the CoE's main job is to spread knowledge, standardize processes, and enable other teams to run their own experiments effectively.
At Farfetch, this started seven years ago when Luis joined. The company was smaller then, around 300 people, and while experimentation was already happening, it was often "ad hoc." As Farfetch grew, they realized they needed a centralized approach to scale.
Different teams were trying to use different tools, with marketing wanting WYSIWYG tools, while engineering was only for server-side. There was a clear need for consistency.
So, Luis proposed setting up a central team, a CoE, to centralize all this work. This small team would handle enablement, implementation, and guidelines for tools, but the actual experiment execution would be done by the different teams.
This approach connects directly with Speero's Org Charts Blueprint. It helps you figure out the best way to structure your experimentation capabilities. It also highlights the importance of understanding the pros and cons of each structure, just like Farfetch had to decide before scaling.
The Two Hats of a CoE Leader
What's really interesting is that Luis wears "two hats". He leads the CoE area, providing expert support around experimentation, and he also leads an engineering team that builds their in-house experimentation platform.
This dual role is super important. Think about it:
- The Evangelist/Leader: This person is all about getting buy-in, spreading the word, and building a culture of experimentation. They make sure everyone understands why experimentation matters. This is only one part of how you can create experimentation culture, though.
- The Platform/Ops Lead: This (tiny) team ensures the tools and processes are solid. They build and maintain the engine that makes experimentation possible.
Often, these are two different people in an organization. But Luis combined both roles to help Farfetch create a unified vision and execution for their experimentation program.
This touches on the XOS Tool Stack Blueprint, which helps you figure out the right software categories for your experimentation program and even budget for them. Having a leader who understands both the strategic and technical aspects of the platform is a huge win.
Another interesting resource for XOS tools is Speero’s free, ungated A/B Testing Tools Comparison page. Use it to explore different A/B testing tools and find the right one without breaking your budget.
Expanding the CoE's Reach: Experimentation Champions
The CoE at Farfetch started with just the front-facing products area, but it has since expanded to include platform service areas (like payment and fraud systems) and even other companies built on Farfetch’s platform.
How did they manage this expansion with a "super tiny team"? Spot all the micro black spots in the company? They did it by creating a "network of experimentation champions." These are people in their regular jobs—product managers, analysts, designers—who are passionate about experimentation.
The CoE trains them, works with them, and has regular check-ins. This way, the CoE can be closer to different areas of the business and understand their unique challenges.
This perfectly illustrates the power of empowering your teams, a concept echoed in the Experimentation Program RASCI Matrix Blueprint. This blueprint helps define who is Responsible, Accountable, Supports, Consults, and is Informed for each experimentation activity. By training and empowering champions, Farfetch is distributing responsibility and fostering a shared ownership of experimentation, leading to increased efficiency.
Farfetch’s CoE also works on ‘ways of working’ with product managers and their leadership so the product teams don’t only run experiments, but run rigid experiments.
Smart KPIs to Account and Align
A common challenge in product-led organizations is how to keep autonomous teams aligned. Each team might have its own goals, but how do they all contribute to the bigger picture? At Farfetch, they call their independent teams "domains," and each domain has its own goals, blending both business and consumer perspectives.
They've done something really smart here: they've implemented "smart KPIs". These aren't just North Star metrics; they create a connection between the day-to-day experiments teams run and their impact on the overall business goals. Luis says this has been a "game changer" for Farfetch.
This directly relates to Speero's Goal Tree Mapping Blueprint. This blueprint helps you visualize the metrics that matter and identify sub-goals that lead to bigger successes.
It's all about aligning your team to work towards the same goal, which is exactly what Farfetch is doing with their smart KPIs. It also helps in breaking down big KPIs into smaller, more manageable metrics that teams can see improvements in faster.
They also have analysts within each domain who help translate experimentation results into financial metrics and connect them back to these smart KPIs. This shows a clear path from experiment to business impact, even if it's still an ongoing challenge to perfectly attribute every impact.
Continuous Learning: Experimenting on Experimentation
When it comes to learning about experimentation, Luis emphasized that real learning often comes from doing. "The best place to learn is actually going to a conference or just having a chat or taking opportunities like this, where we are actually being challenged about what is working and what is not working."
He also shared that they apply the principles of experimentation to their own experimentation program. "All our ceremonies that we have created have been the ones that have informed what works and what doesn't work for us." This is called eating your own dog food (methodology).
For example, their monthly "test and learn sessions" where they share all findings across the company, including their external tenants, are a key part of this. This helps them spread best practices and learn from each other.
This focus on continuous improvement and learning within the experimentation program itself aligns with Speero's Program Metrics Blueprint. This blueprint helps you monitor the success of your experimentation program by tracking metrics beyond just wins and losses, like test velocity and efficiency. It helps you identify bottlenecks and continuously improve your program.
Luis also mentioned that while books like Teresa Torres's Continuous Discovery Habits and Marty Cagan's work are important references, “the real value comes from applying these principles and then discussing the outcomes with others in the industry.” He finds LinkedIn discussions, specifically the comments and debates, to be a valuable resource for learning and finding common ground with others in the field.
Wrapping It Up
The Farfetch story with their Center of Excellence offers a clear roadmap for other organizations looking to scale their experimentation efforts. It boils down to a few key takeaways:
- Empowerment over Centralization: Don't hoard expertise; spread it throughout the organization.
- Strategic Alignment: Connect daily experimentation efforts to broader business goals using clear, shared metrics.
- Continuous Improvement: Apply the principles of experimentation to your own experimentation program. Test your processes, learn from them, and adapt.
- Community and Collaboration: Engage with others in the industry to share learnings and challenge assumptions.
If you're looking to build a robust experimentation program, thinking about how your team is structured and how you empower everyone to "fish" for insights is a great place to start.
.svg)
