Author: Martin P.
Title: Content Marketer
EP 41: Organizational Charts
There are three principal means of acquiring knowledge... observation of nature, reflection, and experimentation. Observation collects facts; reflection combines them; experimentation verifies the result of that combination. — Denis Diderot
Yo yo… What’s up?
Martin P — Speero here.
How’s the new, new format working out?
Yea, this newsletter went through some changes.
But, we’re here to experiment, are we not?
Why don’t you try to experiment?
Share your podcast, blog post, YT show, or event, and I’ll share it in the next newsletter.
Just reply to this email.
Now, the time has come for:
This Week in Experimentation:
Maturing of the week: [Strategy & Culture > Culture > Experimentation Evangelism] Make people hyped about your experiments. Like, really hyped. Check how below.
Blueprint of the week: Org Charts — How can you better build teams and distribute experimentation capabilities? Link.
Talk of the week: Testing Insights with Rohan Katyal, Product Manager at Meta — why you should have a formalized way to share tests results, like scorecards. Link.
Read of the week: The Ultimate List of Product Metrics — Acquisition, Activation, Engagement, Retention, Revenue, and Referral metrics included. Link.
Event of the week: The Implementation Crisis: How to Run Actionable User Research — actionable ways to seamlessly incorporate dozen of user research methodologies into your AB testing roadmap. Link.
Speero Story of the week — Radical Demo Page Optimization: We know that users who navigate to the Demo page are ready to take action and don’t need distractions. Why not simplify the page so the users there can focus on converting?
Maturing of the Week: Experimentation Evangelism
Pillar: Strategy and Culture
Subpillar: Culture
Activity: Experimentation Evangelism
Like other Avant-garde industries and professions, experimentation needs its evangelists.
Especially experimentation.
The initial CRO hype is over.
Results are compound, rather than instant.
Experimentation costs takes time, and C-level drives a hard bargain.
Every time.
This activity is about Spreading/nurturing excitement for experimentation to ensure continued support for the experimentation program.
How Can You Level Up?
Internally:
— Gamify and quizzing. Run polls for guessing the winning test. Give awards for great experiments or learnings.
— Develop and run workshops for Training, Ideation, and Solutions.
— Run hackathons for ideation and voting.
— Group Miro Experiment Ripdowns and Audits.
— Create a database of best testing, research, or data stories. At Speero we’re running Speero Stories where we share fav tests internally and externally. You can find one Story at the bottom of the email.
— Create an experimentation ambassador.
— Develop and/or nurture a data-driven culture. For example, standardize procedures and processes, data collection, data transparency, choose your metrics carefully, etc.
— Appeal to feelings, not reason. Make stories out of experiments, learnings, and insights. Show how experiments benefit your C-level and team members personally.
— Share learnings during a meeting. For instance, at Speero we share a single interesting experiment and what we learned from it in Weekly company meetings. Anyone can add feedback, peer review it, then we all discuss it.
— Create a communication channel for experiments, their results, and learnings. Then automate it. At Speero we use a Slack channel.
— You can also use bulletins and dashboards to share experimentation learnings.
— Highlight a culture of curiosity, learning, and experiments through previous tips.
— Align everyone around a North Star Metric and have a clear metric system. Helpful Blueprints: Goal Tree Map, Strategic Testing Roadmap,
— Choose your battles. Don’t try to run every experiment, but only those with the most impact. Helpful Blueprints: Should I Run an A/B Test?, Experimentation Decision Matrix.
Externally:
External Strategists can help you:
— Uplevel experiments so they're not simply focused on root-level metrics.
— Get leadership buy-in with experimentation maturity audit/projects
— Champion a data-driven organization,
— Establish a process to document shared customer experience vision.
Most Helpful Blueprints:
— How to Drive Engagement and Culture around experimentation. (Most of the tips are from there)
— Test PhaseGate Framework. The heart of test and learn.
Blueprint of the Week: Org Charts
One of the most challenging questions we often hear from companies increasing their experimentation maturity is how they can better build their teams and distribute their experimentation capabilities within the org.
There's obviously not a single answer to this question, but what this blueprint is intended to do is to just go over some common examples we see in real life.
The centralized model just has a single group of experimentation, essentially serving the different teams and product owners.
The center of excellence model has experimentation specialists distributed among the teams and serving the specific teams, but serving over a single experimentation leader in most cases.
And the decentralized model just has individual experimentation teams assigned to every product owner or team. Obviously, this can get way more complex than that, than that.
Talk of the Week: Testing Insights with Rohan Katyal, Product Manager at Meta
Testing should not only stay in the realm of the experimentation team.
That’s why we’re such big fans of scorecards: a formalized way to share test results beyond your team.
Your goal should always be to share information with the people who can benefit from it, in a language that they will understand.
You want this process to be formulaic, but not forgettable.
On Testing Insights, Meta product manager Rohan Katyal explained his philosophy on scorecarding.
Read of the Week: The Ultimate List of Product Metrics
Categories:
- Acquisition Metrics
- Activation Metrics
- Engagement Metrics
- Retention Metrics
- Revenue Metrics
- Referral Metrics
In the AARRR (Acquisition, Activation, Retention, Revenue, Referral) framework, Engagement Metrics can be considered part of the Activation and Retention.
They’re presented as a separate category to emphasize the distinct metrics focusing on user interaction with the product.
Event of the Week: The Implementation Crisis: How to Run Actionable User Research
"I don’t want to do research just for the sake of doing it...”
Overwhelming outputs, C-level knows best, no system, not tied to business goals. There’s an implementation crisis in user research. Experts can’t move the needle after starting one too many times.
Emma Travis from Speero is here to share actionable ways to seamlessly incorporate dozen of user research methodologies into your AB testing roadmap.
.svg)
