Experimentation / CRO

Experimentation and Testing Programs acknowledge that the future is uncertain. These programs focus on getting better data to product and marketing teams to make better decisions.

Research & Strategy

We believe that research is an integral part of experimentation. Our research projects aim to identify optimization opportunities by uncovering what really matters to your website users and customers.

Data and Analytics

90% of the analytics setups we’ve seen are critically flawed. Our data analytics audit services give you the confidence to make better decisions with data you can trust.

Personalization opportunity

A first-timer and a regular see the same homepage.

Help new people get oriented. Let regulars skip ahead.

Direction
Acquisition

Step 1 · Signals

What do we know about this person?

Connected

Completed orders, recognized visits, account state, and browser recognition.

Example eligibility rule

Treat someone as a regular when completed orders > 1 and recognized visits > 2 in the previous 90 days. Treat someone as a first-timer when completed orders = 0 and recognized visits <= 1. Keep everyone else on the default until the evidence is clearer.

Step 2 · The need

What’s really going on?

Anonymous first-timers land on a homepage built for people who already know you, or regulars land on a brochure built for strangers. The session type is sitting right there. The layout does not change.

Avoid this assumption

A "Welcome back, FirstName" bar is not a different homepage.

Step 3 · Worth it?

Do the math before you build.

New vs returning is a coarse split, but it is honest and testable. Orient the new; shorten the path for the known. Hold out the default.

Reach

How many eligible people can you reach in the chosen channel and time period?

Cost

What will content, delivery, QA and ongoing maintenance cost?

Return

What incremental outcome would cover that cost, and what is it worth?

Planning inputs, not measured results. Estimate reach × incremental outcome rate × value per outcome, then subtract cost. Use a range and a consistent time period.

Step 4 · The response

Pick how you help.

Example format: Experience

Swap the first module: orientation for new, resume for returning.

Compare lighter and deeper responses

Choose the lightest response that solves the need.

Assist: Offer useful help

Keep the default. A quiet "new here" vs "pick up where you left off" chip.

Adapt: Change a meaningful part

Swap the first module: orientation for new, resume for returning.

Transform: Tailor the journey

Two first screens, same URL, with a way back to the other.

What to watch out for

  • Treating a cleared-cookie regular as new and wiping their path.
  • A first-timer experience that cannot reach the full catalog.
  • Personalizing on "returning" when the only signal is a marketing cookie.

Step 5 · The moment

Did it help? A/B test it.

Control

One homepage for everyone.

Treatment

New vs returning first screen, with a holdout.

Success measure

First-session task start (search, PDP, signup) vs returning conversion.

Guardrail

Returning bounce. Do not hide the catalog they came to browse.

Randomize first-timers and regulars separately. Within each segment, compare the tailored homepage with the unchanged homepage. Do not combine the two segments into one average.

Win or lose, record what worked, for whom and why. What you learn becomes the next signal.

Tools that can deliver the response

  • Webflow Optimize Runs three experience types on Webflow sites: standard A/B tests, AI-optimized tests that keep shifting traffic to whatever's currently working instead of calling one winner, and manual rules-based personalization for defined audiences.
  • Sitecore Personalize Real-time web personalization and A/B/n testing inside Sitecore, driven by live behavioral and contextual signals.
  • Insider One Cross-channel engagement platform (web, app, messaging) that builds a unified profile from attributes, real-time behavior, and precomputed predictive scores to drive personalization and campaign targeting.
  • Mutiny B2B website personalization tool that identifies visiting companies (via IP/firmographic match) and swaps page content (headlines, proof, CTAs) by account, segment, or industry. As of April 2026 it relaunched as an AI GTM agent, expanding beyond the original website-personalization product.
  • Convert Experiences Website A/B tests and 1:1/segment personalizations built on the same testing infrastructure, with a visual editor and dev-level code hooks.
  • Acquia Personalization No-code website campaigns (A/B, targeted personalization, and content recommendations) built and optimized for Drupal, with a separately documented path for non-Drupal sites.
  • Conductrics Rule-based and algorithmic (contextual bandit) targeting that reacts to in-session behavior, not just page loads.

Try the framework on another situation

They came back mid-search and had to start over. Put them back where they left off.

The ad made a promise the page doesn't keep. Keep that promise on the page they land on.

They landed in the wrong country, currency, or language. Offer the right store. Always let them stay on this one.

Work through it in the hub

Score your personalization program, explore more customer situations, and compare the tools that can deliver them.