Experimentation / CRO

Experimentation and Testing Programs acknowledge that the future is uncertain. These programs focus on getting better data to product and marketing teams to make better decisions.

Research & Strategy

We believe that research is an integral part of experimentation. Our research projects aim to identify optimization opportunities by uncovering what really matters to your website users and customers.

Data and Analytics

90% of the analytics setups we’ve seen are critically flawed. Our data analytics audit services give you the confidence to make better decisions with data you can trust.

Ecommerce optimization series #5: The challenge of finding the optimal product price

By Matt Beischel

How do you find the ideal price to sell your products? 

The price you charge for your products is far more complex than simply cost-plus pricing. Price impacts and can be affected by factors such as;

  • Product vertical, stage in the product lifecycle, and product alternatives.
  • Competition. 
  • Brand positioning, customer experience, and brand perception.
  • Supply and demand
  • Overall business strategy and the costs to operate and produce/sell products.  
  • Broader economic factors, such as a recession, government policies, or higher living costs. 

With so many factors playing a role in price, let’s delve into why ecommerce businesses must overcome the pricing challenge in this current business climate. 

Why pricing tests need your attention

Ecommerce profit margins are being squeezed thanks to rising acquisition, materials, and fulfillment costs. While cost optimization is one part of the puzzle, reassessing your prices to eke out more profit is a no-brainer. After all, a 5% improvement in pricing without volume loss and average margins can boost profits by 30% to 50%. 

The wealth of customer data and nature of online commerce has also opened up a raft of pricing approaches that would have been hard to implement on the high street (even with digital pricing displays). From Amazon making 2.5 million daily price changes to Uber using surge pricing based on demand, weather, and location. 

However, the opportunity that endless customer data and technology presents us with can be a double-edged sword. While A/B price testing might appear to be the perfect way to determine an optimal price and drive profitability, it’s much more complex in practice. Here are the main challenges you’ll face and what you can do about each. 

The Challenge

Legal and ethical implications 

Convert.com recently ran a LinkedIn poll asking if it’s ethical to run A/B pricing tests–46% of respondents felt it was “under certain conditions.”

I imagine those conditions will vary depending on who you ask. As with any divisive topic, there’s often no black or white answer but rather ‘it depends’ based on the nuance of what you plan to do. If your business has documented brand guidelines or a person responsible for ethical governance, I’d always consult with these sources before engaging in pricing tests. 

Consider, is it ethical to;

  • Offer exactly the same product to different customers for both $10 and $30? 
  • Offer cheaper prices to new customers but not your existing customers? 
  • Charge more depending on clothing or shoe size? 
  • Rise the price of home supply products after a natural disaster when demand increases? 

As a general rule of thumb, if you wouldn't feel comfortable running the test in a physical store face to face with customers, don’t do it online. If you are running a pricing test it’s good practice to:

  • Honor lower prices if there's a support issue
  • Make sure the price point is consistently displayed and purchasable
  • Agree on what should be the "default state" price and why

Convert also created a summary of the legal implications of running pricing tests in the USA, EU, and the UK. While on the whole, price testing is legal, there are specific elements that aren’t, and it’s worth seeking additional advice based on where you operate and the type of test you plan to run. The main illegality arises around discriminatory pricing.  

Even when businesses don't intentionally discriminate between customers, things can still go wrong; Tinder, for example, offered lower prices to customers in school or early in their careers. But a customer sued for age discrimination, and Tinder agreed to settle the lawsuit for $24 million (however, the case was overturned) and stopped this pricing practice worldwide. 

Something seemingly innocuous can have significant implications. Setting particular segmentation in your price tests means you might unintentionally discriminate against certain groups of people. Take, for instance, offering different prices based on browsing devices, a data point that could highly correlate with specific economic profiles or ages. 

That said, Codecademy saw massive revenue gains in testing different prices per country according to low vs high GDP (lower prices for lower GDP countries. Here’s Dan Layfield on it:

“The first experiment we ran was around exaggerating the difference in price between the monthly and annual plans in high-GDP countries, which had a massive positive impact on the number of annual subscriptions.”

At Speero we have clients who have better margins for shipping and taxes in the UK compared to the US, so we’re experimenting with lowering pricing there to find out if it nets out with better revenue overall. 

In another example, we’ve played around a ton with discounting, slice-thru-pricing techniques, and coupon codes. When to auto-apply, for whom, etc. 

Oliver Cabell is an upscale sneaker manufacturer and retailer. They were regularly offering a site-wide 15% discount. Through heuristic analysis, we hypothesized that this sale pricing didn’t mesh with the brand; it harmed the perceived value of their luxury products. We tested removing the “perma-discount” in favor of a manually applied promo code. This resulted in a $100k+ lift in monthly revenue and improved pricing display that aligns with the brand. They were able to roll back the abundant slash-thrus, which helps elevate the site experience to match the level of the product in feeling more up-market and lux.

Technical Limitations

The ability to execute on price testing is highly dependent on the type of test and site architecture. Any "painted door" here is generally a no-no. Products being price tested have to actually be purchasable at the price being shown; typically this means duplicating product data, making 2 copies of the product/category pages and using redirection to keep price display consistent across the entire site; ie. category1.html redirects to category1a.html, etc.

You could be somewhat sneaky with "starting at..." pricing, but that's getting into ethically dubious dark pattern territory. When in doubt, always give the user the lower price. Customer service should be informed and in the loop whenever price testing is going on.

What can you do instead?  

As mentioned in the introduction, branding, marketing, and customer experience can impact the price customers are willing to pay. Rather than testing individual product prices based on user behavior, you can look at testing the context around price. Experiment with elements that can support increasing your prices across your product portfolio, from optimizing how prices are displayed to testing copy about why your brand is superior and determines a higher price. 

Changing consumer behavior

The nature of pricing tests means that individuals shopping for the same product will see different prices. If users clear their cookies or browse on another device, they might be bucketed into another test variation and see a different price from before. This is bad from a customer trust and user experience perspective, but it can backfire in a much more significant way. 

Consumers aren’t keen on pricing tests or dynamic pricing practices, despite potentially working in their favor. Seven out of ten Americans oppose personalized pricing, 49% “strongly,” according to a survey of 2,341 adults. When customers discover brands are price testing, they often call it out on social media. The damage of negative PR on customer perception (and potentially share prices) can vastly outweigh any profit gained from optimized prices. 

Another aspect of price testing (and the growing prevalence of dynamic pricing) is that consumers are becoming wise to such practices. As a result, shopping behavior is changing, with users checking prices in an incognito browser, using VPNs, and installing pricing checking browser plug-ins. The act itself of price testing might lead to more price sensitivity or comparison shopping behavior. Not to mention the above behavior would seriously skew your test results. 

What can you do instead?  

Conduct user research to understand what role price plays in the overall shopping experience and what perceptions or issues your target customers have when shopping with you. Use this alongside market analysis to inform a comprehensive pricing strategy rather than dynamically testing individual prices. A solid pricing strategy is an overall approach your business takes to achieve broader goals, and there are many established strategies based on what you want to achieve. E.g., penetration pricing or premium pricing.

Also, consider that you don’t have to test price changes outright. There are other avenues around pricing to consider testing:

  • removing an "always on" discount
  • different types of discounts - flat amount vs percentage off
  • different discount amounts - ie. 10% vs 15%
  • price presentation - ie. $499 vs $500
  • price sortation on curated PLPs - ie. lowest to highest vs highest to lowest

The wrong experimentation method?  

Our thought process around price is complex, multifaceted, and not always logical. While a higher price might perform better in a test, it could also have broader implications beyond that conversion. For example, the price you set can also impact brand perception, customer lifetime value, satisfaction, and more.

A/B pricing tests tell us historically which price led to a purchase decision. Given that so many factors influence the amount a user is prepared to pay, it’s hard to conclude from the A/B test that the results are replicable even if they are statistically significant. For example, your main competitor is running a sale during the period you run your A/B test and discounts all their products–this could influence your test findings. 

What can you do instead?  

Consider if an A/B test is the best experimentation method to answer your hypotheses. Will it give you insights that can inform decisions? Weigh up what you might learn from an A/B test against potential issues and limitations and then compare this to other experimentation methods.

If you decide to run A/B price tests, run qualitative tests in tandem to glean insights that might help you interpret the quantitative results. 

Guardrail metrics are another vital element to ensure that your test doesn’t negatively impact other business areas. Look at multiple metrics, not just total revenue. AOV & RPV are really important, as are metrics around return rates. Consider a scenario where orders dropped by 20% but profit is flat. That might be a win for the business because they can cut warehousing and manpower costs while making the same amount of money and increasing margins. Or consider your price increase test might have won, but it also raised return rates, so overall, the test is reducing your profit margins.

How does this all apply to your business? 

This article is the fifth installment in our nine-part ecommerce optimization series. Keep reading to help inform your organization about the latest challenges and opportunities in ecommerce experimentation.

If you'd like to learn more about conversion rate optimization and how we can help build the necessary processes and methodologies alongside your ecommerce team, contact me at matt@speero.com

Look out for the 6th installment of the nine-part series next week.

Related Posts

Who's currently reading The Experimental Revolution?