Let's talk about how we can tailor Swell to your business.

Learn More

How to Use A/B Testing For Your Loyalty Program the Right Way


How to Use A/B Testing For Your Loyalty Program the Right Way

Every online merchant survives on “conversions”-- turning a typical visitor into someone that produces value for your business. Whether that visitor completes a purchase, subscribes to your newsletter, or simply clicks on your “Learn More” button, each of these actions moves your potential customers along the buyer's journey and can be vital to your success. Customer acquisition is becoming increasingly more expensive and so optimizing your conversion rate will help you keep down marketing costs while you grow your sales. 

Unfortunately, too many modern businesses struggle with low conversion rates. A conversion rate is calculated as the ratio of the number of total visitors on your website to the number of visitors that complete a desired action. The average ecommerce business has a conversion rate for purchases of 1.33%, meaning that for every 1000 people that visit their website, less than 14 of them actually m

ake a purchase. Since only 22% of businesses are satisfied with their conversion rates, there is a lot of improvement needed in this area.

So how can companies increase their conversion rates without overspending on marketing? 

The simple, cost-effective solution is A/B Testing for your loyalty program. 

What is A/B Testing?

A/B Testing, also known as “split testing” or “bucket testing,” is used to experiment with multiple versions of a developing product in order to identify which delivers the optimal conversion rate. Determining which version of your web page “performs better” will then result in increased lead generation, completed purchases, or link/button clicks. In the online world, A/B Testing can be used for any component associated with your business: a landing page, web page, or email campaign.

The process of A/B Testing works like this: a company or individual creates two versions of a product: Version A, the control or existing element, and Version B, the new element. These versions can differ in almost anything--wording, color, font, size, shape, or placement on the website. Once both versions are created, the company must decide on the duration of the test. During that time, both Version A and B will be “live.” As visitors approach the company’s page, 50% of them will be directed to Version A, while the other 50% to Version B. After the test ends, the company can analyze their performance and determine which version resulted in a higher conversion rate.

Let’s walk through an example to better understand this concept.

SafeSoft Solutions, an online business that develops products for customer contact centers, wanted to experiment with displaying price on their landing page. They created two versions (shown below): Version A, their existing landing page, and Version B, a replica that prominently displayed the price ($75 per seat) of their service.


Image from Kissmetrics


After testing out both versions, each with 50% of their customer base, they found that Version B (the one that displayed price) resulted in a 100% increase in conversions. After conducting some interviews, the company found that visitors trusted their service more when the price was prominently displayed. This surprising discovery helped Safesoft Solutions increase trust with potential customers and drive more conversions.

What else, besides displaying price, can you test? The options are endless, but here are a few examples.

  • Headlines
  • Paragraph texts
  • Call-to-action buttons
  • Images
  • Videos
  • Links

How Can A/B Testing Be Useful for My Business?

To make sure you're fully convinced of the value of A/B Testing for your business, we'll highlight some of its most beneficial characteristics.

  • Cost-effective- A/B Testing is known to create significant value at a fraction of the cost. Once you define your goals, you can subscribe to inexpensive websites such as Optimizely ($17/month) or Visual Web Optimizer ($49/month) to test your landing pages or email campaigns.
  • Simple to implement- Conducting A/B Tests can be as simple as implementing a few HTML changes to your webpage. Whether its changing the color of a hero image or modifying a few lines of text, a few minutes of work can give you insight that will transform your conversion rate. In addition, websites such as the ones mentioned above (Optimizely or VWO) make it very easy to connect and test different parts of your pages. Both services can connect with your Shopify+, Wordpress, or Magento accounts.
  • Leads to improved content- A/B Testing provides deep insight into your customers’ preferences. Creating tests for various elements of your website and implementing changes based on the data you gather over time will help you construct your webpages specifically to your customers tastes and generate customer-centric content.  (Creative Momentum)
  • Lowers decision-making risks- By designing A/B Tests carefully, you can test customer reactions in a highly controlled environment. For example, if you're looking to make a widespread shift in messaging and tone on your website, proper A/B testing can help you determine the right direction for change on a small, low-risk, and inexpensive scale. When you ultimately need to make more important decisions, you’ll be more prepared with the insights you've gained from testing.

How to Integrate A/B Testing With Your Loyalty Program




If you’re still reading, you might be interested in understanding how to integrate A/B Testing with your specific loyalty program. In the example with SafeSoft Solutions, we talked about the effects of prominently displaying price on your landing page, but A/B Testing is useful in many other ways, especially when it comes to loyalty programs.

Here are three examples of tests you can run on your loyalty program to optimize how you display and promote your program to customers.

  • Placement of a Button to Sign up for Your Rewards Program- In Version A, you can place your “Sign Up” button to the right of a text box that explains your rewards program. As visitors are reading about your rewards program, they will always have the option to sign up on the right side of the screen. In Version B, you can place the
    “Sign Up” button at the bottom of the page, right after explaining the program.  Then, see if there’s a difference in signups between the two groups. Will more people sign up if the button is on the right (Version A) or at the very end (Version B)?


  • Headers for your Newsletter Signup Page- In Version A, you can use the text “Join the Madness, Get a $5 Discount Code.” In Version B, use the text “Join the Madness, Receive 10% Off Your First Purchase.” You can adjust the numbers so that they create the same discount amount at checkout. Then, see if there’s a difference in newsletter signups between the two groups. Will your customers respond better to a discount in the form of money (Version A) or percent off (Version B) their first purchase?


  • Colors for your Referral Page. Something as small as a change in color might make your program “pop” and get customers to engage at a higher rate. When creating a text that describes your referral program, you can play around with colors. In Version A, you can use a green heading, while in Version B, you can use a blue heading. Make sure that these colors still fit your brand’s color scheme, as it should be consistent with the rest of your website. Then see if there’s a difference in referral engagement between the two groups. What colors will draw your consumer into engaging in your program?


So How Do You Know if Your Results are Significant?


Once you have finished the A/B Tests, it’s important to understand their significance. To calculate it, all you need to know is each version’s (A and B) conversion rate, which you can find by dividing the number of overall conversions by the number of visitors on the specific page and then multiplying by 100 to convert the decimal into a percent. Compare the conversion rates using a tool such as Kissmetrics to determine if your data is significant.


Screen Shot 2017-06-14 at 4.14.59 PM
Image from Get Data Driven


Overall, A/B Testing is incredibly useful in understanding how potential customers react to your website, emails, or other online components in different ways--all with the focus of improving conversions.

So, if A/B Testing is right for your business goals, start testing!

Noam Kahn

Noam Kahn

Content Creator at Swell


Leave a Comment