Tag: testing

How to Prevent A/B Testing from Slowing Down Your Site

You’ve probably used tools like Google Optimize for A/B testing to increase conversion rates on your site.

These tools allow you to test content by showing different variations of the same page to visitors at random. 

A/B testing helps prevent websites from spending time and resources on developing features that turn out to be unpopular with their users. 

Sometimes, however, A/B testing can lead to a slower user experience if the page takes too long to load.

This often happens if the content is being tested too often or if the code is used in a way that slows down the site.

If your content takes too long to load, users may navigate off your site, increasing bounce rates and lowering your chance to convert them. 

In this blog, we’ll cover how to prevent A/B testing from slowing down your site, using tactics such as:

  • making sure that the scripts are implemented directly into the top of the head tag, not using a tag manager 
  • implementing the asynchronous GTM version of Google Optimize 
  • using animations can be used to prevent test experiences from loading too slowly and being too disruptive to user experience

Let’s get started.

How Can A/B Testing Slow Down Your Site?

A/B testing can cause an extra step in loading and displaying web pages.

This happens because two versions of content are being shown to users at random times, collecting data on which page performs better.

All of this back and forth communication can result in a lag in page load time.

It can also cause a flicker of original content (FOOC) that displays for a short moment before the page finishes loading.

A/B testing slows down your site in three ways:

  • making the loading time of your site slower than normal 
  • creating a poor user experience that causes users to leave or prevent them from visiting again later on 
  • delaying any other events, such as an email campaign, because it’s taking longer for pages to load and finish rendering

Page load time is an important metric for your conversions and SEO.

Research has shown the first five seconds of page load time has the biggest impact on conversion rates.

Similarly, 70 percent of consumers say page speed influences their desire to buy.

Ultimately, if you want users to stay on your site and purchase your products, you need to make sure your site is fast. 

google on bounce rates for sites to prevent a/b testing from slowing down your site

How to Prevent A/B Testing from Slowing Down Your Site

To prevent A/B testing from slowing down your site, it’s important to take extra steps to ensure your user experience is not impacted by these tests.

According to Backlinko, the average page load benchmark is 10.3 seconds on desktop and 27.3 seconds on mobile.

If you’re not hitting these markers, you may have a problem.

Whether you’re using Google Optimize or another A/B testing tool, there are a few ways to prevent your site from slowing down.

How to Prevent A/B Testing From Slowing Down Your Site

1. Implement Scripts in the Top of the Head Tag

When you add A/B testing scripts to your site, make sure they are at the top of your head tag and not a tag manager.

This is important because if you make changes to your site, the scripts will be overwritten.

A tag manager is an external script that loads in place of others which can overwrite them without warning and prevent scripts from functioning properly when you make changes to your website.

If you are using the synchronous version of the script, then make sure it is placed after your site’s scripts. 

This will prevent any problems with delays caused by third-party resources on your page, such as ad networks. 

2. Use Asynchronous Tracking

Google Optimize has two versions: synchronous and asynchronous. 

The synchronous versions prevent any content from rendering until it has been fully loaded. This can prevent your A/B tests from loading in a reasonable time. 

The asynchronous versions prevent any content from rendering until it is ready, but this does not prevent the other scripts on the page from being executed immediately.

The asynchronous version is recommended for most users. It loads in a separate thread from the rest of the website, so it does not prevent other critical tasks from being executed prior to its execution.

The async version will prevent certain animations from slowing down your test experiences while still allowing for other elements on the page to play.  

If you use a tag manager like Google Tag Manager (GTM), or another JavaScript management system, it’s important these are implemented asynchronously and not using the standard version of the Optimize snippet.  

There should be no delays in page load time when Google Optimize is running on your website. The async version can prevent this by adding asynchronous to each script call so they don’t block rendering.

This is particularly important if you don’t run any tests or if they are played in a non-interruptive manner across all pages.

3. Incorporate Animations to Improve UX

If you are using Google Optimize, then you can also use animations to prevent test experiences that may load slowly and be too disruptive to the user experience.

Animations can be used to prevent A/B testing from slowing down your site by giving users something fun to focus on while they wait for content delivery.

For example, you can use animations to keep users engaged before a site fully loads, like this.

How to Prevent AB Testing From Slowing Down Your Site Incorporate Animations

This will tell users their content is being loaded and prevent them from leaving the page.

Remember to always center your animations in a place where your user will be focused.

A loading page is a good example of this or a page where the user will be focused on a specific part of the design.

Remember to prevent animations from interrupting other tests and make sure they are implemented correctly across all pages.

4. Reduce the Size of the Snippet

When adding a snippet to your site, try to keep it as small as possible.

This will prevent the script from slowing down other parts of your site, and prevent other scripts on your page from being delayed or interrupted. 

You can do this by using a tag manager, such as Google Tag Manager (GTM). 

GTM will allow you to shorten the snippet or include the snippet only on specific pages. 

Keep in mind that using a tag manager is not necessary for Google Optimize if you just want to add it once across all of your page’s head tags. 

If you prefer to embed the script into each page directly then make sure they are implemented at the top of the head tag. 

5. Test on the Server-Side

When conducting A/B tests on different server sides, the delay is often much less noticeable. 

For example, you might be using PHP instead of JavaScript on your client-side to prevent content from loading slowly and interrupting users who are trying out their new site design. 

Using different server sides works because the async version will prevent browsers from blocking on a callback function, which would prevent all other content from loading while it’s waiting for code to finish running. 

The benefit of doing this is the server-side tests prevent users with slow connections or high latency from seeing delays when loading content. 

If you can’t do this, it’s recommended to use Google Tag Manager to load these scripts asynchronously so they run after page rendering is complete and don’t affect performance. 

Also keep in mind that when testing on different server sides, it might be more difficult to prevent a slower loading experience from interrupting users since there is no way of calling asynchronous JavaScript into service. 

6. Consolidate and Optimize Variation Code

Consolidating and optimizing variation codes can help prevent A/B testing from slowing down your site. 

Variation codes are the code that is used by Optimize for each variation. 

The more complicated your website, the more variations you need to create and the more often these tests run — which results in slower site speed. 

If too many changes are applied at once on a page it can prevent other scripts from running properly or even prevent the page from loading at all.

This is detrimental to your user experience and can prevent testing from allowing you to continue optimizing your website. 

For example, if a user has JavaScript turned off or does not have it enabled they will never reach the variation that contains optimized content for them and this can set back optimizations by several weeks!

This is why it’s so important to consolidate all of your Optimize codes and scripts directly into the head tag of your site.

7. Keep All Data in a Single File

Your website is full of data and assets that need to load before the page is shown to a user. 

When you run an A/B test these assets and data need to be shared between the two experiences, but can also cause a lot of issues if they aren’t carefully managed. 

For example, say your old site used Font Awesome for all its icons and your new website uses Google Fonts as it is more web-friendly. If your site is running an A/B test, your old site will need to use the same Google Fonts as your new one. 

If you don’t manage this correctly it can cause a considerable delay in how fast the page loads for users because of all these extra assets that are loaded on top of each other. 

To prevent A/B testing from slowing down your site, keep all data in a single file. This means you prevent the page from having to make multiple requests for information. 

All experiments should be stored in a single place that is easily accessible by everyone on your team. This can prevent a lot of issues from occurring, as well as making it much easier to track the progress and performance of each test you are running. 

Frequently Asked Questions About Preventing A/B Testing from Slowing Down Your Site

Does Google Optimize slow down your site?

Google Optimize does not have a big effect on page load times. What’s more important is the time it takes your page to load, latency, and visitor connection speeds.

What should you do after an A/B test?

After you complete your A/B testing you should measure your results and take action based on your findings. It’s also recommended to strategize a new A/B test so you can continue learning.

How do I increase my Google page speed?

Page speed comes down to many factors, but optimizing your A/B tests can help prevent testing from slowing down your site.

When should you not use an A/B test?

If you lack meaningful traffic, don’t have the time or resources to dedicate to testing, don’t have a hypothesis to test, or don’t need more traffic, you should not use an A/B test.

{
“@context”: “https://schema.org”,
“@type”: “FAQPage”,
“mainEntity”: [
{
“@type”: “Question”,
“name”: “Does Google Optimize slow down your site?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: ”

Google Optimize does not have a big effect on page load times. What’s more important is the time it takes your page to load, latency, and visitor connection speeds.

”
}
}
, {
“@type”: “Question”,
“name”: “What should you do after an A/B test?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: ”

After you complete your A/B testing you should measure your results and take action based on your findings. It’s also recommended to strategize a new A/B test so you can continue learning.

”
}
}
, {
“@type”: “Question”,
“name”: “How do I increase my Google page speed?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: ”

Page speed comes down to many factors, but optimizing your A/B tests can help prevent testing from slowing down your site.

”
}
}
, {
“@type”: “Question”,
“name”: “When should you not use an A/B test? “,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: ”

If you lack meaningful traffic, don’t have the time or resources to dedicate to testing, don’t have a hypothesis to test, or don’t need more traffic, you should not use an A/B test.

”
}
}
]
}

How to Prevent A/B Testing from Slowing Down Your Site: Conclusion

A/B testing can be a great tool for driving conversions and it’s something every website owner should take advantage of. 

Understanding how to prevent A/B tests from slowing down your site, however, is equally important because slow test experiences are disruptive to the user experience. 

Sites that use A/B testing effectively will see both an increase in traffic and greater audience insights. 

How have you used A/B testing to improve site performance?

Secondary Goals: Track These GA Events If You’re Doing A/B Testing

A/B testing is brilliant. It lets you compare two or more versions of the same page element, paid ad, or another variable to see which one performs the best. In other words, it’s a key way to improve your content, increase user engagement, and boost conversion rates across your site. What’s not to like?

Additionally, with A/B testing, the results are clear and speak for themselves. For example, if you test two versions of a newsletter, it’ll be obvious from the results which version “worked” best based on the numbers.

How do you know why either version A or B performed best, though, and how do you gain deeper insight into your campaign performance?

This is where secondary goals can help you out. Let me show you why secondary goals matter in A/B testing, and how you can use them in your own marketing development.

Primary Vs. Secondary Goals in A/B Testing

Before we get started, let’s be clear on what’s meant by “primary” and “secondary” goals in A/B testing.

A primary goal is, quite simply, your main objective. It’s the priority goal of your campaign or the goal you’re hoping to achieve when you run an A/B test.

For example, say you’re a personal trainer. You have a notice on your landing page, encouraging people to sign up for a free sample week or taster session. Maybe it looks something like this from My Soul Sanctuary:

Secondary Goals in A/B Testing - Personal trainer example

Your primary goal might be identifying how many people click through to complete this form because you’re trying to increase your sign-up numbers.

Secondary goals, on the other hand, give you more insight into user behavior and how people interact with your website. They help you reach your primary goal by providing a detailed insight into your A/B test results.

For example, the personal trainer might also want to know how many people share their content on social media, or sign up for their newsletter while on their website:

Secondary Goals in A/B Testing - Personal trainer newsletter example

Knowing the answers to these questions offers additional insight into how well the content is performing—rather than tracking submission form sign-ups alone.

If you want to increase conversions, grow your business, and improve your ROI, you need to track primary and secondary goals. Otherwise, you only have half the data you need to market your business effectively.

7 Secondary Goals to Track in GA for A/B Testing

Ready to track some secondary goals? To help you get started, here are seven metrics I suggest you measure as part of your A/B testing.

1. Add-to-Cart Actions

Tracking the “add-to-cart” metric allows you to identify how often customers add items to their cart and which pages get the most traction. Knowing how many times the “add-to-cart” action is triggered lets you split your audience into two categories:

  • people who add items to their cart but remove them, i.e., shopping cart abandonment
  • those who view a product page but don’t add the item to their cart

You can use Google Tag Manager (GTM) to track cart actions. GA has detailed instructions for how to do this. Once you’re set up, you can run some different A/B tests. For example, you might test if more people proceed to checkout if there is a discount advertised for the product, and so on.

2. Interaction With Site Features

It sounds obvious, but it’s useful to track how often people interact with certain website features. Otherwise, it’s hard to tell whether your website offers visitors the great user experience they’re looking for.

From an A/B testing perspective, you might track features such as how many times users click CTA buttons, how many users engage with your live chat, and how many people click on your email address to contact you.

The exact features you track vary depending on your business goals. For example, Betterment, an investing website, has multiple different features, such as quizzes and investment calculators worth tracking:

Secondary Goals to Track in GA for A/B Testing - Interaction With Site Features

In the above example, you might be inclined to track if the calculator performs better if it’s placed higher on the page, or if different colors mean more clicks.

3. Rage Clicking on Page Features

If you’re unfamiliar with rage clicking, it’s basically when someone repeatedly clicks on a page element, but nothing happens. This typically occurs because a page element looks clickable even if when it’s not, or because a link on your page isn’t working.

With Google Analytics, you can track, for example, if there’s a single page generating a high amount of rage clicks. Or, you can see if there’s a certain type of page element which generates a lot of rage clicks e.g., a button, line of text, or image.

Rage clicking can frustrate your audience to the point where they lose trust in your business and leave your website, so it’s crucial to track the cause of these events. Again, you can track rage clicks in GA through Google Tag Manager by inserting the appropriate tags into the HTML where you want to start tracking.

4. Highlighting Page Text

Why does it matter if people highlight a portion of text on your page? Well, there are two reasons.

First, they might be highlighting the text so they can take action on it. For example, if lots of people highlight and copy your phone number, then maybe it needs a hyperlink.

On the other hand, people may highlight text to copy it into Google and search for related content. In which case, there’s a chance your website isn’t providing the answers they’re looking for. This last scenario is bad from a marketing perspective, obviously.

How do you track a secondary goal like this? Well, in the first scenario, you might run an A/B test to see if more people call you if you hyperlink your phone number. Compare the results in GA to check if it’s worth keeping the hyperlink or not.

5. Newsletter Sign-ups

Newsletters are a great way to expand your audience reach and deliver high-quality, informative content straight to your subscribers’ inboxes. First, though, you need people to sign up for your newsletter (which isn’t always easy.)

If you’re low on newsletter subscribers, there are a few variables you can play with during A/B testing. For example, you might track if people are more likely to subscribe to your newsletter during the checkout process or if a brighter, more colorful banner on your landing page results in more subscribers.

Newsletters and similar content like free guides and e-books can really help you build brand trust, so this definitely isn’t a secondary goal you should ignore. In fact, every marketer should have it near the top of their priority list.

6. Category and Subcategory Pageviews

Category and subcategory pageviews are equally important.

Your category page contains a list of related pages on your site, so it’s easier for visitors to find what they’re looking for.

Subcategory pages branch off from category pages and allow you to provide more structure to a customer’s web experience.

What should you be tracking on these pages? Well, you might track how many people click on certain subcategory pages, your bounce rate for various pages, and whether there’s a subcategory with very low engagement levels.

Then, you can play with optimizing the names of each page, changing the order of the categories, or making the subcategories clearer and more condensed. GA allows you to track both category and subcategory events, so make full use of the available features.

7. Social Media Sharing Buttons

It’s awesome when people share your content. Not only does it mean you’re resonating with your target audience, but it means they’re introducing other people to your brand. In other words, social media shares count as free marketing, which is always a bonus.

From an A/B testing perspective, you might want to track how many people are sharing your page content, and if there’s any platform outperforming the others. Perhaps no one shares your blogs, but you discover your videos are shared frequently, or maybe more people share your content on Instagram than elsewhere.

GA can help here to an extent, but it’d be worth checking out the analytics tools on your social media platforms, too.

How to Measure Your Secondary Goals for A/B Testing

OK, so you have some goals in mind. Now, you need a means of measuring these goals to see if you’re actually reaching your objectives.

First, you need to establish your baseline measurements. You need to know how your website and all its features are performing right now so you can set an appropriate end goal.

If you don’t already have your baseline measurements, go back and collate some data, and then you’ll be ready to track your progress.

Do you have a baseline? Great. Now let’s consider how Google Optimize can help you measure those all-important secondary goals.

Google Optimize is a GA extension. It allows you to run experiments and track different outcomes, and you can measure the results via Google Analytics. If you don’t already have a Google Optimize account, head to optimize.google.com and click the “Get Started” button.

After creating your profile, link it to your GA account. Google has some comprehensive instructions for this, so check them out if you get stuck.

Once you’re all set up, create an experiment within Google Optimize by heading to the “Experiments” page and clicking the “Create Experiment” option:

How to Measure Your Secondary Goals for A/B Testing - Create an Expirement with Google Optimize

Add your variables, configure your specific objectives, and let your experiment run. Once the experiment concludes, track your results by clicking the “Reporting” tab from the experiment’s page. Repeat the process if you want to try new variants.

To be clear, you’re not restricted to Google Optimize, though. You can also use data from sources like GA and even Facebook Analytics to build a comprehensive understanding of how your secondary goals are performing.

Frequently Asked Questions About Secondary Goals for A/B Testing

A/B testing can seem a little daunting, so here’s a quick recap of the main points I’ve covered to highlight the role of secondary goals in your A/B testing strategy.

What is A/B testing?

A/B testing, or split testing, allows marketers to test two versions of the same variable, such as paid ads or page elements, to identify which version performs better. The idea is to run both versions simultaneously to see which one has the highest impact.

What’s the difference between primary and secondary goals in A/B testing?

In A/B testing, the primary goals relate to the performance of each variable. In other words, primary goals allow you to track how tweaking a variable impacts visitors’ behavior.

Secondary goals, or metrics, give you further insight into how your visitors behave while they are on your website. Secondary goals matter because they help you improve the overall user experience on your site which, in turn, increases your conversions in the long run.

Which secondary goals should you track for A/B testing?

The secondary goals you should track vary depending on your unique goals. However, metrics you should focus on include newsletter signups, add-to-cart actions, and interactions with other site or page features. You might also track conversion rates, depending on your primary goal.

How do you measure secondary goals for A/B testing?

First, measure current performance so you have a benchmark to test against. Then, use analytics tools such as Google Optimize to measure each goal. Compile your results and devise a strategy based on your findings.

{
“@context”: “https://schema.org”,
“@type”: “FAQPage”,
“mainEntity”: [
{
“@type”: “Question”,
“name”: “What is A/B testing?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: ”

A/B testing, or split testing, allows marketers to test two versions of the same variable, such as paid ads or page elements, to identify which version performs better. The idea is to run both versions simultaneously to see which one has the highest impact.

”
}
}
, {
“@type”: “Question”,
“name”: “What’s the difference between primary and secondary goals in A/B testing?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: ”

In A/B testing, the primary goals relate to the performance of each variable. In other words, primary goals allow you to track how tweaking a variable impacts visitors’ behavior.

Secondary goals, or metrics, give you further insight into how your visitors behave while they are on your website. Secondary goals matter because they help you improve the overall user experience on your site which, in turn, increases your conversions in the long run.

”
}
}
, {
“@type”: “Question”,
“name”: “Which secondary goals should you track for A/B testing?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: ”

The secondary goals you should track vary depending on your unique goals. However, metrics you should focus on include newsletter signups, add-to-cart actions, and interactions with other site or page features. You might also track conversion rates, depending on your primary goal.

”
}
}
, {
“@type”: “Question”,
“name”: “How do you measure secondary goals for A/B testing?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: ”

First, measure current performance so you have a benchmark to test against. Then, use analytics tools such as Google Optimize to measure each goal. Compile your results and devise a strategy based on your findings.

”
}
}
]
}

Secondary Goals: Conclusion

When you perform A/B testing, don’t limit yourself to tracking primary goals and metrics. Instead, make sure you’re measuring those all-important secondary goals to gain crucial insight into how your website’s performing and whether the user experience is optimal.

The best part? You don’t need a host of complicated tools to measure secondary metrics. Simply track your analytics data in GA, or check out one or two other measuring tools to build a more comprehensive picture of your performance.

Are you tracking your A/B secondary goals? Which measuring tool do you find most effective?

BillionToOne (YC S17) is hiring software engineers to advance genetic testing

BillionToOne is a YC Top Company (https://www.ycombinator.com/topcompanies/) that has the most advanced genetic tests in the world. Our proprietary DNA sequencing technology has enabled us to launch:

– The world’s first and only blood test to use pregnant mom’s blood to detect fetal diseases like sickle cell disease and cystic fibrosis (www.unityscreen.com)

– The only prenatal test in the US for detecting fetal Rh blood type incompatibility (affects 15% of births)

– The first FDA Emergency Use Authorized COVID-19 test to run on high-throughput Sanger instruments (https://billiontoone.com/covid-19/)

In the coming years, we will also launch a cell-free DNA test for cancer detection and monitoring.

We’re looking to hire a few engineers to help us with a number of projects that are used by patients, physicians, and internal staff.

Our HQ is in Menlo Park, but engineering is fully remote (US based). Tech stack: Python/Django, PostgreSQL, React, etc.

Apply here:
– Frontend / Fullstack: https://apply.workable.com/billiontoone/j/4ED9B27860/
– Backend / Fullstack: https://apply.workable.com/billiontoone/j/14D61DA914/

Comments URL: https://news.ycombinator.com/item?id=27052498

Points: 1

# Comments: 0

A/B Testing: Definition, How it Works, Examples & Tools

A/B testing is a fantastic method for figuring out the best online promotional and marketing strategies for your business. It can be used to test everything from website copy to sales emails to search ads. While it can be time-consuming, the advantages of A/B testing are enough to offset the additional time it takes. Well-planned …

The post A/B Testing: Definition, How it Works, Examples & Tools first appeared on Online Web Store Site.

A/B Testing: Definition, How it Works, Examples & Tools

A/B testing is a fantastic method for figuring out the best online promotional and marketing strategies for your business. It can be used to test everything from website copy to sales emails to search ads. While it can be time-consuming, the advantages of A/B testing are enough to offset the additional time it takes.

Well-planned tests can make a huge difference in the effectiveness of your marketing efforts. Narrowing down the most effective elements of a promotion and then combining them, can create a far more effective marketing plan.

What is A/B Testing and Why Does it Matter?

A/B testing is a marketing strategy that pits two different versions of a website, ad, email, popup, or landing page against each other to see which is most effective.

For example, you might test two different popups to see which drives more webinar sign-ups or two different Google Ads to see which drives more purchases.

On my own site, I spent time A/B testing my popup to find out what encouraged users to engage with my brand.

neil patel A/B testing example

Over time, we found offering a free website analysis (which provides tons of value) was the most effective way to establish expertise and show visitors the value we offer.

Why is A/B Testing Important?

Accurate A/B tests can make a huge difference to your bottom line. By using controlled tests and gathering empirical data, you can figure out exactly which marketing strategies work best for your company and your product.

When you figure that one variation might work two, three, or even four times better than another, the idea that you would run promotions without testing starts to seem a bit ludicrous.

When done consistentyly, testing can improve your bottom line substantially. If you know what works and what doesn’t (and have evidence to back it up) it’s easier to make decisions and craft more effective marketing strategies.

Here are a few other benefits to running regular tests on your website and marketing materials:

  • Helps You Better Understand Your Target Audience: When you see what types of emails, headlines, and other features your audience responds to, you can better understand who your audience is and what they want.
  • Higher Conversion Rates: A/B testing is the single most effective way to increase conversion rates. Knowing what works and what doesn’t gives you actionable data that can help you streamline the conversion process.
  • Stay On Top of Changing Trends: It’s hard to predict what type of content, images, or other features people will respond to. Testing regularly helps you stay ahead of changing consumer behavior.
  • Reduce Bounce Rates: When site visitors see content they like, they stay on your site longer. Testing to find the type of content and marketing materials your users like will help you create a better site — and one that users want to stay on.

How Do You Plan an A/B Test?

The first thing to do when planning an A/B test is to figure out what you want to test. Are you running an on-site test, or an off-site test? If you’re running an on-site test, you’ll want to think of all the sales-related pieces of your website, and then figure out which elements to test.

For example, you might test:

  • headlines
  • calls to action text
  • calls to action location
  • pop up
  • featured images
  • copy
  • the number of fields in a form

With off-site tests, you’re probably testing either an ad or a sales email. Testing ad copy to see which ad drives more conversions can help you focus your advertising efforts. Once you know your ad is converting as well as possible, it’s easier to justify spending more money on it.

The same goes for emails. If you send out two versions to your list (randomly selecting which half gets which email), and then track which one converts better, you can send only that version the next time.

Once you know what you’ll test, make a list of all the variables you’ll test. For example, if you’ve decided to test your call to action, you might test:

  • the location
  • the exact text used
  • the button color or surrounding space

It’s a process, and it’s common for multiple A/B tests to be carried out prior to making a final decision or final change.

Before you start testing, make sure you have a clear idea of the results you’re looking for. You should already know your baseline result, which is the results you’re currently getting. You want to test option A and B against each other, but you also want to know that whichever one does better in the test is also doing better than your current results.

Alternatively, you can use A as your control (leaving whatever you’re currently using) and then use something new for B.

Tests need to be run simultaneously to account for any variations in timing. You can’t test one variation today and the other one tomorrow, because you can’t factor in any variables that might have changed between today and tomorrow. Instead, you need to split the traffic seeing your variations at the same time.

Here’s an A/B testing checklist to review before running your first test:

  • Decide what you want to test.
  • Create two versions of the same ad, landing page, etc.
  • Decide how long your test will run (I suggest at least two weeks, but it may be longer or slightly shorter depending on your traffic and industry)
  • Chose a testing tool to help you run your test. (More on that later.)
  • Launch!
  • After two or so weeks, take a look at the results. Which version won?
  • Rinse, and repeat. A/B testing is most effective when done continually.

What to Use A/B Testing to Test

You can test virtually anything in your marketing materials or on your website: headlines, calls to action, body copy, images, etc. If you can change it, you can test it.

That doesn’t mean you should spend months testing every little thing. Instead, focus on changes most likely to have a big impact on traffic and conversions.

On your website, this likely includes:

  • the headline
  • your call to action
  • any graphic you use in direct correlation to your sales efforts
  • the sales copy or product descriptions
  • feature image

In an email, you might test the title, images, links, CTAs, or segmenting options. In a paid ad, especially a text ad (like a search ad), you have fewer things to change, and so you might test the main headline, the offer, or targeting.

Testing different offers is also important. Just make sure that each person is always offered the same promotion. For example, if a free gift is offered to group A, and a discount is offered to group B, then you want to make sure that group A always contains the same visitors, as does group B.

You can also test the full path of conversion. For example, you might test newsletter A with landing page A, and newsletter B with landing page B. Later, you may want to test newsletter A with landing page B, and vice versa.

This can give you a better idea of what is working, especially if you’re getting mixed results or if the results are very close. Here are a few other tests you can run.

How Much Time Does A/B Testing Take?

A/B testing is not an overnight project. Depending on the amount of traffic you get, you might want to run tests for anywhere from a few days to a couple of weeks. Remember, you only want to run one test at a time for the most accurate results.

Running a test for an insufficient amount of time can skew the results, as you don’t get a large enough group of visitors to be statistically accurate. Running a test for too long can also give skewed results, though, since there are more variables you can’t control over a longer period.

Make sure that you stay abreast of anything that might affect your test results, so that you can account for statistical anomalies in your results. When in doubt, run the test again.

Considering the impact A/B testing can have on your bottom line, it’s worth taking a few weeks to properly conduct tests. Test one variable at a time, and give each test sufficient time to run.

Can I Test More Than One Thing At a Time?

There are two approaches to this question. Say you just want to test your headline, but you have three possible variations. In that case, running a single test and splitting your visitors (or recipients in the case of an email) into three groups instead of two is reasonable, and would likely still be considered an A/B test.

This is more efficient than running three separate tests (A vs. B, B vs. C, and A vs. C). You may want to give your test an extra couple of days to run, so you have enough results to see what actually works.

Testing more than one thing at a time, such as headlines and calls to action, is called a multi-variate test, and is more complicated to run. There are plenty of resources out there for multi-variate testing, but we won’t be covering that when talking about A/B testing.

A/B Testing FAQs

What is A/B Testing and Why Does It Matter?

A/B testing is a marketing strategy that pits two different versions of a website, ad, email, popup, or landing page against each other to see which is most effective. It’s one of the most effective ways to increase conversion rates.

How Do You Plan an A/B Test?

Decide what to test, create two versions, decide on how long to run the test, choose a tool, then see what works!

What to Use A/B Testing to Test

Any part of a paid ad, website, or marketing material including (but not limited to) pop-ups, emails, landing pages, and featured images.

How Much Time Does A/B Testing Take?

Most tests should be run for at least two weeks, but A/B testing should be continual.

Can I Test More Than One Thing At A Time?

Yes, in some cases. In general, it is best to stick to two versions of the same asset.

What A/B Testing Tools Should I Use?

Google’s Optimize is a free, powerful A/B testing tool. Your email platform, landing page tools, or website plugins may also offer this feature. For paid tools, consider Optimizely.

Conclusion

A/B testing is a marketer’s best friend. It allows you to see, for example, what ads drive the most conversions, what offers your audience responds to, or what blog headlines drive the most traffic.

There are a variety of tools you can use to get started, including Google Optimize (which is free!) and Optimizely.

If you’re looking to get started with A/B testing, you can start by learning how to a/b test in Google Analytics.

Are you considering trying A/B testing? What is holding you back?

BillionToOne (YC S17) is hiring engineers to transform DNA testing

Do you want to develop prenatal diagnostics that can affect the lives of millions of expecting parents? BillionToOne (Y Combinator S17) is looking for a Senior Software Engineer. We apply bioengineering and machine learning principles to diagnostics in order to build truly quantitative molecular tests. Our QCT platform improves the resolution of cell-free DNA testing by >1000x fold and enables novel tests for both prenatal and oncology care. As engineer #1, you will work closely with the CTO to build backend infrastructure, bioinformatics data processing pipelines, laboratory automation tools, and web-based tools to communicate genetic results to patients. This is a highly impactful position with the opportunity to own engineering end-to-end from internal prototypes to widely deployed products directly affecting patients.

If you have experience in full stack development, love seeing your work positively affect your colleagues, and thrive in a fast-paced entrepreneurial and collaborative environment, this could be a great opportunity for you.

Apply here: https://apply.workable.com/billiontoone/j/14D61DA914/ or email me at david@billiontoone.com

Comments URL: https://news.ycombinator.com/item?id=21423483

Points: 1

# Comments: 0