Home » Optimization » Everything You Need to Know about A/B Testing – Conversion.vn

What is A / B testing?

A / B testing (also known as split testing) is a process in which two versions (A and B) will be compared against each other in a defined environment and situation and therefore  detertmine which one is more effective. This version can be anything from a banner image, a web page, an ad to an email, and the effectiveness is evaluated based on the tester’s goals for these versions.

A sales website has the goal of encouragin customers to purchase or buy more. A banner ad has the goal of making more customers to click on it. An email is intended to be opened more by customers. Everything has a purpose, to make the customer perform a desired action, this action is called conversion. The rate at which the action is taken is called the conversion rate.


And measuring and evaluating both versions A and B is the same as measuring and evaluating the conversion rate of an ongoing process.

Why doing A / B testing is essential?

If you have a certain number of customers and you want to increase the number of conversions, the first way is to bring more customers to your website or store. The second idea is to increase the conversion rate so that with the same amount of guests available, they create a greater conversion rate. A / B testing helps you do the second approach by allowing you to improve the efficiency of your ongoing processes whether it is web development, application development, advertising or sales.


The cost of recruiting a new customer in the first way is usually not small, while the cost of A / B testing is not much and the small changes can still bring great effects in creating more conversions.

A / B testing process

To perform an A / B testing process (or any other testing process), it is necessary to follow the usual scientific method, comprising the following steps:

1. Making questions:

It is important to ask questions to guide and target the A / B testing process and be clear about what will be the result after the test. The questions may be: “how to reduce the bounce rate for landing page?” Or “how to increase the number of subscribers to the form on the home page?” Or “how to improve the CTR of banner ads? “.

2. Overview study:

It is important to perceive and understand the behavior of customers as they perform conversion by measurement tools for each channel, for the website it may be Google Analytics, for Email it may be the email client, or social listening tools for social networks.

3. Setting a hypothesis:

With the question above and what you know about customer behavior when doing the conversion, try to think of a hypothesis to solve the question set above. “Having a link to the guide page in the footer can reduce the bounce rate,” “making the registration button more prominent will increase the number of subscribers” or “banner with a beautiful girl image will have a higher CTR “are examples of hypotheses for the above questions.

4. Identifying sample and testing duration:

The next step is to determine the number of customers that will be conducted in the A / B testing. The sample size should be large enough to clearly distinguish the two A / B variants after the test. Test time should also be reasonably determined to ensure that results are not affected by seasonal factors, external influences that change the needs and behaviors of customers. You can use an estimator tool to calculate the test running time.

5. Performing the test:

Create a new version B to experiment with the original version A. This version B uses the hypothesis you have set (with link under footer, more prominent sign up button, beautiful girl banner) and will be measured on conversion rate compared with version A.

6. Gathering information and conducting analysis:

After A / B testing, if you see that version B delivers higher conversion rate than version A (bounce rate decreases, subscriber increases, CTR increases), then the version B is more efficient. But if the conversion rate is lower or unchanged then the hypothesis to solve your problem is incorrect. Now we need to go back to step 3 and find a new hypothesis to continue.

7. Providing results to all related parties:

Send information and insights found after the testing process to related departments (programmers, UI / UX designers, optimization team, etc.). Proceed replacing version A with version B if B is more effective after considering all possible situations that may happen after doing so.

Repeat this testing process from the beginning to solve another question or problem.

Application of A / B testing

With A / B testing you can apply and improve a lot of things for the running and development process of the web, online / offline advertising, mobile app and email marketing.

1. For website


This is mostly related to the web interface and user experience (UI / UX) as these are factors that directly affect the user performing conversion on the website. With a website you can do A / B testing with all the elements that can affect user behavior such as image, title, content, call to action, information form, etc. Test in turns elements that can be improved to increase conversion rate. Read more about analytical and A / B testing applications for website.

2. For advertising and sales


For online, A / B testing is often used to measure the effectiveness of different types of ads. For example, when you are writing AdWords ads copy for the same keyword ad group, always write 2 different ad templates and run them in parallel to see which works better after some times. Similar to GDN or Facebook ads, use different ad designs for the same campaign to measure the effectiveness and then choose which design is more effective to run. Optimizing your ads regularly by testing different options will help you continually improve your conversion rate and help your ads run more efficiently.

For offline, A / B testing can often be used to evaluate the efficiency of advertising channels such as newspapers, leaflets, billboards, etc. For example, by using different coupon codes for each ad newspapers, leaflets, or billboards, advertisers can know which template is more effective by seeing which coupon code is used more. In some cases, different phone numbers may be used instead of the coupon code.

Another application in store sales may include placement of products in different locations to measure customer interest and eventually encourage them to buy more. This shows the applicability of A / B testing is very diverse and very flexible depending on the set target.

3. For mobile apps


A / B testing is also used in mobile application development and similar to the website, primarily to improve the UI / UX of the product. For mobile applications, testing is often more difficult in term of technical and behavioral testing. Technically, to test, the app version needs to be updated, approved by the AppStore or Google Play and then comes to the user, so it takes more time. In terms of user behavior, not everyone will immediately update the new version, and the mobile user experience is completely different than on the web.

There are a lot of A / B testing tools available for mobile applications in the market, and I personally have had the opportunity to try out the two tools Splitforce and Apptimize and both are pretty good. You can also refer to the list of 20 A / B testing tools for mobile phones.

4. For email marketing


It has gone the days when hundreds of thousands of emails were sent with the thought that users would read their emails. Email clients are having increasingly sophisticated filters, which push all spam emails to the trash and even so customers are buried by dozens or even hundreds of emails every day. It’s important that customers open our email to read and interact with those emails. The answer is A / B testing. You do not know what title will appeal to the reader to increase the open rate, please test. You do not know which CTA to use to get users to click on the link, please test.

Most email-sending tools, such as MailChimp, BenchmarkEmail, now have the ability to allow A / B testing of sent content to measure the effectiveness of the sent campaign.

Advices when doing the A/B testing

1. DOs

Knowing when to stop the test. Too soon: you will lose valuable parameters to make the right decision. Running the test too long, it is also harmful if the test version has too bad performance which can also affect your conversion rate and the sales.

Maintaining the consistency: When conducting A / B testing there must be a way to remember which users have been choosen with which test version to always display the correct version to avoid affecting the user experience. If a button is changed for testing and this button appears in many places on the website, customers must also see this same button everywhere in the website. Cookies are the most commonly used method.

Testing multiple times: The fact is that not every A / B testing session will deliver the results you want or help you find a solution to the problem. So keep testing more and more, in different directions. If each test improves your conversion rate a bit, doing it many times will add up to a greater effect.

Noting the difference between mobile and desktop traffic: visitors to your website from the mobile and from the desktop may have a completely different behavior on your website depending on the design, UI / UX and if website is mobile-friendly. Therefore, it is important to keep in mind the division of traffic when A / B testing your site, preferably you should test mobile and desktop traffic separately.

2. DONTs

Testing without the same conditions: Always remember that testing both versions A and B must be carried out in parallel. You can not run version A in week 1 and version B in week 2 and think that it will give the correct result.

Too early conclusion: Remember that the results are only valid when they have a big enough numerical value and a corresponding time to determine. You can not decide that version A is better than B or vice versa when they differ only a few conversions or the test time is too short.

Make Current Customers Surprised: It is best to conduct A / B testing by just focusing on new customers because if old customers come in and see everything differently than before, then this probably will affect the conversion rate, especially if you are not sure whether the trial version will be selected.

Letting your intuition decide the result: Sometimes the test results may be completely contrary to what you can think of. Maybe a red CTA on your blue background is dazzling and annoying, but the results may prove to be more effective. What you need is conversion rate, do not let your intuition fight against the test results.

Hope this article will answer the questions of those who are learning about A / B testing process to apply to their work. For questions and comments, leave them below or contact us for an answer.

Facebook Comments


Hien are is Director of Marketing Store Vietnam Ringier AG. Digital Marketing and passionate about sharing knowledge as expected những scraped past tense as multiple users of coal, I was quyết create conversion.vn blog. If need to contact me, please email as through tu.bui@conversion.vn

Leave a Reply

Your email address will not be published. Required fields are marked *

Name *
Email *