How to create an A/B test
Last updated
Last updated
A/B testing is a method used to compare two versions of a website to determine which one performs better in terms of achieving a specific goal, such as increasing conversions. The variations can involve changes in design, layout, content, calls-to-action, or any other element of the webpage that may impact user behavior.
Start with a hypothesis about what changes might improve the performance of your webpage. For example, you might hypothesise that changing the color of a call-to-action button will increase click-through rates.
Log in to the dashboard and navigate to the Experience page from the sidebar.
Click on the 'Create Experience' button.
After giving your A/B test a name and a description, click the "Create Experience" button.
Add the page path(s) on which you want to run the A/B test. For example, if you want to run an A/B test on the pricing page, then enter the pricing page path.
Click on the 'Add pages' button.
From the drop-down, select the required option.
Simple page path match: It checks if the given page path matches exactly.
Page path contains: It checks if the page path contains the given text.
Page path regex matches with: It checks if the page path matches the given regex.
Enter the page path and click on the 'Add' button.
You can create multiple behaviors (variations) on your website or webpage. When you create an A/B test, then by default, a default behavior will be created. This will be your default (control) website, and you will not be allowed to delete default behavior. To add the new behavior
Click on the 'Add Behavior' button.
Then give a proper Name and Description for your behavior.
Click on the 'Add' button.
To edit the webpage, click on the 'open visual editor' button present on the respective behavior.
You can target A/B tests to specific audience segments. This will allow you to gain deeper insights into how different variations resonate with different groups of visitors. For example, you can restrict the A/B test to visitors from the USA using mobile devices.
Click on the 'Add Rules' button.
Choose the audience category you wish to target for the A/B test from the popup.
Then, from the attribute dropdown, select one of the attributes. For the above example, the audience category will be geography, and the attribute will be country.
Then, from the operation dropdown, select one of the operations, such as 'is one of' or 'is not one of' based on your requirement. For the above example, the operation type will be 'is one of'.
Then, provide the value in the input field. For the above example, the value will be the United States. You can enter multiple values. In a few of the cases, you can see the dropdown option under the values section.
Finally, under the targeting behavior dropdown, select the percentage rollout option. If you want to do an A/B test between default and one of the behavior then configure the value as 50% for default and 50% for the newly created behavior. And if you want to do the A/B test between two different behaviors, then configure the value as 50% for both behaviors.
Once the target audience is configured, make sure to click on the 'Save' button.
Running an A/B test without a goal(s) is of no use. Goals define the effectiveness of an experience. For example, if you are running an A/B test on the home page, then the goal event can be clicking on the signup button or the pricing page. You can choose different types of goals based on your requirements.
Page: Number of visitors who visit the given page path after they are part of an experience.
Click: Number of visitors who click on the configured element, such as a button click.
Custom: You can create your own custom events and track them based on your requirements.
Revenue: Tracks the revenue generated from an experience.
In the experience goals tab, perform the below steps to add a goal
Click on the 'Add goals' button.
In the popup, select the type of goal event you want to add.
Then select the goal event that you want to add.
Enter the access code and click on the 'Add' button.
If you don't find your goal event, then you can create a new goal event by clicking on the 'Create a new goal' option.
By default, the first goal event that you add to an A/B test will be considered the primary goal. If you have multiple goal events in an A/B test, then you can make any one of the goal events the primary goal event.
Once an A/B test is configured, make sure to preview the changes you have made to each behaviour before making it live. You can preview the behavior by clicking on the 'preview' button present on the respective behavior.
If you do not see the changes in the preview, then check if the CustomFit code snippet is installed on the website. If you have installed the code snippet, then check whether your browser is blocking the opening of new tabs.
To take the A/B test live click on the power switch toggle.
