Blendee provides the possibility to create A/B/X Tests by taking advantage of the flexibility and infinite possibilities offered by the experience manager, by accessing the Experiments section.

Thanks to Experiments , it is possible to test any content that can be created with Blendee’s features: it will then be up to the platform itself, based on the results, to choose which will be the best option.
Automatic mode, but not only: it is of course also possible to opt for the manual mode which leaves the user free to select the winner on the basis of the best performance obtained by the options tested.

The immediate advantage of the solution adopted by Blendee for the management of A/B/X tests is that you do not have to take over an experience already created, stop it, make changes and then relaunch it. In fact, as we will see, everything happens smoothly and once our flows are set up, everything will be automated.

Now let’s see how to proceed with the configuration. First you need to create/edit an experience, in the example we will create an experience for the abandoned cart and we will check which, between two email templates, will be the most performing.
We will then test which, between two behavioral messages, leads to a greater conversion of carts.

Within the Experience Manager you will find, at the bottom left, the section:

At this point, we proceed by dragging the Testing block into our workflow to start configuring our test:

The first thing that is required is the choice ofthe number to be configured to recognize the test to be performed.

IMPORTANT

We recommend that you choose a “speaking” name for the test so that you can better identify it in the subsequent stages of reading the statistics, thus easily distinguishing it from other tests already present.

Let’s now take a closer look at the differences between the two experiment management modes : Automatic and Manual.

“Automatic” mode

Automatic mode is extremely easy to set up. You will only need to choose the evaluation metric to select the winning test. The available metrics are as follows:

• Views/Openings;
• Click;
• Sales;
• Products Sold;
• Turnover;

“Manual” mode

By selecting the “Manual” mode, the choices to be made are greater but always very simple to configure.

Accuracy

Accuracy indicates the time we want Blendee to keep the test active and, as we will see, involves defining the number of users to be engaged before the test can be considered concluded.
Obviously, a “High” accuracy allows you to have more reliable results, but the time in which the test is performed will be longer.
On the other hand, a “Low” accuracy will involve a small number of users in a short time.

The accuracy should also be chosen according to the type of test to be carried out. For example, if we are testing clicks on a message on the portal and visible to all users registered for newsletters, we can choose a low accuracy as we will not need much time and many users to understand which option is more performing.

Advanced Configuration

Once the accuracy has been selected, you can change the basic parameters by clicking on “Advanced Configuration”:

Within the advanced configuration, the data will be displayed according to the accuracy previously chosen and it is possible, if necessary, to modify the parameters:

• Maximum users involved in the test
• Maximum duration of the test (can be expressed in: hours, minutes, days, months and it is also possible to define a maximum end date of the test)
• Waiting time: allows you to define a minimum waiting time to be taken into account before calculating performance statistics.

The test is then configured, and all that remains is to proceed with the creation of the content to be tested. As mentioned, in our example, first of all we are going to test two emails for users who have an abandoned cart. The test includes the “Automatic” mode with the choice of the winner based on clicks.

Here’s the full test that’s also configured to evaluate the custom message.

As you can see from the image, the links between the content blocks (email and behavioural messages) and the test have a different colour from the other links to allow a better reading of the flow.

IMPORTANT

As you can see from the image, both test blocks are connected to the same output block, this is because the evaluation of the test does not take place with regard to the “paths” that we can create based on the various conditions: the tests aim to evaluate which of two contents/copy/templates is more performing.

It is very important, when testing Web Personalizations (recommendations, behavioural messages, etc…), to place a waiting block so that users do not immediately exit the experience and view one of the tests.

Choosing the “Winner”

Once the experience is launched, Blendee allows you to view separate statistics for each Experiments and choose the winner.
To proceed:

• select the “Experiments” button next to the name of the experience in the list of experiences
• Once clicked, we will choose the experiment to analyze.

In the analysis of statistics, within an experiment in automatic mode, Blendee will automatically select the winner but the user can still change it by clicking on the “Edit” button;

The statistical analysis can then be further deepened by verifying the results broken down by user segment.
In the “Filter by Segments” area, you can select the segments you want to analyze:

By clicking on “Search”, in the area below you will be able to view the results divided by segment.