A01 chaf6542 06 se fm indd

Part 3  Implementation

Multivariate testing

Multivariate testing is a more sophisticated form of AB testing which enables simultaneous 

testing of pages for different combinations of page elements that are being tested. This enables 

selection of the most effective combination of design elements to achieve the desired goal.

An example of a multivariate test is shown in Mini case study 12.1.

Test

A (Control)

B (Test page)

Test 1

Original page

New headline, existing button, existing body copy

Test 2

Original page

Existing headline, new button, existing body copy

Test 3

Original page

Existing headline, existing button, new body copy

Table 12.8

AB test example

Mini Case Study 12.1

The National Express Group is the leading provider of travel solutions in the UK. Around 1 billion journeys 

a year are made worldwide on National Express Group’s bus, train, light rail and express coach and airport 

operations. A significant proportion of ticket bookings are made online through the company’s website at 

www.nationalexpress.com.

Multivariate testing at National Express Group increases  

conversion rate

Figure 12.15

National Express page assessed through multivariate testing

Source: www.national express.com, National Express Group PLC.

M12_CHAF6542_06_SE_C12.indd   636

7/22/14   1:41 AM

---

637

Chapter 12  Digital business service implementation and optimisation

The company used multivariate testing provider Maxymiser to run an experiment to improve conversion 

rate of a fare selection page which was the penultimate step in booking (Figure 12.15). The analysis team 

identified a number of subtle alterations to content (labelled A to E) and calls to action on the page with the 

aim of stimulating visitor engagement and driving a higher percentage of visitors through to successful con-

version without changing the structure of the page or National Express brand identity. In order to aid more 

effective up-sell to insurance  add-  ons, changes to this call to action were also proposed.

It was decided that a multivariate test would be the most effective approach to determine the  best- 

 performing combination of content. The variants jointly developed by Maxymiser and the client were tested 

with all live site visitors and the conversion rate of each combination monitored. They tried 3,500 possible 

page combinations and during the live test the underperforming combinations were taken out to maximise 

conversion rates at every stage.

At the end of the testing period, after reaching statistical validity, results showed that the best combina-

tion of elements showed a 14.11% increase in conversion rates for the page, i.e. 14.11% more visitors were 

sent through to the fourth and final step in the registration process, immediately hitting bottom line revenue 

for National Express (Figure 12.16).

Figure 12.16

Results of multivariate testing for National Express