A01 chaf6542 06 se fm indd

Q. Online channels bring great opportunities to test propositions online. Can 

you advise on approaches to testing propositions?

Roberto Hortal:

 I have used online channels to test propositions in a couple of ways:

● 

Assign propositions randomly to visitors on first arrival to test interest/sales. I would 

typically run this against a large control group (being offered the current main prop‑

osition) to both protect commercial results and detect the effect of any external 

influences that maybe otherwise wrongly influence the experiment. This approach 

can be extended beyond the site, via randomised allocation of marketing messages 

on Display, Search etc. to measure a proposition’s attractiveness. It’s important 

to test all aspects of the proposition: a very successful proposition at attracting 

interest may convert badly if it can’t be priced at a level that matches customer 

expectations.

● 

Provide a modular proposition platform and allow customers to combine proposi‑

tion elements. We can easily analyse popular combinations as well as secondary 

correlations such as the propensity to add a certain ancillary product to a particular 

proposition configuration, and understand the compound impact on profitability, 

retention and advocacy from what we know about each of the modular components 

in isolation.

I’ve found that proposition testing rarely fits neatly an A/B scenario, with tests quickly 

developing into complex multivariate experiments with a significant number of vari‑

ables. It is important to ensure the tests are solidly planned, rigorously executed and 

statistically significant. Free tools like Google Website Optimiser provide information 

to test and get the best out of website contact. However, such tools will not prevent 

badly designed experiments from yielding wrong data. In my experience, the only way 

to ensure valid tests and improved business results is to bring in the best analyti‑

cal brain you can find. Analytics is the first role I fill when I build a digital team from 

scratch, it’s that important.