So far in our five-stop tour of the use of biometrics in conversion optimisation, we have investigated: eye-tracking – for finding out where your customers are looking, EEG – for measuring their motivation or memory load, pupil dilation – for measuring their general arousal, and galvanic skin response, for measuring their emotional arousal.

But these measures can be ambiguous: Knowing that your customers are experiencing emotional arousal is one thing; knowing whether that arousal is positive (e.g., joy) or negative (e.g., frustration) is quite another.

That’s where the final piece of the jigsaw – facial expression analysis – comes in.

Analysing people’s likes and dislikes

Some of the first studies of face-reading technology investigated customers’ tastes – quite literally.

For example, in a 2014 study, Dr Grazina Juodeikiene¹ and her colleagues at Kaunas University of Technology, Lithuania showed that facial expression reading software could predict the ratings that participants would give to 22 different types of sweets and chocolates (even though they didn’t know they were having their expressions analysed).

Traditionally, customers’ responses to different media have been captured by bringing volunteers into the lab and giving them dials to indicate their like (or dislike) of what they’re seeing (particularly for TV shows and movies).

But a 2013 study led by Evan Kodra² at Northeastern University showed that more reliable data could be obtained at a fraction of the cost by using facial expression recognition software instead.

Predicting user preferences and intent to purchase

Although these early studies were conducted offline, more recent studies have demonstrated that the same techniques can be used to investigate consumers’ online responses to websites.

For example, a 2015 study led by Daniel McDuff³ at MIT used customers’ own webcams (with their consent!) to collect over a thousand people’s facial responses to adverts for 170 different products, from pet-care, to chewing gum to pasta.

This resulted in over 3.7 million video frames worth of data, which were coded individually for the facial expressions shown (obviously not by hand, but by a computer algorithm). This data could predict not only customer’s rated “liking” of the various products, but also their “intentions to purchase”.

Creating adverts and websites that strike the right chord

Facial expression recognition technology can also be used to shape individual adverts or websites.

A 2014 study led by Thales Teixeira⁴, a Professor at Harvard Business School, found that – when it comes to entertainment in advertising – there is a sweet spot: a little bit of entertainment (as measured by positive facial responses) results in higher “intention to purchase” ratings, especially when that entertainment is focussed on the brand.

But too much entertainment distracts customers from the brand, and although the facial recognition software shows them to be having the time of their lives, their “intention to purchase” ratings are actually lower.

As this study shows, there is rarely a one-size-fits-all solution. To get the most out of facial expression recognition technology (or any other biometric measure) you need not only the raw data, but expert knowledge of how to interpret it.

And with many years of experience in Biometric Research, we at Endless Gain are both the pioneers and the masters.

References:

1. Juodeikiene, G., Basinskiene, L., Vidmantiene, D., Klupsaite, D. and Bartkiene, E. (2014). The use of face reading technology to predict consumer acceptance of confectionery products. In Proceedings of Foodbalt, the 9th Baltic Conference on Food Science and Technology “Food for Consumer Well-Being” pp. 276-279.
2. Kodra, E., Senechal, T., McDuff, D., & El Kaliouby, R. (2013, April). From dials to facial coding: Automated detection of spontaneous facial expressions for media research. In Automatic Face and Gesture Recognition (FG), 2013 10th IEEE International Conference and Workshops on (pp. 1-6). IEEE.
3. McDuff, D., El Kaliouby, R., Cohn, J. F., & Picard, R. W. (2015). Predicting ad liking and purchase intent: Large-scale analysis of facial responses to ads. IEEE Transactions on Affective Computing, 6(3), 223-235.
4. Teixeira, T., Picard, R., & El Kaliouby, R. (2014). Why, when, and how much to entertain consumers in advertisements? A web-based facial tracking field study. Marketing Science, 33(6), 809-827.

‍