Brands have already begun using facial recognition technology in their Customer Experience. Walmart and KFC use facial recognition technology. Passengers on Delta check in for flights in the Atlanta airport using facial recognition. Even my phone password is my face.
Technology also exists that captures customers’ authentic emotion measurement using facial recognition software. I believe both of these technologies are the future for Customer Experience and, in some cases, the now.
But is it creepy?
We had Professor Bill Hedgcock, associate professor of marketing Carlson School of Management at the University of Minnesota, as a guest on a recent podcast to talk about facial recognition and facial expression analysis technology and application in Customer Experience programs. As an academic marketer and an expert in neurology, Hedgcock does psychology-based research and studies neuromarketing, or how the brain works when we make decisions.
Hedgcock explained the difference between facial recognition and facial expression analysis:
- Facial recognition takes someone’s face and matches it to other images.
- Facial expression analysis figures out what emotion you feel based on how your face looks.
The two technologies involve identifying a face in a screen, what direction it is facing, and landmarks, like where a nose is or where the ears are.
However, at this stage with the landmarks, the two technologies deviate:
- Facial recognition takes all these landmarks and matches them to other images, like your photo software or Facebook.
- Facial expression analysis uses these landmarks to determine if the person is showing joy or sadness or anger.
So, Is it Creepy?
Facial recognition data collection about customers during a Customer Experience is the future. The uncomfortable feelings we have are a reaction to experiencing something that is different from what we have had before. When you look into the details, the odds of harm are low to nonexistent. In other words, once we get used to it, it won’t feel creepy anymore.
When you consider the chance for harm, the regular data that companies collect all the time is more creepy to me than facial recognition and facial expression analysis data. For example, companies now use your Wi-Fi connection to know where you are when you linger in a store and don’t get me started on the sheer amount of information “they” have about your clickstream data (i.e., the websites you visit and in what order). Sure, you know data is collected in these cases, but “they” never tell you how it’s going to be used. Frankly, this data contains more private things about you than information collected about your facial expressions.
How to Help People Get Over the Creepy Feeling
Hedgcock and his team were trying to determine if facial expression data could be used by retail or marketing departments. Hedgcock and the team decided to do a study to determine if people’s emotions could predict their food consumption. They called it “The Creepy Project.” Why? Their unconventional name reminded them to do what they could to minimize any perception of malfeasance.
The research team diffused the discomfort people feel about the technology in a couple of ways:
- They used a camera that was already there and in a public setting. The significance of this fact was that it was not a place where people expected privacy.
- They didn’t record video and the participants were anonymous. They don’t even have examples of actual participants in the study because the software can take an image from the screen to encode what the expression is and never record it.
To help people adapt, companies should provide some benefit to the customer when they use facial recognition and facial expression analysis software. The facial recognition on Facebook is useful because it can match the photos for me and my kids. Google used to mine all kinds of data from people’s Gmail accounts and people were OK with that because they got free email.
So, for example, let’s say you use ice cream to cheer yourself up when you are having a bad day, that you “eat your emotions,” as it were. Imagine if the data collected from this technology developed an app that would recognize when you were feeling down and intervene before you head for the ice cream shop. What’s more, it directs you to get a healthy snack instead. An app like that could be quite useful…to me, anyway.
As NBC News points out, the film Minority Report explored this concept a bit back in the early 2000s.
In the futuristic film, the adverts would use a retinal scan to find you. Using all the data about you, the display would show you a specific offer.
Per Hedgcock, that’s not science fiction anymore. There are companies using this, sans the retinal scan. Now, all they need is your face, so it’s less intrusive than the movie’s predicted technology. Moreover, it isn’t as specific as the movie was. They are using broader demographic information to target their message, e.g., gender or age, to name a couple.
It is targeted marketing. While it sounds creepy at first, it is useful once you experience it. I’d prefer to hear about relevant information than irrelevant information, wouldn’t you? Furthermore, the examples Hedgcock has seen are funny and entertaining. Sometimes the billboard interacts with you by reading your facial expression.
Making Customers Comfortable with Facial Expression Analysis
Facial recognition and facial expression analysis is where things are going. However, there is potential for very bad public relations blow back if this is not handled with sensitivity.
No matter how logical you are in your arguments that the technology is not creepy, people respond to it emotionally just like they do everything else. In other words, creepy is a subjective feeling. If it feels creepy to customers, it’s creepy.
However, you can decrease perceived creepiness in the following ways:
Be transparent about your collection of it.
People might perceive facial recognition technology and facial expression analysis as less creepy if they are aware you are collecting this information and it’s anonymous rather than collecting it without their awareness.
Moreover, it’s the law to notify people you are collecting this data. In our global Customer Experience Consultancy, we’ve been talking to lawyers about this for a client. We learned that you must advertise that you use facial recognition or facial expression analysis. It’s a bit like when you phone a call center and they say, “This call may be recorded for training purposes.”
Provide Value.
Companies have a lot of our data. We are usually okay with it when the company provides value with it. For example, I like when I go on to Amazon and it uses the data it has on me to give me better recommendations.
Many times, I wish Netflix used this data more to do a better job of figuring out what movies and TV shows I wanted to watch. I know they know what TV show I’m watching. I want them to use that data.
Keep it as general as possible.
If the data you collect seems like it’s private, like the customer did not mean to show you this expression, a person will have negative feelings about the technology. Instead, follow Hedgcock’s lead and use a public camera in a public place were other people can see, too, and don’t record the images. Instead, analyze them in real-time (with software) and let the images go afterward. If a person knows their facial expressions aren’t being recorded and stored, they are less likely to object to the use of the technology.
Understand people’s emotions and moods.
There are obvious times where knowing moods is important. So, for example, if someone is in a bad mood, wouldn’t it be great to offer them excellent customer service?
However, there are also some less obvious things. Your mood changes your decision-making in predictable ways. When I’m in a bad mood, I evaluate things less. For others, happiness leads to glossing over details. My point is moods would change your purchases in subtle but predictable ways.
The issue of the technology’s creepiness has to be overcome. Transparency, adding value, keeping it general, and understanding emotions will help.
Facial recognition and facial expression analysis is the next level of authentic emotional data collection from your Customer Experience. It’s time to stop playing about and experimenting with it and put it to work.
Sincerely,
Big Brother*
* I’m just kidding, for heaven’s sake.
If you want to benchmark your organization’s performance in the new world of behavioral economics against other companies, take our short questionnaire. Once you submit, we compare your answers against what we know about the market and send you a free personalized report about where your organization is today.
If you enjoyed this post, you might be interested in the following blogs and podcasts:
Why Too Many Choices Can Be Killing Your Bottom Line
10 Things You Must Know To Establish And Preserve Customer Service Culture
Branding And CX: The Surprising Way They Work Together To Form Customer Loyalty
Colin Shaw is the founder and CEO of Beyond Philosophy, one of the world’s leading Customer experience consultancy & training organizations. Colin is an international author of six bestselling books and an engaging keynote speaker.
Follow Colin Shaw on Twitter @ColinShaw_CX