I’ve discussed before the increasing use of facial recognition technology in advertising, especially in “digital signage.” Most people have heard of the term connected with billboards or other screens that have cameras (and facial-recognition technology) to watch people watching ads in order to improve their marketing. The digital signs log data such as gender, approximate age and how long someone looks at an advertisement. This is supposed to help build a better billboard — one that is tailored specifically to the individual standing in front of it. However, the data-gathering and surveillance practices raise substantial privacy questions. (Disclosure: The Center for Democracy and Technology has released a set of privacy guidelines for digital signage, which I consulted on and contributed to, in the report “Building the Digital Out-Of-Home Privacy Infrastructure.”)
USA Today and the New York Times have detailed safety problems that can arise from these digital billboards. BBC News has reported on the use of digital billboards in the United Kingdom. The Wall Street Journal has reported on digital signage use in Japan. The Los Angeles Times reported on the expansion of these digital billboards and their use of facial-recognition biometric technology in casinos, Chicago-area bars and more.
Now, the Economist takes a look at the use of facial recognition technology in digital signs and how advertisers are gathering data on people beyond just what they’re looking at — the tech can read your mood and check your vital signs, as well:
[M]aking online ads that not only know you are looking at them but also respond to your emotions will soon be possible, thanks to the power of image-processing software and the ubiquity of tiny cameras in computers and mobile devices.
Uses for this technology would not, of course, be confined to advertising. There is ample scope to deploy it in areas like security, computer gaming, education and health care. […]
One of the companies doing such work, Realeyes, which is based in London, has been developing a system that combines eye-spying webcams with emotional analysis. Mihkel Jäätma, who founded the company in 2007, says that his system is able to gauge a person’s mood by plotting the position of facial features, such as eyebrows, mouth and nostrils, and employing clever algorithms to interpret changes in their alignment—as when eyebrows are raised in surprise, say. Add eye-movement tracking, hinting at which display ads were overlooked and which were studied for any period of time, and the approach offers precisely the sort of quantitative data brand managers yearn for. […]
As similar gimmicks become widespread, privacy concerns will invariably mount. People would need to give consent to their webcams being used in this way, Mr Jäätma admits. […]
In fact, webcams that monitor a person’s heart rate are soon to appear. Instead of sticking sensors onto the skin, Philips has developed a vital-signs camera system which the Dutch company says can measure heart and respiration rates extremely accurately. To calculate the heart rate the camera detects tiny changes in the colour of the skin. These changes, imperceptible to the human eye, occur as the heart pumps blood through the body. The person’s breathing rate is measured by detecting the rise and fall of his chest.