BUFFALO, N.Y. -- Inspired by the work of psychologists who study
the human face for clues that someone is telling a high-stakes lie,
UB computer scientists are exploring whether machines can also read
the visual cues that give away deceit.
Results so far are promising: In a study of 40 videotaped
conversations, an automated system that analyzed eye movements
correctly identified whether interview subjects were lying or
telling the truth 82.5 percent of the time.
That's a better accuracy rate than expert human interrogators
typically achieve in lie-detection judgment experiments, said
Ifeoma Nwogu, a research assistant professor at UB's Center for
Unified Biometrics and Sensors (CUBS) who helped develop the
system. In published results, even experienced interrogators
average closer to 65 percent, Nwogu said.
"What we wanted to understand was whether there are signal
changes emitted by people when they are lying, and can machines
detect them? The answer was yes, and yes," said Nwogu, whose full
name is pronounced "e-fo-ma nwo-gu."
The research was peer-reviewed, published and presented as part
of the 2011 IEEE Conference on Automatic Face and Gesture
Nwogu's colleagues on the study included CUBS scientists Nisha
Bhaskaran and Venu Govindaraju, and UB communication professor Mark
G. Frank, a behavioral scientist whose primary area of research has
been facial expressions and deception.
In the past, Frank's attempts to automate deceit detection have
used systems that analyze changes in body heat or examine a slew of
involuntary facial expressions.
The automated UB system tracked a different trait -- eye
movement. The system employed a statistical technique to model how
people moved their eyes in two distinct situations: during regular
conversation, and while fielding a question designed to prompt a
People whose pattern of eye movements changed between the first
and second scenario were assumed to be lying, while those who
maintained consistent eye movement were assumed to be telling the
truth. In other words, when the critical question was asked, a
strong deviation from normal eye movement patterns suggested a
Previous experiments in which human judges coded facial
movements found documentable differences in eye contact at times
when subjects told a high-stakes lie.
What Nwogu and fellow computer scientists did was create an
automated system that could verify and improve upon information
used by human coders to successfully classify liars and truth
tellers. The next step will be to expand the number of subjects
studied and develop automated systems that analyze body language in
addition to eye contact.
Nwogu said that while the sample size was small, the findings
They suggest that computers may be able to learn enough about a
person's behavior in a short time to assist with a task that
challenges even experienced interrogators. The videos used in the
study showed people with various skin colors, head poses, lighting
and obstructions such as glasses.
This does not mean machines are ready to replace human
questioners, however -- only that computers can be a helpful tool
in identifying liars, Nwogu said.
She noted that the technology is not foolproof: A very small
percentage of subjects studied were excellent liars, maintaining
their usual eye movement patterns as they lied. Also, the nature of
an interrogation and interrogators' expertise can influence the
effectiveness of the lie-detection method.
The videos used in the study were culled from a set of 132 that
Frank recorded during a previous experiment.
In Frank's original study, 132 interview subjects were given the
option to "steal" a check made out to a political party or cause
they strongly opposed.
Subjects who took the check but lied about it successfully to a
retired law enforcement interrogator received rewards for
themselves and a group they supported; Subjects caught lying
incurred a penalty: they and their group received no money, but the
group they despised did. Subjects who did not steal the check faced
similar punishment if judged lying, but received a smaller sum for
being judged truthful.
The interrogators opened each interview by posing basic,
everyday questions. Following this mundane conversation, the
interrogators asked about the check. At this critical point, the
monetary rewards and penalties increased the stakes of lying,
creating an incentive to deceive and do it well.
In their study on automated deceit detection, Nwogu and her
colleagues selected 40 videotaped interrogations.
They used the mundane beginning of each to establish what
normal, baseline eye movement looked like for each subject,
focusing on the rate of blinking and the frequency with which
people shifted their direction of gaze.
The scientists then used their automated system to compare each
subject's baseline eye movements with eye movements during the
critical section of each interrogation -- the point at which
interrogators stopped asking everyday questions and began inquiring
about the check.
If the machine detected unusual variations from baseline eye
movements at this time, the researchers predicted the subject was