Humanizing Facial Detection with the Teenie Harris Archive

February 24th, 2017 by Zaria Howard


Oftentimes when doing face detection on a datset, it's more about the location of where people are in the photo. However, when using Microsoft's Face and Emotion API, we also received an estimate of the age, gender, and emotion expression of each face. Seeing that the gender given by Microsoft's API is binary, the main statistic on this attribute is pretty straight forward: 39.5% of the faces are women.

A more complex approach to looking at this data is to see if the emotions and ages captured differ based on gender possibly due to cultural influence or photographer preference. An example of this would be if the women in the pictures tended to be younger (like in mainstream media today). Another example would be if a higher proportion of angry men were captured as opposed to angry women.

The first step is to look at the ranges of age and emotions that we have and see how Microsoft's predictions match up with our own perception. The two women below have the oldest age estimations in the database at 95 years old.

Teenie Harris also captured the cutest baby in two photos. In each picture the baby was estimated to be 0 years old.

These two photos below contain the two most surprised faces. Surprise is the most common emotion after happiness and neutrality in this archive. This suggest that Teenie preferred to cover more positive experiences for the community. Although both pictures below have over a 99% confidence, I'm not sure if I would characterize that face in the Christmas photo by the word "surprise".

Disgust is the least common emotion in the archive with only 3 out of 200,000 faces giving more than a 50% probability of being disgusted. The next least common emotion is fear, and the most fearful guy shown below only had a 65% confidence associated with that emotion of his face. Sadness, although uncommon, usually comes with much higher confidence. Most likely because the face changes when a person is sad is a lot more dramatic and not very ambiguous. The photos below show the most disgusted, most fearful, and the saddest face. It's quite obvious which one is which.

Now that we've seen some of the range of people in the Teenie Archive, it makes more sense to look at this faces as a whole. The histograms below are visual representations of the more significant results.

Although women and men have the same average age at 34 years old, the distributions by age are a little different. The age distribution of men seems to be slightly more spread out with more elderly people and children represented. Women tend to be more represented in the 18-30 age range, the time when women are generally percieved to be at peak attractiveness.

Happiness and Neutral expressions are overwhelmingly the most common in the dataset. However, it is interesting to note that women tend to be happier in the photos and the men tend to appear neutral more frequently. These facts really speak to the ideas of femininity and masculinity that were present during that time period.Generally, the emotions that aren't happiness and neutral tend to have a much more exponential distribution, with over 90% of the faces leaning towards a probability of zero for those emotions. It was very rare to have people on camera that weren't happy or at least neutral.

A link to a python notebook with the results can be found here.