Playing with various facial recognition softwares: iOs Photo app

The 2020 mass protests denouncing racial violence and racism in the US, UK and western countries in general highlighted a new danger: policing authorities using social media pictures to identify and target activists. Mobile face recognition apps now go from a useful feature, helping taking candid (or not) pictures, to miniature, portable back-stabbers. Since iPhones store the time and location associated with pictures, the risks are even greater. So how well can it track and identify a face?

For iPhone users, going scrolling to the bottom your Albums folder will show an interesting sorting option: “People”. The Photo app constantly tries to associate faces that seem to belong to the same person, and will give you the option to select (or remove) photos it perceived as being of one subject. This also trains it to more accurately find this person in future photos.

My unsuspecting younger self was the base image to identify me

I have over 6 years of photos accumulated on my Cloud, so it does an eerily good job at identifying recurring faces, and struggles more with random people or children growing up, so for the purpose of understanding how to escape the face detection and recognition tool I decided to ignore these. I also do not have the option to borrow random children at the moment so my experiment shall include my face only, taken from the safety of my room.

As we know, the Photo app on iOs does a great job at identifying my unobscured face, hair up and down. But how well can it recognise me when I wear accessories?

I posed in front of a neutral background (namely a white wall), wearing the same outfit and at the same time of the day. I then took multiple sets of pictures: wearing black glasses, wearing golden glasses, no glasses. For each of these three sets, I took pictures from the front, but also turning my head as to obscure one eye, and with exagerated top and bottom views. I also took a second series of said pictures wearing a mask, resulting in a total of sixty-five pictures.

I then headed to my People Album, to see how many versions of my face the Photo App identified as being me. Out of the sixty-five, only fifteen pictures made it to the folder, mostly front-facing and looking to the right, and none of them were of me with the face mask or looking up or down.

12 of the 15 pictures recognised as my face

Now, that result probably indicates that most pictures of me are using these angles, and recent studies showed that when wearing a mask, facial recognition algorithms failed over 50% of the time. With most of my face covered by the mask, I wasn’t expecting it to succeed in identifying me. But what if my face was only partially obscured? I went on and took a few pictures with two strands of hair going over my eyes – and the Photo app failed to recognise me again.

My test samples were obviously minuscule, but it did give me a good idea of ways to conceal my identity on pictures, as well as understanding why CV Dazzle-type makeup and ways to disguise or cover key features of the face can play the algorithm.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s