ANI
25 Jan 2021, 22:07 GMT+10
Washington [US], January 25 (ANI): A team from the Computer Vision Center (CVC) and the University of Barcelona has published the results of a study that evaluates the accuracy and bias in gender and skin colour of automatic face recognition algorithms tested with real-world data.
Although the top solutions exceed 99.9 percent of accuracy, researchers have detected some groups that show higher false positive or false negative rates.
Face recognition has been routinely used by both private and governmental organizations worldwide. Automatic face recognition can be used for legitimate and beneficial purposes but at the same time its power and ubiquity increase the potential negative impact that unfair methods can have on society. A necessary, albeit not sufficient, condition for a legitimate deployment of face recognition algorithms is the equal accuracy for all demographic groups.
With this purpose in mind, researchers from the Human Pose Recovery and Behavior Analysis Group at the Computer Vision Center (CVC) - University of Barcelona (UB), led by Sergio Escalera, organized a challenge within the European Conference of Computer Vision (ECCV) 2020.
The results, recently published in the journal Computer Vision - ECCV 2020 Workshops, evaluated the accuracy of the submitted algorithms by the participants on the face verification task in the presence of other confounding attributes.
The challenge was a success, since "it attracted 151 participants, who made more than 1,800 submissions in total, exceeding our expectations regarding the number of participants and submissions" explained Sergio Escalera, also member of the Institute of Mathematics of the UB.
The participants used a not balanced image dataset, which simulates a real world scenario where AI based models are supposed to be trained and evaluated on imbalanced data. In total, they worked with 152,917 images from 6,139 identities.
The images were annotated for two protected attributes: gender and skin colour; and five legitimate attributes: age group, head pose, image source, wearing glasses and a bounding box size.
The obtained results were very promising. Top winning solutions exceeded 99.9 percent of accuracy while achieving very low scores in the proposed bias metrics, "which can be considered a step toward the development of fairer face recognition methods" expounded Julio C. S. Jacques Jr., researcher at the CVC and at the Open University of Catalonia.
The analysis of top 10 teams showed higher false positive rates for females with dark skin tone and for samples where both individuals wear glasses. In contrast, there were higher false negative rates for males with light skin tone and for samples where both individuals were aged 35 and below.
Also, it was found that in the dataset individuals younger than 35 wear glasses less often than older individuals, resulting in a combination of effects of these attributes.
"This was not a surprise, since the adopted dataset was not balanced regarding the different demographic attributes. However, it shows that overall accuracy is not enough when the goal is to build fair face recognition methods, and that future works on the topic must take into account accuracy and bias mitigation together," concluded Julio C. S. Jacques Jr. (ANI)Get a daily dose of Irish Sun news through our daily email, its complimentary and keeps you fully up to date with world and business news as well.
Publish news of your business, community or sports group, personnel appointments, major event and more by submitting a news release to Irish Sun.
More InformationMOSCOW, Russia: Just hours after his sudden dismissal by President Vladimir Putin, Russia's former transport minister, Roman Starovoit,...
DHARAMSHALA, India: The Dalai Lama turned 90 on July 6, celebrated by thousands of followers in the Himalayan town of Dharamshala,...
ZAGREB, Croatia: A massive concert by popular Croatian singer Marko Perković, known by his stage name Thompson, has drawn widespread...
WASHINGTON, D.C.: Elon Musk's entry into the political arena is drawing pushback from top U.S. officials and investors, as his decision...
CULVER CITY, California: TikTok is preparing to roll out a separate version of its app for U.S. users, as efforts to secure a sale...
WASHINGTON, D.C.: President Donald Trump claimed he was unaware that the term shylock is regarded as antisemitic when he used it in...
ZAGREB, Croatia: A massive concert by popular Croatian singer Marko Perković, known by his stage name Thompson, has drawn widespread...
DUBLIN, Ireland: Despite extreme heat gripping much of mainland Europe, Irish holidaymakers are pressing ahead with their travel plans,...
HONG KONG: China has fired back at the European Union in an escalating trade dispute by imposing new restrictions on medical device...
PARIS, France: A strike by French air traffic controllers demanding improved working conditions caused significant disruptions during...
ROME, Italy: Quick thinking by emergency responders helped prevent greater devastation after a gas station explosion in southeastern...
WASHINGTON, D.C.: President Donald Trump is drawing praise from his core supporters after halting key arms shipments to Ukraine, a...