Trained on “people categories” from the ImageNet dataset – the databank used as a standard benchmark in image classification and object detection – the tool labels uploaded photos according to over 20,000 categories.
With 2,833 sub-categories for the person tag, people are classified according to race, nationality, profession, behaviour, etc. Whilst some categorisations may be as banal as ”lass”, or as wildly specific as “sociolinguist”, others are “dubious and cruel”, with some using racist or misogynistic terminology pulled from its learning source – ImageNet. As amusing as an unfortunate label may be among friends, it reveals a darker side to AI. The developers commented, “we want to shed light on what happens when technical systems are trained on problematic training data. AI classifications of people are rarely made visible to the people being classified. ImageNet Roulette provides a glimpse into that process – and to show the ways things can go wrong”.
