Google says its AI image-generator would sometimes ‘overcompensate’ for diversity::Google apologized Friday for its faulty rollout of a new artificial intelligence image-generator, acknowledging that in some cases the tool would “overcompensate” in seeking a diverse range of people even when such a range didn’t make sense.
Also, when the prompt is modified to include “native american” it seems to mostly return the most stereotypically dressed people possible. Like wearing traditional garbs and headdresses when everyone else portrayed is wearing setting appropriate clothing.
Yep, it’s racism piled on top of racism. Aboriginal people are rarely included in the training data, but when they are it’s mostly wearing what they wear for tourists, and rarely what they wear on a day-to-day basis in the modern world. As a result, that’s what you get in the output.
The real fix would be to fix the training data, but that’s difficult. It’s much easier to train the SALAMI on the racist things that you find all over the web, than to be selective and say “sure, this may be on the web, but it isn’t representative of reality”.