Advanced NSFW AI applications fail to perform equally in diverse communities because of cultural, social, and contextual issues. In 2021, research by the AI Now Institute at New York University said that AI models, since they are mostly trained on content from the West, were more likely to misclassify content from non-Western cultures. For example, some cultural symbols or ways of expression in countries from Africa and the Middle East may not denote nudity but, through artificial intelligence systems, they are misinterpreted to mean explicit material. This is a bias that happens because most of the training data the NSFW AI uses stems from regions whose cultural norms, as far as nudity and sexuality are concerned, are very different from the rest of the world.
Advanced NSFW AI usually fails to properly classify contents in the context of communities with alternative lifestyles or non-mainstream ways of sexual expression. A case in point is that in 2020, researchers at the University of Cambridge found AI models failed to identify explicit content if it was queer, non-binary, or transgender-themed. These AI systems, trained on binary gender assumptions, sometimes mislabel content that does not fall into the traditional categories of heteronormative and cisgender. In fact, a 2022 survey conducted by the EFF found that 30% of LGBTQ+ users considered themselves to be unfairly censored by AI content moderation systems due to the misclassification of their content.
Limitations also clearly emerge in communities where high amounts of creative or artistic work are created that happens to feature nudity or sexual themes. In 2023, a report by the Digital Civil Liberties Union underlined how AI models could not tell explicit adult content apart from the nudity in works by famous painters or photographers. With increased governance over creative industries by AI systems, many artists say their work is flagged or removed, leading to lost revenue and suppressed expression. It gets even more complicated when AI flags non-explicit images or videos as explicit, impacting both content creators and consumers.
The performance of AI across diverse communities greatly depends on the region in use due to concerns about access to technology and internet censorship laws. AI models, while doing content moderation in countries like China with strict Internet censorship, tend to over-censor certain content types because of the local laws and norms modeled within. In 2022, for example, China issued new regulations for online firms, including the strict rules on the moderation of adult content. This, in turn, put China’s nsfw AI systems into a situation where most of the non-explicit content was flagged for either political or social reasons, reflecting divergence in performance by geographic location.
Where global communities generate content, navigating languages becomes an added layer of complexity that AI systems often must work through. Text-based systems, such as those analyzing explicit conversations or texts, encounter difficulties when analyzing slang, dialects, or languages that are underrepresented in the training data. In 2021, OpenAI’s GPT-3 language model, when used in a sexual content detection application, misclassified explicit content in non-English languages, especially those using slang or regional terms. This means that it was 25% more prone to errors than content in English, showcasing a very big performance gap for those speaking languages other than English.
Finally, ethical concerns regarding how NSFW AI systems are performing within diverse communities have drawn complaints from civil rights groups. In 2020, the American Civil Liberties Union released a report about how AI-based content moderation disproportionately affects communities of color and non-Western communities. It said that AI systems are more likely to misidentify explicit content from these groups due to either cultural bias or lack of training data. This is part of a larger trend in which the diversity of users is not reflected in the data on which content moderation systems have been trained.
As AI technology continues to evolve, such limitations in performance across diverse communities raise the need to incorporate an inclusive approach to training data, better representation of global cultural contexts, and a deeper understanding of how different societies think about explicit content.