AI\u2019s biases<\/p>\n
Typical social media AI-based moderating systems<\/a> use machine learning classifiers<\/a> to analyze posts and identify and flag content that breaks platform rules. But it\u2019s difficult for a machine to understand the nuances of sarcasm, satire, and slang.<\/p>\n Further, some research (examples here<\/a>, here<\/a>, and here<\/a>) suggests that marginalized groups can be disproportionately affected by AI moderation. Without human oversight, AI can end up penalizing<\/a> the very communities most vulnerable to the hateful content the systems are designed to combat.<\/p>\n Gilbert, who is also the research director of Cornell\u2019s Citizens and Technology Lab, says that \u201cmarginalized and vulnerable populations are among those who experience the highest rates of moderation, and that typically this is a result of \u2018false-positives,\u2019\u201d often driven by instances of counter-speech<\/a>, language reclamation<\/a>, and \u201cresponses to hateful content.\u201d<\/p>\n \u201cFalse positives are an equity issue. They mean that groups that are already marginalized are further silenced and censored,\u201d she added.<\/p>\n AI moderators can also make communities less effective at moderating themselves. On Reddit, for example, some subreddit moderators would prefer to ban users who use hateful or violent rhetoric. But if Reddit\u2019s AI removes such content before a human moderator sees it, those moderators lose the ability to assess whether a ban is warranted.<\/p>\n In terms of giving human mods more control, Reddit this week announced<\/a> expanding testing for Rules Hub, a suite of tools that lets human mods \u201cchoose which rules should be automatically enforced, decide what happens when a rule is triggered (send to queue, filter, or remove), preview the experience before enabling it, and review logs and insights.\u201d Reddit expects Rules Hub to eventually replace the Automod <\/a>tool, which relies primarily on exact keywords.<\/p>\n AI is a tool, not the solution<\/p>\n