Content moderation AI copilot

Created on 27 July 2023, over 1 year ago
Updated 10 February 2024, 10 months ago

Problem/Motivation

There is a need to automate some part of moderation - detect content violating rules (hate, sexual...) and take action (unpublish...).
Applies to text, images and videos.
Useful for sites with user generated content like Open Social (which also already has moderation workflow).

More from Anand on slack:

  • "To save costs, the AI content moderator could skip those users posts which have previously posted more than X amount good content (ECA could make this rule)"
  • "From new users, AI moderator could check 50% of the post or something. There could be plain filter first, which would unpublish the post first, but if that would not find anything then only AI content moderator could check the post."
  • "Also if users are reportin about certain post, that it is not appropriate, then AI could check and make its decicion to unpublish it or change it state for humans to check."
  • "For images and videos, I dont know how chatgpt can do that, but Google has something made for image/video detection for example finding nude etc pictures."

Proposed resolution

Remaining tasks

  • Share issue with Augmentor, Social, ECA and other projects to come up with a common solution.

User interface changes

API changes

Data model changes

Feature request
Status

Active

Component

General

Created by

🇱🇹Lithuania mindaugasd

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Production build 0.71.5 2024