Design Harmful Content Detection for LLMs

System Design Question Preview

You are designing an ML-powered moderation system for a product that hosts or generates user-facing content. The content may include LLM prompts, LLM responses, chat messages, comments, uploaded documents, images, or videos. The system should detect harmful content and decide what action to take before or after the content reaches users.