跳至主要内容

Content moderation helps ensure content security, user experience, and compliance with legal requirements in AI applications.

Enabling Content Moderation​

  • In chatbot applications, go to the orchestration page and select ADD FEATURE > TOOLBOX > Content moderation. Enable Content Moderation Content Moderation Toolbox

Using the OpenAI Moderation API​

  • OpenAI and other LLM providers offer moderation models to block content related to violence, sex, or illegal activities.
  • You can use the OpenAI Moderation API in AgentBuilder to review input or output and provide preset replies. OpenAI Moderation API

Custom Keywords​

  • Developers can define sensitive keywords (e.g., "kill") to trigger moderation and return a default response like "The content is violating usage policies." Custom Keyword Moderation

Moderation Extension​

  • Companies can implement custom sensitive word filtering by writing an API extension. Moderation Extension
  • Example: Block queries about the US president's name and return a policy violation message.

On this page ...

Is this helpful?