OpenAI Moderation API

Free classifier for harmful text and images — standalone or inline with generation. Treat scores as policy signals for user-facing apps, not automatic block decisions.

OpenAI's moderation models detect potentially harmful content across text and images. Use them to enforce application policy: filter UGC, route flagged content to human review, or log audit trails before showing AI-generated output to users. Source: OpenAI, https://developers.openai.com/api/docs/guides/moderation, 2026-06-12

When to consider it

For user-facing apps that accept user input, show model-generated text, or render uploaded images — especially chat, social, creative tools, and any surface where harmful content creates trust, legal, or abuse risk. Treat it as one implementation option when the App Legal Launch Checklist triggers the UGC/moderation gate. This is an output/input guardrail layer in Agent Guardrails Architecture, complementary to (not a replacement for) custom The Eval Loop (Slop Is an Output Problem) rubrics for product-specific quality.

Not a substitute for: prompt-injection defense, PII redaction, tool/action allowlists, or domain-specific policy (those stay separate checks).

Model and cost

Model: omni-moderation-latest — text + image inputs; no audio.
Cost: moderation endpoint is free; image files up to 20 MB.
Caveat: OpenAI continuously upgrades the underlying model; policies that depend on fixed category_scores thresholds may need recalibration over time. Source: OpenAI, https://developers.openai.com/api/docs/guides/moderation, 2026-06-12

Workflows

Workflow	Use when
Standalone classify	Moderate user-submitted text or images without generating a model response (`POST /moderations`).
Inline with generation	Same request returns both generated output and moderation scores (`moderation` object on Responses API).
Post-generation only	Classify model output after the fact if inline moderation is unavailable in your stack.

Standalone moderation

Classify text or images without calling a chat model:

response = client.moderations.create(
    model="omni-moderation-latest",
    input="...text or image input...",
)
result = response.results[0]

Inline moderation (Responses API)

When generation and moderation scores should arrive together, pass a top-level moderation object:

response = client.responses.create(
    model="gpt-5.5",
    input=[{"role": "user", "content": "..."}],
    moderation={"model": "omni-moderation-latest"},
)
input_mod = response.moderation.input
output_mod = response.moderation.output

Input scores at response.moderation.input; output at response.moderation.output.
Same category fields as standalone results.
Streaming: moderation scores arrive after the full output is available, not with partial deltas.
Tool calls: moderation covers tool-call arguments and tool outputs in conversation content — not tool names, descriptions, schemas, or response-format schemas.
Errors: if moderation fails, the field may contain an error object instead of scores — check type before reading flags. Source: OpenAI, https://developers.openai.com/api/docs/guides/moderation, 2026-06-12

Interpreting results

Field	Meaning
`flagged`	`true` if the model classifies content as potentially harmful — good first-pass signal.
`categories`	Per-category boolean violation flags.
`category_scores`	Confidence 0–1 per category; use for logging, routing tiers, and audit — not as hardcoded universal thresholds without tuning.
`category_applied_input_types`	Which input types (`text`, `image`) each category applies to.

Policy, not physics: a safety-aware refusal can still flag if it discusses harmful content. Use scores to implement your policy (block, soft-block, queue for review, rate-limit account) rather than treating flagged as always block.

Supported categories

Category	Applies to	Notes
`harassment`, `harassment/threatening`	Text
`hate`, `hate/threatening`	Text	Non-protected groups (e.g. hobby communities) fall under harassment, not hate.
`illicit`, `illicit/violent`	Text	Instructional illegal activity; violent variant adds weapons/violence refs.
`self-harm`, `self-harm/intent`, `self-harm/instructions`	Text + images
`sexual`, `sexual/minors`	Text + images / text only	Minors category is text-only.
`violence`, `violence/graphic`	Text + images

Image-only requests return score 0 for text-only categories. Source: OpenAI, https://developers.openai.com/api/docs/guides/moderation, 2026-06-12

Integration patterns (user-facing apps)

Ingress — moderate user messages and uploads before they enter the model or appear to other users (input guardrail).
Egress — moderate model output before render; pair with The Eval Loop (Slop Is an Output Problem) for product-quality gates (tone, format, grounding).
Tiered response — flagged → block or replace with generic message; high category_scores without flag → log + sample review; low scores → pass.
Account signals — repeated flags on one account → throttle, shadow-ban, or escalate — moderation as abuse signal, not only content filter.

See Agent Guardrails Architecture for placement in the input/output/action stack and latency ordering (cheap classifier before LLM judge).

Timeline

2026-07-06 | Linked moderation into App Legal Launch Checklist as one implementation option for apps that trigger the UGC liability/moderation launch gate. Source: User request, 2026-07-06
2026-06-12 | Page created from OpenAI moderation guide capture. Documented omni-moderation-latest, standalone vs inline workflows, category matrix, streaming/tool-call limits, free pricing, and user-facing integration patterns as a consideration alongside custom eval guardrails. Source: User capture + OpenAI, https://developers.openai.com/api/docs/guides/moderation, 2026-06-12

Source: User request, 2026-07-06