Harmless response rate

In a maker's safety tests, the share of clearly harmful requests where the model's reply did not help cause the harm.

Why it matters. Higher is safer, but compare only within one maker's methodology.

How it works

Harmless response rate is the share of clearly harmful test requests where the model's reply did not give meaningful help toward the harm. Makers report it in their safety evaluations, usually as a percentage.

Each maker builds its own test sets, categories and grading, so 98% from one maker cannot be compared with 98% from another. Reports may show the rate with and without extra safeguards, or with reasoning on and off.

Read it together with the over-refusal rate: a model can score high on harmlessness simply by refusing too much. For your product, test the kinds of misuse you actually expect.

Worked example

Illustrative: a 99% harmless rate on 2,000 harmful test prompts means about 20 replies gave meaningful help. If a public chatbot gets 5,000 clearly harmful requests a month, a similar rate would mean about 50 harmful replies, which may call for your own moderation layer.

Common mistakes

Comparing harmless rates across makers with different test sets and graders.
Looking at harmlessness without the matching over-refusal rate.
Assuming the maker's test categories cover the misuse your product attracts.

See it on a real card: Safety & risk on Claude Opus 5.5.