Scores & exit codes

What each number means, which direction is good, and what the CLI returns to your shell.

Two directions

SecureAI reports both kinds of metric, and mixing them up inverts the conclusion. Scores out of 100 are higher-is-safer. Rates are higher-is-worse.

Metric	Range	Direction	From
Overall score	0-100	Higher is safer	Any multi-phase assessment
BUILD score	0-100	Higher is safer	`secureai scan`
SDS	0-100	Higher is safer	`secureai validate`
ASR	0-100%	Higher is worse	`secureai test`
Anomaly score	0-1	Higher is worse	`secureai runtime lbf`

Colour banding follows the meaning, not the number: a red 40 score and a red 40% ASR are both bad, despite pointing in opposite directions numerically.

Score bands

Scores out of 100. Higher is safer

Band	Reading
90-100	Strong. Controls are present throughout.
70-89	Acceptable. Gaps exist but none structural.
50-69	At risk. Material controls missing. Not production-ready.
0-49	Unsafe. Do not ship.

70 is the line validate draws by default, which is why --fail-below defaults to it.

Attack Success Rate. Higher is worse

ASR	Reading
0%	No payload in the corpus succeeded.
Under 2%	Strong. A typical release bar.
2-5%	Acceptable for many contexts. The default gate sits at 5%.
5-15%	At risk. Injection is working reliably enough to depend on.
Over 15%	Severe. The system is effectively steerable by users.

Judge ASR per category too

An overall 4% that is entirely one category is a specific, fixable weakness. The same 4% spread evenly across every category usually means the prompt lacks structural defences rather than having one bug.

Anomaly score. Higher is worse

Score	Reading
0.0-0.3	Consistent with the baseline.
0.3-0.6	Drifting. Review the pattern across sessions.
0.6-0.75	Elevated. Below the default threshold but increasing.
Over 0.75	Alert. CBE degrades capability at the default threshold.

Severity levels

Severity	Meaning
Critical	Directly exploitable with serious consequence. Fix before shipping.
High	Exploitable, or a missing control with real exposure. Fix soon.
Medium	A genuine weakness that needs another factor to be exploited.
Low	Hardening. Widens the surface without being a breach on its own.
Informational	Observation. No action required.

Exit codes

Code	Meaning
`0`	Success. The run completed and the gate passed, or no gate was set.
`1`	Gate failed. Findings, ASR, or score crossed the configured threshold. A real result.
`2`	CLI error. Bad flags, unreadable file, unreachable server, expired credentials. Not a security result.

Do not collapse 1 and 2

Treating them the same hides broken automation behind “security failed the build”. A pipeline reporting 2 every night is telling you the token expired, not that the model is unsafe.

The gates

Command	Flag	Default	Exits 1 when
`scan`	`--fail-on`	none (or `fail_on` in config)	A finding is at or above the given severity.
`test`	`--fail-asr`	`5`	Overall ASR exceeds the given percentage.
`validate`	`--fail-below`	`70`	SDS falls below the given score.

Checking the code

Terminal

secureai scan ./prompts --fail-on high
echo "exit: $?"

# Distinguish a gate failure from a broken run
secureai scan ./prompts --fail-on high
case $? in
  0) echo "clean" ;;
  1) echo "findings exceeded the gate" ;;
  2) echo "the scan itself failed" ;;
esac

Score and gate are separate

The score summarises posture across everything found.
The gate is a yes/no decision about one threshold.

A scan can score 88 and still exit 1, because a single Critical finding trips --fail-on critical regardless of how good the average is. One exposed credential is not offset by ninety clean files.

Related: CI integration · Scan rules · Configuration