EMILIA Protocol, Inc.

USteam@emiliaprotocol.ai

sec AI agents qualification evaluation evidence runtime admission Agent evaluations report what happened in a test environment. They do not, by themselves, establish that a measured candidate satisfies a relying party's policy, remains current, matches the runtime candidate, or is authorized to perform a consequential action. This document defines a portable Qualification Statement that binds a candidate, complete evaluation campaign, qualification policy, assignment, and status. It preserves three separate claims: observation, qualification, and authorization. A relying party can accept a current qualification as evidence at runtime, but MUST make a separate exact-action authorization and admission decision.

Introduction Evaluation systems can execute tests, compare models, and fail a build. A consequential system needs an additional portable claim: this exact measured candidate satisfied this policy for this assignment, based on this complete campaign, and the claim remains current now. It also needs to avoid turning that claim into a universal score or transferable authority. This document defines the qualification layer. It does not define a benchmark runner, model leaderboard, certification business, universal intelligence score, or authorization token.

Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals.

Non-Collapsing Claims

Evaluation Attestation

Signed evidence of what was observed during a named campaign over a measured candidate.

Qualification Policy

The relying-party-selected predicates, thresholds, assignment, and validity rules that must be satisfied.

Qualification Statement

A signed claim that the candidate and complete campaign satisfy that policy for that assignment.

Authorization

A separate decision about whether an exact action may execute now under current authority and local policy.

A verifier MUST NOT reinterpret an Evaluation Attestation as a Qualification Statement. A relying party MUST NOT reinterpret a Qualification Statement as authorization. The labels VERIFIED, ACCEPTED, QUALIFIED, SATISFIED, AUTHORIZED, and ADMITTED describe different checks and MUST NOT be collapsed into one boolean.

Artifact Set

Candidate Manifest The Candidate Manifest identifies the exact evaluated configuration. It MUST bind immutable or content-addressed identifiers for code, model, prompt, tools, policy-relevant configuration, and evaluation environment. Mutable model aliases or unpinned remote resources make runtime identity INDETERMINATE unless a profile defines an authenticated resolution procedure.

Evaluation Campaign The Evaluation Campaign binds the Candidate Manifest, assignment, qualification-policy digest, evaluator identity, harness and suite versions, ordered test results, start and completion times, and a complete campaign-graph digest. Selectively omitting a failing result MUST change the graph digest and invalidate the later qualification.

Qualification Statement A Qualification Statement MUST bind exactly three subjects: the Candidate Manifest, the head Evaluation Campaign, and the complete qualification-graph digest. Its signed predicate MUST bind the qualification-policy digest, assignment digest, issuer, creation and expiry times, and a closed result of QUALIFIED or NOT_QUALIFIED. The signed statement MUST use a deterministic serialization or an envelope profile that defines exact signing bytes. An unknown critical member, missing subject, duplicate subject, or graph mismatch is refused.

Qualification Status Qualification status is a separately authenticated, monotonically sequenced chain. Each entry binds the statement, authority, sequence, predecessor digest, state, issuance time, and next-update time. A stale, rolled-back, equivocated, revoked, or missing required status yields NOT_QUALIFIED or INDETERMINATE according to the relying party's profile; it never yields QUALIFIED.

Qualification Verification The relying party MUST pin accepted issuers, evaluators, signature suites, candidate-measurement rules, assignment vocabulary, qualification policies, freshness limits, and status authorities. None of those trust inputs may come solely from the presented bundle. Verification proceeds in this order: envelope and signature; complete campaign graph; candidate identity; assignment and policy binding; predicate replay; statement time; current status; and runtime candidate measurement. The result is QUALIFIED only if every required check passes. A conclusive predicate failure is NOT_QUALIFIED. Missing, ambiguous, unsupported, stale, or unavailable required input is INDETERMINATE and fails closed. At runtime the verifier MUST establish EXACT_MATCH between the measured candidate and Candidate Manifest and IN_SCOPE between the protected request and assignment. A changed prompt, tool, code digest, model, or policy-relevant configuration requires a new qualification unless the qualification policy explicitly covered the exact change.

Runtime Composition A Qualification Statement MAY be carried as evaluation evidence in an Authorization Evidence Chain and evaluated at an Action Evidence Boundary . It can satisfy only the qualification role selected by the relying party. The boundary still MUST establish the exact CAID, verify authority evidence, make a local authorization decision, durably admit the action, and enforce one-time invocation or refusal. Qualification freshness is checked immediately before admission. Qualification supersession after admission affects future admissions; it does not rewrite a provider effect. An uncertain provider result remains an execution-reconciliation matter, not a qualification decision.

Security Considerations Implementations MUST defend against mutable aliases, selective-result omission, evaluator-key substitution, assignment widening, policy substitution, stale or rolled-back status, runtime candidate drift, cross-tenant replay, and treating benchmark success as authority. A qualification can be valid and still be irrelevant to the exact action. A compromised accepted evaluator can issue false observations; relying parties therefore need explicit evaluator roots, revocation, and diversity rules appropriate to risk. Scores are meaningful only within the named task, environment, candidate, harness, rubric, and campaign. Implementations MUST NOT derive a persistent universal reputation label from this protocol.

IANA Considerations This document requests no IANA action.

References Normative References Informative References EMILIA Protocol, Inc. EMILIA Protocol, Inc.

Implementation Status An Apache-2.0 TypeScript reference implementation is published in the EMILIA Protocol repository. It includes exported qualification verifiers, a Promptfoo evaluation-only adapter, Gate composition, in-memory and PostgreSQL admission stores, portable conformance vectors, hostile tests, and a bounded TLA+ lifecycle model. This is same-team implementation and model evidence. It is not independent interoperability, certification, a managed service, or proof of any external evaluation system.