Testing & Validation

A correct post-quantum implementation must prove it matches the standard bit-for-bit and, for regulated environments, earn a FIPS 140-3 certificate. This page covers Known Answer Tests, NIST's ACVP/CAVP programs, and what you actually need to test.

Two kinds of assurance

There are two distinct questions to answer about any PQC deployment. First, does the implementation compute the standardized algorithm correctly? — answered by test vectors and the ACVP/CAVP validation programs. Second, does it resist side-channel leakage? — a separate concern covered under side channels, because passing functional tests says nothing about timing or power leaks.

Known Answer Tests (KATs) and test vectors

For validating FIPS 203/204/205 implementations, use the final-standard test vectors published or generated by NIST CAVP/ACVP; do not mix in intermediate values from the draft-FIPS phase or third-party copies. These Known Answer Tests pair fixed inputs (seeds, messages) with the exact expected outputs (keys, ciphertexts, shared secrets, signatures) for every parameter set, and a conforming implementation must reproduce them byte-for-byte.

• Deterministic per parameter set. KATs are provided for each parameter set (ML-KEM-512/768/1024, ML-DSA-44/65/87, the SLH-DSA variants), seeded so output is reproducible.
• First gate in CI. Run KATs on every build and platform; they catch endianness bugs, NTT errors, and incorrect domain separation immediately.
• Source of truth. Use the official NIST test vectors and the reference implementations that accompany the standards as the authority, not a third-party copy.

ACVP and CAVP

For FIPS 140-3 validation, NIST runs two programs that work together:

• ACVP — Automated Cryptographic Validation Protocol. A protocol and server that issue test vectors to your module and check its responses automatically. It supersedes the old static-file flow with a request/response exchange covering many edge cases.
• CAVP — Cryptographic Algorithm Validation Program. The program under which an algorithm implementation is validated (via ACVP) and issued an algorithm certificate. CAVP validation is a prerequisite for the module-level CMVP / FIPS 140-3 certificate.

The chain is: pass ACVP tests for the algorithm → earn a CAVP certificate → include it in a FIPS 140-3 module validation (CMVP). Only the module certificate, scoped to specific algorithms and parameter sets, lets you claim compliance.

CAVP/ACVP validates the algorithm implementation; CMVP/FIPS 140-3 validates the cryptographic module that contains the algorithm implementation. An algorithm certificate is an essential component of module validation, but algorithm validation alone cannot substitute for a module certificate.

The FIPS algorithm boundary

Validation applies to a defined cryptographic boundary — a specific module version, build, and set of algorithms. ML-KEM, ML-DSA, and SLH-DSA each have their own validation scope. A library "supporting" an algorithm is not the same as that algorithm being inside a validated boundary, so confirm the certificate covers the exact parameter set you ship.

What to actually test

1. KAT conformance for every parameter set, on every target platform and build configuration.
2. Signing determinism modes. ML-DSA supports both a deterministic (hedged-off) and a randomized signing mode — test both, and confirm deterministic mode reproduces the KAT signatures exactly.
3. KEM decapsulation failure handling. Feed malformed and tampered ciphertexts and verify the implementation follows FIPS 203 implicit-rejection logic: the bad-ciphertext path still produces a pseudorandom shared secret, and must not reveal a "decapsulation failure" through a return code, log, timing, or error branch (the failure-oracle risk from side channels).
4. Round-trip property tests. Random keygen → encaps/sign → decaps/verify across many iterations to catch low-probability decapsulation-failure handling.
5. Interoperability. Cross-test against a second independent implementation (e.g. your library vs OpenSSL) to confirm wire-format agreement.
6. Negative tests. Wrong-key, truncated-input, and bit-flipped-signature cases must all be rejected.

Related

Standards & references

• NIST CAVP/ACVP — the algorithm validation program and protocol.
• FIPS 203 (ML-KEM) — standard and Known Answer Tests.
• FIPS 204 (ML-DSA) — standard and Known Answer Tests.
• FIPS 205 (SLH-DSA) — standard and Known Answer Tests.
• NIST PQC project — official test vectors and reference implementations.
• Resources — full standards register