Test evidence report — QMS Nordic platform
| Document ID | QMSN-TER-001 |
| Applies to | QMS Nordic v1.5.0 |
| Part of | The software validation pack |
| Produced | On every change, automatically. This revision reflects the suite as it stood at v1.5.0. |
This report states what our automated tests actually establish, and — more usefully to you — what they do not. It exists because a traceability matrix that names a test tells you a test exists; it does not tell you what the test asserts, whether it runs against a real database, or whether it would notice if the behaviour changed.
This is our evidence, not yours. It reduces what you need to execute under template T4; it does not replace it. An auditor assesses your testing, and no vendor test can cover your configuration, your data, or your process.
1. What runs, and when
The suite runs on every change and blocks the build. A change that breaks a test cannot be released, which is what makes the rest of this document worth reading.
At v1.5.0 the suite is 15 files, 306 tests: 262 executed and 44 skipped.
The 44 skipped are not failures and not hidden. 43 of them require a live database connection and are skipped when one is not configured — so they run in continuous integration against a database rebuilt from the migration history, and are skipped on a developer machine without credentials. Two files are affected in full: esignature-audit.integration (22) and document-lifecycle.integration (21). When you ask for a test run as evidence, ask for one in which those two executed — they produce the isolation, signature and audit-chain evidence, and a run showing 44 skipped produced none of it.
The 44th is a single case in ai-eval held back separately.
2. What each suite establishes
| Suite | Tests | What it establishes |
|---|---|---|
validation-pack | 91 | This pack is internally consistent: every requirement identifier resolves, the URS and matrix carry exactly the registered set, no document cites an ID we do not ship, version stamps agree with package.json, and no repository path leaks into a customer document. |
render-md | 30 | The renderer that produces every page of this pack from its source markdown does not corrupt it — a defect here would silently change what a document says. |
lifecycle-gates | 25 | Document state transitions refuse what they should refuse. This is the suite behind URS-DOC-03. |
validation-pack-docx | 28 | The .docx you download matches the document you read. |
esignature-audit (integration) | 22 | Signature and audit-trail integrity against a real database, including the hash chain and the append-only triggers. |
ai-eval | 22 | AI drafting produces documents of the expected shape, and refuses rather than inventing when it lacks grounding. |
document-lifecycle (integration) | 21 | Lifecycle, access control and retention against a real database. |
helpdesk-email | 19 | Inbound email is attributed to the right organisation and cannot cross between them. |
token-weights | 14 | AI usage is metered at the published rate, including the commercial contract that spoken audio earns the same markup as text. |
ai-draft | 13 | The drafting pipeline records what it generated, continues rather than truncating, and reports provider failures as failures instead of producing an empty document. |
classifier | 7 | Device classification produces the expected class for known inputs. |
rules-engine | 5 | The rule pack derives the obligations it should from a classification. |
docx-roundtrip | 3 | The Edit-in-Word handoff preserves content and custom properties. |
security-headers | 3 | Transport and content-security headers are present. |
tenant-isolation | 3 | The application-layer tenant guard holds: a query from one organisation cannot read another's records. |
checklist-templates-rls (integration) | 6 | Shared audit-checklist templates are readable by every organisation and writable by none, as the application role against a real database. |
periodic-review + (integration) | 13 | Document review reminders: the 30-day window, the weekly dedupe, urgency once overdue, and the daily job end to end (URS-DOC-05). |
reportability | 9 | Vigilance clocks run from the awareness date: Article 87's 2, 10 and 15 days; Part 803's 5 and 30. |
capa-gates (integration) | 5 | CAPA cannot skip its actions, verify early, or close without a verified effectiveness check (§8.5.2(f)). Module evidence M-CAPA. |
complaint-gates (integration) | 5 | A reportable complaint cannot close with its report outstanding; a report cannot be recorded on an unflagged one. M-CMP. |
management-review-gates (integration) | 4 | Planned → conducted → closed in order; no closure without §5.6.3 outputs; frozen once closed. M-MR. |
training-gates (integration) | 4 | Completed by the trainee; verified by a different person, by identity. M-TRN. |
supplier-gates (integration) | 6 | Unique code; five qualification statuses; approval dated; an auditor cannot change it. M-SUP. |
equipment-gates + calibration-gates (integration) | 7 | Unique asset code; status changes with reason; certificates by the right roles with due after performed. M-EQP. |
software-gates (integration) | 4 | A software problem cannot close before its resolution is verified, nor by the engineer who resolved it. M-SW. |
submission-gates (integration) | 4 | Evidence pins to a section; empty external evidence is refused; removal is on the trail. M-SUB. |
3. What this evidence does not establish
Stated plainly, because the gap between "tested" and "validated" is where supplier assessments go wrong.
| It is not a signed results report | There is no per-release document with an approver's signature attesting to a specific run. This report describes the suite; it does not attest to one execution of it. If your procedure requires signed test evidence, that signature is yours to apply after you execute T4. |
| Passing tests are not proof of correctness | A test proves the behaviour it asserts. Several faults reached production during 2026-08 that this suite did not catch — see the release record — because no test asserted the thing that broke. |
| Coverage is not measured | We do not publish a coverage figure, and would not want you to rely on one. Coverage measures which lines ran, not whether the right things were asserted. |
| Nothing here covers your configuration | Your organisation's roles, document types, retention period and rule pack are yours. No test of ours exercises them. |
| AI output is not deterministic | The AI suites assert shape, grounding and refusal behaviour. They cannot assert that a generated document is correct, and you should not treat AI output as validated content. |
4. How to use this under T4
- Read §2 and identify which of your T4 test cases we already exercise.
- For each, decide whether our assertion covers your intended use. Where it does, you may reduce scope and cite this report as the reason.
- For anything in §3, or anything specific to your configuration, execute it yourself. That is the bulk of T4 and is not reducible.
- Record the version you relied on — v1.5.0 — in your T5. Evidence has to stay retrievable at the version you validated against.
5. Asking us for a run
You may ask for the output of a test run at any release, and we will provide it. Ask for one in which the two integration suites executed rather than skipped; a run showing 44 skipped is a run without a database, and the isolation and signature evidence is precisely what those two produce.
QMSN-TER-001 · v1.5.0 · pack rev. J (2026-09-07)
Published at qmsnordic.com/legal/validation/test-evidence. Cite the version above in your validation record — a pack is evidence, and evidence has to stay retrievable at the version you validated against.