Coverage Gap Closure Specification¶

Authors

Matt Cockayne, Claude Opus 4.8 (AI drafting assistant)

Date

20 June 2026

Status

IMPLEMENTED

Summary¶

The 2026-06-17 test-coverage closure plan brought its named packages to the pkg/ ≥90% policy (22 packages, plus the generator ast_extract walkers). A full-repo go test ./... -cover audit on 2026-06-20 shows 44 of 97 tested packages at ≥90% — but that raw denominator is misleading: it counts generated mocks, cmd/* entry points, and pure test-helper packages that carry no shippable logic and should never be measured.

This spec classifies every package currently below 90% into three buckets — exclude-by-design, covered-by-gated-tests, and addressable-hermetic — and lays out a prioritised plan to close the addressable gaps. The point is to make the 90% policy mean something repo-wide: a package is either ≥90% on unit coverage, or it appears on an explicit, justified exclusion list — nothing falls through the cracks silently.

This spec covers unit coverage (the default just test). Several packages have additional gated integration or E2E (Godog) coverage not reflected in the unit number; that is called out per package. The credential/desktop-gated integration work is specified separately in 2026-06-20-desktop-gated-integration-tests.md.

Goals & Non-Goals¶

Goals¶

• A definitive, categorised inventory of sub-90% packages as of 2026-06-20.
• An explicit exclusion list with per-package rationale, so the policy is enforceable: "≥90% OR on the list."
• A prioritised, effort-estimated plan for the genuinely addressable gaps.
• A definition of the countable package set the policy applies to.

Non-Goals¶

• No production code changes are expected (test-only), barring a minimal, additive, precedented test seam where unavoidable (note it in the MR).
• Not chasing 90% on TUI/Bubble-Tea, generated, entry-point, or real-dependency code — those are excluded with rationale, not ignored.
• Not the desktop-gated integration work (keychain/WKD/live-VCS/live-chat) — that is the separate spec linked above.

The countable package set¶

A package counts toward the policy unless it is one of:

• mocks/** — generated by mockery (// Code generated … DO NOT EDIT).
• cmd/** — binary entry points (main): cmd/gtb, cmd/e2e, cmd/gtb-no-aws-smoke, and the thin cmd/changelog / cmd/docs shims.
• Test-only helpers: internal/exectest, pkg/credentials/credtest, test/e2e/support, internal/version (build-stamp only).

These are never measured. Everything else is either ≥90% or on the exclusion list in this spec.

Bucket A — Exclude by design (with rationale)¶

These ship logic but are intentionally not unit-covered to 90%; they are covered by E2E/integration or are untestable without a real external dependency. They are the standing exclusion list.

Rule going forward: adding a package to Bucket A requires a one-line rationale here. The apidiff/coverage reviewer treats anything sub-90 not on this list as a gap to fix.

Bucket B — Generator CLI, covered by @generator BDD E2E (not unit)¶

The internal/cmd/* scaffolding commands are thin wiring over internal/generator; their behaviour is driven end-to-end by the @generator Godog suite (features/generator/*) and the generator_build integration tests, not by unit tests. Their low unit numbers are expected.

Decision (Open Question 1 → A, resolved): these are accepted as E2E-covered and join the exclusion list for the unit policy, on the condition that the @generator suite + generator_build integration remain green. A thin unit pass over the pure flag/arg-parsing and error branches of internal/cmd/generate and internal/cmd/enable is a no-cost bonus only if it falls out naturally (Phase 3).

Bucket C — Addressable hermetic gaps (the real work)¶

Genuinely unit-testable packages below 90%, ordered by gap size. These are the targets.

Status (implementation, 2026-06-20)¶

Phases 1–3 below are complete — every Bucket C package is ≥90% except internal/generator, which reached its OQ3→A pure-helper ceiling:

• Phase 1 (telemetry): telemetry 90.6, setup/telemetry 100, telemetry/logs 100, telemetry/tracing 100, telemetry/metrics 100.
• Phase 2 (command surface): cmd/version 100, cmd/doctor 98.7, openapi 92.3, setup 90.1.
• Phase 3 (larger lifts): gateway 100, utils 92.9, internal/agent 92.2; internal/generator 79.2 (pure-helper ceiling — emission/TTY/AI/network excluded, generator_build-covered; the white-box tests required 7 narrow inline //nolint:gosec // G602 false positive directives on guarded slice indexing — comments only, no logic change).

Phase 4 (complete): the policy is now codified and self-enforcing — .coverage-policy.yaml (threshold + not-counted prefixes + the Bucket A/B and internal/generator exclusion list, each with a rationale), scripts/coverage-policy.sh (flags any countable sub-90 package not excluded), the advisory coverage-policy CI job (allow_failure: true, mirroring apidiff), and a just coverage-policy runner. The check passes clean against main. This spec is IMPLEMENTED.

Testing strategy¶

• Hermetic only: httptest, afero.NewMemMapFs(), logger.NewNoop() / NewBuffer(), in-process fakes; no network/keychain/TTY/wall-clock.
• CI determinism is mandatory — every new test must pass under CI=true (one Phase-10 slice tripped a CI=true-only storage-mode default; lesson applied). Set needed env via t.Setenv (serial, no t.Parallel).
• Table-driven + t.Parallel() where safe; per package: go test ./pkg/X -cover ≥90%, golangci-lint run ./pkg/X clean, go test -race ./pkg/X ok.
• Parallelise across packages with isolated subagents (the Phase-10 pattern), integrating and re-verifying each slice on the main worktree.

Implementation phases (prioritised)¶

1. Telemetry cluster — pkg/telemetry (+ logs/tracing/metrics) and pkg/setup/telemetry. Self-contained, fakeable, four near-misses among them.
2. Command-surface near-misses — pkg/cmd/version, pkg/cmd/doctor, pkg/openapi, pkg/setup (already 89.6%). Cheap output/error-branch passes.
3. Larger lifts — pkg/gateway (38-pt gap), pkg/utils, internal/agent, and the non-AST internal/generator helpers. Optionally the thin internal/cmd/generate/enable unit pass (Bucket B nice-to-have).
4. Codify the policy — publish the Bucket A/B exclusion list and the countable-set definition as a machine-readable manifest (e.g. .coverage-policy.yaml) plus a contributor-guide reference, and add the non-blocking CI advisory job (OQ2 → A) that flags any sub-90 package not on the list, mirroring apidiff's allow_failure: true pattern.

Each phase is one or more MRs, sized like the Phase-10 slices.

Migration & Compatibility¶

Test-only; no API or behaviour change. Phase 4 makes the policy enforceable by publishing the exclusion list — a documentation/process change, not a code one.

Open Questions¶

All resolved at review (2026-06-20); recorded here for provenance.

1. Bucket B generator CLI: accept as E2E-covered, or add a unit pass? — Resolved → A: accept as E2E-covered for the unit policy (they are thin wiring; the security-relevant validation/escaping lives in internal/generator, which Bucket C covers), conditional on the @generator + generator_build suites staying green. A cheap unit pass over generate/enable flag-parsing/error branches is a no-cost bonus only if it falls out naturally.
2. CI advisory for sub-90 packages not on the exclusion list? — Resolved → A: add a non-blocking advisory job (Phase 4), mirroring the apidiff advisory (allow_failure: true) so it never gates merges or flakes the pipeline. Requires a machine-readable exclusion list shared by the job and this spec.
3. internal/generator: unit-test the pure helpers or also emission? — Resolved → A: unit-test the deterministic manifest/merge/validation helpers; leave file-emission to generator_build (which compiles the scaffold — a stronger check than asserting emitted strings).