Migrate GTB to go/config v0.3.x¶

Authors

Matt Cockayne, Claude Opus 4.8 (AI drafting assistant)

Date

20 July 2026

Status

IMPLEMENTED — 20 July 2026

Related

config module extraction (the v0.2.0 extraction this supersedes in practice), module extraction playbook (§ normative guidance that names config.Containable and must be updated), go/config specs: 2026-07-19-store-architecture, 2026-07-20-filesystem-abstraction

Summary¶

go/config v0.3.0 replaced the Viper-backed container with a Store architecture. GTB is on v0.2.0 and depends on the module entirely, so this is not an optional upgrade — it is the migration that unblocks every subsequent config change.

The target is v0.3.1, plus config-afero v0.1.0 for the filesystems GTB hands to config that are not the real one.

Five things changed, and they are not equally expensive:

Motivation¶

GTB cannot stay on v0.2.0. The module is published, the old API is gone, and every future config improvement lands on v0.3.x. But the migration is worth doing on its own merits:

Writes stop destroying user files. pkg/setup and pkg/cmd/config currently write by re-encoding the merged view, so a saved file loses every comment its author wrote and gains every value that arrived from somewhere else. Two helpers exist solely to work around this (D9). Provenance-routed Apply removes both the problem and the workarounds.

Reads become coherent. Configuration is read live today, so two related values can be read either side of a reload. Measured upstream: 1,759 mismatched pairs in 6.76M reads against 0 for snapshot-pinned reads.

The dependency graph shrinks. Viper leaves. GTB's own footprint guard currently permits the Viper stack; after this it forbids it (D12).

Blast radius¶

Measured on de7969b, not estimated.

Read/write split across production code:

GetString 72   IsSet 35   GetBool 23   GetInt 7   Sub 5
GetDuration 2  Has 2      Unmarshal 1  GetFloat 1        →  148 reads
Set 22         WriteConfigAs 2                            →   24 writes
GetViper 11                                               →   11 escape hatches

Roughly 80% is mechanical, because config.Reader is a strict superset of Containable for every read method GTB uses. The remaining 20% is where this spec spends its decisions.

Decisions¶

D1 — Props.Config becomes *config.Store, and reads take a View explicitly¶

This decision sizes every other one, so it comes first.

Containable was live: it wrapped a container that reloaded underneath its readers. Its natural replacement, config.Reader, is satisfied by *config.View — which is pinned to a snapshot and goes stale after a reload. A direct type swap would silently freeze configuration at startup.

Three options were weighed:

(a) Props.Config *config.Store, readers call .View(). Correct, coherent, and costs ~148 edits inserting .View().

(b) A live-forwarding adapter implementing Reader by delegating each method to store.View(). Zero edits — and it reintroduces exactly the tearing the Store exists to prevent, because each call resolves against a different snapshot. This is the option that looks cheapest and is worst: it would give GTB the behaviour upstream measured at 1,759 mismatched pairs and rejected.

© Props.Config config.Reader holding a View. Cheapest of all and simply wrong: the configuration would stop tracking the file.

Decision: (a). Props.Config is a *config.Store. ConfigProvider.GetConfig() returns *config.Store. Read sites become props.Config.View().GetString(...).

Where several values must agree with each other, use store.With(func(v *config.View) error) rather than taking several views — that is what it is for.

D2 — ConfigProvider keeps its shape; a second interface exposes reads¶

ConfigProvider.GetConfig() *config.Store stays the injection point, because writers need the Store and it is the thing that is live.

Consumers that only read should not take a Store. pkg/props gains:

// ConfigReader provides read access to resolved configuration.
type ConfigReader interface {
    Config() *config.View
}

so a component can declare the narrow dependency, and the four *_config_adapter.go files that only read take config.Reader directly rather than any GTB type. That is the local one-method-interface pattern the toolkit already uses, applied to the read surface.

D3 — Props.FS stays afero.Fs; config gets an adapted view of it¶

GTB carries 232 non-test afero references and internal/generator/dryrun.go installs an afero.CopyOnWriteFs as the application filesystem. Converting GTB to config.FS wholesale is a separate, larger change and is not in scope.

Props gains:

// ConfigFS adapts the application filesystem for the config module.
func (p *Props) ConfigFS() config.FS { return configafero.Wrap(p.FS) }

configafero.Wrap claims RealPather and LinkReader only where the wrapped filesystem supports them, which matters here: the dry-run CopyOnWriteFs can read links but has no path the operating system can watch, and claiming otherwise would send a watcher after files that are not there.

Production paths that pass the real filesystem may use config.OS() directly and skip the adapter.

D4 — The eleven Viper escape hatches, individually¶

Nine have direct replacements. Two do not.

The two that need design work:

pkg/cmd/root/root.go — GetViper().MergeConfig(strings.NewReader(cfg.ToJSON())). This deep-merges embedded defaults underneath user config. There is no equivalent because the Store does not need one: defaults are an ordinary layer. Resolved — see D13.

pkg/telemetry/datadir_config_adapter.go — ResolveDataDir(GetViper().ConfigFileUsed()). "The config file used" is not a single value in a layered store; it asks a question the model does not answer. Options, to be settled before implementation:

1. Resolve against the highest-precedence writable file layer — closest to intent, since the data directory sits beside the file a user edits.
2. Take it from store.Sources() filtered to SourceFile, last entry.
3. Give telemetry its own explicit data-directory setting and stop deriving it.

Option 3 is the honest one — deriving a data directory from a config file path was always incidental — but it is a behaviour change for existing installs. Recorded as open question 1.

D5 — ObserveSection sites need no change, and no ApplyInitial¶

Four sites bind typed sections: pkg/http, pkg/grpc, pkg/gateway, pkg/telemetry/observability_otel_config.go. None registers an apply callback; all consume by polling .Value()/.Current().

An earlier assessment held that these needed an ApplyInitial() decision because delivery is now change-only. That was wrong. v0.2.0's ObserveSection also reached its apply callback solely from inside the observer it registered — verified against the v0.2.0 source. Delivery has always been change-only, nothing is accommodating a change, and no ApplyInitial call is required to preserve behaviour.

What does change in those files: the first parameter becomes a config.Binder (*Store satisfies it), and the closures passed to WithSectionDefaultFunc take config.Observed where they took config.Containable.

D6 — Watching must be explicitly wired, and nothing today calls it¶

This is the only change in the migration with no compiler error.

v0.2.0 started an fsnotify watcher inside every constructor, so GTB got hot-reload without a call site. Nothing in the codebase calls a watch API today, because there was never anything to call. v0.3.x requires store.Watch(ctx), and if it is not added the code compiles, the tests pass, and configuration silently stops reloading.

pkg/cmd/root gains the call, with the returned stop tied to the command's context, and OnReloadError wired to the logger so a rejected reload is reported rather than swallowed.

Acceptance for this decision is a test, not a review: an integration test that changes a file on disk and asserts an observer fires. Without it, this decision is unenforceable.

D7 — The generator's four emission sites change in lockstep with setup.Initialiser¶

The generator emits config-typed signatures into every scaffolded project, so its output is the public contract of every downstream tool.

All four become config.Reader. stubs.go duplicates the stub independently for the main.go-preserved path and must change with the template, or a scaffolded project gets one signature from --force and a different one without it.

pkg/setup/init.go:44,46 declares the Initialiser interface the emitted code implements. It must change in the same commit, or generated code stops satisfying it.

Amended 2026-07-20, during implementation: Configure (and therefore its emitted counterpart) takes a new setup.Editor — View() *config.View plus Set(key, value) error — rather than config.Reader. The table above glossed that Configure writes: every initialiser calls Set, which Reader cannot express. IsConfigured and Validate<Name>Config take config.Reader as planned. The editor is backed by the store's Apply, so wizard writes edit the target document in place and template comments survive.

Good news the survey established: the generated pkg/cmd/root/root.go does not construct a container — it builds Props and delegates to GTB's NewCmdRoot. Downstream tools inherit the Store transparently, so these four signatures are the entire generated break surface.

D8 — ValidateStruct should be widened upstream rather than worked around here¶

The emitted Validate<Name>Config breaks in a way a rename does not fix:

func ValidateStruct[T any](cfg *View, opts ...SchemaOption) error   // v0.3.1

It takes a concrete *View, so the generated config.ValidateStruct[FooConfig](cfg) fails even after Containable → Reader, and forcing the generated signature to *config.View would make downstream validation unmockable.

It does not need the concrete type. validateView calls exactly four methods — Get, Has, Keys, Shadowed — and all four are on Reader.

Prototyped upstream and measured, rather than assumed. The change is three signatures:

func ValidateStruct[T any](cfg Reader, opts ...SchemaOption) error   // was *View
func validateView(view Reader, schema *Schema) *ValidationResult      // was *View
func configuredKeys(view Reader) []string                             // was *View

ValidateStruct calls validateView(cfg, schema) directly instead of cfg.Validate(schema). View.Validate is untouched and keeps working.

Results of the prototype:

• config's full suite passes unmodified — no behaviour change.
• A mocks.MockReader validates successfully, which is the point: downstream generated code stays mockable.
• Existing callers passing store.View() compile unchanged, because *View satisfies Reader. The widening is source-compatible.

Decision: take the upstream widening and depend on it. The alternative — emitting *config.View — exports a concrete type into every scaffolded tool to work around a parameter that was over-narrow, and makes every downstream Validate<Name>Config untestable without a real Store.

Release shape: widening a parameter is additive, so it is a feat: and lands as config v0.4.0 rather than a patch. That is the only cost, and it is sequencing rather than risk — recorded as open question 2.

D9 — Two workaround helpers are deleted, not ported¶

pkg/cmd/telemetry/telemetry.go ensureConfigFile and pkg/cmd/root/root.go ensureMinimalConfig both construct a fresh Viper, set one key and write it, explicitly to avoid dumping embedded defaults into a user's file.

That problem does not exist in v0.3.x: Apply edits the target document in place and writes only the key it was given. Both helpers are deleted rather than ported, and their call sites become a single Apply.

D10 — Documentation is swept in three tiers, not uniformly¶

595 lines across ~110 files. Treating them alike would either waste effort rewriting history or leave user-facing instructions wrong.

Tier 1 — correct in place (user-facing). These are copy-paste instructions that no longer compile: docs/how-to/test-configuration.md, config-hot-reload.md, validate-component-config.md, testing.md, embed-custom-assets.md, observe-typed-config.md, add-initialiser.md, plus the transport adapter guides (security-headers.md, add-grpc-service.md, expose-grpc-as-rest.md).

Two need rewriting rather than editing, because the change invalidates their narrative and not just their identifiers:

• config-hot-reload.md teaches implicit always-on watching as the container's defining behaviour. Explicit Store.Watch(ctx) inverts the premise.
• test-configuration.md builds its entire testing story on NewReaderContainer and GetViper(). The replacement story is config.Dir(t.TempDir()) or the published mocks.

Tier 2 — leave alone (historical). Migration notes, implementation notes and IMPLEMENTED specs are dated records of what was true then. Editing them destroys the record. Two exceptions:

• docs/reference/migration/v0.x-config-extracted.md gets a successor note, not an edit. Its thesis — "It is the toolkit's Viper layer: it carries the Viper stack so nothing else has to" — is now exactly inverted, and that is worth showing rather than hiding.
• docs/reference/migration/index.md gains a row.

Tier 3 — forward-looking, must be corrected. docs/development/specs/2026-07-12-go-module-extraction-playbook.md is the only APPROVED, normative spec, and it names config.Containable twice as guidance for future extractions. It will keep propagating a dead type until changed.

Also in scope, and easy to miss: the marketing surface. docs/about/comparison.md argues GTB's value proposition in terms of wrapping Viper ("GTB uses both Cobra and Viper internally"), docs/about/coming-from-other-ecosystems.md says the same, and README.md:57 says "Viper-powered configuration" behind a link that is already stale. These are positioning claims outside where a docs sweep naturally looks, and they will be flatly false.

Diagrams requiring redraw: docs/explanation/concepts/architecture.md has Containable ..> Viper : wraps — an edge to a dependency that no longer exists. interface-design.md reproduces the whole Containable interface verbatim including GetViper() and WriteConfigAs(). validate-component-config.md has an ASCII diagram naming "Viper merge hierarchy" as a box.

D13 — Precedence is already documented, and it translates directly¶

The order is not something to re-derive. It is stated in docs/development/security.md:15 and AGENTS.md:155, and the code at pkg/cmd/root/root.go:189 agrees with it — embedded configs are the base and the main config is merged over them, with the comment saying so explicitly.

Highest to lowest, combining both sources with the project-local layer described at root.go:112:

That maps onto the Store with no interpretation required, because precedence there is the order sources are added:

store, err := config.NewStore(ctx,
    config.WithReaders(config.NamedSource{
        Name: "embedded:defaults.yaml", Content: setup.DefaultConfig,
    }),                                        // 5 — lowest
    config.WithFiles(fsys, cfgPaths...),       // 4 — --config, or the defaults
    config.WithFiles(fsys, projectLocalPath),  // 3 — later wins over the above
    config.WithEnv(props.Tool.EnvPrefix),      // 2
    config.WithFlags(cmd.Flags()),             // 1 — highest
)

Amended 2026-07-20: the setup.DefaultConfig NamedSource shown at layer 5 is superseded — the embedded-defaults layer is the merged assets/config.yaml read through props.Assets, per the segregated default config spec. The order itself is unchanged.

This is a strict improvement, not just a translation. Today the order is an emergent property of where MergeConfig is called and in which direction; a reader has to trace three functions to work it out, and the doc is a separate assertion that could drift. After the migration the call site is the documentation.

One behavioural nuance worth stating rather than discovering. cfg.ToJSON() serialises the resolved main config, so environment-derived values are flattened into the merge at load time. In the Store, environment stays a live layer. Resolved values are identical — the environment does not change under a running process — but provenance improves: Explain can say a value came from env:MYTOOL_SERVER_PORT where today it would name the merged config.

D14 — --config replaces the default paths, is repeatable, and is not the last word¶

--config is load-bearing and undocumented outside development notes, so its exact semantics are recorded here and must survive the migration unchanged.

It is a StringArray with the default paths as its default value (pkg/cmd/root/root.go:803-808). pflag's stringArray replaces on first Set and appends after, which gives override semantics rather than additive ones. Verified against pflag v1.0.10:

no flag                            → [~/.config/<tool>/config.yaml  /etc/<tool>/config.yaml]
--config a.yaml                    → [a.yaml]                    # defaults discarded
--config a.yaml --config b.yaml    → [a.yaml b.yaml]             # repeatable, later wins

This maps onto WithFiles exactly. config's contract is "the first is the base and the last wins", which is the same rule config.Load applies today, so the whole list — however it was arrived at — passes straight through:

config.WithFiles(fsys, append(slices.Clone(cfgPaths), projectLocal)...)

A bug to fix, not behaviour to port. projectConfigPaths (root.go:115-129) appends the discovered project-local file unconditionally, including when the paths came from an explicit --config. So:

cd /repo-with-.mytool.yaml
mytool --config /my/exact/file.yaml     # .mytool.yaml silently overrides it

That contradicts the intent the flag was built for. --config is a StringArray whose default value is the standard paths precisely so that supplying it replaces them outright — the replace-not-append semantics are the design, not an accident of pflag. A flag designed to be the authoritative answer should not then be overridden by a file the user did not name and may not know is there. The failure mode is quiet and bad: a tool invoked with an explicit config runs against different settings, with nothing on the command line to explain why.

The doc comment on discoverProjectConfig shows the gap in reasoning — it says the layer is appended last so it overrides "the global ~/.<tool>/config.yaml". It contemplates the default path. Nobody weighed it against an explicit one.

The fix. Suppress the project-local layer when --config was explicitly set. pflag records this exactly, and it is the right test — it distinguishes "the user named files" from "the defaults happen to be in the slice", which inspecting the paths themselves cannot do:

func projectConfigPaths(props *p.Props, cmd *cobra.Command, base []string) []string {
    // An explicit --config is authoritative: it already replaced the default
    // paths, and a project-local file the user did not name must not override
    // files they did.
    if cmd.Flags().Changed("config") {
        return base
    }
    ...
}

There is exactly one call site (root.go:700), which already holds cmd, so this is a one-line signature change and a one-line guard.

Scope of the change. This is user-visible: a tool run with both --config and a project-local file in scope now uses only the named file. That is the behaviour the flag was intended to have, so it is a fix rather than a feature, but it wants a release note and a test pinning both directions — with --config, the project layer is absent; without it, the project layer still applies.

It does not depend on the config v0.3 swap and could have shipped separately, but it lands as part of this migration (resolved 2026-07-20). The precedence documentation has to be rewritten either way — D10's sweep and this fix touch the same table — and splitting them would mean documenting the old rule and then correcting it a release later. One behaviour change to the precedence order, described once.

The release note is therefore the migration's, and must call this out separately from the config swap: the swap is intended to be behaviour-preserving, and this is the one deliberate user-visible change riding with it. Burying it under "upgraded config" is how a real behaviour change gets missed.

Documentation gap. --config appears nowhere in the user-facing precedence material — not in docs/development/security.md, not in docs/reference/config/index.md, not in AGENTS.md. It is mentioned only in implementation notes and specs. The precedence table in D13 is therefore the first complete statement of the order, and the doc sweep (D10) must add it to the user-facing pages rather than merely correcting what is already there.

D11 — Mocks are regenerated, never hand-edited¶

12 generated mock files under mocks/ cover ConfigProvider, CoreProvider, LoggingConfigProvider and setup.Initialiser. All change signature. mockery regenerates them; the diff is reviewed but not authored.

D12 — The dependency footprint guard inverts¶

GTB's guard currently permits the Viper stack, on the reasoning recorded in the extraction spec that config legitimately carries it. After this migration Viper is absent from the graph entirely, and the guard should forbid github.com/spf13/viper so it cannot return unnoticed.

Verify by removing the entry and watching the guard fail, per the project's own testing guidance — a guard nobody has watched fail is a guard nobody knows works.

Testing strategy¶

The compile gate is internal/generator/compile_integration_test.go. It generates a skeleton, adds commands, injects a replace, and runs go build ./.... It is the only test that proves generated code compiles against the real module, and it will fail until D7 and D8 are complete. It is the acceptance criterion for the generator work.

~20 generator tests break on construction alone — config.NewFilesContainer does not exist in v0.3.x. These are mechanical and should be converted via a single shared test helper rather than 20 independent edits, so the next migration touches one place.

D6 needs a behavioural test. Everything else in this migration fails to compile if done wrong. Watching fails silently, so it needs an integration test that writes a file and asserts an observer fires.

Write-path behaviour changes and needs new assertions. The old code round-tripped the whole merged tree; the new code edits a targeted layer. Expect different bytes on disk, and assert the new guarantee — comments survive, only the named key changes — rather than adjusting old assertions until they pass.

Migration procedure¶

Phased so each phase is independently verifiable, and ordered so the largest mechanical pass happens after the shape is settled.

Phase 0 — upstream. Land ValidateStruct[T](cfg Reader, ...) (D8). Done — config v0.4.0. Widening a parameter is additive, so it released as a minor rather than the patch this originally guessed at.

One thing the D8 prototype missed, recorded because it is the kind of thing that recurs when a concrete parameter becomes an interface: validateView guarded on view == nil, which caught a nil *View while the parameter was concrete. Boxed in an interface that comparison is false — the interface carries a type — so the nil pointer reached Get and panicked, on exactly the input the guard existed for. Fixed with a reflective isNil, and confirmed to panic without it rather than assumed to be covered.

Phase 1 — pkg/props. Props.Config *config.Store, ConfigProvider, ConfigReader, ConfigFS(). Nothing else compiles until this is settled, and it dictates the diff everywhere else (D1, D2, D3). Regenerate mocks.

Phase 2 — construction. pkg/cmd/root bootstrap: one NewStore replacing the Load/LoadEmbed/NewReaderContainer chain, including the embedded-defaults precedence re-derivation (D4). Add Watch (D6) with its test.

Phase 3 — the mechanical pass. Containable → Reader, .View() insertion across ~148 read sites and the nine *_config_adapter.go files. Split by package (open question 5), so each commit is small enough to actually read.

Phase 4 — writes. pkg/setup/{init,ai,github,bitbucket,telemetry} and pkg/cmd/config move to Apply. Delete the two workaround helpers (D9).

Phase 5 — the remaining escape hatches. The nine direct replacements, then the two designed ones (D4).

Phase 6 — generator. Four emission sites, stubs.go, and setup.Initialiser in lockstep (D7). Green compile_integration_test.go is the gate.

Phase 7 — tests. The ~20 construction-only generator tests, plus the 72 test files.

Phase 8 — documentation sweep (D10), three tiers.

Phase 9 — guard and cleanup. Invert the footprint guard (D12), confirm Viper is absent from go.mod and go.sum.

Acceptance criteria¶

1. go build ./... and go test -race ./... pass.
2. github.com/spf13/viper appears in neither go.mod nor go.sum, and the footprint guard forbids it — verified by removing the entry and watching it fail.
3. compile_integration_test.go passes, proving generated projects compile.
4. A test asserts that a file changed on disk reaches an observer (D6).
5. A test asserts a write preserves comments and changes only the named key.
6. No documentation in Tier 1 contains a config API symbol that no longer exists.
7. mockery output is current and unedited by hand.

Open questions¶

1. How should ResolveDataDirFromProps resolve a data directory? Resolved 2026-07-20 by the parity principle. Viper's ConfigFileUsed() returns the last file it loaded — stated in docs/reference/migration/v0.21-config-files-accessor.md:26, which is why ConfigFiles() was added in the first place. Parity is therefore the last file source: store.Sources() filtered to SourceFile, last entry. An explicit telemetry data-directory setting remains the better long-term answer and is out of scope here.
2. Do we take the upstream ValidateStruct widening? Resolved 2026-07-20: yes, released as config v0.4.0. Additive rather than breaking, so it does not disturb the pre-release bedding-in period. See Phase 0 for the nil-guard hazard the prototype missed.
3. Does Props.FS eventually become config.FS? Resolved 2026-07-20: no. Props.FS stays a full afero instance — OsFs live, MemMapFs in tests — because that is how GTB operates today and parity is the goal. config gets an adapted view of it through configafero.Wrap, per D3. Nothing else changes.
4. What is the precedence for embedded defaults after the merge is replaced? Resolved 2026-07-20: the documented order, which the code already matches. See D13.
5. Is a single mechanical commit for Phase 3 reviewable at ~148 sites? Resolved 2026-07-20: split by package. A diff that large gets rubber-stamped, and reviewers skim mechanical changes precisely when the one non-mechanical site is hiding among them. The cost — a longer-lived broken build — is confined to a feature branch nobody else builds from. (Question 6 — whether an explicit --config should suppress the project-local layer — resolved 2026-07-20: yes. It was always intended to be authoritative over both the defaults and the project-local file, which is why it was built as a replacing flag. Recorded in D14.)

Status¶

IMPLEMENTED — 20 July 2026, in the same MR that approved it. All six open questions were resolved the same day (question 2 as config v0.4.0; question 5 as split-by-package; question 6 as D14). Every acceptance criterion holds: build/test/race green module-wide, viper absent from go.mod and go.sum with a guard verified by failure, the generated-project compile gate green, the D6 watch integration test green, comment-preserving targeted writes asserted, tier-1 docs swept, and mocks regenerated. One amendment recorded during implementation: emitted Configure takes setup.Editor (D7 note), since a Reader cannot write.