Configuration¶

The configuration layer has been extracted into the standalone gitlab.com/phpboyscout/go/config module. Its full documentation, the Store/View model, layered sources and precedence, typed sections (UnmarshalSection / ObserveSection), schema validation, explicit hot-reload (Watch), transactional writes (Apply), flag binding, and testing with the published mocks package. Now lives at:

config.go.phpboyscout.uk

API reference: pkg.go.dev/gitlab.com/phpboyscout/go/config. See the migration note for the import-path change.

go-tool-base imports the module directly (no adapter package). This page documents only what GTB layers on top. The conventions and wiring that are framework concerns and deliberately not in the module.

How GTB wires the container¶

The root command builds the store during its pre-run and publishes it as props.Config, so every command and initialiser receives the same instance:

p := &props.Props{
    Config: cfg,   // *config.Store
    Logger: l,
    FS:     fs,
}

// Reads pin an immutable snapshot:
timeout := p.Config.View().GetString("app.timeout")

The store is the live object that owns reloads and writes; reads go through props.Config.View(), which pins a consistent snapshot (a *config.View, satisfying the module's config.Reader). Hot reload is explicit, the root pre-run calls Store.Watch, so file changes propagate without any per-package wiring, and writes go through the store's transactional Apply, which edits the target file in place, preserving comments and writing only the named keys.

The layer declaration, highest precedence first: changed CLI flags; environment variables under the tool's EnvPrefix; the project-local .<tool>.yaml; the config files (--config paths if given, otherwise ~/.<tool>/config.yaml then /etc/<tool>/config.yaml); the tool's ConfigPaths embedded assets; and the merged assets/config.yaml embedded defaults, which always apply. The per-user config outranks the system /etc file, the Unix convention.

That is the default. A tool can declare its own stack in props.Tool.Config.Layers, lowest precedence first, by the names defaults, files, project, env and flags, and the store appends the layers in exactly that order: the declaration is the precedence. Leaving a name out removes that layer. Three orders are refused, by props.New and again when the store is built, because each would make a tool quietly unsafe rather than merely unusual. defaults must be lowest, since nothing below the compiled-in defaults could ever be read. flags must be highest, since a layer above it would mean a flag the user passed did not take. And project must sit below env, since the trust filter below assumes a repository's file cannot outrank the environment. See spec 0204 D1.

File formats are linked, and chosen by extension¶

A config file's format comes from its extension, whether the file is named by --config, found on the default search paths or embedded as one of the tool's ConfigPaths. YAML needs nothing: .yaml, .yml, no extension, and any extension that belongs to no format are all read as YAML, as every file was before formats could be linked. The seven other formats of the go/config family are each a link: a tool reads them only when its main blank-imports the format's package, so a tool that never reads HCL carries none of HCL's dependencies.

A file whose extension belongs to one of these formats, in a tool that does not link it, is refused before anything is read. The refusal names the extensions the tool does accept and the package that would read the file. A read-only format is read like any other layer, and a write goes to the highest-precedence writable one. Each link contributes its codec under setup.SlotConfigCodec, and setup.ConfigCodecFor makes the choice. See spec 0204 D2.

Only config files that exist are declared as layers, a non-existent file contributes nothing and must not become a phantom write target. The one exception is the write destination: the highest-precedence path is always available so config set/unset/edit have somewhere to land and can create the file on first write. Writes therefore go to the per-user config (or the project-local file when one is present), never to a system path the user cannot write.

Two safeguards ride on every config-command write. The written file is re-tightened to 0600: the store deliberately preserves an existing file's mode on write (it treats the mode as the owner's choice), but a GTB config file routinely holds credentials, so the framework actively re-asserts owner-only permissions. And when a write would place a recognised credential in a project-local .<tool>.yaml. A file that may be committed to version control: config set warns, and, in an interactive terminal, asks to confirm. It never blocks the write (a project-local secret can be deliberate), but it nudges toward env-var or keychain storage. See config set and Configure credentials.

Props.Tool.EnvPrefix is propagated into the store as the module's WithEnv layer, so a tool's config keys resolve from MYTOOL_* rather than bare names. See the module's env-prefix rationale.

Project-local trust: security keys are ignored until you trust the file¶

The project-local .<tool>.yaml is convenient, but it arrives with a repository you may not have written. A git clone can ship one. So its security-sensitive keys are ignored unless you explicitly trust the file, while its ordinary workflow keys (logging, output, feature toggles) always apply. The protected set is the keys that govern self-update verification and posture (update.require_signature, update.require_checksum, update.require_external_crosscheck, update.policy, update.key_source, update.external_key_email), telemetry consent (telemetry.enabled), and every credential subtree (*.auth.*, *.api.*, bitbucket.app_password). When any of these is stripped from an untrusted file, the framework logs a WARN naming the file and the ignored keys. Never a silent drop.

One subtree is stripped even from a trusted file: config.sources, which says where configuration comes from (a Consul address, a Vault mount, a bucket). Trusting a repository to set its own log level is a smaller act than trusting it to choose your secret store, so config trust does not re-admit it, and the framework logs a WARN when a trusted file carries it. A source's settings belong in your own config file, the environment, or the tool's embedded defaults. See spec 0204 D5.

Trust is direnv-style. <tool> config trust records the file's absolute path and the SHA-256 of its exact current content in a per-user store (~/.<tool>/trusted-projects.yaml, owner-only, never inside a repository). Editing a trusted file (or a fresh clone swapping it out) changes the hash and revokes trust until you run the command again. Use config trust --list to see what is trusted and config trust --forget to revoke. Until a file is trusted it is also read-only: config set in an untrusted repository routes the write to your own config, not the repository file.

An explicit --config suppresses the project-local layer entirely (naming a file means "use this one"), so this trust gate only applies to the implicitly discovered .<tool>.yaml. CI runs untrusted by default, a pipeline that legitimately depends on project-local security keys should trust the file in a provisioning step or supply those values through the user config or environment. See config trust and the security-decisions record.

Embedded defaults: the assets/config.yaml convention¶

GTB discovers shipped assets at fixed paths inside each registered bundle's embed.FS (directive: //go:embed assets/*):

• assets/config.yaml: the embedded-defaults layer. Merged across every registered bundle and always applied as the lowest-precedence layer, so a user file that omits a key resolves to the shipped default.
• assets/init/config.yaml: the init template: the human-facing document (comments included) that init writes to the user's config file.

The framework's own baseline bundle registers first (inside props.NewAssets), the tool's bundle next, and feature bundles (registered via setup.RegisterAssets) are applied for enabled features at root construction: later bundles override earlier ones in the merged structured reads:

//go:embed assets/*
var assets embed.FS

p := &props.Props{
    Assets: props.NewAssets(props.AssetMap{"root": &assets}),
    // ...
}

Defaults live only here. Never duplicated into default: struct tags, which the module treats as hint text and never applies.

The project-local config layer¶

At startup GTB also looks for a project-local file named .<tool>.yaml (e.g. .myapp.yaml), discovered by walking up from the working directory to the filesystem root. A repo-root convention like .editorconfig. When found it is merged last among the file sources, so it deep-merges over and overrides the per-user ~/.<tool>/config.yaml:

~/.myapp/config.yaml          # global, per-user
/path/to/repo/.myapp.yaml     # project — overrides the global, committed with the repo

The file may be in any format the tool links: discovery looks for .<tool>.yaml and .<tool> plus each linked format's extension, so a tool linking TOML also finds .myapp.toml. The nearest directory holding a candidate wins. Two candidates in the same directory, such as .myapp.yaml beside .myapp.toml, stop the command with both paths named, because silently picking one is exactly how a repository could shadow the file you believe is being read. Whatever the format, the trust filter below strips the same keys: it decodes through the format's own codec first and filters what came out, and its tests run the hostile-clone case once per format. See spec 0204 D16 and D17.

This keeps a project's non-secret settings in the repo that owns them. A tool opts out simply by not having the file; it never errors when absent. Environment variables and flags still override it: it sits in the file tier of the module's precedence chain. An explicit --config suppresses the layer entirely: naming a config file means "use this one", and a project-local file the caller did not name must not override files they did. The filename derives from Props.Tool.Name.

Binding CLI flags¶

GTB registers and binds flags on your behalf, so you rarely touch the module's flag layer (config.WithFlags / config.BindFlag) directly:

portFlags := pflag.NewFlagSet("server", pflag.ContinueOnError)
portFlags.Int("server-port", 8080, "server port")

rootCmd := root.NewCmdRootWithOptions(props,
    root.WithBoundFlags(map[string]*pflag.Flag{"server.port": portFlags.Lookup("server-port")}),
    // or, by convention (--server-port -> server.port):
    root.WithConventionBoundFlags(portFlags),
)

A subcommand's own local flags are bound by the same hyphen-to-dot convention when that command runs. Only flags the user actually changed are bound, which is what keeps a defaulted flag from masking file or env values. See the module's default-clobber warning.

The built-in --debug and --ci flags fold through the same path, so Config.View().GetBool("ci") reflects --ci; --debug additionally retains its immediate effect on the log level.

Config sources¶

A tool can read configuration from somewhere other than files: a Consul prefix, a Vault path, a bucket. Each such source is a named slot the tool declares in props.Tool.Config.Sources and places in props.Tool.Config.Layers like any other layer, so its position is its precedence. Where the slot connects is not declared anywhere in the tool; it is runtime configuration under config.sources.<name>, in the user's own file, the environment or the tool's embedded defaults.

A kind (vault, consul, aws-s3, file, ...) is a link package: blank-importing it registers a factory that builds the backend from a slot's settings, through setup.RegisterConfigSourceKind. When a tool declares sources the store is built in two passes. The first is a store of embedded defaults, the tool's own files, the environment and flags, never the project-local file; each factory reads its slot's settings from that view, so a repository cannot choose where configuration comes from even in principle. The second pass is the full stack with every source in its declared place.

• A slot is required unless it says required: false. A required slot that nobody configured, or whose backend cannot be built, stops the tool with the slot named; an optional one is left out with a warning.
• A slot is read-only unless it says writable: true, so config set never lands in a secret store by accident. The core's sensitive-leak guard still applies to anything a secret source holds.
• An optional slot that cannot be watched drops out of watching alone; a watch failure with sources in the stack is logged at warn, not debug.
• An author override (setup.OverrideConfigSource(name, factory) in the tool's main) replaces the factory for one slot, and is the only way to build etcd, sftp, billy, iofs and afero sources. An override for a slot the tool does not declare stops the tool, so a stale one cannot add a layer.

See spec 0204 D3 to D7, D11 and D19.

The tool's own format¶

A tool writes its own config file in one format, set by props.Tool.Config.Format: yaml (the default), toml, json or hcl, the four the family can edit, and the format must be linked. The file is named for it, config.<ext>, through props.Tool.ConfigFilename(), both on the default search paths and where init writes. props.New refuses a read-only format here, since init writes the file and config set edits it.

Every init template fragment stays YAML, the tool's own included. init merges the fragments across bundles as it always has, then writes the result through the own format's codec (setup.EncodeConfig). The merge decodes and re-encodes, so template comments never reached a user's file in any format; see spec 0204 revision R4. config edit and config unset decode the file they touch through its own codec too, and config edit seeds a new JSON file with {} rather than a comment JSON cannot hold.

Changing a tool's own format never converts anyone's file. A tool that finds its config in the old format and none in the new one refuses to start, naming both paths, rather than run on nothing or let auto-initialise write a fresh file beside the old one. The hint is <tool> config convert --from <old> --to <new> when the tool has the config command, and a manual conversion when it does not; doctor reports the same as a failure. Commands that opt out of the config check, doctor and config convert among them, still run, and an explicit --config is never refused, because it names the file to read. Automatic conversion was rejected: comments and formatting do not survive it, and it would run where nobody is watching (spec 0204 D14).

Initialiser integration¶

Tool initialisers work against two narrow surfaces: IsConfigured(cfg config.Reader) checks existing state against a pinned view, and Configure(p, cfg setup.Editor) writes new values through cfg.Set(...), the setup.Editor routes writes through the store's transactional Apply, so template comments in the user's file survive the wizards.

Sensitive-value masking¶

Masking lives in GTB's config command (pkg/cmd/config/sensitive.go), not in the container: the module never inspects values for sensitivity. config get / config list render secrets as ****<tail> using three independent strategies:

1. Declared literals: every plaintext credential key the credential registry declares for the tool's enabled features (github.auth.value, bitbucket.username, anthropic.api.key, ...) is masked by name, exactly. The pointer keys beside them are not: github.auth.env holds the name of an environment variable and github.auth.keychain a service/account reference, and a listing that hides them cannot show where a credential comes from.
2. Key-name matching: the leaf segment of the dotted key against token, password, secret, apikey, api_key, auth, app_password, plus key when it is the whole leaf. Only the leaf counts, so gitlab.ssh.key.type, update.key_source and signing.key_id are settings and stay readable.
3. Value-content matching: the value against known token patterns (e.g. ghp_…, github_pat_…), whatever the key, so a token pasted into a pointer key is still hidden.

Tool authors extend it via functional options:

cmdconfig.NewCmdConfig(props,
    cmdconfig.WithKeyPattern("credential"),
    cmdconfig.WithValuePattern(regexp.MustCompile(`^sk-[A-Za-z0-9]{32}$`)),
)

Relationship with init and config¶

Both InitCmd and ConfigCmd should be disabled in containerised services where local YAML config is not applicable.

Related¶

• Module docs: config.go.phpboyscout.uk
• How-to: Test code that uses configuration, Bind flags to config, React to config changes, Validate component config
• Assets: Embed custom assets