Skip to content

v0.x: the config stack a tool declares is its precedence

Spec 0204 makes the layer list a tool declares the order its configuration resolves in. Before it, the list said which layers a tool wired, and the order it gave was documentation: the store always appended the layers in the framework's order.

What changed

  • props.Tool.Config.Layers is the stack, lowest precedence first. The store appends each layer in exactly that order, so moving an entry moves its precedence. Unstated, it resolves to the framework default: defaults, files, project, env, flags.
  • Three orders are refused, by props.New, by the store and by the generator, because each makes a tool quietly unsafe:
    • defaults must be the lowest layer when present. A layer below the compiled-in defaults can never be read.
    • flags must be the highest layer when present. A layer above it means a flag the user passed does not take.
    • project must sit below env. The trust filter assumes a repository's file cannot outrank the environment.
  • props.Tool.ConfigLayers is deprecated. A tool still setting it resolves exactly as before: its layers, in the framework's order, whatever order it lists them in.
  • The manifest field moves. properties.config_layers becomes properties.config.layers. The first regenerate moves an existing list into the new field in the framework's order, logs the move with the other derived values, and renders the root as props.Tool{Config: props.ConfigSpec{Layers: ...}}. What the tool resolves does not change.
  • --config-layers and gtb set config.layers take an ordered list, lowest precedence first, and refuse the three orders above.

Config files in other formats

A config file's format now comes from its extension (spec 0204 D2), and every format except YAML is a link: pkg/config/formats/<format>, blank-imported from main, for toml, json, hcl, ini, xml, dotenv (.env) and properties.

  • A --config file with one of those extensions is refused unless the tool links the format. A .json file used to be read as YAML, which mostly worked because JSON is nearly a subset of YAML; it is now refused with a hint naming the link. Link pkg/config/formats/json, or rename the file to .yaml.
  • Any other extension is read as YAML, as before. ~/.mytool.conf keeps working.
  • An embedded ConfigPaths asset in another format is now decoded as that format. props.Assets merged a .toml asset and re-emitted it as TOML, which the store then read as YAML. The tool must link the format for the asset to be read.

The project file in other formats

  • The project-local file may be in any linked format (spec 0204 D16). A tool linking TOML also discovers .<tool>.toml. .<tool>.yml is still not discovered, as before.
  • Two project files in one directory stop the command, naming both. A repository carrying .mytool.yaml beside .mytool.toml must remove one.
  • The trust filter strips the same keys whatever the format (D17), and a trusted project file in a writable format is written in its own format.
  • setup.DiscoverProjectConfig is deprecated in favour of setup.FindProjectConfig, which takes the linked codecs and returns the two-candidate refusal as an error. The deprecated function still finds .<tool>.yaml only.
  • config set warns before writing a credential into a project file of any format. It used to recognise one only by a .yaml name.

The tool's own format

  • props.Tool.Config.Format sets the format of the tool's own config file: yaml (default), toml, json or hcl, linked. props.New refuses a read-only format.
  • The file is named for it: props.Tool.ConfigFilename() returns config.<ext>, used by the default --config paths, by init and by config set's fallback. setup.DefaultConfigFilename is deprecated; it is still config.yaml.
  • init writes the tool's own format. Every init template fragment stays assets/init/config.yaml; init merges them and encodes the result through the own format's codec. A YAML tool's file is byte-for-byte what it was.
  • config edit and config unset read the file through its own codec, where both used to parse every file as YAML.
  • setup.EncodeConfig and setup.DecodeConfig write and read a document through any codec.

Changing a tool's own format

When a tool's own format changes, its users' files do not (spec 0204 D14). A tool that finds config.<old> and no config.<new> in a default location refuses to start, naming both, and the hint gives the command: <tool> config convert --from <old> --to <new>, which writes the new file and leaves the old one in place. Without the config command the hint says to convert by hand. doctor reports the same condition as a failure. An explicit --config is never refused.

Generated projects

gtb generate and gtb set take the formats a tool links and its own format: --config-formats and --config-format, recorded as properties.config.formats and properties.config.format. The generator writes cmd/<name>/config.go with one blank import per linked format and removes it when the list is empty, and renders the own format into props.Tool.Config. YAML is never listed, since it is built in. A from-scratch regenerate manifest now recovers config.layers, config.formats and config.format; the layer list used to be lost.

Config sources

New and additive: nothing changes for a tool that declares none.

  • props.Tool.Config.Sources declares source slots (props.ConfigSource: name, kind, required, writable). A tool declaring sources must place every one in Config.Layers; props.New refuses one that is not placed, a duplicate name, a built-in layer's name, or a name that is not a lower-case command word.
  • setup.RegisterConfigSourceKind is what a kind's link package calls; setup.OverrideConfigSource replaces a slot's factory from the tool's main.
  • props.ValidateConfigSpec validates the whole declaration; props.Tool.ResolvedConfigSpec() is it with the layer list resolved.
  • A config watch failure is logged at warn when the stack holds a source.
  • <tool> init config <name> configures a declared slot. Commands that opt out of the config check treat every slot as optional.
  • Source kinds: file, keychain, vault, consul, aws-s3, aws-ssm, aws-secrets, azure-blob, azure-keyvault, azure-appconfig, gcp-gcs, gcp-secret and gcp-parameter, each a blank import of pkg/config/sources/<kind>. Vault and Consul tokens may come through config.sources.<name>.auth.env, .auth.keychain or .auth.value (refused under CI), or the provider's own variable.
  • setup.ConfigBootstrap gains FS() and CodecFor(path), so a factory reads a file through the tool's filesystem in any linked format. setup.SettingsInitialiser is the form a kind's init config asks.
  • Generated projects declare slots with gtb generate --config-source name=kind, --config-source-optional and --config-source-writable, recorded as properties.config.sources. A slot's layer renders as props.ConfigLayer("<name>"). With no layer list the slots sit above the defaults and below the user's own files. A from-scratch regenerate manifest recovers the slots. generator.ValidateConfigStack checks a layer list and its slots together; ValidateConfigLayers alone refuses a slot's name.
  • --config is no longer a configuration key. It was bound like any other flag, so config list showed a config entry holding the file paths, and that entry hid every config.* key beneath it. Nothing read it.

Migrating a hand-written tool

Replace the deprecated field with the spec, in the order you want the layers to resolve:

// Before
props.Tool{ConfigLayers: []props.ConfigLayer{props.LayerFlags, props.LayerEnv, props.LayerFiles, props.LayerDefaults}}

// After
props.Tool{Config: props.ConfigSpec{Layers: []props.ConfigLayer{props.LayerDefaults, props.LayerFiles, props.LayerEnv, props.LayerFlags}}}

A list written highest-first, as in the example, has to be reversed. Written as it was, props.New refuses it because defaults is not the lowest layer.

Migrating a generated project

Run gtb regenerate project. Nothing else is needed.