job-search/internal/config/README.md

config

Package config loads the JobSearch configuration into a JobSearchConfig struct. Values come from struct-tag defaults, a YAML or JSON config file and environment variables, merged through Viper.

Loading order

Init(cfgFile, &conf) builds the configuration in this order:

1. Loads .env from the working directory into the environment. A missing .env file is an error.
2. Sets the defaults from the default struct tags as Viper defaults.
3. Reads the config file: cfgFile if given, otherwise the first config.yaml found in /etc/job-search then the working directory. If no file is found a warning is logged and the defaults are used. A file that exists but can't be parsed is an error.
4. Reads environment variables prefixed with JOBSEARCH_. Dots in a key become underscores, so logging.level is read from JOBSEARCH_LOGGING_LEVEL.
5. Copies every known key from Viper into conf. A value that fails validation (e.g. an unknown log level) is logged as a warning and the previous value is kept.

When more than one place sets a key, environment variables win over the config file, which wins over the defaults. A debug or error log level already set on conf before Init (from the --debug and --quiet flags) is kept.

Keys

Key	Type	Default	Notes
logging.level	string	warn	One of debug, info, warn, error
logging.file	string		Log file for fetch service (unless --foreground)
sources.<source>.enabled	bool	false	For every source
sources.<source>.interval	duration	10m	For every source; e.g. 30s, 1h
sources.find_a_tender.api_root	string	URL of the Find A Tender service API. Currently https://www.find-tender.service.gov.uk/api/1.0
sources.fts_mock.api_root	string		URL of the mock FTS server (see internal/mock)
sources.sleeper.block_shutdown	bool	false	Test source: if true, its sleep ignores shutdown

The current sources are find_a_tender, fts_mock and sleeper. See "Addition of a new opportunities source" in the top level README for how to add a new source.

Example config.yaml:

logging:
    level: info
    file: ""
sources:
    find_a_tender:
        enabled: true
        interval: 10m
    fts_mock:
        enabled: false
        interval: 1m
        api_root: http://localhost:8080/api/1.0

Sections and struct tags

Each section of the configuration is a plain struct. The reflection code in section.go reads its struct tags, so a section needs no hand-written code per key:

• cfg:"key" is the field's configuration key. Fields without it are not part of the configuration.
• default:"value" is the field's default, parsed as if it were a string passed to SetValue (e.g. "10m", "true").
• A tagged field whose type is a struct is a nested section, addressed as section.key. An embedded struct has its fields promoted, as Go does; every source embeds JobSearchConfigSource for enabled and interval.
• time.Duration fields report reflect.String from TypeOf, accept either a time.Duration or a duration string in SetValue, and are written as strings ("10m0s") by Map, JSON and YAML.
• A section can reject values by implementing validate(key, value) error on its pointer type. JobSearchConfigLogging does this for level.

Each section type has the same small set of methods, each a one-line call into section.go: Map, Flatten, TypeOf, ValueOf and SetValue, plus Keys on some.

Adding a key or section

To add a key to an existing section, add a tagged field to its struct. Nothing else is needed for it to be loaded, printed, and read or written by the config CLI commands.

To add a new section:

1. Create a conf_*.go file with the section struct, a JSCK* constant for its key, and the standard methods (copy an existing section).
2. Add the struct as a tagged field of its parent: JobSearchConfig in config.go, or JobSearchConfigSources in conf_sources.go for a source, etc. The field's cfg tag must match the JSCK* constant.

API

• Init(cfgFile, &conf): load the configuration, as described above.
• conf.TypeOf(key), conf.ValueOf(key), conf.SetValue(key, value): read or set one dotted key on the loaded struct. SetValue converts and validates the value.
• conf.Map() (nested) and conf.Flatten() (dotted keys): the whole configuration as a map.
• conf.String(), conf.JSON(), conf.YAML() and the matching Print* methods: the configuration as text.
• StoreValue(&conf, key, value) / PurgeValue(&conf, key): set or remove a key in the config file Viper loaded. Used by config set and config set --unset. StoreValue validates the value before writing it.
• LoadDefaultConfig, LoadToViper, LoadFromViper: move values between the struct defaults, Viper and a JobSearchConfig.

Files

File	Contents
config.go	JobSearchConfig, defaults, Viper loading, text output
load.go	Init, .env and config file loading
modify.go	StoreValue and PurgeValue
section.go	Struct tag reflection shared by every section
conf_logging.go	logging section and log level helpers
conf_sources.go	sources section and the shared source fields
conf_source_*.go	One file per source section
print.go, debug.go	Printing and debug logging of the configuration

Testing

Viper keeps its state in package globals, so tests that use it call resetViper(t) to reset it before and after the test. Tests of Init use initDir, which changes into a temp directory with its own .env and config.yaml, so the real files in the repository are never read.