149 lines
6.6 KiB
Markdown
149 lines
6.6 KiB
Markdown
# config
|
|
|
|
Package config loads the JobSearch configuration into a `JobSearchConfig`
|
|
struct. Values come from struct-tag defaults, a YAML or JSON config file and
|
|
environment variables, merged through [Viper](https://github.com/spf13/viper).
|
|
|
|
|
|
## Loading order
|
|
`Init(cfgFile, &conf)` builds the configuration in this order:
|
|
|
|
1. Loads `.env` from the working directory into the environment. A missing
|
|
`.env` file is an error.
|
|
2. Sets the defaults from the `default` struct tags as Viper defaults.
|
|
3. Reads the config file: `cfgFile` if given, otherwise the first
|
|
`config.yaml` found in `/etc/job-search` then the working directory. If no
|
|
file is found a warning is logged and the defaults are used. A file that
|
|
exists but can't be parsed is an error.
|
|
4. Reads environment variables prefixed with `JOBSEARCH_`. Dots in a key
|
|
become underscores, so `logging.level` is read from
|
|
`JOBSEARCH_LOGGING_LEVEL`.
|
|
5. Copies every known key from Viper into `conf`. A value that fails
|
|
validation (e.g. an unknown log level) is logged as a warning and the
|
|
previous value is kept.
|
|
|
|
When more than one place sets a key, environment variables win over the
|
|
config file, which wins over the defaults. A `debug` or `error` log level
|
|
already set on `conf` before `Init` (from the `--debug` and `--quiet` flags)
|
|
is kept.
|
|
|
|
|
|
## Keys
|
|
|
|
| Key | Type | Default | Notes |
|
|
|---|---|---|---|
|
|
| `logging.level` | string | `warn` | One of `debug`, `info`, `warn`, `error` |
|
|
| `logging.file` | string | | Log file for `fetch service` (unless `--foreground`); other commands log to the terminal |
|
|
| `database.file` | string | | Path to the SQLite database file, used by the `migrate` commands |
|
|
| `sources.<source>.enabled` | bool | `false` | For every source |
|
|
| `sources.<source>.interval` | duration | `10m` | For every source; e.g. `30s`, `1h` |
|
|
| `sources.find_a_tender.api_root` | string | `https://www.find-tender.service.gov.uk/api/1.0` | URL of the Find a Tender API |
|
|
| `sources.fts_mock.api_root` | string | | URL of the mock FTS server (see `internal/mock`) |
|
|
| `sources.sleeper.block_shutdown` | bool | `false` | Test source: if true, its sleep ignores shutdown |
|
|
|
|
The current sources are `find_a_tender`, `fts_mock` and `sleeper`. See
|
|
"Addition of a new opportunities source" in the top level README for how to add
|
|
a new source.
|
|
|
|
Example `config.yaml`:
|
|
```yaml
|
|
logging:
|
|
level: info
|
|
file: ""
|
|
database:
|
|
file: ./jobsearch.db
|
|
sources:
|
|
find_a_tender:
|
|
enabled: true
|
|
interval: 10m
|
|
fts_mock:
|
|
enabled: false
|
|
interval: 1m
|
|
api_root: http://localhost:8080/api/1.0
|
|
```
|
|
|
|
|
|
## Sections and struct tags
|
|
Each section of the configuration is a plain struct. The reflection code in
|
|
`section.go` reads its struct tags, so a section needs no hand-written code
|
|
per key:
|
|
|
|
- `cfg:"key"` is the field's configuration key. Fields without it are not part
|
|
of the configuration.
|
|
- `default:"value"` is the field's default, parsed as if it were a string
|
|
passed to `SetValue` (e.g. `"10m"`, `"true"`).
|
|
- A tagged field whose type is a struct is a nested section, addressed as
|
|
`section.key`. An embedded struct has its fields promoted, as Go does;
|
|
every source embeds `JobSearchConfigSource` for `enabled` and `interval`.
|
|
- `time.Duration` fields report `reflect.String` from `TypeOf`, accept either a
|
|
`time.Duration` or a duration string in `SetValue`, and are given out as
|
|
strings (`"10m0s"`) by `Map`, `Flatten` and `ValueOf`, and so by `JSON`,
|
|
`YAML` and `String` too.
|
|
- A section can reject values by implementing `validate(key, value) error` on
|
|
its pointer type. `JobSearchConfigLogging` does this for `level`.
|
|
|
|
Every section implements the `Section` interface defined in `section.go`:
|
|
`Map`, `Flatten`, `TypeOf`, `ValueOf` and `SetValue`, each a one-line call into
|
|
`section.go` (some sections also have `Keys`). The methods sit under a
|
|
`// ---------- Section methods (see Section) ----------` banner in each
|
|
section's file. `TestSections` walks `JobSearchConfig` and checks every
|
|
section, including embedded ones, implements `Section` and that its
|
|
`Flatten` keys match its `cfg` tag, so a new section is checked without being
|
|
listed anywhere.
|
|
|
|
|
|
## Adding a key or section
|
|
To add a key to an existing section, add a tagged field to its struct. Nothing
|
|
else is needed for it to be loaded, printed, and read or written by the
|
|
`config` CLI commands.
|
|
|
|
To add a new section:
|
|
1. Create a `conf_*.go` file with the section struct, a `JSCK*` constant for
|
|
its key, and an implementation of `Section` (copy an existing section's
|
|
methods).
|
|
2. Add the struct as a tagged field of its parent: `JobSearchConfig` in
|
|
`config.go`, or `JobSearchConfigSources` in `conf_sources.go` for a source,
|
|
etc. The field's `cfg` tag must match the `JSCK*` constant.
|
|
|
|
Nothing in `section.go` needs changing.
|
|
|
|
|
|
## API
|
|
- `Init(cfgFile, &conf)`: load the configuration, as described above.
|
|
- `conf.TypeOf(key)`, `conf.ValueOf(key)`, `conf.SetValue(key, value)`: read
|
|
or set one dotted key on the loaded struct. `SetValue` converts and
|
|
validates the value.
|
|
- `conf.Map()` (nested) and `conf.Flatten()` (dotted keys): the whole
|
|
configuration as a map.
|
|
- `conf.String()`, `conf.JSON()`, `conf.YAML()` and the matching `Print*`
|
|
methods: the configuration as text, with keys sorted.
|
|
- `StoreValue(&conf, key, value)` / `PurgeValue(&conf, key)`: set or remove a
|
|
key in the config file Viper loaded. Used by `config set` and
|
|
`config set --unset`. `StoreValue` validates the value before writing it,
|
|
and stores it as the field's own type. Only the file's own settings are
|
|
written back; defaults and environment variables never are.
|
|
- `LoadDefaultConfig`, `LoadToViper`, `LoadFromViper`: move values between
|
|
the struct defaults, Viper and a `JobSearchConfig`.
|
|
|
|
|
|
## Files
|
|
|
|
| File | Contents |
|
|
|---|---|
|
|
| `config.go` | `JobSearchConfig`, defaults, Viper loading, text output |
|
|
| `conf_database.go` | `database` section |
|
|
| `load.go` | `Init`, `.env` and config file loading |
|
|
| `modify.go` | `StoreValue` and `PurgeValue` |
|
|
| `section.go` | The `Section` interface and the struct tag reflection shared by every section |
|
|
| `conf_logging.go` | `logging` section and log level helpers |
|
|
| `conf_sources.go` | `sources` section and the shared source fields |
|
|
| `conf_source_*.go` | One file per source section |
|
|
| `print.go`, `debug.go` | Printing and debug logging of the configuration |
|
|
|
|
|
|
## Testing
|
|
Viper keeps its state in package globals, so tests that use it call
|
|
`resetViper(t)` to reset it before and after the test. Tests of `Init`
|
|
use `initDir`, which changes into a temp directory with its own `.env` and
|
|
`config.yaml`, so the real files in the repository are never read.
|