Added unit tests, documentation and function descriptions across the board. Increased cyclop complexity threshold to 15 which is a bit more sane than 10 and excluded _test.go files from linters.
This commit is contained in:
bwilliams committed
1 parent
005b335371
commit
7fd321df97
31 files changed
+4106
-1
No files matched your search
@@ -0,0 +1,133 @@
|
||||
# config
|
||||
|
||||
Package config loads the JobSearch configuration into a `JobSearchConfig`
|
||||
struct. Values come from struct-tag defaults, a YAML or JSON config file and
|
||||
environment variables, merged through [Viper](https://github.com/spf13/viper).
|
||||
|
||||
|
||||
## Loading order
|
||||
`Init(cfgFile, &conf)` builds the configuration in this order:
|
||||
|
||||
1. Loads `.env` from the working directory into the environment. A missing
|
||||
`.env` file is an error.
|
||||
2. Sets the defaults from the `default` struct tags as Viper defaults.
|
||||
3. Reads the config file: `cfgFile` if given, otherwise the first
|
||||
`config.yaml` found in `/etc/job-search` then the working directory. If no
|
||||
file is found a warning is logged and the defaults are used. A file that
|
||||
exists but can't be parsed is an error.
|
||||
4. Reads environment variables prefixed with `JOBSEARCH_`. Dots in a key
|
||||
become underscores, so `logging.level` is read from
|
||||
`JOBSEARCH_LOGGING_LEVEL`.
|
||||
5. Copies every known key from Viper into `conf`. A value that fails
|
||||
validation (e.g. an unknown log level) is logged as a warning and the
|
||||
previous value is kept.
|
||||
|
||||
When more than one place sets a key, environment variables win over the
|
||||
config file, which wins over the defaults. A `debug` or `error` log level
|
||||
already set on `conf` before `Init` (from the `--debug` and `--quiet` flags)
|
||||
is kept.
|
||||
|
||||
|
||||
## Keys
|
||||
|
||||
| Key | Type | Default | Notes |
|
||||
|---|---|---|---|
|
||||
| `logging.level` | string | `warn` | One of `debug`, `info`, `warn`, `error` |
|
||||
| `logging.file` | string | | Log file for `fetch service` (unless `--foreground`) |
|
||||
| `sources.<source>.enabled` | bool | `false` | For every source |
|
||||
| `sources.<source>.interval` | duration | `10m` | For every source; e.g. `30s`, `1h` |
|
||||
| `sources.find_a_tender.api_root` | string | URL of the Find A Tender service API. Currently `https://www.find-tender.service.gov.uk/api/1.0` | |
|
||||
| `sources.fts_mock.api_root` | string | | URL of the mock FTS server (see `internal/mock`) |
|
||||
| `sources.sleeper.block_shutdown` | bool | `false` | Test source: if true, its sleep ignores shutdown |
|
||||
|
||||
The current sources are `find_a_tender`, `fts_mock` and `sleeper`. See
|
||||
"Addition of a new opportunities source" in the top level README for how to add
|
||||
a new source.
|
||||
|
||||
Example `config.yaml`:
|
||||
```yaml
|
||||
logging:
|
||||
level: info
|
||||
file: ""
|
||||
sources:
|
||||
find_a_tender:
|
||||
enabled: true
|
||||
interval: 10m
|
||||
fts_mock:
|
||||
enabled: false
|
||||
interval: 1m
|
||||
api_root: http://localhost:8080/api/1.0
|
||||
```
|
||||
|
||||
|
||||
## Sections and struct tags
|
||||
Each section of the configuration is a plain struct. The reflection code in
|
||||
`section.go` reads its struct tags, so a section needs no hand-written code
|
||||
per key:
|
||||
|
||||
- `cfg:"key"` is the field's configuration key. Fields without it are not part
|
||||
of the configuration.
|
||||
- `default:"value"` is the field's default, parsed as if it were a string
|
||||
passed to `SetValue` (e.g. `"10m"`, `"true"`).
|
||||
- A tagged field whose type is a struct is a nested section, addressed as
|
||||
`section.key`. An embedded struct has its fields promoted, as Go does;
|
||||
every source embeds `JobSearchConfigSource` for `enabled` and `interval`.
|
||||
- `time.Duration` fields report `reflect.String` from `TypeOf`, accept either a
|
||||
`time.Duration` or a duration string in `SetValue`, and are written as
|
||||
strings (`"10m0s"`) by `Map`, `JSON` and `YAML`.
|
||||
- A section can reject values by implementing `validate(key, value) error` on
|
||||
its pointer type. `JobSearchConfigLogging` does this for `level`.
|
||||
|
||||
Each section type has the same small set of methods, each a one-line call into
|
||||
`section.go`: `Map`, `Flatten`, `TypeOf`, `ValueOf` and `SetValue`, plus
|
||||
`Keys` on some.
|
||||
|
||||
|
||||
## Adding a key or section
|
||||
To add a key to an existing section, add a tagged field to its struct. Nothing
|
||||
else is needed for it to be loaded, printed, and read or written by the
|
||||
`config` CLI commands.
|
||||
|
||||
To add a new section:
|
||||
1. Create a `conf_*.go` file with the section struct, a `JSCK*` constant for
|
||||
its key, and the standard methods (copy an existing section).
|
||||
2. Add the struct as a tagged field of its parent: `JobSearchConfig` in
|
||||
`config.go`, or `JobSearchConfigSources` in `conf_sources.go` for a source,
|
||||
etc. The field's `cfg` tag must match the `JSCK*` constant.
|
||||
|
||||
|
||||
## API
|
||||
- `Init(cfgFile, &conf)`: load the configuration, as described above.
|
||||
- `conf.TypeOf(key)`, `conf.ValueOf(key)`, `conf.SetValue(key, value)`: read
|
||||
or set one dotted key on the loaded struct. `SetValue` converts and
|
||||
validates the value.
|
||||
- `conf.Map()` (nested) and `conf.Flatten()` (dotted keys): the whole
|
||||
configuration as a map.
|
||||
- `conf.String()`, `conf.JSON()`, `conf.YAML()` and the matching `Print*`
|
||||
methods: the configuration as text.
|
||||
- `StoreValue(&conf, key, value)` / `PurgeValue(&conf, key)`: set or remove a
|
||||
key in the config file Viper loaded. Used by `config set` and
|
||||
`config set --unset`. `StoreValue` validates the value before writing it.
|
||||
- `LoadDefaultConfig`, `LoadToViper`, `LoadFromViper`: move values between
|
||||
the struct defaults, Viper and a `JobSearchConfig`.
|
||||
|
||||
|
||||
## Files
|
||||
|
||||
| File | Contents |
|
||||
|---|---|
|
||||
| `config.go` | `JobSearchConfig`, defaults, Viper loading, text output |
|
||||
| `load.go` | `Init`, `.env` and config file loading |
|
||||
| `modify.go` | `StoreValue` and `PurgeValue` |
|
||||
| `section.go` | Struct tag reflection shared by every section |
|
||||
| `conf_logging.go` | `logging` section and log level helpers |
|
||||
| `conf_sources.go` | `sources` section and the shared source fields |
|
||||
| `conf_source_*.go` | One file per source section |
|
||||
| `print.go`, `debug.go` | Printing and debug logging of the configuration |
|
||||
|
||||
|
||||
## Testing
|
||||
Viper keeps its state in package globals, so tests that use it call
|
||||
`resetViper(t)` to reset it before and after the test. Tests of `Init`
|
||||
use `initDir`, which changes into a temp directory with its own `.env` and
|
||||
`config.yaml`, so the real files in the repository are never read.
|
||||
Reference in new issue
Block a user