Replaced the Title filter with a general search, run by the backend

GET /api/opportunities/search?q= returns the UUIDs of the matching opportunities, and the frontend filters its list by them. Words without a prompt search the title and description; title:, buyer:, buyer_id: and cpv: search only their own field, and a leading ! leaves out what a term matches. Repeats of one prompt are OR, everything else AND. CPV terms are checked per code: cpv:72 !cpv:7221 needs a code starting 72 that doesn't start 7221. Unknown prompts are reported, and every response carries the help that the search box's "?" panel shows. Migration 00011 adds the description and buyer name to the full-text index, and indexes CPV descriptions.
A CPV code is now stored once, keeping its longest description
2026-10-22 13:10:25 +01:00 · 2026-10-22 09:56:01 +01:00 · 2026-10-21 16:22:45 +01:00 · 2026-10-21 16:07:21 +01:00 · 2026-10-21 15:39:36 +01:00 · 2026-10-21 15:31:35 +01:00
302 changed files with 145308 additions and 796 deletions
@@ -38,6 +38,9 @@ build/
 # Release directory
 release/
-# Generated code directories
+mock-data/
-internal/config/util/
+
-!internal/config/util/util.go
+# The frontend's build output, copied by npm run sync in web/ for embedding.
 # .gitkeep is kept, as Go can't embed a missing directory.
 internal/frontend/dist/*
 !internal/frontend/dist/.gitkeep
@@ -9,17 +9,138 @@
      "cyclop",
      "dogsled",
      "errcheck",
      "errorlint",
      "gocritic",
      "godox",
      "govet",
      "gosec",
      "govet",
      "ineffassign",
      "misspell",
      "nilerr",
      "nilnil",
      "nolintlint",
      "revive",
      "sloglint",
      "thelper",
      "unused",
-      "whitespace"
+      "varnamelen",
      "whitespace",
      "wsl_v5"
    ],
    "settings": {
      "cyclop": {
        "max-complexity": 15
      },
      "misspell": {
        "locale": "UK",
        "ignore-rules": [
          "organization",
          "humanize"
        ]
      },
      "sloglint": {
        "no-mixed-args": true,
        "key-naming-case": "snake",
        "static-msg": true
      },
      "revive": {
        "rules": [
          {
            "name": "context-as-argument",
            "arguments": [
              {
                "allowTypesBefore": "*testing.T"
              }
            ]
          },
          {
            "name": "error-naming"
          },
          {
            "name": "error-strings"
          },
          {
            "name": "errorf"
          },
          {
            "name": "exported"
          },
          {
            "name": "indent-error-flow"
          },
          {
            "name": "receiver-naming"
          },
          {
            "name": "superfluous-else"
          },
          {
            "name": "var-naming"
          }
        ]
      },
      "varnamelen": {
        "ignore-names": [
          "err",
          "ctx",
          "i",
          "idx",
          "msg",
          "w",
          "r",
          "req",
          "res",
          "db",
          "tx",
          "wg",
          "mu",
          "t",
          "ok",
          "id",
          "ip"
        ],
        "ignore-decls": []
      },
      "gocritic": {
        "disable-all": true,
        "enabled-checks": [
          "builtinShadow",
          "emptyStringTest",
          "paramTypeCombine"
        ]
      },
      "wsl_v5": {
        "default": "none",
        "enable": [
          "err"
        ]
      }
    },
    "exclusions": {
      "rules": [
        {
          "path": "_test\\.go",
          "linters": [
            "errcheck",
            "cyclop"
          ]
        },
        {
          "path": "_test\\.go",
          "linters": [
            "gosec"
          ],
          "text": "G104"
        },
        {
          "path": "_test\\.go",
          "linters": [
            "varnamelen"
          ]
        }
      ]
    }
  },
  "formatters": {
    "enable": [
      "gci",
@@ -28,6 +149,18 @@
      "golines"
    ],
    "settings": {
      "gci": {
        "sections": [
          "standard",
          "default",
          "localmodule"
        ]
      },
      "goimports": {
        "local-prefixes": [
          "git.avsdev.uk/Go/job-search"
        ]
      },
      "golines": {
        "max-len": 100,
        "tab-len": 2
@@ -0,0 +1,818 @@
 # Code conventions
 These are the conventions for Go code in this repository. Most formatting
 is enforced by `golangci-lint` (see [Tooling](#tooling)); the rest are
 applied by hand and checked in review.
 Generated code is never edited to follow these conventions. It is
 recognised by Go's standard marker, a comment before the package clause:
 ```go
 // Code generated by ocdsgen from extensions.json. DO NOT EDIT.
 ```
 ## Tooling
 ```bash
 golangci-lint run --build-tags sqlite_fts5 ./...   # lint
 golangci-lint fmt ./...                            # format (gofmt, goimports, gci, golines)
 ```
 - Anything that touches the store needs the `sqlite_fts5` build tag (the
  title search uses SQLite FTS5); `scripts/build.sh` passes it.
 - `golines` limits lines to 100 characters, counting a tab as 2. See
  [Line length](#line-length) for the length to aim for.
 - In `_test.go` files, `.golangci.json` turns off `errcheck` and
  `cyclop`, and gosec's G104 only (it duplicates errcheck). Every other
  linter still runs on tests.
 ### Linters that check these conventions
 Several of the conventions below are checked by linters enabled in
 `.golangci.json`, so `golangci-lint run` reports them rather than review:
 | Linter | Checks | Settings |
 |---|---|---|
 | `revive` | doc comments on exported names ([Doc comments](#doc-comments)), consistent receiver names, `ctx` first ([Functions and types](#functions-and-types)), [initialisms](#initialisms), lowercase error strings and `Err…` names ([Error messages](#error-messages)), [no `else` after a `return`](#no-else-after-a-return) | Only these rules are enabled. `context-as-argument` allows a `*testing.T` before `ctx`, for test helpers. |
 | `errorlint` | errors are compared with `errors.Is` and `errors.As`, not `==` or a type assertion, so wrapped errors still match | |
 | `misspell` | British spelling ([Spelling](#spelling)) in comments, strings and names | `locale: UK`. `organization` and `humanize` are ignored: they are external names (OCDS's `Organization`, the `go-humanize` package). |
 | `sloglint` | snake_case keys, no mixing of key/value pairs with `slog.Attr`, constant messages ([Logging](#logging)); `slog.DiscardHandler` for discarded output | |
 | `thelper` | test helpers call `t.Helper()` first ([Tests](#tests)) | |
 | `gocritic` | neighbouring parameters and results of the same type share one type name, and strings are tested against `""` ([Functions and types](#functions-and-types)); no names shadowing built-ins ([Naming](#naming)) | Only `paramTypeCombine`, `emptyStringTest` and `builtinShadow` are enabled. |
 | `wsl_v5` | no blank line between a call and the `if err != nil` that checks it ([Blank lines](#blank-lines)) | Only the `err` check is enabled; the rest of wsl's rules conflict with the Blank lines convention. |
 | `varnamelen` | short names used over a wide scope ([Naming](#naming)) | Off in `_test.go` files. The short names in the Naming table are allowed. |
 A finding that is a false positive is silenced with a `//nolint`
 directive and a reason, as for any linter (see
 [Other `//nolint` directives](#other-nolint-directives)).
 ## Files and packages
 - File names are lowercase. Compound names run together (`findatender`,
  `ftsmock`, `ratelimits`); underscores separate parts
  (`conf_source_ftsmock.go`, `confirm_action.go`).
 - Never end a file name in `_<GOOS>` or `_<GOARCH>` (e.g. `_js`, `_arm`):
  Go treats it as a build constraint.
 - Package directories are lowercase, with no underscores.
 ## Imports
 Three groups, each separated by a blank line: the standard library, then
 third-party packages, then this module's own packages.
 ```go
 import (
 	"context"
 	"net/http"
 	"github.com/go-chi/chi/v5"
 	"git.avsdev.uk/Go/job-search/internal/config"
 )
 ```
 `golangci-lint fmt` applies this (gci sections `standard`, `default`,
 `localmodule`; goimports' local prefix is the module path).
 ## Naming
 Variables and parameters in application code have descriptive full-word
 names, not single letters or abbreviations.
 | Use | Not |
 |---|---|
 | `listen`, `newConn`, `client`, `limiter` | `ln`, `nc`, `c`, `lim` |
 | `shutdownCtx`, `writeCtx` | `shCtx`, `wctx` |
 | `release`, `opportunity`, `entry`, `value` | `r`, `t`, `e`, `v` |
 These short names are conventional enough to keep:
 | Name | For |
 |---|---|
 | a short method receiver | e.g. `h *Hub`, `t *tcpConn`, `s *Store` |
 | `err`, `ctx` | errors and contexts |
 | `i`, `idx` | a plain index loop |
 | `msg` | a message |
 | `w`, `r` | `http.ResponseWriter` and `*http.Request` in HTTP handlers and middleware, as in the standard library |
 | `req`, `res` | a request and a response elsewhere, e.g. building an `*http.Request` or receiving an `*http.Response` in client code |
 | `db`, `tx` | `*sql.DB` and a database transaction (`*sql.Tx`, `*sqlx.Tx`) |
 | `wg`, `mu` | `sync.WaitGroup` and `sync.Mutex` |
 | `t` | `*testing.T`, including in test helper packages such as `internal/cli/cmdtest` |
 `w` and `r` are used for nothing else: a release is `release`, an
 `io.Writer` is `out`, a `*rand.Rand` is `rng`.
 Names never shadow Go's predeclared identifiers: not `min`, `max`,
 `len`, `cap`, `new`, `copy`, `clear`, `close`, `error`, `string` and so
 on. A gap's bounds are `minGap` and `maxGap`, not `min` and `max`, so the
 built-in `min` and `max` still work inside the function.
 `tx` means only a database transaction. It is not used for transmit (or
 `rx` for receive), as in tx/rx: name those `transmit` and `receive`, or
 more specifically (e.g. `sent`, `received`).
 Test files are exempt from this rule: short names (`h`, `c`, `e`, `ln`,
 `tc`) are fine there, and existing ones shouldn't be renamed.
 ### Initialisms
 Initialisms and acronyms keep one case throughout a name, as in the
 standard library: `ID`, `URL`, `URI`, `JSON`, `HTTP`, `API`, `UUID`,
 `SQL`. So `ReleaseID`, `APIRoot`, `UUIDType` and `baseURL`, not
 `ReleaseId`, `ApiRoot` or `baseUrl`. At the start of an unexported name
 the whole initialism is lowercase: `uuid`, `apiRoot`, `jsonHandler`.
 `revive` checks this (`var-naming`).
 ### `must` functions
 A name starting with `must` (or `Must`) promises that the function never
 returns an error to its caller, as in the standard library's
 `regexp.MustCompile` and `template.Must`:
 - In tests, a `must…` helper takes `t` first, calls `t.Helper()`, and
  stops the test with `t.Fatal` on error: `mustSave(t, s, opportunity)`.
 - In application code, a `must…` function panics on error, and is only
  used where the input is known to be valid, such as constants or
  package-level setup.
 - A function that hides an error some other way, such as returning a zero
  value or `nil`, isn't named `must…`: `rawJSON` returns `nil` for a value
  it can't marshal.
 ## Functions and types
 - A function that takes a context takes it first, named `ctx`:
  `func (s *Store) Get(ctx context.Context, id string)`.
 - Constructors are `New` when a package has one main type
  (`tcpserver.New`), otherwise `NewX` (`NewHub`, `NewKeyedLimiter`).
 - All of a type's methods use the same receiver name, usually its first
  letter (`h *Hub`, `t *tcpConn`), for value and pointer receivers alike.
 - A string is tested for emptiness against `""`, not with `len`:
  `if file == ""`, not `if len(file) == 0`. It says what is meant, and
  reads the same as any other string comparison.
 - Neighbouring parameters or results of the same type share one type
  name, as in the standard library: `func f(ocid, id string)`, not
  `func f(ocid string, id string)`. Parameters are never reordered just
  to share a type.
 ### No `else` after a `return`
 When an `if` block ends in a `return` (or `break`, `continue`, `panic`),
 there is no `else`: the code that would be in it follows the `if`,
 un-indented. The usual path then reads straight down the left edge of
 the function, with the early exits indented, which is how Go code is
 normally read (see Effective Go and the Go Code Review Comments, "Indent
 Error Flow").
 ```go
 if blockShutdown {
 	return sleep(taskCtx, true, sleepDuration)
 }
 return sleep(ctx, false, sleepDuration)
 ```
 Not:
 ```go
 if blockShutdown {
 	return sleep(taskCtx, true, sleepDuration)
 } else {
 	return sleep(ctx, false, sleepDuration)
 }
 ```
 Where there are several outcomes, none of which is the usual path, a
 `switch` with no condition reads better than nested `if`/`else`:
 ```go
 switch {
 case errors.As(err, &notFound):
 	slog.Warn("no config file found; using defaults")
 case err != nil:
 	return err
 default:
 	slog.Info("using config file", "file", viper.ConfigFileUsed())
 }
 ```
 `revive` checks this (`indent-error-flow`, `superfluous-else`).
 ## Layout
 These rules are applied by hand and checked in review; `golines` only
 enforces the 100-character limit. They apply to tests as well as
 application code, except where a rule says otherwise.
 ### Line length
 Lines should be at most 80 characters (counting a tab as 2), so code
 reads well side by side and in diffs. 100 is the absolute maximum,
 enforced by `golines`.
 Go past 80 only when breaking the line would make it harder to read:
 - a line ending in a `//nolint` directive, which must stay on the line it
  applies to;
 - struct tags;
 - string literals that would lose meaning or searchability if split:
  URLs, error messages, regular expressions, JSON fixtures, help text;
 - rows of table-driven tests.
 Don't run `golines` at 80 to do this: it splits method chains, separates
 slog keys from their values and moves `//nolint` directives off their
 lines.
 ### Assignments in `if` and `for`
 When an `if` statement's assignment and check don't fit within the line
 length together, the assignment is taken out of the `if`, rather than the
 call being wrapped inside it. That keeps the condition readable,
 particularly an `if err != nil`, which would otherwise be left on the
 call's closing line, after its arguments:
 ```go
 _, err = tx.ExecContext(
 	ctx,
 	deleteOpportunityClassifications,
 	opportunity.UUID,
 )
 if err != nil {
 	return err
 }
 ```
 Not:
 ```go
 if _, err := tx.ExecContext(
 	ctx,
 	deleteOpportunityClassifications,
 	opportunity.UUID,
 ); err != nil {
 	return err
 }
 ```
 The same applies to a `for` statement's init statement, and to a range
 expression that doesn't fit on the `for` line: it is assigned to a
 variable first, so the loop's header stays on one line, next to its body.
 ```go
 marks := []struct {
 	filter MarkFilter
 	clause string
 }{
 	{filter: filter.Hidden, clause: filterOpportunityMarkHidden},
 	{filter: filter.Skipped, clause: filterOpportunityMarkSkipped},
 }
 for _, mark := range marks {
 ```
 Tests are exempt from this rule.
 ### Wrapped calls and declarations
 When a call doesn't fit on one line, it breaks straight after the opening
 bracket and the closing bracket goes on a line of its own. If the
 arguments all fit on one line between them, they may stay together on
 that line, followed by a comma:
 ```go
 ctx, stop := signal.NotifyContext(
 	context.Background(), os.Interrupt, syscall.SIGTERM,
 )
 ```
 Otherwise each argument goes on its own line, followed by a comma. The
 arguments are either all on one line or one per line, never grouped two
 or three to a line:
 ```go
 server := tcpserver.New(
 	&conf.Server,
 	websocket.NewHub(conf.Server.WebSocket, websocket.WithLogger(log)),
 	tcpserver.WithLogger(log),
 )
 ```
 Not:
 ```go
 server := tcpserver.New(&conf.Server,
 	websocket.NewHub(conf.Server.WebSocket, websocket.WithLogger(log)),
 	tcpserver.WithLogger(log))
 ```
 A function declaration whose parameters don't fit on one line wraps the
 same way:
 ```go
 func insertReleaseWithTags(
 	ctx context.Context, tx *sqlx.Tx, release types.OpportunityRelease,
 ) (bool, error) {
 ```
 A `log/slog` call that fits on one line stays on one line, like any other
 call. When it doesn't, its attributes are the exception to the rule above:
 the message goes on its own line, then each field name stays on the same
 line as its value, one field per line, even when the fields would all fit
 together on one line.
 ```go
 slog.Info(
 	"fetched opportunities",
 	"source", "find_a_tender",
 	"count", len(opportunities),
 	"since", since,
 )
 ```
 A `//nolint` directive on a wrapped call goes on its first line, after the
 opening bracket (see [Ignoring an error](#ignoring-an-error)).
 ### Method chains
 A chain of calls that fits on one line stays on one line. One that
 doesn't breaks after each `.`, one call per line, indented once, with
 the dot at the end of the line:
 ```go
 findATender.httpClient = resty.New().
 	SetTimeout(60 * time.Second).
 	SetRetryCount(1)
 ```
 A short accessor at the start of a chain, such as `h.logger()` or
 `cmd.Flags()`, stays with the call after it; if that call doesn't fit,
 its arguments are wrapped instead (see
 [Wrapped calls and declarations](#wrapped-calls-and-declarations)):
 ```go
 h.logger().Info(
 	"client connected",
 	"client", client.id,
 	"ip", ip,
 )
 ```
 Not:
 ```go
 h.logger().
 	Info("client connected", "client", client.id, "ip", ip)
 ```
 ### Long conditions
 A condition that doesn't fit on one line breaks after a `||` or `&&`,
 which ends the line, and continues indented once. Each line is filled up
 to the line length, keeping related comparisons together, rather than
 one comparison per line:
 ```go
 if len(cat.Buyers) == 0 || len(cat.Suppliers) == 0 ||
 	len(cat.Subjects) == 0 || len(cat.Methods) == 0 {
 ```
 A condition is never broken inside a call's brackets; if a call in it
 doesn't fit, its result is assigned to a variable first (see
 [Assignments in `if` and `for`](#assignments-in-if-and-for)). A
 condition that runs to more than about three lines is better given a
 name, as a boolean variable or a small function.
 ### Blank lines
 A function body is split into steps, with one blank line between them. A
 step is a statement, or a few closely related ones, together with its
 error handling. Go itself has no rule for this (`gofmt` only collapses
 several blank lines into one, and the `whitespace` linter only rejects
 blank lines at the start or end of a block), so it is applied by hand.
 - A call and the `if err != nil` that checks it are never separated by a
  blank line. `wsl_v5` checks this; the rest of the rule is applied
  by hand.
 - A `defer` that releases something goes straight after the check of the
  call that acquired it, with no blank line, so acquiring and releasing
  read as one step.
 - A function or block of only one or two steps needs no blank lines.
 - A final `return` after several (three or more) steps is set off by a
  blank line. The `return` itself isn't counted as a step.
 ```go
 statement, err := tx.PrepareNamedContext(
 	ctx, upsertClassificationCodeReturningID,
 )
 if err != nil {
 	return err
 }
 defer func() {
 	err = errors.Join(err, statement.Close())
 }()
 for i := range opportunity.Classifications {
 	...
 }
 return nil
 ```
 ### Function literals
 A function literal (`func(...) { ... }` written inline) is never written on
 one line, however short its body:
 ```go
 t.Cleanup(func() {
 	server.Close()
 })
 ```
 Not `t.Cleanup(func() { server.Close() })`.
 This is about function literals only. Named one-line functions and methods,
 such as a config section's `Map() map[string]any { return mapOf(c) }`, are
 unaffected.
 ### Struct literals
 A struct literal is always wrapped, one field per line, and names its
 fields:
 ```go
 envelope := websocket.Envelope{
 	Type: websocket.TypePing,
 	ID:   "1",
 }
 ```
 Not `websocket.Envelope{Type: websocket.TypePing, ID: "1"}`, and never
 positional (`websocket.Envelope{"ping", "1", nil, ""}`).
 - Empty struct literals (`netip.Addr{}`, `time.Time{}`, `struct{}{}`,
  `&Server{}`) have nothing to wrap and stay on one line.
 - Rows of a table-driven test (`{"size", "4096", "4KiB"},`) stay one row per
  line, so the table stays easy to scan. A struct literal used as a value
  inside a row is still wrapped.
 - Rows of a short lookup table in application code (a slice or map
  literal of small structs, used as data) may each stay on one line if
  they fit within the line length, so the table is easy to scan. They
  still name their fields:
  `{filter: filter.Hidden, clause: filterOpportunityMarkHidden},`.
 - This is about struct literals only; slice and map literals
  (`[]string{"a", "b"}`) may stay on one line.
 ## Comments
 ### Doc comments
 Every exported identifier has a doc comment, and it starts with the name
 it documents.
 ```go
 // NewHub creates a hub.
 func NewHub(conf config.JSCServerWebSocket, opts ...Option) *Hub {
 ```
 ### Grouped declarations
 Related package-level constants and variables go in one block, with a
 comment for the group and one for each member.
 ```go
 // Errors returned by CSRF.Validate for a token that is rejected.
 var (
 	// ErrCSRFMissing is returned when no token is presented.
 	ErrCSRFMissing = errors.New("csrf token missing")
 	// ErrCSRFExpired is returned for a token older than the token TTL.
 	ErrCSRFExpired = errors.New("csrf token expired")
 )
 ```
 ### Section banners
 A file is divided into sections with a banner, followed by a blank line
 so godoc doesn't attach the banner to the next declaration:
 ```go
 // ---------- tcpConn ----------
 // tcpConn adapts a net.Conn ...
 type tcpConn struct {
 ```
 In a file with more than one type (such as a config section and its
 nested section), each type gets a `==========` sub-section, which then
 has its own `----------` banners:
 ```go
 // ========== HSTS Headers ==========
 // JSCServerHeadersHSTS configures the Strict-Transport-Security header.
 type JSCServerHeadersHSTS struct {
 	...
 }
 // ---------- Section methods (see Section) ----------
 ```
 `internal/store/queries.go` uses `----------` banners for its per-table
 groups of queries.
 ### Trailing comments
 A comment after code on the same line is for a short note, and only when
 the line still fits within the [line length](#line-length). Otherwise
 the comment goes on its own line above the code it describes:
 ```go
 	// location is the zone for updatedFrom/updatedTo without an offset;
 	// nil means UTC.
 	location *time.Location
 ```
 Not a trailing comment that runs the line past 80, and never a comment
 after the closing bracket of a wrapped call.
 A comment on a row of a table-driven test stays on the row, like the row
 itself (see [Line length](#line-length)).
 `//nolint` directives are exempt: a directive must be on the line the
 linter reports, so it stays a trailing comment even when that takes the
 line past 80 (see [Ignoring an error](#ignoring-an-error)).
 ### Spelling
 Comments and documentation use British English ("normalised", "honoured",
 "recognise", "behaviour"). Identifiers, and anything that must match an
 external name, keep their own spelling (`Authorization`, `Color` in a
 third-party API).
 ## Errors
 ### Error messages
 - Error strings are lowercase and don't end with punctuation, as they are
  usually wrapped into a longer message (`revive`'s `error-strings`
  reports capitalised ones): `errors.New("store: not found")`.
 - An error from another call is wrapped with `%w`, adding what was being
  done, so the chain reads as a sentence:
  `fmt.Errorf("listen %s: %w", addr, err)`.
 ### Package prefixes
 An error names the package it came from once, where it is created, as
 the standard library does (`sql: no rows in result set`, `http: server
 closed`) and Effective Go recommends:
 - An error a package creates itself, a sentinel, `errors.New`, or
  `fmt.Errorf` without `%w`, starts with the package name:
  `errors.New("store: not found")`,
  `fmt.Errorf("util: byte size %q is too large", text)`.
 - Wrapping an error adds what was being done, never a package name:
  `fmt.Errorf("inserting release %s/%s: %w", uuid, id, err)`, not
  `"store: inserting release …"`. The wrapped error already says where it
  came from, so a chain reads `writing opportunity ocds-1: store: …`
  rather than repeating `store:` at every level.
 Two kinds of error are exempt and have no prefix:
 - Errors from `package main` and the command packages in `internal/cli`.
  Their text is the final message shown to the user, who has no use for
  an internal package name.
 - Error text that is part of a protocol sent to a client, such as the
  mock Find a Tender API's error responses, which match the real API's.
 ### Sentinel errors
 Errors that callers test for are sentinel values named `Err…` (exported)
 or `err…` (unexported), checked with `errors.Is`. The sentinels in a file
 go in one commented `var` block (see
 [Grouped declarations](#grouped-declarations)).
 They are declared near the top of the file: straight after the imports
 and any constants, before the file's types and functions. A reader then
 sees what the file can fail with before reading how, and finds a
 sentinel without searching the file. This applies to every package-level
 error variable, in tests as well.
 ```go
 import (
 	"errors"
 	"time"
 )
 // ErrWonAndLost is returned by OpportunityMarks.Validate when both Won and
 // Lost are set.
 var ErrWonAndLost = errors.New("types: an opportunity cannot be both won and lost")
 // Opportunity is a procurement opportunity ...
 type Opportunity struct {
 ```
 Not declared next to the type or function that returns it, partway down
 the file.
 ### Ignoring an error
 An error is never ignored with `_ = f()`. Instead, the call says why
 with a `//nolint` directive naming only the linters that actually report
 the line (`nolintlint` reports any that don't):
 ```go
 json.NewEncoder(w).Encode(problem) //nolint:errcheck,gosec // the client may have gone
 ```
 - Most calls need `errcheck,gosec`. `hash.Write` needs only `gosec`
  (errcheck excludes it); `fmt.Fprint*` needs only `errcheck` (gosec
  excludes it).
 - A deferred call needs only `errcheck`, since gosec doesn't check
  deferred calls:
  ```go
  defer rows.Close() //nolint:errcheck // read only
  ```
 - For a call spanning several lines, the directive goes on the first
  line, after the opening parenthesis:
  ```go
  tcpSocket.SetKeepAliveConfig(net.KeepAliveConfig{ //nolint:errcheck,gosec // best effort
  ```
  A linter that reports one argument rather than the call (such as
  `sloglint` on a message that isn't a constant) needs the directive on
  that argument's line instead. One that reports several lines of a
  statement, such as a table of test cases, takes the directive on a line
  of its own directly above the statement.
 - A call whose error both linters already exclude (such as
  `crypto/rand.Read`, which never returns one) gets a plain comment
  rather than a directive, which would be reported as unused:
  ```go
  rand.Read(key) // crypto/rand.Read never returns an error
  ```
 - A multi-value assignment such as `value, _ := f()` can't take a
  directive (errcheck doesn't check blank assignments, so the directive
  would be unused) and stays as it is. These are mostly cobra flag
  getters, which only fail for an undefined flag. Where ignoring the
  error is ambiguous or potentially dangerous in future, an explanation
  comment can be added.
 ### Errors that matter
 When closing something can lose data (a file being written, a database
 handle), its error is joined into the function's returned error with
 `errors.Join`, through a named return.
 ### Named results
 A function names its results only when it needs to:
 - **A deferred function changes the error.** This is the only reason to
  name `err` for its own sake (see [Errors that matter](#errors-that-matter)).
 - **The names say which result is which.** When several results share a
  type, naming them documents the order, as the Go Code Review Comments
  suggest: `func releaseIDs(raw json.RawMessage) (ocid, id string, err error)`.
  Go names all of a function's results or none, so `err` is named here too.
 Otherwise results are unnamed. A function with named results:
 - never uses a bare `return`: each `return` lists what it returns, so a
  reader doesn't have to track the named variables to know;
 - where a deferred function changes `err`, assigns to it with `=`, never
  `:=`, in nested blocks. `err :=` there declares a new variable that
  shadows the result, and the error assigned to it never reaches the
  deferred `errors.Join`:
  ```go
  for i := range opportunity.Classifications {
  	err = statement.GetContext(ctx, &classification.ID, classification)
  	if err != nil {
  		return err
  	}
  }
  ```
 ### Other `//nolint` directives
 Otherwise `//nolint` is only for false positives that the code can't
 express, always with a reason. For example, a seeded `math/rand` in the
 mock, or a gosec finding explained in the comment above it:
 ```go
 // Secure is false only on plain HTTP (local development), where
 // browsers refuse Secure cookies; HttpOnly and SameSite are always set.
 http.SetCookie(w, &http.Cookie{ //nolint:gosec // G124: see above
 ```
 ## Logging
 - Logging uses `log/slog`. A component (a type such as `Hub`, or a
  package-level entry point such as `fetcher.Run`) logs to `slog.Default()`
  unless it is given a logger with a `WithLogger` option, so callers that
  are happy with the default pass nothing, and tests can still capture or
  discard its output.
 - The logger is given to the constructor (or entry point) as an option,
  never set afterwards: a setter called once goroutines are running would
  be a data race. Options follow the functional options pattern used by
  `store.Open`.
 - With no logger given, the component looks up `slog.Default()` when it
  logs, not when it is created, so it follows `logging.Setup` and a test's
  `slog.SetDefault` even if it was created first.
  ```go
  // WithLogger makes the hub log to logger instead of slog.Default().
  func WithLogger(logger *slog.Logger) Option {
  	return func(h *Hub) {
  		h.log = logger
  	}
  }
  // logger returns the logger the hub logs to, tagged with its component.
  func (h *Hub) logger() *slog.Logger {
  	log := h.log
  	if log == nil {
  		log = slog.Default()
  	}
  	return log.With("component", "ws")
  }
  ```
  ```go
  hub := websocket.NewHub(conf.Server.WebSocket)                            // slog.Default()
  hub := websocket.NewHub(conf.Server.WebSocket, websocket.WithLogger(log)) // given logger
  ```
 - Field keys are snake_case and name one thing each: `client`, `ip`,
  `request_id`, `retry_in`. Durations, sizes and counts are logged with
  the matching `slog` type, not as pre-formatted strings.
 - A component tags its logger with `"component"` in one place (its
  `logger()` method, as above), whether the logger was given or is the
  default, rather than on every line.
 - Messages are short lowercase phrases ("client connected", "tcp accept
  failed"); the details go in fields.
 ## Configuration
 ### Keys
 Configuration keys (the `cfg` struct tags) are snake_case:
 `max_message_size`, `connection_limit`. Viper lowercases every key it
 reads, so a camelCase key such as `minVersion` is never loaded from a
 config file or environment variable.
 ### Zero values
 Every setting's doc comment (and its row in `internal/config/README.md`)
 says what 0 or empty means, and the code treats it consistently:
 - A timeout of 0 or less means **no limit**, and an interval of 0 or less
  means **off** (e.g. `ping_interval`), never "already expired". Where off
  makes no sense, or no limit is unsafe, the setting rejects 0 instead (a
  source's `interval`: a source is turned off with `enabled`;
  `server.timeouts.read_header`: a client could hold a connection open for
  ever). Code applies a timeout with a helper rather than
  `context.WithTimeout(ctx, 0)`:
  ```go
  writeCtx, cancel := util.WithTimeout(ctx, conf.WriteTimeout) // no deadline if 0 or less
  ```
  Where a library reads 0 as some other timeout (`http.Server`'s
  `IdleTimeout` uses `ReadTimeout`), the code passes it a negative timeout
  instead, which such libraries read as no limit.
 - A size of 0 means **the default** (`max_message_size` of `0B` reads
  32KiB), and -1 means **no limit** (`util.ByteSizeNoLimit`); no other size
  may be negative. A setting for which no limit is unsafe rejects -1 in its
  validation and says why (`max_message_size` does: a client could exhaust
  the server's memory).
 - An empty list means **none** (no trusted origins, no exempt paths), and
  is stored as `nil`.
 ## Tests
 - Tests are table-driven where several cases share a check, with a name
  or input field identifying each case in failure messages.
 - Helpers call `t.Helper()` first, and register their own teardown with
  `t.Cleanup` rather than returning a cleanup function.
 - Tests that touch global state (Viper, `slog.Default`, the working
  directory, environment variables) don't call `t.Parallel()`, and reset
  what they change: `resetViper(t)`, `t.Chdir`, `t.Setenv`.
 - Network tests listen on `127.0.0.1:0` and use the address they were
  given, never a fixed port.
 - Short names are fine in tests (see [Naming](#naming)), and setup calls
  may ignore errors with a bare call (errcheck is off for tests).
@@ -2,3 +2,346 @@
 JobSearch is a service for scraping for contract opportunities. It is primarily
 a re-write of the existing nodeJS JobSearch codebase.
 It currently fetches releases from the UK Government
 [Find a Tender](https://www.find-tender.service.gov.uk/) service (FTS), which
 publishes them in the [OCDS](https://standard.open-contracting.org/) format.
 A mock of the FTS API is included for development.
 ## Building
 ```bash
 scripts/build.sh                    # build/job-search-<version>
 scripts/build.sh --release          # also package release/job-search-<version>-<arch>.tar.gz
 ```
 The build script builds the frontend and copies it into place for embedding
 (`npm run build` and `npm run sync` in `web/`, which need Node and npm; it
 runs `npm ci` first for a release, or when `web/node_modules` is missing). It
 then runs `go generate ./...`, which needs `ocdsgen` and `swag` installed
 (`scripts/install-dev-tools.sh`; see
 [Installation of dev tools](#installation-of-dev-tools)), and stamps the
 version from the latest git tag. A build from untagged commits is marked as a development build
 and isn't packaged unless `--force` is also given.
 The frontend (`web/`) is embedded in the binary. Without the build script,
 build it and copy it into place first, or the binary serves no frontend
 (`job-search serve` warns):
 ```bash
 cd web && npm ci && npm run build && npm run sync   # web/dist -> internal/frontend/dist
 ```
 Setting `server.frontend.dir` serves a build from disk instead.
 During development, `go run -tags sqlite_fts5 ./cmd/jobsearch` works as well.
 The `sqlite_fts5` tag is needed for the database's title search (see
 [Testing](#testing)).
 ## Running as a service
 `deploy/job-search.service` is a systemd unit running `job-search serve`
 (the server and every enabled source's fetcher) as a `job-search` system
 user, from `/usr/local/bin/job-search` with `/etc/job-search/config.yaml`.
 Its comments give the install steps. It keeps its data in
 `/var/lib/job-search` and any log file in `/var/log/job-search`, the only
 places it can write, so set `database.file` there. Logs go to the journal
 (`journalctl -u job-search`) unless `logging.file` is set to a file in
 `/var/log/job-search`, which then needs rotating (below).
 - The database isn't migrated on start: run `job-search migrate up`
  (`--create` the first time) as the `job-search` user before the first
  start and after each upgrade.
 - A failing source stops `serve` on purpose; the unit restarts it after
  30s.
 ### Rotating the logs
 `job-search` opens its log file once and has no signal to reopen it, so
 logrotate must copy the file and truncate it in place (`copytruncate`); a
 line written between the two is lost. `deploy/job-search.logrotate` does
 this weekly for every `*.log` in `/var/log/job-search` (the log and, if
 `server.access_log` is set there, the access log), keeping 8 weeks,
 compressed. `/var/log/job-search` belongs to the service user, so it rotates
 as that user (`su job-search job-search`); logrotate refuses otherwise.
 ```bash
 # from the release tarball's directory (deploy/job-search.logrotate in the repo)
 sudo install -m 644 job-search.logrotate /etc/logrotate.d/job-search
 sudo logrotate --debug /etc/logrotate.d/job-search   # dry run: what it would do
 sudo logrotate --force /etc/logrotate.d/job-search   # rotate now, to check
 ```
 ## Configuration
 Configuration is read from, in increasing priority:
 1. Built-in defaults
 2. `config.yaml` in `/etc/job-search` or the working directory, or the file
   given with `--config`
 3. `JOBSEARCH_*` environment variables, e.g. `JOBSEARCH_LOGGING_LEVEL=debug`
 A `.env` file in the working directory, if there is one, is loaded into the
 environment first.
 Example `config.yaml`:
 ```yaml
 logging:
    level: info
    file: /var/log/job-search.log
 database:
    file: /var/lib/job-search/jobsearch.db
 sources:
    find_a_tender:
        enabled: true
        interval: 10m
 ```
 See [internal/config](internal/config/README.md) for every key and how the
 configuration code works.
 ## Usage
 Global flags: `--config`/`-c` (config file), `--debug`/`-d` and
 `--quiet`/`-q` (errors only).
 | Command | Purpose |
 |---|---|
 | `job-search config init [path]` | Create a config file holding the defaults (`--all` also writes the test sources, `fts_mock` and `sleeper`) |
 | `job-search config show` | Print the configuration (`--json` or `--yaml` for those formats) |
 | `job-search config get <key>` | Print one value, e.g. `logging.level` |
 | `job-search config set <key> <value>` | Validate and write one value to the config file |
 | `job-search config set --unset <key>` | Remove a value from the config file |
 | `job-search config generate-csrf-key` | Store a new random `server.csrf.tokens.key` in the config file, so CSRF tokens survive restarts and work across instances (`--print` only prints one; `--force` replaces an existing key) |
 | `job-search migrate` | Apply every pending database migration (same as `migrate up`) |
 | `job-search migrate up` | Apply pending migrations (`--steps N` for only N; `--create` to create a missing database file without asking) |
 | `job-search migrate down --steps N` | Roll back the last N migrations (`--steps 0` rolls back all of them, deleting all data, and needs `--yes`) |
 | `job-search migrate status` | Show the database version and which migrations are applied or pending |
 | `job-search skip-rules check [file]` | Check a skip rules file (default: `sources.skip_rules_file`) |
 | `job-search skip-rules apply [file]` | Mark the stored opportunities that match the skip rules Skipped, listing them (`--dry-run` only lists them) |
 | `job-search version` | Print version information |
 `job-search serve` describes its REST API in OpenAPI 3.1 at
 `/api/openapi.json`, and shows it in Swagger UI at `/api/docs`. The
 description is generated from the handlers' annotations in `internal/api` by
 `go generate`.
 The `migrate` commands work on the database in `database.file`. The store
 refuses to open a database that isn't at this build's schema version, so run
 `job-search migrate` after installing a new version.
 ### Skip rules
 `sources.skip_rules_file` names a JSON file of rules that mark new
 opportunities Skipped as `fetcher service` and `job-search serve` fetch them,
 by buyer, title words or CPV codes. The file is read again when it changes;
 an invalid edit is logged and the last good rules kept. Opportunities already
 stored are only changed by `job-search skip-rules apply`. See
 [internal/skiprules](internal/skiprules/README.md) for the format.
 ## Fetch tool
 `fetcher` reads the same configuration as `job-search`:
 | Command | Purpose |
 |---|---|
 | `fetcher service` | Poll every enabled source on its interval until stopped (Ctrl+C or SIGTERM) |
 | `fetcher fts-once` | Fetch from Find a Tender once and print the releases as JSON |
 `service` logs to `logging.file` if it's set, or to the terminal with
 `--foreground`. Both commands take `--since "YYYY-MM-DD[ hh:mm:ss]"`.
 Without it, `service` (and `job-search serve`) starts each source from the
 newest release stored for it, so a restart carries on where it left off; with
 nothing stored, Find a Tender's client fetches the last day. `--since`
 overrides that for every source.
 When the start is more than 7 days ago (an early `--since`, or a long time
 offline), the Find a Tender and EU Supply sources first backfill it in 7-day
 windows,
 oldest first, saving each before fetching the next. A window that fails is
 saved as far as it got and fetched again after the source's interval. The
 API returns the newest releases first, so if a backfill is stopped partway
 through a window, restarting without `--since` skips the rest of that
 window: restart with `--since` set to the window's start (logged as
 `backfilling`) to fill it.
 A rate limit is waited out, as the API asks, up to
 `sources.<source>.rate_limit_retries` times per request. A failed fetch
 keeps what it fetched before the failure.
 ### EU Supply
 The `eu_supply` source scrapes the public tenders on EU Supply's UK portal
 (`uk.eu-supply.com`), stored with the source `EU Supply - UK`. The site has
 no API and publishes no rate limit, so the client waits a random time from
 `request_min_gap` to `request_max_gap` (2s to 10s) between its requests. Each
 tender not stored yet has its own page read too, for its description and
 CPV codes. Most of its tenders are also published on Find a Tender: a new
 one is stored Hidden and Skipped when another source's opportunity released
 within 30 days of it has a title at least 90% alike, and its buyer or its
 description matches too (80% of the shorter one's words). See
 [internal/eusupply](internal/eusupply/README.md).
 `fts-once` can be limited with `--planning`, `--opportunities` or
 `--awards`, and with `--ignore-rate-limit` it fails on a rate limit instead
 of waiting.
 During development, `go run ./cmd/fetcher` works as well.
 ## Mock Find a Tender server
 To develop without using the real API, run the mock and enable the `fts_mock`
 source:
 ```bash
 go run ./cmd/mock-fts --min-gap 5s --max-gap 10s --rate-limit 1000
 ```
 ```yaml
 sources:
    fts_mock:
        enabled: true
        interval: 1m
        api_root: http://localhost:8080/api/1.0
 ```
 See [internal/mock](internal/mock/README.md) for its options and behaviour.
 ## WebSocket test tool
 `ws-test` reads the same configuration as `job-search` for its defaults:
 ```bash
 go run ./cmd/ws-test dial --http-upgrade --ping  # ping job-search serve's /ws endpoint
 go run ./cmd/ws-test dial --direct --ping        # ping ws-test serve
 go run ./cmd/ws-test dial --http-upgrade         # send each line of stdin, print replies
 go run ./cmd/ws-test serve                       # plain TCP server for websocket hub clients
 ```
 `dial` needs one of `--http-upgrade`, to upgrade an HTTP connection to a
 WebSocket on `/ws` as `job-search serve` expects, or `--direct`, to connect
 straight to `ws-test serve` over plain TCP with one JSON message per line.
 It also takes `--message`, `--listen` and `--timeout`; `--url` and `--origin`
 (to test the Origin check) with `--http-upgrade`, and `--addr` with
 `--direct`. See `ws-test dial --help`.
 `ws-test serve` is a test server only: don't run it alongside
 `job-search serve`. Both listen on `server.host` and `server.port`, so the
 second one fails with "address already in use".
 ## Project layout
 | Path | Contents |
 |---|---|
 | `cmd/jobsearch` | The `job-search` command |
 | `cmd/fetcher` | The `fetcher` command (Find a Tender fetch, the polling service) |
 | `cmd/mock-fts` | The mock FTS server |
 | `cmd/ws-test` | The `ws-test` WebSocket test tool |
 | `internal/cli` | Cobra commands and shared CLI setup |
 | `internal/config` | Configuration loading, validation and editing |
 | `internal/fetcher` | Runs a worker per enabled source |
 | `internal/search` | Parsing of the general opportunity search (`title:`, `buyer:`, `buyer_id:`, `cpv:`, `!`) and its help |
 | `internal/skiprules` | The rules that mark new opportunities Skipped (see [internal/skiprules](internal/skiprules/README.md)) |
 | `internal/store` | SQLite storage of opportunities, and its migrations (see [internal/store](internal/store/README.md)) |
 | `internal/findatender` | FTS API client and conversion from OCDS to JobSearch types |
 | `internal/eusupply` | EU Supply UK portal scraper and conversion to JobSearch types (see [internal/eusupply](internal/eusupply/README.md)) |
 | `internal/types` | JobSearch's own opportunity types |
 | `internal/util` | The periodic `Worker` used by the fetcher |
 | `internal/logging` | slog setup and the HTTP access log |
 | `internal/mock` | Mock external services |
 | `internal/version` | Version information set at build time |
 | `internal/frontend` | Serves the frontend's build output; embeds `dist/`, copied there from `web/dist` by `npm run sync` |
 | `web` | The frontend (see [web](web/README.md)) |
 | `deploy` | The systemd unit and logrotate configuration (see [Running as a service](#running-as-a-service)) |
 ## Testing
 The store's title search needs SQLite's FTS5 module, which is only compiled in
 with the `sqlite_fts5` build tag, so tests (and lint) need it. Without it the
 store tests fail with `no such module: fts5`.
 ```bash
 go test -race -tags sqlite_fts5 ./internal/...
 go test -tags sqlite_fts5 -coverprofile=cover.out ./internal/... && go tool cover -func=cover.out
 golangci-lint run --build-tags sqlite_fts5 ./...
 ```
 Tests don't need network access; HTTP calls go to local `httptest` servers.
 ## Addition of a new opportunities source
 Using a source called `example` as an illustration:
 1. **Configuration section:** create `internal/config/conf_source_example.go`,
   copying an existing source such as `conf_source_ftsmock.go`:
   - Add a `JSCKSourceExample = "example"` constant.
   - Define `JSCSourceExample`, embedding `JSCSourcesSource`
     (for `enabled` and `interval`), plus any fields of its own with `cfg` and
     optional `default` tags, e.g. ``APIRoot string `cfg:"api_root"` ``.
   - Implement the `Section` interface (see `internal/config/section.go`):
     copy the `Map`, `Flatten`, `TypeOf`, `ValueOf` and `SetValue` methods
     from an existing source, under a
     `// ---------- Section methods (see Section) ----------` banner, renaming
     the receiver type and using the new constant in `Flatten`.
 2. **Register the section:** add
   ``Example JSCSourceExample `cfg:"example"` `` to
   `JSCSources` in `internal/config/conf_sources.go`. The `cfg` tag
   must match the constant. The source's keys (`sources.example.*`) are then
   loaded, validated and available to the `config` commands with no further
   code.
 3. **Client package:** create a package that fetches from the source and
   returns `[]*types.Opportunity` (see `internal/types`). If the source
   publishes OCDS, `internal/findatender` may already be suitable, as it is
   for `fts_mock`: create it with the source's own API root.
 4. **Worker:** create `internal/fetcher/worker_example.go`, following
   `worker_ftsmock.go`:
   - Add an `exampleSource = "Example"` constant: the worker's name, and the
     `Source` its opportunities are stored with (set it on each one if the
     client doesn't, as `fetchFTSMock` does).
   - `runExample(ctx, logger, conf, save, since)` builds a `util.Worker`
     with `exampleSource`, `conf.Interval` and a task, then calls `Run`.
   - The task fetches everything since its `since` argument, then returns
     `save(ctx, logger, opportunities)`, which stores them when the fetcher
     has a store. Wrap blocking calls in `util.AwaitTask` so shutdown isn't
     held up, and return `ErrNoNewReleases` when nothing is found, so the
     checkpoint still moves forward. Any other error makes the next run
     retry from the same point.
 5. **Start it:** in `Run` in `internal/fetcher/fetcher.go`, add
   ```go
   if f.conf.Example.Enabled {
       start(exampleSource, func(ctx context.Context, since time.Time) error {
           return runExample(ctx, logger, &f.conf.Example, f.save, since)
       })
   }
   ```
   `start` passes the worker its starting point: `--since`, or the newest
   release stored with `exampleSource`.
 6. **Tests:** `TestSections` checks the new section implements `Section`
   and that its keys match its `cfg` tag, with no changes needed. In
   `internal/config/conf_sections_test.go`, add the section to
   `TestConcreteSourceSections` and update the key count in
   `TestSourcesSection`. Test the client and worker using `httptest` servers
   rather than the real service.
 7. **Docs:** add the source and its keys to the key table in
   `internal/config/README.md`.
 ## Installation of dev tools
 `scripts/install-dev-tools.sh` installs everything needed to develop and
 build the project: the Go tools below, with `go install`, and the frontend's
 npm packages (`npm ci` in `web/`). It needs Go and Node.js 24 or later (with
 npm), as the frontend's scripts run TypeScript directly. Run it again after
 a tool's version in it changes.
 ```bash
 scripts/install-dev-tools.sh
 ```
 | Tool | Version | Used for |
 |---|---|---|
 | `ocdsgen` (`git.avsdev.uk/Go/go-ocds`) | v1.0.0 | Generates the OCDS types in `internal/findatender/ocds` (`ocds_gen.go`) from `extensions.json`, run by `go generate` |
 | `swag` | v2.0.0-rc6 | Generates the API's OpenAPI 3.1 description from the handlers' comments (`swag init --v3.1`), run by `go generate` |
 | `golangci-lint` | latest | Lints the Go code with the project's `.golangci.json` |
 | `goose` | latest | Creates new migration files (see `internal/store`); the migrations themselves are applied by `job-search migrate` |
 | `cobra-cli` | latest | Adds cobra commands |
 The generators are pinned, as another version could generate different code.
 `swag`'s v2, the version with OpenAPI 3 support, has only had release
 candidates so far. `go generate ./...` and the build script run the
 generators from PATH, so ensure that the GOBIN path (`$HOME/go/bin` by
 default) is added to the environment before running any of the tools; the
 script says if it isn't.
@@ -0,0 +1,82 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 // Command fetcher, for fetching opportunities from JobSearch's sources. Run
 // `fetcher --help` for usage.
 package main
 import (
 	"fmt"
 	"log/slog"
 	"os"
 	"strings"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/cli/fetch"
 )
 // newRootCmd builds the command tree. It returns a fresh tree on each call so
 // tests can run commands in isolation.
 func newRootCmd() *cobra.Command {
 	opts := &cli.Options{}
 	root := &cobra.Command{
 		Use:   "fetcher",
 		Short: "Fetch tools for JobSearch",
 		Long: `fetcher fetches from Find a Tender once and prints the releases as
 JSON (fts-once), or polls every enabled source on its interval until stopped
 (service). It reads the JobSearch configuration.
 	`,
 		SilenceUsage:  true, // don't print usage on runtime errors
 		SilenceErrors: true, // main prints the error once
 		CompletionOptions: cobra.CompletionOptions{
 			HiddenDefaultCmd: true,
 		},
 		PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
 			cli.SetupInitialLogging(cmd)
 			slog.Debug("command persistent pre-run", "command", "root")
 			slog.Debug(
 				"command",
 				"path", strings.Join(strings.Split(cmd.CommandPath(), " ")[1:], " "),
 			)
 			return nil
 		},
 		PersistentPostRunE: func(cmd *cobra.Command, args []string) error {
 			slog.Debug("command persistent post-run", "command", "root")
 			return opts.Finish()
 		},
 	}
 	cobra.EnableTraverseRunHooks = true
 	opts.Bind(root)
 	root.AddCommand(fetch.NewCommands(opts)...)
 	return root
 }
 func main() {
 	if err := newRootCmd().Execute(); err != nil {
 		fmt.Fprintln(os.Stderr, "fetcher:", err)
 		os.Exit(1)
 	}
 }
@@ -0,0 +1,30 @@
 package main
 import (
 	"slices"
 	"testing"
 )
 // The commands are tested in internal/cli/fetch. This test checks newRootCmd
 // puts them together.
 func TestCommandTree(t *testing.T) {
 	root := newRootCmd()
 	var names []string
 	for _, c := range root.Commands() {
 		names = append(names, c.Name())
 	}
 	for _, want := range []string{"fts-once", "service"} {
 		if !slices.Contains(names, want) {
 			t.Errorf("fetcher has no %q command; has %v", want, names)
 		}
 	}
 	for _, flag := range []string{"config", "debug", "quiet"} {
 		if root.PersistentFlags().Lookup(flag) == nil {
 			t.Errorf("fetcher has no --%s flag", flag)
 		}
 	}
 }
@@ -0,0 +1,90 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 // Command job-search. Run `job-search --help` for usage.
 package main
 import (
 	"fmt"
 	"log/slog"
 	"os"
 	"strings"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/cli/config"
 	"git.avsdev.uk/Go/job-search/internal/cli/migrate"
 	"git.avsdev.uk/Go/job-search/internal/cli/serve"
 	"git.avsdev.uk/Go/job-search/internal/cli/skiprules"
 	"git.avsdev.uk/Go/job-search/internal/cli/version"
 )
 // newRootCmd builds the command tree. It returns a fresh tree on each call so
 // tests can run commands in isolation.
 func newRootCmd() *cobra.Command {
 	opts := &cli.Options{}
 	root := &cobra.Command{
 		Use:   "job-search",
 		Short: "JobSearch - AVSDev Future Opportunities!",
 		Long: `JobSearch is a scraping tool for searching for possible contract
 opportunities.
 	`,
 		SilenceUsage:  true, // don't print usage on runtime errors
 		SilenceErrors: true, // main prints the error once
 		CompletionOptions: cobra.CompletionOptions{
 			HiddenDefaultCmd: true,
 		},
 		PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
 			cli.SetupInitialLogging(cmd)
 			slog.Debug("command persistent pre-run", "command", "root")
 			slog.Debug(
 				"command",
 				"path", strings.Join(strings.Split(cmd.CommandPath(), " ")[1:], " "),
 			)
 			return nil
 		},
 		PersistentPostRunE: func(cmd *cobra.Command, args []string) error {
 			slog.Debug("command persistent post-run", "command", "root")
 			return opts.Finish()
 		},
 	}
 	cobra.EnableTraverseRunHooks = true
 	opts.Bind(root)
 	root.AddCommand(
 		version.NewCommand(),
 		config.NewCommand(opts),
 		migrate.NewCommand(opts),
 		serve.NewCommand(opts),
 		skiprules.NewCommand(opts),
 	)
 	return root
 }
 func main() {
 	if err := newRootCmd().Execute(); err != nil {
 		fmt.Fprintln(os.Stderr, "job-search:", err)
 		os.Exit(1)
 	}
 }
@@ -0,0 +1,61 @@
 package main
 import (
 	"slices"
 	"testing"
 	"git.avsdev.uk/Go/job-search/internal/cli/cmdtest"
 )
 // The commands are tested in their own packages under internal/cli. These
 // tests check newRootCmd puts them together.
 func TestCommandTree(t *testing.T) {
 	root := newRootCmd()
 	var names []string
 	for _, c := range root.Commands() {
 		names = append(names, c.Name())
 	}
 	commands := []string{"config", "migrate", "serve", "skip-rules", "version"}
 	for _, want := range commands {
 		if !slices.Contains(names, want) {
 			t.Errorf("job-search has no %q command; has %v", want, names)
 		}
 	}
 	for _, flag := range []string{"config", "debug", "quiet"} {
 		if root.PersistentFlags().Lookup(flag) == nil {
 			t.Errorf("job-search has no --%s flag", flag)
 		}
 	}
 }
 func TestVersion(t *testing.T) {
 	cmdtest.Setup(t, "")
 	cmdtest.Contains(t, cmdtest.MustRun(t, newRootCmd, "version"), "JobSearch - ")
 }
 // A command that loads the configuration runs through the real root: its
 // flags and hooks, and the subcommand beneath it.
 func TestRootRunsCommands(t *testing.T) {
 	cmdtest.Setup(t, "logging:\n  level: info\n")
 	cmdtest.Contains(
 		t,
 		cmdtest.MustRun(t, newRootCmd, "config", "get", "logging.level"),
 		"info <string>",
 	)
 	cmdtest.Contains(
 		t,
 		cmdtest.MustRun(t, newRootCmd, "config", "get", "logging.level", "--debug"),
 		"debug <string>",
 	)
 }
 func TestUnknownCommand(t *testing.T) {
 	cmdtest.Setup(t, "")
 	if _, err := cmdtest.Run(t, newRootCmd, "", "nope"); err == nil {
 		t.Error("an unknown command was accepted")
 	}
 }
@@ -0,0 +1,380 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 // Mock of the Find a Tender (FTS) OCDS release package endpoint:
 //
 //	GET /api/1.0/ocdsReleasePackages?updatedFrom=2026-10-06T00:00:00&limit=5
 package main
 import (
 	"context"
 	"errors"
 	"fmt"
 	"log/slog"
 	"net"
 	"net/http"
 	"os"
 	"os/signal"
 	"path/filepath"
 	"syscall"
 	"time"
 	_ "time/tzdata"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/logging"
 	mock "git.avsdev.uk/Go/job-search/internal/mock/findatender"
 )
 // options holds the values of every command-line flag.
 type options struct {
 	addr          string
 	baseURL       string
 	dataDir       string
 	reset         bool
 	backfill      time.Duration
 	minGap        time.Duration
 	maxGap        time.Duration
 	rateLimit     int
 	rateWindow    time.Duration
 	accessLogPath string
 	logFormat     string
 	timeZone      string
 }
 const longHelp = `Mock of the Find a Tender (FTS) OCDS release package endpoint:
  GET /api/1.0/ocdsReleasePackages?updatedFrom=2026-10-06T00:00:00&limit=5
 Query parameters (mirroring the real service):
  updatedFrom  yyyy-MM-ddTHH:mm:ss   releases updated at or after this time
  updatedTo    yyyy-MM-ddTHH:mm:ss   releases updated at or before this time (optional)
  limit        1..100 (default 100)  page size
  cursor       opaque token from links.next
 Like the real service, updatedFrom and updatedTo are UK local time
 (Europe/London, so BST in summer); change this with --tz. Values with an
 explicit offset or Z are taken as given. Release dates are stored and
 returned in UTC.
 Data: the catalog of buyers, suppliers, tender subjects, procurement methods
 and classifications that releases are generated from is embedded in the
 build from internal/mock/findatender/data/catalog.json. The mock stops with an
 error if it is invalid.
 Paging: each package's uri and links.next start with --base-url, which
 clients follow as given. Without it, the base URL is taken from --addr, with
 an unspecified host (":8080", "0.0.0.0", "::") as localhost. Set it when
 clients reach the mock by another address, such as through a proxy or a
 mapped container port.
 Releases are generated from that catalog and appended to
 <data-dir>/releases.jsonl, so they survive restarts. A background worker
 publishes a new release every --min-gap to --max-gap. On startup the mock
 fills any gap since the last stored release, going back at most --backfill
 on a fresh data directory. Use --reset to discard stored releases.
 Rate limiting: each client (by remote IP) may make --rate-limit requests per
 --rate-window. Windows are fixed and anchored on the client's first request
 of the current clock hour. Over the limit the mock returns 429 Too Many
 Requests, a Retry-After header, and the body
 "Rate limit of N exceeded. Please retry after X seconds."
 Access logging: every request is written in Apache/nginx "combined" format
 (or "common" with --log-format) to stdout, or to the --access-log file.
 Startup and error messages go to stderr.`
 func main() {
 	// Cancelled on Ctrl+C or SIGTERM; Cobra hands it to the command.
 	ctx, stop := signal.NotifyContext(
 		context.Background(), os.Interrupt, syscall.SIGTERM,
 	)
 	defer stop()
 	if err := newRootCmd().ExecuteContext(ctx); err != nil {
 		// Cobra has already printed the error.
 		stop()
 		os.Exit(1)
 	}
 }
 func newRootCmd() *cobra.Command {
 	opts := &options{}
 	cmd := &cobra.Command{
 		Use:          "fts-mock",
 		Short:        "Mock of the Find a Tender OCDS release package API",
 		Long:         longHelp,
 		Args:         cobra.NoArgs,
 		SilenceUsage: true,
 		Example: `
  fts-mock --addr :8080
  fts-mock --data-dir ./mock-data --reset
  fts-mock --min-gap 5s --max-gap 10s --rate-limit 1000`,
 		PreRunE: func(cmd *cobra.Command, args []string) error {
 			err := opts.validate()
 			if err != nil {
 				return err
 			}
 			if opts.baseURL == "" {
 				opts.baseURL, err = baseURLFromAddr(opts.addr)
 			}
 			return err
 		},
 		RunE: func(cmd *cobra.Command, args []string) error {
 			return run(cmd.Context(), opts)
 		},
 	}
 	flags := cmd.Flags()
 	flags.SortFlags = false // show flags in the order defined here
 	flags.StringVar(&opts.addr, "addr", ":8080", "listen address")
 	flags.StringVar(
 		&opts.baseURL,
 		"base-url",
 		"",
 		"public base URL used in uri and links.next (default from --addr)",
 	)
 	flags.StringVar(
 		&opts.dataDir,
 		"data-dir",
 		"./mock-data",
 		"directory where releases.jsonl will be stored",
 	)
 	flags.BoolVar(
 		&opts.reset,
 		"reset",
 		false,
 		"delete stored releases and regenerate the --backfill history",
 	)
 	flags.DurationVar(
 		&opts.backfill,
 		"backfill",
 		72*time.Hour,
 		"history to generate when the data directory is empty",
 	)
 	flags.DurationVar(
 		&opts.minGap,
 		"min-gap",
 		30*time.Second,
 		"minimum time between generated releases",
 	)
 	flags.DurationVar(
 		&opts.maxGap,
 		"max-gap",
 		5*time.Minute,
 		"maximum time between generated releases",
 	)
 	flags.IntVar(
 		&opts.rateLimit, "rate-limit", 12, "requests allowed per window per client",
 	)
 	flags.DurationVar(
 		&opts.rateWindow, "rate-window", 5*time.Minute, "rate limit window length",
 	)
 	flags.StringVar(
 		&opts.accessLogPath,
 		"access-log",
 		"-",
 		`access log file, or "-" for stdout`,
 	)
 	flags.StringVar(
 		&opts.logFormat,
 		"log-format",
 		"combined",
 		`access log format: "combined" or "common"`,
 	)
 	flags.StringVar(
 		&opts.timeZone,
 		"tz",
 		"Europe/London",
 		`time zone for updatedFrom/updatedTo without an offset ("Local" for this machine's)`,
 	)
 	return cmd
 }
 // validate checks flag values before anything is started.
 func (o *options) validate() error {
 	if o.logFormat != "combined" && o.logFormat != "common" {
 		return fmt.Errorf(
 			`--log-format must be "combined" or "common", got %q`, o.logFormat,
 		)
 	}
 	if o.minGap <= 0 || o.maxGap < o.minGap {
 		return errors.New("--min-gap must be positive and no larger than --max-gap")
 	}
 	if o.rateLimit < 1 || o.rateWindow <= 0 {
 		return errors.New(
 			"--rate-limit must be at least 1 and --rate-window positive",
 		)
 	}
 	if _, err := time.LoadLocation(o.timeZone); err != nil {
 		return fmt.Errorf("--tz: %w", err)
 	}
 	return nil
 }
 // baseURLFromAddr returns the base URL clients reach a mock listening on
 // addr by: http://host:port, with localhost for an unspecified host. Port 0
 // is refused, as the port isn't known until the mock is listening.
 func baseURLFromAddr(addr string) (string, error) {
 	host, port, err := net.SplitHostPort(addr)
 	if err != nil {
 		return "", fmt.Errorf("--addr: %w", err)
 	}
 	if port == "0" {
 		return "", errors.New("--addr with port 0 needs --base-url")
 	}
 	switch host {
 	case "", "0.0.0.0", "::":
 		host = "localhost"
 	}
 	return "http://" + net.JoinHostPort(host, port), nil
 }
 // run starts the mock and blocks until ctx is cancelled or the server fails.
 func run(ctx context.Context, opts *options) (err error) {
 	location, err := time.LoadLocation(opts.timeZone) // already validated
 	if err != nil {
 		return fmt.Errorf("--tz: %w", err)
 	}
 	if err = os.MkdirAll(opts.dataDir, 0o750); err != nil {
 		return fmt.Errorf("creating data directory: %w", err)
 	}
 	// Load the catalog.
 	catalog, err := mock.LoadCatalog()
 	if err != nil {
 		return fmt.Errorf("loading catalog: %w", err)
 	}
 	slog.Info(
 		"catalog loaded",
 		"buyers", len(catalog.Buyers),
 		"suppliers", len(catalog.Suppliers),
 		"subjects", len(catalog.Subjects),
 		"methods", len(catalog.Methods),
 		"classifications", len(catalog.Classifications),
 	)
 	// Open (or reset) the store.
 	storePath := filepath.Join(opts.dataDir, mock.ReleasesFile)
 	if opts.reset {
 		if err = os.Remove(storePath); err != nil &&
 			!errors.Is(err, os.ErrNotExist) {
 			return fmt.Errorf("resetting store: %w", err)
 		}
 		slog.Info("store reset", "path", storePath)
 	}
 	store, err := mock.OpenStore(storePath)
 	if err != nil {
 		return fmt.Errorf("opening store: %w", err)
 	}
 	defer func() {
 		err = errors.Join(err, store.Close())
 	}()
 	// Backfill releases, then keep generating them.
 	generator := mock.NewGenerator(catalog, store, time.Now().UnixNano())
 	generated, err := generator.CatchUp(
 		time.Now(), opts.backfill, opts.minGap, opts.maxGap,
 	)
 	if err != nil {
 		return fmt.Errorf("backfilling releases: %w", err)
 	}
 	slog.Info("store ready", "releases", store.Len(), "generated", generated)
 	// The generator and server share a context that also ends if the server
 	// fails, so the generator is never left running after run returns.
 	ctx, cancel := context.WithCancel(ctx)
 	defer cancel()
 	workerDone := make(chan struct{})
 	go func() {
 		defer close(workerDone)
 		generator.Run(ctx, opts.minGap, opts.maxGap)
 	}()
 	// Serve the API.
 	accessLogger, closeAccessLog, err := logging.OpenAccessLog(
 		opts.accessLogPath,
 	)
 	if err != nil {
 		cancel()
 		<-workerDone
 		return err
 	}
 	defer func() {
 		err = errors.Join(err, closeAccessLog())
 	}()
 	rateLimiter := mock.NewRateLimiter(opts.rateLimit, opts.rateWindow)
 	mux := http.NewServeMux()
 	mux.HandleFunc(
 		mock.BasePath,
 		rateLimiter.Middleware(mock.NewHandler(store, opts.baseURL, location)),
 	)
 	httpServer := &http.Server{
 		Addr: opts.addr,
 		Handler: logging.AccessLog(
 			accessLogger, opts.logFormat == "combined", mux,
 		),
 		ReadHeaderTimeout: 10 * time.Second,
 	}
 	go func() {
 		<-ctx.Done()
 		// ctx is done by now; the shutdown gets its own deadline.
 		shutdownCtx, cancelShutdown := context.WithTimeout(
 			context.WithoutCancel(ctx), 5*time.Second,
 		)
 		defer cancelShutdown()
 		if err := httpServer.Shutdown(shutdownCtx); err != nil {
 			slog.Error("shutdown failed", "error", err)
 		}
 	}()
 	slog.Info(
 		"mock listening",
 		"addr", opts.addr,
 		"base_url", opts.baseURL,
 		"path", mock.BasePath,
 		"time_zone", location.String(),
 	)
 	serveErr := httpServer.ListenAndServe()
 	cancel() // stop the worker (and the shutdown goroutine) if the server failed
 	<-workerDone
 	if serveErr != nil && !errors.Is(serveErr, http.ErrServerClosed) {
 		return fmt.Errorf("serving: %w", serveErr)
 	}
 	slog.Info("stopped")
 	return nil
 }
@@ -0,0 +1,37 @@
 package main
 import "testing"
 // Without --base-url, the base URL comes from --addr, with localhost for an
 // unspecified host.
 func TestBaseURLFromAddr(t *testing.T) {
 	tests := []struct {
 		addr string
 		want string
 	}{
 		{":8080", "http://localhost:8080"},
 		{"0.0.0.0:8080", "http://localhost:8080"},
 		{"[::]:8080", "http://localhost:8080"},
 		{"127.0.0.1:18080", "http://127.0.0.1:18080"},
 		{"[::1]:8080", "http://[::1]:8080"},
 		{"mock.example.com:80", "http://mock.example.com:80"},
 	}
 	for _, tt := range tests {
 		got, err := baseURLFromAddr(tt.addr)
 		if err != nil || got != tt.want {
 			t.Errorf(
 				"baseURLFromAddr(%q) = %q, %v; want %q",
 				tt.addr, got, err, tt.want,
 			)
 		}
 	}
 }
 // An address without a port, or with port 0, is refused.
 func TestBaseURLFromAddrRefused(t *testing.T) {
 	for _, addr := range []string{"localhost", ":0"} {
 		if got, err := baseURLFromAddr(addr); err == nil {
 			t.Errorf("baseURLFromAddr(%q) = %q; want an error", addr, got)
 		}
 	}
 }
@@ -1,118 +0,0 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 package cmd
 import (
 	"log/slog"
 	"os"
 	"strings"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	cmdConfig "git.avsdev.uk/Go/job-search/internal/cli/config"
 	cmdVersion "git.avsdev.uk/Go/job-search/internal/cli/version"
 	"git.avsdev.uk/Go/job-search/internal/config"
 	"git.avsdev.uk/Go/job-search/internal/log"
 	"github.com/spf13/cobra"
 )
 var rootCmd = &cobra.Command{
 	Use:   "job-search",
 	Short: "JobSearch - AVSDev Future Opportunities!",
 	Long: `JobSearch is a scraping tool for searching for possible contract
 opportunities.
 	`,
 	CompletionOptions: cobra.CompletionOptions{
 		HiddenDefaultCmd: true,
 	},
 	PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
 		// Check if the command is expecting to use terminal for messages
 		if val, ok := cmd.Annotations["terminal_log"]; ok && val == "1" {
 			log.LogToTerminal()
 		}
 		// Check for debug level via args
 		dbgCount, _ := cmd.Flags().GetCount("debug")
 		conf := config.JobSearchConfig{
 			Logging: config.JobSearchConfigLogging{},
 		}
 		if dbgCount > 1 {
 			conf.Logging.Level = "debug"
 		} else if dbgCount > 0 {
 			conf.Logging.Level = "info"
 		}
 		if conf.Logging.LogLevel() <= slog.LevelDebug {
 			log.EnableDebug()
 		} else if conf.Logging.LogLevel() <= slog.LevelInfo {
 			log.EnableInfo()
 		}
 		// With logging set up, these will be visible if trace is active
 		slog.Debug("root.PersistentPreRun")
 		slog.Debug(
 			"command: \"" +
 				strings.Join(strings.Split(cmd.CommandPath(), " ")[1:], " ") +
 				"\"",
 		)
 		// Always load config unless command indicates not to
 		if val, ok := cmd.Annotations["no_config"]; !ok || val == "0" {
 			// Load config - from specified file if provided
 			cfgFile, _ := cmd.Flags().GetString("config")
 			if err := config.Init(cfgFile, &conf); err != nil {
 				return err
 			}
 			// Attach config to command context
 			ctx := cli.WithConfig(cmd.Context(), conf)
 			cmd.SetContext(ctx)
 			// Change logging level based off newly loaded config
 			if conf.Logging.LogLevel() <= slog.LevelDebug {
 				log.EnableDebug()
 			}
 		}
 		return nil
 	},
 	PersistentPostRun: func(cmd *cobra.Command, args []string) {
 		slog.Debug("root.PersistentPostRun")
 	},
 }
 // Execute, called by main.main() handles all child commands and appropriate
 // flags.
 func Execute() {
 	err := rootCmd.Execute()
 	if err != nil {
 		os.Exit(1)
 	}
 }
 func init() {
 	cobra.EnableTraverseRunHooks = true
 	log.Init()
 	pFlags := rootCmd.PersistentFlags()
 	pFlags.StringP(
 		"config", "c", "",
 		"config file (default: "+config.EtcConfigPath+"/config.yaml)")
 	pFlags.CountP("debug", "d", "enable debug output (repeat for trace)")
 	rootCmd.AddCommand(cmdVersion.NewCommand())
 	rootCmd.AddCommand(cmdConfig.NewCommand())
 }
@@ -0,0 +1,82 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 // Command ws-test, for testing JobSearch's WebSocket servers. Run
 // `ws-test --help` for usage.
 package main
 import (
 	"fmt"
 	"log/slog"
 	"os"
 	"strings"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/cli/ws"
 )
 // newRootCmd builds the command tree. It returns a fresh tree on each call so
 // tests can run commands in isolation.
 func newRootCmd() *cobra.Command {
 	opts := &cli.Options{}
 	root := &cobra.Command{
 		Use:   "ws-test",
 		Short: "WebSocket test tools for JobSearch",
 		Long: `ws-test runs a plain TCP server for websocket hub clients (serve), and
 tests the JobSearch HTTP server's /ws endpoint (dial). Defaults come from the
 JobSearch configuration.
 	`,
 		SilenceUsage:  true, // don't print usage on runtime errors
 		SilenceErrors: true, // main prints the error once
 		CompletionOptions: cobra.CompletionOptions{
 			HiddenDefaultCmd: true,
 		},
 		PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
 			cli.SetupInitialLogging(cmd)
 			slog.Debug("command persistent pre-run", "command", "root")
 			slog.Debug(
 				"command",
 				"path", strings.Join(strings.Split(cmd.CommandPath(), " ")[1:], " "),
 			)
 			return nil
 		},
 		PersistentPostRunE: func(cmd *cobra.Command, args []string) error {
 			slog.Debug("command persistent post-run", "command", "root")
 			return opts.Finish()
 		},
 	}
 	cobra.EnableTraverseRunHooks = true
 	opts.Bind(root)
 	root.AddCommand(ws.NewCommands(opts)...)
 	return root
 }
 func main() {
 	if err := newRootCmd().Execute(); err != nil {
 		fmt.Fprintln(os.Stderr, "ws-test:", err)
 		os.Exit(1)
 	}
 }
@@ -0,0 +1,30 @@
 package main
 import (
 	"slices"
 	"testing"
 )
 // The commands are tested in internal/cli/ws. This test checks newRootCmd
 // puts them together.
 func TestCommandTree(t *testing.T) {
 	root := newRootCmd()
 	var names []string
 	for _, c := range root.Commands() {
 		names = append(names, c.Name())
 	}
 	for _, want := range []string{"dial", "serve"} {
 		if !slices.Contains(names, want) {
 			t.Errorf("ws-test has no %q command; has %v", want, names)
 		}
 	}
 	for _, flag := range []string{"config", "debug", "quiet"} {
 		if root.PersistentFlags().Lookup(flag) == nil {
 			t.Errorf("ws-test has no --%s flag", flag)
 		}
 	}
 }
@@ -0,0 +1,18 @@
 # logrotate configuration for job-search's logs (see README.md, "Rotating
 # the logs"). Install as /etc/logrotate.d/job-search; it is in the release
 # tarball next to the binary.
 #
 # job-search opens its log file once and has no signal to reopen it, so the
 # file is copied and then truncated in place (copytruncate). A line written
 # between the copy and the truncation is lost.
 /var/log/job-search/*.log {
    weekly
    rotate 8
    compress
    delaycompress
    missingok
    notifempty
    copytruncate
    # /var/log/job-search belongs to the service user, so rotate as it.
    su job-search job-search
 }
@@ -0,0 +1,80 @@
 # job-search: serves the API and frontend, and runs the enabled sources'
 # fetchers (job-search serve). See README.md, "Running as a service".
 #
 # Install, from the release tarball's directory (job-search-<version>-<arch>):
 #   sudo useradd --system --home-dir /var/lib/job-search --shell /usr/sbin/nologin job-search
 #   sudo install -m 755 job-search /usr/local/bin/job-search
 #   sudo install -d -m 750 -o root -g job-search /etc/job-search
 #   sudo install -m 640 -o root -g job-search config.yaml /etc/job-search/config.yaml
 #   sudo install -d -m 750 -o job-search -g job-search /var/lib/job-search /var/log/job-search
 #   sudo -u job-search /usr/local/bin/job-search --config /etc/job-search/config.yaml migrate up --create
 #   sudo install -m 644 job-search.service /etc/systemd/system/
 #   sudo systemctl daemon-reload
 #   sudo systemctl enable --now job-search
 #
 # /etc/job-search/config.yaml should set the database:
 #   database:
 #       file: /var/lib/job-search/jobsearch.db
 # Logs go to the journal (journalctl -u job-search), unless logging.file is
 # set to log to a file instead, rotated with job-search.logrotate:
 #   logging:
 #       file: /var/log/job-search/job-search.log
 #   sudo install -m 644 job-search.logrotate /etc/logrotate.d/job-search
 #
 # After each upgrade, migrate the database before restarting:
 #   sudo -u job-search /usr/local/bin/job-search --config /etc/job-search/config.yaml migrate up
 #   sudo systemctl restart job-search
 [Unit]
 Description=Job Search: procurement opportunities server and fetcher
 Documentation=https://git.avsdev.uk/Go/job-search
 Wants=network-online.target
 After=network-online.target
 [Service]
 Type=simple
 User=job-search
 Group=job-search
 # /var/lib/job-search and /var/log/job-search, created and owned by the
 # service user; the only places it can write.
 StateDirectory=job-search
 StateDirectoryMode=0750
 LogsDirectory=job-search
 LogsDirectoryMode=0750
 WorkingDirectory=/var/lib/job-search
 ExecStart=/usr/local/bin/job-search --config /etc/job-search/config.yaml serve
 # serve shuts down cleanly on SIGTERM: the HTTP server drains and in-flight
 # fetches get up to 10s.
 KillSignal=SIGTERM
 TimeoutStopSec=30
 # A failing source stops serve on purpose; restart to retry it.
 Restart=on-failure
 RestartSec=30
 UMask=0027
 # Hardening
 NoNewPrivileges=yes
 CapabilityBoundingSet=
 AmbientCapabilities=
 ProtectSystem=strict
 ProtectHome=yes
 PrivateTmp=yes
 PrivateDevices=yes
 ProtectKernelTunables=yes
 ProtectKernelModules=yes
 ProtectKernelLogs=yes
 ProtectControlGroups=yes
 ProtectClock=yes
 ProtectHostname=yes
 RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
 RestrictNamespaces=yes
 RestrictRealtime=yes
 RestrictSUIDSGID=yes
 LockPersonality=yes
 MemoryDenyWriteExecute=yes
 SystemCallArchitectures=native
 SystemCallFilter=@system-service
 SystemCallFilter=~@privileged @resources
 [Install]
 WantedBy=multi-user.target
@@ -3,24 +3,39 @@ module git.avsdev.uk/Go/job-search
 go 1.26.8
 require (
 	github.com/coder/websocket v1.8.15
 	github.com/dustin/go-humanize v1.0.1
 	github.com/go-chi/chi/v5 v5.4.0
 	github.com/jmoiron/sqlx v1.4.0
 	github.com/joho/godotenv v1.5.1
 	github.com/klauspost/compress v1.20.3
 	github.com/lmittmann/tint v1.2.0
 	github.com/mattn/go-sqlite3 v1.14.52
 	github.com/pires/go-proxyproto v0.15.0
 	github.com/pressly/goose/v3 v3.28.0
 	github.com/spf13/cobra v1.10.2
 	github.com/spf13/viper v1.21.0
 	go.yaml.in/yaml/v3 v3.0.5
 	golang.org/x/net v0.58.0
 	golang.org/x/time v0.16.0
 	resty.dev/v3 v3.0.0-rc.4
 )
 require (
-	github.com/fsnotify/fsnotify v1.9.0 // indirect
+	github.com/fsnotify/fsnotify v1.10.1 // indirect
 	github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
 	github.com/inconshreveable/mousetrap v1.1.0 // indirect
 	github.com/mfridman/interpolate v0.0.2 // indirect
 	github.com/pelletier/go-toml/v2 v2.2.4 // indirect
 	github.com/sagikazarmark/locafero v0.11.0 // indirect
 	github.com/sethvargo/go-retry v0.4.0 // indirect
 	github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 // indirect
 	github.com/spf13/afero v1.15.0 // indirect
 	github.com/spf13/cast v1.10.0 // indirect
 	github.com/spf13/pflag v1.0.10 // indirect
 	github.com/subosito/gotenv v1.6.0 // indirect
-	golang.org/x/sys v0.29.0 // indirect
+	go.uber.org/multierr v1.11.0 // indirect
-	golang.org/x/text v0.28.0 // indirect
+	golang.org/x/sync v0.22.0 // indirect
 	golang.org/x/sys v0.47.0 // indirect
 	golang.org/x/text v0.41.0 // indirect
 )
@@ -1,19 +1,66 @@
 filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
 filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
 filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
 github.com/coder/websocket v1.8.15 h1:6B2JPeOGlpff2Uz6vOEH1Vzpi0iUz20A+lPVhPHtNUA=
 github.com/coder/websocket v1.8.15/go.mod h1:NX3SzP+inril6yawo5CQXx8+fk145lPDC6pumgx0mVg=
 github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
-github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
+github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
-github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
+github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
 github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
 github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
 github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho=
 github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
 github.com/go-chi/chi/v5 v5.4.0 h1:+ufzvFTGKyin26Or5DLrlIDq4ygIDBVcj7oVNeeaYFE=
 github.com/go-chi/chi/v5 v5.4.0/go.mod h1:d1IG7GtKPSjPyBP365PH8sAGVHk3qfB/IhjFNRH1wWM=
 github.com/go-sql-driver/mysql v1.8.1/go.mod h1:wEBSXgmK//2ZFJyE+qWnIsVGmvmEKlqwuVSjsCm7DZg=
 github.com/go-sql-driver/mysql v1.10.0 h1:Q+1LV8DkHJvSYAdR83XzuhDaTykuDx0l6fkXxoWCWfw=
 github.com/go-sql-driver/mysql v1.10.0/go.mod h1:M+cqaI7+xxXGG9swrdeUIoPG3Y3KCkF0pZej+SK+nWk=
 github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
 github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
 github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
 github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
 github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
 github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
 github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
 github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
 github.com/jmoiron/sqlx v1.4.0 h1:1PLqN7S1UYp5t4SrVVnt4nUVNemrDAtxlulVe+Qgm3o=
 github.com/jmoiron/sqlx v1.4.0/go.mod h1:ZrZ7UsYB/weZdl2Bxg6jCRO9c3YHl8r3ahlKmRT4JLY=
 github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
 github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
 github.com/klauspost/compress v1.20.3 h1:zWNoFqq/lSuYQRim3dUXlXeA0IqwuZGicdG5ZTfY5j4=
 github.com/klauspost/compress v1.20.3/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
 github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
 github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
 github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
 github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
 github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
 github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
 github.com/lmittmann/tint v1.2.0 h1:AogHRHy8HUJUnNJBHJlYa+fR4YY8mko2cnCp67xn9JY=
 github.com/lmittmann/tint v1.2.0/go.mod h1:HIS3gSy7qNwGCj+5oRjAutErFBl4BzdQP6cJZ0NfMwE=
 github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
 github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
 github.com/mattn/go-sqlite3 v1.14.22/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
 github.com/mattn/go-sqlite3 v1.14.52 h1:wVbm2Qnf4OXkqhBTSPuCRZDRnxfbVrrmiCEroVdog8U=
 github.com/mattn/go-sqlite3 v1.14.52/go.mod h1:6JTjA44L93a0QCyJef5YvlPoKXntQPjzWv5gtm9sB6w=
 github.com/mfridman/interpolate v0.0.2 h1:pnuTK7MQIxxFz1Gr+rjSIx9u7qVjf5VOoM/u6BbAxPY=
 github.com/mfridman/interpolate v0.0.2/go.mod h1:p+7uk6oE07mpE/Ik1b8EckO0O4ZXiGAfshKBWLUM9Xg=
 github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
 github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
 github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
 github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
 github.com/pires/go-proxyproto v0.15.0 h1:dTshmNbFm/D+0+sbrxUuddPOZ5Y0B7c5NhtsBkm6LqI=
 github.com/pires/go-proxyproto v0.15.0/go.mod h1:OXsCrKwrK2tXS9YrI5tkHx5xaQlO8FH3lFW76orFh24=
 github.com/pressly/goose/v3 v3.28.0 h1:D2M+iL31GmpZxSHOhX8mqyqAT3CXnokUmm0eKoSP+Vc=
 github.com/pressly/goose/v3 v3.28.0/go.mod h1:v26MOuB8bL3kzzrt3Vqhb3R0PRVsl8hFQKdrht/L6Rk=
 github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
 github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
 github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
 github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
 github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
 github.com/sagikazarmark/locafero v0.11.0 h1:1iurJgmM9G3PA/I+wWYIOw/5SyBtxapeHDcg+AAIFXc=
 github.com/sagikazarmark/locafero v0.11.0/go.mod h1:nVIGvgyzw595SUSUE6tvCp3YYTeHs15MvlmU87WwIik=
 github.com/sethvargo/go-retry v0.4.0 h1:9qy1OoIAxBL+gBYnkTnTnWle5wlfsXQlwRzIbbpdqPw=
 github.com/sethvargo/go-retry v0.4.0/go.mod h1:tvsjdKG6xfiCx4LSiUZ06kcv38xvdVQwv8R6/VnnVWg=
 github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 h1:+jumHNA0Wrelhe64i8F6HNlS8pkoyMv5sreGx2Ry5Rw=
 github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8/go.mod h1:3n1Cwaq1E1/1lhQhtRK2ts/ZwZEhjcQeJQ1RuC6Q/8U=
 github.com/spf13/afero v1.15.0 h1:b/YBCLWAJdFWJTN9cLhiXXcD7mzKn9Dm86dNnfyQw1I=
@@ -22,19 +69,38 @@ github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
 github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
 github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
 github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
 github.com/spf13/pflag v1.0.9 h1:9exaQaMOCwffKiiiYk6/BndUBv+iRViNW+4lEMi0PvY=
 github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
 github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
 github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
 github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU=
 github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY=
 github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
 github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
 github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
 github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
 go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
 go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
 go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
 go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
 go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
-golang.org/x/sys v0.29.0 h1:TPYlXGxvx1MGTn2GiZDhnjPA9wZzZeGKHHmKhHYvgaU=
+golang.org/x/net v0.58.0 h1:ynWG7rqYi4ccpTEuPZ2QGWHktVEM9DMCj9yzDE0Q7To=
-golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/net v0.58.0/go.mod h1:YwCddHnFlT7eLQqVprV19OnhLGtc5xOKgE0RyqgfWAU=
-golang.org/x/text v0.28.0 h1:rhazDwis8INMIwQ4tpjLDzUhx6RlXqZNPEM0huQojng=
+golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
-golang.org/x/text v0.28.0/go.mod h1:U8nCwOR8jO/marOQ0QbDiOngZVEBB7MAiitBuMjXiNU=
+golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
 golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
 golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
 golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=
 golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=
 golang.org/x/time v0.16.0 h1:vMb6ptszcQMkcwiRTAuNNU50gom6++Q/6gY2hDM6VDE=
 golang.org/x/time v0.16.0/go.mod h1:rVKOqvZeKvrDKTQiAHJ7wmwP0RzleSphoEA9RcdLA0s=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 modernc.org/libc v1.75.6 h1:yKk8qo+Di4gkmvRboK8ocCqH22FiUCR6jRy2OwtCRus=
 modernc.org/libc v1.75.6/go.mod h1:bO5o2ztHxBb2rjz0PgdHN0sSMw57CgxGFLZ3Qd/QpVQ=
 modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
 modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
 modernc.org/memory v1.12.1 h1:nFMiWrpStgZczNl6XI9GnIk/rWhYIyHGUaR04pGbp9g=
 modernc.org/memory v1.12.1/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
 modernc.org/sqlite v1.57.0 h1:qNQP6xnx5M0ISNtlnxoOX0+cD5bJ0/gr9aMmndFczzg=
 modernc.org/sqlite v1.57.0/go.mod h1:yCJ2cmAaIkHQ25oXWrF8H4O1lIfPYPR26yCEDj2P3pQ=
 resty.dev/v3 v3.0.0-rc.4 h1:n3360yEcFiyrX5Vkc9XG3Y8kMM24alRBxJ9Ev1mLmqs=
 resty.dev/v3 v3.0.0-rc.4/go.mod h1:NTOerrC/4T7/FE6tXIZGIysXXBdgNqwMZuKtxpea9NM=
@@ -0,0 +1 @@
 openapi/
@@ -0,0 +1,58 @@
 package api
 import (
 	"github.com/go-chi/chi/v5"
 	"git.avsdev.uk/Go/job-search/internal/events"
 	"git.avsdev.uk/Go/job-search/internal/store"
 )
 // API is the REST API, mounted under /api. Create it with New, giving it what
 // its resources need as options, and register its routes with Routes. The
 // server applies the API's middleware (CSRF, NoStore, RequireJSON, rate
 // limits); the API only registers handlers.
 type API struct {
 	store  *store.Store // nil: no opportunity routes
 	events *events.Bus  // nil: nothing is published
 }
 // Option configures the API.
 type Option func(*API)
 // WithStore makes the API serve the opportunities in opportunityStore under
 // /opportunities, and their classifications under /classifications. Without
 // it there are neither.
 func WithStore(opportunityStore *store.Store) Option {
 	return func(a *API) {
 		a.store = opportunityStore
 	}
 }
 // WithEvents makes the API publish its changes to bus, such as an
 // events.OpportunityMarksUpdated event when an opportunity's marks change.
 func WithEvents(bus *events.Bus) Option {
 	return func(a *API) {
 		a.events = bus
 	}
 }
 // New returns the API.
 func New(opts ...Option) *API {
 	api := &API{}
 	for _, option := range opts {
 		option(api)
 	}
 	return api
 }
 // Routes registers the API's routes on router (mounted under /api): its
 // description (see openapi.go), and the opportunities and classifications if
 // it has a store.
 func (a *API) Routes(router chi.Router) {
 	docsRoutes(router)
 	if a.store != nil {
 		newOpportunityRoutes(a.store, a.events).Routes(router)
 		newClassificationRoutes(a.store).Routes(router)
 	}
 }
@@ -0,0 +1,54 @@
 package api
 import (
 	"net/http"
 	"github.com/go-chi/chi/v5"
 	"git.avsdev.uk/Go/job-search/internal/httpx"
 	"git.avsdev.uk/Go/job-search/internal/store"
 )
 // classificationRoutes wires HTTP handlers to the classification codes in
 // the opportunity store. They're read-only.
 type classificationRoutes struct {
 	store *store.Store
 }
 // newClassificationRoutes returns the classification handlers for
 // opportunityStore.
 func newClassificationRoutes(
 	opportunityStore *store.Store,
 ) *classificationRoutes {
 	return &classificationRoutes{
 		store: opportunityStore,
 	}
 }
 // Routes registers the classification routes on router (mounted under
 // /api).
 func (h *classificationRoutes) Routes(router chi.Router) {
 	router.Get("/classifications", h.list)
 }
 // list serves GET /classifications: every classification code that at least
 // one opportunity has, ordered by code.
 //
 //	@Summary		List classifications
 //	@Description	Every classification code (CPV and others) that at least one
 //	@Description	stored opportunity has, once each, ordered by code: the
 //	@Description	codes worth filtering opportunities by.
 //	@Tags			classifications
 //	@Produce		json
 //	@Success		200	{array}		types.ClassificationCode
 //	@Failure		500	{object}	httpx.Problem
 //	@Router			/classifications [get]
 func (h *classificationRoutes) list(w http.ResponseWriter, r *http.Request) {
 	classifications, err := h.store.ListClassifications(r.Context())
 	if err != nil {
 		httpx.WriteProblem(w, r, http.StatusInternalServerError, "")
 		return
 	}
 	httpx.WriteJSON(w, http.StatusOK, classifications)
 }
@@ -0,0 +1,41 @@
 package api
 import (
 	"encoding/json"
 	"net/http"
 	"testing"
 	"git.avsdev.uk/Go/job-search/internal/types"
 )
 // The classifications the opportunities have are listed once each, by code.
 func TestListClassifications(t *testing.T) {
 	url := opportunitiesServer(t, 3)
 	var classifications []types.ClassificationCode
 	status := request(t, "GET", url+"/classifications", &classifications)
 	if status != http.StatusOK {
 		t.Fatalf("status %d", status)
 	}
 	if len(classifications) != 2 ||
 		classifications[0].Code != "03000000" ||
 		classifications[1].Code != "72000000" ||
 		classifications[0].ID == 0 {
 		t.Errorf("got %+v; want 03000000 then 72000000, with ids", classifications)
 	}
 }
 // With no opportunities, the list is [] in JSON, not null.
 func TestListClassificationsEmpty(t *testing.T) {
 	url := opportunitiesServer(t, 0)
 	res, err := http.Get(url + "/classifications")
 	if err != nil {
 		t.Fatal(err)
 	}
 	defer res.Body.Close()
 	var raw json.RawMessage
 	json.NewDecoder(res.Body).Decode(&raw)
 	if string(raw) != "[]" {
 		t.Errorf("body = %s; want []", raw)
 	}
 }
@@ -0,0 +1,42 @@
 package api
 import (
 	"fmt"
 	"strconv"
 	"time"
 )
 // intParam parses the query parameter text as an integer from low to high,
 // or returns fallback if it is empty. Its error is meant for the client.
 func intParam(text string, fallback, low, high int) (int, error) {
 	if text == "" {
 		return fallback, nil
 	}
 	number, err := strconv.Atoi(text)
 	if err != nil || number < low || number > high {
 		return 0, fmt.Errorf(
 			"parameter must be an integer between %d and %d", low, high,
 		)
 	}
 	return number, nil
 }
 // timeParam parses the query parameter name's text as an RFC 3339 time, or
 // returns the zero time if it is empty. Its error is meant for the client.
 func timeParam(name, text string) (time.Time, error) {
 	if text == "" {
 		return time.Time{}, nil
 	}
 	when, err := time.Parse(time.RFC3339, text)
 	if err != nil {
 		return time.Time{}, fmt.Errorf(
 			"%s must be an RFC 3339 date and time, e.g. 2026-10-12T00:00:00+01:00",
 			name,
 		)
 	}
 	return when, nil
 }
@@ -0,0 +1,24 @@
 // Package api implements the REST API mounted under /api. New assembles it
 // (api.go): the opportunities, read-only apart from their marks
 // (opportunities.go), their classifications (classifications.go), and the
 // API's own OpenAPI 3.1 description and docs viewer (openapi.go). New
 // resources are added in New and Routes, so the server only mounts the API.
 //
 // The description is generated from the swag annotations on this package's
 // handlers by go generate (swag v2, see the README's dev tools), into
 // openapi/swagger.json. The annotations below are the API's general
 // information.
 //
 //	@title			JobSearch API
 //	@version		1.0
 //	@description	Procurement opportunities fetched by JobSearch, and the user's
 //	@description	marks on them. Errors are RFC 9457 problem details.
 //	@servers.url	/api
 //
 //	@securityDefinitions.apikey	CSRF
 //	@in							header
 //	@name						X-CSRF-Token
 //	@description				From GET /api/csrf; send its cookie too.
 package api
 //go:generate swag init --v3.1 --generalInfo doc.go --dir . --parseInternal --parseDependencyLevel 1 --requiredByDefault --outputTypes json --output openapi
@@ -0,0 +1,54 @@
 package api
 import (
 	_ "embed"
 	"net/http"
 	"github.com/go-chi/chi/v5"
 )
 // openAPISpec is the API's OpenAPI 3.1 description, generated by go generate
 // (see doc.go).
 //
 //go:embed openapi/swagger.json
 var openAPISpec []byte
 // docsPage shows openapi.json, relative to /api/docs, in Swagger UI, loaded
 // from jsDelivr at a fixed version. A content security policy
 // (server.headers.content_security_policy) must allow that.
 const docsPage = `<!doctype html>
 <html lang="en">
 <head>
 <meta charset="utf-8">
 <meta name="viewport" content="width=device-width, initial-scale=1">
 <title>JobSearch API</title>
 <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/swagger-ui-dist@5.17.14/swagger-ui.css">
 </head>
 <body>
 <div id="ui"></div>
 <script src="https://cdn.jsdelivr.net/npm/swagger-ui-dist@5.17.14/swagger-ui-bundle.js"></script>
 <script>
 SwaggerUIBundle({url: "openapi.json", dom_id: "#ui"});
 </script>
 </body>
 </html>
 `
 // docsRoutes registers the API's description on router: GET /openapi.json,
 // the OpenAPI 3.1 description, and GET /docs, a viewer of it.
 func docsRoutes(router chi.Router) {
 	router.Get("/openapi.json", serveOpenAPISpec)
 	router.Get("/docs", serveDocsPage)
 }
 // serveOpenAPISpec serves GET /openapi.json.
 func serveOpenAPISpec(w http.ResponseWriter, r *http.Request) {
 	w.Header().Set("Content-Type", "application/json")
 	w.Write(openAPISpec) //nolint:errcheck,gosec // the client may have gone
 }
 // serveDocsPage serves GET /docs.
 func serveDocsPage(w http.ResponseWriter, r *http.Request) {
 	w.Header().Set("Content-Type", "text/html; charset=utf-8")
 	w.Write([]byte(docsPage)) //nolint:errcheck,gosec // the client may have gone
 }
@@ -0,0 +1,136 @@
 package api
 import (
 	"encoding/json"
 	"io"
 	"net/http"
 	"net/http/httptest"
 	"strings"
 	"testing"
 	"github.com/go-chi/chi/v5"
 	"git.avsdev.uk/Go/job-search/internal/store"
 )
 // The description is served as JSON, and the docs page points at it.
 func TestDocsRoutes(t *testing.T) {
 	router := chi.NewRouter()
 	New().Routes(router)
 	server := httptest.NewServer(router)
 	t.Cleanup(server.Close)
 	res, err := http.Get(server.URL + "/openapi.json")
 	if err != nil {
 		t.Fatal(err)
 	}
 	defer res.Body.Close()
 	var spec struct {
 		OpenAPI string `json:"openapi"`
 	}
 	if err := json.NewDecoder(res.Body).Decode(&spec); err != nil {
 		t.Fatal(err)
 	}
 	if res.Header.Get("Content-Type") != "application/json" ||
 		spec.OpenAPI != "3.1.0" {
 		t.Errorf(
 			"openapi.json: %s, openapi %q; want JSON, 3.1.0",
 			res.Header.Get("Content-Type"), spec.OpenAPI,
 		)
 	}
 	res, err = http.Get(server.URL + "/docs")
 	if err != nil {
 		t.Fatal(err)
 	}
 	defer res.Body.Close()
 	page, _ := io.ReadAll(res.Body)
 	if !strings.Contains(string(page), `url: "openapi.json"`) {
 		t.Errorf("docs page doesn't load openapi.json:\n%s", page)
 	}
 }
 // Every route the API serves, apart from its description's own, is in the
 // description, and the description has no route that isn't served, so the
 // annotations can't fall behind as resources are added.
 func TestOpenAPICoversRoutes(t *testing.T) {
 	var spec struct {
 		Paths map[string]map[string]json.RawMessage `json:"paths"`
 	}
 	if err := json.Unmarshal(openAPISpec, &spec); err != nil {
 		t.Fatal(err)
 	}
 	documented := map[string]bool{}
 	for path, operations := range spec.Paths {
 		for method := range operations {
 			documented[strings.ToUpper(method)+" "+path] = true
 		}
 	}
 	// A store that is never opened: registering routes doesn't use it.
 	router := chi.NewRouter()
 	New(WithStore(&store.Store{})).Routes(router)
 	description := map[string]bool{
 		"GET /openapi.json": true,
 		"GET /docs":         true,
 	}
 	served := map[string]bool{}
 	err := chi.Walk(router, func(
 		method, route string, _ http.Handler, _ ...func(http.Handler) http.Handler,
 	) error {
 		if !description[method+" "+route] {
 			served[method+" "+route] = true
 		}
 		return nil
 	})
 	if err != nil {
 		t.Fatal(err)
 	}
 	for route := range served {
 		if !documented[route] {
 			t.Errorf("%s is served but not in the OpenAPI description", route)
 		}
 	}
 	for route := range documented {
 		if !served[route] {
 			t.Errorf("%s is in the OpenAPI description but not served", route)
 		}
 	}
 }
 // The description's list of marks matches OpportunityMarks, so its Enums
 // annotation can't fall behind the struct.
 func TestOpenAPIMarkEnum(t *testing.T) {
 	var spec struct {
 		Paths map[string]map[string]struct {
 			Parameters []struct {
 				Name   string `json:"name"`
 				Schema struct {
 					Enum []string `json:"enum"`
 				} `json:"schema"`
 			} `json:"parameters"`
 		} `json:"paths"`
 	}
 	if err := json.Unmarshal(openAPISpec, &spec); err != nil {
 		t.Fatal(err)
 	}
 	want := strings.Join(markNames, ",")
 	checked := 0
 	operations := spec.Paths["/opportunities/{uuid}/marks/{mark}"]
 	for method, operation := range operations {
 		for _, parameter := range operation.Parameters {
 			if parameter.Name != "mark" {
 				continue
 			}
 			checked++
 			if got := strings.Join(parameter.Schema.Enum, ","); got != want {
 				t.Errorf("%s mark enum = %s; want %s", method, got, want)
 			}
 		}
 	}
 	if checked != 2 {
 		t.Errorf("checked the mark of %d routes; want 2 (PUT, DELETE)", checked)
 	}
 }
@@ -0,0 +1,448 @@
 package api
 import (
 	"errors"
 	"fmt"
 	"net/http"
 	"reflect"
 	"strings"
 	"sync"
 	"time"
 	"github.com/go-chi/chi/v5"
 	"git.avsdev.uk/Go/job-search/internal/events"
 	"git.avsdev.uk/Go/job-search/internal/httpx"
 	"git.avsdev.uk/Go/job-search/internal/search"
 	"git.avsdev.uk/Go/job-search/internal/store"
 	"git.avsdev.uk/Go/job-search/internal/types"
 )
 // markIndex maps the name of each mark that can be set and cleared, as named
 // in URLs, to the index of its field in types.OpportunityMarks. markNames
 // lists the names in field order. Both are read from the struct (see
 // findMarks), so a mark added to it can be changed over the API.
 var markIndex, markNames = findMarks()
 // OpportunityPage is a page of opportunities, as GET /opportunities returns
 // it. HasMore reports that another page follows, at Offset+Limit.
 //
 // Batch is the hash of the newest change batch when the page was read, or
 // "" if none has been recorded: given as changed_after, it gets the
 // opportunities changed since. A page of changed opportunities is all of
 // them, so Limit is their number and HasMore is false.
 type OpportunityPage struct {
 	Opportunities []types.Opportunity `json:"opportunities"`
 	Offset        int                 `json:"offset"`
 	Limit         int                 `json:"limit"`
 	HasMore       bool                `json:"has_more"`
 	Batch         string              `json:"batch"`
 }
 // OpportunitySearch is the result of a general search, as GET
 // /opportunities/search returns it: the UUIDs of the opportunities that
 // match, in no particular order. MatchesAll reports a search with nothing to
 // search for (blank, or only unknown prompts), which matches every
 // opportunity; UUIDs is then empty. UnknownPrompts are the prompts in the
 // search that aren't known, as typed, without their colon. Prompts and
 // Syntax are the help for the search box.
 type OpportunitySearch struct {
 	UUIDs          []string         `json:"uuids"`
 	MatchesAll     bool             `json:"matches_all"`
 	UnknownPrompts []string         `json:"unknown_prompts"`
 	Prompts        []search.Prompt  `json:"prompts"`
 	Syntax         []search.Example `json:"syntax"`
 }
 // opportunityRoutes wires HTTP handlers to the opportunity store.
 // Opportunities are read-only; only their marks can be changed.
 type opportunityRoutes struct {
 	store  *store.Store
 	events *events.Bus // nil: nothing is published
 	// marksMu makes changing one mark (read the marks, change one, write
 	// them all back) atomic, so concurrent changes to different marks of an
 	// opportunity don't undo each other.
 	marksMu sync.Mutex
 }
 // newOpportunityRoutes returns the opportunity handlers for
 // opportunityStore, publishing mark changes to bus if it isn't nil.
 func newOpportunityRoutes(
 	opportunityStore *store.Store, bus *events.Bus,
 ) *opportunityRoutes {
 	return &opportunityRoutes{
 		store:  opportunityStore,
 		events: bus,
 	}
 }
 // Routes registers the opportunity routes on router (mounted under /api).
 func (h *opportunityRoutes) Routes(router chi.Router) {
 	router.Get("/opportunities", h.list)
 	router.Get("/opportunities/search", h.search)
 	router.Get("/opportunities/{uuid}", h.get)
 	router.Put("/opportunities/{uuid}/marks/{mark}", h.setMark)
 	router.Delete("/opportunities/{uuid}/marks/{mark}", h.clearMark)
 }
 // list serves GET /opportunities: a page of opportunities, newest release
 // first, with their classifications and marks but not their releases. The
 // page is set by the offset and limit (default 50, at most 100) query
 // parameters, and released_from leaves out opportunities whose latest release
 // is before it. With changed_after, it serves the opportunities changed since
 // that batch instead (see listChanged).
 //
 //	@Summary		List opportunities
 //	@Description	A page of opportunities, newest release first, with their
 //	@Description	classifications and marks but not their releases. has_more
 //	@Description	says whether another page follows, at offset+limit. batch
 //	@Description	is the newest change batch when the page was read.
 //	@Description	With changed_after (a batch from a page or an
 //	@Description	OpportunitiesUpdated event), the page is instead every
 //	@Description	opportunity changed since that batch (new, given new
 //	@Description	releases or with changed marks), and batch is the newest
 //	@Description	batch they include. offset and limit can't be given with
 //	@Description	changed_after. An unknown batch is 410 Gone: reload the
 //	@Description	whole list. released_from (RFC 3339, inclusive) leaves out
 //	@Description	opportunities whose latest release is before it, in a page
 //	@Description	or with changed_after.
 //	@Tags			opportunities
 //	@Produce		json
 //	@Param			offset			query		int		false	"Opportunities to skip"	minimum(0)	default(0)
 //	@Param			limit			query		int		false	"Page size"				minimum(1)	maximum(100)	default(50)
 //	@Param			changed_after	query		string	false	"List the opportunities changed after this batch instead"
 //	@Param			released_from	query		string	false	"Only opportunities whose latest release is at or after this time"	format(date-time)
 //	@Success		200				{object}	OpportunityPage
 //	@Failure		400				{object}	httpx.Problem	"Invalid offset, limit or released_from, or offset or limit with changed_after"
 //	@Failure		410				{object}	httpx.Problem	"Unknown batch"
 //	@Failure		500				{object}	httpx.Problem
 //	@Router			/opportunities [get]
 func (h *opportunityRoutes) list(w http.ResponseWriter, r *http.Request) {
 	query := r.URL.Query()
 	releasedFrom, err := timeParam(
 		"released_from", query.Get("released_from"),
 	)
 	if err != nil {
 		httpx.WriteProblem(w, r, http.StatusBadRequest, err.Error())
 		return
 	}
 	if query.Has("changed_after") {
 		h.listChanged(w, r, releasedFrom)
 		return
 	}
 	offset, offsetErr := intParam(query.Get("offset"), 0, 0, 1<<30)
 	limit, limitErr := intParam(query.Get("limit"), 50, 1, 100)
 	err = errors.Join(offsetErr, limitErr)
 	if err != nil {
 		httpx.WriteProblem(w, r, http.StatusBadRequest, err.Error())
 		return
 	}
 	// Read first, so nothing changed while the page is read is missed by
 	// asking for the changes after it.
 	batch, err := h.store.LatestChangeBatch(r.Context())
 	if err != nil {
 		httpx.WriteProblem(w, r, http.StatusInternalServerError, "")
 		return
 	}
 	// One more than the page, to tell whether another page follows.
 	opportunities, err := h.store.ListOpportunities(
 		r.Context(),
 		store.OpportunityFilter{
 			Limit:        limit + 1,
 			Offset:       offset,
 			ReleasedFrom: releasedFrom,
 		},
 	)
 	if err != nil {
 		httpx.WriteProblem(w, r, http.StatusInternalServerError, "")
 		return
 	}
 	page := OpportunityPage{
 		Opportunities: opportunities,
 		Offset:        offset,
 		Limit:         limit,
 		HasMore:       len(opportunities) > limit,
 		Batch:         batch,
 	}
 	if page.HasMore {
 		page.Opportunities = opportunities[:limit]
 	}
 	httpx.WriteJSON(w, http.StatusOK, page)
 }
 // listChanged serves GET /opportunities?changed_after=batch: every
 // opportunity changed after the change batch, as one page, with the newest
 // batch it includes, leaving out those whose latest release is before
 // releasedFrom unless it is zero. An unknown batch, such as one from another
 // database, is 410 Gone, as the client can't catch up from it.
 func (h *opportunityRoutes) listChanged(
 	w http.ResponseWriter, r *http.Request, releasedFrom time.Time,
 ) {
 	query := r.URL.Query()
 	if query.Has("offset") || query.Has("limit") {
 		httpx.WriteProblem(
 			w,
 			r,
 			http.StatusBadRequest,
 			"offset and limit can't be used with changed_after",
 		)
 		return
 	}
 	opportunities, batch, err := h.store.ListChangedOpportunities(
 		r.Context(), query.Get("changed_after"), releasedFrom,
 	)
 	if errors.Is(err, store.ErrUnknownBatch) {
 		httpx.WriteProblem(
 			w,
 			r,
 			http.StatusGone,
 			"unknown batch: reload the whole list",
 		)
 		return
 	}
 	if err != nil {
 		httpx.WriteProblem(w, r, http.StatusInternalServerError, "")
 		return
 	}
 	httpx.WriteJSON(w, http.StatusOK, OpportunityPage{
 		Opportunities: opportunities,
 		Limit:         len(opportunities),
 		Batch:         batch,
 	})
 }
 // search serves GET /opportunities/search: the UUIDs of the opportunities
 // matching the general search in the q query parameter (see package
 // search), with the help for the search box. released_from bounds it as it
 // bounds list.
 //
 //	@Summary		Search opportunities
 //	@Description	The UUIDs of the opportunities matching a general search,
 //	@Description	in no particular order. q is words and "quoted phrases",
 //	@Description	each optionally with a prompt (title:, buyer:, buyer_id:,
 //	@Description	cpv:) and a leading ! to leave out what it matches; prompts
 //	@Description	and syntax explain it. A search with nothing to search for
 //	@Description	(blank, or only unknown prompts) has matches_all set and no
 //	@Description	uuids. unknown_prompts lists the prompts that aren't known,
 //	@Description	as typed; their terms are ignored. released_from (RFC 3339,
 //	@Description	inclusive) leaves out opportunities whose latest release is
 //	@Description	before it.
 //	@Tags			opportunities
 //	@Produce		json
 //	@Param			q				query		string	false	"The search"
 //	@Param			released_from	query		string	false	"Only opportunities whose latest release is at or after this time"	format(date-time)
 //	@Success		200				{object}	OpportunitySearch
 //	@Failure		400				{object}	httpx.Problem	"Invalid released_from"
 //	@Failure		500				{object}	httpx.Problem
 //	@Router			/opportunities/search [get]
 func (h *opportunityRoutes) search(w http.ResponseWriter, r *http.Request) {
 	query := r.URL.Query()
 	releasedFrom, err := timeParam(
 		"released_from", query.Get("released_from"),
 	)
 	if err != nil {
 		httpx.WriteProblem(w, r, http.StatusBadRequest, err.Error())
 		return
 	}
 	parsed := search.Parse(query.Get("q"))
 	result := OpportunitySearch{
 		UUIDs:          []string{},
 		MatchesAll:     parsed.MatchesAll(),
 		UnknownPrompts: parsed.UnknownPrompts,
 		Prompts:        search.Prompts,
 		Syntax:         search.Syntax,
 	}
 	if !result.MatchesAll {
 		result.UUIDs, err = h.store.SearchOpportunities(
 			r.Context(), parsed, releasedFrom,
 		)
 		if err != nil {
 			httpx.WriteProblem(w, r, http.StatusInternalServerError, "")
 			return
 		}
 	}
 	httpx.WriteJSON(w, http.StatusOK, result)
 }
 // get serves GET /opportunities/{uuid}: one opportunity with its
 // classifications, marks and releases.
 //
 //	@Summary		Get an opportunity
 //	@Description	One opportunity with its classifications, marks and all of its
 //	@Description	releases.
 //	@Tags			opportunities
 //	@Produce		json
 //	@Param			uuid	path		string	true	"The opportunity's UUID (its OCID)"
 //	@Success		200		{object}	types.Opportunity
 //	@Failure		404		{object}	httpx.Problem	"No such opportunity"
 //	@Failure		500		{object}	httpx.Problem
 //	@Router			/opportunities/{uuid} [get]
 func (h *opportunityRoutes) get(w http.ResponseWriter, r *http.Request) {
 	opportunity, err := h.store.GetOpportunity(
 		r.Context(), chi.URLParam(r, "uuid"),
 	)
 	if err != nil {
 		h.fail(w, r, err)
 		return
 	}
 	httpx.WriteJSON(w, http.StatusOK, opportunity)
 }
 // setMark serves PUT /opportunities/{uuid}/marks/{mark}, setting the mark.
 //
 //	@Summary		Set a mark
 //	@Description	Sets one of the opportunity's marks, keeping the others, and
 //	@Description	returns all of them. Setting a mark that is already set
 //	@Description	changes nothing. won and lost exclude each other.
 //	@Tags			marks
 //	@Produce		json
 //	@Security		CSRF
 //	@Param			uuid	path		string	true	"The opportunity's UUID (its OCID)"
 //	@Param			mark	path		string	true	"The mark"	Enums(hidden, skipped, interested, bid_submitted, won, lost)
 //	@Success		200		{object}	types.OpportunityMarks
 //	@Failure		403		{object}	httpx.Problem	"No valid CSRF token"
 //	@Failure		404		{object}	httpx.Problem	"No such opportunity or mark"
 //	@Failure		409		{object}	httpx.Problem	"won and lost together"
 //	@Failure		500		{object}	httpx.Problem
 //	@Router			/opportunities/{uuid}/marks/{mark} [put]
 func (h *opportunityRoutes) setMark(w http.ResponseWriter, r *http.Request) {
 	h.changeMark(w, r, true)
 }
 // clearMark serves DELETE /opportunities/{uuid}/marks/{mark}, clearing the
 // mark.
 //
 //	@Summary		Clear a mark
 //	@Description	Clears one of the opportunity's marks, keeping the others, and
 //	@Description	returns all of them. Clearing a mark that isn't set changes
 //	@Description	nothing.
 //	@Tags			marks
 //	@Produce		json
 //	@Security		CSRF
 //	@Param			uuid	path		string	true	"The opportunity's UUID (its OCID)"
 //	@Param			mark	path		string	true	"The mark"	Enums(hidden, skipped, interested, bid_submitted, won, lost)
 //	@Success		200		{object}	types.OpportunityMarks
 //	@Failure		403		{object}	httpx.Problem	"No valid CSRF token"
 //	@Failure		404		{object}	httpx.Problem	"No such opportunity or mark"
 //	@Failure		500		{object}	httpx.Problem
 //	@Router			/opportunities/{uuid}/marks/{mark} [delete]
 func (h *opportunityRoutes) clearMark(w http.ResponseWriter, r *http.Request) {
 	h.changeMark(w, r, false)
 }
 // changeMark sets the mark named in the URL to value, keeping the
 // opportunity's other marks, and responds with all of its marks. Setting won
 // while lost is set, or lost while won is, is refused with 409 Conflict. A
 // change is published as an events.OpportunityMarksUpdated event; one that
 // changes nothing is neither stored nor published.
 func (h *opportunityRoutes) changeMark(
 	w http.ResponseWriter, r *http.Request, value bool,
 ) {
 	uuid, name := chi.URLParam(r, "uuid"), chi.URLParam(r, "mark")
 	h.marksMu.Lock()
 	defer h.marksMu.Unlock()
 	opportunity, err := h.store.GetOpportunity(r.Context(), uuid)
 	if err != nil {
 		h.fail(w, r, err)
 		return
 	}
 	marks := opportunity.Marks
 	marks.UUID = uuid
 	before := marks
 	if !setMark(&marks, name, value) {
 		httpx.WriteProblem(
 			w,
 			r,
 			http.StatusNotFound,
 			fmt.Sprintf("no mark %q: marks are %v", name, markNames),
 		)
 		return
 	}
 	if marks == before {
 		httpx.WriteJSON(w, http.StatusOK, marks)
 		return
 	}
 	err = h.store.SetMarks(r.Context(), marks)
 	if err != nil {
 		h.fail(w, r, err)
 		return
 	}
 	httpx.WriteJSON(w, http.StatusOK, marks)
 	// Still under marksMu, so events are published in the order the
 	// changes were made.
 	if h.events != nil {
 		h.events.Publish(events.Event{
 			Type:    events.OpportunityMarksUpdated,
 			Payload: marks,
 		})
 	}
 }
 // findMarks returns the marks of types.OpportunityMarks: every bool field,
 // named by its JSON tag, mapped to the field's index, and the names in field
 // order.
 func findMarks() (map[string]int, []string) {
 	index := map[string]int{}
 	var names []string
 	marksType := reflect.TypeFor[types.OpportunityMarks]()
 	for i := range marksType.NumField() {
 		field := marksType.Field(i)
 		if field.Type.Kind() != reflect.Bool {
 			continue
 		}
 		name, _, _ := strings.Cut(field.Tag.Get("json"), ",")
 		index[name] = i
 		names = append(names, name)
 	}
 	return index, names
 }
 // setMark sets the mark of marks named name (see markIndex) to value, and
 // reports whether name is a mark.
 func setMark(marks *types.OpportunityMarks, name string, value bool) bool {
 	i, ok := markIndex[name]
 	if !ok {
 		return false
 	}
 	reflect.ValueOf(marks).Elem().Field(i).SetBool(value)
 	return true
 }
 // fail writes the problem response for a store error.
 func (h *opportunityRoutes) fail(
 	w http.ResponseWriter, r *http.Request, err error,
 ) {
 	switch {
 	case errors.Is(err, store.ErrNotFound):
 		httpx.WriteProblem(w, r, http.StatusNotFound, "opportunity not found")
 	case errors.Is(err, types.ErrWonAndLost):
 		httpx.WriteProblem(
 			w,
 			r,
 			http.StatusConflict,
 			"an opportunity cannot be both won and lost",
 		)
 	default:
 		httpx.WriteProblem(w, r, http.StatusInternalServerError, "")
 	}
 }
@@ -0,0 +1,537 @@
 package api
 import (
 	"context"
 	"encoding/json"
 	"fmt"
 	"net/http"
 	"net/http/httptest"
 	"path/filepath"
 	"slices"
 	"strings"
 	"sync"
 	"testing"
 	"time"
 	"github.com/go-chi/chi/v5"
 	"git.avsdev.uk/Go/job-search/internal/events"
 	"git.avsdev.uk/Go/job-search/internal/store"
 	"git.avsdev.uk/Go/job-search/internal/types"
 )
 // opportunitiesServer serves the API, given opts as well as its store, over
 // a store holding count opportunities, ocds-1 the oldest, and returns its
 // URL.
 func opportunitiesServer(t *testing.T, count int, opts ...Option) string {
 	t.Helper()
 	opportunityStore, err := store.Open(
 		context.Background(),
 		filepath.Join(t.TempDir(), "api.db"),
 		store.WithAutoMigrate(),
 	)
 	if err != nil {
 		t.Fatal(err)
 	}
 	t.Cleanup(func() {
 		opportunityStore.Close()
 	})
 	for i := 1; i <= count; i++ {
 		opportunity := &types.Opportunity{
 			UUID:     fmt.Sprintf("ocds-%d", i),
 			UUIDType: "ocid",
 			Title:    fmt.Sprintf("Opportunity %d", i),
 			Source:   "FindATender",
 			// Two codes, shared: ocds-1, ocds-3... have 72000000, the
 			// others 03000000.
 			Classifications: []types.ClassificationCode{
 				{
 					IsCPV:          true,
 					Code:           []string{"03000000", "72000000"}[i%2],
 					Classification: []string{"Agriculture", "IT services"}[i%2],
 				},
 			},
 			Releases: []types.OpportunityRelease{
 				{
 					ReleaseID:   "r1",
 					ReleaseDate: time.Date(2026, 10, i, 12, 0, 0, 0, time.UTC),
 					JSON:        "{}",
 				},
 			},
 		}
 		_, err := opportunityStore.SaveOpportunity(
 			context.Background(), opportunity,
 		)
 		if err != nil {
 			t.Fatal(err)
 		}
 	}
 	router := chi.NewRouter()
 	New(append([]Option{WithStore(opportunityStore)}, opts...)...).Routes(router)
 	server := httptest.NewServer(router)
 	t.Cleanup(server.Close)
 	return server.URL
 }
 // request sends a bodyless request and decodes a JSON reply into reply (if
 // not nil), returning the status.
 func request(t *testing.T, method, url string, reply any) int {
 	t.Helper()
 	req, err := http.NewRequest(method, url, nil)
 	if err != nil {
 		t.Fatal(err)
 	}
 	res, err := http.DefaultClient.Do(req)
 	if err != nil {
 		t.Fatal(err)
 	}
 	defer res.Body.Close()
 	if reply != nil && res.StatusCode < 300 {
 		if err := json.NewDecoder(res.Body).Decode(reply); err != nil {
 			t.Fatal(err)
 		}
 	}
 	return res.StatusCode
 }
 // uuids returns the UUIDs of a page's opportunities.
 func uuids(page OpportunityPage) []string {
 	var ids []string
 	for _, opportunity := range page.Opportunities {
 		ids = append(ids, opportunity.UUID)
 	}
 	return ids
 }
 // Opportunities are listed newest first, a page at a time, from
 // released_from if given (ocds-N is released 2026-10-0N at 12:00 UTC).
 func TestListOpportunitiesPaging(t *testing.T) {
 	url := opportunitiesServer(t, 5)
 	all := []string{"ocds-5", "ocds-4", "ocds-3", "ocds-2", "ocds-1"}
 	tests := []struct {
 		query   string
 		want    []string
 		hasMore bool
 	}{
 		{"?limit=2", []string{"ocds-5", "ocds-4"}, true},
 		{"?limit=2&offset=2", []string{"ocds-3", "ocds-2"}, true},
 		{"?limit=2&offset=4", []string{"ocds-1"}, false},
 		{"?limit=5", all, false},
 		{"?offset=10", nil, false},
 		{"", all, false},
 		{"?released_from=", all, false},
 		{"?released_from=2026-10-03T12:00:00Z", []string{"ocds-5", "ocds-4", "ocds-3"}, false},
 		{"?released_from=2026-10-03T13:00:00%2B01:00", []string{"ocds-5", "ocds-4", "ocds-3"}, false},
 		{"?released_from=2026-10-03T12:00:01Z", []string{"ocds-5", "ocds-4"}, false},
 		{"?released_from=2026-10-02T00:00:00Z&limit=1", []string{"ocds-5"}, true},
 		{"?released_from=2026-10-02T00:00:00Z&limit=2&offset=2", []string{"ocds-3", "ocds-2"}, false},
 	}
 	for _, tt := range tests {
 		var page OpportunityPage
 		status := request(t, "GET", url+"/opportunities"+tt.query, &page)
 		if status != 200 {
 			t.Fatalf("%s: status %d", tt.query, status)
 		}
 		got := strings.Join(uuids(page), ",")
 		want := strings.Join(tt.want, ",")
 		if got != want || page.HasMore != tt.hasMore {
 			t.Errorf(
 				"%s: got %s, has_more %v; want %s, %v",
 				tt.query, got, page.HasMore, want, tt.hasMore,
 			)
 		}
 	}
 	var page OpportunityPage
 	request(t, "GET", url+"/opportunities", &page)
 	if page.Limit != 50 || page.Offset != 0 {
 		t.Errorf(
 			"defaults: limit %d, offset %d; want 50, 0", page.Limit, page.Offset,
 		)
 	}
 }
 // An empty page is [] in JSON, not null.
 func TestListOpportunitiesEmpty(t *testing.T) {
 	url := opportunitiesServer(t, 0)
 	res, err := http.Get(url + "/opportunities")
 	if err != nil {
 		t.Fatal(err)
 	}
 	defer res.Body.Close()
 	var raw map[string]json.RawMessage
 	json.NewDecoder(res.Body).Decode(&raw)
 	if string(raw["opportunities"]) != "[]" {
 		t.Errorf("opportunities = %s; want []", raw["opportunities"])
 	}
 }
 // Bad paging or released_from is 400. An unencoded "+" in released_from
 // arrives as a space, so it is refused too.
 func TestListOpportunitiesBadPaging(t *testing.T) {
 	url := opportunitiesServer(t, 1)
 	queries := []string{
 		"?limit=0",
 		"?limit=101",
 		"?limit=x",
 		"?offset=-1",
 		"?released_from=x",
 		"?released_from=2026-10-03",
 		"?released_from=2026-10-03T13:00:00+01:00",
 	}
 	for _, query := range queries {
 		status := request(t, "GET", url+"/opportunities"+query, nil)
 		if status != 400 {
 			t.Errorf("%s: status %d; want 400", query, status)
 		}
 	}
 }
 // A page names the newest change batch, and changed_after lists what
 // changed after a batch: here, a change of marks.
 func TestListChangedOpportunities(t *testing.T) {
 	url := opportunitiesServer(t, 3)
 	var page OpportunityPage
 	request(t, "GET", url+"/opportunities", &page)
 	if page.Batch == "" {
 		t.Fatal("the page names no batch")
 	}
 	var changed OpportunityPage
 	status := request(
 		t, "GET", url+"/opportunities?changed_after="+page.Batch, &changed,
 	)
 	if status != 200 || len(changed.Opportunities) != 0 ||
 		changed.Opportunities == nil || changed.Batch != page.Batch {
 		t.Fatalf(
 			"nothing changed: status %d, %+v; want [] and the same batch",
 			status, changed,
 		)
 	}
 	request(t, "PUT", url+"/opportunities/ocds-2/marks/skipped", nil)
 	status = request(
 		t, "GET", url+"/opportunities?changed_after="+page.Batch, &changed,
 	)
 	if status != 200 || strings.Join(uuids(changed), ",") != "ocds-2" ||
 		!changed.Opportunities[0].Marks.Skipped {
 		t.Fatalf("after a mark: status %d, %+v; want ocds-2", status, changed)
 	}
 	if changed.Batch == page.Batch || changed.Limit != 1 || changed.HasMore {
 		t.Errorf(
 			"after a mark: batch %q, limit %d, has_more %v; "+
 				"want a newer batch, 1, false",
 			changed.Batch, changed.Limit, changed.HasMore,
 		)
 	}
 	// released_from leaves out ocds-2 (released on the 2nd), not the batch.
 	var bounded OpportunityPage
 	status = request(
 		t,
 		"GET",
 		url+"/opportunities?released_from=2026-10-03T00:00:00Z"+
 			"&changed_after="+page.Batch,
 		&bounded,
 	)
 	if status != 200 || len(bounded.Opportunities) != 0 ||
 		bounded.Batch != changed.Batch {
 		t.Errorf(
 			"released_from the 3rd: status %d, %+v; want [] and batch %q",
 			status, bounded, changed.Batch,
 		)
 	}
 	status = request(
 		t,
 		"GET",
 		url+"/opportunities?changed_after="+page.Batch+
 			"&released_from=2026-10-02T12:00:00Z",
 		&bounded,
 	)
 	if status != 200 || strings.Join(uuids(bounded), ",") != "ocds-2" {
 		t.Errorf(
 			"released_from the 2nd: status %d, %+v; want ocds-2",
 			status, bounded,
 		)
 	}
 	var latest OpportunityPage
 	request(t, "GET", url+"/opportunities?limit=1", &latest)
 	if latest.Batch != changed.Batch {
 		t.Errorf("page batch %q; want %q", latest.Batch, changed.Batch)
 	}
 }
 // An unknown batch is 410 Gone, and paging can't be used with changed_after;
 // a bad released_from is 400 there too.
 func TestListChangedOpportunitiesRefused(t *testing.T) {
 	url := opportunitiesServer(t, 1)
 	var page OpportunityPage
 	request(t, "GET", url+"/opportunities", &page)
 	tests := []struct {
 		query string
 		want  int
 	}{
 		{"?changed_after=unknown", http.StatusGone},
 		{"?changed_after=", http.StatusGone},
 		{"?changed_after=" + page.Batch + "&limit=5", http.StatusBadRequest},
 		{"?offset=0&changed_after=" + page.Batch, http.StatusBadRequest},
 		{"?changed_after=" + page.Batch + "&released_from=x", http.StatusBadRequest},
 	}
 	for _, tt := range tests {
 		status := request(t, "GET", url+"/opportunities"+tt.query, nil)
 		if status != tt.want {
 			t.Errorf("%s: status %d; want %d", tt.query, status, tt.want)
 		}
 	}
 }
 // One opportunity comes with its releases; a missing one is 404.
 func TestGetOpportunity(t *testing.T) {
 	url := opportunitiesServer(t, 2)
 	var opportunity types.Opportunity
 	status := request(t, "GET", url+"/opportunities/ocds-2", &opportunity)
 	if status != 200 {
 		t.Fatalf("status %d", status)
 	}
 	if opportunity.UUID != "ocds-2" || len(opportunity.Releases) != 1 {
 		t.Errorf("got %+v; want ocds-2 with its release", opportunity)
 	}
 	status = request(t, "GET", url+"/opportunities/nope", nil)
 	if status != 404 {
 		t.Errorf("missing: status %d; want 404", status)
 	}
 }
 // Opportunities are read-only: there is no way to write one.
 func TestOpportunitiesReadOnly(t *testing.T) {
 	url := opportunitiesServer(t, 1)
 	for _, method := range []string{"POST", "PUT", "PATCH", "DELETE"} {
 		status := request(t, method, url+"/opportunities/ocds-1", nil)
 		if status != http.StatusMethodNotAllowed {
 			t.Errorf("%s: status %d; want 405", method, status)
 		}
 	}
 	if status := request(t, "POST", url+"/opportunities", nil); status != 405 {
 		t.Errorf("POST /opportunities: status %d; want 405", status)
 	}
 }
 // A mark is set with PUT and cleared with DELETE, leaving the others, and
 // both reply with all the marks.
 func TestChangeMark(t *testing.T) {
 	url := opportunitiesServer(t, 1)
 	marks := url + "/opportunities/ocds-1/marks/"
 	var got types.OpportunityMarks
 	request(t, "PUT", marks+"interested", &got)
 	request(t, "PUT", marks+"bid_submitted", &got)
 	if !got.Interested || !got.BidSubmitted || got.UUID != "ocds-1" {
 		t.Fatalf("after two PUTs: %+v", got)
 	}
 	request(t, "DELETE", marks+"interested", &got)
 	if got.Interested || !got.BidSubmitted {
 		t.Errorf("after DELETE: %+v; want only bid_submitted", got)
 	}
 	// The change is stored, and setting a set mark again is harmless.
 	request(t, "PUT", marks+"bid_submitted", &got)
 	var opportunity types.Opportunity
 	request(t, "GET", url+"/opportunities/ocds-1", &opportunity)
 	if !opportunity.Marks.BidSubmitted || opportunity.Marks.Interested {
 		t.Errorf("stored marks: %+v", opportunity.Marks)
 	}
 }
 // Won and lost exclude each other; unknown marks and opportunities are 404.
 func TestChangeMarkRefused(t *testing.T) {
 	url := opportunitiesServer(t, 1)
 	marks := url + "/opportunities/ocds-1/marks/"
 	request(t, "PUT", marks+"won", nil)
 	status := request(t, "PUT", marks+"lost", nil)
 	if status != http.StatusConflict {
 		t.Errorf("lost while won: status %d; want 409", status)
 	}
 	status = request(t, "PUT", marks+"shiny", nil)
 	if status != 404 {
 		t.Errorf("unknown mark: status %d; want 404", status)
 	}
 	status = request(t, "PUT", url+"/opportunities/nope/marks/won", nil)
 	if status != 404 {
 		t.Errorf("unknown opportunity: status %d; want 404", status)
 	}
 	// Clearing won makes lost allowed.
 	request(t, "DELETE", marks+"won", nil)
 	status = request(t, "PUT", marks+"lost", nil)
 	if status != 200 {
 		t.Errorf("lost after clearing won: status %d; want 200", status)
 	}
 }
 // Concurrent changes to different marks of one opportunity all stick.
 func TestChangeMarkConcurrent(t *testing.T) {
 	url := opportunitiesServer(t, 1)
 	names := []string{"hidden", "skipped", "interested", "bid_submitted", "won"}
 	var wg sync.WaitGroup
 	for _, name := range names {
 		wg.Add(1)
 		go func() {
 			defer wg.Done()
 			request(t, "PUT", url+"/opportunities/ocds-1/marks/"+name, nil)
 		}()
 	}
 	wg.Wait()
 	var opportunity types.Opportunity
 	request(t, "GET", url+"/opportunities/ocds-1", &opportunity)
 	m := opportunity.Marks
 	if !m.Hidden || !m.Skipped || !m.Interested || !m.BidSubmitted || !m.Won {
 		t.Errorf("marks after concurrent PUTs: %+v; want all five set", m)
 	}
 }
 // The marks are OpportunityMarks' bool fields, by JSON name, and setting one
 // by name sets only that field.
 func TestFindMarks(t *testing.T) {
 	want := "hidden,skipped,interested,bid_submitted,won,lost"
 	if got := strings.Join(markNames, ","); got != want {
 		t.Fatalf("markNames = %s; want %s", got, want)
 	}
 	for _, name := range markNames {
 		var marks types.OpportunityMarks
 		if !setMark(&marks, name, true) {
 			t.Fatalf("setMark(%q) = false", name)
 		}
 		encoded, err := json.Marshal(marks)
 		if err != nil {
 			t.Fatal(err)
 		}
 		var fields map[string]any
 		json.Unmarshal(encoded, &fields)
 		for field, value := range fields {
 			if set := value == true; set != (field == name) {
 				t.Errorf("setting %s: %s = %v", name, field, value)
 			}
 		}
 	}
 	var marks types.OpportunityMarks
 	if setMark(&marks, "uuid", true) || setMark(&marks, "shiny", true) {
 		t.Error("setMark accepted a name that isn't a mark")
 	}
 }
 // A change to a mark is published with all the marks; a change that changes
 // nothing, or is refused, isn't.
 func TestChangeMarkPublishes(t *testing.T) {
 	var bus events.Bus
 	var mu sync.Mutex
 	var published []types.OpportunityMarks
 	bus.Subscribe(func(event events.Event) {
 		if event.Type != events.OpportunityMarksUpdated {
 			t.Errorf("event type %q", event.Type)
 		}
 		mu.Lock()
 		defer mu.Unlock()
 		published = append(published, event.Payload.(types.OpportunityMarks))
 	})
 	url := opportunitiesServer(t, 1, WithEvents(&bus))
 	marks := url + "/opportunities/ocds-1/marks/"
 	request(t, "PUT", marks+"interested", nil)    // published
 	request(t, "PUT", marks+"interested", nil)    // already set: not published
 	request(t, "DELETE", marks+"skipped", nil)    // not set: not published
 	request(t, "PUT", marks+"shiny", nil)         // no such mark: refused
 	request(t, "PUT", marks+"won", nil)           // published
 	request(t, "PUT", marks+"lost", nil)          // won and lost: refused
 	request(t, "DELETE", marks+"interested", nil) // published
 	mu.Lock()
 	defer mu.Unlock()
 	want := []types.OpportunityMarks{
 		{
 			UUID:       "ocds-1",
 			Interested: true,
 		},
 		{
 			UUID:       "ocds-1",
 			Interested: true,
 			Won:        true,
 		},
 		{
 			UUID: "ocds-1",
 			Won:  true,
 		},
 	}
 	if len(published) != len(want) {
 		t.Fatalf("published %+v; want %+v", published, want)
 	}
 	for i := range want {
 		if published[i] != want[i] {
 			t.Errorf("event %d: %+v; want %+v", i, published[i], want[i])
 		}
 	}
 }
 // GET /opportunities/search returns the matching UUIDs, bounded by
 // released_from, with the help; a search with nothing to search for matches
 // all, and unknown prompts are listed.
 func TestSearchOpportunities(t *testing.T) {
 	url := opportunitiesServer(t, 5) + "/opportunities/search"
 	from3 := "&released_from=2026-10-03T00:00:00Z"
 	cases := []struct {
 		query      string
 		uuids      []string
 		matchesAll bool
 		unknown    []string
 	}{
 		{"", []string{}, true, []string{}},
 		{"?q=", []string{}, true, []string{}},
 		{"?q=cpv:72", []string{"ocds-1", "ocds-3", "ocds-5"}, false, []string{}},
 		{"?q=cpv:72" + from3, []string{"ocds-3", "ocds-5"}, false, []string{}},
 		{"?q=opportunity%20!cpv:72", []string{"ocds-2", "ocds-4"}, false, []string{}},
 		{"?q=nothing", []string{}, false, []string{}},
 		{"?q=foo:x%20Bar:y", []string{}, true, []string{"foo", "Bar"}},
 		{"?q=foo:x%20title:3", []string{"ocds-3"}, false, []string{"foo"}},
 	}
 	for _, tc := range cases {
 		var got OpportunitySearch
 		status := request(t, http.MethodGet, url+tc.query, &got)
 		slices.Sort(got.UUIDs)
 		if status != http.StatusOK || !slices.Equal(got.UUIDs, tc.uuids) ||
 			got.MatchesAll != tc.matchesAll ||
 			!slices.Equal(got.UnknownPrompts, tc.unknown) {
 			t.Errorf("%s: status %d, %+v; want %v, matches_all %v, unknown %v",
 				tc.query, status, got, tc.uuids, tc.matchesAll, tc.unknown)
 		}
 		if len(got.Prompts) == 0 || len(got.Syntax) == 0 {
 			t.Errorf("%s: no help: %+v", tc.query, got)
 		}
 	}
 	// Empty lists are [], never null.
 	var raw map[string]json.RawMessage
 	request(t, http.MethodGet, url+"?q=nothing", &raw)
 	for _, field := range []string{"uuids", "unknown_prompts"} {
 		if string(raw[field]) != "[]" {
 			t.Errorf("%s = %s; want []", field, raw[field])
 		}
 	}
 	status := request(t, http.MethodGet, url+"?q=x&released_from=x", nil)
 	if status != http.StatusBadRequest {
 		t.Errorf("bad released_from: status %d; want 400", status)
 	}
 }
@@ -0,0 +1,215 @@
 package cli
 import (
 	"log/slog"
 	"os"
 	"path/filepath"
 	"strings"
 	"testing"
 	"github.com/spf13/cobra"
 	"github.com/spf13/viper"
 	"git.avsdev.uk/Go/job-search/internal/logging"
 )
 // The commands themselves are tested through the real command tree in
 // cmd/jobsearch. These tests cover what that can't reach easily: switching
 // to file logging, and Confirm's answers.
 // setupDir changes into a new temporary directory holding an empty .env and
 // config.yaml with the given contents, and isolates Viper and logging.
 func setupDir(t *testing.T, config string) {
 	t.Helper()
 	t.Chdir(t.TempDir())
 	writeFile(t, ".env", "")
 	writeFile(t, "config.yaml", config)
 	viper.Reset()
 	oldLog := slog.Default()
 	t.Cleanup(func() {
 		viper.Reset()
 		if err := logging.Shutdown(); err != nil {
 			t.Error(err)
 		}
 		slog.SetDefault(oldLog)
 		logging.SetLevel(slog.LevelInfo)
 	})
 }
 func writeFile(t *testing.T, path, contents string) {
 	t.Helper()
 	if err := os.WriteFile(path, []byte(contents), 0o600); err != nil {
 		t.Fatal(err)
 	}
 }
 // newCommand returns a command with the shared flags bound to opts, as the
 // root command has, after parsing args.
 func newCommand(t *testing.T, opts *Options, args ...string) *cobra.Command {
 	t.Helper()
 	cmd := &cobra.Command{
 		Use: "test",
 	}
 	opts.Bind(cmd)
 	if err := cmd.ParseFlags(args); err != nil {
 		t.Fatal(err)
 	}
 	return cmd
 }
 func TestRunSetupSwitchesToLogFile(t *testing.T) {
 	setupDir(t, "logging:\n  level: info\n  file: ./app.log\n")
 	opts := &Options{}
 	cmd := newCommand(t, opts)
 	conf, err := opts.RunSetup(cmd, true, false)
 	if err != nil {
 		t.Fatal(err)
 	}
 	if conf.Logging.File != "./app.log" {
 		t.Errorf("Logging.File = %q", conf.Logging.File)
 	}
 	if !cmd.SilenceUsage {
 		t.Error("RunSetup did not silence usage for runtime errors")
 	}
 	if !logging.IsFileLogger() {
 		t.Fatal("logging was not switched to the log file")
 	}
 	slog.Info("written to the log file")
 	if err := opts.Finish(); err != nil {
 		t.Fatal(err)
 	}
 	if logging.IsFileLogger() {
 		t.Error("Finish left the log file open")
 	}
 	b, err := os.ReadFile("app.log")
 	if err != nil {
 		t.Fatal(err)
 	}
 	if !strings.Contains(string(b), "written to the log file") {
 		t.Errorf("app.log = %q", b)
 	}
 }
 func TestRunSetupWithoutLogFileStaysOnTerminal(t *testing.T) {
 	setupDir(t, "logging:\n  level: info\n")
 	opts := &Options{}
 	if _, err := opts.RunSetup(newCommand(t, opts), true, false); err != nil {
 		t.Fatal(err)
 	}
 	if logging.IsFileLogger() {
 		t.Error("switched to file logging with no log file configured")
 	}
 	if err := opts.Finish(); err != nil {
 		t.Errorf("Finish without a log file: %v", err)
 	}
 }
 func TestRunSetupTerminalLogIgnoresLogFile(t *testing.T) {
 	setupDir(t, "logging:\n  file: ./app.log\n")
 	opts := &Options{}
 	if _, err := opts.RunSetup(newCommand(t, opts), true, true); err != nil {
 		t.Fatal(err)
 	}
 	if logging.IsFileLogger() {
 		t.Error("switched to file logging when terminal logging was asked for")
 	}
 	if _, err := os.Stat("app.log"); err == nil {
 		t.Error("app.log was created")
 	}
 }
 func TestRunSetupBadLogFile(t *testing.T) {
 	setupDir(t, "logging:\n  file: ./missing-dir/app.log\n")
 	opts := &Options{}
 	_, err := opts.RunSetup(newCommand(t, opts), true, false)
 	if err == nil || !strings.Contains(err.Error(), "opening log file") {
 		t.Errorf("err = %v, want opening log file", err)
 	}
 }
 // Without loading, the configuration only reflects the flags.
 func TestRunSetupWithoutLoadingConfig(t *testing.T) {
 	setupDir(t, "logging:\n  level: info\n")
 	for args, want := range map[string]string{
 		"":        "",
 		"--debug": "debug",
 		"--quiet": "error",
 		"-d -q":   "debug", // --debug wins
 	} {
 		opts := &Options{}
 		conf, err := opts.RunSetup(
 			newCommand(t, opts, strings.Fields(args)...), false, true,
 		)
 		if err != nil {
 			t.Fatal(err)
 		}
 		if conf.Logging.Level != want {
 			t.Errorf(
 				"%q: Logging.Level = %q, want %q", args, conf.Logging.Level, want,
 			)
 		}
 	}
 }
 func TestSetupInitialLogging(t *testing.T) {
 	setupDir(t, "")
 	for args, wantDebug := range map[string]bool{
 		"--quiet": false,
 		"--debug": true,
 		"-q -d":   true, // --debug wins
 	} {
 		logging.SetLevel(slog.LevelInfo)
 		SetupInitialLogging(newCommand(t, &Options{}, strings.Fields(args)...))
 		if got := logging.IsDebugEnabled(); got != wantDebug {
 			t.Errorf("%q: debug enabled = %v, want %v", args, got, wantDebug)
 		}
 	}
 }
 func TestConfirm(t *testing.T) {
 	for input, want := range map[string]bool{
 		"y\n":              true,
 		"YES\n":            true,
 		"  Yes  \n":        true,
 		"n\n":              false,
 		"No\n":             false,
 		"maybe\n\ny\n":     true, // asks again until it gets an answer
 		"what?\nno\nyes\n": false,
 	} {
 		if got := confirmWithInput(t, input); got != want {
 			t.Errorf("Confirm with input %q = %v, want %v", input, got, want)
 		}
 	}
 }
 // confirmWithInput runs Confirm with input as standard input, discarding the
 // prompt it prints.
 func confirmWithInput(t *testing.T, input string) bool {
 	t.Helper()
 	dir := t.TempDir()
 	in := filepath.Join(dir, "stdin")
 	writeFile(t, in, input)
 	stdin, err := os.Open(filepath.Clean(in))
 	if err != nil {
 		t.Fatal(err)
 	}
 	defer stdin.Close()
 	stdout, err := os.Create(filepath.Clean(filepath.Join(dir, "stdout")))
 	if err != nil {
 		t.Fatal(err)
 	}
 	defer stdout.Close()
 	return Confirm(stdout, stdin, "Proceed?")
 }
@@ -0,0 +1,159 @@
 // Package cmdtest runs job-search commands in tests, much as net/http/httptest
 // serves handlers. Commands read .env and config.yaml from the working
 // directory and print to os.Stdout, so Setup gives each test its own working
 // directory and Run swaps os.Stdin and os.Stdout for files.
 //
 // Tests using it change global state (the working directory, os.Stdin,
 // os.Stdout, Viper and the logging level), so they must not run in parallel.
 package cmdtest
 import (
 	"log/slog"
 	"os"
 	"path/filepath"
 	"strings"
 	"testing"
 	"github.com/spf13/cobra"
 	"github.com/spf13/viper"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/logging"
 )
 // Setup changes into a new temporary directory holding an empty .env and,
 // unless config is empty, config.yaml with those contents. It returns the
 // directory.
 func Setup(t *testing.T, config string) string {
 	t.Helper()
 	dir := t.TempDir()
 	t.Chdir(dir)
 	WriteFile(t, ".env", "")
 	if config != "" {
 		WriteFile(t, "config.yaml", config)
 	}
 	return dir
 }
 // WriteFile writes contents to path, failing the test on error.
 func WriteFile(t *testing.T, path, contents string) {
 	t.Helper()
 	if err := os.WriteFile(path, []byte(contents), 0o600); err != nil {
 		t.Fatal(err)
 	}
 }
 // ReadFile returns the contents of path, failing the test on error.
 func ReadFile(t *testing.T, path string) string {
 	t.Helper()
 	b, err := os.ReadFile(filepath.Clean(path))
 	if err != nil {
 		t.Fatal(err)
 	}
 	return string(b)
 }
 // Exists reports whether path exists.
 func Exists(path string) bool {
 	_, err := os.Stat(path)
 	return err == nil
 }
 // NewRoot returns a root command set up like job-search's: the shared flags
 // bound to opts, logging set up before every command, opts.Finish run after,
 // and cmds added beneath it.
 func NewRoot(opts *cli.Options, cmds ...*cobra.Command) *cobra.Command {
 	root := &cobra.Command{
 		Use:           "job-search",
 		SilenceUsage:  true,
 		SilenceErrors: true,
 		PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
 			cli.SetupInitialLogging(cmd)
 			return nil
 		},
 		PersistentPostRunE: func(cmd *cobra.Command, args []string) error {
 			return opts.Finish()
 		},
 	}
 	cobra.EnableTraverseRunHooks = true
 	opts.Bind(root)
 	root.AddCommand(cmds...)
 	return root
 }
 // Run runs args on a command tree from newRoot, with stdin as standard
 // input, and returns everything written to standard output (including log
 // messages). Viper and the logging level are reset around each run, as they
 // are global.
 func Run(
 	t *testing.T, newRoot func() *cobra.Command, stdin string, args ...string,
 ) (string, error) {
 	t.Helper()
 	viper.Reset()
 	t.Cleanup(viper.Reset)
 	dir := t.TempDir()
 	inPath, outPath := filepath.Join(dir, "stdin"), filepath.Join(dir, "stdout")
 	WriteFile(t, inPath, stdin)
 	input, err := os.Open(filepath.Clean(inPath))
 	if err != nil {
 		t.Fatal(err)
 	}
 	defer input.Close() //nolint:errcheck // read only
 	out, err := os.Create(filepath.Clean(outPath))
 	if err != nil {
 		t.Fatal(err)
 	}
 	oldIn, oldOut, oldLog := os.Stdin, os.Stdout, slog.Default()
 	os.Stdin, os.Stdout = input, out
 	defer func() {
 		os.Stdin, os.Stdout = oldIn, oldOut
 		slog.SetDefault(oldLog)
 		logging.SetLevel(slog.LevelInfo)
 	}()
 	root := newRoot()
 	root.SetArgs(args)
 	runErr := root.Execute()
 	if err := out.Close(); err != nil {
 		t.Fatal(err)
 	}
 	return ReadFile(t, outPath), runErr
 }
 // MustRun is Run with no input, for commands that must succeed.
 func MustRun(
 	t *testing.T, newRoot func() *cobra.Command, args ...string,
 ) string {
 	t.Helper()
 	out, err := Run(t, newRoot, "", args...)
 	if err != nil {
 		t.Fatalf("job-search %s: %v\n%s", strings.Join(args, " "), err, out)
 	}
 	return out
 }
 // Contains fails the test unless out contains every string in want.
 func Contains(t *testing.T, out string, want ...string) {
 	t.Helper()
 	for _, wanted := range want {
 		if !strings.Contains(out, wanted) {
 			t.Errorf("output does not contain %q:\n%s", wanted, out)
 		}
 	}
 }
 // ErrContains fails the test unless err is an error containing want.
 func ErrContains(t *testing.T, err error, want string) {
 	t.Helper()
 	if err == nil || !strings.Contains(err.Error(), want) {
 		t.Errorf("err = %v; want an error containing %q", err, want)
 	}
 }
@@ -1,41 +1,31 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 package config
 import (
 	"log/slog"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 )
-func NewCommand() *cobra.Command {
+// NewCommand returns the "config" command, which shows and edits the
 // configuration.
 func NewCommand(opts *cli.Options) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "config",
 		Short: "Configuration management",
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
-			slog.Debug("config.PersistentPreRun")
+			slog.Debug("command persistent pre-run", "command", "config")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
-			slog.Debug("config.PersistentPostRun")
+			slog.Debug("command persistent post-run", "command", "config")
 		},
 	}
-	cmd.AddCommand(commandShow())
+	cmd.AddCommand(commandShow(opts))
-	cmd.AddCommand(commandGet())
+	cmd.AddCommand(commandGet(opts))
-	cmd.AddCommand(commandSet())
+	cmd.AddCommand(commandSet(opts))
 	cmd.AddCommand(commandInit())
 	cmd.AddCommand(commandGenerateCSRFKey(opts))
 	return cmd
 }
@@ -0,0 +1,267 @@
 package config
 import (
 	"strings"
 	"testing"
 	"github.com/spf13/cobra"
 	"go.yaml.in/yaml/v3"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/cli/cmdtest"
 )
 // These tests run the config commands as a user would, through a root
 // command like job-search's (see cmdtest).
 const appConfig = `logging:
  level: info
 sources:
  sleeper:
    interval: 2m
 `
 func newRoot() *cobra.Command {
 	opts := &cli.Options{}
 	return cmdtest.NewRoot(opts, NewCommand(opts))
 }
 func run(t *testing.T, args ...string) (string, error) {
 	t.Helper()
 	return cmdtest.Run(t, newRoot, "", args...)
 }
 func mustRun(t *testing.T, args ...string) string {
 	t.Helper()
 	return cmdtest.MustRun(t, newRoot, args...)
 }
 // fileSettings returns the settings in the YAML file at path, flattened to
 // dotted keys.
 func fileSettings(t *testing.T, path string) map[string]any {
 	t.Helper()
 	var nested map[string]any
 	if err := yaml.Unmarshal(
 		[]byte(cmdtest.ReadFile(t, path)), &nested,
 	); err != nil {
 		t.Fatal(err)
 	}
 	flat := map[string]any{}
 	var walk func(prefix string, m map[string]any)
 	walk = func(prefix string, m map[string]any) {
 		for k, v := range m {
 			if sub, ok := v.(map[string]any); ok {
 				walk(prefix+k+".", sub)
 			} else {
 				flat[prefix+k] = v
 			}
 		}
 	}
 	walk("", nested)
 	return flat
 }
 func TestConfigShow(t *testing.T) {
 	cmdtest.Setup(t, appConfig)
 	out := mustRun(t, "config", "show")
 	cmdtest.Contains(
 		t,
 		out,
 		"logging.level = info <string>",
 		"sources.sleeper.interval = 2m0s <string>",
 	)
 	// Keys are listed in order, the same every time.
 	if again := mustRun(t, "config", "show"); again != out {
 		t.Errorf(
 			"config show output changed between runs:\n%s\n---\n%s", out, again,
 		)
 	}
 	database := strings.Index(out, "database.file")
 	logging := strings.Index(out, "logging.level")
 	if database > logging {
 		t.Errorf("keys not sorted:\n%s", out)
 	}
 	out = mustRun(t, "config", "show", "--json")
 	cmdtest.Contains(t, out, `"level": "info"`, `"interval": "2m0s"`)
 	out = mustRun(t, "config", "show", "--yaml")
 	cmdtest.Contains(t, out, "level: info", "interval: 2m0s")
 }
 func TestConfigGet(t *testing.T) {
 	cmdtest.Setup(t, appConfig)
 	for _, tt := range []struct {
 		key, want string
 	}{
 		{"logging.level", "info <string>"},
 		{"sources.sleeper.interval", "2m0s <string>"},
 		{"sources.sleeper.enabled", "false <bool>"},
 	} {
 		t.Run(tt.key, func(t *testing.T) {
 			cmdtest.Contains(t, mustRun(t, "config", "get", tt.key), tt.want)
 		})
 	}
 	if _, err := run(t, "config", "get", "no.such.key"); err == nil {
 		t.Error("get of an unknown key succeeded")
 	}
 	if _, err := run(t, "config", "get"); err == nil {
 		t.Error("get without a key succeeded")
 	}
 }
 func TestConfigSet(t *testing.T) {
 	cmdtest.Setup(t, appConfig)
 	out := mustRun(t, "config", "set", "logging.level", "debug")
 	cmdtest.Contains(t, out, "logging.level = debug <string>")
 	cmdtest.Contains(
 		t, mustRun(t, "config", "get", "logging.level"), "debug <string>",
 	)
 	out = mustRun(t, "config", "set", "sources.sleeper.enabled", "true")
 	cmdtest.Contains(t, out, "sources.sleeper.enabled = true <bool>")
 	// Durations are stored as given out, e.g. 5m0s.
 	out = mustRun(t, "config", "set", "sources.sleeper.interval", "5m")
 	cmdtest.Contains(t, out, "sources.sleeper.interval = 5m0s <string>")
 	// Only the settings the file had, plus those set, are written: no
 	// defaults.
 	want := map[string]any{
 		"logging.level":            "debug",
 		"sources.sleeper.enabled":  true,
 		"sources.sleeper.interval": "5m0s",
 	}
 	if got := fileSettings(t, "config.yaml"); !equalSettings(got, want) {
 		t.Errorf("config.yaml settings = %v; want %v", got, want)
 	}
 	for _, args := range [][]string{
 		{"config", "set", "logging.level", "loud"},          // rejected by validation
 		{"config", "set", "sources.sleeper.enabled", "yes"}, // not a bool
 		{"config", "set", "sources.sleeper.interval", "soon"},
 		{"config", "set", "no.such.key", "1"},
 		{"config", "set", "logging.level"}, // no value, no --unset
 	} {
 		if _, err := run(t, args...); err == nil {
 			t.Errorf("%v succeeded", args)
 		}
 	}
 	if got := fileSettings(t, "config.yaml"); !equalSettings(got, want) {
 		t.Errorf("failed sets changed config.yaml: %v", got)
 	}
 }
 func equalSettings(a, b map[string]any) bool {
 	if len(a) != len(b) {
 		return false
 	}
 	for k, v := range a {
 		if b[k] != v {
 			return false
 		}
 	}
 	return true
 }
 func TestConfigSetUnset(t *testing.T) {
 	cmdtest.Setup(t, appConfig)
 	mustRun(t, "config", "set", "--unset", "sources.sleeper.interval")
 	// Only the unset key is gone: no defaults are written in its place.
 	want := map[string]any{"logging.level": "info"}
 	if got := fileSettings(t, "config.yaml"); !equalSettings(got, want) {
 		t.Errorf("config.yaml settings = %v; want %v", got, want)
 	}
 	// Back to the default.
 	cmdtest.Contains(
 		t,
 		mustRun(t, "config", "get", "sources.sleeper.interval"),
 		"10m0s <string>",
 	)
 }
 func TestConfigSetWithoutConfigFile(t *testing.T) {
 	cmdtest.Setup(t, "")
 	_, err := run(t, "config", "set", "logging.level", "debug")
 	cmdtest.ErrContains(t, err, "no configuration file exists")
 }
 func TestConfigInit(t *testing.T) {
 	cmdtest.Setup(t, "")
 	out := mustRun(t, "config", "init", "./new.yaml")
 	cmdtest.Contains(t, out, "writing default configuration to file: ./new.yaml")
 	cmdtest.Contains(
 		t, cmdtest.ReadFile(t, "new.yaml"), "level: warn", "interval: 10m0s",
 	)
 	_, err := run(t, "config", "init", "./new.yaml")
 	cmdtest.ErrContains(t, err, "already exists")
 	if _, err := run(t, "config", "init", "a.yaml", "b.yaml"); err == nil {
 		t.Error("init with two paths succeeded")
 	}
 }
 // The test sources are left out of the file unless --all is given.
 func TestConfigInitTestSources(t *testing.T) {
 	cmdtest.Setup(t, "")
 	mustRun(t, "config", "init", "./default.yaml")
 	written := cmdtest.ReadFile(t, "default.yaml")
 	cmdtest.Contains(t, written, "find_a_tender:", "eu_supply:")
 	for _, hidden := range []string{"fts_mock", "sleeper"} {
 		if strings.Contains(written, hidden) {
 			t.Errorf("init wrote %s without --all:\n%s", hidden, written)
 		}
 	}
 	mustRun(t, "config", "init", "--all", "./all.yaml")
 	cmdtest.Contains(
 		t, cmdtest.ReadFile(t, "all.yaml"), "fts_mock:", "sleeper:",
 	)
 	// The test sources still load and are set as usual.
 	mustRun(
 		t, "--config", "default.yaml",
 		"config", "set", "sources.sleeper.enabled", "true",
 	)
 	cmdtest.Contains(t, cmdtest.ReadFile(t, "default.yaml"), "sleeper:")
 }
 // With no path (or a blank one), init searches the default locations. The
 // working directory is one, so its config.yaml is found. (Not finding one
 // would write to /etc/job-search, which a test can't do.)
 func TestConfigInitDefaultLocations(t *testing.T) {
 	cmdtest.Setup(t, appConfig)
 	for _, args := range [][]string{
 		{"config", "init"},
 		{"config", "init", "  "},
 	} {
 		t.Run(strings.Join(args, " "), func(t *testing.T) {
 			_, err := run(t, args...)
 			cmdtest.ErrContains(t, err, "already exists")
 		})
 	}
 }
 // init takes the path as an argument, so --config isn't shown in its help.
 func TestConfigInitHelpHidesConfigFlag(t *testing.T) {
 	cmdtest.Setup(t, "")
 	out := mustRun(t, "config", "init", "--help")
 	cmdtest.Contains(t, out, "init [config_path]", "--debug")
 	if strings.Contains(out, "--config") {
 		t.Errorf("init help shows --config:\n%s", out)
 	}
 }
@@ -0,0 +1,129 @@
 package config
 import (
 	"crypto/rand"
 	"encoding/hex"
 	"errors"
 	"fmt"
 	"log/slog"
 	"os"
 	"github.com/spf13/cobra"
 	"github.com/spf13/viper"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/config"
 )
 // csrfKeySetting is the setting generate-csrf-key stores the key in.
 const csrfKeySetting = "server.csrf.tokens.key"
 // csrfKeyBytes is the length of a generated key: 32 bytes, as the server
 // generates for itself when no key is set.
 const csrfKeyBytes = 32
 // commandGenerateCSRFKey returns the "config generate-csrf-key" command,
 // which stores a new random CSRF key in the configuration file, or prints it.
 func commandGenerateCSRFKey(opts *cli.Options) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "generate-csrf-key",
 		Args:  cobra.NoArgs,
 		Short: "Generate a CSRF key and store it in the configuration file",
 		Long: `Generate a random key for signing CSRF tokens and store it in
 server.csrf.tokens.key in the configuration file. Without a key, each server
 process signs with a random key of its own, so tokens don't survive a restart
 and aren't valid across instances. Give every instance the same key, and
 restart them to use it.
 The key isn't printed. With --print, it is only printed, and the
 configuration file isn't changed: use that to set
 JOBSEARCH_SERVER_CSRF_TOKENS_KEY instead, or another instance's file.
 An existing key is only replaced with --force, as a new key invalidates
 every token already issued. A key set by JOBSEARCH_SERVER_CSRF_TOKENS_KEY
 overrides the file, so the command refuses to store one while it is set.`,
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug(
 				"command persistent pre-run", "command", "config.generate-csrf-key",
 			)
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug(
 				"command persistent post-run", "command", "config.generate-csrf-key",
 			)
 		},
 		RunE: func(cmd *cobra.Command, args []string) error {
 			slog.Debug("command run", "command", "config.generate-csrf-key")
 			printOnly, printErr := cmd.Flags().GetBool("print")
 			force, forceErr := cmd.Flags().GetBool("force")
 			err := errors.Join(printErr, forceErr)
 			if err != nil {
 				return err
 			}
 			key := generateCSRFKey()
 			if printOnly {
 				fmt.Fprintln(cmd.OutOrStdout(), key) //nolint:errcheck // printing only
 				return nil
 			}
 			envVar := config.EnvVarName(csrfKeySetting)
 			if os.Getenv(envVar) != "" {
 				return fmt.Errorf(
 					"%s is set and overrides the configuration file: "+
 						"unset it first, or use --print to make a key for it",
 					envVar,
 				)
 			}
 			conf, err := opts.RunSetup(cmd, true, true)
 			if err != nil {
 				return fmt.Errorf("load configuration: %w", err)
 			}
 			if viper.ConfigFileUsed() == "" {
 				return errors.New("no configuration file exists")
 			}
 			if conf.Server.CSRF.Tokens.Key != "" && !force {
 				return fmt.Errorf(
 					"%s is already set: --force replaces it, which invalidates "+
 						"every token already issued",
 					csrfKeySetting,
 				)
 			}
 			_, _, err = config.StoreValue(conf, csrfKeySetting, key)
 			if err != nil {
 				return err
 			}
 			fmt.Fprintf( //nolint:errcheck // printing only
 				cmd.OutOrStdout(),
 				"%s set in %s; restart the server to use it, and give every "+
 					"instance the same key\n",
 				csrfKeySetting,
 				viper.ConfigFileUsed(),
 			)
 			return nil
 		},
 	}
 	cmd.Flags().Bool(
 		"print", false, "only print the key; don't change the configuration file",
 	)
 	cmd.Flags().Bool(
 		"force", false, "replace an existing key, invalidating issued tokens",
 	)
 	cmd.MarkFlagsMutuallyExclusive("print", "force")
 	return cmd
 }
 // generateCSRFKey returns a new random key, hex encoded as
 // server.csrf.tokens.key expects.
 func generateCSRFKey() string {
 	key := make([]byte, csrfKeyBytes)
 	rand.Read(key) // crypto/rand.Read never returns an error
 	return hex.EncodeToString(key)
 }
@@ -0,0 +1,106 @@
 package config
 import (
 	"encoding/hex"
 	"strings"
 	"testing"
 	"git.avsdev.uk/Go/job-search/internal/cli/cmdtest"
 )
 // fileCSRFKey returns the CSRF key in config.yaml, or "" if it has none.
 func fileCSRFKey(t *testing.T) string {
 	t.Helper()
 	key, _ := fileSettings(t, "config.yaml")[csrfKeySetting].(string)
 	return key
 }
 // checkCSRFKey fails the test unless key is csrfKeyBytes of hex.
 func checkCSRFKey(t *testing.T, key string) {
 	t.Helper()
 	decoded, err := hex.DecodeString(key)
 	if err != nil || len(decoded) != csrfKeyBytes {
 		t.Fatalf("key %q is not %d bytes of hex", key, csrfKeyBytes)
 	}
 }
 // The key is stored in the configuration file, without being printed, and
 // the file's other settings are kept.
 func TestGenerateCSRFKey(t *testing.T) {
 	cmdtest.Setup(t, appConfig)
 	out := mustRun(t, "config", "generate-csrf-key")
 	cmdtest.Contains(t, out, csrfKeySetting+" set in ")
 	key := fileCSRFKey(t)
 	checkCSRFKey(t, key)
 	if strings.Contains(out, key) {
 		t.Errorf("the key was printed: %s", out)
 	}
 	if fileSettings(t, "config.yaml")["sources.sleeper.interval"] != "2m" {
 		t.Errorf("other settings were changed: %v", fileSettings(t, "config.yaml"))
 	}
 }
 // An existing key is only replaced with --force.
 func TestGenerateCSRFKeyExisting(t *testing.T) {
 	cmdtest.Setup(t, appConfig)
 	mustRun(t, "config", "generate-csrf-key")
 	first := fileCSRFKey(t)
 	_, err := run(t, "config", "generate-csrf-key")
 	cmdtest.ErrContains(t, err, "already set: --force replaces it")
 	if fileCSRFKey(t) != first {
 		t.Fatal("the key was replaced without --force")
 	}
 	mustRun(t, "config", "generate-csrf-key", "--force")
 	second := fileCSRFKey(t)
 	checkCSRFKey(t, second)
 	if second == first {
 		t.Error("--force kept the same key")
 	}
 }
 // --print only prints a key, and needs no configuration file.
 func TestGenerateCSRFKeyPrint(t *testing.T) {
 	cmdtest.Setup(t, "")
 	key := strings.TrimSpace(mustRun(t, "config", "generate-csrf-key", "--print"))
 	checkCSRFKey(t, key)
 	other := strings.TrimSpace(
 		mustRun(t, "config", "generate-csrf-key", "--print"),
 	)
 	if other == key {
 		t.Error("two keys were the same")
 	}
 }
 // A key in the environment variable would override the file, so none is
 // stored while it is set.
 func TestGenerateCSRFKeyEnvironment(t *testing.T) {
 	cmdtest.Setup(t, appConfig)
 	t.Setenv("JOBSEARCH_SERVER_CSRF_TOKENS_KEY", generateCSRFKey())
 	_, err := run(t, "config", "generate-csrf-key")
 	cmdtest.ErrContains(t, err, "JOBSEARCH_SERVER_CSRF_TOKENS_KEY is set")
 	if fileCSRFKey(t) != "" {
 		t.Error("a key was stored")
 	}
 }
 func TestGenerateCSRFKeyWithoutConfigFile(t *testing.T) {
 	cmdtest.Setup(t, "")
 	_, err := run(t, "config", "generate-csrf-key")
 	cmdtest.ErrContains(t, err, "no configuration file exists")
 }
 func TestGenerateCSRFKeyFlags(t *testing.T) {
 	cmdtest.Setup(t, appConfig)
 	_, err := run(t, "config", "generate-csrf-key", "--print", "--force")
 	cmdtest.ErrContains(t, err, "none of the others")
 	if _, err := run(t, "config", "generate-csrf-key", "extra"); err == nil {
 		t.Error("an argument was accepted")
 	}
 }
@@ -1,83 +1,46 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 package config
 import (
 	"errors"
 	"fmt"
 	"log/slog"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 )
-func commandGet() *cobra.Command {
+func commandGet(opts *cli.Options) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "get [config_key]",
 		Args:  cobra.ExactArgs(1),
 		// Args: func(cmd *cobra.Command, args []string) error {
 		// 	// Optionally run one of the validators provided by cobra
 		// 	if err := cobra.ExactArgs(1)(cmd, args); err != nil {
 		// 		return err
 		// 	}
 		// 	conf, ok := cli.ConfigFromContext(cmd.Context())
 		// 	if !ok {
 		// 		return errors.New("no configuration loaded")
 		// 	}
 		// 	flatConf := config.FlattenConfig(conf)
 		// 	if !util.HasKey(flatConf, args[0]) {
 		// 		return fmt.Errorf("invalid config key specified: %s", args[0])
 		// 	}
 		// 	return nil
 		// },
 		Short: "Get a value from configuration",
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
-			slog.Debug("config.get.PersistentPreRun")
+			slog.Debug("command persistent pre-run", "command", "config.get")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
-			slog.Debug("config.get.PersistentPostRun")
+			slog.Debug("command persistent post-run", "command", "config.get")
 		},
 		RunE: func(cmd *cobra.Command, args []string) error {
-			slog.Debug("config.get.Run")
+			slog.Debug("command run", "command", "config.get")
 			if err := cobra.ExactArgs(1)(cmd, args); err != nil {
 				return err
 			}
-			conf, ok := cli.ConfigFromContext(cmd.Context())
+			conf, err := opts.RunSetup(cmd, true, true)
-			if !ok {
+			if err != nil {
-				return errors.New("no configuration loaded")
+				return fmt.Errorf("load configuration: %w", err)
 			}
-			v, err := conf.ValueOf(args[0])
+			value, err := conf.ValueOf(args[0])
 			if err != nil {
 				return err
 			}
-			t, _ := conf.TypeOf(args[0])
+			kind, _ := conf.TypeOf(args[0])
-			fmt.Println(v, "<"+t.String()+">")
+			fmt.Println(value, "<"+kind.String()+">")
 			return nil
 		},
 		Annotations: map[string]string{
 			"terminal_log": "1",
 		},
 	}
 	return cmd
 }
@@ -1,19 +1,3 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 package config
 import (
@@ -21,25 +5,31 @@ import (
 	"fmt"
 	"log/slog"
 	"os"
 	"strings"
 	"git.avsdev.uk/Go/job-search/internal/config"
 	"github.com/spf13/cobra"
 	"github.com/spf13/viper"
 	"git.avsdev.uk/Go/job-search/internal/config"
 )
 func commandInit() *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "init [config_path]",
 		Args:  cobra.MatchAll(cobra.MinimumNArgs(0), cobra.MaximumNArgs(1)),
-		Short: "Create an empty configuration file",
+		Short: "Create a configuration file holding the defaults",
 		Long: `Create a configuration file holding the defaults, at config_path or,
 without one, the first config.yaml found in /etc/job-search then the working
 directory (/etc/job-search/config.yaml if there is none). The test sources
 (fts_mock and sleeper) are left out unless --all is given.`,
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
-			slog.Debug("config.init.PersistentPreRun")
+			slog.Debug("command persistent pre-run", "command", "config.init")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
-			slog.Debug("config.init.PersistentPostRun")
+			slog.Debug("command persistent post-run", "command", "config.init")
 		},
 		RunE: func(cmd *cobra.Command, args []string) error {
-			slog.Debug("config.init.Run")
+			slog.Debug("command run", "command", "config.init")
 			if err := cobra.MinimumNArgs(0)(cmd, args); err != nil {
 				return err
@@ -48,10 +38,18 @@ func commandInit() *cobra.Command {
 				return err
 			}
-			cfgFile, _ := cmd.Flags().GetString("config")
+			// Flags and arguments are valid from here on, so errors are runtime
-			if cfgFile != "" {
+			// failures: report them without cobra's usage text.
-				// Use config file from the flag.
+			cmd.SilenceUsage = true
-				viper.SetConfigFile(cfgFile)
+
 			// With no path, the default locations are used.
 			configFile := ""
 			if len(args) > 0 {
 				configFile = strings.TrimSpace(args[0])
 			}
 			if configFile != "" {
 				// Use the config file given.
 				viper.SetConfigFile(configFile)
 			} else {
 				// Search config in /etc or local directory.
 				viper.AddConfigPath(config.EtcConfigPath)
@@ -64,33 +62,40 @@ func commandInit() *cobra.Command {
 				return errors.New("configuration file already exists")
 			}
-			_, ok := err.(viper.ConfigFileNotFoundError)
+			var notFound viper.ConfigFileNotFoundError
-			if !errors.Is(err, os.ErrNotExist) && !ok {
+			if !errors.Is(err, os.ErrNotExist) && !errors.As(err, &notFound) {
 				// Config file was found but another error was produced
 				return err
 			}
-			// Load default configuration
+			path := viper.ConfigFileUsed()
-			config.LoadDefaultConfig()
+			if path == "" {
-
+				path = config.EtcConfigPath + "/config.yaml"
 			if viper.ConfigFileUsed() == "" {
 				viper.SetConfigFile(config.EtcConfigPath + "/config.yaml")
 			}
-			fmt.Print(
+			// A Viper of its own, holding only the keys to write: the global
-				"writing default configuration to file: " +
+			// one would write every default, the hidden sections' included.
-					viper.ConfigFileUsed() +
+			all, _ := cmd.Flags().GetBool("all")
-					"\n")
+			generated := viper.New()
-
+			for key, value := range config.InitDefaults(all) {
-			if err := viper.WriteConfig(); err != nil {
+				generated.Set(key, value)
 				return err
 			}
-			return nil
+			generated.SetConfigFile(path)
-		},
+
-		Annotations: map[string]string{
+			fmt.Print("writing default configuration to file: " + path + "\n")
-			"no_config":    "1",
+
-			"terminal_log": "1",
+			return generated.WriteConfig()
 		},
 	}
 	cmd.Flags().Bool(
 		"all", false, "also write the test sources (fts_mock and sleeper)",
 	)
 	// Config flag should be hidden as the config path should be provided
 	cmd.SetHelpFunc(func(command *cobra.Command, args []string) {
 		command.Flags().MarkHidden("config") //nolint:errcheck,gosec // a root flag, so it always exists
 		command.Parent().HelpFunc()(command, args)
 	})
 	return cmd
 }
@@ -1,106 +1,30 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 package config
 import (
 	"errors"
 	"fmt"
 	"log/slog"
-	"reflect"
+
-	"strconv"
+	"github.com/spf13/cobra"
 	"github.com/spf13/viper"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/config"
 	"github.com/spf13/cobra"
 	"github.com/spf13/viper"
 )
-func setConfig(conf config.JobSearchConfig, key string, value string) error {
+func commandSet(opts *cli.Options) *cobra.Command {
 	var confValue any = nil
 	var confType reflect.Kind
 	var err error
 	confType, err = conf.TypeOf(key)
 	if err != nil {
 		return err
 	}
 	switch confType {
 	case reflect.Bool:
 		confValue, err = strconv.ParseBool(value)
 	case reflect.String:
 		confValue = value
 	case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
 		confValue, err = strconv.ParseInt(value, 10, 64)
 	case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
 		confValue, err = strconv.ParseUint(value, 10, 64)
 	case reflect.Float32, reflect.Float64:
 		confValue, err = strconv.ParseFloat(value, 64)
 	default:
 		err = errors.ErrUnsupported
 	}
 	if err != nil {
 		return fmt.Errorf("config value type not accepted, expected <%s>", confType)
 	}
 	viper.Set(key, confValue)
 	if err := viper.WriteConfig(); err != nil {
 		return err
 	}
 	fmt.Print(key, " = ", confValue, " <", confType, ">\n")
 	return nil
 }
 func unsetConfig(conf config.JobSearchConfig, key string) error {
 	if _, err := conf.TypeOf(key); err != nil {
 		return err
 	}
 	v := viper.New()
 	v.SetConfigFile(viper.ConfigFileUsed())
 	keys := viper.AllKeys()
 	for i := range keys {
 		k := keys[i]
 		if k != key {
 			v.Set(k, viper.Get(k))
 		}
 	}
 	if err := v.WriteConfig(); err != nil {
 		return err
 	}
 	return nil
 }
 func commandSet() *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "set [config_key] [config_value]",
 		Args:  cobra.MatchAll(cobra.MinimumNArgs(1), cobra.MaximumNArgs(2)),
 		Short: "Set a value to configuration",
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
-			slog.Debug("config.set.PersistentPreRun")
+			slog.Debug("command persistent pre-run", "command", "config.set")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
-			slog.Debug("config.set.PersistentPostRun")
+			slog.Debug("command persistent post-run", "command", "config.set")
 		},
 		RunE: func(cmd *cobra.Command, args []string) error {
-			slog.Debug("config.set.Run")
+			slog.Debug("command run", "command", "config.set")
 			if err := cobra.MinimumNArgs(1)(cmd, args); err != nil {
 				return err
@@ -117,9 +41,9 @@ func commandSet() *cobra.Command {
 				return err
 			}
-			conf, ok := cli.ConfigFromContext(cmd.Context())
+			conf, err := opts.RunSetup(cmd, true, true)
-			if !ok {
+			if err != nil {
-				return errors.New("no configuration loaded")
+				return fmt.Errorf("load configuration: %w", err)
 			}
 			if viper.ConfigFileUsed() == "" {
@@ -127,15 +51,20 @@ func commandSet() *cobra.Command {
 			}
 			if unset {
-				return unsetConfig(conf, args[0])
+				return config.PurgeValue(conf, args[0])
 			} else {
 				return setConfig(conf, args[0], args[1])
 			}
-		},
+
-		Annotations: map[string]string{
+			confValue, confType, err := config.StoreValue(conf, args[0], args[1])
-			"terminal_log": "1",
+			if err != nil {
 				return err
 			}
 			fmt.Print(args[0], " = ", confValue, " <", confType, ">\n")
 			return nil
 		},
 	}
 	cmd.Flags().BoolP("unset", "x", false, "clear configuration setting")
 	return cmd
 }
@@ -1,62 +1,45 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 package config
 import (
-	"errors"
+	"fmt"
 	"log/slog"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 )
-func commandShow() *cobra.Command {
+func commandShow(opts *cli.Options) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "show",
 		Short: "Print config information",
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
-			slog.Debug("config.show.PersistentPreRun")
+			slog.Debug("command persistent pre-run", "command", "config.show")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
-			slog.Debug("config.show.PersistentPostRun")
+			slog.Debug("command persistent post-run", "command", "config.show")
 		},
 		RunE: func(cmd *cobra.Command, args []string) error {
-			slog.Debug("config.show.Run")
+			slog.Debug("command run", "command", "config.show")
-			conf, ok := cli.ConfigFromContext(cmd.Context())
+			conf, err := opts.RunSetup(cmd, true, true)
-			if !ok {
+			if err != nil {
-				return errors.New("no configuration loaded")
+				return fmt.Errorf("load configuration: %w", err)
 			}
 			if yaml, _ := cmd.Flags().GetBool("yaml"); yaml {
-				conf.PrintYaml()
+				conf.PrintYAML()
 			} else if json, _ := cmd.Flags().GetBool("json"); json {
-				conf.PrintJson()
+				conf.PrintJSON()
 			} else {
 				conf.PrintTerminal()
 			}
 			return nil
 		},
 		Annotations: map[string]string{
 			"terminal_log": "1",
 		},
 	}
 	cmd.Flags().Bool("json", false, "format configuration as json")
 	cmd.Flags().Bool("yaml", false, "format configuration as yaml")
 	return cmd
 }
@@ -0,0 +1,39 @@
 package cli
 import (
 	"bufio"
 	"fmt"
 	"io"
 	"os"
 	"strings"
 )
 // Confirm asks the user the yes/no question prompt on stdin and reports whether
 // they answered yes. "y" and "yes" confirm, "n" and "no" refuse, in any case;
 // anything else asks again. It exits the program if stdin can't be read.
 func Confirm(out io.Writer, in io.Reader, prompt string) bool {
 	reader := bufio.NewReader(in)
 	for {
 		_, err := fmt.Fprintf(out, "%s [y/n]: ", prompt)
 		if err != nil {
 			fmt.Fprintf(os.Stderr, "error asking question: %s\n", err)
 			os.Exit(1)
 		}
 		response, err := reader.ReadString('\n')
 		if err != nil {
 			fmt.Fprintf(os.Stderr, "error reading answer: %s\n", err)
 			os.Exit(1)
 		}
 		response = strings.ToLower(strings.TrimSpace(response))
 		switch response {
 		case "y", "yes":
 			return true
 		case "n", "no":
 			return false
 		}
 	}
 }
@@ -0,0 +1,18 @@
 // Package fetch holds the fetcher commands: "fts-once" fetches from Find a
 // Tender once and prints the releases, and "service" polls every enabled
 // source on its interval.
 package fetch
 import (
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 )
 // NewCommands returns the fetcher commands, "fts-once" and "service".
 func NewCommands(opts *cli.Options) []*cobra.Command {
 	return []*cobra.Command{
 		commandFTSOnce(opts),
 		commandService(opts),
 	}
 }
@@ -0,0 +1,156 @@
 package fetch
 import (
 	"encoding/json"
 	"errors"
 	"fmt"
 	"log/slog"
 	"slices"
 	"time"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/findatender"
 	"git.avsdev.uk/Go/job-search/internal/types"
 )
 // Release tags for each stage of a procedure, from the OCDS releaseTag
 // codelist. Cancellations are left out, as a cancelled procedure is no longer
 // at any stage.
 var stageTags = map[string][]string{
 	"planning":    {"planning", "planningUpdate"},
 	"opportunity": {"tender", "tenderAmendment", "tenderUpdate"},
 	"award": {
 		"award", "awardUpdate",
 		"contract", "contractUpdate", "contractAmendment",
 		"implementation", "implementationUpdate", "contractTermination",
 	},
 }
 // filter keeps the opportunities whose latest release is at stage. Releases
 // are oldest first, so the last one is the procedure's current stage.
 func filter(
 	opportunities []*types.Opportunity, stage string,
 ) []*types.Opportunity {
 	var matching []*types.Opportunity
 	for _, opportunity := range opportunities {
 		if len(opportunity.Releases) == 0 {
 			continue
 		}
 		latest := opportunity.Releases[len(opportunity.Releases)-1]
 		for _, tag := range latest.Tags {
 			if slices.Contains(stageTags[stage], tag.Tag) {
 				matching = append(matching, opportunity)
 				break
 			}
 		}
 	}
 	return matching
 }
 // commandFTSOnce returns the "fts-once" command, which fetches from Find a
 // Tender once and prints the releases as JSON.
 func commandFTSOnce(opts *cli.Options) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "fts-once",
 		Short: "Fetch from Find a Tender once and print the releases as JSON",
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent pre-run", "command", "fts-once")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent post-run", "command", "fts-once")
 		},
 		RunE: func(cmd *cobra.Command, args []string) error {
 			slog.Debug("command run", "command", "fts-once")
 			conf, err := opts.RunSetup(cmd, true, true)
 			if err != nil {
 				return fmt.Errorf("load configuration: %w", err)
 			}
 			since, sinceErr := cmd.Flags().GetTime("since")
 			if sinceErr != nil {
 				return sinceErr
 			}
 			if conf.Sources.FindATender.APIRoot == "" {
 				return errors.New("sources.find_a_tender.api_root is not set")
 			}
 			findATender := findatender.New(conf.Sources.FindATender.APIRoot)
 			defer findATender.Shutdown()
 			ignoreRateLimit, _ := cmd.Flags().GetBool("ignore-rate-limit")
 			rateLimitRetries := conf.Sources.FindATender.RateLimitRetries
 			if ignoreRateLimit {
 				rateLimitRetries = 0
 			}
 			results, err := findATender.FetchAll(findatender.RequestOptions{
 				Since:            since,
 				RateLimitRetries: rateLimitRetries,
 			})
 			if err != nil {
 				return err
 			}
 			planning, _ := cmd.Flags().GetBool("planning")
 			opportunities, _ := cmd.Flags().GetBool("opportunities")
 			awards, _ := cmd.Flags().GetBool("awards")
 			filterTag := ""
 			if planning {
 				filterTag = "planning"
 			}
 			if opportunities {
 				filterTag = "opportunity"
 			}
 			if awards {
 				filterTag = "award"
 			}
 			if filterTag != "" {
 				results = filter(results, filterTag)
 			}
 			encoded, err := json.MarshalIndent(results, "", "  ")
 			if err != nil {
 				return err
 			}
 			fmt.Println(string(encoded))
 			return nil
 		},
 	}
 	cmd.Flags().Time(
 		"since",
 		time.Time{},
 		[]string{
 			time.DateOnly, time.DateTime,
 		},
 		"contract updates published since YYYY-MM-DD[ hh:mm:ss]",
 	)
 	cmd.Flags().Bool(
 		"planning",
 		false,
 		"only return planning (i.e. exclude opportunities and awards)",
 	)
 	cmd.Flags().Bool(
 		"opportunities",
 		false,
 		"only return opportunities (i.e. exclude awards and planning)",
 	)
 	cmd.Flags().Bool(
 		"awards",
 		false,
 		"only return awards (i.e. exclude planning and opportunities)",
 	)
 	cmd.Flags().Bool(
 		"ignore-rate-limit", false, "ignore the rate limit and fail if exceeded",
 	)
 	cmd.MarkFlagsMutuallyExclusive("planning", "opportunities", "awards")
 	return cmd
 }
@@ -0,0 +1,66 @@
 package fetch
 import (
 	"io"
 	"slices"
 	"strings"
 	"testing"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/types"
 )
 // opp makes an opportunity whose releases, oldest first, have these tags.
 func opp(uuid string, tags ...string) *types.Opportunity {
 	o := &types.Opportunity{
 		UUID: uuid,
 	}
 	for _, tag := range tags {
 		o.Releases = append(o.Releases, types.OpportunityRelease{
 			Tags: []types.OpportunityReleaseTag{{
 				Tag: tag,
 			}},
 		})
 	}
 	return o
 }
 func TestFilter(t *testing.T) {
 	opps := []*types.Opportunity{
 		opp("planned", "planning"),
 		opp("open", "planning", "tender"),
 		opp("extended", "tender", "tenderUpdate"),
 		opp("awarded", "planning", "tender", "award"),
 		opp("cancelled", "tender", "tenderCancellation"),
 		opp("no releases"),
 	}
 	tests := map[string][]string{
 		"planning":    {"planned"},
 		"opportunity": {"open", "extended"},
 		"award":       {"awarded"},
 	}
 	for stage, want := range tests {
 		t.Run(stage, func(t *testing.T) {
 			var got []string
 			for _, o := range filter(opps, stage) {
 				got = append(got, o.UUID)
 			}
 			if !slices.Equal(got, want) {
 				t.Errorf("filter(%s) = %v; want %v", stage, got, want)
 			}
 		})
 	}
 }
 // Conflicting filters are rejected by cobra before anything is fetched.
 func TestOnceFiltersExclusive(t *testing.T) {
 	cmd := commandFTSOnce(&cli.Options{})
 	cmd.SetArgs([]string{"--planning", "--awards"})
 	cmd.SetOut(io.Discard)
 	cmd.SetErr(io.Discard)
 	err := cmd.Execute()
 	if err == nil || !strings.Contains(err.Error(), "none of the others can be") {
 		t.Fatalf("err = %v; want a mutually exclusive flags error", err)
 	}
 }
@@ -0,0 +1,92 @@
 package fetch
 import (
 	"context"
 	"errors"
 	"fmt"
 	"log/slog"
 	"os"
 	"os/signal"
 	"syscall"
 	"time"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/fetcher"
 )
 func commandService(opts *cli.Options) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "service",
 		Short: "Run every enabled source's fetcher on its interval until stopped",
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent pre-run", "command", "service")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent post-run", "command", "service")
 		},
 		RunE: func(cmd *cobra.Command, args []string) (err error) {
 			slog.Debug("command run", "command", "service")
 			foreground, _ := cmd.Flags().GetBool("foreground")
 			conf, err := opts.RunSetup(cmd, true, foreground)
 			if err != nil {
 				return fmt.Errorf("load configuration: %w", err)
 			}
 			since, sinceErr := cmd.Flags().GetTime("since")
 			if sinceErr != nil {
 				return sinceErr
 			}
 			// ctx is cancelled on the first SIGINT (Ctrl+C) or SIGTERM.
 			ctx, stop := signal.NotifyContext(
 				context.Background(), os.Interrupt, syscall.SIGTERM,
 			)
 			defer stop()
 			// After the first signal, restore default signal handling so a second
 			// Ctrl+C force-kills the process if graceful shutdown hangs.
 			go func() {
 				<-ctx.Done()
 				stop()
 			}()
 			fetcherOptions, err := cli.SkipRulesOptions(conf)
 			if err != nil {
 				return err
 			}
 			opportunityStore, openErr := cli.OpenConfiguredStore(ctx, conf)
 			switch {
 			case errors.Is(openErr, cli.ErrNoDatabase):
 				slog.Warn("not storing fetched opportunities", "reason", openErr)
 			case openErr != nil:
 				return openErr
 			default:
 				defer func() {
 					err = errors.Join(err, opportunityStore.Close())
 				}()
 				fetcherOptions = append(
 					fetcherOptions, fetcher.WithStore(opportunityStore),
 				)
 			}
 			return fetcher.New(&conf.Sources, fetcherOptions...).Run(ctx, since)
 		},
 	}
 	cmd.Flags().Time(
 		"since",
 		time.Time{},
 		[]string{
 			time.DateOnly, time.DateTime,
 		},
 		"contract updates published since YYYY-MM-DD[ hh:mm:ss] "+
 			"(default: each source's newest stored release, or the last day)",
 	)
 	cmd.Flags().Bool("foreground", false, "run in the foreground")
 	return cmd
 }
@@ -0,0 +1,89 @@
 package cli_test
 import (
 	"fmt"
 	"os"
 	"strings"
 	"testing"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/cli/cmdtest"
 )
 // The shared flags (--config, --debug and --quiet), run through a root
 // command like job-search's and a command that loads the configuration, as
 // every real command does.
 const appConfig = "logging:\n  level: info\n"
 // newRoot returns a root command with a "level" command that loads the
 // configuration and prints its log level.
 func newRoot() *cobra.Command {
 	opts := &cli.Options{}
 	return cmdtest.NewRoot(opts, &cobra.Command{
 		Use: "level",
 		RunE: func(cmd *cobra.Command, args []string) error {
 			conf, err := opts.RunSetup(cmd, true, true)
 			if err != nil {
 				return err
 			}
 			fmt.Println("level:", conf.Logging.Level)
 			return nil
 		},
 	})
 }
 // --debug and --quiet override the configured log level; --debug wins.
 func TestLogLevelFlags(t *testing.T) {
 	cmdtest.Setup(t, appConfig)
 	for _, tt := range []struct {
 		args []string
 		want string
 	}{
 		{nil, "level: info"},
 		{[]string{"--debug"}, "level: debug"},
 		{[]string{"-d"}, "level: debug"},
 		{[]string{"--quiet"}, "level: error"},
 		{[]string{"-q"}, "level: error"},
 		{[]string{"-q", "-d"}, "level: debug"},
 	} {
 		args := append([]string{"level"}, tt.args...)
 		t.Run(strings.Join(args, " "), func(t *testing.T) {
 			cmdtest.Contains(t, cmdtest.MustRun(t, newRoot, args...), tt.want)
 		})
 	}
 }
 // --config points the command at another file.
 func TestConfigFlag(t *testing.T) {
 	cmdtest.Setup(t, appConfig)
 	cmdtest.WriteFile(t, "other.yaml", "logging:\n  level: error\n")
 	for _, flag := range []string{"--config", "-c"} {
 		out := cmdtest.MustRun(t, newRoot, "level", flag, "other.yaml")
 		cmdtest.Contains(t, out, "level: error")
 	}
 	_, err := cmdtest.Run(t, newRoot, "", "level", "--config", "missing.yaml")
 	if err == nil {
 		t.Error("a missing --config file was accepted")
 	}
 }
 // A .env file is optional, but one that can't be parsed is an error.
 func TestEnvFile(t *testing.T) {
 	cmdtest.Setup(t, appConfig)
 	if err := os.Remove(".env"); err != nil {
 		t.Fatal(err)
 	}
 	if _, err := cmdtest.Run(t, newRoot, "", "level"); err != nil {
 		t.Errorf("without a .env file: %v; want no error", err)
 	}
 	cmdtest.WriteFile(t, ".env", "NOT VALID\n")
 	_, err := cmdtest.Run(t, newRoot, "", "level")
 	cmdtest.ErrContains(t, err, ".env")
 }
@@ -0,0 +1,29 @@
 package cli
 import (
 	"log/slog"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/logging"
 )
 // SetupInitialLogging starts logging to the terminal, at the level set by
 // cmd's --quiet and --debug flags (--debug wins), before any configuration
 // is loaded.
 func SetupInitialLogging(cmd *cobra.Command) {
 	// Start logging to terminal first - any commands that want to use file
 	// logging should activate it themselves
 	logging.LogToTerminal()
 	// Check for quiet flag
 	if quiet, _ := cmd.Flags().GetBool("quiet"); quiet {
 		logging.SetLevel(slog.LevelError)
 	}
 	// Check for debug logging enabled (overrides quiet flag)
 	debugEnabled, _ := cmd.Flags().GetBool("debug")
 	if debugEnabled {
 		logging.EnableDebug()
 	}
 }
@@ -0,0 +1,153 @@
 package migrate
 import (
 	"context"
 	"database/sql"
 	"errors"
 	"fmt"
 	"io"
 	"os"
 	"path/filepath"
 	"time"
 	"github.com/pressly/goose/v3"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/store"
 )
 // openMigrator loads the configuration and returns a migrator over the
 // store's embedded migrations for the configured database, opened with the
 // store's connection settings. The caller must close the returned *sql.DB.
 //
 // If the database file doesn't exist, create decides between asking whether
 // to create it (up) and failing (down, status: nothing to work on).
 func openMigrator(
 	cmd *cobra.Command, opts *cli.Options, create, createConfirmed bool,
 ) (*goose.Provider, *sql.DB, error) {
 	conf, err := opts.RunSetup(cmd, true, true)
 	if err != nil {
 		return nil, nil, fmt.Errorf("load configuration: %w", err)
 	}
 	file := conf.Database.File
 	if file == "" {
 		return nil, nil, errors.New(
 			"no database file specified. Update your configuration",
 		)
 	}
 	if _, err := os.Stat(file); errors.Is(err, os.ErrNotExist) {
 		if !create {
 			return nil, nil, fmt.Errorf("database file '%s' does not exist", file)
 		}
 		if !createConfirmed {
 			question := fmt.Sprintf(
 				"Database file '%s' does not exist. Would you like to create it?", file,
 			)
 			if !cli.Confirm(cmd.OutOrStdout(), cmd.InOrStdin(), question) {
 				return nil, nil, errors.New("no database file")
 			}
 		}
 	} else if err != nil {
 		return nil, nil, err
 	}
 	db, err := store.OpenDB(file)
 	if err != nil {
 		return nil, nil, err
 	}
 	migrator, err := store.NewMigrator(db)
 	if err != nil {
 		return nil, nil, errors.Join(err, db.Close())
 	}
 	return migrator, db, nil
 }
 // closeDB closes db when deferred, adding any error to *err.
 func closeDB(db *sql.DB, err *error) {
 	*err = errors.Join(*err, db.Close())
 }
 // errWriter writes until a write fails, then keeps the error, so a run of
 // prints needs checking only once.
 type errWriter struct {
 	out io.Writer
 	err error
 }
 // printf writes like fmt.Fprintf, unless an earlier write failed.
 func (e *errWriter) printf(format string, args ...any) {
 	if e.err == nil {
 		_, e.err = fmt.Fprintf(e.out, format, args...)
 	}
 }
 // printResults lists migrations that ran, e.g.
 //
 //	applied   00003_create_release_tags_table.sql  (2ms)
 func printResults(out io.Writer, results []*goose.MigrationResult) error {
 	output := &errWriter{
 		out: out,
 	}
 	for _, result := range results {
 		verb := "applied"
 		if result.Direction == "down" {
 			verb = "reverted"
 		}
 		output.printf(
 			"  %-9s %s  (%s)\n",
 			verb,
 			filepath.Base(result.Source.Path),
 			result.Duration.Round(time.Millisecond),
 		)
 	}
 	return output.err
 }
 // failure prints what ran before a failed migration and describes the
 // failure. Each migration runs in a transaction, so the failed one has been
 // rolled back and the database is at the last version that succeeded.
 func failure(out io.Writer, err error) error {
 	var partialErr *goose.PartialError
 	if !errors.As(err, &partialErr) {
 		return err
 	}
 	printErr := printResults(out, partialErr.Applied)
 	name := "unknown migration"
 	if partialErr.Failed != nil && partialErr.Failed.Source != nil {
 		name = filepath.Base(partialErr.Failed.Source.Path)
 	}
 	return errors.Join(
 		fmt.Errorf(
 			"migration %s failed and was rolled back; the database is at the last "+
 				"version that succeeded.\nMigration error: %w",
 			name,
 			partialErr.Err,
 		),
 		printErr,
 	)
 }
 // printVersion reports where the database ended up.
 func printVersion(
 	ctx context.Context, out io.Writer, migrator *goose.Provider,
 ) error {
 	dbVersion, err := migrator.GetDBVersion(ctx)
 	if err != nil {
 		return err
 	}
 	if dbVersion == 0 {
 		_, err = fmt.Fprintln(out, "Database cleared")
 	} else {
 		_, err = fmt.Fprintf(out, "Database migrated to version %d\n", dbVersion)
 	}
 	return err
 }
@@ -0,0 +1,116 @@
 package migrate
 import (
 	"bytes"
 	"errors"
 	"strings"
 	"testing"
 	"time"
 	"github.com/pressly/goose/v3"
 )
 // errWrite is the error failingWriter fails every write with.
 var errWrite = errors.New("write failed")
 // The commands are tested through the real command tree in cmd/jobsearch.
 // These tests cover reporting a failed migration, which a working set of
 // migrations can't trigger.
 func result(path, direction string) *goose.MigrationResult {
 	return &goose.MigrationResult{
 		Source: &goose.Source{
 			Path: "migrations/" + path,
 		},
 		Direction: direction,
 		Duration:  1500 * time.Microsecond,
 	}
 }
 func TestFailurePartial(t *testing.T) {
 	cause := errors.New("no such table: missing")
 	var out bytes.Buffer
 	err := failure(&out, &goose.PartialError{
 		Applied: []*goose.MigrationResult{result("00001_first.sql", "up")},
 		Failed:  result("00002_second.sql", "up"),
 		Err:     cause,
 	})
 	if want := "  applied   00001_first.sql  (2ms)\n"; out.String() != want {
 		t.Errorf("output = %q, want %q", out.String(), want)
 	}
 	if !errors.Is(err, cause) {
 		t.Errorf("err = %v, want it to wrap the migration error", err)
 	}
 	for _, want := range []string{
 		"00002_second.sql",
 		"rolled back",
 		"last version that succeeded",
 	} {
 		if err == nil || !strings.Contains(err.Error(), want) {
 			t.Errorf("err = %v, want it to mention %q", err, want)
 		}
 	}
 }
 func TestFailureWithoutFailedSource(t *testing.T) {
 	err := failure(&bytes.Buffer{}, &goose.PartialError{
 		Err: errors.New("boom"),
 	})
 	if err == nil || !strings.Contains(err.Error(), "unknown migration") {
 		t.Errorf("err = %v, want unknown migration", err)
 	}
 }
 func TestFailureOtherError(t *testing.T) {
 	cause := errors.New("database is locked")
 	var out bytes.Buffer
 	if err := failure(
 		&out,
 		cause,
 	); err != cause { //nolint:errorlint // checks the error is returned unchanged
 		t.Errorf("err = %v, want the error unchanged", err)
 	}
 	if out.Len() != 0 {
 		t.Errorf("printed %q for a non-migration error", out.String())
 	}
 }
 func TestPrintResults(t *testing.T) {
 	var out bytes.Buffer
 	err := printResults(&out, []*goose.MigrationResult{
 		result("00001_first.sql", "up"),
 		result("00002_second.sql", "down"),
 	})
 	if err != nil {
 		t.Fatal(err)
 	}
 	want := "  applied   00001_first.sql  (2ms)\n" +
 		"  reverted  00002_second.sql  (2ms)\n"
 	if out.String() != want {
 		t.Errorf("output = %q, want %q", out.String(), want)
 	}
 }
 // failingWriter fails every write.
 type failingWriter struct{}
 func (failingWriter) Write([]byte) (int, error) { return 0, errWrite }
 // A write error is returned, and the failed migration is still reported.
 func TestWriteErrors(t *testing.T) {
 	results := []*goose.MigrationResult{result("00001_first.sql", "up")}
 	if err := printResults(failingWriter{}, results); !errors.Is(err, errWrite) {
 		t.Errorf("printResults: err = %v, want the write error", err)
 	}
 	cause := errors.New("boom")
 	err := failure(failingWriter{}, &goose.PartialError{
 		Applied: results,
 		Failed:  result("00002_second.sql", "up"),
 		Err:     cause,
 	})
 	if !errors.Is(err, cause) || !errors.Is(err, errWrite) {
 		t.Errorf("failure: err = %v, want both the migration and write errors", err)
 	}
 }
@@ -0,0 +1,74 @@
 package migrate
 import (
 	"errors"
 	"fmt"
 	"log/slog"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/store"
 )
 func commandDown(opts *cli.Options) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "down",
 		Short: "Migrate SQLite database to a previous version",
 		Long: `Roll back the most recent migrations.
 --steps 0 rolls back every migration which deletes all data and must be
 confirmed by providing -yes.`,
 		Args: cobra.NoArgs,
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent pre-run", "command", "migrate.down")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent post-run", "command", "migrate.down")
 		},
 		RunE: func(cmd *cobra.Command, args []string) (err error) {
 			slog.Debug("command run", "command", "migrate.down")
 			yes, _ := cmd.Flags().GetBool("yes")
 			steps, _ := cmd.Flags().GetUint("steps")
 			migrator, db, err := openMigrator(cmd, opts, false, false)
 			if err != nil {
 				return err
 			}
 			defer closeDB(db, &err)
 			results, err := store.StepDown(cmd.Context(), migrator, int(steps), yes)
 			if len(results) == 0 && err == nil {
 				fmt.Println("No changes made to database (empty)")
 				return nil
 			}
 			if errors.Is(err, store.ErrWouldEmptyDatabase) {
 				return fmt.Errorf(
 					"%w\n\nTo confirm you wish to proceed, pass the --yes flag", err,
 				)
 			}
 			if printErr := printResults(cmd.OutOrStdout(), results); printErr != nil {
 				return errors.Join(err, printErr)
 			}
 			if err != nil {
 				return failure(cmd.OutOrStdout(), err)
 			}
 			return printVersion(cmd.Context(), cmd.OutOrStdout(), migrator)
 		},
 	}
 	cmd.Flags().Uint(
 		"steps", 0, "number of migrations to roll back. 0 = all (deletes all data)",
 	)
 	if err := cmd.MarkFlagRequired("steps"); err != nil {
 		panic(err) // only fails if the flag isn't defined
 	}
 	cmd.Flags().BoolP(
 		"yes", "y", false, "confirm you are intending to roll back every migration",
 	)
 	return cmd
 }
@@ -0,0 +1,44 @@
 // Package migrate provides the "migrate" command and its subcommands, which
 // apply, roll back and report the store's migrations (goose, embedded in the
 // store package) on the configured SQLite database.
 //
 // goose runs each migration in its own transaction, so a failed migration is
 // rolled back and leaves the database at the previous version.
 package migrate
 import (
 	"log/slog"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 )
 // NewCommand returns the "migrate" command, which migrates the database schema
 // (see internal/store).
 func NewCommand(opts *cli.Options) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "migrate",
 		Short: "Migrate SQLite database to latest version",
 		Long: `Migrate the SQLite database to the latest version.
 Running without a subcommand is the same as running "migrate up": every pending
 migration is applied.`,
 		Args: cobra.NoArgs,
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent pre-run", "command", "migrate")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent post-run", "command", "migrate")
 		},
 		RunE: func(cmd *cobra.Command, args []string) error {
 			slog.Debug("command run", "command", "migrate")
 			return runStepUp(cmd, opts, 0, false)
 		},
 	}
 	cmd.AddCommand(commandUp(opts))
 	cmd.AddCommand(commandDown(opts))
 	cmd.AddCommand(commandStatus(opts))
 	return cmd
 }
@@ -0,0 +1,197 @@
 package migrate
 import (
 	"fmt"
 	"path/filepath"
 	"strings"
 	"testing"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/cli/cmdtest"
 	"git.avsdev.uk/Go/job-search/internal/store"
 )
 // Run with: go test -tags sqlite_fts5 ./...
 //
 // These tests run the migrate commands as a user would, through a root
 // command like job-search's (see cmdtest).
 // dbConfig is a config.yaml with a database in the test's directory.
 const dbConfig = "logging:\n  level: info\ndatabase:\n  file: ./test.db\n"
 func newRoot() *cobra.Command {
 	opts := &cli.Options{}
 	return cmdtest.NewRoot(opts, NewCommand(opts))
 }
 func run(t *testing.T, stdin string, args ...string) (string, error) {
 	t.Helper()
 	return cmdtest.Run(t, newRoot, stdin, args...)
 }
 func mustRun(t *testing.T, args ...string) string {
 	t.Helper()
 	return cmdtest.MustRun(t, newRoot, args...)
 }
 // latestVersion is the schema version of this build's newest migration.
 func latestVersion(t *testing.T) int64 {
 	t.Helper()
 	db, err := store.OpenDB(filepath.Join(t.TempDir(), "version.db"))
 	if err != nil {
 		t.Fatal(err)
 	}
 	defer db.Close()
 	m, err := store.NewMigrator(db)
 	if err != nil {
 		t.Fatal(err)
 	}
 	return store.LatestVersion(m)
 }
 func TestMigrateUpCreateThenStatus(t *testing.T) {
 	cmdtest.Setup(t, dbConfig)
 	latest := latestVersion(t)
 	out := mustRun(t, "migrate", "up", "--create")
 	cmdtest.Contains(
 		t,
 		out,
 		"applied   00001_create_opportunities_table.sql",
 		fmt.Sprintf("Database migrated to version %d", latest),
 	)
 	if !cmdtest.Exists("test.db") {
 		t.Fatal("test.db was not created")
 	}
 	out = mustRun(t, "migrate", "status")
 	cmdtest.Contains(
 		t,
 		out,
 		fmt.Sprintf("Database version %d of %d (up-to-date)", latest, latest),
 		"00001_create_opportunities_table.sql",
 	)
 	out = mustRun(t, "migrate", "up")
 	cmdtest.Contains(t, out, "No changes made to database (up-to-date)")
 }
 // Plain "migrate" is "migrate up" with every pending migration.
 func TestMigrateWithoutSubcommand(t *testing.T) {
 	cmdtest.Setup(t, dbConfig)
 	cmdtest.WriteFile(t, "test.db", "") // exists, so no question is asked
 	out := mustRun(t, "migrate")
 	cmdtest.Contains(
 		t, out, fmt.Sprintf("Database migrated to version %d", latestVersion(t)),
 	)
 }
 func TestMigrateUpAsksBeforeCreating(t *testing.T) {
 	cmdtest.Setup(t, dbConfig)
 	out, err := run(t, "n\n", "migrate", "up")
 	cmdtest.ErrContains(t, err, "no database file")
 	cmdtest.Contains(
 		t,
 		out,
 		"Database file './test.db' does not exist. Would you like to create it?",
 	)
 	if cmdtest.Exists("test.db") {
 		t.Error("test.db was created after answering no")
 	}
 	out, err = run(t, "y\n", "migrate", "up")
 	if err != nil {
 		t.Fatalf("answering yes: %v\n%s", err, out)
 	}
 	if !cmdtest.Exists("test.db") {
 		t.Error("test.db was not created after answering yes")
 	}
 }
 func TestMigrateUpBySteps(t *testing.T) {
 	cmdtest.Setup(t, dbConfig)
 	latest := latestVersion(t)
 	out := mustRun(t, "migrate", "up", "--create", "--steps", "2")
 	cmdtest.Contains(t, out, "Database migrated to version 2")
 	out = mustRun(t, "migrate", "status")
 	cmdtest.Contains(
 		t,
 		out,
 		fmt.Sprintf("Database version 2 of %d (%d pending)", latest, latest-2),
 	)
 }
 func TestMigrateDown(t *testing.T) {
 	cmdtest.Setup(t, dbConfig)
 	latest := latestVersion(t)
 	mustRun(t, "migrate", "up", "--create")
 	out := mustRun(t, "migrate", "down", "--steps", "1")
 	cmdtest.Contains(
 		t,
 		out,
 		"reverted",
 		fmt.Sprintf("Database migrated to version %d", latest-1),
 	)
 	// Rolling back everything deletes all data, so it must be confirmed.
 	for _, args := range [][]string{
 		{"migrate", "down", "--steps", "0"},
 		{"migrate", "down", "--steps", "99"},
 	} {
 		t.Run(strings.Join(args, " "), func(t *testing.T) {
 			_, err := run(t, "", args...)
 			cmdtest.ErrContains(t, err, "pass the --yes flag")
 		})
 	}
 	cmdtest.Contains(
 		t,
 		mustRun(t, "migrate", "status"),
 		fmt.Sprintf("Database version %d of %d", latest-1, latest),
 	)
 	out = mustRun(t, "migrate", "down", "--steps", "0", "--yes")
 	cmdtest.Contains(t, out, "Database cleared")
 	out = mustRun(t, "migrate", "down", "--steps", "1")
 	cmdtest.Contains(t, out, "No changes made to database (empty)")
 }
 func TestMigrateDownRequiresSteps(t *testing.T) {
 	cmdtest.Setup(t, dbConfig)
 	mustRun(t, "migrate", "up", "--create")
 	_, err := run(t, "", "migrate", "down")
 	cmdtest.ErrContains(t, err, "steps")
 }
 func TestMigrateMissingDatabase(t *testing.T) {
 	cmdtest.Setup(t, dbConfig)
 	for _, args := range [][]string{
 		{"migrate", "status"},
 		{"migrate", "down", "--steps", "1"},
 	} {
 		t.Run(strings.Join(args, " "), func(t *testing.T) {
 			_, err := run(t, "", args...)
 			cmdtest.ErrContains(t, err, "does not exist")
 		})
 	}
 	if cmdtest.Exists("test.db") {
 		t.Error("status or down created the database")
 	}
 }
 func TestMigrateWithoutDatabaseConfigured(t *testing.T) {
 	cmdtest.Setup(t, "logging:\n  level: info\n")
 	_, err := run(t, "", "migrate", "status")
 	cmdtest.ErrContains(t, err, "no database file specified")
 }
@@ -0,0 +1,112 @@
 package migrate
 import (
 	"context"
 	"errors"
 	"io"
 	"log/slog"
 	"path/filepath"
 	"text/tabwriter"
 	"github.com/pressly/goose/v3"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/store"
 )
 // printStatus lists every migration as applied (with when) or pending,
 // after a summary line, e.g.
 //
 //	Database version 4 of 6 (2 pending)
 //	  1  applied  2026-10-07 10:12:01  00001_create_opportunities_table.sql
 //	  ...
 //	  5  pending                       00005_create_opportunity_classifications_table.sql
 func printStatus(
 	ctx context.Context, out io.Writer, migrator *goose.Provider,
 ) error {
 	statuses, err := migrator.Status(ctx)
 	if err != nil {
 		return err
 	}
 	current, err := migrator.GetDBVersion(ctx)
 	if err != nil {
 		return err
 	}
 	latest := store.LatestVersion(migrator)
 	pending := 0
 	for _, status := range statuses {
 		if status.State == goose.StatePending {
 			pending++
 		}
 	}
 	output := &errWriter{
 		out: out,
 	}
 	switch {
 	case current > latest:
 		output.printf(
 			"Database version %d is newer than this build (latest %d)\n",
 			current,
 			latest,
 		)
 	case pending == 0:
 		output.printf("Database version %d of %d (up-to-date)\n", current, latest)
 	default:
 		output.printf(
 			"Database version %d of %d (%d pending)\n", current, latest, pending,
 		)
 	}
 	if output.err != nil {
 		return output.err
 	}
 	table := tabwriter.NewWriter(out, 0, 0, 2, ' ', 0)
 	output = &errWriter{
 		out: table,
 	}
 	for _, status := range statuses {
 		applied := ""
 		if status.State == goose.StateApplied {
 			applied = status.AppliedAt.Local().Format("2006-01-02 15:04:05")
 		}
 		output.printf(
 			"  %d\t%s\t%s\t%s\n",
 			status.Source.Version,
 			status.State,
 			applied,
 			filepath.Base(status.Source.Path),
 		)
 	}
 	return errors.Join(output.err, table.Flush())
 }
 func commandStatus(opts *cli.Options) *cobra.Command {
 	return &cobra.Command{
 		Use:   "status",
 		Short: "Show which migrations have been applied to the SQLite database",
 		Args:  cobra.NoArgs,
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent pre-run", "command", "migrate.status")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent post-run", "command", "migrate.status")
 		},
 		RunE: func(cmd *cobra.Command, args []string) (err error) {
 			slog.Debug("command run", "command", "migrate.status")
 			migrator, db, err := openMigrator(cmd, opts, false, false)
 			if err != nil {
 				return err
 			}
 			defer closeDB(db, &err)
 			return printStatus(cmd.Context(), cmd.OutOrStdout(), migrator)
 		},
 	}
 }
@@ -0,0 +1,71 @@
 package migrate
 import (
 	"errors"
 	"fmt"
 	"log/slog"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/store"
 )
 // runStepUp applies steps pending migrations (0 means all), printing each one
 // and the resulting version. create says the user already agreed to create a
 // missing database file; otherwise they are asked.
 func runStepUp(
 	cmd *cobra.Command, opts *cli.Options, steps int, create bool,
 ) (err error) {
 	migrator, db, err := openMigrator(cmd, opts, true, create)
 	if err != nil {
 		return err
 	}
 	defer closeDB(db, &err)
 	results, err := store.StepUp(cmd.Context(), migrator, int(steps))
 	if len(results) == 0 && err == nil {
 		_, err = fmt.Fprintln(
 			cmd.OutOrStdout(), "No changes made to database (up-to-date)",
 		)
 		return err
 	}
 	if printErr := printResults(cmd.OutOrStdout(), results); printErr != nil {
 		return errors.Join(err, printErr)
 	}
 	if err != nil {
 		return failure(cmd.OutOrStdout(), err)
 	}
 	return printVersion(cmd.Context(), cmd.OutOrStdout(), migrator)
 }
 func commandUp(opts *cli.Options) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "up",
 		Short: "Migrate SQLite database to the latest version, or by a number of steps",
 		Args:  cobra.NoArgs,
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent pre-run", "command", "migrate.up")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent post-run", "command", "migrate.up")
 		},
 		RunE: func(cmd *cobra.Command, args []string) error {
 			slog.Debug("command run", "command", "migrate.up")
 			steps, _ := cmd.Flags().GetUint("steps")
 			create, _ := cmd.Flags().GetBool("create")
 			return runStepUp(cmd, opts, int(steps), create)
 		},
 	}
 	cmd.Flags().Uint("steps", 0, "number of migrations to apply. 0 = all pending")
 	cmd.Flags().Bool(
 		"create", false, "create the database file if it does not already exist",
 	)
 	return cmd
 }
@@ -0,0 +1,85 @@
 package cli
 import (
 	"fmt"
 	"log/slog"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/config"
 	"git.avsdev.uk/Go/job-search/internal/logging"
 )
 // Options holds the flags shared by every command. Bind registers them on the
 // root command and RunSetup acts on them.
 type Options struct {
 	configFile string
 	debug      bool
 	quiet      bool
 }
 // Bind registers the shared flags as persistent flags on the root command cmd.
 func (o *Options) Bind(cmd *cobra.Command) {
 	flags := cmd.PersistentFlags()
 	flags.StringVarP(&o.configFile, "config", "c", "", "configuration file")
 	flags.BoolVarP(&o.debug, "debug", "d", false, "enable debug messages")
 	flags.BoolVarP(
 		&o.quiet, "quiet", "q", false, "don't report info or warning messages",
 	)
 }
 // RunSetup prepares a command to run and returns its configuration. When
 // loadConfig is true it loads the configuration (from the --config file if
 // given), with --debug and --quiet overriding its log level, and then, unless
 // terminalLog is true, switches logging to the configured log file (staying
 // on the terminal, with a warning, if none is configured).
 func (o *Options) RunSetup(
 	cmd *cobra.Command, loadConfig, terminalLog bool,
 ) (*config.JobSearchConfig, error) {
 	// Flags and arguments are valid from here on, so errors are runtime
 	// failures: report them without cobra's usage text.
 	cmd.SilenceUsage = true
 	// Check for debug level or quiet suppression via args
 	conf := &config.JobSearchConfig{
 		Logging: config.JSCLogging{},
 	}
 	if o.debug {
 		conf.Logging.Level = "debug"
 	} else if o.quiet {
 		conf.Logging.Level = "error"
 	}
 	// Only load config if command requests it (nearly always true)
 	if loadConfig {
 		// Load config - from specified file if provided
 		if err := config.Init(o.configFile, conf); err != nil {
 			return nil, err
 		}
 		// Change logging level based off newly loaded config
 		if conf.Logging.LogLevel() <= slog.LevelDebug && !logging.IsDebugEnabled() {
 			logging.EnableDebug()
 		}
 		// Check if the logging should be switched to file
 		if !terminalLog {
 			if conf.Logging.File == "" {
 				slog.Warn("no log file configured; logging to the terminal")
 			} else if err := logging.LogToFile(conf.Logging.File); err != nil {
 				return nil, fmt.Errorf("opening log file: %w", err)
 			}
 		}
 	}
 	return conf, nil
 }
 // Finish shuts down any open file logger
 func (o *Options) Finish() error {
 	if logging.IsFileLogger() {
 		return logging.Shutdown()
 	}
 	return nil
 }
@@ -0,0 +1,246 @@
 package serve
 import (
 	"context"
 	"errors"
 	"fmt"
 	"log/slog"
 	"os"
 	"os/signal"
 	"strconv"
 	"strings"
 	"syscall"
 	"time"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/config"
 	"git.avsdev.uk/Go/job-search/internal/events"
 	"git.avsdev.uk/Go/job-search/internal/fetcher"
 	"git.avsdev.uk/Go/job-search/internal/frontend"
 	"git.avsdev.uk/Go/job-search/internal/logging"
 	"git.avsdev.uk/Go/job-search/internal/server"
 )
 // NewCommand returns the serve command, which serves the REST API, the
 // WebSocket endpoint and the frontend, and runs a worker for every enabled
 // source.
 func NewCommand(opts *cli.Options) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "serve",
 		Short: "Serve the REST API, WebSocket and frontend, and fetch from sources",
 		Long: `Serve the REST API under /api, the WebSocket endpoint under /ws, and the
 frontend under /: the files in server.frontend.dir if it is set, or else the
 build embedded in the binary (copied there by npm run sync in web/ before
 building), if there is one.
 It also runs a worker for every enabled source, as fetcher service does,
 saving what they fetch to database.file if it is set, and pushing an
 OpportunitiesUpdated event to every WebSocket client when a save stores new
 opportunities or new releases. A source that stops with an error stops the
 server too, and the server stopping stops the sources. With database.file
 set, the API serves the stored opportunities under /api/opportunities.`,
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent pre-run", "command", "serve")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent post-run", "command", "serve")
 		},
 		Args: cobra.NoArgs,
 		RunE: func(cmd *cobra.Command, _ []string) (err error) {
 			slog.Debug("command run", "command", "serve")
 			addrOverride, _ := cmd.Flags().GetString("listen")
 			hostOverride, _ := cmd.Flags().GetString("host")
 			portOverride, _ := cmd.Flags().GetInt("port")
 			foreground, _ := cmd.Flags().GetBool("foreground")
 			accessLogOverride, _ := cmd.Flags().GetString("access-log")
 			since, _ := cmd.Flags().GetTime("since")
 			if addrOverride != "" {
 				listenParts := strings.Split(addrOverride, ":")
 				if len(listenParts) != 2 {
 					return errors.New(
 						"--listen should be in the format of 127.0.0.1:9393",
 					)
 				}
 				portOverride, err = strconv.Atoi(listenParts[1])
 				if err != nil {
 					return fmt.Errorf("failed to parse --listen port: %w", err)
 				}
 				hostOverride = listenParts[0]
 			}
 			conf, err := opts.RunSetup(cmd, true, foreground)
 			if err != nil {
 				return fmt.Errorf("load configuration: %w", err)
 			}
 			if portOverride != 0 {
 				conf.Server.Port = portOverride
 			}
 			if hostOverride != "" {
 				conf.Server.Host = hostOverride
 			}
 			if cmd.Flags().Changed("access-log") {
 				conf.Server.AccessLog = accessLogOverride
 			}
 			options, closeAccessLog, err := serverOptions(&conf.Server)
 			if err != nil {
 				return err
 			}
 			defer func() {
 				err = errors.Join(err, closeAccessLog())
 			}()
 			// ctx is cancelled on the first SIGINT (Ctrl+C) or SIGTERM.
 			ctx, stop := signal.NotifyContext(
 				context.Background(), os.Interrupt, syscall.SIGTERM,
 			)
 			defer stop()
 			// After the first signal, restore default signal handling so a second
 			// Ctrl+C force-kills the process if graceful shutdown hangs.
 			go func() {
 				<-ctx.Done()
 				stop()
 			}()
 			// The store is shared by the sources, which save to it, and the API,
 			// which serves it.
 			fetcherOptions, err := cli.SkipRulesOptions(conf)
 			if err != nil {
 				return err
 			}
 			opportunityStore, openErr := cli.OpenConfiguredStore(ctx, conf)
 			switch {
 			case errors.Is(openErr, cli.ErrNoDatabase):
 				slog.Warn(
 					"opportunities aren't stored or served", "reason", openErr,
 				)
 			case openErr != nil:
 				return openErr
 			default:
 				defer func() {
 					err = errors.Join(err, opportunityStore.Close())
 				}()
 				fetcherOptions = append(
 					fetcherOptions, fetcher.WithStore(opportunityStore),
 				)
 				options = append(
 					options, server.WithStore(opportunityStore),
 				)
 			}
 			// Events from the sources and the API, such as new opportunities and
 			// changed marks, are pushed to every WebSocket client.
 			var bus events.Bus
 			options = append(options, server.WithEvents(&bus))
 			appServer, err := server.New(&conf.Server, options...)
 			if err != nil {
 				return err
 			}
 			// runCtx stops whichever of the server and the sources is still
 			// running once the other has stopped.
 			runCtx, cancelRun := context.WithCancel(ctx)
 			defer cancelRun()
 			fetcherOptions = append(fetcherOptions, fetcher.WithEvents(&bus))
 			sourceFetcher := fetcher.New(&conf.Sources, fetcherOptions...)
 			fetcherDone := make(chan error, 1)
 			go func() {
 				fetchErr := sourceFetcher.Run(runCtx, since)
 				// Only a failure stops the server: with no sources enabled, Run
 				// returns nil straight away.
 				if fetchErr != nil {
 					cancelRun()
 				}
 				fetcherDone <- fetchErr
 			}()
 			serverErr := appServer.Run(runCtx)
 			cancelRun()
 			return errors.Join(serverErr, <-fetcherDone)
 		},
 	}
 	flags := cmd.Flags()
 	flags.String(
 		"listen", "", "listen address (default from config, 127.0.0.1:9393)",
 	)
 	flags.String(
 		"host", "", "listen address (default from config, 127.0.0.1)",
 	)
 	flags.Int("port", 0, "listen address (default from config, 9393)")
 	flags.Bool("foreground", false, "run in the foreground")
 	flags.String(
 		"access-log",
 		"",
 		`access log file, or "-" for stdout; "" for none `+
 			"(default from config, server.access_log)",
 	)
 	flags.Time(
 		"since",
 		time.Time{},
 		[]string{
 			time.DateOnly, time.DateTime,
 		},
 		"contract updates published since YYYY-MM-DD[ hh:mm:ss] "+
 			"(default: each source's newest stored release, or the last day)",
 	)
 	cmd.MarkFlagsMutuallyExclusive("listen", "host")
 	cmd.MarkFlagsMutuallyExclusive("listen", "port")
 	return cmd
 }
 // serverOptions returns the server's options for conf: the access log, if
 // server.access_log is set, and the frontend's files: those in
 // server.frontend.dir if it is set (checked first), or else the build
 // embedded in the binary, if there is one. closeAccessLog closes the access
 // log; call it once the server has stopped.
 func serverOptions(
 	conf *config.JSCServer,
 ) ([]server.Option, func() error, error) {
 	var options []server.Option
 	closeAccessLog := func() error {
 		return nil
 	}
 	if conf.AccessLog != "" {
 		accessLog, closeFile, err := logging.OpenAccessLog(conf.AccessLog)
 		if err != nil {
 			return nil, nil, err
 		}
 		closeAccessLog = closeFile
 		options = append(options, server.WithAccessLog(accessLog))
 	}
 	if dir := conf.Frontend.Dir; dir != "" {
 		files := os.DirFS(dir)
 		err := frontend.CheckFS(files)
 		if err != nil {
 			return nil, nil, errors.Join(
 				fmt.Errorf("server.frontend.dir %s: %w", dir, err),
 				closeAccessLog(),
 			)
 		}
 		options = append(options, server.WithFrontend(files))
 		return options, closeAccessLog, nil
 	}
 	files := frontend.Embedded()
 	if files == nil {
 		slog.Warn("no frontend embedded and server.frontend.dir not set")
 		return options, closeAccessLog, nil
 	}
 	options = append(options, server.WithFrontend(files))
 	return options, closeAccessLog, nil
 }
@@ -0,0 +1,27 @@
 package cli
 import (
 	"fmt"
 	"git.avsdev.uk/Go/job-search/internal/config"
 	"git.avsdev.uk/Go/job-search/internal/fetcher"
 	"git.avsdev.uk/Go/job-search/internal/skiprules"
 )
 // SkipRulesOptions opens the rules file at sources.skip_rules_file and
 // returns the fetcher option that applies it, or none if no file is
 // configured. A missing or invalid file is an error, so a command doesn't
 // start without the rules it was given.
 func SkipRulesOptions(conf *config.JobSearchConfig) ([]fetcher.Option, error) {
 	path := conf.Sources.SkipRulesFile
 	if path == "" {
 		return nil, nil
 	}
 	rulesFile, err := skiprules.Open(path)
 	if err != nil {
 		return nil, fmt.Errorf("sources.skip_rules_file: %w", err)
 	}
 	return []fetcher.Option{fetcher.WithSkipRules(rulesFile)}, nil
 }
@@ -0,0 +1,174 @@
 package skiprules
 import (
 	"context"
 	"errors"
 	"fmt"
 	"io"
 	"log/slog"
 	"text/tabwriter"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/skiprules"
 	"git.avsdev.uk/Go/job-search/internal/store"
 	"git.avsdev.uk/Go/job-search/internal/types"
 )
 // applyPageSize is how many opportunities apply reads at a time.
 const applyPageSize = 500
 // match is a stored opportunity the rules skip, and the filter that does.
 type match struct {
 	opportunity types.Opportunity
 	filter      string
 }
 // commandApply returns the "apply" command, which marks the stored
 // opportunities that match the rules Skipped.
 func commandApply(opts *cli.Options) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "apply [file]",
 		Short: "Mark stored opportunities that match skip rules Skipped",
 		Long: `Mark the stored opportunities that match the skip rules Skipped,
 listing each one. The rules are read from the file given, or else from
 sources.skip_rules_file. Only the Skipped mark is ever set, and never
 cleared: opportunities already skipped are left alone. With --dry-run, the
 matches are listed but nothing is changed.
 Each change is recorded for clients to catch up from, but isn't pushed to
 open browsers as a running "serve" pushes its own changes.`,
 		Args: cobra.MaximumNArgs(1),
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent pre-run", "command", "skip-rules.apply")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent post-run", "command", "skip-rules.apply")
 		},
 		RunE: func(cmd *cobra.Command, args []string) (err error) {
 			slog.Debug("command run", "command", "skip-rules.apply")
 			dryRun, _ := cmd.Flags().GetBool("dry-run")
 			conf, err := opts.RunSetup(cmd, true, true)
 			if err != nil {
 				return fmt.Errorf("load configuration: %w", err)
 			}
 			rules, _, err := loadRules(conf, args)
 			if err != nil {
 				return err
 			}
 			opportunityStore, err := cli.OpenConfiguredStore(cmd.Context(), conf)
 			if err != nil {
 				return err
 			}
 			defer func() {
 				err = errors.Join(err, opportunityStore.Close())
 			}()
 			matches, err := findMatches(cmd.Context(), opportunityStore, rules)
 			if err != nil {
 				return err
 			}
 			err = printMatches(cmd.OutOrStdout(), matches)
 			if err != nil {
 				return err
 			}
 			if dryRun {
 				_, err = fmt.Fprintf(
 					cmd.OutOrStdout(),
 					"%d opportunities would be skipped (dry run)\n",
 					len(matches),
 				)
 				return err
 			}
 			err = skipAll(cmd.Context(), opportunityStore, matches)
 			if err != nil {
 				return err
 			}
 			_, err = fmt.Fprintf(
 				cmd.OutOrStdout(), "Skipped %d opportunities\n", len(matches),
 			)
 			return err
 		},
 	}
 	cmd.Flags().Bool("dry-run", false, "list the matches without changing them")
 	return cmd
 }
 // findMatches returns the stored opportunities not yet skipped that rules
 // match, newest first. They are all read before any is changed, so changing
 // them doesn't move the pages being read.
 func findMatches(
 	ctx context.Context, opportunityStore *store.Store, rules *skiprules.Rules,
 ) ([]match, error) {
 	var matches []match
 	for offset := 0; ; offset += applyPageSize {
 		page, err := opportunityStore.ListOpportunities(
 			ctx,
 			store.OpportunityFilter{
 				Skipped: store.MarkUnset,
 				Limit:   applyPageSize,
 				Offset:  offset,
 			},
 		)
 		if err != nil {
 			return nil, fmt.Errorf("listing opportunities: %w", err)
 		}
 		for i := range page {
 			filter, matched := rules.Match(&page[i])
 			if matched {
 				matches = append(matches, match{
 					opportunity: page[i],
 					filter:      filter,
 				})
 			}
 		}
 		if len(page) < applyPageSize {
 			return matches, nil
 		}
 	}
 }
 // printMatches lists matches, one per line: UUID, filter and title.
 func printMatches(out io.Writer, matches []match) error {
 	table := tabwriter.NewWriter(out, 0, 0, 2, ' ', 0)
 	for _, found := range matches {
 		_, err := fmt.Fprintf(
 			table,
 			"%s\t%s\t%s\n",
 			found.opportunity.UUID,
 			found.filter,
 			found.opportunity.Title,
 		)
 		if err != nil {
 			return err
 		}
 	}
 	return table.Flush()
 }
 // skipAll sets the Skipped mark of every match, keeping its other marks.
 func skipAll(
 	ctx context.Context, opportunityStore *store.Store, matches []match,
 ) error {
 	for _, found := range matches {
 		marks := found.opportunity.Marks
 		marks.Skipped = true
 		if err := opportunityStore.SetMarks(ctx, marks); err != nil {
 			return fmt.Errorf(
 				"skipping %s: %w", found.opportunity.UUID, err,
 			)
 		}
 	}
 	return nil
 }
@@ -0,0 +1,44 @@
 package skiprules
 import (
 	"fmt"
 	"log/slog"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 )
 // commandCheck returns the "check" command, which reports whether a rules
 // file is valid.
 func commandCheck(opts *cli.Options) *cobra.Command {
 	return &cobra.Command{
 		Use:   "check [file]",
 		Short: "Check a skip rules file (default: sources.skip_rules_file)",
 		Args:  cobra.MaximumNArgs(1),
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent pre-run", "command", "skip-rules.check")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent post-run", "command", "skip-rules.check")
 		},
 		RunE: func(cmd *cobra.Command, args []string) error {
 			slog.Debug("command run", "command", "skip-rules.check")
 			conf, err := opts.RunSetup(cmd, true, true)
 			if err != nil {
 				return fmt.Errorf("load configuration: %w", err)
 			}
 			rules, path, err := loadRules(conf, args)
 			if err != nil {
 				return err
 			}
 			_, err = fmt.Fprintf(
 				cmd.OutOrStdout(), "%s is valid: %s\n", path, describeRules(rules),
 			)
 			return err
 		},
 	}
 }
@@ -0,0 +1,72 @@
 // Package skiprules provides the "skip-rules" command and its subcommands,
 // which check a skip rules file and apply it to the opportunities already
 // stored (see internal/skiprules).
 package skiprules
 import (
 	"errors"
 	"fmt"
 	"log/slog"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/config"
 	"git.avsdev.uk/Go/job-search/internal/skiprules"
 )
 // NewCommand returns the "skip-rules" command, with "check" and "apply".
 func NewCommand(opts *cli.Options) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "skip-rules",
 		Short: "Check skip rules, or apply them to stored opportunities",
 		Long: `Skip rules mark new opportunities Skipped as the sources fetch
 them. They are read from sources.skip_rules_file, and read again when it
 changes.
 "check" reports whether a rules file is valid, and "apply" marks the stored
 opportunities that match Skipped. Both read sources.skip_rules_file unless
 given another file.`,
 		Args: cobra.NoArgs,
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent pre-run", "command", "skip-rules")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent post-run", "command", "skip-rules")
 		},
 	}
 	cmd.AddCommand(commandCheck(opts))
 	cmd.AddCommand(commandApply(opts))
 	return cmd
 }
 // loadRules returns the rules in the file named by args, if given, or else
 // in sources.skip_rules_file, and the file's path.
 func loadRules(
 	conf *config.JobSearchConfig, args []string,
 ) (*skiprules.Rules, string, error) {
 	path := conf.Sources.SkipRulesFile
 	if len(args) > 0 {
 		path = args[0]
 	}
 	if path == "" {
 		return nil, "", errors.New(
 			"no rules file given, and sources.skip_rules_file is not set",
 		)
 	}
 	rules, err := skiprules.Load(path)
 	if err != nil {
 		return nil, "", err
 	}
 	return rules, path, nil
 }
 // describeRules returns a summary of rules for output, e.g. "3 filters (2
 // active)".
 func describeRules(rules *skiprules.Rules) string {
 	filters, active := rules.Filters()
 	return fmt.Sprintf("%d filters (%d active)", filters, active)
 }
@@ -0,0 +1,175 @@
 package skiprules
 import (
 	"context"
 	"strings"
 	"testing"
 	"time"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/cli/cmdtest"
 	"git.avsdev.uk/Go/job-search/internal/store"
 	"git.avsdev.uk/Go/job-search/internal/types"
 )
 // Run with: go test -tags sqlite_fts5 ./...
 //
 // These tests run the skip-rules commands as a user would, through a root
 // command like job-search's (see cmdtest).
 // conf is a config.yaml with a database and rules file in the test's
 // directory.
 const conf = "logging:\n  level: info\n" +
 	"database:\n  file: ./test.db\n" +
 	"sources:\n  skip_rules_file: ./rules.json\n"
 // rules skips buyer X's opportunities.
 const rules = `[{"comment": "buyer X", "active": true,
 	"rules": [{"rule_type": "buyer", "values": ["X"]}]}]`
 func newRoot() *cobra.Command {
 	opts := &cli.Options{}
 	return cmdtest.NewRoot(opts, NewCommand(opts))
 }
 // storeOpportunities creates test.db holding ocds-1 (buyer X), ocds-2 (buyer
 // Y) and ocds-3 (buyer X, already skipped and interested).
 func storeOpportunities(t *testing.T) {
 	t.Helper()
 	ctx := context.Background()
 	s, err := store.Open(ctx, "test.db", store.WithAutoMigrate())
 	if err != nil {
 		t.Fatal(err)
 	}
 	defer s.Close()
 	buyers := []string{"X", "Y", "X"}
 	for i, buyer := range buyers {
 		uuid := "ocds-" + string(rune('1'+i))
 		_, err := s.SaveOpportunity(ctx, &types.Opportunity{
 			UUID:     uuid,
 			UUIDType: "ocid",
 			Title:    "Opportunity " + uuid,
 			BuyerID:  buyer,
 			Releases: []types.OpportunityRelease{
 				{
 					ReleaseID:   "r1",
 					ReleaseDate: time.Date(2026, 10, 1+i, 12, 0, 0, 0, time.UTC),
 					JSON:        "{}",
 				},
 			},
 		})
 		if err != nil {
 			t.Fatal(err)
 		}
 	}
 	err = s.SetMarks(ctx, types.OpportunityMarks{
 		UUID:       "ocds-3",
 		Skipped:    true,
 		Interested: true,
 	})
 	if err != nil {
 		t.Fatal(err)
 	}
 }
 // marks returns the stored marks of uuid.
 func marks(t *testing.T, uuid string) types.OpportunityMarks {
 	t.Helper()
 	s, err := store.Open(context.Background(), "test.db")
 	if err != nil {
 		t.Fatal(err)
 	}
 	defer s.Close()
 	opportunity, err := s.GetOpportunity(context.Background(), uuid)
 	if err != nil {
 		t.Fatal(err)
 	}
 	return opportunity.Marks
 }
 func TestCheck(t *testing.T) {
 	cmdtest.Setup(t, conf)
 	cmdtest.WriteFile(t, "rules.json", rules)
 	cmdtest.WriteFile(t, "other.json", `[]`)
 	cmdtest.WriteFile(t, "bad.json", `[{"comment": "c"}]`)
 	out := cmdtest.MustRun(t, newRoot, "skip-rules", "check")
 	cmdtest.Contains(t, out, "./rules.json is valid: 1 filters (1 active)")
 	out = cmdtest.MustRun(t, newRoot, "skip-rules", "check", "other.json")
 	cmdtest.Contains(t, out, "other.json is valid: 0 filters (0 active)")
 	_, err := cmdtest.Run(t, newRoot, "", "skip-rules", "check", "bad.json")
 	cmdtest.ErrContains(t, err, `bad.json: filter 1 ("c"): skiprules: active`)
 	_, err = cmdtest.Run(t, newRoot, "", "skip-rules", "check", "none.json")
 	cmdtest.ErrContains(t, err, "none.json")
 }
 func TestCheckNoFile(t *testing.T) {
 	cmdtest.Setup(t, "logging:\n  level: info\n")
 	_, err := cmdtest.Run(t, newRoot, "", "skip-rules", "check")
 	cmdtest.ErrContains(t, err, "sources.skip_rules_file is not set")
 }
 func TestApplyDryRun(t *testing.T) {
 	cmdtest.Setup(t, conf)
 	cmdtest.WriteFile(t, "rules.json", rules)
 	storeOpportunities(t)
 	out := cmdtest.MustRun(t, newRoot, "skip-rules", "apply", "--dry-run")
 	cmdtest.Contains(
 		t,
 		out,
 		"ocds-1  buyer X  Opportunity ocds-1",
 		"1 opportunities would be skipped (dry run)",
 	)
 	if strings.Contains(out, "ocds-2") || strings.Contains(out, "ocds-3") {
 		t.Errorf("listed a non-match or an already skipped one:\n%s", out)
 	}
 	if marks(t, "ocds-1").Skipped {
 		t.Error("a dry run skipped ocds-1")
 	}
 }
 // apply sets only Skipped, on the matches not yet skipped, and leaves the
 // other marks alone.
 func TestApply(t *testing.T) {
 	cmdtest.Setup(t, conf)
 	cmdtest.WriteFile(t, "rules.json", rules)
 	storeOpportunities(t)
 	out := cmdtest.MustRun(t, newRoot, "skip-rules", "apply")
 	cmdtest.Contains(t, out, "ocds-1  buyer X", "Skipped 1 opportunities")
 	if !marks(t, "ocds-1").Skipped {
 		t.Error("ocds-1 wasn't skipped")
 	}
 	if marks(t, "ocds-2").Skipped {
 		t.Error("ocds-2 (buyer Y) was skipped")
 	}
 	if got := marks(t, "ocds-3"); !got.Skipped || !got.Interested {
 		t.Errorf("ocds-3's marks changed: %+v", got)
 	}
 	out = cmdtest.MustRun(t, newRoot, "skip-rules", "apply")
 	cmdtest.Contains(t, out, "Skipped 0 opportunities")
 }
 // A rules file given on the command line is used instead of the
 // configured one.
 func TestApplyGivenFile(t *testing.T) {
 	cmdtest.Setup(t, conf)
 	cmdtest.WriteFile(t, "rules.json", rules)
 	cmdtest.WriteFile(t, "y.json", strings.ReplaceAll(rules, `"X"`, `"Y"`))
 	storeOpportunities(t)
 	out := cmdtest.MustRun(t, newRoot, "skip-rules", "apply", "y.json")
 	cmdtest.Contains(t, out, "ocds-2", "Skipped 1 opportunities")
 	if !marks(t, "ocds-2").Skipped || marks(t, "ocds-1").Skipped {
 		t.Error("y.json's rules weren't the ones applied")
 	}
 }
@@ -0,0 +1,43 @@
 package cli
 import (
 	"os"
 	"path/filepath"
 	"strings"
 	"testing"
 	"git.avsdev.uk/Go/job-search/internal/config"
 )
 // SkipRulesOptions gives no option without a file, one with a valid file,
 // and refuses a missing or invalid file, naming the setting.
 func TestSkipRulesOptions(t *testing.T) {
 	dir := t.TempDir()
 	valid := filepath.Join(dir, "valid.json")
 	invalid := filepath.Join(dir, "invalid.json")
 	os.WriteFile(valid, []byte(`[]`), 0o600)
 	os.WriteFile(invalid, []byte(`[{}]`), 0o600)
 	tests := []struct {
 		file    string
 		options int
 		err     string
 	}{
 		{"", 0, ""},
 		{valid, 1, ""},
 		{filepath.Join(dir, "missing.json"), 0, "sources.skip_rules_file: "},
 		{invalid, 0, "sources.skip_rules_file: skip rules "},
 	}
 	for _, tt := range tests {
 		conf := &config.JobSearchConfig{}
 		conf.Sources.SkipRulesFile = tt.file
 		options, err := SkipRulesOptions(conf)
 		if len(options) != tt.options {
 			t.Errorf("%q: %d options; want %d", tt.file, len(options), tt.options)
 		}
 		if (err == nil) != (tt.err == "") ||
 			err != nil && !strings.Contains(err.Error(), tt.err) {
 			t.Errorf("%q: err %v; want %q", tt.file, err, tt.err)
 		}
 	}
 }
@@ -0,0 +1,56 @@
 package cli
 import (
 	"context"
 	"errors"
 	"fmt"
 	"os"
 	"git.avsdev.uk/Go/job-search/internal/config"
 	"git.avsdev.uk/Go/job-search/internal/store"
 )
 // ErrNoDatabase is returned by OpenConfiguredStore when database.file isn't
 // set.
 var ErrNoDatabase = errors.New("database.file is not set")
 // OpenConfiguredStore opens the store at database.file, or returns
 // ErrNoDatabase if none is configured. A missing file or a database that
 // needs migrating fails with a message saying how to create or migrate it; a
 // missing file isn't created, as store.Open would. The caller must close the
 // store.
 func OpenConfiguredStore(
 	ctx context.Context, conf *config.JobSearchConfig,
 ) (*store.Store, error) {
 	file := conf.Database.File
 	if file == "" {
 		return nil, ErrNoDatabase
 	}
 	_, err := os.Stat(file)
 	if errors.Is(err, os.ErrNotExist) {
 		return nil, fmt.Errorf(
 			"database %s does not exist: run `job-search migrate up --create`",
 			file,
 		)
 	}
 	if err != nil {
 		return nil, fmt.Errorf("database %s: %w", file, err)
 	}
 	opportunityStore, err := store.Open(
 		ctx,
 		file,
 		store.WithChangeBatchRetention(conf.Database.ChangeBatchRetention),
 	)
 	if errors.Is(err, store.ErrNeedsMigration) {
 		return nil, fmt.Errorf(
 			"database %s: %w: run `job-search migrate`", file, err,
 		)
 	}
 	if err != nil {
 		return nil, fmt.Errorf("database %s: %w", file, err)
 	}
 	return opportunityStore, nil
 }
@@ -0,0 +1,66 @@
 package cli
 import (
 	"context"
 	"errors"
 	"os"
 	"path/filepath"
 	"strings"
 	"testing"
 	"git.avsdev.uk/Go/job-search/internal/config"
 	"git.avsdev.uk/Go/job-search/internal/store"
 )
 // No database.file means no store, reported as ErrNoDatabase.
 func TestOpenConfiguredStoreNone(t *testing.T) {
 	opportunityStore, err := OpenConfiguredStore(
 		context.Background(), &config.JobSearchConfig{},
 	)
 	if opportunityStore != nil || !errors.Is(err, ErrNoDatabase) {
 		t.Errorf(
 			"OpenConfiguredStore = %v, %v; want nil, ErrNoDatabase",
 			opportunityStore, err,
 		)
 	}
 }
 // A missing database isn't created; one that needs migrating says how to
 // migrate it; a migrated one opens.
 func TestOpenConfiguredStore(t *testing.T) {
 	conf := &config.JobSearchConfig{}
 	conf.Database.File = filepath.Join(t.TempDir(), "jobsearch.db")
 	_, err := OpenConfiguredStore(context.Background(), conf)
 	if err == nil || !strings.Contains(err.Error(), "migrate up --create") {
 		t.Fatalf("missing: err = %v; want a hint to create it", err)
 	}
 	_, statErr := os.Stat(conf.Database.File)
 	if !errors.Is(statErr, os.ErrNotExist) {
 		t.Fatal("the missing database file was created")
 	}
 	// An empty file is a database at version 0.
 	if err := os.WriteFile(conf.Database.File, nil, 0o600); err != nil {
 		t.Fatal(err)
 	}
 	_, err = OpenConfiguredStore(context.Background(), conf)
 	if !errors.Is(err, store.ErrNeedsMigration) ||
 		!strings.Contains(err.Error(), "run `job-search migrate`") {
 		t.Fatalf("unmigrated: err = %v; want ErrNeedsMigration and a hint", err)
 	}
 	migrated, err := store.Open(
 		context.Background(), conf.Database.File, store.WithAutoMigrate(),
 	)
 	if err != nil {
 		t.Fatal(err)
 	}
 	migrated.Close()
 	opportunityStore, err := OpenConfiguredStore(context.Background(), conf)
 	if err != nil {
 		t.Fatalf("migrated: err = %v", err)
 	}
 	opportunityStore.Close()
 }
@@ -1,43 +1,30 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 package version
 import (
 	"fmt"
 	"log/slog"
 	"git.avsdev.uk/Go/job-search/internal/version"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/version"
 )
 // NewCommand returns the "version" command.
 func NewCommand() *cobra.Command {
 	cmd := &cobra.Command{
 		Use:          "version",
 		Short:        "Print version information",
 		SilenceUsage: true,
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
-			slog.Debug("version.PersistentPreRun")
+			slog.Debug("command persistent pre-run", "command", "version")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
-			slog.Debug("version.PersistentPostRun")
+			slog.Debug("command persistent post-run", "command", "version")
 		},
 		Run: func(cmd *cobra.Command, args []string) {
-			slog.Debug("version.Run")
+			slog.Debug("command run", "command", "version")
-			fmt.Printf("PBCNinja - %s\n", version.Version)
+			fmt.Printf("JobSearch - %s\n", version.Version)
 			if version.BuildTime != "unknown" {
 				fmt.Printf("Built: %s\n", version.BuildTime)
 			}
@@ -45,10 +32,6 @@ func NewCommand() *cobra.Command {
 				fmt.Printf("Commit: %s\n", version.GitCommit)
 			}
 		},
 		Annotations: map[string]string{
 			"no_config":    "1",
 			"terminal_log": "1",
 		},
 	}
 	return cmd
 }
@@ -1,43 +0,0 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 package cli
 import (
 	"context"
 	"git.avsdev.uk/Go/job-search/internal/config"
 )
 // Define a private key type to avoid collisions
 type configContextKey struct {
 	name string
 }
 // Define the key
 var configKey = &configContextKey{"config"}
 // Public accessor to store a value to ctx
 func WithConfig(ctx context.Context, conf config.JobSearchConfig) context.Context {
 	// No allocation here since userIDKey is a pointer to a struct
 	return context.WithValue(ctx, configKey, conf)
 }
 // Public accessor to fetch a value from ctx
 func ConfigFromContext(ctx context.Context) (config.JobSearchConfig, bool) {
 	v, ok := ctx.Value(configKey).(config.JobSearchConfig)
 	return v, ok
 }
@@ -0,0 +1,17 @@
 // Package ws holds the ws-test commands: "serve" runs the plain TCP server
 // for websocket hub clients, and "dial" tests the HTTP server's /ws endpoint.
 package ws
 import (
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 )
 // NewCommands returns the ws-test commands, "serve" and "dial".
 func NewCommands(opts *cli.Options) []*cobra.Command {
 	return []*cobra.Command{
 		commandServe(opts),
 		commandDial(opts),
 	}
 }
@@ -0,0 +1,520 @@
 package ws
 import (
 	"bufio"
 	"context"
 	"encoding/json"
 	"errors"
 	"fmt"
 	"io"
 	"log/slog"
 	"net"
 	"net/http"
 	"os"
 	"os/signal"
 	"strconv"
 	"strings"
 	"syscall"
 	"time"
 	"github.com/coder/websocket"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/config"
 	"git.avsdev.uk/Go/job-search/internal/util"
 	hub "git.avsdev.uk/Go/job-search/internal/websocket"
 )
 // pingID is the ID of the ping --ping sends, which the pong it waits for
 // carries back.
 const pingID = "ping"
 // dialSettings are the dial command's flag values. direct chooses a plain
 // TCP connection to addr (--direct) over an HTTP upgrade at url
 // (--http-upgrade).
 type dialSettings struct {
 	direct   bool
 	url      string
 	addr     string
 	origin   string
 	messages []string
 	ping     bool
 	listen   bool
 	timeout  time.Duration
 }
 // connection is what dial talks to: a *websocket.Conn with --http-upgrade,
 // or a *directConn with --direct.
 type connection interface {
 	Read(ctx context.Context) (websocket.MessageType, []byte, error)
 	Write(
 		ctx context.Context,
 		messageType websocket.MessageType,
 		payload []byte,
 	) error
 	Close(code websocket.StatusCode, reason string) error
 	CloseNow() error
 }
 // commandDial returns the "dial" command, which connects to the WebSocket
 // hub of job-search serve (--http-upgrade) or ws-test serve (--direct),
 // sends messages and prints the replies.
 func commandDial(opts *cli.Options) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "dial",
 		Short: "Connect to a WebSocket hub, send messages and print the replies",
 		Long: `Connect to a server's WebSocket hub and print every message the server sends,
 starting with its welcome. One of these is required:
  --http-upgrade  upgrade an HTTP connection to a WebSocket on the /ws path,
                  as job-search serve expects
  --direct        connect straight to ws-test serve over plain TCP, with one
                  JSON message per line in each direction
 With --ping, it sends a ping and waits for the pong, failing if none arrives
 within --timeout.
 With --message, each one is sent in turn and the reply to it printed.
 With --ping or --message, dial then closes the connection, or with --listen
 prints every message the server pushes until the connection closes or Ctrl+C
 is pressed.
 With neither, each line read from stdin is sent as a message (blank lines are
 skipped), until Ctrl+C is pressed or the server closes the connection. The end
 of stdin stops the sending, but the connection stays open.
 With --http-upgrade, the URL defaults to the configured server.host and
 server.port (wss:// if server.tls.enabled is set). Set --origin to test the
 Origin check: an upgrade from an origin that isn't the server's own or one of
 server.csrf.cross_origin_protection.trusted_origins is refused with 403.
 With --direct, --addr defaults to the configured server.host and server.port.
 There is no HTTP request, so --url and --origin don't apply.`,
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent pre-run", "command", "dial")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent post-run", "command", "dial")
 		},
 		Args: cobra.NoArgs,
 		RunE: func(cmd *cobra.Command, args []string) error {
 			slog.Debug("command run", "command", "dial")
 			settings, err := dialFlags(cmd)
 			if err != nil {
 				return err
 			}
 			// Only one of url and addr can be set, for the chosen mode.
 			if settings.url == "" && settings.addr == "" {
 				conf, err := opts.RunSetup(cmd, true, true)
 				if err != nil {
 					return fmt.Errorf("load configuration: %w", err)
 				}
 				if settings.direct {
 					settings.addr = defaultAddr(conf.Server)
 				} else {
 					settings.url = defaultURL(conf.Server)
 				}
 			}
 			// ctx is cancelled on the first SIGINT (Ctrl+C) or SIGTERM.
 			ctx, stop := signal.NotifyContext(
 				context.Background(), os.Interrupt, syscall.SIGTERM,
 			)
 			defer stop()
 			return dial(ctx, cmd.InOrStdin(), cmd.OutOrStdout(), settings)
 		},
 	}
 	flags := cmd.Flags()
 	flags.Bool(
 		"http-upgrade",
 		false,
 		"upgrade an HTTP connection to a WebSocket on /ws (job-search serve)",
 	)
 	flags.Bool(
 		"direct",
 		false,
 		"connect straight over plain TCP, one JSON message per line "+
 			"(ws-test serve)",
 	)
 	flags.String(
 		"url",
 		"",
 		"with --http-upgrade, the WebSocket URL "+
 			"(default from config, ws://127.0.0.1:9393/ws)",
 	)
 	flags.String(
 		"addr",
 		"",
 		"with --direct, the address to connect to "+
 			"(default from config, 127.0.0.1:9393)",
 	)
 	flags.String(
 		"origin",
 		"",
 		"with --http-upgrade, the Origin header to send, as a browser would "+
 			"(default none)",
 	)
 	flags.StringArray(
 		"message",
 		nil,
 		"message to send, as JSON; repeat to send several (default: stdin)",
 	)
 	flags.Bool(
 		"ping",
 		false,
 		"send a ping and wait for the pong (before any --message)",
 	)
 	flags.Bool(
 		"listen",
 		false,
 		"with --ping or --message, keep printing pushed messages until the "+
 			"connection closes or Ctrl+C",
 	)
 	flags.Duration(
 		"timeout",
 		10*time.Second,
 		"how long to wait to connect and for each reply; 0s for no limit",
 	)
 	cmd.MarkFlagsOneRequired("http-upgrade", "direct")
 	cmd.MarkFlagsMutuallyExclusive("http-upgrade", "direct")
 	cmd.MarkFlagsMutuallyExclusive("direct", "url")
 	cmd.MarkFlagsMutuallyExclusive("direct", "origin")
 	cmd.MarkFlagsMutuallyExclusive("http-upgrade", "addr")
 	return cmd
 }
 // dialFlags reads the dial command's flags. The flags are all defined by
 // commandDial, so getting them can't fail.
 func dialFlags(cmd *cobra.Command) (dialSettings, error) {
 	flags := cmd.Flags()
 	direct, directErr := flags.GetBool("direct")
 	url, urlErr := flags.GetString("url")
 	addr, addrErr := flags.GetString("addr")
 	origin, originErr := flags.GetString("origin")
 	messages, messagesErr := flags.GetStringArray("message")
 	ping, pingErr := flags.GetBool("ping")
 	listen, listenErr := flags.GetBool("listen")
 	timeout, timeoutErr := flags.GetDuration("timeout")
 	err := errors.Join(
 		directErr,
 		urlErr,
 		addrErr,
 		originErr,
 		messagesErr,
 		pingErr,
 		listenErr,
 		timeoutErr,
 	)
 	if err != nil {
 		return dialSettings{}, err
 	}
 	return dialSettings{
 		direct:   direct,
 		url:      url,
 		addr:     addr,
 		origin:   origin,
 		messages: messages,
 		ping:     ping,
 		listen:   listen,
 		timeout:  timeout,
 	}, nil
 }
 // defaultURL is the /ws URL of the server conf configures: ws://, or wss://
 // with TLS, at defaultAddr.
 func defaultURL(conf config.JSCServer) string {
 	scheme := "ws"
 	if conf.TLS.Enabled {
 		scheme = "wss"
 	}
 	return scheme + "://" + defaultAddr(conf) + "/ws"
 }
 // defaultAddr is the host:port of the server conf configures. An empty host
 // (every interface) is dialled as 127.0.0.1.
 func defaultAddr(conf config.JSCServer) string {
 	host := conf.Host
 	if host == "" {
 		host = "127.0.0.1"
 	}
 	return net.JoinHostPort(host, strconv.Itoa(conf.Port))
 }
 // dial connects as settings choose (see connect) and prints the messages
 // received to out, each after "<". It sends each of settings.messages,
 // printed after ">", or without them each line of input (see interact),
 // which isn't printed again.
 func dial(
 	ctx context.Context,
 	input io.Reader,
 	out io.Writer,
 	settings dialSettings,
 ) error {
 	connectCtx, cancel := util.WithTimeout(ctx, settings.timeout)
 	defer cancel()
 	conn, err := connect(connectCtx, out, settings)
 	if err != nil {
 		return err
 	}
 	defer conn.CloseNow() //nolint:errcheck // finished with; it may already be closed
 	if !settings.ping && len(settings.messages) == 0 {
 		return interact(ctx, input, out, conn, settings.timeout)
 	}
 	// The server sends a welcome first.
 	err = receive(ctx, out, conn, settings.timeout)
 	if err != nil {
 		return err
 	}
 	if settings.ping {
 		err = pingPong(ctx, out, conn, settings.timeout)
 		if err != nil {
 			return err
 		}
 	}
 	for _, message := range settings.messages {
 		fmt.Fprintf(out, "> %s\n", message) //nolint:errcheck // printing only
 		err = send(ctx, conn, message, settings.timeout)
 		if err != nil {
 			return err
 		}
 		err = receive(ctx, out, conn, settings.timeout)
 		if err != nil {
 			return err
 		}
 	}
 	if settings.listen {
 		return listen(ctx, out, conn)
 	}
 	return conn.Close(websocket.StatusNormalClosure, "")
 }
 // connect opens a WebSocket at settings.url, or with settings.direct a
 // plain TCP connection to settings.addr, giving up when ctx is done. It
 // prints how it connected to out.
 func connect(
 	ctx context.Context,
 	out io.Writer,
 	settings dialSettings,
 ) (connection, error) {
 	if settings.direct {
 		conn, err := dialDirect(ctx, settings.addr)
 		if err != nil {
 			return nil, fmt.Errorf("connecting to %s: %w", settings.addr, err)
 		}
 		fmt.Fprintf(out, "connected directly to %s\n", settings.addr) //nolint:errcheck // printing only
 		return conn, nil
 	}
 	header := http.Header{}
 	if settings.origin != "" {
 		header.Set("Origin", settings.origin)
 	}
 	conn, res, err := websocket.Dial(
 		ctx,
 		settings.url,
 		&websocket.DialOptions{
 			HTTPHeader: header,
 		},
 	)
 	if err != nil {
 		if res != nil {
 			return nil, fmt.Errorf("upgrade refused: %s", res.Status)
 		}
 		return nil, fmt.Errorf("dialling %s: %w", settings.url, err)
 	}
 	// dial shows whatever the server sends, and an event can be large (one
 	// after a catch-up lists every new opportunity), so there's no limit.
 	conn.SetReadLimit(-1)
 	fmt.Fprintf(out, "connected (%s)\n", res.Status) //nolint:errcheck // printing only
 	return conn, nil
 }
 // pingPong sends a ping on conn and waits at most timeout for its pong,
 // printing each message received meanwhile to out.
 func pingPong(
 	ctx context.Context,
 	out io.Writer,
 	conn connection,
 	timeout time.Duration,
 ) error {
 	ping, err := json.Marshal(hub.Envelope{
 		Type: hub.TypePing,
 		ID:   pingID,
 	})
 	if err != nil {
 		return fmt.Errorf("encoding the ping: %w", err)
 	}
 	fmt.Fprintf(out, "> %s\n", ping) //nolint:errcheck // printing only
 	err = send(ctx, conn, string(ping), timeout)
 	if err != nil {
 		return err
 	}
 	waitCtx, cancel := util.WithTimeout(ctx, timeout)
 	defer cancel()
 	for {
 		_, data, readErr := conn.Read(waitCtx)
 		if readErr != nil {
 			if ctx.Err() == nil && waitCtx.Err() != nil {
 				return fmt.Errorf("no pong within %s", timeout)
 			}
 			return fmt.Errorf("waiting for the pong: %w", readErr)
 		}
 		fmt.Fprintf(out, "< %s\n", data) //nolint:errcheck // printing only
 		// Anything that isn't the pong, such as a push, is only printed.
 		var reply hub.Envelope
 		if json.Unmarshal(data, &reply) == nil &&
 			reply.Type == hub.TypePong && reply.ID == pingID {
 			return nil
 		}
 	}
 }
 // interact prints each message received on conn to out, and sends each line
 // of input as a message, skipping blank lines, until ctx is cancelled
 // (Ctrl+C) or the server closes the connection. The end of input stops the
 // sending; the connection stays open.
 func interact(
 	ctx context.Context,
 	input io.Reader,
 	out io.Writer,
 	conn connection,
 	timeout time.Duration,
 ) error {
 	listening := make(chan error, 1)
 	go func() {
 		listening <- listen(ctx, out, conn)
 	}()
 	// Reading input can't be interrupted, so it has a goroutine of its own,
 	// left blocked if ctx is cancelled first. Its error, if any, is sent
 	// before lines is closed.
 	lines := make(chan string)
 	inputErr := make(chan error, 1)
 	go func() {
 		defer close(lines)
 		scanner := bufio.NewScanner(input)
 		for scanner.Scan() {
 			lines <- scanner.Text()
 		}
 		inputErr <- scanner.Err()
 	}()
 	for {
 		select {
 		case <-ctx.Done():
 			return <-listening
 		case err := <-listening:
 			return err
 		case line, ok := <-lines:
 			if !ok {
 				err := <-inputErr
 				if err != nil {
 					return fmt.Errorf("reading input: %w", err)
 				}
 				// The end of input: stop sending, keep listening.
 				lines = nil
 				continue
 			}
 			if strings.TrimSpace(line) == "" {
 				continue
 			}
 			err := send(ctx, conn, line, timeout)
 			if err != nil {
 				return err
 			}
 		}
 	}
 }
 // send writes message to conn as a text frame, waiting at most timeout.
 func send(
 	ctx context.Context,
 	conn connection,
 	message string,
 	timeout time.Duration,
 ) error {
 	writeCtx, cancel := util.WithTimeout(ctx, timeout)
 	defer cancel()
 	err := conn.Write(writeCtx, websocket.MessageText, []byte(message))
 	if err != nil {
 		return fmt.Errorf("sending: %w", err)
 	}
 	return nil
 }
 // receive reads one message from conn, waiting at most timeout, and prints
 // it to out.
 func receive(
 	ctx context.Context,
 	out io.Writer,
 	conn connection,
 	timeout time.Duration,
 ) error {
 	readCtx, cancel := util.WithTimeout(ctx, timeout)
 	defer cancel()
 	_, data, err := conn.Read(readCtx)
 	if err != nil {
 		return fmt.Errorf("waiting for a reply: %w", err)
 	}
 	fmt.Fprintf(out, "< %s\n", data) //nolint:errcheck // printing only
 	return nil
 }
 // listen prints each message received on conn to out until ctx is cancelled
 // (Ctrl+C) or the server closes the connection, either of which is a normal
 // end.
 func listen(ctx context.Context, out io.Writer, conn connection) error {
 	for {
 		_, data, err := conn.Read(ctx)
 		status := websocket.CloseStatus(err)
 		switch {
 		case err == nil:
 			fmt.Fprintf(out, "< %s\n", data) //nolint:errcheck // printing only
 		case ctx.Err() != nil:
 			// Ctrl+C: dial closes the connection.
 			return nil //nolint:nilerr // a normal end
 		case errors.Is(err, io.EOF):
 			// A direct connection has no close status.
 			fmt.Fprintln(out, "closed by the server") //nolint:errcheck // printing only
 			return nil
 		case status != -1:
 			fmt.Fprintf(out, "closed by the server (%d)\n", status) //nolint:errcheck // printing only
 			return nil
 		default:
 			return fmt.Errorf("reading: %w", err)
 		}
 	}
 }
@@ -0,0 +1,412 @@
 package ws
 import (
 	"bytes"
 	"context"
 	"errors"
 	"io"
 	"log/slog"
 	"net/http"
 	"net/http/httptest"
 	"strings"
 	"sync"
 	"testing"
 	"time"
 	coder "github.com/coder/websocket"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/cli/cmdtest"
 	"git.avsdev.uk/Go/job-search/internal/config"
 	"git.avsdev.uk/Go/job-search/internal/websocket"
 )
 func newRoot() *cobra.Command {
 	opts := &cli.Options{}
 	return cmdtest.NewRoot(opts, NewCommands(opts)...)
 }
 // wsServer serves a hub that trusts https://admin.example.com on a test HTTP
 // server, and returns the hub and its ws:// URL.
 func wsServer(t *testing.T) (*websocket.Hub, string) {
 	t.Helper()
 	hub := websocket.NewHub(
 		config.JSCServerWebSocket{
 			WriteTimeout: time.Second,
 		},
 		websocket.WithLogger(slog.New(slog.DiscardHandler)),
 		websocket.WithTrustedOrigins([]string{"https://admin.example.com"}),
 	)
 	server := httptest.NewServer(hub)
 	t.Cleanup(func() {
 		ctx, cancel := context.WithTimeout(context.Background(), time.Second)
 		defer cancel()
 		hub.Shutdown(ctx)
 		server.Close()
 	})
 	return hub, "ws" + strings.TrimPrefix(server.URL, "http")
 }
 // ping is a message the server answers with a pong.
 const ping = `{"type":"ping","id":"1"}`
 // With --message, dial prints the welcome, sends the message and prints the
 // reply.
 func TestDialPing(t *testing.T) {
 	cmdtest.Setup(t, "")
 	_, url := wsServer(t)
 	out := cmdtest.MustRun(
 		t, newRoot, "dial", "--http-upgrade", "--url", url, "--message", ping,
 	)
 	cmdtest.Contains(
 		t,
 		out,
 		"connected (101 Switching Protocols)",
 		`< {"type":"welcome"`,
 		`> {"type":"ping","id":"1"}`,
 		`< {"type":"pong","id":"1"`,
 	)
 }
 // --ping sends a ping and waits for its pong, then closes the connection.
 func TestDialPingFlag(t *testing.T) {
 	cmdtest.Setup(t, "")
 	_, url := wsServer(t)
 	out := cmdtest.MustRun(
 		t, newRoot, "dial", "--http-upgrade", "--url", url, "--ping",
 	)
 	cmdtest.Contains(
 		t,
 		out,
 		`> {"type":"ping","id":"ping"}`,
 		`< {"type":"pong","id":"ping"`,
 	)
 }
 // --ping fails if no pong arrives within --timeout.
 func TestDialPingTimeout(t *testing.T) {
 	cmdtest.Setup(t, "")
 	// A server that sends a welcome, then never replies.
 	server := httptest.NewServer(http.HandlerFunc(
 		func(w http.ResponseWriter, r *http.Request) {
 			conn, err := coder.Accept(w, r, nil)
 			if err != nil {
 				return
 			}
 			defer conn.CloseNow()
 			conn.Write(r.Context(), coder.MessageText, []byte(`{"type":"welcome"}`))
 			for {
 				if _, _, err := conn.Read(r.Context()); err != nil {
 					return
 				}
 			}
 		},
 	))
 	t.Cleanup(server.Close)
 	url := "ws" + strings.TrimPrefix(server.URL, "http")
 	_, err := cmdtest.Run(
 		t,
 		newRoot,
 		"",
 		"dial", "--http-upgrade", "--url", url, "--ping", "--timeout", "100ms",
 	)
 	cmdtest.ErrContains(t, err, "no pong within 100ms")
 }
 // Each --message is sent in turn, and the reply to each printed.
 func TestDialMessages(t *testing.T) {
 	cmdtest.Setup(t, "")
 	_, url := wsServer(t)
 	out := cmdtest.MustRun(
 		t,
 		newRoot,
 		"dial", "--http-upgrade", "--url", url,
 		"--message", `{"type":"echo","id":"7","payload":{"a":1}}`,
 		"--message", `{"type":"nonsense","id":"8"}`,
 	)
 	cmdtest.Contains(
 		t,
 		out,
 		`< {"type":"echo","id":"7","payload":{"a":1}}`,
 		`< {"type":"error","id":"8"`,
 	)
 }
 // An upgrade from a foreign origin is refused; a trusted one is accepted.
 func TestDialOrigin(t *testing.T) {
 	cmdtest.Setup(t, "")
 	_, url := wsServer(t)
 	_, err := cmdtest.Run(
 		t,
 		newRoot,
 		"",
 		"dial", "--http-upgrade", "--url", url, "--message", ping,
 		"--origin", "https://evil.example",
 	)
 	cmdtest.ErrContains(t, err, "upgrade refused: 403 Forbidden")
 	out := cmdtest.MustRun(
 		t,
 		newRoot,
 		"dial", "--http-upgrade", "--url", url, "--message", ping,
 		"--origin", "https://admin.example.com",
 	)
 	cmdtest.Contains(t, out, "connected (101 Switching Protocols)")
 }
 // With --listen, pushed messages are printed until the server closes the
 // connection.
 func TestDialListen(t *testing.T) {
 	hub, url := wsServer(t)
 	out := &syncBuffer{}
 	done := make(chan error, 1)
 	go func() {
 		done <- dial(context.Background(), nil, out, dialSettings{
 			url:      url,
 			messages: []string{ping},
 			listen:   true,
 			timeout:  time.Second,
 		})
 	}()
 	// Push only once the client has its pong, and shut down only once it has
 	// the push: Shutdown closes clients without waiting for their queues.
 	waitFor(t, out, `< {"type":"pong","id":"1"`)
 	hub.Push("event", map[string]any{"n": 1})
 	waitFor(t, out, `< {"type":"event","payload":{"n":1}}`)
 	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
 	defer cancel()
 	hub.Shutdown(ctx)
 	select {
 	case err := <-done:
 		if err != nil {
 			t.Fatalf("dial = %v; want nil when the server closes", err)
 		}
 	case <-time.After(time.Second):
 		t.Fatal("dial did not return when the server closed")
 	}
 	waitFor(t, out, "closed by the server (1001)")
 }
 // A pushed message larger than the WebSocket library's default read limit
 // (32KiB), as a catch-up event can be, is printed in full.
 func TestDialLargeMessage(t *testing.T) {
 	hub, url := wsServer(t)
 	out := &syncBuffer{}
 	done := make(chan error, 1)
 	go func() {
 		done <- dial(context.Background(), nil, out, dialSettings{
 			url:      url,
 			messages: []string{ping},
 			listen:   true,
 			timeout:  time.Second,
 		})
 	}()
 	waitFor(t, out, `< {"type":"pong","id":"1"`)
 	large := strings.Repeat("x", 100_000)
 	hub.Push("event", map[string]string{"large": large, "end": "here"})
 	// encoding/json sorts a map's keys.
 	waitFor(t, out, `{"end":"here","large":"`+large+`"}}`)
 	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
 	defer cancel()
 	hub.Shutdown(ctx)
 	select {
 	case err := <-done:
 		if err != nil {
 			t.Fatalf("dial = %v; want nil when the server closes", err)
 		}
 	case <-time.After(time.Second):
 		t.Fatal("dial did not return when the server closed")
 	}
 }
 // syncBuffer is a bytes.Buffer safe to write from one goroutine while
 // another reads it.
 type syncBuffer struct {
 	mu     sync.Mutex
 	buffer bytes.Buffer
 }
 func (b *syncBuffer) Write(data []byte) (int, error) {
 	b.mu.Lock()
 	defer b.mu.Unlock()
 	return b.buffer.Write(data)
 }
 func (b *syncBuffer) String() string {
 	b.mu.Lock()
 	defer b.mu.Unlock()
 	return b.buffer.String()
 }
 // waitFor fails the test unless out comes to contain every string in want
 // within a second.
 func waitFor(t *testing.T, out *syncBuffer, want ...string) {
 	t.Helper()
 	deadline := time.Now().Add(time.Second)
 	for {
 		missing := ""
 		for _, wanted := range want {
 			if !strings.Contains(out.String(), wanted) {
 				missing = wanted
 				break
 			}
 		}
 		if missing == "" {
 			return
 		}
 		if time.Now().After(deadline) {
 			t.Fatalf("output does not contain %q:\n%s", missing, out)
 		}
 		time.Sleep(5 * time.Millisecond)
 	}
 }
 // Without --message, each line of stdin is sent, skipping blank ones. The end
 // of stdin doesn't end dial: pushes are still printed until Ctrl+C.
 func TestDialStdin(t *testing.T) {
 	hub, url := wsServer(t)
 	stdin, input := io.Pipe()
 	out := &syncBuffer{}
 	ctx, cancel := context.WithCancel(context.Background())
 	defer cancel()
 	done := make(chan error, 1)
 	go func() {
 		done <- dial(ctx, stdin, out, dialSettings{
 			url:     url,
 			timeout: time.Second,
 		})
 	}()
 	io.WriteString(input, ping+"\n\n"+`{"type":"echo","id":"2"}`+"\n")
 	input.Close()
 	waitFor(
 		t,
 		out,
 		`< {"type":"welcome"`,
 		`< {"type":"pong","id":"1"`,
 		`< {"type":"echo","id":"2"}`,
 	)
 	if strings.Contains(out.String(), `"type":"error"`) {
 		t.Errorf("a blank line was sent:\n%s", out)
 	}
 	hub.Push("event", map[string]any{"n": 1})
 	waitFor(t, out, `< {"type":"event","payload":{"n":1}}`)
 	cancel() // Ctrl+C
 	select {
 	case err := <-done:
 		if err != nil {
 			t.Fatalf("dial = %v; want nil after Ctrl+C", err)
 		}
 	case <-time.After(time.Second):
 		t.Fatal("dial did not return after Ctrl+C")
 	}
 }
 // Without --message, a failure reading stdin ends dial with an error, unlike
 // the end of stdin.
 func TestDialStdinError(t *testing.T) {
 	_, url := wsServer(t)
 	stdin, input := io.Pipe()
 	out := &syncBuffer{}
 	done := make(chan error, 1)
 	go func() {
 		done <- dial(context.Background(), stdin, out, dialSettings{
 			url:     url,
 			timeout: time.Second,
 		})
 	}()
 	waitFor(t, out, `< {"type":"welcome"`)
 	input.CloseWithError(errors.New("disk on fire"))
 	select {
 	case err := <-done:
 		if err == nil ||
 			!strings.Contains(err.Error(), "reading input: disk on fire") {
 			t.Fatalf("dial = %v; want the input error", err)
 		}
 	case <-time.After(time.Second):
 		t.Fatal("dial did not return after the input error")
 	}
 }
 // Without --message, dial ends when the server closes the connection.
 func TestDialStdinServerClose(t *testing.T) {
 	hub, url := wsServer(t)
 	stdin, _ := io.Pipe() // never written: stdin stays open
 	out := &syncBuffer{}
 	done := make(chan error, 1)
 	go func() {
 		done <- dial(context.Background(), stdin, out, dialSettings{
 			url:     url,
 			timeout: time.Second,
 		})
 	}()
 	waitFor(t, out, `< {"type":"welcome"`)
 	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
 	defer cancel()
 	hub.Shutdown(ctx)
 	select {
 	case err := <-done:
 		if err != nil {
 			t.Fatalf("dial = %v; want nil when the server closes", err)
 		}
 	case <-time.After(time.Second):
 		t.Fatal("dial did not return when the server closed")
 	}
 	waitFor(t, out, "closed by the server (1001)")
 }
 // Without --url, dial uses the configured host and port.
 func TestDefaultURL(t *testing.T) {
 	tests := []struct {
 		name string
 		conf config.JSCServer
 		want string
 	}{
 		{"configured", config.JSCServer{
 			Host: "10.0.0.1",
 			Port: 8080,
 		}, "ws://10.0.0.1:8080/ws"},
 		{"every interface", config.JSCServer{
 			Port: 9393,
 		}, "ws://127.0.0.1:9393/ws"},
 		{"IPv6", config.JSCServer{
 			Host: "::1",
 			Port: 9393,
 		}, "ws://[::1]:9393/ws"},
 		{"TLS", config.JSCServer{
 			Host: "h",
 			Port: 443,
 			TLS: config.JSCServerTLS{
 				Enabled: true,
 			},
 		}, "wss://h:443/ws"},
 	}
 	for _, tt := range tests {
 		if got := defaultURL(tt.conf); got != tt.want {
 			t.Errorf("%s: defaultURL = %q; want %q", tt.name, got, tt.want)
 		}
 	}
 }
@@ -0,0 +1,117 @@
 package ws
 import (
 	"bufio"
 	"context"
 	"errors"
 	"io"
 	"net"
 	"strings"
 	"time"
 	"github.com/coder/websocket"
 )
 // maxDirectLineBytes is the longest line a directConn reads.
 const maxDirectLineBytes = 1 << 20
 // errBinaryMessage is returned for a binary message, which a directConn
 // can't send.
 var errBinaryMessage = errors.New("only text messages can be sent directly")
 // directConn is a plain TCP connection to ws-test serve, without an HTTP
 // upgrade: each message is one line of JSON in both directions (see
 // websocket.Hub.ServeTCP). It has the methods dial uses of a
 // *websocket.Conn.
 type directConn struct {
 	conn    net.Conn
 	scanner *bufio.Scanner
 }
 // dialDirect connects to addr, giving up when ctx is done.
 func dialDirect(ctx context.Context, addr string) (*directConn, error) {
 	var dialer net.Dialer
 	conn, err := dialer.DialContext(ctx, "tcp", addr)
 	if err != nil {
 		return nil, err
 	}
 	scanner := bufio.NewScanner(conn)
 	scanner.Buffer(nil, maxDirectLineBytes)
 	return &directConn{
 		conn:    conn,
 		scanner: scanner,
 	}, nil
 }
 // Read returns the next line as a text message, skipping the blank lines the
 // server sends to check the connection. Cancelling ctx interrupts it. The
 // server closing the connection is reported as io.EOF.
 func (d *directConn) Read(
 	ctx context.Context,
 ) (websocket.MessageType, []byte, error) {
 	stop := context.AfterFunc(ctx, func() {
 		d.conn.SetReadDeadline(time.Now()) //nolint:errcheck,gosec // only interrupts the read
 	})
 	defer stop()
 	for d.scanner.Scan() {
 		line := strings.TrimSuffix(d.scanner.Text(), "\r")
 		if strings.TrimSpace(line) == "" {
 			continue
 		}
 		return websocket.MessageText, []byte(line), nil
 	}
 	err := d.scanner.Err()
 	switch {
 	case ctx.Err() != nil:
 		return 0, nil, ctx.Err()
 	case err == nil:
 		return 0, nil, io.EOF
 	}
 	return 0, nil, err
 }
 // Write sends payload as one line. Cancelling ctx interrupts it.
 func (d *directConn) Write(
 	ctx context.Context,
 	messageType websocket.MessageType,
 	payload []byte,
 ) error {
 	if messageType != websocket.MessageText {
 		return errBinaryMessage
 	}
 	stop := context.AfterFunc(ctx, func() {
 		d.conn.SetWriteDeadline(time.Now()) //nolint:errcheck,gosec // only interrupts the write
 	})
 	defer stop()
 	line := make([]byte, 0, len(payload)+1)
 	line = append(append(line, payload...), '\n')
 	_, err := d.conn.Write(line)
 	if err != nil && ctx.Err() != nil {
 		return ctx.Err()
 	}
 	return err
 }
 // Close closes the connection. Plain TCP has no close handshake, so code and
 // reason aren't sent.
 func (d *directConn) Close(code websocket.StatusCode, reason string) error {
 	return d.CloseNow()
 }
 // CloseNow closes the connection. Closing it again isn't an error.
 func (d *directConn) CloseNow() error {
 	err := d.conn.Close()
 	if errors.Is(err, net.ErrClosed) {
 		return nil
 	}
 	return err
 }
@@ -0,0 +1,239 @@
 package ws
 import (
 	"context"
 	"errors"
 	"io"
 	"log/slog"
 	"net"
 	"testing"
 	"time"
 	coder "github.com/coder/websocket"
 	"git.avsdev.uk/Go/job-search/internal/cli/cmdtest"
 	"git.avsdev.uk/Go/job-search/internal/config"
 	"git.avsdev.uk/Go/job-search/internal/websocket"
 )
 // directServer serves a hub's plain TCP clients on a local port, as ws-test
 // serve does, and returns the hub and its address.
 func directServer(t *testing.T) (*websocket.Hub, string) {
 	t.Helper()
 	hub := websocket.NewHub(
 		config.JSCServerWebSocket{
 			WriteTimeout: time.Second,
 		},
 		websocket.WithLogger(slog.New(slog.DiscardHandler)),
 	)
 	listener, err := net.Listen("tcp", "127.0.0.1:0")
 	if err != nil {
 		t.Fatal(err)
 	}
 	ctx, cancel := context.WithCancel(context.Background())
 	go func() {
 		for {
 			conn, err := listener.Accept()
 			if err != nil {
 				return
 			}
 			go hub.ServeTCP(ctx, conn)
 		}
 	}()
 	t.Cleanup(func() {
 		listener.Close()
 		shutdownCtx, cancelShutdown := context.WithTimeout(
 			context.Background(), time.Second,
 		)
 		defer cancelShutdown()
 		hub.Shutdown(shutdownCtx)
 		cancel()
 	})
 	return hub, listener.Addr().String()
 }
 // With --direct, --ping and --message work over plain TCP as they do over an
 // HTTP upgrade.
 func TestDialDirect(t *testing.T) {
 	cmdtest.Setup(t, "")
 	_, addr := directServer(t)
 	out := cmdtest.MustRun(
 		t,
 		newRoot,
 		"dial", "--direct", "--addr", addr, "--ping",
 		"--message", `{"type":"echo","id":"7","payload":{"a":1}}`,
 	)
 	cmdtest.Contains(
 		t,
 		out,
 		"connected directly to "+addr,
 		`< {"type":"welcome"`,
 		`> {"type":"ping","id":"ping"}`,
 		`< {"type":"pong","id":"ping"`,
 		`< {"type":"echo","id":"7","payload":{"a":1}}`,
 	)
 }
 // With --direct and --listen, pushed messages are printed until the server
 // closes the connection, which has no close status.
 func TestDialDirectListen(t *testing.T) {
 	hub, addr := directServer(t)
 	out := &syncBuffer{}
 	done := make(chan error, 1)
 	go func() {
 		done <- dial(context.Background(), nil, out, dialSettings{
 			direct:   true,
 			addr:     addr,
 			messages: []string{ping},
 			listen:   true,
 			timeout:  time.Second,
 		})
 	}()
 	waitFor(t, out, `< {"type":"pong","id":"1"`)
 	hub.Push("event", map[string]any{"n": 1})
 	waitFor(t, out, `< {"type":"event","payload":{"n":1}}`)
 	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
 	defer cancel()
 	hub.Shutdown(ctx)
 	select {
 	case err := <-done:
 		if err != nil {
 			t.Fatalf("dial = %v; want nil when the server closes", err)
 		}
 	case <-time.After(time.Second):
 		t.Fatal("dial did not return when the server closed")
 	}
 	waitFor(t, out, `"error":"server shutting down"`, "closed by the server\n")
 }
 // With --direct and no messages, each line of stdin is sent.
 func TestDialDirectStdin(t *testing.T) {
 	_, addr := directServer(t)
 	stdin, input := io.Pipe()
 	out := &syncBuffer{}
 	ctx, cancel := context.WithCancel(context.Background())
 	defer cancel()
 	done := make(chan error, 1)
 	go func() {
 		done <- dial(ctx, stdin, out, dialSettings{
 			direct:  true,
 			addr:    addr,
 			timeout: time.Second,
 		})
 	}()
 	io.WriteString(input, ping+"\n")
 	waitFor(t, out, `< {"type":"welcome"`, `< {"type":"pong","id":"1"`)
 	cancel() // Ctrl+C
 	select {
 	case err := <-done:
 		if err != nil {
 			t.Fatalf("dial = %v; want nil after Ctrl+C", err)
 		}
 	case <-time.After(time.Second):
 		t.Fatal("dial did not return after Ctrl+C")
 	}
 }
 // --direct fails clearly when nothing is listening.
 func TestDialDirectRefused(t *testing.T) {
 	listener, err := net.Listen("tcp", "127.0.0.1:0")
 	if err != nil {
 		t.Fatal(err)
 	}
 	addr := listener.Addr().String()
 	listener.Close()
 	err = dial(context.Background(), nil, io.Discard, dialSettings{
 		direct:  true,
 		addr:    addr,
 		timeout: time.Second,
 	})
 	if err == nil {
 		t.Fatal("dial succeeded with nothing listening")
 	}
 	cmdtest.ErrContains(t, err, "connecting to "+addr)
 }
 // One of --http-upgrade and --direct is required, and each refuses the other's
 // flags.
 func TestDialModeFlags(t *testing.T) {
 	cmdtest.Setup(t, "")
 	tests := []struct {
 		name string
 		args []string
 		want string
 	}{
 		{"neither", []string{"--ping"}, "at least one of the flags"},
 		{"both", []string{"--http-upgrade", "--direct"}, "none of the others"},
 		{"direct with url", []string{"--direct", "--url", "ws://h/ws"}, "none of the others"},
 		{"direct with origin", []string{"--direct", "--origin", "https://o"}, "none of the others"},
 		{"upgrade with addr", []string{"--http-upgrade", "--addr", "h:1"}, "none of the others"},
 	}
 	for _, tt := range tests {
 		args := append([]string{"dial"}, tt.args...)
 		_, err := cmdtest.Run(t, newRoot, "", args...)
 		if err == nil {
 			t.Errorf("%s: dial was accepted", tt.name)
 			continue
 		}
 		cmdtest.ErrContains(t, err, tt.want)
 	}
 }
 // A directConn sends only text messages, and a closed one closes again
 // without an error.
 func TestDirectConn(t *testing.T) {
 	_, addr := directServer(t)
 	conn, err := dialDirect(context.Background(), addr)
 	if err != nil {
 		t.Fatal(err)
 	}
 	err = conn.Write(context.Background(), coder.MessageBinary, []byte("x"))
 	if !errors.Is(err, errBinaryMessage) {
 		t.Errorf("binary Write = %v; want errBinaryMessage", err)
 	}
 	if err := conn.CloseNow(); err != nil {
 		t.Errorf("CloseNow = %v", err)
 	}
 	if err := conn.CloseNow(); err != nil {
 		t.Errorf("second CloseNow = %v; want nil", err)
 	}
 }
 // Without --addr, --direct uses the configured host and port.
 func TestDefaultAddr(t *testing.T) {
 	tests := []struct {
 		name string
 		conf config.JSCServer
 		want string
 	}{
 		{"configured", config.JSCServer{
 			Host: "10.0.0.1",
 			Port: 8080,
 		}, "10.0.0.1:8080"},
 		{"every interface", config.JSCServer{
 			Port: 9393,
 		}, "127.0.0.1:9393"},
 		{"IPv6", config.JSCServer{
 			Host: "::1",
 			Port: 9393,
 		}, "[::1]:9393"},
 	}
 	for _, tt := range tests {
 		if got := defaultAddr(tt.conf); got != tt.want {
 			t.Errorf("%s: defaultAddr = %q; want %q", tt.name, got, tt.want)
 		}
 	}
 }
@@ -0,0 +1,101 @@
 package ws
 import (
 	"context"
 	"errors"
 	"fmt"
 	"log/slog"
 	"os"
 	"os/signal"
 	"strconv"
 	"strings"
 	"syscall"
 	"github.com/spf13/cobra"
 	"git.avsdev.uk/Go/job-search/internal/cli"
 	"git.avsdev.uk/Go/job-search/internal/websocket"
 	"git.avsdev.uk/Go/job-search/internal/websocket/tcpserver"
 )
 // commandServe returns the "serve" command, which runs a plain TCP server
 // for websocket hub clients (see tcpserver).
 func commandServe(opts *cli.Options) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "serve",
 		Short: "Run JobSearch TCP Websocket server",
 		PersistentPreRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent pre-run", "command", "serve")
 		},
 		PersistentPostRun: func(cmd *cobra.Command, args []string) {
 			slog.Debug("command persistent post-run", "command", "serve")
 		},
 		RunE: func(cmd *cobra.Command, args []string) error {
 			slog.Debug("command run", "command", "serve")
 			addrOverride, _ := cmd.Flags().GetString("listen")
 			hostOverride, _ := cmd.Flags().GetString("host")
 			portOverride, _ := cmd.Flags().GetInt("port")
 			foreground, _ := cmd.Flags().GetBool("foreground")
 			if addrOverride != "" {
 				listenParts := strings.Split(addrOverride, ":")
 				if len(listenParts) != 2 {
 					return errors.New(
 						"--listen should be in the format of 127.0.0.1:9393",
 					)
 				}
 				port, err := strconv.Atoi(listenParts[1])
 				if err != nil {
 					return fmt.Errorf("failed to parse --listen port: %w", err)
 				}
 				hostOverride = listenParts[0]
 				portOverride = port
 			}
 			conf, err := opts.RunSetup(cmd, true, foreground)
 			if err != nil {
 				return fmt.Errorf("load configuration: %w", err)
 			}
 			if portOverride != 0 {
 				conf.Server.Port = portOverride
 			}
 			if hostOverride != "" {
 				conf.Server.Host = hostOverride
 			}
 			// ctx is cancelled on the first SIGINT (Ctrl+C) or SIGTERM.
 			ctx, stop := signal.NotifyContext(
 				context.Background(), os.Interrupt, syscall.SIGTERM,
 			)
 			defer stop()
 			// After the first signal, restore default signal handling so a second
 			// Ctrl+C force-kills the process if graceful shutdown hangs.
 			go func() {
 				<-ctx.Done()
 				stop()
 			}()
 			hub := websocket.NewHub(conf.Server.WebSocket)
 			server := tcpserver.New(&conf.Server, hub)
 			return server.Run(ctx)
 		},
 	}
 	flags := cmd.Flags()
 	flags.String(
 		"listen", "", "listen address (default from config, 127.0.0.1:9393)",
 	)
 	flags.String(
 		"host", "", "listen address (default from config, 127.0.0.1)",
 	)
 	flags.Int("port", 0, "listen address (default from config, 9393)")
 	flags.Bool("foreground", false, "run in the foreground")
 	cmd.MarkFlagsMutuallyExclusive("listen", "host")
 	cmd.MarkFlagsMutuallyExclusive("listen", "port")
 	return cmd
 }
@@ -0,0 +1,377 @@
 # config
 Package config loads the JobSearch configuration into a `JobSearchConfig`
 struct. Values come from struct-tag defaults, a YAML or JSON config file and
 environment variables, merged through [Viper](https://github.com/spf13/viper).
 ## Loading order
 `Init(cfgFile, &conf)` builds the configuration in this order:
 1. Loads `.env` from the working directory into the environment, if there
   is one. A `.env` file that can't be read or parsed is an error.
 2. Sets the defaults from the `default` struct tags as Viper defaults.
 3. Reads the config file: `cfgFile` if given, otherwise the first
   `config.yaml` found in `/etc/job-search` then the working directory. If no
   file is found a warning is logged and the defaults are used. A file that
   exists but can't be parsed is an error.
 4. Reads environment variables prefixed with `JOBSEARCH_`. Dots in a key
   become underscores, so `logging.level` is read from
   `JOBSEARCH_LOGGING_LEVEL`.
 5. Copies every known key from Viper into `conf`. A value that fails
   validation (e.g. an unknown log level) is logged as a warning and the
   previous value is kept.
 When more than one place sets a key, environment variables win over the
 config file, which wins over the defaults.
 ## Configuration Keys
 Configuration is managed in sections:
 | Key | Section description |
 |---|---|
 | `logging` | Paths to log files and log level control |
 | `database` | Database connection/file details |
 | `sources` | Opportunity source settings |
 | `server` | Server configuration including CSRF protection, Websockets, Proxy handling and rate limits |
 ### Logging
 | Key | Type | Default | Notes |
 |---|---|---|---|
 | `logging.level` | string | `warn` | One of `debug`, `info`, `warn`, `error` |
 | `logging.file` | string | | Log file (JSON lines) for long-running commands such as `fetcher service`. Empty: log to the terminal |
 Logging always starts on the terminal. Long-running commands switch to the file
 in `logging.file` once the configuration is loaded, unless it is empty (they
 then stay on the terminal, with a warning) or `--foreground` is passed.
 All other commands log to the terminal.
 The `--debug` and `--quiet` flags force the log level to `debug` and `error`
 respectively, ignoring the `logging.level` configuration. `--debug` takes
 priority over `--quiet`.
 ### Database
 | Key | Type | Default | Notes |
 |---|---|---|---|
 | `database.file` | string | | Path to the SQLite database file. Empty: commands that need the database fail |
 | `database.change_batch_retention` | duration | `720h` | How long a batch of changed opportunities is kept for clients to catch up from (`?changed_after=`); older ones are deleted as new ones are recorded. A client whose batch has gone reloads the whole list. 0 or less: kept for ever |
 Database is currently restricted to SQLite v3. At this point, no other engine is
 supported.
 ### Opportunity Sources
 | Key | Type | Default | Notes |
 |---|---|---|---|
 | `sources.skip_rules_file` | string | | JSON file of rules that mark new opportunities Skipped (see `internal/skiprules`); read again when it changes. Empty: none. Missing or invalid: `fetcher service` and `serve` refuse to start |
 | `sources.<source>.enabled` | bool | `false` | For every source |
 | `sources.<source>.interval` | duration | `10m` | For every source; e.g. `30s`, `1h`. Must be greater than 0; turn a source off with `enabled` |
 | `sources.find_a_tender.api_root` | string | `https://www.find-tender.service.gov.uk/api/1.0` | URL of the Find a Tender API. Empty: the source fails to start if enabled, and `fetcher fts-once` fails |
 | `sources.find_a_tender.rate_limit_retries` | int | `5` | How many times a request refused for the API's rate limit is waited out (for its `Retry-After`) and retried. 0 fails at the first; can't be negative. `fetcher fts-once --ignore-rate-limit` uses 0 |
 | `sources.fts_mock.api_root` | string | | URL of the mock FTS server (see `internal/mock`). Empty: the source fails to start if enabled |
 | `sources.fts_mock.rate_limit_retries` | int | `5` | As for `find_a_tender` |
 | `sources.eu_supply.base_url` | string | `https://uk.eu-supply.com` | Root of EU Supply's UK portal (see `internal/eusupply`) |
 | `sources.eu_supply.request_min_gap` | duration | `2s` | The shortest wait between two requests to the portal. Must be greater than 0: the site publishes no rate limit |
 | `sources.eu_supply.request_max_gap` | duration | `10s` | The longest wait between two requests; each wait is random, from `request_min_gap`. Must be greater than 0; less than `request_min_gap` is read as `request_min_gap` |
 | `sources.sleeper.block_shutdown` | bool | `false` | Test source: if true, its sleep ignores shutdown |
 The current sources are `find_a_tender`, `fts_mock`, `eu_supply` and
 `sleeper`. See
 "Addition of a new opportunities source" in the top level README for how to add
 a new source.
 ### Server
 The `server` section is split into a number of sub-sections:
 | Key | Section description |
 |---|---|
 | `server` | Top level server configuration settings |
 | `server.timeouts` | HTTP read, write and idle timeouts, and the shutdown timeout |
 | `server.rate_limits` | Global (every request) and API rate limits. Also see `server.websocket.connection_limit` for WebSocket and plain TCP connection limits |
 | `server.proxy` | PROXY protocol and proxy header settings |
 | `server.csrf` | CSRF and cross-origin protection settings |
 | `server.tls` | TLS certificate, minimum version and HTTP redirect |
 | `server.headers` | HTTP header settings including policy headers |
 | `server.frontend` | Where the frontend's build output is and how it is cached |
 | `server.websocket` | Websocket configuration |
 #### Server
 | Key | Type | Default | Notes |
 |---|---|---|---|
 | `server.host` | string | `127.0.0.1` | Address to listen on; empty: every interface |
 | `server.port` | int | `9393` | Port to listen on; `0`: any free port |
 | `server.max_header_bytes` | byte size | `8KB` | Largest request header block; `0B` for 8KB. `-1` (no limit) is rejected, as the HTTP server always has one |
 | `server.max_body_bytes` | byte size | `1MB` | Largest request body; `0B` for 1MB, `-1` for no limit |
 | `server.access_log` | string | | File the access log is appended to, one line per request in the Combined Log Format; `-`: stdout; empty: no access log. `serve --access-log` overrides it |
 #### Timeouts
 | Key | Type | Default | Notes |
 |---|---|---|---|
 | `server.timeouts.read_header` | duration | `5s` | Reading a request's headers. Must be greater than 0, as a client could otherwise hold a connection open for ever |
 | `server.timeouts.read` | duration | `15s` | Reading a whole request; `0s` for none |
 | `server.timeouts.write` | duration | `30s` | Writing a response; `0s` for none |
 | `server.timeouts.idle` | duration | `2m` | Waiting for the next request on a keep-alive connection; `0s` for none |
 | `server.timeouts.shutdown` | duration | `15s` | How long shutdown waits for open requests; `0s`: until they finish |
 #### Rate Limits
 | Key | Type | Default | Notes |
 |---|---|---|---|
 | `server.rate_limits.<rule>.enabled` | bool | `false` | `<rule>` is `global` or `api` (see rate limit rules below) |
 | `server.rate_limits.<rule>.per_second` | int | `50` | Sustained requests per second per client IP. `0` or less: the rule is off |
 | `server.rate_limits.<rule>.burst` | int | `100` | Requests allowed in a burst. Below `1`: 1 |
 | `server.rate_limits.idle_ttl` | duration | `10m` | How long a client's limiter is kept after its last request; `0s`: for ever |
 #### PROXY Protocol and Proxy Headers
 | Key | Type | Default | Notes |
 |---|---|---|---|
 | `server.proxy.trusted` | list | `127.0.0.1/32` | IP addresses and CIDR ranges of proxies whose forwarding headers are trusted; empty: none |
 | `server.proxy.client_ip_headers` | list | `X-Forwarded-For, Forwarded, X-Real-IP` | Headers read for the client IP from a trusted proxy, in priority order; empty: always the connection's address |
 | `server.proxy.strip_untrusted` | bool | `true` | Remove forwarding headers from requests not from a trusted proxy |
 | `server.proxy.proxy_protocol.enabled` | bool | `false` | Accept the PROXY protocol from trusted sources |
 | `server.proxy.proxy_protocol.trusted_sources` | list | | Addresses and CIDR ranges allowed to send a PROXY header; empty: `server.proxy.trusted` |
 | `server.proxy.proxy_protocol.required` | bool | `false` | Reject connections from trusted sources without a PROXY header |
 | `server.proxy.proxy_protocol.read_header_timeout` | duration | `5s` | How long to wait for the PROXY header; `0s` for none |
 #### CSRF & Cross-Origin Protection
 | Key | Type | Default | Notes |
 |---|---|---|---|
 | `server.csrf.cross_origin_protection.enabled` | bool | `true` | Reject cross-origin unsafe requests (`Sec-Fetch-Site`/`Origin`) |
 | `server.csrf.cross_origin_protection.trusted_origins` | list | | Extra origins (`https://admin.example.com`) allowed to make them, and to open WebSockets; empty: none |
 | `server.csrf.tokens.enabled` | bool | `true` | Require a signed double-submit token on unsafe requests |
 | `server.csrf.tokens.key` | string | | Hex HMAC key; empty: a random per-process key, so tokens don't survive a restart. `job-search config generate-csrf-key` stores one |
 | `server.csrf.tokens.cookie_name` | string | `js_csrf` | Empty: `js_csrf` |
 | `server.csrf.tokens.header_name` | string | `X-CSRF-Token` | Empty: `X-CSRF-Token` |
 | `server.csrf.tokens.token_ttl` | duration | `12h` | `0s`: 12h |
 | `server.csrf.exempt_paths` | list | | ServeMux-style patterns skipped by both CSRF layers (e.g. `/api/webhooks/`); empty: none |
 #### TLS
 | Key | Type | Default | Notes |
 |---|---|---|---|
 | `server.tls.enabled` | bool | `false` | Serve HTTPS |
 | `server.tls.cert_file` | string | | Certificate file; must be set when TLS is enabled |
 | `server.tls.key_file` | string | | Private key file; must be set when TLS is enabled |
 | `server.tls.min_version` | string | `1.3` | `1.2` or `1.3`; anything else, including empty, means 1.2 |
 | `server.tls.redirect_http_port` | int | | Port of a plain HTTP listener that redirects to HTTPS; `0`: none |
 #### Headers
 | Key | Type | Default | Notes |
 |---|---|---|---|
 | `server.headers.allowed_hosts` | list | | Allowed `Host` headers, `*.example.com` supported; empty disables the check |
 | `server.headers.content_security_policy` | string | | Security response headers; empty leaves the header unset |
 | `server.headers.frame_options` | string | | |
 | `server.headers.referrer_policy` | string | | |
 | `server.headers.permissions_policy` | string | | |
 | `server.headers.cross_origin_opener_policy` | string | | |
 | `server.headers.cross_origin_resource_policy` | string | | |
 | `server.headers.extra_response_headers` | list of objects | | Extra headers set on every response, each `{header: ..., value: ...}`. `header` must be a valid header name and `value` must not contain a line break |
 | `server.headers.remove_response_headers` | list | | Headers stripped from every response (e.g. `Server`); empty: none |
 | `server.headers.hsts.enabled` | bool | `false` | Send `Strict-Transport-Security` on HTTPS requests |
 | `server.headers.hsts.max_age` | duration | `8760h` | `0s`: the header isn't sent |
 | `server.headers.hsts.include_subdomains` | bool | `true` | |
 | `server.headers.hsts.preload` | bool | `false` | |
 #### Frontend
 | Key | Type | Default | Notes |
 |---|---|---|---|
 | `server.frontend.dir` | string | | Directory of the frontend's build output, holding `index.html`; empty: the build embedded in the binary (see `npm run sync` in `web/`), or no frontend if it has none |
 | `server.frontend.immutable_prefix` | string | | Where Vite emits content-hashed files, cached for a year; empty: `/assets/` |
 | `server.frontend.index_cache_control` | string | | `Cache-Control` of HTML and other files without a hash; empty: `no-cache` |
 #### Websockets
 | Key | Type | Default | Notes |
 |---|---|---|---|
 | `server.websocket.enable_compression` | bool | `false` | permessage-deflate |
 | `server.websocket.max_message_size` | byte size | `32KiB` | Largest message read from a client; `0B` for 32KiB. `-1` (no limit) is rejected, as a client could then exhaust the server's memory |
 | `server.websocket.connection_limit.*` | | | A rate limit rule for new connections from each client IP, WebSocket and plain TCP alike; limiters are kept for `server.rate_limits.idle_ttl` |
 | `server.websocket.message_rate_limit.*` | | | A rate limit rule for messages from each connection |
 | `server.websocket.write_timeout` | duration | `10s` | `0s` for none |
 | `server.websocket.ping_interval` | duration | `30s` | How long a connection may be idle before it is pinged; `0s`: no pings |
 | `server.websocket.pong_timeout` | duration | `10s` | `0s` for none |
 ## Complex Types
 A rate limit rule (`JSCRateLimitRule`) has `enabled`, `per_second` and `burst`.
 It is off unless `enabled` and `per_second` is greater than 0; a `burst` below
 1 means 1. It is used under several keys: `server.rate_limits.global`,
 `server.rate_limits.api`, `server.websocket.connection_limit` and
 `server.websocket.message_rate_limit`. Rate limits are implemented using the
 token bucket algorithm.
 Lists can be given as YAML/JSON lists or as a comma-separated string (e.g. in
 an environment variable). A list of objects such as `server.headers.extra_response_headers`
 is a YAML/JSON list of objects in a config file, or a JSON array in an
 environment variable or `config set`:
 `'[{"header":"X-Robots-Tag","value":"noindex"}]'`.
 ## Example Configuration
 Example `config.yaml`:
 ```yaml
 logging:
    level: info
    file: ""
 database:
    file: ./jobsearch.db
    change_batch_retention: 720h
 sources:
    find_a_tender:
        enabled: true
        interval: 10m
    fts_mock:
        enabled: false
        interval: 1m
        api_root: http://localhost:8080/api/1.0
 server:
    host: 0.0.0.0
    port: 9393
    max_body_bytes: 2MB
    proxy:
        trusted: [10.0.0.0/8]
    rate_limits:
        api:
            enabled: true
            per_second: 20
            burst: 40
    headers:
        allowed_hosts: [jobs.example.com]
        extra_response_headers:
            - header: X-Robots-Tag
              value: noindex
        remove_response_headers: [Server]
        hsts:
            enabled: true
 ```
 ## Sections and struct tags
 Each section of the configuration is a plain struct. The reflection code in
 `section.go` reads its struct tags, so a section needs no hand-written code
 per key:
 - `cfg:"key"` is the field's configuration key. Fields without it are not part
  of the configuration.
 - `default:"value"` is the field's default, parsed as if it were a string
  passed to `SetValue` (e.g. `"10m"`, `"true"`).
 - `init:"hidden"` on a nested section's field leaves the section out of the
  file `config init` writes, unless `--all` is given (`InitDefaults`). The
  test sources, `fts_mock` and `sleeper`, have it. The section is otherwise
  as any other: loaded, and read and set by the `config` commands.
 - A tagged field whose type is a struct is a nested section, addressed as
  `section.key`. An embedded struct has its fields promoted, as Go does;
  every source embeds `JSCSourcesSource` for `enabled` and `interval`.
 - `time.Duration` fields report `reflect.String` from `TypeOf`, accept either a
  `time.Duration` or a duration string in `SetValue`, and are given out as
  strings (`"10m0s"`) by `Map`, `Flatten` and `ValueOf`, and so by `JSON`,
  `YAML` and `String` too.
 - `util.ByteSize` fields work the same way, with size strings such as
  `"32KiB"` or `"1MB"` (KB, MB, ... are powers of 1000; KiB, MiB, ... powers
  of 1024), and are given out exactly in the largest unit that divides them
  (`"1MB"`). `-1` means no limit (`util.ByteSizeNoLimit`); no other size may
  be negative. A setting for which no limit is unsafe rejects `-1`.
 - `[]string` fields report `reflect.Slice` from `TypeOf` and accept either a
  `[]string` or a comma-separated string such as `"a, b"` in `SetValue` (and so
  in default tags). An empty list is stored as `nil`.
 - A list of sections, `[]S` for a section struct `S` (e.g. `[]JSCServerHeadersExtra`),
  is a single key reporting `reflect.Slice`. It is given out as a
  `[]map[string]any`, one map per item, and `SetValue` accepts a `[]S`, a list
  of maps (as Viper reads a YAML/JSON list of objects) or a JSON array string.
  Each item starts from its defaults and its keys are set as `SetValue` sets
  them. `S` implements `Section` and, like a type used under several keys, has
  no `JSCK*` constant.
 - A section can reject values by implementing `validate(key, value) error` on
  its pointer type. `JSCLogging` does this for `level`, `JSCSourcesSource` for
  `interval`, `JSCServer` for `max_header_bytes`, `JSCServerTimeouts` for
  `read_header` (rejecting 0 or less) and `JSCServerWebSocket` for
  `max_message_size` (rejecting `-1`), and `JSCServerHeadersExtra` for
  `header` and `value`. Each item in a list of
  sections is validated whole, so a key left out of an item is checked too.
 Every section implements the `Section` interface defined in `section.go`:
 `Map`, `Flatten`, `TypeOf`, `ValueOf` and `SetValue`, each a one-line call into
 `section.go` (some sections also have `Keys`). The methods sit under a
 `// ---------- Section methods (see Section) ----------` banner in each
 section's file. A section's `Flatten` keys are prefixed with its `JSCK*`
 constant, except for a section type used under several keys, such as
 `JSCRateLimitRule`: it has no constant and its keys are unprefixed.
 `TestSections` walks `JobSearchConfig` and checks every
 section, including embedded ones, implements `Section` and that its
 `Flatten` keys match its `cfg` tag, and that its `Map`, `TypeOf`, `ValueOf`
 and `SetValue` agree with the whole configuration's view of the same keys (the
 items of a list of sections are checked the same way), so a new section is
 checked without being listed anywhere.
 ## Adding a key or section
 To add a key to an existing section, add a tagged field to its struct. Nothing
 else is needed for it to be loaded, printed, and read or written by the
 `config` CLI commands.
 To add a new section:
 1. Create a `conf_*.go` file with the section struct, a `JSCK*` constant for
   its key, and an implementation of `Section` (copy an existing section's
   methods). A type that will be used under several keys, like
   `JSCRateLimitRule`, has no constant and passes `""` to `flattenOf`.
 2. Add the struct as a tagged field of its parent: `JobSearchConfig` in
   `config.go`, or `JSCSources` in `conf_sources.go` for a source,
   etc. The field's `cfg` tag must match the `JSCK*` constant. Add
   `init:"hidden"` if it's only for testing.
 Nothing in `section.go` needs changing.
 ## API
 - `Init(cfgFile, &conf)`: load the configuration, as described above.
 - `conf.TypeOf(key)`, `conf.ValueOf(key)`, `conf.SetValue(key, value)`: read
  or set one dotted key on the loaded struct. `SetValue` converts and
  validates the value.
 - `conf.Map()` (nested) and `conf.Flatten()` (dotted keys): the whole
  configuration as a map.
 - `conf.String()`, `conf.JSON()`, `conf.YAML()` and the matching `Print*`
  methods: the configuration as text, with keys sorted.
 - `StoreValue(&conf, key, value)` / `PurgeValue(&conf, key)`: set or remove a
  key in the config file Viper loaded. Used by `config set` and
  `config set --unset`. `StoreValue` validates the value before writing it,
  and stores it as the field's own type. Only the file's own settings are
  written back; defaults and environment variables never are.
 - `LoadDefaultConfig`, `LoadToViper`, `LoadFromViper`: move values between
  the struct defaults, Viper and a `JobSearchConfig`.
 ## Files
 | File | Contents |
 |---|---|
 | `config.go` | `JobSearchConfig`, defaults, Viper loading, text output |
 | `conf_database.go` | `database` section |
 | `load.go` | `Init`, `.env` and config file loading |
 | `modify.go` | `StoreValue` and `PurgeValue` |
 | `section.go` | The `Section` interface and the struct tag reflection shared by every section |
 | `conf_logging.go` | `logging` section and log level helpers |
 | `conf_sources.go` | `sources` section and the shared source fields |
 | `conf_source_*.go` | One file per source section |
 | `conf_server.go` | `server` section |
 | `conf_server_*.go` | One file per `server` sub-section (`csrf`, `frontend`, `headers`, `proxy`, `rate_limits`, `timeouts`, `tls`, `websocket`) |
 | `conf_ratelimitrule.go` | `JSCRateLimitRule`, the rate limit rule shared by several keys |
 | `print.go`, `debug.go` | Printing and debug logging of the configuration |
 ## Testing
 Viper keeps its state in package globals, so tests that use it call
 `resetViper(t)` to reset it before and after the test. Tests of `Init`
 use `initDir`, which changes into a temp directory with its own `.env` and
 `config.yaml`, so the real files in the repository are never read.
@@ -0,0 +1,47 @@
 package config
 import (
 	"reflect"
 	"time"
 )
 // JSCKDatabase is the database section's configuration key.
 const JSCKDatabase string = "database"
 // JSCDatabase configures the SQLite database. File is the path to
 // the database file; empty means none, and commands that need the database
 // fail with a message saying so. ChangeBatchRetention is how long a batch of
 // changed opportunities is kept for clients to catch up from (see
 // store.WithChangeBatchRetention); 0 or less means they are kept for ever.
 type JSCDatabase struct {
 	File                 string        `cfg:"file"`
 	ChangeBatchRetention time.Duration `cfg:"change_batch_retention" default:"720h"`
 }
 // ---------- Section methods (see Section) ----------
 // Keys returns the keys directly in the section.
 func (c *JSCDatabase) Keys() []string { return keysOf(c) }
 // Map implements Section.
 func (c JSCDatabase) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCDatabase) Flatten() map[string]any {
 	return flattenOf(JSCKDatabase, c)
 }
 // TypeOf implements Section.
 func (c JSCDatabase) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCDatabase) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCDatabase) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
@@ -1,19 +1,3 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 package config
 import (
@@ -22,73 +6,65 @@ import (
 	"reflect"
 )
-type JobSearchConfigLogging struct {
+// JSCKLogging is the logging section's configuration key.
-	Level string
+const JSCKLogging string = "logging"
 // JSCLogging configures logging. Level is one of debug, info,
 // warn or error. File, if set, is where commands that log to a file write;
 // empty means they log to the terminal.
 type JSCLogging struct {
 	Level string `cfg:"level" default:"warn"`
 	File  string `cfg:"file"`
 }
-// Default configuration
+// ---------- Section methods (see Section) ----------
-func defaultConfigLogging() JobSearchConfigLogging {
+
-	return JobSearchConfigLogging{
+// Map implements Section.
-		Level: "warn",
+func (c JSCLogging) Map() map[string]any { return mapOf(c) }
-	}
+
 // Flatten implements Section.
 func (c JSCLogging) Flatten() map[string]any {
 	return flattenOf(JSCKLogging, c)
 }
-// Converts configuration to a map
+// TypeOf implements Section.
-func (conf JobSearchConfigLogging) Map() map[string]any {
+func (c JSCLogging) TypeOf(key string) (reflect.Kind, error) {
-	return map[string]any{
+	return typeOf(c, key)
 		"level": conf.Level,
 	}
 }
-// Converts configuration to a flat dotted format
+// ValueOf implements Section.
-func (conf JobSearchConfigLogging) Flatten() map[string]any {
+func (c JSCLogging) ValueOf(key string) (any, error) {
-	confMap := conf.Map()
+	return valueOf(c, key)
 	retMap := map[string]any{}
 	for k, v := range confMap {
 		retMap["logging."+k] = v
 	}
 	return retMap
 }
-// Returns the expected type of a configuration key value
+// SetValue implements Section.
-func (c JobSearchConfigLogging) TypeOf(key string) (reflect.Kind, error) {
+func (c *JSCLogging) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
 // Keys returns the keys directly in the section.
 func (c *JSCLogging) Keys() []string { return keysOf(c) }
 // ---------- Section specific methods ----------
 // validate checks values before they are stored (see validator).
 func (c *JSCLogging) validate(key string, value any) error {
 	switch key {
 	case "level":
-		return reflect.String, nil
+		switch value.(string) {
 	default:
 		return reflect.Invalid, errors.New("unknown configuration key")
 	}
 }
 // Returns the value of a configuration key
 func (c JobSearchConfigLogging) ValueOf(key string) (any, error) {
 	switch key {
 	case "level":
 		return c.Level, nil
 	default:
 		return "", errors.New("unknown configuration key")
 	}
 }
 // Sets the value of a configuration key
 func (c *JobSearchConfigLogging) SetValue(key string, value any) error {
 	switch key {
 	case "level":
 		newLevel := value.(string)
 		switch newLevel {
 		case "debug", "info", "warn", "error":
 			c.Level = value.(string)
 		default:
 			return errors.New(
-				"invalid log level. Must be one of: debug, info, warn, error")
+				"config: invalid log level: must be one of debug, info, warn, error",
 			)
 		}
 	default:
 		return errors.New("unknown configuration key")
 	}
 	return nil
 }
-func ConfigLevelToLogLevel(level string) slog.Level {
+// LevelToLogLevel converts a configured level name to an slog level. An
 // unknown name gives 99, above every real level, so nothing is logged.
 func LevelToLogLevel(level string) slog.Level {
 	switch level {
 	case "debug":
 		return slog.LevelDebug
@@ -103,6 +79,8 @@ func ConfigLevelToLogLevel(level string) slog.Level {
 	}
 }
-func (c *JobSearchConfigLogging) LogLevel() slog.Level {
+// LogLevel returns the configured level as an slog level. An unknown level
-	return ConfigLevelToLogLevel(c.Level)
+// gives 99, above every real level, so nothing is logged.
 func (c *JSCLogging) LogLevel() slog.Level {
 	return LevelToLogLevel(c.Level)
 }
@@ -0,0 +1,36 @@
 package config
 import "reflect"
 // JSCRateLimitRule configures one rate limit: PerSecond sustained, with
 // bursts of up to Burst. It is off unless Enabled and PerSecond is greater
 // than 0; a Burst below 1 means 1. The type is used under several keys so it
 // has no JSCK* constant and its Flatten keys are not prefixed.
 type JSCRateLimitRule struct {
 	Enabled   bool `cfg:"enabled"    default:"false"`
 	PerSecond int  `cfg:"per_second" default:"50"`
 	Burst     int  `cfg:"burst"      default:"100"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCRateLimitRule) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCRateLimitRule) Flatten() map[string]any { return flattenOf("", c) }
 // TypeOf implements Section.
 func (c JSCRateLimitRule) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCRateLimitRule) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCRateLimitRule) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
@@ -0,0 +1,850 @@
 package config
 import (
 	"bytes"
 	"encoding/json"
 	"log/slog"
 	"reflect"
 	"strings"
 	"testing"
 	"time"
 	"github.com/spf13/viper"
 	"go.yaml.in/yaml/v3"
 )
 // resetViper isolates tests that touch the global Viper instance.
 func resetViper(t *testing.T) {
 	t.Helper()
 	viper.Reset()
 	t.Cleanup(viper.Reset)
 }
 // captureLogs sends the default slog logger to a buffer until the test ends.
 func captureLogs(t *testing.T, level slog.Level) *bytes.Buffer {
 	t.Helper()
 	var buf bytes.Buffer
 	def := slog.Default()
 	handler := slog.NewTextHandler(&buf, &slog.HandlerOptions{
 		Level: level,
 	})
 	slog.SetDefault(slog.New(handler))
 	t.Cleanup(func() {
 		slog.SetDefault(def)
 	})
 	return &buf
 }
 func defaultConfig(t *testing.T) JobSearchConfig {
 	t.Helper()
 	var conf JobSearchConfig
 	if err := applyDefaults(&conf); err != nil {
 		t.Fatalf("defaults: %v", err)
 	}
 	return conf
 }
 func TestDefaults(t *testing.T) {
 	conf := defaultConfig(t)
 	if conf.Logging.Level != "warn" {
 		t.Errorf("Logging.Level = %q; want warn", conf.Logging.Level)
 	}
 	const apiRoot = "https://www.find-tender.service.gov.uk/api/1.0"
 	if conf.Sources.FindATender.APIRoot != apiRoot {
 		t.Errorf("FindATender.APIRoot = %q", conf.Sources.FindATender.APIRoot)
 	}
 	if conf.Sources.FTSMock.APIRoot != "" {
 		t.Errorf("FTSMock.APIRoot = %q; want empty", conf.Sources.FTSMock.APIRoot)
 	}
 	if conf.Sources.FindATender.RateLimitRetries != 5 ||
 		conf.Sources.FTSMock.RateLimitRetries != 5 {
 		t.Errorf(
 			"RateLimitRetries = %d, %d; want 5, 5",
 			conf.Sources.FindATender.RateLimitRetries,
 			conf.Sources.FTSMock.RateLimitRetries,
 		)
 	}
 	for name, src := range map[string]JSCSourcesSource{
 		"find_a_tender": conf.Sources.FindATender.JSCSourcesSource,
 		"fts_mock":      conf.Sources.FTSMock.JSCSourcesSource,
 		"sleeper":       conf.Sources.Sleeper.JSCSourcesSource,
 	} {
 		if src.Enabled || src.Interval != 10*time.Minute {
 			t.Errorf("%s source = %+v; want disabled, 10m", name, src)
 		}
 	}
 }
 func TestLoggingSection(t *testing.T) {
 	c := JSCLogging{
 		Level: "info",
 		File:  "/tmp/x.log",
 	}
 	if got, want := c.Keys(), []string{
 		"level",
 		"file",
 	}; !reflect.DeepEqual(got, want) {
 		t.Errorf("Keys = %v; want %v", got, want)
 	}
 	if got, want := c.Map(), map[string]any{
 		"level": "info",
 		"file":  "/tmp/x.log",
 	}; !reflect.DeepEqual(got, want) {
 		t.Errorf("Map = %v; want %v", got, want)
 	}
 	if got, want := c.Flatten(), map[string]any{
 		"logging.level": "info",
 		"logging.file":  "/tmp/x.log",
 	}; !reflect.DeepEqual(got, want) {
 		t.Errorf("Flatten = %v; want %v", got, want)
 	}
 	if k, err := c.TypeOf("file"); err != nil || k != reflect.String {
 		t.Errorf("TypeOf(file) = %v, %v", k, err)
 	}
 	if v, err := c.ValueOf("level"); err != nil || v != "info" {
 		t.Errorf("ValueOf(level) = %v, %v", v, err)
 	}
 	if _, err := c.TypeOf("nope"); err == nil {
 		t.Error("TypeOf(nope) succeeded")
 	}
 	if err := c.SetValue("file", "/var/log/js.log"); err != nil ||
 		c.File != "/var/log/js.log" {
 		t.Errorf("SetValue(file) = %v, File = %q", err, c.File)
 	}
 }
 func TestLoggingLevelValidation(t *testing.T) {
 	var c JSCLogging
 	for _, lvl := range []string{"debug", "info", "warn", "error"} {
 		if err := c.SetValue("level", lvl); err != nil || c.Level != lvl {
 			t.Errorf("SetValue(level, %q) = %v, Level = %q", lvl, err, c.Level)
 		}
 	}
 	for _, lvl := range []string{"", "trace", "WARN", "fatal"} {
 		if err := c.SetValue("level", lvl); err == nil ||
 			!strings.Contains(err.Error(), "invalid log level") {
 			t.Errorf("SetValue(level, %q) = %v; want invalid log level", lvl, err)
 		}
 	}
 	if c.Level != "error" {
 		t.Errorf("rejected level was stored: %q", c.Level)
 	}
 	// Validation also applies through the top-level config.
 	conf := defaultConfig(t)
 	if err := conf.SetValue("logging.level", "loud"); err == nil {
 		t.Error("JobSearchConfig.SetValue(logging.level, loud) succeeded")
 	}
 	// Other keys are not validated.
 	if err := c.validate("file", "anything"); err != nil {
 		t.Errorf("validate(file) = %v", err)
 	}
 }
 func TestLevelToLogLevel(t *testing.T) {
 	tests := map[string]slog.Level{
 		"debug": slog.LevelDebug,
 		"info":  slog.LevelInfo,
 		"warn":  slog.LevelWarn,
 		"error": slog.LevelError,
 		"":      99,
 		"bogus": 99,
 	}
 	for in, want := range tests {
 		if got := LevelToLogLevel(in); got != want {
 			t.Errorf("LevelToLogLevel(%q) = %v; want %v", in, got, want)
 		}
 		c := JSCLogging{
 			Level: in,
 		}
 		if got := c.LogLevel(); got != want {
 			t.Errorf("LogLevel() with %q = %v; want %v", in, got, want)
 		}
 	}
 }
 func TestSourceSection(t *testing.T) {
 	c := JSCSourcesSource{
 		Enabled:  true,
 		Interval: time.Minute,
 	}
 	if got, want := c.Keys(), []string{
 		"enabled",
 		"interval",
 	}; !reflect.DeepEqual(got, want) {
 		t.Errorf("Keys = %v; want %v", got, want)
 	}
 	want := map[string]any{"enabled": true, "interval": "1m0s"}
 	if got := c.Map(); !reflect.DeepEqual(got, want) {
 		t.Errorf("Map = %v; want %v", got, want)
 	}
 	wantFlat := map[string]any{"enabled": true, "interval": "1m0s"}
 	if got := c.Flatten(); !reflect.DeepEqual(got, wantFlat) {
 		t.Errorf("Flatten = %v; want %v (no prefix)", got, wantFlat)
 	}
 	if k, err := c.TypeOf("interval"); err != nil || k != reflect.String {
 		t.Errorf("TypeOf(interval) = %v, %v; want String", k, err)
 	}
 	if k, err := c.TypeOf("enabled"); err != nil || k != reflect.Bool {
 		t.Errorf("TypeOf(enabled) = %v, %v; want Bool", k, err)
 	}
 	if v, err := c.ValueOf("interval"); err != nil || v != "1m0s" {
 		t.Errorf("ValueOf(interval) = %v, %v", v, err)
 	}
 	if err := c.SetValue("interval", "5s"); err != nil ||
 		c.Interval != 5*time.Second {
 		t.Errorf("SetValue(interval, 5s) = %v, Interval = %v", err, c.Interval)
 	}
 	if err := c.SetValue("interval", "never"); err == nil {
 		t.Error("SetValue(interval, never) succeeded")
 	}
 }
 func TestConcreteSourceSections(t *testing.T) {
 	tests := []struct {
 		prefix        string
 		section       Section
 		apiRoot       bool
 		blockShutdown bool
 	}{
 		{JSCKSourceFindATender, &JSCSourceFindATender{}, true, false},
 		{JSCKSourceFTSMock, &JSCSourceFTSMock{}, true, false},
 		{JSCKSourceSleeper, &JSCSourceSleeper{}, false, true},
 	}
 	for _, tt := range tests {
 		t.Run(tt.prefix, func(t *testing.T) {
 			s := tt.section
 			// Embedded source fields are promoted.
 			if err := s.SetValue("enabled", "true"); err != nil {
 				t.Fatalf("SetValue(enabled): %v", err)
 			}
 			if err := s.SetValue("interval", 2*time.Minute); err != nil {
 				t.Fatalf("SetValue(interval): %v", err)
 			}
 			if v, err := s.ValueOf("enabled"); err != nil || v != true {
 				t.Errorf("ValueOf(enabled) = %v, %v", v, err)
 			}
 			if k, err := s.TypeOf("interval"); err != nil || k != reflect.String {
 				t.Errorf("TypeOf(interval) = %v, %v", k, err)
 			}
 			wantMap := map[string]any{"enabled": true, "interval": "2m0s"}
 			wantFlat := map[string]any{
 				tt.prefix + ".enabled":  true,
 				tt.prefix + ".interval": "2m0s",
 			}
 			if tt.apiRoot {
 				if err := s.SetValue("api_root", "http://localhost:8080"); err != nil {
 					t.Fatalf("SetValue(api_root): %v", err)
 				}
 				if v, err := s.ValueOf("api_root"); err != nil ||
 					v != "http://localhost:8080" {
 					t.Errorf("ValueOf(api_root) = %v, %v", v, err)
 				}
 				wantMap["api_root"] = "http://localhost:8080"
 				wantFlat[tt.prefix+".api_root"] = "http://localhost:8080"
 				// The Find a Tender sources' own validate must still check
 				// interval, as JSCSourcesSource's does.
 				if err := s.SetValue("rate_limit_retries", "3"); err != nil {
 					t.Fatalf("SetValue(rate_limit_retries): %v", err)
 				}
 				if err := s.SetValue("rate_limit_retries", "-1"); err == nil {
 					t.Error("SetValue(rate_limit_retries, -1) succeeded")
 				}
 				if err := s.SetValue("interval", "0s"); err == nil {
 					t.Error("SetValue(interval, 0s) succeeded")
 				}
 				wantMap["rate_limit_retries"] = 3
 				wantFlat[tt.prefix+".rate_limit_retries"] = 3
 			} else if _, err := s.TypeOf("api_root"); err == nil {
 				t.Error("TypeOf(api_root) succeeded on a source without one")
 			}
 			if tt.blockShutdown {
 				if err := s.SetValue("block_shutdown", "true"); err != nil {
 					t.Fatalf("SetValue(block_shutdown): %v", err)
 				}
 				if k, err := s.TypeOf("block_shutdown"); err != nil ||
 					k != reflect.Bool {
 					t.Errorf("TypeOf(block_shutdown) = %v, %v", k, err)
 				}
 				wantMap["block_shutdown"] = true
 				wantFlat[tt.prefix+".block_shutdown"] = true
 			} else if _, err := s.TypeOf("block_shutdown"); err == nil {
 				t.Error("TypeOf(block_shutdown) succeeded on a source without one")
 			}
 			if got := s.Map(); !reflect.DeepEqual(got, wantMap) {
 				t.Errorf("Map = %v; want %v", got, wantMap)
 			}
 			if got := s.Flatten(); !reflect.DeepEqual(got, wantFlat) {
 				t.Errorf("Flatten = %v; want %v", got, wantFlat)
 			}
 			if _, err := s.ValueOf("nope"); err == nil {
 				t.Error("ValueOf(nope) succeeded")
 			}
 		})
 	}
 }
 // InitDefaults leaves out the sections tagged init:"hidden" unless asked.
 func TestInitDefaults(t *testing.T) {
 	all := InitDefaults(true)
 	shown := InitDefaults(false)
 	if all["sources.sleeper.enabled"] != false ||
 		all["sources.fts_mock.interval"] != "10m0s" {
 		t.Errorf("all = %v; want the test sources' defaults", all)
 	}
 	for key := range all {
 		hidden := strings.HasPrefix(key, "sources.fts_mock.") ||
 			strings.HasPrefix(key, "sources.sleeper.")
 		if _, found := shown[key]; found == hidden {
 			t.Errorf("%s: in the generated defaults = %v", key, found)
 		}
 	}
 	if shown["sources.eu_supply.base_url"] != "https://uk.eu-supply.com" {
 		t.Errorf("eu_supply missing: %v", shown)
 	}
 }
 func TestEUSupplySection(t *testing.T) {
 	conf := defaultConfig(t)
 	got := conf.Sources.EUSupply
 	if got.Enabled || got.Interval != 10*time.Minute ||
 		got.BaseURL != "https://uk.eu-supply.com" ||
 		got.RequestMinGap != 2*time.Second ||
 		got.RequestMaxGap != 10*time.Second {
 		t.Errorf("defaults = %+v", got)
 	}
 	c := JSCSourceEUSupply{}
 	if err := c.SetValue("request_min_gap", "5s"); err != nil ||
 		c.RequestMinGap != 5*time.Second {
 		t.Errorf("SetValue(request_min_gap, 5s): %v, %v", err, c.RequestMinGap)
 	}
 	for _, key := range []string{"request_min_gap", "request_max_gap"} {
 		if err := c.SetValue(key, "0s"); err == nil {
 			t.Errorf("SetValue(%s, 0s) succeeded", key)
 		}
 	}
 	// Its own validate must still check interval, as JSCSourcesSource's does.
 	if err := c.SetValue("interval", "0s"); err == nil {
 		t.Error("SetValue(interval, 0s) succeeded")
 	}
 	if v, err := c.ValueOf("base_url"); err != nil || v != "" {
 		t.Errorf("ValueOf(base_url) = %v, %v", v, err)
 	}
 }
 func TestSourcesSection(t *testing.T) {
 	var c JSCSources
 	if err := c.SetValue("fts_mock.api_root", "http://mock"); err != nil {
 		t.Fatalf("SetValue: %v", err)
 	}
 	if c.FTSMock.APIRoot != "http://mock" {
 		t.Errorf("FTSMock.APIRoot = %q", c.FTSMock.APIRoot)
 	}
 	if v, err := c.ValueOf("fts_mock.api_root"); err != nil ||
 		v != "http://mock" {
 		t.Errorf("ValueOf = %v, %v", v, err)
 	}
 	if k, err := c.TypeOf("sleeper.enabled"); err != nil || k != reflect.Bool {
 		t.Errorf("TypeOf(sleeper.enabled) = %v, %v", k, err)
 	}
 	if _, err := c.TypeOf("sleeper"); err == nil {
 		t.Error("TypeOf(sleeper) succeeded on a section key")
 	}
 	m := c.Map()
 	for _, k := range []string{"find_a_tender", "fts_mock", "sleeper"} {
 		if _, ok := m[k].(map[string]any); !ok {
 			t.Errorf("Map()[%q] = %#v; want nested map", k, m[k])
 		}
 	}
 	flat := c.Flatten()
 	if flat["sources.fts_mock.api_root"] != "http://mock" {
 		t.Errorf("Flatten missing sources.fts_mock.api_root: %v", flat)
 	}
 	if len(flat) != 17 {
 		t.Errorf("Flatten has %d keys; want 17: %v", len(flat), flat)
 	}
 	if err := c.SetValue("skip_rules_file", "rules.json"); err != nil ||
 		c.SkipRulesFile != "rules.json" {
 		t.Errorf("SetValue(skip_rules_file): %v, %q", err, c.SkipRulesFile)
 	}
 }
 func TestJobSearchConfigMap(t *testing.T) {
 	conf := defaultConfig(t)
 	m := conf.Map()
 	logging, ok := m["logging"].(map[string]any)
 	if !ok || logging["level"] != "warn" {
 		t.Errorf("Map()[logging] = %#v", m["logging"])
 	}
 	sources, ok := m["sources"].(map[string]any)
 	if !ok {
 		t.Fatalf("Map()[sources] = %#v", m["sources"])
 	}
 	fat, ok := sources["find_a_tender"].(map[string]any)
 	if !ok || fat["interval"] != "10m0s" {
 		t.Errorf("Map()[sources][find_a_tender] = %#v", sources["find_a_tender"])
 	}
 }
 func TestString(t *testing.T) {
 	conf := defaultConfig(t)
 	s := conf.String()
 	for _, want := range []string{
 		"logging.level = warn <string>\n",
 		"sources.sleeper.enabled = false <bool>\n",
 		"sources.find_a_tender.interval = 10m0s <string>\n",
 	} {
 		if !strings.Contains(s, want) {
 			t.Errorf("String() missing %q:\n%s", want, s)
 		}
 	}
 	if n := strings.Count(s, "\n"); n != len(conf.Flatten()) {
 		t.Errorf("String() has %d lines; want %d", n, len(conf.Flatten()))
 	}
 }
 // JSON and YAML give the configuration as nested sections.
 func TestSerialised(t *testing.T) {
 	conf := defaultConfig(t)
 	tests := []struct {
 		name      string
 		output    string
 		unmarshal func([]byte, any) error
 	}{
 		{"JSON", conf.JSON(), json.Unmarshal},
 		{"YAML", conf.YAML(), yaml.Unmarshal},
 	}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
 			var m map[string]map[string]any
 			if err := tt.unmarshal([]byte(tt.output), &m); err != nil {
 				t.Fatalf("%s() doesn't parse: %v\n%s", tt.name, err, tt.output)
 			}
 			if m["logging"]["level"] != "warn" {
 				t.Errorf("logging.level = %#v", m["logging"]["level"])
 			}
 			for _, source := range []string{"fts_mock", "sleeper"} {
 				if _, ok := m["sources"][source]; !ok {
 					t.Errorf("missing sources.%s: %v", source, m["sources"])
 				}
 			}
 		})
 	}
 }
 // Output of JSON()/YAML() can be read back as a config file.
 func TestSerialisedRoundTrip(t *testing.T) {
 	orig := defaultConfig(t)
 	orig.Logging.Level = "info"
 	orig.Sources.FTSMock.Enabled = true
 	orig.Sources.FTSMock.APIRoot = "http://mock"
 	orig.Server.WebSocket.MaxMessageSize = 64 << 10
 	orig.Server.MaxBodyBytes = 2 << 20
 	orig.Server.Headers.ExtraResponseHeaders = []JSCServerHeadersExtra{
 		{
 			Header: "X-Robots-Tag",
 			Value:  "noindex",
 		},
 		{
 			Header: "Cache-Control",
 			Value:  "no-store, max-age=0",
 		},
 	}
 	orig.Server.Headers.RemoveResponseHeaders = []string{"Server", "X-Powered-By"}
 	for _, format := range []string{"json", "yaml"} {
 		t.Run(format, func(t *testing.T) {
 			resetViper(t)
 			out := orig.JSON()
 			if format == "yaml" {
 				out = orig.YAML()
 			}
 			viper.SetConfigType(format)
 			if err := viper.ReadConfig(strings.NewReader(out)); err != nil {
 				t.Fatalf("ReadConfig: %v", err)
 			}
 			var got JobSearchConfig
 			LoadFromViper(&got)
 			if !reflect.DeepEqual(got, orig) {
 				t.Errorf("round trip =\n%+v\nwant\n%+v", got, orig)
 			}
 		})
 	}
 }
 func TestLoadToViper(t *testing.T) {
 	resetViper(t)
 	conf := defaultConfig(t)
 	LoadToViper(conf)
 	for key, want := range conf.Flatten() {
 		if got := viper.Get(key); !reflect.DeepEqual(got, want) {
 			t.Errorf("viper.Get(%q) = %#v; want %#v", key, got, want)
 		}
 	}
 }
 func TestLoadDefaultConfig(t *testing.T) {
 	resetViper(t)
 	LoadDefaultConfig()
 	if got := viper.GetString("logging.level"); got != "warn" {
 		t.Errorf("logging.level = %q; want warn", got)
 	}
 	if got := viper.GetDuration(
 		"sources.sleeper.interval",
 	); got != 10*time.Minute {
 		t.Errorf("sources.sleeper.interval = %v; want 10m", got)
 	}
 }
 func TestLoadFromViper(t *testing.T) {
 	resetViper(t)
 	LoadDefaultConfig()
 	viper.Set("logging.level", "debug")
 	viper.Set("logging.file", "/tmp/js.log")
 	viper.Set("sources.sleeper.enabled", "true") // string, read as bool
 	viper.Set("sources.fts_mock.interval", "30s")
 	viper.Set("sources.fts_mock.api_root", "http://mock")
 	viper.Set("unknown.key", "ignored")
 	var conf JobSearchConfig
 	LoadFromViper(&conf)
 	want := defaultConfig(t)
 	want.Logging.Level = "debug"
 	want.Logging.File = "/tmp/js.log"
 	want.Sources.Sleeper.Enabled = true
 	want.Sources.FTSMock.Interval = 30 * time.Second
 	want.Sources.FTSMock.APIRoot = "http://mock"
 	if !reflect.DeepEqual(conf, want) {
 		t.Errorf("LoadFromViper =\n%+v\nwant\n%+v", conf, want)
 	}
 }
 func TestViperRoundTrip(t *testing.T) {
 	resetViper(t)
 	orig := defaultConfig(t)
 	orig.Logging.Level = "error"
 	orig.Sources.FindATender.Enabled = true
 	orig.Sources.FindATender.Interval = 45 * time.Second
 	orig.Server.Headers.ExtraResponseHeaders = []JSCServerHeadersExtra{
 		{
 			Header: "X-Robots-Tag",
 			Value:  "noindex",
 		},
 	}
 	LoadToViper(orig)
 	var got JobSearchConfig
 	LoadFromViper(&got)
 	if !reflect.DeepEqual(got, orig) {
 		t.Errorf("round trip =\n%+v\nwant\n%+v", got, orig)
 	}
 }
 // A list of sections can be given as a JSON array string, as it would be in
 // an environment variable.
 func TestLoadFromViperSectionListString(t *testing.T) {
 	resetViper(t)
 	viper.Set(
 		"server.headers.extra_response_headers",
 		`[{"header": "X-Robots-Tag", "value": "noindex"}]`,
 	)
 	conf := defaultConfig(t)
 	LoadFromViper(&conf)
 	want := []JSCServerHeadersExtra{
 		{
 			Header: "X-Robots-Tag",
 			Value:  "noindex",
 		},
 	}
 	if !reflect.DeepEqual(conf.Server.Headers.ExtraResponseHeaders, want) {
 		t.Errorf(
 			"ExtraResponseHeaders = %+v; want %+v",
 			conf.Server.Headers.ExtraResponseHeaders,
 			want,
 		)
 	}
 }
 func TestHeadersExtraValidation(t *testing.T) {
 	var c JSCServerHeadersExtra
 	for _, h := range []string{"X-Robots-Tag", "x-a", "A1!#$%&'*+-.^_`|~"} {
 		if err := c.SetValue("header", h); err != nil || c.Header != h {
 			t.Errorf("SetValue(header, %q) = %v, Header = %q", h, err, c.Header)
 		}
 	}
 	for _, h := range []string{
 		"", "X Robots", "X-A:", "X-A\r\nX-B", "Ünicode", "(x)",
 	} {
 		if err := c.SetValue("header", h); err == nil ||
 			!strings.Contains(err.Error(), "invalid header name") {
 			t.Errorf("SetValue(header, %q) = %v; want invalid header name", h, err)
 		}
 	}
 	if err := c.SetValue("value", "no-store, max-age=0"); err != nil {
 		t.Errorf("SetValue(value): %v", err)
 	}
 	for _, v := range []string{"a\nb", "a\rb", "a\r\nX-Evil: 1"} {
 		if err := c.SetValue("value", v); err == nil ||
 			!strings.Contains(err.Error(), "invalid header value") {
 			t.Errorf("SetValue(value, %q) = %v; want invalid header value", v, err)
 		}
 	}
 	// Items in the list are validated whole, so a missing header is rejected
 	// too, and the list is left as it was.
 	const key = "server.headers.extra_response_headers"
 	conf := defaultConfig(t)
 	for _, value := range []any{
 		`[{"value": "noindex"}]`,
 		`[{"header": "X-A", "value": "1"}, {"header": "bad name", "value": "2"}]`,
 		`[{"header": "X-A", "value": "a\nb"}]`,
 		[]JSCServerHeadersExtra{
 			{
 				Value: "noindex",
 			},
 		},
 	} {
 		if err := conf.SetValue(key, value); err == nil ||
 			!strings.Contains(err.Error(), "invalid header") {
 			t.Errorf("SetValue(%s, %#v) = %v; want invalid header", key, value, err)
 		}
 		if conf.Server.Headers.ExtraResponseHeaders != nil {
 			t.Errorf(
 				"rejected list was stored: %+v",
 				conf.Server.Headers.ExtraResponseHeaders,
 			)
 		}
 	}
 	// From Viper, an invalid list is skipped with a warning.
 	resetViper(t)
 	logs := captureLogs(t, slog.LevelWarn)
 	viper.Set(key, []any{map[string]any{"value": "noindex"}})
 	LoadFromViper(&conf)
 	if conf.Server.Headers.ExtraResponseHeaders != nil ||
 		!strings.Contains(logs.String(), "key="+key) {
 		t.Errorf(
 			"invalid list from Viper: stored %+v, logs %s",
 			conf.Server.Headers.ExtraResponseHeaders,
 			logs,
 		)
 	}
 }
 // Invalid values from Viper are skipped with a warning, leaving the existing
 // value.
 func TestLoadFromViperInvalidValues(t *testing.T) {
 	resetViper(t)
 	logs := captureLogs(t, slog.LevelWarn)
 	viper.Set("logging.level", "shouty")
 	viper.Set("sources.sleeper.interval", "eventually")
 	conf := defaultConfig(t)
 	LoadFromViper(&conf)
 	if conf.Logging.Level != "warn" {
 		t.Errorf("Logging.Level = %q; want warn kept", conf.Logging.Level)
 	}
 	if conf.Sources.Sleeper.Interval != 10*time.Minute {
 		t.Errorf(
 			"Sleeper.Interval = %v; want 10m kept", conf.Sources.Sleeper.Interval,
 		)
 	}
 	for _, key := range []string{"logging.level", "sources.sleeper.interval"} {
 		if !strings.Contains(logs.String(), "key="+key) {
 			t.Errorf("no warning logged for %s: %s", key, logs)
 		}
 	}
 }
 func TestDatabaseSection(t *testing.T) {
 	c := JSCDatabase{
 		File:                 "./jobsearch.db",
 		ChangeBatchRetention: 48 * time.Hour,
 	}
 	keys := []string{"file", "change_batch_retention"}
 	if got := c.Keys(); !reflect.DeepEqual(got, keys) {
 		t.Errorf("Keys = %v; want %v", got, keys)
 	}
 	if got, want := c.Map(), map[string]any{
 		"file":                   "./jobsearch.db",
 		"change_batch_retention": "48h0m0s",
 	}; !reflect.DeepEqual(got, want) {
 		t.Errorf("Map = %v; want %v", got, want)
 	}
 	want := map[string]any{
 		"database.file":                   "./jobsearch.db",
 		"database.change_batch_retention": "48h0m0s",
 	}
 	if got := c.Flatten(); !reflect.DeepEqual(got, want) {
 		t.Errorf("Flatten = %v; want %v", got, want)
 	}
 	if k, err := c.TypeOf("file"); err != nil || k != reflect.String {
 		t.Errorf("TypeOf(file) = %v, %v", k, err)
 	}
 	if v, err := c.ValueOf("file"); err != nil || v != "./jobsearch.db" {
 		t.Errorf("ValueOf(file) = %v, %v", v, err)
 	}
 	if _, err := c.TypeOf("nope"); err == nil {
 		t.Error("TypeOf(nope) succeeded")
 	}
 	if _, err := c.ValueOf("nope"); err == nil {
 		t.Error("ValueOf(nope) succeeded")
 	}
 	if err := c.SetValue("file", "/var/lib/js.db"); err != nil ||
 		c.File != "/var/lib/js.db" {
 		t.Errorf("SetValue(file) = %v, File = %q", err, c.File)
 	}
 	if err := c.SetValue("file", 42); err == nil {
 		t.Error("SetValue(file, 42) succeeded")
 	}
 	// The root configuration reaches it as database.file.
 	conf := defaultConfig(t)
 	if err := conf.SetValue("database.file", "./other.db"); err != nil ||
 		conf.Database.File != "./other.db" {
 		t.Errorf("SetValue(database.file) = %v, File = %q", err, conf.Database.File)
 	}
 	// 30 days by default.
 	retention := defaultConfig(t).Database.ChangeBatchRetention
 	if retention != 720*time.Hour {
 		t.Errorf("default change_batch_retention = %v; want 720h", retention)
 	}
 	err := conf.SetValue("database.change_batch_retention", "0s")
 	if err != nil || conf.Database.ChangeBatchRetention != 0 {
 		t.Errorf(
 			"SetValue(change_batch_retention) = %v, retention = %v",
 			err, conf.Database.ChangeBatchRetention,
 		)
 	}
 }
 func TestViperValue(t *testing.T) {
 	resetViper(t)
 	viper.Set("s", "text")
 	viper.Set("i", "42") // strings are converted
 	viper.Set("b", "true")
 	viper.Set("env_list", "a, b")           // e.g. from an environment variable
 	viper.Set("file_list", []any{"a", "b"}) // e.g. from a YAML list
 	for _, tt := range []struct {
 		key  string
 		kind reflect.Kind
 		want any
 	}{
 		{"s", reflect.String, "text"},
 		{"i", reflect.Int, 42},
 		{"b", reflect.Bool, true},
 		{"env_list", reflect.Slice, "a, b"}, // split by SetValue
 		{"file_list", reflect.Slice, []string{"a", "b"}},
 	} {
 		got, ok := viperValue(tt.key, tt.kind)
 		if !ok || !reflect.DeepEqual(got, tt.want) {
 			t.Errorf(
 				"viperValue(%q, %v) = %#v, %v; want %#v",
 				tt.key,
 				tt.kind,
 				got,
 				ok,
 				tt.want,
 			)
 		}
 	}
 	unhandled := []reflect.Kind{reflect.Float64, reflect.Int8, reflect.Map}
 	for _, kind := range unhandled {
 		if got, ok := viperValue("i", kind); ok {
 			t.Errorf("viperValue(i, %v) = %#v, true; want unhandled", kind, got)
 		}
 	}
 }
 // max_message_size can't be -1 (no limit), directly or through the whole
 // configuration; 0 (the default) and explicit sizes are accepted.
 func TestWebSocketMaxMessageSizeNoLimit(t *testing.T) {
 	var c JSCServerWebSocket
 	if err := c.SetValue("max_message_size", "-1"); err == nil ||
 		!strings.Contains(err.Error(), "can't be -1") {
 		t.Errorf("SetValue(max_message_size, -1) = %v; want rejected", err)
 	}
 	if c.MaxMessageSize != 0 {
 		t.Errorf("rejected size was stored: %v", c.MaxMessageSize)
 	}
 	for _, text := range []string{"0B", "64KiB"} {
 		if err := c.SetValue("max_message_size", text); err != nil {
 			t.Errorf("SetValue(max_message_size, %s): %v", text, err)
 		}
 	}
 	conf := defaultConfig(t)
 	if err := conf.SetValue(
 		"server.websocket.max_message_size", "-1",
 	); err == nil {
 		t.Error(
 			"JobSearchConfig.SetValue(server.websocket.max_message_size, -1) succeeded",
 		)
 	}
 }
 // A source's interval must be greater than 0, through the embedded section's
 // promoted validate; a source is turned off with enabled.
 func TestSourceIntervalMustBePositive(t *testing.T) {
 	conf := defaultConfig(t)
 	for _, key := range []string{
 		"sources.find_a_tender.interval",
 		"sources.fts_mock.interval",
 		"sources.sleeper.interval",
 	} {
 		for _, value := range []string{"0s", "-1m"} {
 			err := conf.SetValue(key, value)
 			if err == nil || !strings.Contains(err.Error(), "greater than 0") {
 				t.Errorf("SetValue(%s, %s) = %v; want rejected", key, value, err)
 			}
 		}
 		if err := conf.SetValue(key, "1m"); err != nil {
 			t.Errorf("SetValue(%s, 1m): %v", key, err)
 		}
 	}
 	if conf.Sources.FindATender.Interval.Minutes() != 1 {
 		t.Errorf(
 			"FindATender.Interval = %v; want 1m", conf.Sources.FindATender.Interval,
 		)
 	}
 }
 // server.timeouts.read_header must be greater than 0; the other server
 // timeouts accept 0 (no limit).
 func TestServerReadHeaderTimeoutMustBePositive(t *testing.T) {
 	conf := defaultConfig(t)
 	for _, value := range []string{"0s", "-1s"} {
 		err := conf.SetValue("server.timeouts.read_header", value)
 		if err == nil || !strings.Contains(err.Error(), "greater than 0") {
 			t.Errorf("SetValue(read_header, %s) = %v; want rejected", value, err)
 		}
 	}
 	if conf.Server.Timeouts.ReadHeaderTimeout != 5*time.Second {
 		t.Errorf(
 			"rejected read_header was stored: %v",
 			conf.Server.Timeouts.ReadHeaderTimeout,
 		)
 	}
 	for _, key := range []string{"read", "write", "idle", "shutdown"} {
 		if err := conf.SetValue("server.timeouts."+key, "0s"); err != nil {
 			t.Errorf("SetValue(%s, 0s): %v", key, err)
 		}
 	}
 }
@@ -0,0 +1,75 @@
 package config
 import (
 	"errors"
 	"reflect"
 	"git.avsdev.uk/Go/job-search/internal/util"
 )
 // JSCKServer is the server section's configuration key.
 const JSCKServer string = "server"
 // JSCServer configures the outward server. Host is the address to listen
 // on (empty means every interface) and Port the port (0 means any free
 // port). MaxHeaderBytes is the largest request header block; 0 means the
 // default (8KB), and -1 (no limit) is rejected (see validate). MaxBodyBytes is
 // the largest request body; 0 means the default (1MB) and -1 no limit.
 // AccessLog is the file the access log is appended to, one line per request
 // in the Combined Log Format; "-" means stdout, and empty means no access
 // log.
 type JSCServer struct {
 	Host           string              `cfg:"host"             default:"127.0.0.1"`
 	Port           int                 `cfg:"port"             default:"9393"`
 	MaxHeaderBytes util.ByteSize       `cfg:"max_header_bytes" default:"8KB"`
 	MaxBodyBytes   util.ByteSize       `cfg:"max_body_bytes"   default:"1MB"`
 	AccessLog      string              `cfg:"access_log"`
 	Timeouts       JSCServerTimeouts   `cfg:"timeouts"`
 	RateLimits     JSCServerRateLimits `cfg:"rate_limits"`
 	Proxy          JSCServerProxy      `cfg:"proxy"`
 	CSRF           JSCServerCSRF       `cfg:"csrf"`
 	TLS            JSCServerTLS        `cfg:"tls"`
 	Headers        JSCServerHeaders    `cfg:"headers"`
 	Frontend       JSCServerFrontend   `cfg:"frontend"`
 	WebSocket      JSCServerWebSocket  `cfg:"websocket"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCServer) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCServer) Flatten() map[string]any {
 	return flattenOf(JSCKServer, c)
 }
 // TypeOf implements Section.
 func (c JSCServer) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCServer) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCServer) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
 // ---------- Section specific methods ----------
 // validate checks values before they are stored (see validator).
 // max_header_bytes can't be -1 (no limit): the HTTP server always limits the
 // header block, so -1 would quietly mean its own default instead.
 func (c *JSCServer) validate(key string, value any) error {
 	if key == "max_header_bytes" &&
 		value.(util.ByteSize) == util.ByteSizeNoLimit {
 		return errors.New(
 			"config: max_header_bytes can't be -1 (no limit): set an explicit size, or 0 for the default",
 		)
 	}
 	return nil
 }
@@ -0,0 +1,124 @@
 package config
 import (
 	"reflect"
 	"time"
 )
 // JSCKServerCSRF is the server CSRF section's configuration key.
 const JSCKServerCSRF string = "csrf"
 // JSCServerCSRF configures the server's CSRF protection: its cross-origin
 // check and its tokens. ExemptPaths are ServeMux-style patterns skipped by
 // both (e.g. "/api/webhooks/"); empty means none are skipped.
 type JSCServerCSRF struct {
 	CrossOriginProtection JSCServerCSRFCrossOrigin `cfg:"cross_origin_protection"`
 	Tokens                JSCServerCSRFTokens      `cfg:"tokens"`
 	ExemptPaths           []string                 `cfg:"exempt_paths"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCServerCSRF) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCServerCSRF) Flatten() map[string]any {
 	return flattenOf(JSCKServerCSRF, c)
 }
 // TypeOf implements Section.
 func (c JSCServerCSRF) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCServerCSRF) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCServerCSRF) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
 // ========== Cross-Origin Protection ==========
 // JSCKServerCSRFCrossOrigin is the server CSRF cross-origin protection
 // section's configuration key.
 const JSCKServerCSRFCrossOrigin string = "cross_origin_protection"
 // JSCServerCSRFCrossOrigin configures CSRF protection by checking requests'
 // origin. TrustedOrigins are extra origins ("https://admin.example.com")
 // allowed to make cross-origin unsafe requests; empty means none.
 type JSCServerCSRFCrossOrigin struct {
 	Enabled        bool     `cfg:"enabled"         default:"true"`
 	TrustedOrigins []string `cfg:"trusted_origins"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCServerCSRFCrossOrigin) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCServerCSRFCrossOrigin) Flatten() map[string]any {
 	return flattenOf(JSCKServerCSRFCrossOrigin, c)
 }
 // TypeOf implements Section.
 func (c JSCServerCSRFCrossOrigin) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCServerCSRFCrossOrigin) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCServerCSRFCrossOrigin) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
 // ========== CSRF Tokens ==========
 // JSCKServerCSRFTokens is the server CSRF tokens section's configuration key.
 const JSCKServerCSRFTokens string = "tokens"
 // JSCServerCSRFTokens configures CSRF protection by tokens. Key is the hex
 // HMAC key tokens are signed with; empty means a random key for the process,
 // so tokens don't survive a restart. An empty CookieName or HeaderName, or a
 // TokenTTL of 0 or less, means the default.
 type JSCServerCSRFTokens struct {
 	Enabled    bool          `cfg:"enabled"     default:"true"`
 	Key        string        `cfg:"key"`
 	CookieName string        `cfg:"cookie_name" default:"js_csrf"`
 	HeaderName string        `cfg:"header_name" default:"X-CSRF-Token"`
 	TokenTTL   time.Duration `cfg:"token_ttl"   default:"12h"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCServerCSRFTokens) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCServerCSRFTokens) Flatten() map[string]any {
 	return flattenOf(JSCKServerCSRFTokens, c)
 }
 // TypeOf implements Section.
 func (c JSCServerCSRFTokens) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCServerCSRFTokens) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCServerCSRFTokens) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
@@ -0,0 +1,45 @@
 package config
 import (
 	"reflect"
 )
 // JSCKServerFrontend is the frontend section's configuration key.
 const JSCKServerFrontend string = "frontend"
 // JSCServerFrontend configures how the server serves the frontend. Dir is
 // the directory of the frontend's build output (holding index.html); empty
 // means the build embedded in the binary, or no frontend if it has none.
 // ImmutablePrefix is where Vite emits content-hashed files, cached for a
 // year; empty means "/assets/". IndexCacheControl is the Cache-Control of
 // HTML and other files without a hash; empty means "no-cache".
 type JSCServerFrontend struct {
 	Dir               string `cfg:"dir"`
 	ImmutablePrefix   string `cfg:"immutable_prefix"`
 	IndexCacheControl string `cfg:"index_cache_control"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCServerFrontend) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCServerFrontend) Flatten() map[string]any {
 	return flattenOf(JSCKServerFrontend, c)
 }
 // TypeOf implements Section.
 func (c JSCServerFrontend) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCServerFrontend) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCServerFrontend) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
@@ -0,0 +1,176 @@
 package config
 import (
 	"errors"
 	"reflect"
 	"strings"
 	"time"
 )
 // JSCKServerHeaders is the server headers section's configuration key.
 const JSCKServerHeaders string = "headers"
 // JSCServerHeaders configures the server's request checks and security
 // response headers. AllowedHosts restricts the Host header, and supports
 // "*.example.com"; empty disables the check. Each policy header is sent with
 // its configured value; empty means it isn't sent. ExtraResponseHeaders are
 // set on every response, and RemoveResponseHeaders stripped from every
 // response before it is sent; empty means none.
 type JSCServerHeaders struct {
 	AllowedHosts []string `cfg:"allowed_hosts"`
 	ContentSecurityPolicy     string               `cfg:"content_security_policy"`
 	FrameOptions              string               `cfg:"frame_options"`
 	ReferrerPolicy            string               `cfg:"referrer_policy"`
 	PermissionsPolicy         string               `cfg:"permissions_policy"`
 	CrossOriginOpenerPolicy   string               `cfg:"cross_origin_opener_policy"`
 	CrossOriginResourcePolicy string               `cfg:"cross_origin_resource_policy"`
 	HSTS                      JSCServerHeadersHSTS `cfg:"hsts"`
 	ExtraResponseHeaders  []JSCServerHeadersExtra `cfg:"extra_response_headers"`
 	RemoveResponseHeaders []string                `cfg:"remove_response_headers"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCServerHeaders) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCServerHeaders) Flatten() map[string]any {
 	return flattenOf(JSCKServerHeaders, c)
 }
 // TypeOf implements Section.
 func (c JSCServerHeaders) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCServerHeaders) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCServerHeaders) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
 // ========== HSTS Headers ==========
 // JSCKServerHeadersHSTS is the HSTS section's configuration key.
 const JSCKServerHeadersHSTS string = "hsts"
 // JSCServerHeadersHSTS configures the Strict-Transport-Security header, sent
 // on HTTPS requests when enabled. A MaxAge of 0 or less means the header
 // isn't sent.
 type JSCServerHeadersHSTS struct {
 	Enabled           bool          `cfg:"enabled"            default:"false"`
 	MaxAge            time.Duration `cfg:"max_age"            default:"8760h"`
 	IncludeSubdomains bool          `cfg:"include_subdomains" default:"true"`
 	Preload           bool          `cfg:"preload"            default:"false"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCServerHeadersHSTS) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCServerHeadersHSTS) Flatten() map[string]any {
 	return flattenOf(JSCKServerHeadersHSTS, c)
 }
 // TypeOf implements Section.
 func (c JSCServerHeadersHSTS) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCServerHeadersHSTS) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCServerHeadersHSTS) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
 // ========== Extra Headers ==========
 // JSCServerHeadersExtra configures one extra response header. It is used as
 // the items of the extra_response_headers list, so it has no JSCK* constant
 // and its Flatten keys are not prefixed. Header must be a valid header name,
 // and Value must not hold a line break (see validate); an empty Value means
 // the header isn't sent.
 type JSCServerHeadersExtra struct {
 	Header string `cfg:"header"`
 	Value  string `cfg:"value"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCServerHeadersExtra) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCServerHeadersExtra) Flatten() map[string]any {
 	return flattenOf("", c)
 }
 // TypeOf implements Section.
 func (c JSCServerHeadersExtra) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCServerHeadersExtra) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCServerHeadersExtra) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
 // ---------- Section specific methods ----------
 // validate checks values before they are stored (see validator). header must
 // be a valid header field name (an RFC 9110 token) and value must not hold a
 // line break, so neither can break the response's header block.
 func (c *JSCServerHeadersExtra) validate(key string, value any) error {
 	switch key {
 	case "header":
 		if !isHeaderToken(value.(string)) {
 			return errors.New(
 				"config: invalid header name: must be a non-empty token such as X-Robots-Tag",
 			)
 		}
 	case "value":
 		if strings.ContainsAny(value.(string), "\r\n") {
 			return errors.New(
 				"config: invalid header value: must not contain a line break",
 			)
 		}
 	}
 	return nil
 }
 // isHeaderToken reports whether name is a non-empty RFC 9110 token, the syntax
 // of a header field name.
 func isHeaderToken(name string) bool {
 	if name == "" {
 		return false
 	}
 	for _, char := range name {
 		switch {
 		case char >= 'a' && char <= 'z',
 			char >= 'A' && char <= 'Z',
 			char >= '0' && char <= '9':
 		case strings.ContainsRune("!#$%&'*+-.^_`|~", char):
 		default:
 			return false
 		}
 	}
 	return true
 }
@@ -0,0 +1,88 @@
 package config
 import (
 	"reflect"
 	"time"
 )
 // JSCKServerProxy is the server proxy section's configuration key.
 const JSCKServerProxy string = "proxy"
 // JSCServerProxy configures which proxies the server trusts and how it finds
 // the client's address behind them. TrustedProxies are IP addresses and CIDR
 // ranges; empty means no proxy is trusted. ClientIPHeaders are the headers
 // read for the client's address from a trusted proxy, in priority order;
 // empty means the connection's own address is always used.
 type JSCServerProxy struct {
 	TrustedProxies  []string               `cfg:"trusted"           default:"127.0.0.1/32"`
 	ClientIPHeaders []string               `cfg:"client_ip_headers" default:"X-Forwarded-For, Forwarded, X-Real-IP"`
 	StripUntrusted  bool                   `cfg:"strip_untrusted"   default:"true"`
 	ProxyProtocol   JSCServerProxyProtocol `cfg:"proxy_protocol"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCServerProxy) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCServerProxy) Flatten() map[string]any {
 	return flattenOf(JSCKServerProxy, c)
 }
 // TypeOf implements Section.
 func (c JSCServerProxy) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCServerProxy) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCServerProxy) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
 // ========== PROXY protocol ==========
 // JSCKServerProxyProtocol is the server proxy PROXY protocol section's
 // configuration key.
 const JSCKServerProxyProtocol string = "proxy_protocol"
 // JSCServerProxyProtocol configures accepting the PROXY protocol from
 // trusted proxies. TrustedSources may send PROXY headers; empty means
 // Proxy.TrustedProxies. ReadHeaderTimeout is how long to wait for the PROXY
 // header; 0 or less means no limit.
 type JSCServerProxyProtocol struct {
 	Enabled           bool          `cfg:"enabled"             default:"false"`
 	TrustedSources    []string      `cfg:"trusted_sources"`
 	Required          bool          `cfg:"required"            default:"false"`
 	ReadHeaderTimeout time.Duration `cfg:"read_header_timeout" default:"5s"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCServerProxyProtocol) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCServerProxyProtocol) Flatten() map[string]any {
 	return flattenOf(JSCKServerProxyProtocol, c)
 }
 // TypeOf implements Section.
 func (c JSCServerProxyProtocol) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCServerProxyProtocol) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCServerProxyProtocol) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
@@ -0,0 +1,44 @@
 package config
 import (
 	"reflect"
 	"time"
 )
 // JSCKServerRateLimits is the server rate limits section's configuration key.
 const JSCKServerRateLimits string = "rate_limits"
 // JSCServerRateLimits configures the server's rate limits. Global limits
 // every request from each client, and API each client's API requests (see
 // JSCRateLimitRule). IdleTTL is how long a client's limiter is kept after its
 // last request; 0 or less means limiters are never evicted.
 type JSCServerRateLimits struct {
 	Global  JSCRateLimitRule `cfg:"global"`
 	API     JSCRateLimitRule `cfg:"api"`
 	IdleTTL time.Duration    `cfg:"idle_ttl" default:"10m"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCServerRateLimits) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCServerRateLimits) Flatten() map[string]any {
 	return flattenOf(JSCKServerRateLimits, c)
 }
 // TypeOf implements Section.
 func (c JSCServerRateLimits) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCServerRateLimits) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCServerRateLimits) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
@@ -0,0 +1,64 @@
 package config
 import (
 	"errors"
 	"reflect"
 	"time"
 )
 // JSCKServerTimeouts is the server timeouts section's configuration key.
 const JSCKServerTimeouts string = "timeouts"
 // JSCServerTimeouts configures the server's HTTP timeouts, as those of
 // http.Server: ReadHeaderTimeout for reading a request's headers,
 // ReadTimeout for the whole request, WriteTimeout for writing the response
 // and IdleTimeout for waiting for the next request on a keep-alive
 // connection. 0 or less means no limit, except that ReadHeaderTimeout must be
 // greater than 0 (see validate). ShutdownTimeout is how long shutdown waits
 // for clients to close; 0 or less means it waits until they have.
 type JSCServerTimeouts struct {
 	ReadHeaderTimeout time.Duration `cfg:"read_header" default:"5s"`
 	ReadTimeout       time.Duration `cfg:"read"        default:"15s"`
 	WriteTimeout      time.Duration `cfg:"write"       default:"30s"`
 	IdleTimeout       time.Duration `cfg:"idle"        default:"2m"`
 	ShutdownTimeout   time.Duration `cfg:"shutdown"    default:"15s"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCServerTimeouts) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCServerTimeouts) Flatten() map[string]any {
 	return flattenOf(JSCKServerTimeouts, c)
 }
 // TypeOf implements Section.
 func (c JSCServerTimeouts) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCServerTimeouts) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCServerTimeouts) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
 // ---------- Section specific methods ----------
 // validate checks values before they are stored (see validator).
 // read_header must be greater than 0: without it a client could hold a
 // connection open for ever by sending its headers slowly (slowloris).
 func (c *JSCServerTimeouts) validate(key string, value any) error {
 	if key == "read_header" && value.(time.Duration) <= 0 {
 		return errors.New(
 			"config: read_header must be greater than 0, as a client could otherwise hold a connection open for ever",
 		)
 	}
 	return nil
 }
@@ -0,0 +1,46 @@
 package config
 import (
 	"reflect"
 )
 // JSCKServerTLS is the TLS section's configuration key.
 const JSCKServerTLS string = "tls"
 // JSCServerTLS configures the server's TLS connections. CertFile and KeyFile
 // must be set when TLS is enabled. MinVersion is "1.2" or "1.3"; anything
 // else, including empty, means 1.2. RedirectHTTPPort, if set (e.g. 80),
 // starts a plain HTTP listener on that port that redirects everything to
 // HTTPS; 0 means none.
 type JSCServerTLS struct {
 	Enabled          bool   `cfg:"enabled"            default:"false"`
 	CertFile         string `cfg:"cert_file"`
 	KeyFile          string `cfg:"key_file"`
 	MinVersion       string `cfg:"min_version"        default:"1.3"`
 	RedirectHTTPPort int    `cfg:"redirect_http_port"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCServerTLS) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCServerTLS) Flatten() map[string]any {
 	return flattenOf(JSCKServerTLS, c)
 }
 // TypeOf implements Section.
 func (c JSCServerTLS) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCServerTLS) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCServerTLS) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
@@ -0,0 +1,68 @@
 package config
 import (
 	"errors"
 	"reflect"
 	"time"
 	"git.avsdev.uk/Go/job-search/internal/util"
 )
 // JSCKServerWebSocket is the websocket section's configuration key.
 const JSCKServerWebSocket string = "websocket"
 // JSCServerWebSocket configures the server's websocket connections. A
 // browser's upgrade must come from the server's own origin or one of
 // csrf.cross_origin_protection.trusted_origins. MaxMessageSize is the
 // largest message read from a client; 0 means the default (32KiB), and -1
 // (no limit) is rejected (see validate). A WriteTimeout or PongTimeout of 0
 // or less means no limit, and a PingInterval of 0 or less turns pings off.
 type JSCServerWebSocket struct {
 	EnableCompression bool             `cfg:"enable_compression" default:"false"`
 	MaxMessageSize    util.ByteSize    `cfg:"max_message_size"   default:"32KiB"`
 	ConnectionLimit   JSCRateLimitRule `cfg:"connection_limit"`
 	MessageRateLimit  JSCRateLimitRule `cfg:"message_rate_limit"`
 	WriteTimeout      time.Duration    `cfg:"write_timeout"      default:"10s"`
 	PingInterval      time.Duration    `cfg:"ping_interval"      default:"30s"`
 	PongTimeout       time.Duration    `cfg:"pong_timeout"       default:"10s"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCServerWebSocket) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCServerWebSocket) Flatten() map[string]any {
 	return flattenOf(JSCKServerWebSocket, c)
 }
 // TypeOf implements Section.
 func (c JSCServerWebSocket) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCServerWebSocket) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCServerWebSocket) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
 // ---------- Section specific methods ----------
 // validate checks values before they are stored (see validator).
 // max_message_size can't be -1 (no limit): a client could then send a
 // message of any size, and the server holds each message in memory.
 func (c *JSCServerWebSocket) validate(key string, value any) error {
 	if key == "max_message_size" &&
 		value.(util.ByteSize) == util.ByteSizeNoLimit {
 		return errors.New(
 			"config: max_message_size can't be -1 (no limit): set an explicit size, or 0 for the default",
 		)
 	}
 	return nil
 }
@@ -0,0 +1,62 @@
 package config
 import (
 	"errors"
 	"reflect"
 	"time"
 )
 // JSCKSourceEUSupply is the EU Supply source's configuration key.
 const JSCKSourceEUSupply string = "eu_supply"
 // JSCSourceEUSupply configures scraping the public tenders on EU Supply's
 // UK portal at BaseURL (see internal/eusupply). The client waits a random
 // time from RequestMinGap to RequestMaxGap after each request; both must be
 // greater than 0, as the site's own limit isn't published, and a
 // RequestMaxGap less than RequestMinGap is read as RequestMinGap.
 type JSCSourceEUSupply struct {
 	JSCSourcesSource
 	BaseURL       string        `cfg:"base_url"        default:"https://uk.eu-supply.com"`
 	RequestMinGap time.Duration `cfg:"request_min_gap" default:"2s"`
 	RequestMaxGap time.Duration `cfg:"request_max_gap" default:"10s"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCSourceEUSupply) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCSourceEUSupply) Flatten() map[string]any {
 	return flattenOf(JSCKSourceEUSupply, c)
 }
 // TypeOf implements Section.
 func (c JSCSourceEUSupply) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCSourceEUSupply) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCSourceEUSupply) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
 // ---------- Section specific methods ----------
 // validate checks values before they are stored (see validator), as
 // JSCSourcesSource's does, which this one hides. request_min_gap and
 // request_max_gap must be greater than 0.
 func (c *JSCSourceEUSupply) validate(key string, value any) error {
 	if (key == "request_min_gap" || key == "request_max_gap") &&
 		value.(time.Duration) <= 0 {
 		return errors.New(
 			"config: request_min_gap and request_max_gap must be greater than 0, as EU Supply publishes no rate limit",
 		)
 	}
 	return c.JSCSourcesSource.validate(key, value)
 }
@@ -0,0 +1,51 @@
 package config
 import "reflect"
 // JSCKSourceFindATender is the Find a Tender source's configuration key.
 const JSCKSourceFindATender string = "find_a_tender"
 // JSCSourceFindATender configures fetching from the Find a Tender
 // OCDS API at APIRoot. Empty: the source fails to start if it is enabled,
 // and `fetcher fts-once` fails. RateLimitRetries is how many times a request
 // refused for the API's rate limit is waited out and retried; 0 fails at the
 // first, and it can't be negative.
 type JSCSourceFindATender struct {
 	JSCSourcesSource
 	APIRoot          string `cfg:"api_root"           default:"https://www.find-tender.service.gov.uk/api/1.0"`
 	RateLimitRetries int    `cfg:"rate_limit_retries" default:"5"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCSourceFindATender) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCSourceFindATender) Flatten() map[string]any {
 	return flattenOf(JSCKSourceFindATender, c)
 }
 // TypeOf implements Section.
 func (c JSCSourceFindATender) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCSourceFindATender) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCSourceFindATender) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
 // ---------- Section specific methods ----------
 // validate checks values before they are stored (see validator), as
 // JSCSourcesSource's does, which this one hides. rate_limit_retries can't be
 // negative.
 func (c *JSCSourceFindATender) validate(key string, value any) error {
 	return validateRateLimitRetries(&c.JSCSourcesSource, key, value)
 }
@@ -0,0 +1,50 @@
 package config
 import "reflect"
 // JSCKSourceFTSMock is the Find a Tender mock source's configuration key.
 const JSCKSourceFTSMock string = "fts_mock"
 // JSCSourceFTSMock configures fetching from a Find a Tender mock
 // server (cmd/mock-fts) at APIRoot, which has no default. Empty: the source
 // fails to start if it is enabled. RateLimitRetries is as for
 // JSCSourceFindATender.
 type JSCSourceFTSMock struct {
 	JSCSourcesSource
 	APIRoot          string `cfg:"api_root"`
 	RateLimitRetries int    `cfg:"rate_limit_retries" default:"5"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCSourceFTSMock) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCSourceFTSMock) Flatten() map[string]any {
 	return flattenOf(JSCKSourceFTSMock, c)
 }
 // TypeOf implements Section.
 func (c JSCSourceFTSMock) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCSourceFTSMock) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCSourceFTSMock) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
 // ---------- Section specific methods ----------
 // validate checks values before they are stored (see validator), as
 // JSCSourcesSource's does, which this one hides. rate_limit_retries can't be
 // negative.
 func (c *JSCSourceFTSMock) validate(key string, value any) error {
 	return validateRateLimitRetries(&c.JSCSourcesSource, key, value)
 }
@@ -0,0 +1,39 @@
 package config
 import "reflect"
 // JSCKSourceSleeper is the sleeper source's configuration key.
 const JSCKSourceSleeper string = "sleeper"
 // JSCSourceSleeper configures the sleeper, a test source that
 // simulates work by sleeping. With BlockShutdown its work ignores
 // cancellation, to test shutdown waiting on a non-cooperative task.
 type JSCSourceSleeper struct {
 	JSCSourcesSource
 	BlockShutdown bool `cfg:"block_shutdown"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCSourceSleeper) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCSourceSleeper) Flatten() map[string]any {
 	return flattenOf(JSCKSourceSleeper, c)
 }
 // TypeOf implements Section.
 func (c JSCSourceSleeper) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCSourceSleeper) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCSourceSleeper) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
@@ -0,0 +1,116 @@
 package config
 import (
 	"errors"
 	"reflect"
 	"time"
 )
 // ========== Sources ==========
 // JSCKSources is the sources section's configuration key.
 const JSCKSources string = "sources"
 // JSCSources configures each source of opportunities, one field
 // per source. Each field's cfg tag must match the section's JSCK* constant.
 // The test sources (FTSMock and Sleeper) are tagged init:"hidden", so
 // `config init` only writes them with --all.
 // SkipRulesFile is the JSON file of rules that mark new opportunities
 // Skipped (see internal/skiprules), read again when it changes; empty means
 // none.
 type JSCSources struct {
 	SkipRulesFile string               `cfg:"skip_rules_file"`
 	FindATender   JSCSourceFindATender `cfg:"find_a_tender"`
 	FTSMock       JSCSourceFTSMock     `cfg:"fts_mock"        init:"hidden"`
 	EUSupply      JSCSourceEUSupply    `cfg:"eu_supply"`
 	Sleeper       JSCSourceSleeper     `cfg:"sleeper"         init:"hidden"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCSources) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCSources) Flatten() map[string]any {
 	return flattenOf(JSCKSources, c)
 }
 // TypeOf implements Section.
 func (c JSCSources) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCSources) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCSources) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
 // ========== Generic source ==========
 // JSCSourcesSource holds the settings every source has, and is embedded
 // in each source's section. Interval is how often an enabled source fetches;
 // it must be greater than 0 (validate rejects 0 or less: a source is turned
 // off with Enabled, not with its interval).
 type JSCSourcesSource struct {
 	Enabled  bool          `cfg:"enabled"`
 	Interval time.Duration `cfg:"interval" default:"10m"`
 }
 // ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JSCSourcesSource) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JSCSourcesSource) Flatten() map[string]any {
 	return flattenOf("", c)
 }
 // TypeOf implements Section.
 func (c JSCSourcesSource) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 // ValueOf implements Section.
 func (c JSCSourcesSource) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 // SetValue implements Section.
 func (c *JSCSourcesSource) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
 // Keys returns the keys directly in the section.
 func (c *JSCSourcesSource) Keys() []string { return keysOf(c) }
 // validate checks values before they are stored (see validator); it is
 // promoted to each source's section. interval must be greater than 0.
 func (c *JSCSourcesSource) validate(key string, value any) error {
 	if key == "interval" && value.(time.Duration) <= 0 {
 		return errors.New(
 			"config: interval must be greater than 0; set enabled to false to turn the source off",
 		)
 	}
 	return nil
 }
 // validateRateLimitRetries is the validate of a Find a Tender source's
 // section: source's checks, then rate_limit_retries, which can't be
 // negative.
 func validateRateLimitRetries(
 	source *JSCSourcesSource, key string, value any,
 ) error {
 	if key == "rate_limit_retries" && value.(int) < 0 {
 		return errors.New(
 			"config: rate_limit_retries can't be negative; 0 fails at the first rate limit",
 		)
 	}
 	return source.validate(key, value)
 }
@@ -1,114 +1,179 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 package config
 import (
-	"errors"
+	"encoding/json"
 	"fmt"
 	"log/slog"
 	"maps"
 	"reflect"
 	"slices"
 	"strings"
 	"github.com/spf13/viper"
 	"go.yaml.in/yaml/v3"
 )
 // EnvPrefix prefixes environment variables that override configuration keys,
 // e.g. JOBSEARCH_LOGGING_LEVEL for logging.level.
 const EnvPrefix string = "JOBSEARCH"
 // EtcConfigPath is searched for config.* when no config file is given, before
 // the working directory.
 const EtcConfigPath string = "/etc/job-search"
 // JobSearchConfig is the application's whole configuration, one field per
 // section. Each field's cfg tag must match the section's JSCK* constant.
 type JobSearchConfig struct {
-	Logging JobSearchConfigLogging
+	Database JSCDatabase `cfg:"database"`
 	Logging  JSCLogging  `cfg:"logging"`
 	Server   JSCServer   `cfg:"server"`
 	Sources  JSCSources  `cfg:"sources"`
 }
-// Default configuration
+// LoadDefaultConfig sets Viper's defaults from the configuration's default
 // struct tags.
 func LoadDefaultConfig() {
-	conf := JobSearchConfig{
+	var conf JobSearchConfig
-		Logging: defaultConfigLogging(),
+	if err := applyDefaults(&conf); err != nil {
 		panic(fmt.Sprintf("config: %v", err))
 	}
 	LoadToViper(conf)
 }
-// Converts configuration to a map
+// InitDefaults returns the default configuration as dotted keys, as
-func (conf JobSearchConfig) Map() map[string]any {
+// Flatten gives it, for `config init` to write. Sections tagged
-	confMap := map[string]any{}
+// init:"hidden" (the test sources) are left out unless includeHidden is set.
-	confMap["logging"] = conf.Logging.Map()
+func InitDefaults(includeHidden bool) map[string]any {
-	return confMap
+	var conf JobSearchConfig
 	if err := applyDefaults(&conf); err != nil {
 		panic(fmt.Sprintf("config: %v", err))
 	}
 	flat := conf.Flatten()
 	if includeHidden {
 		return flat
 	}
-// Converts configuration to a flat dotted format
+	hidden := hiddenFromInitKeys("", reflect.TypeFor[JobSearchConfig]())
-func (conf JobSearchConfig) Flatten() map[string]any {
+	for _, section := range hidden {
-	confFlat := map[string]any{}
+		for key := range flat {
-	maps.Copy(confFlat, conf.Logging.Flatten())
+			if strings.HasPrefix(key, section+".") {
-	return confFlat
+				delete(flat, key)
 			}
 		}
 	}
 	return flat
 }
-// Returns the expected type of a configuration key value
+// ---------- Section methods (see Section) ----------
 // Map implements Section.
 func (c JobSearchConfig) Map() map[string]any { return mapOf(c) }
 // Flatten implements Section.
 func (c JobSearchConfig) Flatten() map[string]any { return flattenOf("", c) }
 // TypeOf implements Section.
 func (c JobSearchConfig) TypeOf(key string) (reflect.Kind, error) {
-	keyParts := strings.Split(key, ".")
+	return typeOf(c, key)
 	switch keyParts[0] {
 	case "logging":
 		return c.Logging.TypeOf(strings.Join(keyParts[1:], "."))
 	default:
 		return reflect.Invalid, errors.New("unknown configuration key")
 	}
 }
-// Returns the expected type of a configuration value
+// ValueOf implements Section.
 func (c JobSearchConfig) ValueOf(key string) (any, error) {
-	keyParts := strings.Split(key, ".")
+	return valueOf(c, key)
 	switch keyParts[0] {
 	case "logging":
 		return c.Logging.ValueOf(strings.Join(keyParts[1:], "."))
 	default:
 		return nil, errors.New("unknown configuration key")
 	}
 }
-// Returns the expected type of a configuration value
+// SetValue implements Section.
 func (c *JobSearchConfig) SetValue(key string, value any) error {
-	keyParts := strings.Split(key, ".")
+	return setValue(c, key, value)
 	switch keyParts[0] {
 	case "logging":
 		return c.Logging.SetValue(strings.Join(keyParts[1:], "."), value)
 	default:
 		return errors.New("unknown configuration key")
 	}
 }
 // LoadFromViper copies every key Viper knows that conf also has into conf.
 // Unknown keys are skipped; invalid values are logged and ignored.
 func LoadFromViper(conf *JobSearchConfig) {
 	keys := viper.AllKeys()
-	for idx := range keys {
+	for i := range keys {
-		key := keys[idx]
+		key := keys[i]
 		keyType, _ := conf.TypeOf(key)
 		if keyType == reflect.Invalid {
 			continue
 		}
-		switch keyType {
+		value, ok := viperValue(key, keyType)
-		case reflect.String:
+		if !ok {
-			conf.SetValue(key, viper.GetString(key))
+			slog.Error(
-		case reflect.Int:
+				"unknown config key type",
-			conf.SetValue(key, viper.GetInt(key))
+				"key", key,
-		case reflect.Bool:
+				"kind", keyType.String(),
-			conf.SetValue(key, viper.GetBool(key))
+			)
 			continue
 		}
 		if err := conf.SetValue(key, value); err != nil {
 			slog.Warn("ignoring invalid config value", "key", key, "error", err)
 		}
 	}
 }
 // viperValue reads key from Viper as a value of kind. Its bool is false for a
 // kind LoadFromViper doesn't handle.
 func viperValue(key string, kind reflect.Kind) (any, bool) {
 	switch kind {
 	case reflect.String:
 		return viper.GetString(key), true
 	case reflect.Int:
 		return viper.GetInt(key), true
 	case reflect.Bool:
 		return viper.GetBool(key), true
 	case reflect.Slice:
 		// A string, e.g. from an environment variable, is passed on for
 		// SetValue to split on commas; GetStringSlice would split it on
 		// spaces. A list of objects (a list of sections) is passed on as it
 		// is, as GetStringSlice would flatten it.
 		switch value := viper.Get(key).(type) {
 		case string:
 			return value, true
 		case []map[string]any: // as LoadToViper sets it
 			return value, true
 		case []any:
 			if len(value) > 0 {
 				if _, ok := value[0].(map[string]any); ok {
 					return value, true
 				}
 			}
 		}
 		return viper.GetStringSlice(key), true
 	}
 	return nil, false
 }
 // LoadToViper sets every key of conf as a Viper default.
 func LoadToViper(conf JobSearchConfig) {
 	flatMap := conf.Flatten()
 	for key := range flatMap {
 		viper.SetDefault(key, flatMap[key])
 	}
 }
 // String formats the configuration one "key = value <type>" line per key, with
 // dotted keys, sorted by key.
 func (c JobSearchConfig) String() string {
 	var builder strings.Builder
 	flat := c.Flatten()
 	for _, key := range slices.Sorted(maps.Keys(flat)) {
 		kind, _ := c.TypeOf(key)
 		fmt.Fprint(&builder, key, " = ", flat[key], " <", kind, ">\n")
 	}
 	return builder.String()
 }
 // JSON formats the configuration as indented JSON.
 func (c JobSearchConfig) JSON() string {
 	confMap := c.Map()
 	encoded, _ := json.MarshalIndent(confMap, "", "  ")
 	return string(encoded)
 }
 // YAML formats the configuration as YAML.
 func (c JobSearchConfig) YAML() string {
 	confMap := c.Map()
 	encoded, _ := yaml.Marshal(confMap)
 	return string(encoded)
 }
@@ -0,0 +1,243 @@
 package config
 import (
 	"fmt"
 	"maps"
 	"reflect"
 	"strings"
 	"testing"
 )
 // Kinds that LoadFromViper knows how to read.
 var viperKinds = map[reflect.Kind]bool{
 	reflect.String: true,
 	reflect.Int:    true,
 	reflect.Bool:   true,
 	reflect.Slice:  true,
 }
 // Every key round-trips through TypeOf/ValueOf/SetValue and has a type that
 // LoadFromViper can read.
 func TestKeysRoundTrip(t *testing.T) {
 	var conf JobSearchConfig
 	if err := applyDefaults(&conf); err != nil {
 		t.Fatalf("defaults: %v", err)
 	}
 	for key, want := range conf.Flatten() {
 		kind, err := conf.TypeOf(key)
 		if err != nil {
 			t.Errorf("TypeOf(%q): %v", key, err)
 		} else if !viperKinds[kind] {
 			t.Errorf(
 				"TypeOf(%q) = %v, which LoadFromViper does not handle", key, kind,
 			)
 		}
 		got, err := conf.ValueOf(key)
 		if err != nil || !reflect.DeepEqual(got, want) {
 			t.Errorf("ValueOf(%q) = %#v, %v; want %#v", key, got, err, want)
 		}
 		if err := conf.SetValue(key, got); err != nil {
 			t.Errorf("SetValue(%q, %#v): %v", key, got, err)
 		}
 	}
 }
 // sectionType is the Section interface type.
 var sectionType = reflect.TypeOf((*Section)(nil)).Elem()
 // Every section, including embedded ones, implements Section, and its
 // Flatten keys (which use the section's JSCK* constant) line up with its cfg
 // tag in the parent. A section type used under more than one key anywhere in
 // the config, such as JSCRateLimitRule, has no constant, so its keys are
 // unprefixed. Each section's Map, TypeOf, ValueOf and SetValue agree with the
 // whole configuration's view of the same keys, and so do those of each item
 // in a list of sections. Sections are found by walking JobSearchConfig, so a
 // new section is checked without being listed anywhere.
 func TestSections(t *testing.T) {
 	conf := defaultConfig(t)
 	// An item in each list of sections, so the items' methods are checked.
 	conf.Server.Headers.ExtraResponseHeaders = []JSCServerHeadersExtra{
 		{
 			Header: "X-Robots-Tag",
 			Value:  "noindex",
 		},
 	}
 	if !reflect.PointerTo(reflect.TypeOf(conf)).Implements(sectionType) {
 		t.Error("*JobSearchConfig does not implement Section")
 	}
 	all := conf.Flatten()
 	uses := map[reflect.Type]int{}
 	countSections(uses, reflect.TypeOf(conf))
 	checkSections(t, &conf, all, uses, "", reflect.ValueOf(&conf).Elem())
 }
 // countSections counts how many fields of each section type there are in the
 // tree under section.
 func countSections(uses map[reflect.Type]int, section reflect.Type) {
 	for _, f := range fieldsOf(section) {
 		if f.section {
 			uses[f.fieldType]++
 			countSections(uses, f.fieldType)
 		}
 	}
 }
 func checkSections(
 	t *testing.T,
 	conf *JobSearchConfig,
 	all map[string]any,
 	uses map[reflect.Type]int,
 	prefix string,
 	section reflect.Value,
 ) {
 	t.Helper()
 	for _, f := range fieldsOf(section.Type()) {
 		key := joinKey(prefix, f.key)
 		if isSectionList(f.fieldType) {
 			checkSectionList(t, all, key, section.FieldByIndex(f.index))
 		}
 		if !f.section {
 			continue
 		}
 		fv := section.FieldByIndex(f.index)
 		name := fv.Type().Name()
 		if !fv.Addr().Type().Implements(sectionType) {
 			t.Errorf("*%s does not implement Section", name)
 			continue
 		}
 		checkEmbedded(t, fv.Type())
 		keys := fv.Addr().Interface().(Section).Flatten()
 		keyPrefix := prefix
 		if uses[f.fieldType] > 1 {
 			keyPrefix = key
 		}
 		for k := range keys {
 			if _, ok := all[joinKey(keyPrefix, k)]; !ok {
 				t.Errorf(
 					"%s.Flatten() key %q does not match its cfg tag %q", name, k, key,
 				)
 			}
 		}
 		// The section's own keys, relative to it, with the whole
 		// configuration's values and kinds for them.
 		want := map[string]any{}
 		kinds := map[string]reflect.Kind{}
 		for k, v := range all {
 			if rel, ok := strings.CutPrefix(k, key+"."); ok {
 				want[rel] = v
 				kinds[rel], _ = conf.TypeOf(k)
 			}
 		}
 		checkSectionMethods(t, name, fv, want, kinds)
 		checkSections(t, conf, all, uses, key, fv)
 	}
 }
 // checkSectionList checks the items of the list of sections at key implement
 // Section and that their methods agree with the list's value in all.
 func checkSectionList(
 	t *testing.T, all map[string]any, key string, list reflect.Value,
 ) {
 	t.Helper()
 	name := list.Type().Elem().Name()
 	if !reflect.PointerTo(list.Type().Elem()).Implements(sectionType) {
 		t.Errorf("*%s (items of %s) does not implement Section", name, key)
 		return
 	}
 	items, _ := all[key].([]map[string]any)
 	if len(items) != list.Len() {
 		t.Errorf("%s has %d items; Flatten gives %d", key, list.Len(), len(items))
 		return
 	}
 	for i := range list.Len() {
 		item, itemName := list.Index(i), fmt.Sprintf("%s[%d]", key, i)
 		want := flattenMap("", items[i])
 		// Items are not prefixed, and their kinds are their fields' kinds.
 		got := item.Addr().Interface().(Section).Flatten()
 		if !reflect.DeepEqual(got, want) {
 			t.Errorf("%s.Flatten() =\n%v\nwant\n%v", itemName, got, want)
 		}
 		kinds := map[string]reflect.Kind{}
 		for k := range want {
 			_, f, _ := lookup(item, k)
 			kinds[k] = kindOf(f.fieldType)
 		}
 		checkSectionMethods(t, itemName, item, want, kinds)
 	}
 }
 // checkSectionMethods checks section's Map, TypeOf, ValueOf and SetValue
 // against want, its keys and values, and kinds, their kinds.
 // Setting each key to its own value must succeed and leave section unchanged.
 func checkSectionMethods(
 	t *testing.T,
 	name string,
 	section reflect.Value,
 	want map[string]any,
 	kinds map[string]reflect.Kind,
 ) {
 	t.Helper()
 	s := section.Addr().Interface().(Section)
 	if got := flattenMap("", s.Map()); !reflect.DeepEqual(got, want) {
 		t.Errorf("%s.Map() =\n%v\nwant\n%v", name, got, want)
 	}
 	before := section.Interface()
 	for k, v := range want {
 		if kind, err := s.TypeOf(k); err != nil || kind != kinds[k] {
 			t.Errorf("%s.TypeOf(%q) = %v, %v; want %v", name, k, kind, err, kinds[k])
 		}
 		if got, err := s.ValueOf(k); err != nil || !reflect.DeepEqual(got, v) {
 			t.Errorf("%s.ValueOf(%q) = %#v, %v; want %#v", name, k, got, err, v)
 		}
 		if err := s.SetValue(k, v); err != nil {
 			t.Errorf("%s.SetValue(%q, %#v): %v", name, k, v, err)
 		}
 	}
 	if !reflect.DeepEqual(section.Interface(), before) {
 		t.Errorf("%s changed by setting its keys to their own values", name)
 	}
 }
 // flattenMap flattens a map with nested maps, as Map gives, into dotted keys
 // prefixed with prefix (if non-empty).
 func flattenMap(prefix string, m map[string]any) map[string]any {
 	flat := map[string]any{}
 	for k, v := range m {
 		if nested, ok := v.(map[string]any); ok {
 			maps.Copy(flat, flattenMap(joinKey(prefix, k), nested))
 		} else {
 			flat[joinKey(prefix, k)] = v
 		}
 	}
 	return flat
 }
 // checkEmbedded checks that structs embedded in a section, such as
 // JSCSourcesSource, implement Section too.
 func checkEmbedded(t *testing.T, section reflect.Type) {
 	t.Helper()
 	for i := 0; i < section.NumField(); i++ {
 		sf := section.Field(i)
 		if !sf.Anonymous || sf.Type.Kind() != reflect.Struct {
 			continue
 		}
 		if !reflect.PointerTo(sf.Type).Implements(sectionType) {
 			t.Errorf(
 				"*%s (embedded in %s) does not implement Section",
 				sf.Type.Name(),
 				section.Name(),
 			)
 		}
 		checkEmbedded(t, sf.Type)
 	}
 }
@@ -1,22 +1,7 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 package config
 import (
 	"context"
 	"fmt"
 	"log/slog"
 	"os"
@@ -25,36 +10,81 @@ import (
 	"strings"
 	"github.com/spf13/viper"
 	"git.avsdev.uk/Go/job-search/internal/logging"
 )
-// Debug: Prints detected environment variables
+// printEnv logs each JOBSEARCH* environment variable at debug level. On a
 // terminal it logs one line and prints the variables under it as
 // "name=value" lines; elsewhere each variable is a log entry of its own.
 func printEnv() {
-	slog.Debug("Environment:")
+	logger := slog.Default()
-	env := os.Environ()
+	terminal := logging.WritesToTerminal(logger.Handler())
-	slices.Sort(env)
+	if terminal && !logger.Enabled(context.Background(), slog.LevelDebug) {
-	for _, e := range env {
+		return
 		pair := strings.SplitN(e, "=", 2)
 		if strings.HasPrefix(pair[0], EnvPrefix) {
 			slog.Debug("\t" + pair[0] + "=" + pair[1])
 	}
 	var variables []string
 	for _, variable := range os.Environ() {
 		if strings.HasPrefix(variable, EnvPrefix) {
 			variables = append(variables, variable)
 		}
 	}
 	slices.Sort(variables)
 	if terminal {
 		logger.Debug("environment variables", "count", len(variables))
 	}
 	for _, variable := range variables {
 		if terminal {
 			logging.PrintToTerminal("\t" + variable)
 			continue
 		}
 		name, value, _ := strings.Cut(variable, "=")
 		logger.Debug("environment variable", "name", name, "value", value)
 	}
 }
-// Debug: Prints loaded configuration
+// printLoadedConfig logs every key set by the config file or an environment
 // variable (see isExplicitlySet), with its value and type, at debug level. On
 // a terminal it logs one line and prints the values under it as
 // "key = value <type>" lines; elsewhere each value is a log entry of its own.
 func printLoadedConfig() {
-	slog.Debug("Loaded config:")
+	logger := slog.Default()
-	keys := viper.AllKeys()
+	terminal := logging.WritesToTerminal(logger.Handler())
-	for i := range keys {
+	if terminal && !logger.Enabled(context.Background(), slog.LevelDebug) {
-		k := keys[i]
+		return
-		v := viper.Get(k)
+	}
-		t := reflect.TypeOf(v)
+
-		slog.Debug(
+	var keys []string
-			"\t" + k + " = " + fmt.Sprintf("%#v", v) + " <" + t.Kind().String() + ">")
+	for _, key := range viper.AllKeys() {
 		if isExplicitlySet(key) {
 			keys = append(keys, key)
 		}
 	}
-// Debug: Prints parsed configuration
+	if terminal {
 		logger.Debug("loaded config values", "count", len(keys))
 	}
 	for _, key := range keys {
 		value := viper.Get(key)
 		kind := reflect.TypeOf(value).Kind().String()
 		if terminal {
 			logging.PrintToTerminal(
 				fmt.Sprintf("\t%s = %#v <%s>", key, value, kind),
 			)
 			continue
 		}
 		logger.Debug(
 			"loaded config value",
 			"key", key,
 			"value", value,
 			"type", kind,
 		)
 	}
 }
 // printConfig logs conf at debug level.
 func printConfig(conf JobSearchConfig) {
 	slog.Debug("Config:")
 	conf.PrintLog(slog.LevelDebug)
 }
@@ -1,42 +1,59 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 package config
 import (
 	"errors"
 	"fmt"
 	"io/fs"
 	"log/slog"
 	"os"
 	"strings"
 	"github.com/joho/godotenv"
 	"github.com/spf13/viper"
 )
-// Reads in dotenv file for configuration
+// envKeyReplacer turns a dotted key into its environment variable's suffix
 // (logging.level to LOGGING_LEVEL, once upper-cased).
 var envKeyReplacer = strings.NewReplacer(".", "_")
 // EnvVarName returns the environment variable that sets key, e.g.
 // JOBSEARCH_LOGGING_LEVEL for logging.level.
 func EnvVarName(key string) string {
 	return EnvPrefix + "_" + strings.ToUpper(envKeyReplacer.Replace(key))
 }
 // isExplicitlySet reports whether key's value in Viper was set by the config
 // file or by its environment variable, rather than coming from its default.
 // As Viper does, it ignores an empty environment variable. Values set with
 // viper.Set (e.g. by StoreValue) are not counted.
 func isExplicitlySet(key string) bool {
 	if viper.InConfig(key) {
 		return true
 	}
 	value, ok := os.LookupEnv(EnvVarName(key))
 	return ok && value != ""
 }
 // loadEnv loads environment variables from a .env file in the working
 // directory, if there is one. One that can't be read or parsed is an error.
 func loadEnv() error {
-	if err := godotenv.Load(); err != nil {
+	err := godotenv.Load()
 	if errors.Is(err, fs.ErrNotExist) {
 		return nil
 	}
 	if err != nil {
 		return fmt.Errorf("invalid .env file: %w", err)
 	}
 	return nil
 }
-// Reads in config file and ENV variables if set to form a configuration.
+// loadConfig sets up Viper with the defaults, configFile (or config.* found in
-func loadConfig(cfgFile string) error {
+// EtcConfigPath or the working directory) and JOBSEARCH_* environment
-	if cfgFile != "" {
+// variables. A missing config file is only a warning.
 func loadConfig(configFile string) error {
 	if configFile != "" {
 		// Use config file from the flag.
-		viper.SetConfigFile(cfgFile)
+		viper.SetConfigFile(configFile)
 	} else {
 		// Search config in /etc or local directory.
 		viper.AddConfigPath(EtcConfigPath)
@@ -49,27 +66,29 @@ func loadConfig(cfgFile string) error {
 	LoadDefaultConfig()
 	viper.SetEnvPrefix(EnvPrefix)
 	viper.SetEnvKeyReplacer(envKeyReplacer)
 	viper.AutomaticEnv() // read in environment variables that match
 	err := viper.ReadInConfig()
-	if err != nil {
+	var notFound viper.ConfigFileNotFoundError
-		if _, ok := err.(viper.ConfigFileNotFoundError); ok {
+	switch {
-			// Indicates an explicitly set config file is not found or
+	case errors.As(err, &notFound):
-			// that no config file was found in any search path
+		// No config file was found in any search path.
-			slog.Warn("No config file found!")
+		slog.Warn("no config file found; using defaults")
-		} else {
+	case err != nil:
-			// Config file was found but another error was produced
+		// A config file was found but couldn't be read or parsed.
 		return err
 	default:
 		slog.Info("using config file", "file", viper.ConfigFileUsed())
 	}
-	} else {
+
 		// Config file found and successfully parsed
 		slog.Info("Using config file: " + viper.ConfigFileUsed())
 	}
 	return nil
 }
-// Initilises environment file configuration and general app configuration
+// Init loads the .env file and the configuration (see loadConfig) into conf.
-func Init(cfgFile string, conf *JobSearchConfig) error {
+// A debug or error log level already set in conf, from --debug or --quiet,
 // is kept over the loaded one.
 func Init(configFile string, conf *JobSearchConfig) error {
 	if err := loadEnv(); err != nil {
 		return err
 	}
@@ -77,14 +96,18 @@ func Init(cfgFile string, conf *JobSearchConfig) error {
 		printEnv()
 	}
-	if err := loadConfig(cfgFile); err != nil {
+	if err := loadConfig(configFile); err != nil {
 		return err
 	}
-	oldLogLevel := conf.Logging.Level
+	// Load configuration from Viper, ensure -debug and -quiet are retained
 	oldLogLevel := conf.Logging.LogLevel()
 	LoadFromViper(conf)
-	if ConfigLevelToLogLevel(oldLogLevel) < conf.Logging.LogLevel() {
+	if oldLogLevel == slog.LevelDebug {
-		conf.Logging.Level = oldLogLevel
+		conf.Logging.Level = "debug"
 	}
 	if oldLogLevel == slog.LevelError {
 		conf.Logging.Level = "error"
 	}
 	if conf.Logging.LogLevel() <= slog.LevelDebug {
@@ -0,0 +1,298 @@
 package config
 import (
 	"log/slog"
 	"os"
 	"path/filepath"
 	"strings"
 	"testing"
 	"time"
 	"github.com/spf13/viper"
 )
 // initDir makes a temp working directory holding a .env file (if env is not
 // nil) and a config.yaml (if config is not empty), and returns its path.
 func initDir(t *testing.T, env []string, config string) string {
 	t.Helper()
 	resetViper(t)
 	dir := t.TempDir()
 	t.Chdir(dir)
 	if env != nil {
 		// godotenv sets variables without cleaning up after the test.
 		for _, kv := range env {
 			key, _, _ := strings.Cut(kv, "=")
 			t.Cleanup(func() {
 				os.Unsetenv(key)
 			})
 		}
 		writeFile(t, filepath.Join(dir, ".env"), strings.Join(env, "\n")+"\n")
 	}
 	if config != "" {
 		writeFile(t, filepath.Join(dir, "config.yaml"), config)
 	}
 	return dir
 }
 func writeFile(t *testing.T, path, contents string) {
 	t.Helper()
 	if err := os.WriteFile(path, []byte(contents), 0o600); err != nil {
 		t.Fatal(err)
 	}
 }
 const testConfigYAML = `logging:
  level: info
 sources:
  fts_mock:
    enabled: true
    interval: 30s
 `
 func TestInitLoadsEnvAndConfigFile(t *testing.T) {
 	initDir(t, []string{"JOBSEARCH_TEST_ENV=from-dotenv"}, testConfigYAML)
 	captureLogs(t, slog.LevelError)
 	var conf JobSearchConfig
 	if err := Init("", &conf); err != nil {
 		t.Fatal(err)
 	}
 	if got := os.Getenv("JOBSEARCH_TEST_ENV"); got != "from-dotenv" {
 		t.Errorf("JOBSEARCH_TEST_ENV = %q; want it loaded from .env", got)
 	}
 	if conf.Logging.Level != "info" {
 		t.Errorf("Logging.Level = %q; want info", conf.Logging.Level)
 	}
 	if !conf.Sources.FTSMock.Enabled ||
 		conf.Sources.FTSMock.Interval != 30*time.Second {
 		t.Errorf("FTSMock = %+v; want enabled every 30s", conf.Sources.FTSMock)
 	}
 	// Keys missing from the file keep their defaults.
 	if conf.Sources.FindATender.Interval != 10*time.Minute {
 		t.Errorf(
 			"FindATender.Interval = %v; want default 10m",
 			conf.Sources.FindATender.Interval,
 		)
 	}
 }
 func TestInitExplicitConfigFile(t *testing.T) {
 	dir := initDir(t, []string{}, "logging:\n  level: info\n")
 	other := filepath.Join(dir, "other.yaml")
 	writeFile(t, other, "logging:\n  level: error\n")
 	captureLogs(t, slog.LevelError)
 	var conf JobSearchConfig
 	if err := Init(other, &conf); err != nil {
 		t.Fatal(err)
 	}
 	if conf.Logging.Level != "error" {
 		t.Errorf(
 			"Logging.Level = %q; want error from %s", conf.Logging.Level, other,
 		)
 	}
 }
 func TestInitNoConfigFile(t *testing.T) {
 	initDir(t, []string{}, "")
 	logs := captureLogs(t, slog.LevelWarn)
 	var conf JobSearchConfig
 	if err := Init("", &conf); err != nil {
 		t.Fatal(err)
 	}
 	if !strings.Contains(logs.String(), "no config file found") {
 		t.Errorf("no warning logged: %s", logs)
 	}
 	if conf.Logging.Level != "warn" {
 		t.Errorf("Logging.Level = %q; want default warn", conf.Logging.Level)
 	}
 }
 // The -debug and -quiet levels set before Init win over the config file.
 func TestInitKeepsCommandLineLevel(t *testing.T) {
 	for _, level := range []string{"debug", "error"} {
 		t.Run(level, func(t *testing.T) {
 			initDir(t, []string{}, "logging:\n  level: info\n")
 			logs := captureLogs(t, slog.LevelDebug)
 			conf := JobSearchConfig{
 				Logging: JSCLogging{
 					Level: level,
 				},
 			}
 			if err := Init("", &conf); err != nil {
 				t.Fatal(err)
 			}
 			if conf.Logging.Level != level {
 				t.Errorf("Logging.Level = %q; want %s kept", conf.Logging.Level, level)
 			}
 			// At debug, the environment and loaded config are logged.
 			gotDebug := strings.Contains(logs.String(), "loaded config value")
 			if gotDebug != (level == "debug") {
 				t.Errorf(
 					"debug output logged = %v at level %s: %s", gotDebug, level, logs,
 				)
 			}
 		})
 	}
 }
 // A .env file is optional.
 func TestInitWithoutEnvFile(t *testing.T) {
 	initDir(t, nil, testConfigYAML)
 	var conf JobSearchConfig
 	if err := Init("", &conf); err != nil {
 		t.Fatalf("err = %v; want nil without a .env file", err)
 	}
 }
 func TestInitErrors(t *testing.T) {
 	t.Run("invalid .env file", func(t *testing.T) {
 		dir := initDir(t, nil, testConfigYAML)
 		writeFile(t, filepath.Join(dir, ".env"), "NOT VALID\n")
 		var conf JobSearchConfig
 		if err := Init("", &conf); err == nil ||
 			!strings.Contains(err.Error(), ".env") {
 			t.Fatalf("err = %v; want an invalid .env file error", err)
 		}
 	})
 	t.Run("invalid config file", func(t *testing.T) {
 		initDir(t, []string{}, "logging: [unclosed\n")
 		captureLogs(t, slog.LevelError)
 		var conf JobSearchConfig
 		if err := Init("", &conf); err == nil {
 			t.Fatal("expected an error for invalid YAML")
 		}
 	})
 	t.Run("missing explicit config file", func(t *testing.T) {
 		dir := initDir(t, []string{}, "")
 		captureLogs(t, slog.LevelError)
 		var conf JobSearchConfig
 		if err := Init(filepath.Join(dir, "missing.yaml"), &conf); err == nil {
 			t.Fatal("expected an error for a missing config file")
 		}
 	})
 }
 // Environment variables override the config file, with nested keys joined
 // by underscores.
 func TestInitEnvOverride(t *testing.T) {
 	initDir(t, []string{}, testConfigYAML)
 	captureLogs(t, slog.LevelError)
 	t.Setenv("JOBSEARCH_LOGGING_LEVEL", "error")
 	t.Setenv("JOBSEARCH_SOURCES_FTS_MOCK_INTERVAL", "45s")
 	var conf JobSearchConfig
 	if err := Init("", &conf); err != nil {
 		t.Fatal(err)
 	}
 	if conf.Logging.Level != "error" {
 		t.Errorf(
 			"Logging.Level = %q; want error from the environment", conf.Logging.Level,
 		)
 	}
 	if conf.Sources.FTSMock.Interval != 45*time.Second {
 		t.Errorf(
 			"FTSMock.Interval = %v; want 45s from the environment",
 			conf.Sources.FTSMock.Interval,
 		)
 	}
 }
 func TestEnvVarName(t *testing.T) {
 	tests := map[string]string{
 		"logging.level":                             "JOBSEARCH_LOGGING_LEVEL",
 		"sources.fts_mock.interval":                 "JOBSEARCH_SOURCES_FTS_MOCK_INTERVAL",
 		"server.websocket.connection_limit.enabled": "JOBSEARCH_SERVER_WEBSOCKET_CONNECTION_LIMIT_ENABLED",
 	}
 	for key, want := range tests {
 		if got := EnvVarName(key); got != want {
 			t.Errorf("EnvVarName(%q) = %q; want %q", key, got, want)
 		}
 	}
 }
 // explicitConfigYAML sets a top-level and a nested key in a config file.
 const explicitConfigYAML = `logging:
  level: info
 server:
  websocket:
    connection_limit:
      enabled: true
 `
 // loadExplicit sets up Viper as Init does, with the defaults, the config
 // file explicitConfigYAML and the JOBSEARCH_* environment variables.
 func loadExplicit(t *testing.T) {
 	t.Helper()
 	resetViper(t)
 	LoadDefaultConfig()
 	viper.SetEnvPrefix(EnvPrefix)
 	viper.SetEnvKeyReplacer(envKeyReplacer)
 	viper.AutomaticEnv()
 	viper.SetConfigType("yaml")
 	if err := viper.ReadConfig(
 		strings.NewReader(explicitConfigYAML),
 	); err != nil {
 		t.Fatal(err)
 	}
 }
 // A key is explicitly set by the config file or a non-empty environment
 // variable, not by its default or viper.Set.
 func TestIsExplicitlySet(t *testing.T) {
 	t.Setenv("JOBSEARCH_DATABASE_FILE", "/tmp/js.db")
 	t.Setenv("JOBSEARCH_LOGGING_FILE", "")
 	loadExplicit(t)
 	viper.Set("sources.sleeper.enabled", true)
 	tests := map[string]bool{
 		"logging.level": true, // config file
 		"server.websocket.connection_limit.enabled": true, // nested, config file
 		"database.file":                           true,  // environment
 		"logging.file":                            false, // empty environment variable
 		"sources.sleeper.interval":                false, // default only
 		"sources.sleeper.enabled":                 false, // viper.Set
 		"server.websocket.connection_limit.burst": false, // default, beside a set key
 	}
 	for key, want := range tests {
 		if got := isExplicitlySet(key); got != want {
 			t.Errorf("isExplicitlySet(%q) = %v; want %v", key, got, want)
 		}
 	}
 }
 // printLoadedConfig logs only the keys set by the config file or the
 // environment.
 func TestPrintLoadedConfig(t *testing.T) {
 	t.Setenv("JOBSEARCH_DATABASE_FILE", "/tmp/js.db")
 	loadExplicit(t)
 	logs := captureLogs(t, slog.LevelDebug)
 	printLoadedConfig()
 	out := logs.String()
 	for _, want := range []string{
 		`key=logging.level value=info type=string`,
 		`key=server.websocket.connection_limit.enabled value=true type=bool`,
 		`key=database.file value=/tmp/js.db type=string`,
 	} {
 		if !strings.Contains(out, want) {
 			t.Errorf("missing %q in:\n%s", want, out)
 		}
 	}
 	for _, unwanted := range []string{
 		"sources.",
 		"logging.file",
 		"connection_limit.burst",
 	} {
 		if strings.Contains(out, unwanted) {
 			t.Errorf("logged default key %q:\n%s", unwanted, out)
 		}
 	}
 }
@@ -0,0 +1,127 @@
 package config
 import (
 	"errors"
 	"fmt"
 	"reflect"
 	"strconv"
 	"github.com/spf13/viper"
 )
 // StoreValue parses value as the type of key, checks conf accepts it, sets it
 // in conf and Viper and writes it to the config file Viper is using. Only the
 // file's own settings are written back, not defaults or environment
 // variables. It returns the stored value (as ValueOf gives it) and its kind.
 func StoreValue(
 	conf *JobSearchConfig, key, value string,
 ) (any, reflect.Kind, error) {
 	return storeValue(conf, key, value)
 }
 // storeValue is StoreValue for any section, so key types that
 // JobSearchConfig doesn't have yet can be tested.
 func storeValue(
 	section Section, key, value string,
 ) (any, reflect.Kind, error) {
 	confType, err := section.TypeOf(key)
 	if err != nil {
 		return nil, confType, err
 	}
 	// Check the value is of the right kind first, for a clearer error than
 	// SetValue gives.
 	if _, err := parseValue(confType, value); err != nil {
 		return nil, confType, fmt.Errorf(
 			"config: value type not accepted, expected <%s>", confType,
 		)
 	}
 	// SetValue parses the string into the field's own type (e.g. int8,
 	// float32 or time.Duration) and rejects values the configuration itself
 	// would not accept, e.g. an unknown log level or an unparseable duration.
 	if err := section.SetValue(key, value); err != nil {
 		return nil, confType, err
 	}
 	confValue, err := section.ValueOf(key)
 	if err != nil {
 		return nil, confType, err
 	}
 	viper.Set(key, confValue)
 	file, err := readUsedConfigFile()
 	if err != nil {
 		return nil, confType, err
 	}
 	file.Set(key, confValue)
 	if err := file.WriteConfig(); err != nil {
 		return nil, confType, err
 	}
 	return confValue, confType, nil
 }
 // parseValue parses value as a value of kind, returning
 // errors.ErrUnsupported for a kind it can't parse.
 func parseValue(kind reflect.Kind, value string) (any, error) {
 	switch kind {
 	case reflect.Bool:
 		return strconv.ParseBool(value)
 	case reflect.String:
 		return value, nil
 	case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
 		return strconv.ParseInt(value, 10, 64)
 	case reflect.Uint,
 		reflect.Uint8,
 		reflect.Uint16,
 		reflect.Uint32,
 		reflect.Uint64:
 		return strconv.ParseUint(value, 10, 64)
 	case reflect.Float32, reflect.Float64:
 		return strconv.ParseFloat(value, 64)
 	case reflect.Slice:
 		return splitList(value), nil
 	}
 	return nil, errors.ErrUnsupported
 }
 // PurgeValue rewrites the config file Viper is using without key. The
 // file's other settings are kept as they are; defaults and environment
 // variables are not written to it.
 func PurgeValue(conf *JobSearchConfig, key string) error {
 	if _, err := conf.TypeOf(key); err != nil {
 		return err
 	}
 	file, err := readUsedConfigFile()
 	if err != nil {
 		return err
 	}
 	out := viper.New()
 	out.SetConfigFile(file.ConfigFileUsed())
 	for _, fileKey := range file.AllKeys() {
 		if fileKey != key {
 			out.Set(fileKey, file.Get(fileKey))
 		}
 	}
 	return out.WriteConfig()
 }
 // readUsedConfigFile reads the config file Viper is using into a new Viper, so
 // it holds only the file's settings.
 func readUsedConfigFile() (*viper.Viper, error) {
 	path := viper.ConfigFileUsed()
 	if path == "" {
 		return nil, errors.New("config: no configuration file in use")
 	}
 	file := viper.New()
 	file.SetConfigFile(path)
 	if err := file.ReadInConfig(); err != nil {
 		return nil, err
 	}
 	return file, nil
 }
@@ -0,0 +1,379 @@
 package config
 import (
 	"errors"
 	"os"
 	"path/filepath"
 	"reflect"
 	"strings"
 	"testing"
 	"time"
 	"github.com/spf13/viper"
 )
 // useConfigFile points Viper at a new YAML config file holding contents.
 func useConfigFile(t *testing.T, contents string) string {
 	t.Helper()
 	resetViper(t)
 	path := filepath.Join(t.TempDir(), "config.yaml")
 	if err := os.WriteFile(path, []byte(contents), 0o600); err != nil {
 		t.Fatal(err)
 	}
 	viper.SetConfigFile(path)
 	if err := viper.ReadInConfig(); err != nil {
 		t.Fatal(err)
 	}
 	return path
 }
 // readConfigFile reads path with a fresh Viper.
 func readConfigFile(t *testing.T, path string) *viper.Viper {
 	t.Helper()
 	v := viper.New()
 	v.SetConfigFile(path)
 	if err := v.ReadInConfig(); err != nil {
 		t.Fatal(err)
 	}
 	return v
 }
 func TestStoreValue(t *testing.T) {
 	tests := []struct {
 		key, value string
 		want       any
 		kind       reflect.Kind
 	}{
 		{"logging.level", "debug", "debug", reflect.String},
 		{"sources.sleeper.enabled", "true", true, reflect.Bool},
 		{"sources.fts_mock.api_root", "http://mock", "http://mock", reflect.String},
 		{"sources.fts_mock.interval", "30s", "30s", reflect.String},
 	}
 	for _, tt := range tests {
 		t.Run(tt.key, func(t *testing.T) {
 			path := useConfigFile(t, "logging:\n  level: warn\n")
 			conf := defaultConfig(t)
 			got, kind, err := StoreValue(&conf, tt.key, tt.value)
 			if err != nil {
 				t.Fatal(err)
 			}
 			if got != tt.want || kind != tt.kind {
 				t.Errorf("got %#v <%s>; want %#v <%s>", got, kind, tt.want, tt.kind)
 			}
 			if v := readConfigFile(t, path).Get(tt.key); v != tt.want {
 				t.Errorf("file has %s = %#v; want %#v", tt.key, v, tt.want)
 			}
 		})
 	}
 }
 // A list of sections is given to StoreValue as a JSON array, written to the
 // file as a list of objects and read back from it.
 func TestStoreValueSectionList(t *testing.T) {
 	path := useConfigFile(t, "logging:\n  level: warn\n")
 	conf := defaultConfig(t)
 	const key = "server.headers.extra_response_headers"
 	_, kind, err := StoreValue(
 		&conf, key, `[{"header": "X-Robots-Tag", "value": "noindex"}]`,
 	)
 	if err != nil || kind != reflect.Slice {
 		t.Fatalf("StoreValue = <%s>, %v", kind, err)
 	}
 	want := []JSCServerHeadersExtra{
 		{
 			Header: "X-Robots-Tag",
 			Value:  "noindex",
 		},
 	}
 	if !reflect.DeepEqual(conf.Server.Headers.ExtraResponseHeaders, want) {
 		t.Errorf(
 			"conf has %+v; want %+v",
 			conf.Server.Headers.ExtraResponseHeaders,
 			want,
 		)
 	}
 	resetViper(t)
 	viper.SetConfigFile(path)
 	if err := viper.ReadInConfig(); err != nil {
 		t.Fatal(err)
 	}
 	reread := defaultConfig(t)
 	LoadFromViper(&reread)
 	if !reflect.DeepEqual(reread.Server.Headers.ExtraResponseHeaders, want) {
 		t.Errorf(
 			"file read back as %+v; want %+v",
 			reread.Server.Headers.ExtraResponseHeaders,
 			want,
 		)
 	}
 }
 func TestStoreValueKeepsOtherKeys(t *testing.T) {
 	path := useConfigFile(t, "logging:\n  level: warn\n  file: /tmp/js.log\n")
 	conf := defaultConfig(t)
 	if _, _, err := StoreValue(&conf, "logging.level", "error"); err != nil {
 		t.Fatal(err)
 	}
 	v := readConfigFile(t, path)
 	if got := v.GetString("logging.file"); got != "/tmp/js.log" {
 		t.Errorf("logging.file = %q; want it kept", got)
 	}
 }
 func TestStoreValueErrors(t *testing.T) {
 	tests := []struct {
 		name, key, value, wantErr string
 	}{
 		{"unknown key", "nope", "x", ""},
 		{"section key", "sources", "x", ""},
 		{"bad bool", "sources.sleeper.enabled", "maybe", "expected <bool>"},
 		{"invalid log level", "logging.level", "shouty", "invalid log level"},
 		{"bad duration", "sources.fts_mock.interval", "eventually", "duration"},
 	}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
 			path := useConfigFile(t, "logging:\n  level: warn\n")
 			//nolint:gosec // G304: reading a file the test created in t.TempDir
 			before, _ := os.ReadFile(path)
 			conf := defaultConfig(t)
 			_, _, err := StoreValue(&conf, tt.key, tt.value)
 			if err == nil {
 				t.Fatal("expected an error")
 			}
 			if !strings.Contains(err.Error(), tt.wantErr) {
 				t.Errorf("err = %v; want it to mention %q", err, tt.wantErr)
 			}
 			//nolint:gosec // G304: reading a file the test created in t.TempDir
 			if after, _ := os.ReadFile(path); string(after) != string(before) {
 				t.Errorf("config file changed:\n%s", after)
 			}
 		})
 	}
 }
 func TestStoreValueNoConfigFile(t *testing.T) {
 	resetViper(t)
 	conf := defaultConfig(t)
 	if _, _, err := StoreValue(&conf, "logging.level", "debug"); err == nil {
 		t.Fatal("expected an error with no config file to write")
 	}
 }
 func TestPurgeValue(t *testing.T) {
 	path := useConfigFile(t, "logging:\n  level: debug\n  file: /tmp/js.log\n")
 	conf := defaultConfig(t)
 	if err := PurgeValue(&conf, "logging.file"); err != nil {
 		t.Fatal(err)
 	}
 	v := readConfigFile(t, path)
 	if v.IsSet("logging.file") {
 		t.Errorf("logging.file still set: %v", v.Get("logging.file"))
 	}
 	if got := v.GetString("logging.level"); got != "debug" {
 		t.Errorf("logging.level = %q; want it kept", got)
 	}
 }
 func TestPurgeValueErrors(t *testing.T) {
 	path := useConfigFile(t, "logging:\n  level: debug\n")
 	//nolint:gosec // G304: reading a file the test created in t.TempDir
 	before, _ := os.ReadFile(path)
 	conf := defaultConfig(t)
 	if err := PurgeValue(&conf, "nope"); err == nil {
 		t.Error("expected an error for an unknown key")
 	}
 	//nolint:gosec // G304: reading a file the test created in t.TempDir
 	if after, _ := os.ReadFile(path); string(after) != string(before) {
 		t.Errorf("config file changed:\n%s", after)
 	}
 	resetViper(t)
 	if err := PurgeValue(&conf, "logging.level"); err == nil {
 		t.Error("expected an error with no config file to write")
 	}
 }
 func TestStoreValueUpdatesConf(t *testing.T) {
 	useConfigFile(t, "logging:\n  level: warn\n")
 	conf := defaultConfig(t)
 	if _, _, err := StoreValue(
 		&conf, "sources.fts_mock.interval", "30s",
 	); err != nil {
 		t.Fatal(err)
 	}
 	if conf.Sources.FTSMock.Interval != 30*time.Second {
 		t.Errorf("FTSMock.Interval = %v; want 30s", conf.Sources.FTSMock.Interval)
 	}
 }
 // parseValue parses the kinds a key can have; no current key is an integer
 // or float, so StoreValue can't reach those cases yet.
 func TestParseValue(t *testing.T) {
 	for _, tt := range []struct {
 		kind  reflect.Kind
 		value string
 		want  any
 	}{
 		{reflect.Bool, "true", true},
 		{reflect.String, "text", "text"},
 		{reflect.Int, "-42", int64(-42)},
 		{reflect.Int8, "7", int64(7)},
 		{reflect.Uint16, "65535", uint64(65535)},
 		{reflect.Float32, "1.5", 1.5},
 		{reflect.Slice, " a, b ,, c ", []string{"a", "b", "c"}},
 	} {
 		got, err := parseValue(tt.kind, tt.value)
 		if err != nil || !reflect.DeepEqual(got, tt.want) {
 			t.Errorf(
 				"parseValue(%v, %q) = %#v, %v; want %#v",
 				tt.kind,
 				tt.value,
 				got,
 				err,
 				tt.want,
 			)
 		}
 	}
 	for _, tt := range []struct {
 		kind  reflect.Kind
 		value string
 	}{
 		{reflect.Bool, "maybe"},
 		{reflect.Int, "1.5"},
 		{reflect.Uint, "-1"},
 		{reflect.Float64, "x"},
 	} {
 		if got, err := parseValue(tt.kind, tt.value); err == nil {
 			t.Errorf(
 				"parseValue(%v, %q) = %#v; want an error", tt.kind, tt.value, got,
 			)
 		}
 	}
 	_, err := parseValue(reflect.Map, "a")
 	if !errors.Is(err, errors.ErrUnsupported) {
 		t.Errorf("parseValue(map) err = %v; want ErrUnsupported", err)
 	}
 }
 // numberSection has the numeric kinds JobSearchConfig doesn't use yet, to
 // test storeValue with them.
 type numberSection struct {
 	Count int     `cfg:"count"`
 	Small int8    `cfg:"small"`
 	Big   uint16  `cfg:"big"`
 	Ratio float32 `cfg:"ratio"`
 }
 func (c numberSection) Map() map[string]any { return mapOf(c) }
 func (c numberSection) Flatten() map[string]any { return flattenOf("", c) }
 func (c numberSection) TypeOf(key string) (reflect.Kind, error) {
 	return typeOf(c, key)
 }
 func (c numberSection) ValueOf(key string) (any, error) {
 	return valueOf(c, key)
 }
 func (c *numberSection) SetValue(key string, value any) error {
 	return setValue(c, key, value)
 }
 // Numbers are stored as the field's own type, not as the int64, uint64 or
 // float64 that parsing produces.
 func TestStoreValueNumbers(t *testing.T) {
 	for _, tt := range []struct {
 		key, value string
 		want       any
 		kind       reflect.Kind
 	}{
 		{"count", "-42", -42, reflect.Int},
 		{"small", "7", int8(7), reflect.Int8},
 		{"big", "65535", uint16(65535), reflect.Uint16},
 		{"ratio", "1.5", float32(1.5), reflect.Float32},
 	} {
 		t.Run(tt.key, func(t *testing.T) {
 			path := useConfigFile(t, "count: 1\n")
 			var s numberSection
 			got, kind, err := storeValue(&s, tt.key, tt.value)
 			if err != nil {
 				t.Fatal(err)
 			}
 			if got != tt.want || kind != tt.kind {
 				t.Errorf("got %#v <%s>; want %#v <%s>", got, kind, tt.want, tt.kind)
 			}
 			if v, _ := s.ValueOf(tt.key); v != tt.want {
 				t.Errorf("section has %s = %#v; want %#v", tt.key, v, tt.want)
 			}
 			if v := readConfigFile(t, path).GetString(tt.key); v != tt.value {
 				t.Errorf("file has %s = %q; want %q", tt.key, v, tt.value)
 			}
 		})
 	}
 	for key, value := range map[string]string{
 		"count": "1.5",  // not an integer
 		"small": "300",  // too big for an int8
 		"big":   "-1",   // negative
 		"ratio": "lots", // not a number
 	} {
 		useConfigFile(t, "count: 1\n")
 		if _, _, err := storeValue(&numberSection{}, key, value); err == nil {
 			t.Errorf("storeValue(%s, %q) succeeded", key, value)
 		}
 	}
 }
 // Only the file's own settings are written back: defaults (and environment
 // variables) loaded into Viper stay out of the file.
 func TestStoreAndPurgeWriteOnlyFileSettings(t *testing.T) {
 	path := useConfigFile(t, "logging:\n  level: warn\n  file: /tmp/js.log\n")
 	LoadDefaultConfig()
 	conf := defaultConfig(t)
 	if _, _, err := StoreValue(
 		&conf, "sources.sleeper.enabled", "true",
 	); err != nil {
 		t.Fatal(err)
 	}
 	want := map[string]any{
 		"logging.level":           "warn",
 		"logging.file":            "/tmp/js.log",
 		"sources.sleeper.enabled": true,
 	}
 	checkFileSettings(t, path, want)
 	if err := PurgeValue(&conf, "logging.file"); err != nil {
 		t.Fatal(err)
 	}
 	delete(want, "logging.file")
 	checkFileSettings(t, path, want)
 }
 // checkFileSettings checks the config file at path holds exactly want.
 func checkFileSettings(t *testing.T, path string, want map[string]any) {
 	t.Helper()
 	v := readConfigFile(t, path)
 	got := map[string]any{}
 	for _, k := range v.AllKeys() {
 		got[k] = v.Get(k)
 	}
 	if !reflect.DeepEqual(got, want) {
 		//nolint:gosec // G304: reading a file the test created in t.TempDir
 		b, _ := os.ReadFile(path)
 		t.Errorf("file settings = %v; want %v\n%s", got, want, b)
 	}
 }
@@ -1,75 +1,58 @@
 /*
 Copyright © 2026 Craig Williams <craig@avsdev.uk>
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU Lesser General Public License for more details.
 You should have received a copy of the GNU Lesser General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 package config
 import (
-	"encoding/json"
+	"context"
 	"fmt"
 	"log/slog"
 	"maps"
 	"slices"
-	"go.yaml.in/yaml/v3"
+	"git.avsdev.uk/Go/job-search/internal/logging"
 )
-// Prints configuration to stdout in flat dotted format
+// PrintTerminal prints the configuration to stdout as String formats it.
-func (conf JobSearchConfig) PrintTerminal() {
+func (c JobSearchConfig) PrintTerminal() {
-	for k, v := range conf.Flatten() {
+	fmt.Print(c.String() + "\n")
 		typ, _ := conf.TypeOf(k)
 		fmt.Print(k, " = ", v, " <", typ, ">\n")
 	}
 }
-// Prints configuration to stdout as JSON
+// PrintJSON prints the configuration to stdout as JSON.
-func (conf JobSearchConfig) PrintJson() {
+func (c JobSearchConfig) PrintJSON() {
-	confMap := conf.Map()
+	fmt.Print(c.JSON() + "\n")
 	jbytes, _ := json.MarshalIndent(confMap, "", "  ")
 	fmt.Print(string(jbytes) + "\n")
 }
-// Prints configuration to stdout as YAML
+// PrintYAML prints the configuration to stdout as YAML.
-func (conf JobSearchConfig) PrintYaml() {
+func (c JobSearchConfig) PrintYAML() {
-	confMap := conf.Map()
+	fmt.Print(c.YAML() + "\n")
 	jbytes, _ := yaml.Marshal(confMap)
 	fmt.Print(string(jbytes) + "\n")
 }
-// Prints configuration to logrus log at debug, info (default) or error level
+// PrintLog logs each configuration key and value, sorted by key, at atLevel:
-func (conf JobSearchConfig) PrintLog(atLevel slog.Level) {
+// debug, warn, error, or info for any other level. On a terminal it logs one
-	var logFn func(args ...interface{})
+// line and prints the values under it as "key = value" lines; elsewhere each
-
+// value is a log entry of its own.
 func (c JobSearchConfig) PrintLog(atLevel slog.Level) {
 	switch atLevel {
-	case slog.LevelDebug:
+	case slog.LevelDebug, slog.LevelWarn, slog.LevelError:
 		logFn = func(args ...interface{}) {
 			slog.Debug(fmt.Sprint(args...))
 		}
 	case slog.LevelError:
 		logFn = func(args ...interface{}) {
 			slog.Error(fmt.Sprint(args...))
 		}
 	case slog.LevelWarn:
 		logFn = func(args ...interface{}) {
 			slog.Warn(fmt.Sprint(args...))
 		}
 	default:
-		logFn = func(args ...interface{}) {
+		atLevel = slog.LevelInfo
 			slog.Info(fmt.Sprint(args...))
 		}
 	}
-	for k, v := range conf.Flatten() {
+	ctx := context.Background()
-		logFn("\t", k, " = ", v)
+	logger := slog.Default()
 	flat := c.Flatten()
 	keys := slices.Sorted(maps.Keys(flat))
 	if logging.WritesToTerminal(logger.Handler()) {
 		if !logger.Enabled(ctx, atLevel) {
 			return
 		}
 		logger.Log(ctx, atLevel, "config values", "count", len(keys))
 		for _, key := range keys {
 			logging.PrintToTerminal(fmt.Sprintf("\t%s = %v", key, flat[key]))
 		}
 		return
 	}
 	for _, key := range keys {
 		logger.Log(ctx, atLevel, "config value", "key", key, "value", flat[key])
 	}
 }
@@ -0,0 +1,145 @@
 package config
 import (
 	"encoding/json"
 	"log/slog"
 	"os"
 	"path/filepath"
 	"slices"
 	"strings"
 	"testing"
 	"go.yaml.in/yaml/v3"
 )
 // captureStdout returns what fn prints to os.Stdout.
 func captureStdout(t *testing.T, fn func()) string {
 	t.Helper()
 	path := filepath.Join(t.TempDir(), "stdout")
 	f, err := os.Create(filepath.Clean(path))
 	if err != nil {
 		t.Fatal(err)
 	}
 	old := os.Stdout
 	os.Stdout = f
 	t.Cleanup(func() {
 		os.Stdout = old
 	})
 	fn()
 	os.Stdout = old
 	if err := f.Close(); err != nil {
 		t.Fatal(err)
 	}
 	b, err := os.ReadFile(filepath.Clean(path))
 	if err != nil {
 		t.Fatal(err)
 	}
 	return string(b)
 }
 func TestPrintTerminal(t *testing.T) {
 	conf := defaultConfig(t)
 	got := captureStdout(t, conf.PrintTerminal)
 	if want := conf.String() + "\n"; got != want {
 		t.Errorf("PrintTerminal printed %q; want %q", got, want)
 	}
 }
 // String lists every key once, sorted, so the output is the same each run.
 func TestStringSorted(t *testing.T) {
 	conf := defaultConfig(t)
 	var keys []string
 	for _, line := range strings.Split(strings.TrimSpace(conf.String()), "\n") {
 		key, _, _ := strings.Cut(line, " = ")
 		keys = append(keys, key)
 	}
 	if !slices.IsSorted(keys) || len(keys) != len(conf.Flatten()) {
 		t.Errorf("String keys = %v; want each key once, sorted", keys)
 	}
 	if first := conf.String(); conf.String() != first {
 		t.Error("String output differs between calls")
 	}
 }
 // PrintJSON and PrintYAML print the configuration in their format.
 func TestPrintFormats(t *testing.T) {
 	conf := defaultConfig(t)
 	tests := []struct {
 		name      string
 		print     func()
 		unmarshal func([]byte, any) error
 	}{
 		{"PrintJSON", conf.PrintJSON, json.Unmarshal},
 		{"PrintYAML", conf.PrintYAML, yaml.Unmarshal},
 	}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
 			out := captureStdout(t, tt.print)
 			var m map[string]any
 			if err := tt.unmarshal([]byte(out), &m); err != nil {
 				t.Fatalf("%s output doesn't parse: %v\n%s", tt.name, err, out)
 			}
 			level := m["logging"].(map[string]any)["level"]
 			if level != "warn" {
 				t.Errorf("logging.level = %v; want warn", level)
 			}
 		})
 	}
 }
 // PrintLog logs one line per key, at the level asked for (info for any
 // level other than debug, warn and error).
 func TestPrintLog(t *testing.T) {
 	conf := defaultConfig(t)
 	for _, tt := range []struct {
 		at   slog.Level
 		want string
 	}{
 		{slog.LevelDebug, "level=DEBUG"},
 		{slog.LevelInfo, "level=INFO"},
 		{slog.LevelWarn, "level=WARN"},
 		{slog.LevelError, "level=ERROR"},
 		{slog.LevelError + 4, "level=INFO"},
 	} {
 		logs := captureLogs(t, slog.LevelDebug)
 		conf.PrintLog(tt.at)
 		lines := strings.Split(strings.TrimSpace(logs.String()), "\n")
 		if len(lines) != len(conf.Flatten()) {
 			t.Errorf(
 				"PrintLog(%v) logged %d lines; want %d",
 				tt.at,
 				len(lines),
 				len(conf.Flatten()),
 			)
 		}
 		for _, l := range lines {
 			if !strings.Contains(l, tt.want) {
 				t.Errorf("PrintLog(%v) line %q; want %s", tt.at, l, tt.want)
 			}
 		}
 		if !strings.Contains(logs.String(), "key=logging.level value=warn") {
 			t.Errorf("PrintLog(%v) missing logging.level:\n%s", tt.at, logs)
 		}
 		// Each line is msg="config value" key=<key> value=<value>.
 		flat := conf.Flatten()
 		var keys []string
 		for _, l := range lines {
 			_, rest, _ := strings.Cut(l, ` key=`)
 			key, _, _ := strings.Cut(rest, " ")
 			if _, ok := flat[key]; !ok {
 				t.Errorf("PrintLog(%v) line %q: no key", tt.at, l)
 			}
 			keys = append(keys, key)
 		}
 		if !slices.IsSorted(keys) {
 			t.Errorf("PrintLog(%v) keys not sorted: %v", tt.at, keys)
 		}
 	}
 }
@@ -0,0 +1,603 @@
 package config
 import (
 	"encoding/json"
 	"errors"
 	"fmt"
 	"reflect"
 	"slices"
 	"strconv"
 	"strings"
 	"time"
 	"git.avsdev.uk/Go/job-search/internal/util"
 )
 // Configuration sections are plain structs described with struct tags:
 //
 //	cfg:"key"        the field's configuration key. Fields without it are
 //	                 not part of the configuration.
 //	default:"value"  the field's default, parsed the same way as a string
 //	                 passed to SetValue (e.g. "10m", "true", "42").
 //
 // A tagged field whose type is a struct (other than time.Time) is a nested
 // section, addressed as "section.key". An embedded struct has its tagged
 // fields promoted into the enclosing section, just as Go promotes fields.
 //
 // time.Duration fields report reflect.String from TypeOf and accept either a
 // time.Duration or a duration string such as "90s" in SetValue.
 // util.ByteSize fields work the same way, with size strings such as "32KiB"
 // (or "-1" for no limit).
 //
 // []string fields report reflect.Slice and accept either a []string or a
 // comma-separated string such as "a, b" in SetValue (and so in default tags).
 // An empty list is stored as nil.
 //
 // A list of sections, []S where S is a section struct (e.g. a list of extra
 // headers), is a single key reporting reflect.Slice. It is given out as a
 // []map[string]any, one map per item as Map gives it, and SetValue accepts
 // a []S, a list of maps (as Viper reads a YAML or JSON list of objects) or a
 // JSON array string. Each item starts from its defaults; its keys are set as
 // SetValue sets them, so values may be strings to parse. If the item type has
 // a validator, every field of each item is validated, including ones not set.
 //
 // A section can check values before they are stored by implementing
 // validator on its pointer type.
 // The struct tags section.go reads.
 const (
 	// keyTag holds a field's configuration key.
 	keyTag = "cfg"
 	// defaultTag holds a field's default value.
 	defaultTag = "default"
 	// initTag set to initHidden on a section field leaves the section out of
 	// the file `config init` writes (see InitDefaults), as for test sources.
 	initTag    = "init"
 	initHidden = "hidden"
 )
 // The unknown key error and the field types section.go handles specially.
 var (
 	// errUnknownKey is returned for a key that names no field.
 	errUnknownKey = errors.New("config: unknown configuration key")
 	// durationType is time.Duration, read and given out as a string.
 	durationType = reflect.TypeOf(time.Duration(0))
 	// byteSizeType is util.ByteSize, read and given out as a string.
 	byteSizeType = reflect.TypeOf(util.ByteSize(0))
 	// timeType is time.Time, a struct that is a value, not a section.
 	timeType = reflect.TypeOf(time.Time{})
 	// stringSliceType is []string, set from a comma-separated list.
 	stringSliceType = reflect.TypeOf([]string{})
 )
 // Section is the method set of every configuration section, including
 // JobSearchConfig itself, implemented on the section's pointer type. Keys are
 // dotted paths relative to the section, e.g. "sleeper.interval" in JSCSources.
 // Durations and byte sizes are given out as strings such as "10m0s" and
 // "32KiB", matching TypeOf's reflect.String for them. TestSections checks every
 // section, found by walking JobSearchConfig, implements it.
 type Section interface {
 	// Map returns the section as a map of keys to values, with nested
 	// sections as nested maps.
 	Map() map[string]any
 	// Flatten returns the section as a flat map of dotted keys, each
 	// prefixed with the section's own key (its JSCK* constant), so a
 	// section's keys match how its parent flattens it. JobSearchConfig, the
 	// embedded JSCSourcesSource and types used under several keys, such as
 	// JSCRateLimitRule, have no prefix.
 	Flatten() map[string]any
 	// TypeOf returns the kind of the value at key, which is how it is read
 	// from Viper, or an error for an unknown key.
 	TypeOf(key string) (reflect.Kind, error)
 	// ValueOf returns the value at key, or an error for an unknown key.
 	ValueOf(key string) (any, error)
 	// SetValue stores value at key. value may be of the field's type or a
 	// string to parse (e.g. "10m", "true", "42"). The section's validation,
 	// if any, runs first; on error the field is unchanged.
 	SetValue(key string, value any) error
 }
 // validator is implemented by sections that need to reject some values.
 // key is the field's own key within the section; value has already been
 // converted to the field's type.
 type validator interface {
 	validate(key string, value any) error
 }
 // field is a section struct field that is part of the configuration (see
 // fieldsOf).
 type field struct {
 	key          string
 	index        []int // path for reflect.Value.FieldByIndex
 	fieldType    reflect.Type
 	section      bool
 	defaultValue string
 	hasDefault   bool
 	// hiddenFromInit is set by initTag (see InitDefaults).
 	hiddenFromInit bool
 }
 // isSectionList reports whether listType is a list of sections, []S for a
 // section struct S.
 func isSectionList(listType reflect.Type) bool {
 	return listType.Kind() == reflect.Slice &&
 		listType.Elem().Kind() == reflect.Struct &&
 		listType.Elem() != timeType
 }
 // fieldsOf lists the configuration fields of a section type in declaration
 // order, with the fields of embedded structs promoted into it.
 func fieldsOf(structType reflect.Type) []field {
 	var fields []field
 	for i := 0; i < structType.NumField(); i++ {
 		structField := structType.Field(i)
 		if structField.Anonymous && structField.Type.Kind() == reflect.Struct {
 			for _, embeddedField := range fieldsOf(structField.Type) {
 				embeddedField.index = append([]int{i}, embeddedField.index...)
 				fields = append(fields, embeddedField)
 			}
 			continue
 		}
 		key, ok := structField.Tag.Lookup(keyTag)
 		if !ok || key == "" || key == "-" || !structField.IsExported() {
 			continue
 		}
 		defaultValue, hasDefault := structField.Tag.Lookup(defaultTag)
 		fields = append(fields, field{
 			key:       key,
 			index:     []int{i},
 			fieldType: structField.Type,
 			section: structField.Type.Kind() == reflect.Struct &&
 				structField.Type != timeType,
 			defaultValue:   defaultValue,
 			hasDefault:     hasDefault,
 			hiddenFromInit: structField.Tag.Get(initTag) == initHidden,
 		})
 	}
 	return fields
 }
 // structValue dereferences value down to the section struct it holds.
 func structValue(value any) reflect.Value {
 	reflected := reflect.ValueOf(value)
 	for reflected.Kind() == reflect.Pointer {
 		reflected = reflected.Elem()
 	}
 	if reflected.Kind() != reflect.Struct {
 		panic(fmt.Sprintf("config: %T is not a configuration section", value))
 	}
 	return reflected
 }
 // lookup resolves a dotted key to the leaf field and the section holding it.
 func lookup(section reflect.Value, key string) (reflect.Value, field, error) {
 	head, rest, nested := strings.Cut(key, ".")
 	for _, configField := range fieldsOf(section.Type()) {
 		if configField.key != head {
 			continue
 		}
 		switch {
 		case configField.section && nested:
 			return lookup(section.FieldByIndex(configField.index), rest)
 		case !configField.section && !nested:
 			return section, configField, nil
 		}
 		// Key matched but the shape didn't: a section with no sub-key, or a
 		// value with one.
 		return reflect.Value{}, field{}, errUnknownKey
 	}
 	return reflect.Value{}, field{}, errUnknownKey
 }
 func joinKey(prefix, key string) string {
 	if prefix == "" {
 		return key
 	}
 	return prefix + "." + key
 }
 // keysOf returns the keys directly in a section.
 func keysOf(value any) []string {
 	fields := fieldsOf(structValue(value).Type())
 	keys := make([]string, 0, len(fields))
 	for _, configField := range fields {
 		keys = append(keys, configField.key)
 	}
 	return keys
 }
 // mapOf converts a section to a map, with nested sections as nested maps.
 func mapOf(value any) map[string]any {
 	return mapOfValue(structValue(value))
 }
 // mapOfValue is mapOf for a section struct value.
 func mapOfValue(section reflect.Value) map[string]any {
 	values := map[string]any{}
 	for _, configField := range fieldsOf(section.Type()) {
 		fieldValue := section.FieldByIndex(configField.index)
 		if configField.section {
 			values[configField.key] = mapOfValue(fieldValue)
 		} else {
 			values[configField.key] = exportValue(fieldValue)
 		}
 	}
 	return values
 }
 // flattenOf converts a section to a flat map of dotted keys, each prefixed
 // with prefix (if non-empty).
 func flattenOf(prefix string, value any) map[string]any {
 	flat := map[string]any{}
 	flattenInto(flat, prefix, structValue(value))
 	return flat
 }
 // flattenInto adds section's keys to m, dotted and prefixed with prefix (if
 // non-empty), descending into nested sections.
 func flattenInto(flat map[string]any, prefix string, section reflect.Value) {
 	for _, configField := range fieldsOf(section.Type()) {
 		fieldValue := section.FieldByIndex(configField.index)
 		key := joinKey(prefix, configField.key)
 		if configField.section {
 			flattenInto(flat, key, fieldValue)
 		} else {
 			flat[key] = exportValue(fieldValue)
 		}
 	}
 }
 // hiddenFromInitKeys returns the dotted keys of the sections under section
 // (prefixed with prefix, if non-empty) whose field has initTag set to
 // initHidden, descending into nested sections.
 func hiddenFromInitKeys(prefix string, sectionType reflect.Type) []string {
 	var hidden []string
 	for _, configField := range fieldsOf(sectionType) {
 		if !configField.section {
 			continue
 		}
 		key := joinKey(prefix, configField.key)
 		if configField.hiddenFromInit {
 			hidden = append(hidden, key)
 			continue
 		}
 		hidden = append(hidden, hiddenFromInitKeys(key, configField.fieldType)...)
 	}
 	return hidden
 }
 // exportValue returns a field's value as Map, Flatten and ValueOf give it
 // out: durations and byte sizes become strings such as "1m30s" and "32KiB",
 // so values agree with kindOf and can be passed back to SetValue; []string is
 // copied; a list of sections becomes a []map[string]any (nil if empty);
 // everything else is unchanged.
 func exportValue(fieldValue reflect.Value) any {
 	switch fieldValue.Type() {
 	case durationType:
 		return time.Duration(fieldValue.Int()).String()
 	case byteSizeType:
 		return util.ByteSize(fieldValue.Int()).String()
 	}
 	if isSectionList(fieldValue.Type()) {
 		var items []map[string]any
 		for i := range fieldValue.Len() {
 			items = append(items, mapOfValue(fieldValue.Index(i)))
 		}
 		return items
 	}
 	value := fieldValue.Interface()
 	if list, ok := value.([]string); ok {
 		// A copy, so changing it doesn't change the configuration.
 		return slices.Clone(list)
 	}
 	return value
 }
 // splitList splits a comma-separated list, trimming spaces around each item
 // and dropping empty ones, so "" is an empty list.
 func splitList(list string) []string {
 	var items []string
 	for item := range strings.SplitSeq(list, ",") {
 		if item = strings.TrimSpace(item); item != "" {
 			items = append(items, item)
 		}
 	}
 	return items
 }
 // kindOf is the kind reported by TypeOf, i.e. how the value is read from
 // Viper. Durations and byte sizes are read as strings and parsed by SetValue.
 func kindOf(fieldType reflect.Type) reflect.Kind {
 	if fieldType == durationType || fieldType == byteSizeType {
 		return reflect.String
 	}
 	return fieldType.Kind()
 }
 // typeOf returns the kind of the dotted key in the section value (see
 // kindOf), or errUnknownKey.
 func typeOf(value any, key string) (reflect.Kind, error) {
 	_, configField, err := lookup(structValue(value), key)
 	if err != nil {
 		return reflect.Invalid, err
 	}
 	return kindOf(configField.fieldType), nil
 }
 // valueOf returns the value of the dotted key in the section value, or
 // errUnknownKey.
 func valueOf(value any, key string) (any, error) {
 	section, configField, err := lookup(structValue(value), key)
 	if err != nil {
 		return nil, err
 	}
 	return exportValue(section.FieldByIndex(configField.index)), nil
 }
 // setValue sets a dotted key on the section that ptr points to.
 func setValue(ptr any, key string, value any) error {
 	section, configField, err := lookup(structValue(ptr), key)
 	if err != nil {
 		return err
 	}
 	return setField(section, configField, value)
 }
 // setField converts value to the field's type and, if the section has a
 // validator, checks it, before setting the field. On error the field is left
 // unchanged.
 func setField(section reflect.Value, configField field, value any) error {
 	newValue, err := convert(value, configField.fieldType)
 	if err != nil {
 		return err
 	}
 	if checker, ok := section.Addr().Interface().(validator); ok {
 		err := checker.validate(configField.key, newValue.Interface())
 		if err != nil {
 			return err
 		}
 	}
 	section.FieldByIndex(configField.index).Set(newValue)
 	return nil
 }
 // convert turns value into a value of type targetType. Values already of the
 // right type are used as-is, except that an empty []string becomes nil, as an
 // empty comma-separated list does (see splitList); strings are parsed. Lists of
 // sections are converted by convertSectionList.
 func convert(value any, targetType reflect.Type) (reflect.Value, error) {
 	if isSectionList(targetType) {
 		return convertSectionList(value, targetType)
 	}
 	if value != nil {
 		reflected := reflect.ValueOf(value)
 		if reflected.Type().AssignableTo(targetType) {
 			if targetType == stringSliceType && reflected.Len() == 0 {
 				return reflect.Zero(targetType), nil
 			}
 			return reflected, nil
 		}
 		if text, ok := value.(string); ok {
 			return parseString(text, targetType)
 		}
 	}
 	return reflect.Value{}, fmt.Errorf("config: unsupported type %T", value)
 }
 // convertSectionList turns value into a list of sections of type listType: a
 // value of type listType, a list of maps of keys to values, or a JSON array of
 // objects. nil and empty lists become a nil list.
 func convertSectionList(
 	value any, listType reflect.Type,
 ) (reflect.Value, error) {
 	if text, ok := value.(string); ok {
 		var items []map[string]any
 		if strings.TrimSpace(text) != "" {
 			if err := json.Unmarshal([]byte(text), &items); err != nil {
 				return reflect.Value{}, fmt.Errorf(
 					"unable to convert to a list of %s: %w", listType.Elem(), err,
 				)
 			}
 		}
 		value = items
 	}
 	if value == nil {
 		return reflect.Zero(listType), nil
 	}
 	reflected := reflect.ValueOf(value)
 	if reflected.Kind() != reflect.Slice {
 		return reflect.Value{}, fmt.Errorf("config: unsupported type %T", value)
 	}
 	if reflected.Len() == 0 {
 		return reflect.Zero(listType), nil
 	}
 	if reflected.Type().AssignableTo(listType) {
 		for i := range reflected.Len() {
 			if err := validateSection(reflected.Index(i)); err != nil {
 				return reflect.Value{}, fmt.Errorf("item %d: %w", i, err)
 			}
 		}
 		return reflected, nil
 	}
 	list := reflect.MakeSlice(listType, 0, reflected.Len())
 	for i := range reflected.Len() {
 		item, err := sectionFromMap(reflected.Index(i).Interface(), listType.Elem())
 		if err != nil {
 			return reflect.Value{}, fmt.Errorf("item %d: %w", i, err)
 		}
 		list = reflect.Append(list, item)
 	}
 	return list, nil
 }
 // sectionFromMap builds a section of type structType from value, a map of its
 // keys to values, starting from its defaults. Values that are neither strings
 // nor of the field's type (e.g. numbers read from YAML) are formatted and
 // parsed. The finished section is validated as a whole (see
 // validateSection), so a key left out of the map can't skip its validation.
 func sectionFromMap(value any, structType reflect.Type) (reflect.Value, error) {
 	item, ok := value.(map[string]any)
 	if !ok {
 		return reflect.Value{}, fmt.Errorf("config: unsupported type %T", value)
 	}
 	section := reflect.New(structType).Elem()
 	if err := applyDefaultsTo("", section); err != nil {
 		return reflect.Value{}, err
 	}
 	for itemKey, itemValue := range item {
 		holder, configField, err := lookup(section, itemKey)
 		if err != nil {
 			return reflect.Value{}, fmt.Errorf("%w %q", err, itemKey)
 		}
 		if _, isString := itemValue.(string); itemValue != nil && !isString &&
 			!reflect.TypeOf(itemValue).AssignableTo(configField.fieldType) {
 			itemValue = fmt.Sprint(itemValue)
 		}
 		if err := setField(holder, configField, itemValue); err != nil {
 			return reflect.Value{}, fmt.Errorf("%s: %w", itemKey, err)
 		}
 	}
 	if err := validateSection(section); err != nil {
 		return reflect.Value{}, err
 	}
 	return section, nil
 }
 // validateSection runs the section's validator, if it has one, on every value
 // field directly in it, so values that were never set (defaults or zero
 // values) are checked too.
 func validateSection(section reflect.Value) error {
 	checker, ok := reflect.New(section.Type()).Interface().(validator)
 	if !ok {
 		return nil
 	}
 	for _, configField := range fieldsOf(section.Type()) {
 		if configField.section {
 			continue
 		}
 		value := section.FieldByIndex(configField.index).Interface()
 		if err := checker.validate(configField.key, value); err != nil {
 			return fmt.Errorf("%s: %w", configField.key, err)
 		}
 	}
 	return nil
 }
 // parseString parses text as a value of type targetType. It supports strings,
 // bools, signed and unsigned integers, floats, time.Duration, ByteSize and
 // []string (from a comma-separated list, see splitList).
 func parseString(text string, targetType reflect.Type) (reflect.Value, error) {
 	parsed := reflect.New(targetType).Elem()
 	switch targetType {
 	case durationType:
 		duration, err := time.ParseDuration(text)
 		if err != nil {
 			return reflect.Value{}, fmt.Errorf(
 				"unable to convert to duration: %w", err,
 			)
 		}
 		parsed.SetInt(int64(duration))
 		return parsed, nil
 	case byteSizeType:
 		size, err := util.ParseByteSize(text)
 		if err != nil {
 			return reflect.Value{}, fmt.Errorf(
 				"unable to convert to byte size: %w", err,
 			)
 		}
 		parsed.SetInt(int64(size))
 		return parsed, nil
 	case stringSliceType:
 		parsed.Set(reflect.ValueOf(splitList(text)))
 		return parsed, nil
 	}
 	var err error
 	switch targetType.Kind() {
 	case reflect.String:
 		parsed.SetString(text)
 	case reflect.Bool:
 		var flag bool
 		flag, err = strconv.ParseBool(text)
 		parsed.SetBool(flag)
 	case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
 		var number int64
 		number, err = strconv.ParseInt(text, 10, targetType.Bits())
 		parsed.SetInt(number)
 	case reflect.Uint,
 		reflect.Uint8,
 		reflect.Uint16,
 		reflect.Uint32,
 		reflect.Uint64:
 		var number uint64
 		number, err = strconv.ParseUint(text, 10, targetType.Bits())
 		parsed.SetUint(number)
 	case reflect.Float32, reflect.Float64:
 		var number float64
 		number, err = strconv.ParseFloat(text, targetType.Bits())
 		parsed.SetFloat(number)
 	default:
 		return reflect.Value{}, fmt.Errorf(
 			"config: unsupported type string for %s", targetType,
 		)
 	}
 	if err != nil {
 		return reflect.Value{}, fmt.Errorf(
 			"unable to convert %q to %s: %w", text, targetType, err,
 		)
 	}
 	return parsed, nil
 }
 // applyDefaults sets every field that has a default tag, recursing into
 // nested sections. ptr must point to a section struct.
 func applyDefaults(ptr any) error {
 	return applyDefaultsTo("", structValue(ptr))
 }
 // applyDefaultsTo sets every field in section that has a default tag,
 // recursing into nested sections. prefix is the section's dotted key, used in
 // error messages.
 func applyDefaultsTo(prefix string, section reflect.Value) error {
 	for _, configField := range fieldsOf(section.Type()) {
 		key := joinKey(prefix, configField.key)
 		if configField.section {
 			nested := section.FieldByIndex(configField.index)
 			if err := applyDefaultsTo(key, nested); err != nil {
 				return err
 			}
 			continue
 		}
 		if !configField.hasDefault {
 			continue
 		}
 		err := setField(section, configField, configField.defaultValue)
 		if err != nil {
 			return fmt.Errorf("invalid default for %s: %w", key, err)
 		}
 	}
 	return nil
 }
@@ -0,0 +1,407 @@
 package config
 import (
 	"errors"
 	"reflect"
 	"strings"
 	"testing"
 	"time"
 	"git.avsdev.uk/Go/job-search/internal/util"
 )
 // errRejected is the error testValidated.validate rejects "bad" with.
 var errRejected = errors.New("rejected")
 // Test sections exercising every field shape section.go supports.
 type testInner struct {
 	Name string `cfg:"name" default:"inner"`
 }
 // testItem is the item type of a list of sections.
 type testItem struct {
 	Name string        `cfg:"name" default:"item"`
 	N    int           `cfg:"n"`
 	Wait time.Duration `cfg:"wait"`
 }
 type testEmbedded struct {
 	On bool `cfg:"on" default:"true"`
 }
 type testSection struct {
 	testEmbedded
 	Str      string        `cfg:"str"        default:"hello"`
 	Int      int           `cfg:"int"        default:"-42"`
 	Int8     int8          `cfg:"int8"`
 	Uint     uint          `cfg:"uint"       default:"7"`
 	Uint16   uint16        `cfg:"uint16"`
 	Float    float64       `cfg:"float"      default:"1.5"`
 	Float32  float32       `cfg:"float32"`
 	Dur      time.Duration `cfg:"dur"        default:"90s"`
 	Size     util.ByteSize `cfg:"size"       default:"32KiB"`
 	When     time.Time     `cfg:"when"`
 	Slice    []string      `cfg:"slice"`
 	Items    []testItem    `cfg:"items"`
 	Inner    testInner     `cfg:"inner"`
 	NoTag    string
 	Skipped  string `cfg:"-"`
 	Empty    string `cfg:""`
 	unexport string `cfg:"unexported"` //nolint:unused // checks unexported fields are ignored
 }
 type testValidated struct {
 	Value string `cfg:"value"`
 }
 func (c *testValidated) validate(key string, value any) error {
 	if key == "value" && value.(string) == "bad" {
 		return errRejected
 	}
 	return nil
 }
 func TestKeysOf(t *testing.T) {
 	got := keysOf(&testSection{})
 	want := []string{"on", "str", "int", "int8", "uint", "uint16", "float",
 		"float32", "dur", "size", "when", "slice", "items", "inner"}
 	if !reflect.DeepEqual(got, want) {
 		t.Errorf("keysOf = %v; want %v", got, want)
 	}
 }
 func TestFieldsOfSkipsUntagged(t *testing.T) {
 	for _, f := range fieldsOf(reflect.TypeOf(testSection{})) {
 		switch f.key {
 		case "", "-", "unexported":
 			t.Errorf("fieldsOf included field with key %q", f.key)
 		}
 		if f.key == "when" && f.section {
 			t.Error("time.Time field treated as a section")
 		}
 		if f.key == "inner" && !f.section {
 			t.Error("struct field not treated as a section")
 		}
 		if f.key == "items" && f.section {
 			t.Error("list of sections treated as a section")
 		}
 	}
 }
 func TestMapOf(t *testing.T) {
 	var s testSection
 	if err := applyDefaults(&s); err != nil {
 		t.Fatalf("applyDefaults: %v", err)
 	}
 	m := mapOf(s)
 	if got := m["str"]; got != "hello" {
 		t.Errorf("m[str] = %#v", got)
 	}
 	if got := m["on"]; got != true {
 		t.Errorf("m[on] (promoted) = %#v", got)
 	}
 	if got := m["dur"]; got != "1m30s" {
 		t.Errorf("m[dur] = %#v; want the duration as a string", got)
 	}
 	inner, ok := m["inner"].(map[string]any)
 	if !ok {
 		t.Fatalf("m[inner] = %#v; want nested map", m["inner"])
 	}
 	if inner["name"] != "inner" {
 		t.Errorf("m[inner][name] = %#v", inner["name"])
 	}
 	if _, ok := m["inner.name"]; ok {
 		t.Error("mapOf produced a flattened key")
 	}
 	// Pointer and value give the same result.
 	if !reflect.DeepEqual(m, mapOf(&s)) {
 		t.Error("mapOf(&s) != mapOf(s)")
 	}
 }
 func TestFlattenOfPrefix(t *testing.T) {
 	m := flattenOf("p", testSection{
 		Dur: 90 * time.Second,
 	})
 	for _, k := range []string{"p.str", "p.on", "p.inner.name"} {
 		if _, ok := m[k]; !ok {
 			t.Errorf("flattenOf missing %q in %v", k, m)
 		}
 	}
 	if got := m["p.dur"]; got != "1m30s" {
 		t.Errorf("m[p.dur] = %#v; want the duration as a string", got)
 	}
 }
 func TestStructValuePanics(t *testing.T) {
 	defer func() {
 		if recover() == nil {
 			t.Error("structValue(int) did not panic")
 		}
 	}()
 	structValue(42)
 }
 func TestApplyDefaults(t *testing.T) {
 	var s testSection
 	if err := applyDefaults(&s); err != nil {
 		t.Fatalf("applyDefaults: %v", err)
 	}
 	want := testSection{
 		testEmbedded: testEmbedded{
 			On: true,
 		},
 		Str:   "hello",
 		Int:   -42,
 		Uint:  7,
 		Float: 1.5,
 		Dur:   90 * time.Second,
 		Size:  32 << 10,
 		Inner: testInner{
 			Name: "inner",
 		},
 	}
 	if !reflect.DeepEqual(s, want) {
 		t.Errorf("applyDefaults = %+v; want %+v", s, want)
 	}
 }
 func TestApplyDefaultsInvalid(t *testing.T) {
 	var bad struct {
 		N int `cfg:"n" default:"nope"`
 	}
 	err := applyDefaults(&bad)
 	if err == nil || !strings.Contains(err.Error(), "invalid default for n") {
 		t.Errorf("applyDefaults = %v; want invalid default error", err)
 	}
 	var badNested struct {
 		Sub struct {
 			D time.Duration `cfg:"d" default:"forever"`
 		} `cfg:"sub"`
 	}
 	err = applyDefaults(&badNested)
 	if err == nil || !strings.Contains(err.Error(), "sub.d") {
 		t.Errorf("applyDefaults nested = %v; want error naming sub.d", err)
 	}
 }
 func TestTypeOf(t *testing.T) {
 	tests := []struct {
 		key  string
 		want reflect.Kind
 	}{
 		{"str", reflect.String},
 		{"int", reflect.Int},
 		{"uint", reflect.Uint},
 		{"float", reflect.Float64},
 		{"on", reflect.Bool},
 		{"dur", reflect.String},  // durations are read as strings
 		{"size", reflect.String}, // byte sizes are read as strings
 		{"when", reflect.Struct},
 		{"slice", reflect.Slice},
 		{"items", reflect.Slice},
 		{"inner.name", reflect.String},
 	}
 	for _, tt := range tests {
 		got, err := typeOf(testSection{}, tt.key)
 		if err != nil || got != tt.want {
 			t.Errorf("typeOf(%q) = %v, %v; want %v", tt.key, got, err, tt.want)
 		}
 	}
 }
 func TestUnknownKeys(t *testing.T) {
 	keys := []string{
 		"missing",
 		"inner",         // section without sub-key
 		"str.sub",       // value with a sub-key
 		"inner.missing", // unknown nested key
 		"NoTag",         // untagged field
 		"unexported",
 		"",
 	}
 	var s testSection
 	for _, key := range keys {
 		if k, err := typeOf(s, key); !errors.Is(err, errUnknownKey) ||
 			k != reflect.Invalid {
 			t.Errorf("typeOf(%q) = %v, %v; want Invalid, errUnknownKey", key, k, err)
 		}
 		if v, err := valueOf(s, key); !errors.Is(err, errUnknownKey) || v != nil {
 			t.Errorf("valueOf(%q) = %v, %v; want nil, errUnknownKey", key, v, err)
 		}
 		if err := setValue(&s, key, "x"); !errors.Is(err, errUnknownKey) {
 			t.Errorf("setValue(%q) = %v; want errUnknownKey", key, err)
 		}
 	}
 }
 func TestSetValue(t *testing.T) {
 	tests := []struct {
 		key   string
 		value any
 		want  any
 	}{
 		// strings are parsed
 		{"str", "world", "world"},
 		{"on", "false", false},
 		{"int", "123", 123},
 		{"int8", "-128", int8(-128)},
 		{"uint", "9", uint(9)},
 		{"uint16", "65535", uint16(65535)},
 		{"float", "2.25", 2.25},
 		{"float32", "0.5", float32(0.5)},
 		{"dur", "1h30m", "1h30m0s"}, // durations are given out as strings
 		{"size", "1 mb", "1MB"},     // byte sizes are given out as strings
 		{"size", "1.5KiB", "1536B"},
 		{"size", "4096", "4KiB"},
 		{"size", "-1", "-1"}, // no limit
 		{"slice", " a, b ,, c ", []string{"a", "b", "c"}},
 		{"slice", "", []string(nil)},
 		{"inner.name", "nested", "nested"},
 		// values of the field's type are used as-is
 		{"int", 5, 5},
 		{"on", true, true},
 		{"dur", 3 * time.Second, "3s"},
 		{"size", util.ByteSize(2048), "2KiB"},
 		{"slice", []string{"a", "b"}, []string{"a", "b"}},
 		{"slice", []string{}, []string(nil)}, // empty lists are stored as nil
 		// lists of sections are given out as lists of maps; items start
 		// from their defaults and their values are parsed
 		{
 			"items",
 			`[{"n": "1", "wait": "2s"}, {"name": "b", "n": 2}]`,
 			[]map[string]any{
 				{"name": "item", "n": 1, "wait": "2s"},
 				{"name": "b", "n": 2, "wait": "0s"},
 			},
 		},
 		{"items", []any{map[string]any{"name": "a", "n": 3}}, []map[string]any{
 			{"name": "a", "n": 3, "wait": "0s"},
 		}},
 		{"items", []map[string]any{{"name": "a"}}, []map[string]any{
 			{"name": "a", "n": 0, "wait": "0s"},
 		}},
 		{"items", []testItem{{
 			Name: "a",
 			N:    4,
 		}}, []map[string]any{
 			{"name": "a", "n": 4, "wait": "0s"},
 		}},
 		{"items", "", []map[string]any(nil)},
 		{"items", "[]", []map[string]any(nil)},
 		{"items", []testItem{}, []map[string]any(nil)},
 		{"items", nil, []map[string]any(nil)},
 		{"when", time.Unix(0, 0).UTC(), time.Unix(0, 0).UTC()},
 	}
 	for _, tt := range tests {
 		var s testSection
 		if err := setValue(&s, tt.key, tt.value); err != nil {
 			t.Errorf("setValue(%q, %#v): %v", tt.key, tt.value, err)
 			continue
 		}
 		got, err := valueOf(s, tt.key)
 		if err != nil || !reflect.DeepEqual(got, tt.want) {
 			t.Errorf(
 				"after setValue(%q, %#v): valueOf = %#v, %v; want %#v",
 				tt.key,
 				tt.value,
 				got,
 				err,
 				tt.want,
 			)
 		}
 	}
 }
 func TestSetValueErrors(t *testing.T) {
 	tests := []struct {
 		key   string
 		value any
 		want  string
 	}{
 		{"int", "abc", "unable to convert"},
 		{"int8", "200", "unable to convert"}, // out of range
 		{"uint", "-1", "unable to convert"},
 		{"float", "x", "unable to convert"},
 		{"on", "maybe", "unable to convert"},
 		{"dur", "soon", "unable to convert to duration"},
 		{"items", "not json", "unable to convert to a list of config.testItem"},
 		{"items", `[{"bogus": 1}]`, `item 0: config: unknown configuration key "bogus"`},
 		{"items", `[{"n": "x"}]`, "item 0: n: unable to convert"},
 		{"items", []any{"a"}, "item 0: config: unsupported type string"},
 		{"items", 5, "unsupported type int"},
 		{"size", "lots", "unable to convert to byte size"},
 		{"size", "-1KB", "unable to convert to byte size"},
 		{"size", "16EiB", "too large"},
 		{"when", "2024-01-01", "unsupported type string"},
 		{"int", 1.5, "unsupported type float64"},
 		{"int", int64(1), "unsupported type int64"},
 		{"str", nil, "unsupported type <nil>"},
 	}
 	for _, tt := range tests {
 		var s testSection
 		before := s
 		err := setValue(&s, tt.key, tt.value)
 		if err == nil || !strings.Contains(err.Error(), tt.want) {
 			t.Errorf(
 				"setValue(%q, %#v) = %v; want error containing %q",
 				tt.key,
 				tt.value,
 				err,
 				tt.want,
 			)
 		}
 		if !reflect.DeepEqual(s, before) {
 			t.Errorf(
 				"setValue(%q, %#v) modified the section on error", tt.key, tt.value,
 			)
 		}
 	}
 }
 // A []string is given out as a copy, so changing it leaves the section as it
 // was.
 func TestValueOfSliceIsACopy(t *testing.T) {
 	s := testSection{
 		Slice: []string{"a", "b"},
 	}
 	got, err := valueOf(s, "slice")
 	if err != nil {
 		t.Fatal(err)
 	}
 	got.([]string)[0] = "changed"
 	if s.Slice[0] != "a" {
 		t.Errorf("changing valueOf's slice changed the section: %v", s.Slice)
 	}
 }
 func TestSetValueValidator(t *testing.T) {
 	var s testValidated
 	if err := setValue(&s, "value", "good"); err != nil || s.Value != "good" {
 		t.Errorf("setValue(good) = %v, Value = %q", err, s.Value)
 	}
 	if err := setValue(&s, "value", "bad"); !errors.Is(err, errRejected) {
 		t.Errorf("setValue(bad) = %v; want errRejected", err)
 	}
 	if s.Value != "good" {
 		t.Errorf("rejected value was stored: %q", s.Value)
 	}
 	// The validator of the section holding the field is used for nested keys.
 	var outer struct {
 		V testValidated `cfg:"v"`
 	}
 	if err := setValue(&outer, "v.value", "bad"); !errors.Is(err, errRejected) {
 		t.Errorf("nested setValue(bad) = %v; want errRejected", err)
 	}
 }
 // parseString handles string fields too, although convert assigns a string
 // to a string field before reaching it.
 func TestParseStringString(t *testing.T) {
 	v, err := parseString("text", reflect.TypeOf(""))
 	if err != nil || v.Interface() != "text" {
 		t.Errorf("parseString(text, string) = %v, %v", v, err)
 	}
 }
@@ -0,0 +1,47 @@
 # EU Supply
 A scraper for the public tenders on EU Supply's UK portal,
 [uk.eu-supply.com](https://uk.eu-supply.com). The site (Mercell CTM) has no
 API, so the client posts its public search form and reads the HTML. It
 replaces an old Node scraper, which also covered the Sellafield and Shared
 Systems portals; only the UK portal is still used.
 ## Requests
 | Request | Path | Gives |
 |---|---|---|
 | Search | `POST /ctm/Supplier/PublicTenders` | Up to 100 tenders a page: id, reference, title, publication date, response deadline, process, buyers, countries |
 | Tender page | `GET /ctm/Supplier/PublicPurchase/<PID>/0/0?returnUrl=ctm/Supplier/PublicTenders&b=UK` | Description, type of contract, CPV codes, buyer address, lots |
 - The search filters on the **publication date**, by whole UK days
  (`SearchFilter.FromDate`/`ToDate` as `DD/MM/YYYY`, both included). Only
  issued tenders are searched (`SearchFilter.PublishType` 1), not awarded
  ones.
 - The form must be posted whole, with the portal's own defaults
  (`searchForm`). With the old scraper's values (`OperatorId` 1 and an
  empty `SavedCategoryId`), the portal answers 500.
 - Times are UK time. Each date cell's tooltip `title` holds the full time
  (`21/10/2026 12:45:00`); the cell's text has only the date.
 - A tender's PID (its pages' id) is only in its search result's link
  (`rwlentrance_s.asp?PID=…`), which redirects to its page.
 - A tender page's fields are each a `div.ctm-content-label` (`CPV codes:`)
  followed by a `<p>` with the value.
 - The system certificate authorities are enough: the old scraper's own CA
  file isn't needed.
 ## Rate limit
 The site publishes no rate limit, so the client sets its own: it makes one
 request at a time, waiting a random time from the minimum to the maximum gap
 (`WithRequestGap`; the source's `request_min_gap` and `request_max_gap`,
 2s to 10s by default) after each before making the next.
 ## Opportunities
 `Convert` makes one opportunity per tender, with the source `EU Supply - UK`
 and the UUID `eusupply-uk-<tender id>`. It has one release (id: the tender
 id), dated its publication, tagged `tender`, whose JSON is the `Tender` as
 read. CPV codes are stored without their check digit, as Find a Tender's
 are. The package sets no marks: the fetcher marks the tenders it takes as
 repeats of another source's opportunities Hidden and Skipped (see
 `internal/fetcher/match.go`).