Added unit tests, documentation and function descriptions across the board. Increased cyclop complexity threshold to 15 which is a bit more sane than 10 and excluded _test.go files from linters.
This commit is contained in:
bwilliams committed
1 parent
005b335371
commit
7fd321df97
31 files changed
+4106
-1
No files matched your search
@@ -0,0 +1,115 @@
|
||||
# mock
|
||||
|
||||
Mock versions of the external services JobSearch fetches from, for local
|
||||
development and testing without hitting the real APIs or their rate limits.
|
||||
|
||||
| Package | Mocks | Command |
|
||||
|---|---|---|
|
||||
| `findATender` | UK Government Find a Tender service (FTS) OCDS API | `cmd/mock-fts` |
|
||||
|
||||
|
||||
## findATender
|
||||
A mock of the FTS release package endpoint:
|
||||
|
||||
```
|
||||
GET /api/1.0/ocdsReleasePackages?updatedFrom=2026-10-06T00:00:00&limit=5
|
||||
```
|
||||
|
||||
It generates realistic, fictional OCDS releases from a catalog of buyers,
|
||||
suppliers and tender subjects, keeps publishing new ones while it runs, and
|
||||
behaves like the real service: paging, query validation and rate limiting.
|
||||
The JobSearch `fts_mock` source points at it.
|
||||
|
||||
|
||||
### Running
|
||||
```bash
|
||||
go run ./cmd/mock-fts --addr :8080
|
||||
```
|
||||
|
||||
Then enable the `fts_mock` source in `config.yaml`:
|
||||
```yaml
|
||||
sources:
|
||||
fts_mock:
|
||||
enabled: true
|
||||
interval: 1m
|
||||
api_root: http://localhost:8080/api/1.0
|
||||
```
|
||||
|
||||
Useful flags (`go run ./cmd/mock-fts --help` lists them all):
|
||||
|
||||
| Flag | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `--addr` | `:8080` | Listen address |
|
||||
| `--base-url` | `http://localhost:8080` | Public base URL used in `uri` and `links.next` |
|
||||
| `--data-dir` | `./mock-data` | Where `releases.jsonl` is stored |
|
||||
| `--reset` | | Delete stored releases and regenerate the backfill |
|
||||
| `--backfill` | `72h` | History to generate on an empty data directory |
|
||||
| `--min-gap`, `--max-gap` | `30s`, `5m` | Range of time between generated releases |
|
||||
| `--rate-limit`, `--rate-window` | `12`, `5m` | Requests allowed per client per window |
|
||||
| `--access-log` | `-` (stdout) | Access log file |
|
||||
| `--log-format` | `combined` | Access log format: `combined` or `common` |
|
||||
| `--tz` | `Europe/London` | Time zone for query times with no offset |
|
||||
|
||||
For quick testing, shorten the gaps and raise the rate limit:
|
||||
```bash
|
||||
go run ./cmd/mock-fts --min-gap 5s --max-gap 10s --rate-limit 1000
|
||||
```
|
||||
|
||||
|
||||
### API behaviour
|
||||
Query parameters, as on the real service:
|
||||
|
||||
| Parameter | Format | Meaning |
|
||||
|---|---|---|
|
||||
| `updatedFrom` | `yyyy-MM-ddTHH:mm:ss` | Releases dated at or after this time |
|
||||
| `updatedTo` | `yyyy-MM-ddTHH:mm:ss` | Releases dated at or before this time (optional) |
|
||||
| `limit` | `1`–`100`, default `100` | Page size |
|
||||
| `cursor` | opaque | Next page; taken from `links.next` |
|
||||
|
||||
- Like FTS, query times are UK local time (BST in summer) unless they carry an
|
||||
explicit offset or `Z`. Release dates are stored and returned in UTC.
|
||||
- Results are in date order. When there are more, `links.next` holds the URL
|
||||
of the next page, with the original query and a `cursor`.
|
||||
- Releases dated after the current time are never returned.
|
||||
- Invalid parameters get `400 Bad Request` with a message, and methods other
|
||||
than `GET` get `405 Method Not Allowed`.
|
||||
- Each client (by remote IP) may make `--rate-limit` requests per
|
||||
`--rate-window`. Windows are fixed and start at the client's first request
|
||||
in the current clock hour. Over the limit the mock returns
|
||||
`429 Too Many Requests` with a `Retry-After` header and the body
|
||||
`Rate limit of N exceeded. Please retry after X seconds.`
|
||||
|
||||
|
||||
### Generated data
|
||||
- **Catalog** (`findATender/data/catalog.json`): the buyers, suppliers,
|
||||
tender subjects, procurement methods and classifications that releases are
|
||||
drawn from. It's embedded in the binary, so rebuild after changing it.
|
||||
Subjects refer to classifications by `{scheme}-{id}` keys (e.g.
|
||||
`CPV-90513000`). The mock refuses to start if the catalog has an empty pool
|
||||
or a key that doesn't resolve.
|
||||
- **Procedures**: a new release usually starts a new procedure, with a new
|
||||
`ocds-mock-` ocid, as a `tender` (or sometimes `planning`). About a third of
|
||||
the time it moves an open procedure on instead, from `planning` to `tender`,
|
||||
`tender` to `tenderUpdate` (a deadline extension) or `award`. So the same
|
||||
ocid appears in several releases, as it does on the real service.
|
||||
- **Store** (`<data-dir>/releases.jsonl`): every release is appended as one
|
||||
line of JSON, so releases survive restarts. On startup the mock reloads the
|
||||
file, skips any corrupt lines, and generates releases to fill the time since
|
||||
the last one (or `--backfill`, if that's shorter).
|
||||
|
||||
|
||||
### Code
|
||||
|
||||
| File | Contents |
|
||||
|---|---|
|
||||
| `doc.go` | Package documentation and data file names |
|
||||
| `catalog.go` | Loading and validating the embedded catalog |
|
||||
| `generator.go` | Making releases, catching up on startup, and the background publisher |
|
||||
| `store.go` | The in-memory, file-backed release store |
|
||||
| `handler.go` | The HTTP handler: query parsing, paging and cursors |
|
||||
| `ratelimit.go` | The per-client rate limiter and its middleware |
|
||||
| `ocds/` | Generated OCDS types (`ocds_gen.go`, from `ocdsgen`); don't edit by hand |
|
||||
| `testdata/` | A small catalog used by the tests |
|
||||
|
||||
The mock's `ocds` package is separate from `internal/findATender/ocds`, which
|
||||
the JobSearch client uses.
|
||||
Reference in new issue
Block a user