Commit Graph
70 Commits
Author SHA1 Message Date
bwilliams 045492a148 First pass implementation of an SQLite backing store for fetched opportunities 2026-10-13 12:52:17 +01:00
bwilliams 9b0e6dccfe Added source and user marks to opportunities type. Any ignored errors are deferred to //nolint with a comment instead of being silently dropped 2026-10-13 12:40:11 +01:00
bwilliams 6e6dec68c0 Fixed lint issues and missing documentation 2026-10-13 12:21:32 +01:00
bwilliams 7fd321df97 Added unit tests, documentation and function descriptions across the board. Increased cyclop complexity threshold to 15 which is a bit more sane than 10 and excluded _test.go files from linters. 2026-10-09 18:27:51 +01:00
bwilliams 005b335371 Minor fixes:
- Fixed function name typo
- Fetcher/Run now returns a combination of errors if any workers error out. All other workers are ordered to shutdown if any one does error out
- findATender now returns a pointer instead of an instance from New
2026-10-09 18:26:10 +01:00
bwilliams 5cb3601e1f Added fetch service and a mock Find a Tender server
Fetching:
- Add `fetch service`, which runs a worker per enabled source until
  SIGINT/SIGTERM (internal/fetcher)
- Add util.Worker: runs a task on an interval with a since-checkpoint
  that only advances on success, and a shutdown timeout for in-flight
  work; plus AwaitTask/AwaitTaskE for calls that ignore cancellation
- Add fts_mock and sleeper sources
- Sleeper: sleeps periodically to emulate work being done. Can be
  instructed to block the shutdown signal or handle it cleanly

Mock FTS server (cmd/mock-fts, internal/mock/findATender):
- Serves GET /api/1.0/ocdsReleasePackages with FTS paging, query
  validation, UK local time handling and per-client rate limiting
- Generates fictional releases from an embedded catalog, moving
  procedures through planning, tender and award, and keeps them in a
  JSON Lines store that survives restarts
- Add logging.AccessLog for combined/common access logs
2026-10-09 18:02:32 +01:00
bwilliams dcb714dc19 Some bug fixes:
- logging: LogToFile shadowed the package-level fileLogger, so
  IsFileLogger was always false and Shutdown never closed the file
- logging: replace IsJSONHandler's unsafe reflection into slog
  internals with a *slog.JSONHandler type check
- cli: fail if the log file can't be opened instead of silently
  logging to the terminal; return the log shutdown error from Finish
- build.sh: -X flags used the wrong module path, so the version, build
  time and commit were never stamped
2026-10-09 17:58:48 +01:00
bwilliams 399eccf2de Find a Tender API client updates
- Convert OCDS releases to types.Opportunity (internal/types); FetchOnce
  and FetchAll now return opportunities, built from each ocid's latest
  release
- Include each item's main classification as well as its additional ones
- Return ErrUnexpectedStatus for non-200 responses and
  ErrRateLimitExceeded for a repeated 429; close discarded bodies
- Stop fetchAll on a repeated next link and close each page's body once
  read rather than deferring in the loop
- Default a zero Since to 24 hours ago
- Merge date repair and releaseIDs into api.go
- Filter --planning/--opportunities/--awards on each procedure's current
  stage, using OCDS release tags; --opportunities previously matched a
  nonexistent "opportunity" tag and returned nothing
- Reject conflicting filters before fetching (cobra flag group)
2026-10-09 17:56:55 +01:00
bwilliams 69620e5f7d Unit tests added for internal/config 2026-10-09 17:07:50 +01:00
bwilliams 09644d0691 Moved config set/unset to internal/config/modify.go as Store and Purge. 2026-10-09 17:07:06 +01:00
bwilliams 539e88e73c removed copyright/license header from all files since it was getting tedious. It remains in the cmd main.go and there is still the LICENSE file 2026-10-09 12:27:35 +01:00
bwilliams e43b7a8370 Added fetch script for UK Gov FindATender service which is based on OCDS (with issues...) 2026-10-09 10:08:11 +01:00
bwilliams ff33ef082a Refactored configuration to make sections less code intensive to add 2026-10-09 09:30:15 +01:00
bwilliams 8b0cecb51b Updated build script to produce release archives for git and to be more verbose 2026-10-09 09:30:08 +01:00
bwilliams dd26976f76 Moved package log to logging 2026-10-09 09:30:02 +01:00
bwilliams da81e6ccd9 Silence usage on commands that apply 2026-10-09 09:29:53 +01:00
bwilliams 6436253629 Refactored main.go into cmd/jobsearch/main.go (in case separate commands are introduced later) which has required the options mechanism to change slightly as well as the logging. 2026-10-09 09:29:42 +01:00
bwilliams de3ebf68c5 Added an unsafe reflect to detect if a log handler is JSON enabled or not 2026-09-29 11:55:45 +01:00
bwilliams a0fad30e99 Config to string, JSON and YAML methods added and updated print methods to use new methods 2026-09-29 11:55:11 +01:00
bwilliams e715024cbd Initial project v0.0.0 2026-09-23 14:32:50 +01:00