Commit Graph
9 Commits
Author SHA1 Message Date
bwilliams 5ec4b65b91 Added unit tests for CLI 2026-10-13 13:22:22 +01:00
bwilliams 045492a148 First pass implementation of an SQLite backing store for fetched opportunities 2026-10-13 12:52:17 +01:00
bwilliams 6e6dec68c0 Fixed lint issues and missing documentation 2026-10-13 12:21:32 +01:00
bwilliams 005b335371 Minor fixes:
- Fixed function name typo
- Fetcher/Run now returns a combination of errors if any workers error out. All other workers are ordered to shutdown if any one does error out
- findATender now returns a pointer instead of an instance from New
2026-10-09 18:26:10 +01:00
bwilliams 5cb3601e1f Added fetch service and a mock Find a Tender server
Fetching:
- Add `fetch service`, which runs a worker per enabled source until
  SIGINT/SIGTERM (internal/fetcher)
- Add util.Worker: runs a task on an interval with a since-checkpoint
  that only advances on success, and a shutdown timeout for in-flight
  work; plus AwaitTask/AwaitTaskE for calls that ignore cancellation
- Add fts_mock and sleeper sources
- Sleeper: sleeps periodically to emulate work being done. Can be
  instructed to block the shutdown signal or handle it cleanly

Mock FTS server (cmd/mock-fts, internal/mock/findATender):
- Serves GET /api/1.0/ocdsReleasePackages with FTS paging, query
  validation, UK local time handling and per-client rate limiting
- Generates fictional releases from an embedded catalog, moving
  procedures through planning, tender and award, and keeps them in a
  JSON Lines store that survives restarts
- Add logging.AccessLog for combined/common access logs
2026-10-09 18:02:32 +01:00
bwilliams 399eccf2de Find a Tender API client updates
- Convert OCDS releases to types.Opportunity (internal/types); FetchOnce
  and FetchAll now return opportunities, built from each ocid's latest
  release
- Include each item's main classification as well as its additional ones
- Return ErrUnexpectedStatus for non-200 responses and
  ErrRateLimitExceeded for a repeated 429; close discarded bodies
- Stop fetchAll on a repeated next link and close each page's body once
  read rather than deferring in the loop
- Default a zero Since to 24 hours ago
- Merge date repair and releaseIDs into api.go
- Filter --planning/--opportunities/--awards on each procedure's current
  stage, using OCDS release tags; --opportunities previously matched a
  nonexistent "opportunity" tag and returned nothing
- Reject conflicting filters before fetching (cobra flag group)
2026-10-09 17:56:55 +01:00
bwilliams e43b7a8370 Added fetch script for UK Gov FindATender service which is based on OCDS (with issues...) 2026-10-09 10:08:11 +01:00
bwilliams 6436253629 Refactored main.go into cmd/jobsearch/main.go (in case separate commands are introduced later) which has required the options mechanism to change slightly as well as the logging. 2026-10-09 09:29:42 +01:00
bwilliams e715024cbd Initial project 2026-09-23 14:32:50 +01:00