Commit Graph
7 Commits
Author SHA1 Message Date
bwilliams c9f42b62ad Added websocket hub for handling websocket clients, with a wsserve command
which runs a plain TCP server for receiving websocket hub clients, with
server config, CSRF and rate limit middleware.

- internal/websocket: Hub pushing messages to WebSocket clients (ServeHTTP,
  example.com/ws upgrade with optional CSRF check) and plain TCP clients (ServeTCP,
  newline-delimited JSON envelopes); ping/pong keepalive, per-client message
  rate limit, slow consumer handling, graceful Shutdown, debug logging of
  messages sent and received
- internal/websocket/tcpserver: standalone TCP server for a hub, with accept
  back-off, a per-IP connection limit (server.websocket.connection_limit) and
  graceful shutdown
- internal/cli/wsserve: `wsserve` command running the TCP server, with
  --listen/--host/--port overrides
- internal/middleware: keyed token bucket rate limiter, real client IP from
  trusted proxies
- internal/httpx: RFC 9457 problem responses, request IDs in context
- internal/config: server section (host, port, shutdown timeout, rate limiter
  idle TTL, websocket); util.ByteSize settings (e.g. "32KiB"); lists of
  sections as single keys; empty lists stored as nil; isExplicitlySet for
  printLoadedConfig; JobSearchConfigSource renamed to JSCSourcesSource;
  TestSections checks every section's methods
- internal/util: ByteSize, parsed and formatted with go-humanize
2026-10-14 15:55:25 +01:00
bwilliams 5ec4b65b91 Added unit tests for CLI 2026-10-13 13:22:22 +01:00
bwilliams 045492a148 First pass implementation of an SQLite backing store for fetched opportunities 2026-10-13 12:52:17 +01:00
bwilliams 005b335371 Minor fixes:
- Fixed function name typo
- Fetcher/Run now returns a combination of errors if any workers error out. All other workers are ordered to shutdown if any one does error out
- findATender now returns a pointer instead of an instance from New
2026-10-09 18:26:10 +01:00
bwilliams 399eccf2de Find a Tender API client updates
- Convert OCDS releases to types.Opportunity (internal/types); FetchOnce
  and FetchAll now return opportunities, built from each ocid's latest
  release
- Include each item's main classification as well as its additional ones
- Return ErrUnexpectedStatus for non-200 responses and
  ErrRateLimitExceeded for a repeated 429; close discarded bodies
- Stop fetchAll on a repeated next link and close each page's body once
  read rather than deferring in the loop
- Default a zero Since to 24 hours ago
- Merge date repair and releaseIDs into api.go
- Filter --planning/--opportunities/--awards on each procedure's current
  stage, using OCDS release tags; --opportunities previously matched a
  nonexistent "opportunity" tag and returned nothing
- Reject conflicting filters before fetching (cobra flag group)
2026-10-09 17:56:55 +01:00
bwilliams e43b7a8370 Added fetch script for UK Gov FindATender service which is based on OCDS (with issues...) 2026-10-09 10:08:11 +01:00
bwilliams 6436253629 Refactored main.go into cmd/jobsearch/main.go (in case separate commands are introduced later) which has required the options mechanism to change slightly as well as the logging. 2026-10-09 09:29:42 +01:00