bwilliams
5cb3601e1f
Fetching: - Add `fetch service`, which runs a worker per enabled source until SIGINT/SIGTERM (internal/fetcher) - Add util.Worker: runs a task on an interval with a since-checkpoint that only advances on success, and a shutdown timeout for in-flight work; plus AwaitTask/AwaitTaskE for calls that ignore cancellation - Add fts_mock and sleeper sources - Sleeper: sleeps periodically to emulate work being done. Can be instructed to block the shutdown signal or handle it cleanly Mock FTS server (cmd/mock-fts, internal/mock/findATender): - Serves GET /api/1.0/ocdsReleasePackages with FTS paging, query validation, UK local time handling and per-client rate limiting - Generates fictional releases from an embedded catalog, moving procedures through planning, tender and award, and keeps them in a JSON Lines store that survives restarts - Add logging.AccessLog for combined/common access logs
JobSearch
JobSearch is a service for scraping for contract opportunities. It is primarily a re-write of the existing nodeJS JobSearch codebase.
Installation of dev tools
A number of tools have been used in the development of the project. Installation
of these tools can be done using go install.
To install the tools:
go install github.com/spf13/cobra-cli@latest
go install git.avsdev.uk/Go/go-ocds/cmd/ocdsgen@latest
Ensure that the GOBIN path ($HOME/go/bin) is added to the environment before running any of the tools.