Made each source start from its newest stored release, with --since as an override

Without --since, fetcher service and job-search serve start each source
from the newest release stored for it (the new store.LatestReleaseDate,
by Source), so a restart carries on where it left off instead of fetching
the last day again; with nothing stored, the client's default still
applies. --since overrides it for every source. The Find a Tender source's
name is now a constant, as the Source its opportunities are stored with.

The README's guide to adding a source is brought up to date with the
current worker, start and config type names.
This commit is contained in:
bwilliams committed 2026-10-16 11:37:09 +01:00
1 parent 3c4fde8dfd
commit 284a589627
10 files changed
+256 -22

No files matched your search

+5 -1
View File
@@ -12,6 +12,10 @@ import (
"git.avsdev.uk/Go/job-search/internal/util"
)
// findATenderSource is the Source of the opportunities Find a Tender's
// converter makes, and so of the ones the Find a Tender source fetches.
const findATenderSource = "FindATender"
// runFindATender polls Find a Tender every configured interval until ctx
// is cancelled, passing what each run fetches to save.
func runFindATender(
@@ -27,7 +31,7 @@ func runFindATender(
worker := util.Worker{
Logger: tagged(logger),
Name: "FindATender",
Name: findATenderSource,
Interval: conf.Interval,
Task: func(ctx context.Context, since time.Time) error {
opportunities, err := fetchFindATender(ctx, logger, conf, since)