Added change batches, so clients can catch up on changed opportunities

Every save that stores new opportunities or new releases records them in a
change batch (migration 00008: opportunity_change_batches and
opportunity_change_batch_opportunities; the opportunities table is
unchanged), as does every change of marks. A batch is named by a random
SHA-256 hash, so it can't be guessed, and ordered by its created_at, which
is kept strictly increasing even if the clock goes back.

GET /api/opportunities pages now carry the newest batch, read before the
page, and GET /api/opportunities?changed_after=<batch> returns every
opportunity changed since that batch, unpaged, with the newest batch it
includes. Offset and limit can't be combined with it, and an unknown batch
is 410 Gone, telling the client to reload the whole list.

OpportunitiesUpdated now carries its save's batch, and a "large" flag: over
50 opportunities, new and updated are left empty and only the summary
counts them, so a big catch-up no longer sends tens of thousands of UUIDs.
This commit is contained in:
bwilliams committed 2026-10-16 15:25:48 +01:00
1 parent 51d6470649
commit 155b84caea
13 files changed
+699 -29

No files matched your search

@@ -0,0 +1,33 @@
-- +goose Up
-- +goose StatementBegin
-- A batch of changed opportunities: those one save stored for the first time
-- or gave new releases, or the one whose marks were changed. A client asks
-- for what changed after a batch it has seen by the batch's hash.
CREATE TABLE IF NOT EXISTS "opportunity_change_batches" (
"id" INTEGER PRIMARY KEY,
-- A random hash naming the batch to clients, so a batch can't be guessed
-- or worked out from another.
"hash" TEXT NOT NULL UNIQUE,
-- When the batch was recorded, in UTC. Each batch's is later than every
-- earlier batch's, so the batches after one are those created after it.
"created_at" DATETIME NOT NULL
);
CREATE INDEX IF NOT EXISTS "opportunity_change_batches_created_at"
ON "opportunity_change_batches" ("created_at");
-- The opportunities in each batch.
CREATE TABLE IF NOT EXISTS "opportunity_change_batch_opportunities" (
"batch_id" INTEGER NOT NULL
REFERENCES "opportunity_change_batches" ("id") ON DELETE CASCADE,
"opportunity_uuid" TEXT NOT NULL
REFERENCES "opportunities" ("uuid") ON DELETE CASCADE,
PRIMARY KEY ("batch_id", "opportunity_uuid")
) WITHOUT ROWID;
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
DROP TABLE IF EXISTS "opportunity_change_batch_opportunities";
DROP TABLE IF EXISTS "opportunity_change_batches";
-- +goose StatementEnd