Go/job-search
1
0 Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Added CODE_CONVENTIONS.md and linters to enforce it, and applied it across the codebase

Browse Source 
CODE_CONVENTIONS.md sets out the conventions for Go code in this
repository: naming, layout, comments, errors, logging, configuration and
tests. .golangci.json enables the linters that check the parts that can be
checked (each limited to the rules the conventions use); the rest is
applied by hand.

Applied across the codebase:
- Naming: full-word names in application code, initialisms in one case
  (BuyerID, PrintJSON, PrintYAML, ocdsGetBuyerID), no names shadowing
  built-ins (min/max are minGap/maxGap), mustRaw renamed rawJSON as it
  returns nil rather than panicking, and config.ConfigLevelToLogLevel
  renamed LevelToLogLevel.
- Layout: lines kept to 80 characters where possible, assignments taken
  out of if and for statements that don't fit, wrapped arguments all on
  one line or one per line, steps separated by blank lines, no else after
  a return, function and struct literals wrapped.
- Comments: doc comments on every exported name (including each section's
  Map, Flatten, TypeOf, ValueOf and SetValue), section banners, British
  spelling.
- Errors: a created error starts with its package name, a wrapping error
  adds only what was being done, sentinels in one commented var block,
  errors.Is and errors.As in place of == and type assertions, and no
  err := shadowing of a named result. That fixes the mock server losing
  its access log's close error.
- Logging: the fetcher passes its workers an untagged logger, so
  findatender's lines are tagged with their own component rather than
  with fetcher's as well; cmd/mock-fts logs "stopped" with slog.
- Configuration: util.WithTimeout (moved from the websocket package)
  applies a timeout of 0 or less as no limit, used for the websocket
  write and ping, TCP close and tcpserver shutdown timeouts; every setting
  documents what 0 or empty means, including the rate limit rule's keys
  in internal/config/README.md.
- Tests: related tests merged into table-driven ones with named cases.

Separately from CODE_CONVENTIONS.md, internal/store/README.md gains a soft
SQL style convention for the queries in internal/store/queries.go, and the
queries are reformatted to follow it.
This commit is contained in:
bwilliams committed 2026-10-15 16:26:42 +01:00
1 parent 465358609a
commit 67215a6aaa
104 files changed
+3189 -1206
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

internal/store/README.md
+12
View File
@@ -78,6 +78,18 @@ Reads select whole rows (`SELECT *`, `t.*`, `c.*`) which map to a Go struct. sql
`ListOpportunities` is assembled by `listOpportunitiesQuery` from fragments in `queries.go`: a base `SELECT`, one `filterOpportunity*` clause per filter that is set, and the ordering and paging tail. Each filter fragment takes exactly one argument, except the `filterOpportunityMark*` fragments, which take none (`MarkUnset` puts `NOT` in front of them). Adding a filter means one fragment and one line in `listOpportunitiesQuery`. The classifications and the marks for a page of opportunities are then loaded with one `IN (...)` query each.
### SQL style
A soft convention for the SQL in `queries.go`, followed so the queries look alike and read easily. A query that reads better another way can depart from it.
- A statement starts on the line after the opening backtick. A fragment combined into another query (`filterOpportunity*`) starts straight after it.
- Keywords are uppercase (`SELECT`, `INSERT INTO`, `ON CONFLICT`); table and column names are the lowercase snake_case of the schema.
- A short statement may stay on one line (`SELECT t.* FROM opportunities t WHERE t.uuid = ?`). A longer one has one clause per line, each at the start of the line: `SELECT`, `FROM`, `JOIN`, `WHERE`, `ORDER BY`, `LIMIT`.
- A column list that doesn't fit on one line goes inside its brackets, indented one tab and filled to about 80 characters per line. The `VALUES` list wraps at the same columns, so each line of values matches the line of columns above it.
- `ON CONFLICT … DO UPDATE SET` puts one `column = excluded.column` per line, indented one tab.
- A subquery (`EXISTS (…)`, `IN (…)`) is indented one tab inside its brackets, with the closing bracket at the start of a line.
- Tables are aliased consistently: `t` for `opportunities`, `c` for `classification_codes`, `tc` for `opportunity_classifications`, `m` for `opportunity_marks`.
## Store API
```go
internal/store/migrate.go
+5 -17
View File
@@ -17,21 +17,6 @@ import (
//go:embed migrations/*.sql
var migrationsFS embed.FS
// ErrNeedsMigration is returned by Open when the database has pending
// migrations. Apply them with the migrate command, or pass
// WithAutoMigrate.
var ErrNeedsMigration = errors.New("store: database needs migrating")
// ErrSchemaTooNew is returned by Open when the database has migrations
// this build doesn't know about, i.e. it was migrated by a newer build.
var ErrSchemaTooNew = errors.New("store: database schema is newer than this build")
// ErrWouldEmptyDatabase is returned by StepDown when the migration would drop
// all of the tables and thus empty the database.
var ErrWouldEmptyDatabase = errors.New(
"migration cancelled; database would be emptied by undoing all migrations",
)
// NewMigrator returns a goose Provider over the store's embedded migrations,
// for applying, rolling back or inspecting them. Each migration runs in its
// own transaction, so a failed migration leaves the database at the previous
@@ -96,13 +81,15 @@ func checkNotNewer(ctx context.Context, provider *goose.Provider) error {
if err != nil {
return err
}
if latest := LatestVersion(provider); current > latest {
return fmt.Errorf(
"database is at version %d but this build only knows up to %d; use a newer build",
"store: database is at version %d but this build only knows up to %d; use a newer build",
current,
latest,
)
}
return nil
}
@@ -116,7 +103,7 @@ func prepareSchema(ctx context.Context, db *sql.DB, opts options) error {
if opts.autoMigrate {
if _, err := provider.Up(ctx); err != nil {
return fmt.Errorf("store: migrating database: %w", err)
return fmt.Errorf("migrating database: %w", err)
}
}
@@ -184,6 +171,7 @@ func StepDown(
applied++
}
}
all := steps == 0 || steps >= applied
if all && !confirmDrop {
return results, ErrWouldEmptyDatabase
internal/store/migrate_test.go
+20 -18
View File
@@ -29,10 +29,12 @@ func newMigrator(t *testing.T) (*goose.Provider, *sql.DB) {
t.Cleanup(func() {
db.Close()
})
m, err := store.NewMigrator(db)
if err != nil {
t.Fatal(err)
}
return m, db
}
@@ -105,7 +107,8 @@ func TestOpenRejectsNewerSchema(t *testing.T) {
"check": nil,
"auto-migrate": {store.WithAutoMigrate()},
} {
if _, err := store.Open(ctx, path, opts...); !errors.Is(err, store.ErrSchemaTooNew) {
_, err := store.Open(ctx, path, opts...)
if !errors.Is(err, store.ErrSchemaTooNew) {
t.Errorf("%s: err = %v, want ErrSchemaTooNew", name, err)
}
}
@@ -135,10 +138,12 @@ func TestMigrateDownAllAndUpAgain(t *testing.T) {
t.Fatalf("down to 0: %v", err)
}
var tables int
if err := db.QueryRow(`
query := `
SELECT count(*) FROM sqlite_master
WHERE type IN ('table', 'index', 'trigger')
AND name NOT LIKE 'sqlite_%' AND name != 'goose_db_version'`).Scan(&tables); err != nil {
AND name NOT LIKE 'sqlite_%'
AND name != 'goose_db_version'`
if err := db.QueryRow(query).Scan(&tables); err != nil {
t.Fatal(err)
}
if tables != 0 {
@@ -229,7 +234,10 @@ func TestDownAllRequiresConfirmation(t *testing.T) {
}
hasErrored := false
for name, steps := range map[string]int{"steps 0": 0, "more steps than applied": 99} {
for name, steps := range map[string]int{
"steps 0": 0,
"more steps than applied": 99,
} {
_, err := store.StepDown(ctx, m, steps, false)
if !errors.Is(err, store.ErrWouldEmptyDatabase) {
t.Errorf("%s: %s", name, err)
@@ -277,20 +285,11 @@ func TestNewerDatabaseIsRefused(t *testing.T) {
t.Fatal(err)
}
if _, err := store.StepUp(
ctx,
m,
0,
); err == nil ||
if _, err := store.StepUp(ctx, m, 0); err == nil ||
!strings.Contains(err.Error(), "newer build") {
t.Errorf("up: err = %v, want a newer-build error", err)
}
if _, err := store.StepDown(
ctx,
m,
1,
false,
); err == nil ||
if _, err := store.StepDown(ctx, m, 1, false); err == nil ||
!strings.Contains(err.Error(), "newer build") {
t.Errorf("down: err = %v, want a newer-build error", err)
}
@@ -307,7 +306,9 @@ func TestFailedMigrationIsRolledBack(t *testing.T) {
// Occupy migration 4's table name with a view. Its CREATE TABLE IF NOT
// EXISTS then skips, and the CREATE INDEX that follows fails because
// views can't be indexed, part-way through the migration.
if _, err := db.Exec(`CREATE VIEW classification_codes AS SELECT 1 AS x`); err != nil {
if _, err := db.Exec(
`CREATE VIEW classification_codes AS SELECT 1 AS x`,
); err != nil {
t.Fatal(err)
}
@@ -319,8 +320,9 @@ func TestFailedMigrationIsRolledBack(t *testing.T) {
t.Errorf("version after failure: %d, want 3", v)
}
var indexes int
db.QueryRow(`SELECT count(*) FROM sqlite_master WHERE name LIKE 'classification_codes_%'`).
Scan(&indexes)
query := `SELECT count(*) FROM sqlite_master
WHERE name LIKE 'classification_codes_%'`
db.QueryRow(query).Scan(&indexes)
if indexes != 0 {
t.Errorf("failed migration left %d objects behind", indexes)
}
internal/store/queries.go
+18 -12
View File
@@ -18,23 +18,27 @@ package store
// ---------- opportunities ----------
// selectOpportunityLatestReleaseDate reads the date SaveOpportunity compares against.
// selectOpportunityLatestReleaseDate reads the date SaveOpportunity compares
// against.
const selectOpportunityLatestReleaseDate = `
SELECT latest_release_date FROM opportunities WHERE uuid = ?`
// upsertOpportunity inserts or updates a opportunity. Named parameters (types.Opportunity).
// upsertOpportunity inserts or updates a opportunity. Named parameters
// (types.Opportunity).
//
// An UPDATE on conflict, not a REPLACE: REPLACE would delete the row first,
// cascading to the opportunity's releases and changing its id.
const upsertOpportunity = `
INSERT INTO opportunities (
uuid, uuid_type, title, status, description, buyer_name, buyer_id,
main_category, procurement_method, multi_lot, sme_suitable, vcse_suitable,
start_date, end_date, gross_value, has_documents, latest_release_date
main_category, procurement_method, multi_lot, sme_suitable,
vcse_suitable, start_date, end_date, gross_value, has_documents,
latest_release_date
) VALUES (
:uuid, :uuid_type, :title, :status, :description, :buyer_name, :buyer_id,
:main_category, :procurement_method, :multi_lot, :sme_suitable, :vcse_suitable,
:start_date, :end_date, :gross_value, :has_documents, :latest_release_date
:main_category, :procurement_method, :multi_lot, :sme_suitable,
:vcse_suitable, :start_date, :end_date, :gross_value, :has_documents,
:latest_release_date
)
ON CONFLICT (uuid) DO UPDATE SET
uuid_type = excluded.uuid_type,
@@ -162,7 +166,8 @@ INSERT INTO opportunity_release_tags (uuid, release_id, tag)
VALUES (:uuid, :release_id, :tag)
ON CONFLICT DO NOTHING`
// selectReleaseTagsByOpportunity reads the tags of all of a opportunity's releases.
// selectReleaseTagsByOpportunity reads the tags of all of a opportunity's
// releases.
const selectReleaseTagsByOpportunity = `
SELECT *
FROM opportunity_release_tags
@@ -180,13 +185,14 @@ ORDER BY release_id, tag`
const upsertClassificationCodeReturningID = `
INSERT INTO classification_codes (is_cpv, code, classification)
VALUES (:is_cpv, :code, :classification)
ON CONFLICT (is_cpv, code, classification) DO UPDATE SET is_cpv = excluded.is_cpv
ON CONFLICT (is_cpv, code, classification)
DO UPDATE SET is_cpv = excluded.is_cpv
RETURNING id`
// ---------- opportunity_classifications ----------
// deleteOpportunityClassifications clears a opportunity's mappings before they are
// rewritten.
// deleteOpportunityClassifications clears a opportunity's mappings before they
// are rewritten.
const deleteOpportunityClassifications = `
DELETE FROM opportunity_classifications WHERE opportunity_uuid = ?`
@@ -197,8 +203,8 @@ VALUES (?, ?)
ON CONFLICT DO NOTHING`
// selectClassificationsForOpportunities reads the classifications of several
// opportunities. "IN (?)" is expanded by sqlx.In, which must be followed by Rebind.
// c.* must map entirely onto types.ClassificationCode.
// opportunities. "IN (?)" is expanded by sqlx.In, which must be followed by
// Rebind. c.* must map entirely onto types.ClassificationCode.
const selectClassificationsForOpportunities = `
SELECT tc.opportunity_uuid, c.*
FROM opportunity_classifications tc
internal/store/queries_test.go
+81 -28
View File
@@ -30,26 +30,50 @@ type queryCheck struct {
// missing here.
var checks = map[string]queryCheck{
// opportunities
"selectOpportunityLatestReleaseDate": {sql: selectOpportunityLatestReleaseDate},
"upsertOpportunity": {sql: upsertOpportunity, named: true},
"selectOpportunityByUUID": {sql: selectOpportunityByUUID},
"selectOpportunityLatestReleaseDate": {
sql: selectOpportunityLatestReleaseDate,
},
"upsertOpportunity": {sql: upsertOpportunity, named: true},
"selectOpportunityByUUID": {sql: selectOpportunityByUUID},
"listOpportunitiesSelect": {
sql: listOpportunitiesSelect + listOpportunitiesOrderAndPage,
},
"listOpportunitiesOrderAndPage": {
sql: listOpportunitiesSelect + listOpportunitiesOrderAndPage,
},
"filterOpportunityStatus": {sql: withFilter(filterOpportunityStatus)},
"filterOpportunityClassificationCode": {sql: withFilter(filterOpportunityClassificationCode)},
"filterOpportunityTitleMatch": {sql: withFilter(filterOpportunityTitleMatch)},
"filterOpportunityReleasedFrom": {sql: withFilter(filterOpportunityReleasedFrom)},
"filterOpportunityReleasedBefore": {sql: withFilter(filterOpportunityReleasedBefore)},
"filterOpportunityMarkHidden": {sql: withFilter(filterOpportunityMarkHidden)},
"filterOpportunityMarkSkipped": {sql: withFilter(filterOpportunityMarkSkipped)},
"filterOpportunityMarkInterested": {sql: withFilter(filterOpportunityMarkInterested)},
"filterOpportunityMarkBidSubmitted": {sql: withFilter(filterOpportunityMarkBidSubmitted)},
"filterOpportunityMarkWon": {sql: withFilter(filterOpportunityMarkWon)},
"filterOpportunityMarkLost": {sql: withFilter(filterOpportunityMarkLost)},
"filterOpportunityStatus": {
sql: withFilter(filterOpportunityStatus),
},
"filterOpportunityClassificationCode": {
sql: withFilter(filterOpportunityClassificationCode),
},
"filterOpportunityTitleMatch": {
sql: withFilter(filterOpportunityTitleMatch),
},
"filterOpportunityReleasedFrom": {
sql: withFilter(filterOpportunityReleasedFrom),
},
"filterOpportunityReleasedBefore": {
sql: withFilter(filterOpportunityReleasedBefore),
},
"filterOpportunityMarkHidden": {
sql: withFilter(filterOpportunityMarkHidden),
},
"filterOpportunityMarkSkipped": {
sql: withFilter(filterOpportunityMarkSkipped),
},
"filterOpportunityMarkInterested": {
sql: withFilter(filterOpportunityMarkInterested),
},
"filterOpportunityMarkBidSubmitted": {
sql: withFilter(filterOpportunityMarkBidSubmitted),
},
"filterOpportunityMarkWon": {
sql: withFilter(filterOpportunityMarkWon),
},
"filterOpportunityMarkLost": {
sql: withFilter(filterOpportunityMarkLost),
},
// opportunity_ocds_releases
"insertRelease": {sql: insertRelease, named: true},
@@ -61,12 +85,22 @@ var checks = map[string]queryCheck{
"selectReleaseTagsByOpportunity": {sql: selectReleaseTagsByOpportunity},
// classification_codes
"upsertClassificationCodeReturningID": {sql: upsertClassificationCodeReturningID, named: true},
"upsertClassificationCodeReturningID": {
sql: upsertClassificationCodeReturningID,
named: true,
},
// opportunity_classifications
"deleteOpportunityClassifications": {sql: deleteOpportunityClassifications},
"insertOpportunityClassification": {sql: insertOpportunityClassification},
"selectClassificationsForOpportunities": {sql: selectClassificationsForOpportunities, in: true},
"deleteOpportunityClassifications": {
sql: deleteOpportunityClassifications,
},
"insertOpportunityClassification": {
sql: insertOpportunityClassification,
},
"selectClassificationsForOpportunities": {
sql: selectClassificationsForOpportunities,
in: true,
},
// opportunity_marks
"insertOpportunityMarks": {sql: insertOpportunityMarks, named: true},
@@ -75,12 +109,17 @@ var checks = map[string]queryCheck{
}
func withFilter(fragment string) string {
return listOpportunitiesSelect + "\nWHERE " + fragment + listOpportunitiesOrderAndPage
return listOpportunitiesSelect + "\nWHERE " + fragment +
listOpportunitiesOrderAndPage
}
func openTestDB(t *testing.T) *sqlx.DB {
t.Helper()
s, err := Open(context.Background(), filepath.Join(t.TempDir(), "queries.db"), WithAutoMigrate())
s, err := Open(
context.Background(),
filepath.Join(t.TempDir(), "queries.db"),
WithAutoMigrate(),
)
if err != nil {
t.Fatalf("Open: %v", err)
}
@@ -144,11 +183,13 @@ func TestListOpportunitiesQueryAllFilters(t *testing.T) {
Limit: 10,
Offset: 5,
})
stmt, err := db.Preparex(query)
if err != nil {
t.Fatalf("prepare:\n%s\n%v", query, err)
}
defer stmt.Close()
var rows []opportunityRow
if err := stmt.Select(&rows, args...); err != nil {
t.Errorf("run with %d args:\n%s\n%v", len(args), query, err)
@@ -162,6 +203,7 @@ func TestQueriesAllListed(t *testing.T) {
if err != nil {
t.Fatal(err)
}
var declared []string
for _, decl := range file.Decls {
gen, ok := decl.(*ast.GenDecl)
@@ -174,17 +216,21 @@ func TestQueriesAllListed(t *testing.T) {
}
}
}
sort.Strings(declared)
if len(declared) == 0 {
t.Fatal("found no constants in queries.go")
}
for _, name := range declared {
if _, ok := checks[name]; !ok {
t.Errorf("queries.go constant %s is not listed in checks", name)
}
}
for name := range checks {
if i := sort.SearchStrings(declared, name); i == len(declared) || declared[i] != name {
if i := sort.SearchStrings(declared, name); i == len(declared) ||
declared[i] != name {
t.Errorf("checks lists %s, which queries.go does not declare", name)
}
}
@@ -194,20 +240,27 @@ func TestQueriesAllListed(t *testing.T) {
// together, for writes that bypass OpportunityMarks.Validate.
func TestMarksCheckConstraint(t *testing.T) {
db := openTestDB(t)
if _, err := db.Exec(`INSERT INTO opportunities
insert := `INSERT INTO opportunities
(uuid, uuid_type, title, status, latest_release_date)
VALUES ('ocds-1', 'ocid', 'T', 'active', '2026-01-01T00:00:00Z')`); err != nil {
VALUES ('ocds-1', 'ocid', 'T', 'active', '2026-01-01T00:00:00Z')`
if _, err := db.Exec(insert); err != nil {
t.Fatal(err)
}
if _, err := db.Exec(`INSERT INTO opportunity_marks (uuid, won, lost)
VALUES ('ocds-1', 1, 1)`); err == nil {
insertWonAndLost := `INSERT INTO opportunity_marks (uuid, won, lost)
VALUES ('ocds-1', 1, 1)`
if _, err := db.Exec(insertWonAndLost); err == nil {
t.Error("inserting won and lost together succeeded")
}
if _, err := db.Exec(`INSERT INTO opportunity_marks (uuid, won)
VALUES ('ocds-1', 1)`); err != nil {
insertWon := `INSERT INTO opportunity_marks (uuid, won)
VALUES ('ocds-1', 1)`
if _, err := db.Exec(insertWon); err != nil {
t.Fatalf("inserting won: %v", err)
}
if _, err := db.Exec(`UPDATE opportunity_marks SET lost = 1 WHERE uuid = 'ocds-1'`); err == nil {
updateLost := `UPDATE opportunity_marks SET lost = 1 WHERE uuid = 'ocds-1'`
if _, err := db.Exec(updateLost); err == nil {
t.Error("updating to won and lost together succeeded")
}
}
internal/store/store.go
+187 -113
View File
@@ -18,8 +18,25 @@ import (
"git.avsdev.uk/Go/job-search/internal/types"
)
// ErrNotFound is returned when a requested opportunity does not exist.
var ErrNotFound = errors.New("store: not found")
// Errors returned by the store.
var (
// ErrNotFound is returned when a requested opportunity does not exist.
ErrNotFound = errors.New("store: not found")
// ErrNeedsMigration is returned by Open when the database has pending
// migrations. Apply them with the migrate command, or pass
// WithAutoMigrate.
ErrNeedsMigration = errors.New("store: database needs migrating")
// ErrSchemaTooNew is returned by Open when the database has migrations
// this build doesn't know about, i.e. it was migrated by a newer build.
ErrSchemaTooNew = errors.New(
"store: database schema is newer than this build",
)
// ErrWouldEmptyDatabase is returned by StepDown when the migration would
// drop all of the tables and thus empty the database.
ErrWouldEmptyDatabase = errors.New(
"store: migration cancelled; database would be emptied by undoing all migrations",
)
)
// DefaultListLimit is used when OpportunityFilter.Limit is zero or negative.
const DefaultListLimit = 50
@@ -32,14 +49,15 @@ type Store struct {
// SaveMultipleResult reports what SaveOpportunities changed.
type SaveMultipleResult struct {
// NewOpportunities is the number of opportunities that were not already stored.
// NewOpportunities is the number of opportunities that were not already
// stored.
NewOpportunities int
// NewReleases is the number of releases that were not already stored.
NewReleases int
// UpdatedOpportunities is incremented for each opportunity row and its classifications
// that are updated (when every release in the save was older than the
// one the stored opportunity was last updated from). It does not include new
// opportunities.
// UpdatedOpportunities is incremented for each opportunity row and its
// classifications that are updated (when every release in the save was older
// than the one the stored opportunity was last updated from). It does not
// include new opportunities.
UpdatedOpportunities int
// OpportunityResults is a slice of the results of each opportunity saved.
OpportunityResults []SaveResult
@@ -47,8 +65,8 @@ type SaveMultipleResult struct {
// SaveResult reports what SaveOpportunity changed.
type SaveResult struct {
// OpportunityUpdated is true if the opportunity row and its classifications were
// written. It is false when every release in the save was older than the
// OpportunityUpdated is true if the opportunity row and its classifications
// were written. It is false when every release in the save was older than the
// one the stored opportunity was last updated from.
OpportunityUpdated bool
// NewReleases is the number of releases that were not already stored.
@@ -58,6 +76,7 @@ type SaveResult struct {
// MarkFilter filters ListOpportunities on one of an opportunity's marks.
type MarkFilter int
// Values of MarkFilter.
const (
// MarkAny does not filter. It is the zero value.
MarkAny MarkFilter = iota
@@ -68,8 +87,8 @@ const (
MarkUnset
)
// OpportunityFilter selects opportunities for ListOpportunities. Zero-valued fields do not
// filter.
// OpportunityFilter selects opportunities for ListOpportunities. Zero-valued
// fields do not filter.
type OpportunityFilter struct {
Status string
// ClassificationCode matches opportunities with a classification of this code
@@ -117,14 +136,18 @@ func prepareOpportunity(opportunity *types.Opportunity) error {
opportunity.Marks.UUID = opportunity.UUID
if err := opportunity.Marks.Validate(); err != nil {
return fmt.Errorf("store: opportunity %s: %w", opportunity.UUID, err)
return fmt.Errorf("opportunity %s: %w", opportunity.UUID, err)
}
latest := opportunity.LatestReleaseDate
for i := range opportunity.Releases {
release := &opportunity.Releases[i]
if release.ReleaseID == "" {
return fmt.Errorf("store: opportunity %s: release %d has no release id", opportunity.UUID, i)
return fmt.Errorf(
"store: opportunity %s: release %d has no release id",
opportunity.UUID,
i,
)
}
release.UUID = opportunity.UUID
release.ReleaseDate = release.ReleaseDate.UTC()
@@ -138,7 +161,9 @@ func prepareOpportunity(opportunity *types.Opportunity) error {
}
if latest.IsZero() {
return fmt.Errorf("store: opportunity %s has no release date", opportunity.UUID)
return fmt.Errorf(
"store: opportunity %s has no release date", opportunity.UUID,
)
}
opportunity.LatestReleaseDate = latest.UTC()
@@ -188,12 +213,15 @@ func Open(ctx context.Context, path string, opts ...Option) (*Store, error) {
if err != nil {
return nil, err
}
if err := sqldb.PingContext(ctx); err != nil {
return nil, errors.Join(err, sqldb.Close())
}
if err := prepareSchema(ctx, sqldb, settings); err != nil {
return nil, errors.Join(err, sqldb.Close())
}
return &Store{
db: sqlx.NewDb(sqldb, "sqlite3"),
}, nil
@@ -207,10 +235,10 @@ func (s *Store) Close() error { return s.db.Close() }
// reads fail loudly ("missing destination name ...") until it is mapped,
// instead of being silently ignored.
//
// opportunityRow gives the opportunities table's internal id column a home, so it stays
// out of types.Opportunity. sqlx maps the embedded Opportunity's fields as if declared
// here. The other tables have no internal columns and read straight into
// internal/types.
// opportunityRow gives the opportunities table's internal id column a home, so
// it stays out of types.Opportunity. sqlx maps the embedded Opportunity's
// fields as if declared here. The other tables have no internal columns and
// read straight into internal/types.
type opportunityRow struct {
ID int64 `db:"id"`
types.Opportunity
@@ -230,8 +258,7 @@ type opportunityRow struct {
// opportunity is normalised in place (see prepareOpportunity), and when it is
// written its classifications' IDs are filled in.
func (s *Store) SaveOpportunity(
ctx context.Context,
opportunity *types.Opportunity,
ctx context.Context, opportunity *types.Opportunity,
) (SaveResult, error) {
if err := prepareOpportunity(opportunity); err != nil {
return SaveResult{}, err
@@ -243,14 +270,16 @@ func (s *Store) SaveOpportunity(
}
defer tx.Rollback() //nolint:errcheck // no-op after Commit
res, _, err := saveOpportunity(ctx, tx, opportunity)
result, _, err := saveOpportunity(ctx, tx, opportunity)
if err != nil {
return SaveResult{}, err
}
if err := tx.Commit(); err != nil {
return SaveResult{}, err
}
return res, nil
return result, nil
}
// SaveOpportunities stores each of opportunities and its releases, all in one
@@ -282,28 +311,29 @@ func (s *Store) SaveOpportunities(
}
defer tx.Rollback() //nolint:errcheck // no-op after Commit
allRes := SaveMultipleResult{
total := SaveMultipleResult{
OpportunityResults: make([]SaveResult, len(opportunities)),
}
for i, opportunity := range opportunities {
res, isNew, err := saveOpportunity(ctx, tx, opportunity)
result, isNew, err := saveOpportunity(ctx, tx, opportunity)
if err != nil {
return SaveMultipleResult{}, err
}
allRes.OpportunityResults[i] = res
allRes.NewReleases += res.NewReleases
total.OpportunityResults[i] = result
total.NewReleases += result.NewReleases
switch {
case isNew:
allRes.NewOpportunities++
case res.OpportunityUpdated:
allRes.UpdatedOpportunities++
total.NewOpportunities++
case result.OpportunityUpdated:
total.UpdatedOpportunities++
}
}
if err := tx.Commit(); err != nil {
return SaveMultipleResult{}, err
}
return allRes, nil
return total, nil
}
// saveOpportunity does the work of SaveOpportunity and SaveOpportunities for
@@ -311,28 +341,31 @@ func (s *Store) SaveOpportunities(
// not already stored.
func saveOpportunity(
ctx context.Context, tx *sqlx.Tx, opportunity *types.Opportunity,
) (res SaveResult, isNew bool, err error) {
var stored time.Time
err = tx.GetContext(ctx, &stored, selectOpportunityLatestReleaseDate, opportunity.UUID)
) (SaveResult, bool, error) {
var (
result SaveResult
isNew bool
stored time.Time
)
err := tx.GetContext(
ctx, &stored, selectOpportunityLatestReleaseDate, opportunity.UUID,
)
switch {
case errors.Is(err, sql.ErrNoRows):
isNew = true
res.OpportunityUpdated = true
result.OpportunityUpdated = true
case err != nil:
return SaveResult{}, false, fmt.Errorf(
"store: reading opportunity %s: %w",
opportunity.UUID,
err,
"reading opportunity %s: %w", opportunity.UUID, err,
)
default:
res.OpportunityUpdated = !opportunity.LatestReleaseDate.Before(stored)
result.OpportunityUpdated = !opportunity.LatestReleaseDate.Before(stored)
}
if res.OpportunityUpdated {
if result.OpportunityUpdated {
if err := writeOpportunity(ctx, tx, opportunity); err != nil {
return SaveResult{}, false, fmt.Errorf(
"store: writing opportunity %s: %w",
opportunity.UUID,
err,
"writing opportunity %s: %w", opportunity.UUID, err,
)
}
}
@@ -343,10 +376,11 @@ func saveOpportunity(
return SaveResult{}, false, err
}
if inserted {
res.NewReleases++
result.NewReleases++
}
}
return res, isNew, nil
return result, isNew, nil
}
// insertReleaseWithTags stores release and its tags unless it is already
@@ -357,77 +391,96 @@ func insertReleaseWithTags(
result, err := tx.NamedExecContext(ctx, insertRelease, release)
if err != nil {
return false, fmt.Errorf(
"store: inserting release %s/%s: %w",
"inserting release %s/%s: %w",
release.UUID,
release.ReleaseID,
err,
)
}
if affected, err := result.RowsAffected(); err != nil || affected == 0 {
return false, err // n == 0: already stored
return false, err // affected == 0: already stored
}
for _, tag := range release.Tags {
if _, err := tx.NamedExecContext(ctx, insertReleaseTag, tag); err != nil {
return false, fmt.Errorf(
"store: inserting tag for %s/%s: %w", release.UUID, release.ReleaseID, err,
"inserting tag for %s/%s: %w",
release.UUID,
release.ReleaseID,
err,
)
}
}
return true, nil
}
// writeOpportunity upserts opportunity's row, gives it a marks row if it has none
// (existing marks are kept), and replaces its classifications, filling in
// writeOpportunity upserts opportunity's row, gives it a marks row if it has
// none (existing marks are kept), and replaces its classifications, filling in
// their IDs.
func writeOpportunity(
ctx context.Context,
tx *sqlx.Tx,
opportunity *types.Opportunity,
ctx context.Context, tx *sqlx.Tx, opportunity *types.Opportunity,
) (err error) {
if _, err := tx.NamedExecContext(ctx, upsertOpportunity, opportunity); err != nil {
return err
}
if _, err := tx.NamedExecContext(ctx, insertOpportunityMarks, opportunity.Marks); err != nil {
return err
}
if _, err := tx.ExecContext(ctx, deleteOpportunityClassifications, opportunity.UUID); err != nil {
_, err = tx.NamedExecContext(ctx, upsertOpportunity, opportunity)
if err != nil {
return err
}
statement, err := tx.PrepareNamedContext(ctx, upsertClassificationCodeReturningID)
_, err = tx.NamedExecContext(ctx, insertOpportunityMarks, opportunity.Marks)
if err != nil {
return err
}
_, err = tx.ExecContext(
ctx, deleteOpportunityClassifications, opportunity.UUID,
)
if err != nil {
return err
}
statement, err := tx.PrepareNamedContext(
ctx, upsertClassificationCodeReturningID,
)
if err != nil {
return err
}
defer func() {
err = errors.Join(err, statement.Close())
}()
for i := range opportunity.Classifications {
classification := &opportunity.Classifications[i]
if err := statement.GetContext(ctx, &classification.ID, classification); err != nil {
err = statement.GetContext(ctx, &classification.ID, classification)
if err != nil {
return err
}
if _, err := tx.ExecContext(
ctx,
insertOpportunityClassification,
opportunity.UUID,
classification.ID,
); err != nil {
_, err = tx.ExecContext(
ctx, insertOpportunityClassification, opportunity.UUID, classification.ID,
)
if err != nil {
return err
}
}
return nil
}
// SetMarks replaces the marks of the opportunity marks.UUID, or returns
// ErrNotFound. Marks that fail marks.Validate (e.g. types.ErrWonAndLost) are
// not stored.
func (s *Store) SetMarks(ctx context.Context, marks types.OpportunityMarks) error {
func (s *Store) SetMarks(
ctx context.Context, marks types.OpportunityMarks,
) error {
if marks.UUID == "" {
return errors.New("store: marks have no UUID")
}
if err := marks.Validate(); err != nil {
return fmt.Errorf("store: opportunity %s: %w", marks.UUID, err)
return fmt.Errorf("opportunity %s: %w", marks.UUID, err)
}
_, err := s.db.NamedExecContext(ctx, upsertOpportunityMarks, marks)
if isForeignKeyError(err) {
return ErrNotFound
@@ -439,12 +492,15 @@ func (s *Store) SetMarks(ctx context.Context, marks types.OpportunityMarks) erro
// breaks a foreign key, e.g. one referring to a missing opportunity.
func isForeignKeyError(err error) bool {
var sqliteErr sqlite3.Error
return errors.As(err, &sqliteErr) && sqliteErr.ExtendedCode == sqlite3.ErrConstraintForeignKey
return errors.As(err, &sqliteErr) &&
sqliteErr.ExtendedCode == sqlite3.ErrConstraintForeignKey
}
// GetOpportunity returns the opportunity with its classifications, marks and
// all of its releases (with tags), or ErrNotFound.
func (s *Store) GetOpportunity(ctx context.Context, uuid string) (*types.Opportunity, error) {
func (s *Store) GetOpportunity(
ctx context.Context, uuid string,
) (*types.Opportunity, error) {
var row opportunityRow
err := s.db.GetContext(ctx, &row, selectOpportunityByUUID, uuid)
if errors.Is(err, sql.ErrNoRows) {
@@ -453,6 +509,7 @@ func (s *Store) GetOpportunity(ctx context.Context, uuid string) (*types.Opportu
if err != nil {
return nil, err
}
opportunities := []types.Opportunity{row.Opportunity}
if err := s.loadClassifications(ctx, opportunities); err != nil {
return nil, err
@@ -460,9 +517,11 @@ func (s *Store) GetOpportunity(ctx context.Context, uuid string) (*types.Opportu
if err := s.loadMarks(ctx, opportunities); err != nil {
return nil, err
}
if opportunities[0].Releases, err = s.ListReleases(ctx, uuid); err != nil {
return nil, err
}
return &opportunities[0], nil
}
@@ -470,29 +529,32 @@ func (s *Store) GetOpportunity(ctx context.Context, uuid string) (*types.Opportu
// classifications and marks but without releases (use ListReleases or
// GetOpportunity for those).
func (s *Store) ListOpportunities(
ctx context.Context,
filter OpportunityFilter,
ctx context.Context, filter OpportunityFilter,
) ([]types.Opportunity, error) {
query, args := listOpportunitiesQuery(filter)
var rows []opportunityRow
if err := s.db.SelectContext(ctx, &rows, query, args...); err != nil {
return nil, err
}
opportunities := make([]types.Opportunity, len(rows))
for i, row := range rows {
opportunities[i] = row.Opportunity
}
if err := s.loadClassifications(ctx, opportunities); err != nil {
return nil, err
}
if err := s.loadMarks(ctx, opportunities); err != nil {
return nil, err
}
return opportunities, nil
}
// listOpportunitiesQuery assembles the ListOpportunities query from the fragments in
// queries.go, with one argument per filter that is set (none for mark filters).
// listOpportunitiesQuery assembles the ListOpportunities query from the
// fragments in queries.go, with one argument per filter that is set (none for
// mark filters).
func listOpportunitiesQuery(filter OpportunityFilter) (string, []any) {
var where []string
var args []any
@@ -500,6 +562,7 @@ func listOpportunitiesQuery(filter OpportunityFilter) (string, []any) {
where = append(where, clause)
args = append(args, argument)
}
if filter.Status != "" {
add(filterOpportunityStatus, filter.Status)
}
@@ -515,35 +578,19 @@ func listOpportunitiesQuery(filter OpportunityFilter) (string, []any) {
if !filter.ReleasedBefore.IsZero() {
add(filterOpportunityReleasedBefore, filter.ReleasedBefore.UTC())
}
for _, mark := range []struct {
marks := []struct {
filter MarkFilter
clause string
}{
{
filter: filter.Hidden,
clause: filterOpportunityMarkHidden,
},
{
filter: filter.Skipped,
clause: filterOpportunityMarkSkipped,
},
{
filter: filter.Interested,
clause: filterOpportunityMarkInterested,
},
{
filter: filter.BidSubmitted,
clause: filterOpportunityMarkBidSubmitted,
},
{
filter: filter.Won,
clause: filterOpportunityMarkWon,
},
{
filter: filter.Lost,
clause: filterOpportunityMarkLost,
},
} {
{filter: filter.Hidden, clause: filterOpportunityMarkHidden},
{filter: filter.Skipped, clause: filterOpportunityMarkSkipped},
{filter: filter.Interested, clause: filterOpportunityMarkInterested},
{filter: filter.BidSubmitted, clause: filterOpportunityMarkBidSubmitted},
{filter: filter.Won, clause: filterOpportunityMarkWon},
{filter: filter.Lost, clause: filterOpportunityMarkLost},
}
for _, mark := range marks {
switch mark.filter {
case MarkSet:
where = append(where, mark.clause)
@@ -557,14 +604,19 @@ func listOpportunitiesQuery(filter OpportunityFilter) (string, []any) {
query += "\nWHERE " + strings.Join(where, "\n\tAND ")
}
query += listOpportunitiesOrderAndPage
return query, append(args, filter.limit(), filter.Offset)
}
// loadClassifications fills in Classifications for all opportunities with one query.
func (s *Store) loadClassifications(ctx context.Context, opportunities []types.Opportunity) error {
// loadClassifications fills in Classifications for all opportunities with one
// query.
func (s *Store) loadClassifications(
ctx context.Context, opportunities []types.Opportunity,
) error {
if len(opportunities) == 0 {
return nil
}
uuids := make([]string, len(opportunities))
byUUID := make(map[string]*types.Opportunity, len(opportunities))
for i := range opportunities {
@@ -576,27 +628,36 @@ func (s *Store) loadClassifications(ctx context.Context, opportunities []types.O
if err != nil {
return err
}
// opportunity_uuid says which opportunity each row belongs to.
var rows []struct {
OpportunityUUID string `db:"opportunity_uuid"`
types.ClassificationCode
}
if err := s.db.SelectContext(ctx, &rows, s.db.Rebind(query), args...); err != nil {
err = s.db.SelectContext(ctx, &rows, s.db.Rebind(query), args...)
if err != nil {
return err
}
for _, releaseRow := range rows {
opportunity := byUUID[releaseRow.OpportunityUUID]
opportunity.Classifications = append(opportunity.Classifications, releaseRow.ClassificationCode)
for _, classificationRow := range rows {
opportunity := byUUID[classificationRow.OpportunityUUID]
opportunity.Classifications = append(
opportunity.Classifications, classificationRow.ClassificationCode,
)
}
return nil
}
// loadMarks fills in Marks for all opportunities with one query. An
// opportunity with no marks row gets zero marks carrying its UUID.
func (s *Store) loadMarks(ctx context.Context, opportunities []types.Opportunity) error {
func (s *Store) loadMarks(
ctx context.Context, opportunities []types.Opportunity,
) error {
if len(opportunities) == 0 {
return nil
}
uuids := make([]string, len(opportunities))
byUUID := make(map[string]*types.Opportunity, len(opportunities))
for i := range opportunities {
@@ -611,42 +672,55 @@ func (s *Store) loadMarks(ctx context.Context, opportunities []types.Opportunity
if err != nil {
return err
}
var rows []types.OpportunityMarks
if err := s.db.SelectContext(ctx, &rows, s.db.Rebind(query), args...); err != nil {
err = s.db.SelectContext(ctx, &rows, s.db.Rebind(query), args...)
if err != nil {
return err
}
for _, row := range rows {
byUUID[row.UUID].Marks = row
}
return nil
}
// ListReleases returns a opportunity's releases with their tags, oldest first.
func (s *Store) ListReleases(ctx context.Context, uuid string) ([]types.OpportunityRelease, error) {
func (s *Store) ListReleases(
ctx context.Context, uuid string,
) ([]types.OpportunityRelease, error) {
releases := []types.OpportunityRelease{}
if err := s.db.SelectContext(ctx, &releases, selectReleasesByOpportunity, uuid); err != nil {
err := s.db.SelectContext(ctx, &releases, selectReleasesByOpportunity, uuid)
if err != nil {
return nil, err
}
byID := make(map[string]*types.OpportunityRelease, len(releases))
for i := range releases {
byID[releases[i].ReleaseID] = &releases[i]
}
var tags []types.OpportunityReleaseTag
if err := s.db.SelectContext(ctx, &tags, selectReleaseTagsByOpportunity, uuid); err != nil {
err = s.db.SelectContext(ctx, &tags, selectReleaseTagsByOpportunity, uuid)
if err != nil {
return nil, err
}
for _, tag := range tags {
if release := byID[tag.ReleaseID]; release != nil {
release.Tags = append(release.Tags, tag)
}
}
return releases, nil
}
// ReleaseExists reports whether a release is already stored, so a fetcher
// can skip converting it.
func (s *Store) ReleaseExists(ctx context.Context, uuid, releaseID string) (bool, error) {
func (s *Store) ReleaseExists(
ctx context.Context, uuid, releaseID string,
) (bool, error) {
var exists bool
err := s.db.GetContext(ctx, &exists, selectReleaseExists, uuid, releaseID)
return exists, err
internal/store/store_test.go
+146 -65
View File
@@ -30,7 +30,11 @@ func day(d int) time.Time {
return time.Date(2026, time.March, d, 9, 30, 0, 0, time.UTC)
}
func opportunity(uuid, title string, releaseDay int, releaseIDs ...string) *types.Opportunity {
func opportunity(
uuid, title string,
releaseDay int,
releaseIDs ...string,
) *types.Opportunity {
start, end := day(20), day(28)
t := &types.Opportunity{
UUID: uuid,
@@ -39,7 +43,7 @@ func opportunity(uuid, title string, releaseDay int, releaseIDs ...string) *type
Status: "active",
Description: "A description",
BuyerName: "Example Council",
BuyerId: "GB-EXAMPLE",
BuyerID: "GB-EXAMPLE",
MainCategory: "services",
ProcurementMethod: "open",
MultiLot: true,
@@ -49,8 +53,16 @@ func opportunity(uuid, title string, releaseDay int, releaseIDs ...string) *type
GrossValue: 125000.50,
HasDocuments: true,
Classifications: []types.ClassificationCode{
{IsCPV: true, Code: "72000000", Classification: "IT services"},
{IsCPV: true, Code: "03000000", Classification: "Agricultural products"},
{
IsCPV: true,
Code: "72000000",
Classification: "IT services",
},
{
IsCPV: true,
Code: "03000000",
Classification: "Agricultural products",
},
},
// No UUID: SaveOpportunity must fill it in.
Marks: types.OpportunityMarks{
@@ -58,18 +70,29 @@ func opportunity(uuid, title string, releaseDay int, releaseIDs ...string) *type
BidSubmitted: true,
},
}
for i, id := range releaseIDs {
t.Releases = append(t.Releases, types.OpportunityRelease{
ReleaseID: id,
ReleaseDate: day(releaseDay).Add(time.Duration(i) * time.Minute),
JSON: `{"id":"` + id + `"}`,
Tags: []types.OpportunityReleaseTag{{Tag: "opportunity"}, {Tag: "planning"}},
Tags: []types.OpportunityReleaseTag{
{
Tag: "opportunity",
},
{
Tag: "planning",
},
},
})
}
return t
}
func mustSave(t *testing.T, s *store.Store, td *types.Opportunity) store.SaveResult {
func mustSave(
t *testing.T, s *store.Store, td *types.Opportunity,
) store.SaveResult {
t.Helper()
res, err := s.SaveOpportunity(context.Background(), td)
if err != nil {
@@ -116,24 +139,30 @@ func TestSaveAndGetRoundTrip(t *testing.T) {
}
got := mustGet(t, s, "ocds-1")
if got.Title != in.Title || got.BuyerName != in.BuyerName || got.BuyerId != in.BuyerId ||
got.MainCategory != in.MainCategory || got.ProcurementMethod != in.ProcurementMethod ||
got.UUIDType != "ocid" || got.Status != "active" || got.Description != in.Description ||
!got.MultiLot || !got.SMESuitable || got.VCSESuitable || !got.HasDocuments ||
got.GrossValue != in.GrossValue {
if got.Title != in.Title || got.BuyerName != in.BuyerName ||
got.BuyerID != in.BuyerID || got.MainCategory != in.MainCategory ||
got.ProcurementMethod != in.ProcurementMethod || got.UUIDType != "ocid" ||
got.Status != "active" || got.Description != in.Description ||
!got.MultiLot || !got.SMESuitable || got.VCSESuitable ||
!got.HasDocuments || got.GrossValue != in.GrossValue {
t.Errorf("opportunity fields differ:\n got %+v\nwant %+v", got, in)
}
if got.StartDate == nil || !got.StartDate.Equal(day(20)) || got.EndDate == nil ||
!got.EndDate.Equal(day(28)) {
if got.StartDate == nil || !got.StartDate.Equal(day(20)) ||
got.EndDate == nil || !got.EndDate.Equal(day(28)) {
t.Errorf("dates: got %v / %v", got.StartDate, got.EndDate)
}
if want := day(1).Add(time.Minute); !got.LatestReleaseDate.Equal(want) {
t.Errorf("LatestReleaseDate = %v, want %v", got.LatestReleaseDate, want)
}
if len(got.Classifications) != 2 || got.Classifications[0].Code != "03000000" ||
got.Classifications[1].Code != "72000000" || got.Classifications[0].ID == 0 {
t.Errorf("classifications: got %+v (leading zero must survive)", got.Classifications)
if len(got.Classifications) != 2 ||
got.Classifications[0].Code != "03000000" ||
got.Classifications[1].Code != "72000000" ||
got.Classifications[0].ID == 0 {
t.Errorf(
"classifications: got %+v (leading zero must survive)",
got.Classifications,
)
}
wantMarks := types.OpportunityMarks{
@@ -153,9 +182,8 @@ func TestSaveAndGetRoundTrip(t *testing.T) {
!r.ReleaseDate.Equal(day(1)) {
t.Errorf("release: got %+v", r)
}
if len(r.Tags) != 2 ||
r.Tags[0].Tag != "opportunity" || r.Tags[1].Tag != "planning" ||
r.Tags[0].ReleaseID != "r1" ||
if len(r.Tags) != 2 || r.Tags[0].Tag != "opportunity" ||
r.Tags[1].Tag != "planning" || r.Tags[0].ReleaseID != "r1" ||
r.Tags[0].UUID != "ocds-1" {
t.Errorf("tags: got %+v", r.Tags)
}
@@ -178,7 +206,9 @@ func TestReleasesAreWriteOnce(t *testing.T) {
again := opportunity("ocds-1", "Title", 1, "r1")
again.Releases[0].JSON = `{"changed":true}`
again.Releases[0].Tags = []types.OpportunityReleaseTag{{Tag: "award"}}
again.Releases[0].Tags = []types.OpportunityReleaseTag{{
Tag: "award",
}}
if res := mustSave(t, s, again); res.NewReleases != 0 {
t.Errorf("re-save: NewReleases = %d, want 0", res.NewReleases)
}
@@ -206,7 +236,9 @@ func TestOlderReleaseDoesNotOverwriteOpportunity(t *testing.T) {
t.Errorf("opportunity was overwritten by an older release: %+v", got)
}
if len(got.Releases) != 2 || got.Releases[0].ReleaseID != "r1" {
t.Errorf("older release should still be stored, oldest first: %+v", got.Releases)
t.Errorf(
"older release should still be stored, oldest first: %+v", got.Releases,
)
}
}
@@ -220,7 +252,11 @@ func TestNewerReleaseUpdatesOpportunityAndSearchIndex(t *testing.T) {
newer := opportunity("ocds-1", "Website redesign", 2, "r2")
newer.Status = "complete"
newer.Classifications = []types.ClassificationCode{
{IsCPV: true, Code: "72000000", Classification: "IT services (amended)"},
{
IsCPV: true,
Code: "72000000",
Classification: "IT services (amended)",
},
}
if res := mustSave(t, s, newer); !res.OpportunityUpdated {
t.Fatalf("newer save not applied: %+v", res)
@@ -246,7 +282,12 @@ func TestNewerReleaseUpdatesOpportunityAndSearchIndex(t *testing.T) {
t.Fatal(err)
}
if len(ts) != want {
t.Errorf("search %q after title change: %d results, want %d", title, len(ts), want)
t.Errorf(
"search %q after title change: %d results, want %d",
title,
len(ts),
want,
)
}
}
}
@@ -257,7 +298,8 @@ func TestClassificationRowsAreShared(t *testing.T) {
b := opportunity("ocds-b", "B", 1, "r1")
mustSave(t, s, a)
mustSave(t, s, b)
if a.Classifications[0].ID == 0 || a.Classifications[0].ID != b.Classifications[0].ID {
if a.Classifications[0].ID == 0 ||
a.Classifications[0].ID != b.Classifications[0].ID {
t.Errorf(
"identical classifications should share an ID: %d vs %d",
a.Classifications[0].ID,
@@ -267,7 +309,11 @@ func TestClassificationRowsAreShared(t *testing.T) {
c := opportunity("ocds-c", "C", 1, "r1")
c.Classifications = []types.ClassificationCode{
{IsCPV: false, Code: "72000000", Classification: "IT services"},
{
IsCPV: false,
Code: "72000000",
Classification: "IT services",
},
}
mustSave(t, s, c)
if c.Classifications[0].ID == a.Classifications[0].ID {
@@ -279,13 +325,17 @@ func TestSameReleaseIDInDifferentOpportunities(t *testing.T) {
s := openStore(t)
a := opportunity("ocds-a", "A", 1, "001")
b := opportunity("ocds-b", "B", 1, "001")
b.Releases[0].Tags = []types.OpportunityReleaseTag{{Tag: "award"}}
b.Releases[0].Tags = []types.OpportunityReleaseTag{{
Tag: "award",
}}
if res := mustSave(t, s, a); res.NewReleases != 1 {
t.Fatal(res)
}
if res := mustSave(t, s, b); res.NewReleases != 1 {
t.Fatalf("release id reused under another OCID must be stored: %+v", res)
}
got := mustGet(t, s, "ocds-b").Releases
if len(got) != 1 || len(got[0].Tags) != 1 || got[0].Tags[0].Tag != "award" {
t.Errorf("tags leaked between opportunities: %+v", got)
@@ -300,7 +350,11 @@ func TestListOpportunitiesFilters(t *testing.T) {
roads := opportunity("ocds-2", "Road resurfacing works", 2, "r1")
roads.Status = "complete"
roads.Classifications = []types.ClassificationCode{
{IsCPV: true, Code: "45233142", Classification: "Road-repair works"},
{
IsCPV: true,
Code: "45233142",
Classification: "Road-repair works",
},
}
hosting := opportunity("ocds-3", "Managed cloud hosting", 3, "r1")
for _, td := range []*types.Opportunity{cloud, roads, hosting} {
@@ -312,7 +366,11 @@ func TestListOpportunitiesFilters(t *testing.T) {
f store.OpportunityFilter
want []string
}{
{"all, newest first", store.OpportunityFilter{}, []string{"ocds-3", "ocds-2", "ocds-1"}},
{
"all, newest first",
store.OpportunityFilter{},
[]string{"ocds-3", "ocds-2", "ocds-1"},
},
{"status", store.OpportunityFilter{
Status: "complete",
}, []string{"ocds-2"}},
@@ -385,9 +443,13 @@ func TestListOpportunitiesFilters(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if len(got) != 1 || len(got[0].Classifications) != 1 || got[0].Releases != nil ||
got[0].Marks.UUID != "ocds-2" || !got[0].Marks.Interested {
t.Errorf("list should include classifications and marks and omit releases: %+v", got)
if len(got) != 1 || len(got[0].Classifications) != 1 ||
got[0].Releases != nil || got[0].Marks.UUID != "ocds-2" ||
!got[0].Marks.Interested {
t.Errorf(
"list should include classifications and marks and omit releases: %+v",
got,
)
}
}
@@ -402,26 +464,28 @@ func TestReleaseExistsAndListReleases(t *testing.T) {
}{{"ocds-1", "r1", true}, {"ocds-1", "r3", false}, {"ocds-2", "r1", false}} {
got, err := s.ReleaseExists(ctx, tc.uuid, tc.id)
if err != nil || got != tc.want {
t.Errorf("ReleaseExists(%s, %s) = %v, %v; want %v", tc.uuid, tc.id, got, err, tc.want)
t.Errorf(
"ReleaseExists(%s, %s) = %v, %v; want %v",
tc.uuid,
tc.id,
got,
err,
tc.want,
)
}
}
rs, err := s.ListReleases(ctx, "ocds-1")
if err != nil || len(rs) != 2 || rs[0].ReleaseID != "r1" || rs[1].ReleaseID != "r2" ||
len(rs[1].Tags) != 2 {
if err != nil || len(rs) != 2 || rs[0].ReleaseID != "r1" ||
rs[1].ReleaseID != "r2" || len(rs[1].Tags) != 2 {
t.Errorf("ListReleases: %+v, %v", rs, err)
}
}
func TestGetOpportunityNotFound(t *testing.T) {
s := openStore(t)
if _, err := s.GetOpportunity(
context.Background(),
"missing",
); !errors.Is(
err,
store.ErrNotFound,
) {
_, err := s.GetOpportunity(context.Background(), "missing")
if !errors.Is(err, store.ErrNotFound) {
t.Errorf("err = %v, want ErrNotFound", err)
}
}
@@ -473,8 +537,12 @@ func TestResaveLoadedOpportunity(t *testing.T) {
ids := []int64{loaded.Classifications[0].ID, loaded.Classifications[1].ID}
loaded.Title = "Amended"
loaded.Releases = append(loaded.Releases, types.OpportunityRelease{
ReleaseID: "r2", ReleaseDate: day(2), JSON: `{}`,
Tags: []types.OpportunityReleaseTag{{Tag: "opportunityAmendment"}},
ReleaseID: "r2",
ReleaseDate: day(2),
JSON: `{}`,
Tags: []types.OpportunityReleaseTag{{
Tag: "opportunityAmendment",
}},
})
res := mustSave(t, s, loaded)
if !res.OpportunityUpdated || res.NewReleases != 1 {
@@ -482,10 +550,12 @@ func TestResaveLoadedOpportunity(t *testing.T) {
}
got := mustGet(t, s, "ocds-1")
if got.Title != "Amended" || len(got.Releases) != 2 || !got.LatestReleaseDate.Equal(day(2)) {
if got.Title != "Amended" || len(got.Releases) != 2 ||
!got.LatestReleaseDate.Equal(day(2)) {
t.Errorf("resave not applied: %+v", got)
}
if got.Classifications[0].ID != ids[0] || got.Classifications[1].ID != ids[1] {
if got.Classifications[0].ID != ids[0] ||
got.Classifications[1].ID != ids[1] {
t.Errorf(
"classification IDs changed on resave: %v -> %d, %d",
ids,
@@ -527,10 +597,7 @@ func TestSetMarksErrors(t *testing.T) {
UUID: "missing",
Hidden: true,
},
); !errors.Is(
err,
store.ErrNotFound,
) {
); !errors.Is(err, store.ErrNotFound) {
t.Errorf("unknown opportunity: err = %v, want ErrNotFound", err)
}
if err := s.SetMarks(ctx, types.OpportunityMarks{
@@ -565,7 +632,9 @@ func TestSaveKeepsExistingMarks(t *testing.T) {
newest := opportunity("ocds-1", "Amended again", 3, "r3")
newest.Marks = types.OpportunityMarks{}
if _, err := s.SaveOpportunities(ctx, []*types.Opportunity{newest}); err != nil {
if _, err := s.SaveOpportunities(
ctx, []*types.Opportunity{newest},
); err != nil {
t.Fatal(err)
}
if got := mustGet(t, s, "ocds-1").Marks; got != want {
@@ -580,9 +649,13 @@ func TestSaveOpportunitiesStoresMarks(t *testing.T) {
b.Marks = types.OpportunityMarks{
Skipped: true,
}
if _, err := s.SaveOpportunities(context.Background(), []*types.Opportunity{a, b}); err != nil {
if _, err := s.SaveOpportunities(
context.Background(), []*types.Opportunity{a, b},
); err != nil {
t.Fatal(err)
}
for uuid, want := range map[string]types.OpportunityMarks{
"ocds-a": {UUID: "ocds-a", Interested: true, BidSubmitted: true},
"ocds-b": {UUID: "ocds-b", Skipped: true},
@@ -610,7 +683,10 @@ func TestListOpportunitiesMarkFilters(t *testing.T) {
Lost: true,
})
mustSetMarks(t, s, types.OpportunityMarks{
UUID: "ocds-4", Interested: true, BidSubmitted: true, Won: true,
UUID: "ocds-4",
Interested: true,
BidSubmitted: true,
Won: true,
})
set, unset := store.MarkSet, store.MarkUnset
@@ -619,7 +695,11 @@ func TestListOpportunitiesMarkFilters(t *testing.T) {
f store.OpportunityFilter
want []string
}{
{"any", store.OpportunityFilter{}, []string{"ocds-4", "ocds-3", "ocds-2", "ocds-1"}},
{
"any",
store.OpportunityFilter{},
[]string{"ocds-4", "ocds-3", "ocds-2", "ocds-1"},
},
{"hidden", store.OpportunityFilter{
Hidden: set,
}, []string{"ocds-2"}},
@@ -698,8 +778,8 @@ func TestListOpportunitiesMarkFilters(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if len(got) != 4 || !got[0].Marks.Won || !got[1].Marks.Lost || !got[2].Marks.Hidden ||
got[3].Marks.Hidden || !got[3].Marks.Interested {
if len(got) != 4 || !got[0].Marks.Won || !got[1].Marks.Lost ||
!got[2].Marks.Hidden || got[3].Marks.Hidden || !got[3].Marks.Interested {
t.Errorf("list marks: %+v", got)
}
}
@@ -727,19 +807,20 @@ func TestWonAndLostAreExclusive(t *testing.T) {
Won: true,
Lost: true,
}
if _, err := s.SaveOpportunity(ctx, in); !errors.Is(err, types.ErrWonAndLost) {
_, err := s.SaveOpportunity(ctx, in)
if !errors.Is(err, types.ErrWonAndLost) {
t.Errorf("SaveOpportunity(won and lost): err = %v, want ErrWonAndLost", err)
}
if _, err := s.SaveOpportunities(
ctx,
[]*types.Opportunity{in},
); !errors.Is(
err,
types.ErrWonAndLost,
) {
t.Errorf("SaveOpportunities(won and lost): err = %v, want ErrWonAndLost", err)
_, err = s.SaveOpportunities(ctx, []*types.Opportunity{in})
if !errors.Is(err, types.ErrWonAndLost) {
t.Errorf(
"SaveOpportunities(won and lost): err = %v, want ErrWonAndLost", err,
)
}
if _, err := s.GetOpportunity(ctx, "ocds-2"); !errors.Is(err, store.ErrNotFound) {
_, err = s.GetOpportunity(ctx, "ocds-2")
if !errors.Is(err, store.ErrNotFound) {
t.Errorf("rejected opportunity was stored: err = %v", err)
}