Added websocket hub for handling websocket clients, with a wsserve command
which runs a plain TCP server for receiving websocket hub clients, with server config, CSRF and rate limit middleware. - internal/websocket: Hub pushing messages to WebSocket clients (ServeHTTP, example.com/ws upgrade with optional CSRF check) and plain TCP clients (ServeTCP, newline-delimited JSON envelopes); ping/pong keepalive, per-client message rate limit, slow consumer handling, graceful Shutdown, debug logging of messages sent and received - internal/websocket/tcpserver: standalone TCP server for a hub, with accept back-off, a per-IP connection limit (server.websocket.connection_limit) and graceful shutdown - internal/cli/wsserve: `wsserve` command running the TCP server, with --listen/--host/--port overrides - internal/middleware: keyed token bucket rate limiter, real client IP from trusted proxies - internal/httpx: RFC 9457 problem responses, request IDs in context - internal/config: server section (host, port, shutdown timeout, rate limiter idle TTL, websocket); util.ByteSize settings (e.g. "32KiB"); lists of sections as single keys; empty lists stored as nil; isExplicitlySet for printLoadedConfig; JobSearchConfigSource renamed to JSCSourcesSource; TestSections checks every section's methods - internal/util: ByteSize, parsed and formatted with go-humanize
This commit is contained in:
1 parent
803a8929e6
commit
c9f42b62ad
42 files changed
+3619
-57
No files matched your search
@@ -40,11 +40,26 @@ is kept.
|
||||
| `sources.find_a_tender.api_root` | string | `https://www.find-tender.service.gov.uk/api/1.0` | URL of the Find a Tender API |
|
||||
| `sources.fts_mock.api_root` | string | | URL of the mock FTS server (see `internal/mock`) |
|
||||
| `sources.sleeper.block_shutdown` | bool | `false` | Test source: if true, its sleep ignores shutdown |
|
||||
| `server.host` | string | `127.0.0.1` | Address to listen on |
|
||||
| `server.port` | int | `9393` | Port to listen on |
|
||||
| `server.timeouts.shutdown` | duration | `15s` | How long shutdown waits for clients to close |
|
||||
| `server.rate_limits.idle_ttl` | duration | `10m` | How long a client's rate limiter is kept after its last request |
|
||||
| `server.websocket.enable_compression` | bool | `false` | permessage-deflate |
|
||||
| `server.websocket.max_message_size` | byte size | `32KiB` | Largest message read from a client; `0B` for 32KiB |
|
||||
| `server.websocket.connection_limit.*` | | | A rate limit rule for new connections from each client IP, WebSocket and plain TCP alike; limiters are kept for `server.rate_limits.idle_ttl` |
|
||||
| `server.websocket.message_rate_limit.*` | | | A rate limit rule for messages from each connection |
|
||||
| `server.websocket.write_timeout` | duration | `10s` | `0s` for none |
|
||||
| `server.websocket.ping_interval` | duration | `30s` | |
|
||||
| `server.websocket.pong_timeout` | duration | `10s` | `0s` for none |
|
||||
|
||||
The current sources are `find_a_tender`, `fts_mock` and `sleeper`. See
|
||||
"Addition of a new opportunities source" in the top level README for how to add
|
||||
a new source.
|
||||
|
||||
A rate limit rule (`JSCRateLimitRule`) has `enabled`, `per_second` and `burst`.
|
||||
It is used under `server.websocket.connection_limit` and
|
||||
`server.websocket.message_rate_limit`.
|
||||
|
||||
Example `config.yaml`:
|
||||
```yaml
|
||||
logging:
|
||||
@@ -79,6 +94,9 @@ per key:
|
||||
`time.Duration` or a duration string in `SetValue`, and are given out as
|
||||
strings (`"10m0s"`) by `Map`, `Flatten` and `ValueOf`, and so by `JSON`,
|
||||
`YAML` and `String` too.
|
||||
- `util.ByteSize` fields work the same way, with size strings such as `"32KiB"`
|
||||
or `"1MB"` (KB, MB, ... are powers of 1000; KiB, MiB, ... powers of 1024),
|
||||
and are given out exactly in the largest unit that divides them (`"1MB"`).
|
||||
- A section can reject values by implementing `validate(key, value) error` on
|
||||
its pointer type. `JobSearchConfigLogging` does this for `level`.
|
||||
|
||||
@@ -138,6 +156,9 @@ Nothing in `section.go` needs changing.
|
||||
| `conf_logging.go` | `logging` section and log level helpers |
|
||||
| `conf_sources.go` | `sources` section and the shared source fields |
|
||||
| `conf_source_*.go` | One file per source section |
|
||||
| `conf_server.go` | `server` section |
|
||||
| `conf_server_*.go` | One file per `server` sub-section (`rate_limits`, `timeouts`, `websocket`) |
|
||||
| `conf_ratelimitrule.go` | `JSCRateLimitRule`, the rate limit rule shared by several keys |
|
||||
| `print.go`, `debug.go` | Printing and debug logging of the configuration |
|
||||
|
||||
|
||||
|
||||
Reference in new issue
Block a user