MHSanaei d1e733b9e9 perf(clients): chunk IN queries and de-quadratic bulk delete/group/list ...

Bulk client operations bound their entire working set in a single
WHERE x IN (...) clause, which exceeds PostgreSQL's 65535-parameter limit
(and SQLite's 32766) and gives the planner a pathological query, so they
failed outright on inbounds/selections larger than the limit. Every such
query is now chunked at 400 items:

- BulkDelete / delete-all-clients: six IN queries chunked, and the
  per-row delete tombstone (which swept the whole in-memory map on every
  call, O(N^2)) replaced with a single bulk sweep.
- BulkAdjust: record and inbound-mapping lookups chunked.
- AddToGroup / RemoveFromGroup (bulk add/remove to group): three IN
  queries chunked.
- replaceGroupValue (rename/delete group): inbound-mapping lookup chunked.
- List (all-clients listing): link and traffic lookups chunked.

Measured on PostgreSQL 16: delete-all-clients on a 100k-client inbound
now completes in ~7s (previously crashed at the parameter limit); bulk
add/remove to group ~6s and full client list ~1s at 100k.

sync_scale_postgres_test.go adds skip-gated benchmarks for delete-all,
group add/remove, and list.

2026-06-04 20:35:30 +02:00

13 KiB

Raw Blame History

View Raw