-
v0.17.0 Stable
released this
2026-11-06 17:29:28 +00:00 | 13 commits to main since this releaseVSKI v0.17.0 — Performance
v0.17.0 is a planned, performance-focused release. The write path is
rebuilt around a group-commit write queue, the storage layer's WAL
lifecycle and defaults are tuned for sustained throughput, and the
per-request costs around authentication, settings and realtime are
reduced. The result: 10,000+ requests per second sustained across
every benchmark scenario — reads, creates, updates and mixed traffic —
with zero errors, and flat tail latency under load.scenario (10k rows, 60 s steps) sustained RPS p95 at 10k errors read-point 10,000+ 0.40 ms 0 write-create (~150 B) 10,000+ 37 ms* 0 write-update 10,000+ 8.6 ms 0 mixed-80-20 10,000+ 0.67 ms 0 * near-saturation batching (see below). Full methodology and the
Postgres/MySQL reference comparison:benchmarks/BENCHMARK.md— at
10k mixed, vski's 0.67 ms p95 also leads the Postgres (4.05 ms) and
MySQL (23.0 ms) reference services on this rig.Safe upgrade: no schema migrations; API keys are re-hashed lazily on
first use; every new behavior is on by default and individually
disableable via environment variables.Write queue: group commit for record writes
SQLite allows one writer at a time, so the server now makes every write
cycle count: a per-database write queue funnels concurrent write
requests through a single writer goroutine that commits them together in
one transaction. N concurrent requests pay one commit cycle instead of
N — and on a busy server that is the difference between the writer
thrashing the write lock and staying quiet enough that readers never
notice it (mixed 80/20 traffic sustains 10,000+ RPS at 0.67 ms p95,
ahead of the Postgres reference service's 4.05 ms on the same rig).- Applies to record CRUD (single and bulk), graph node/edge writes, and
vector embedding upserts. - Correct by construction: a request is acknowledged only after its
batch commits, so a read following an acknowledged write always sees
it. No eventual consistency anywhere. - A failing request (e.g. a UNIQUE violation) is rolled back to its
savepoint and fails alone; concurrent requests in the same batch
still commit. - A lone write under no load skips batching entirely and commits
immediately — zero added latency at low traffic. - Under overload beyond capacity the queue applies backpressure:
requests batch-wait (that is the ~37 ms p95 on pure write-create at
10k) and fail loudly afterWRITE_QUEUE_SUBMIT_WAIT_MS(5 s) instead
of piling up unbounded. Errors stay at zero up to capacity.
Configuration (all optional):
WRITE_QUEUE_ENABLED(defaulttrue),
WRITE_QUEUE_DEPTH(4096),WRITE_QUEUE_MAX_BATCH(128),
WRITE_QUEUE_WINDOW_MS(2),WRITE_QUEUE_SUBMIT_WAIT_MS(5000).Storage layer: the server owns the WAL lifecycle
- Auto-checkpointing is disabled on every connection; a background loop
checkpoints databases whose WAL exceeds
WAL_CHECKPOINT_THRESHOLD_MB(16 MB) — a PASSIVE checkpoint first
(no write lock), then a TRUNCATE bounded by a 50 ms busy budget,
retried every second. Under normal and mixed traffic the WAL stays
in the low megabytes. Under extreme bulk-ingest bursts (10 KB records
at 10k RPS ≈ 100 MB/s) it can grow into the gigabytes until the burst
pauses and checkpoints reclaim the space automatically — size disk
headroom accordingly for sustained bulk ingest, or tune the
threshold. - Small WALs are periodically backfilled so slow
synchronous=NORMAL
trickles still get fsynced. - Pools are bounded (data 120 connections, aux 20) with idle pinned to
the open cap;SYNCHRONOUSdefaults toNORMAL— the recommended
WAL-mode setting, and with group commit also the fast one.
SQLite binding: pooled-connection hardening
Pooled SQLite connections closed mid-run (idle trims, bad-connection
discards) could leak file descriptors and crash under sustained load.
The vendoredgo-sqlite3fork fixes the shared WAL/shm handling, and
the pinned-pool configuration removes the churn path entirely.Faster authentication and settings
- API keys are stored as indexed SHA-256 hashes: key lookup is one
indexed row read. Legacy bcrypt-hashed keys keep working and are
transparently re-hashed the first time they are presented. _system_settingsreads are cached (short TTL): values such as
the auth collection name were previously re-read and JSON-parsed on
every authenticated request.
Realtime and stats
- The realtime gateway's per-topic event store enforces its size cap by
actual serialized size with deterministic trimming; consumer-group
offsets survive store compaction. - API hit statistics moved off the request path entirely: hits buffer in
memory and flush as multi-row inserts (500 rows / 5 s), so stats
collection never competes with user writes.
Collections
- Older collections get their automatic
createdindex backfilled on
start (consistent with newly created ones), keeping sorted and
paginated list queries fast on upgraded deployments.
Benchmarks
The repository ships a dockerized k6 harness (
benchmarks/) comparing
vski against minimal Go APIs over Postgres and MySQL across
read/write/mixed/auth scenarios with a rate ladder and early saturation
stop. The v0.17.0 numbers above are reproducible withmake vskiand
published viamake publish(benchmarks/BENCHMARK.md, raw runs under
benchmarks/results/).The report also includes measured read-side guidance for large
collections: a selective filter on a 100k-row table goes from 1,250 to
10,000 RPS with one index on the filtered column (collection index
management), and skipping the per-request listing count with
skipTotal=truedoubles listing throughput at 100k records (5,000 →
10,000 RPS).Downloads
-
Source code (ZIP)
3 downloads
-
Source code (TAR.GZ)
2 downloads
- Applies to record CRUD (single and bulk), graph node/edge writes, and