• v0.18.0 929e559150

    v0.18.0 Stable

    x released this 2026-11-15 17:34:34 +00:00 | 3 commits to main since this release

    VSKI v0.18.0: Connectors

    Connectors are now a first-class platform feature. A definition is a
    declarative provider manifest, a connector is one configured instance of
    it, and an account is an authorized credential. One generic engine handles
    OAuth grants, API keys, basic passwords, custom headers, and public APIs.
    Provider tokens stay on the server: pipelines, plugins, and triggers call
    through an authenticated proxy instead of holding credentials.

    Definitions are data. Nothing is built into the binary, so the provider
    catalog is exactly what your IaC declares and the dashboard shows.

    SDKs: @vski/sdk 2.11.0 and vski-sdk 0.20.0.

    Upgrade notes:

    • Four system collections (_connector_definitions, _connectors,
      _connector_accounts, _connector_oauth_states) are created
      automatically on start. Existing collections are untouched.
    • Existing deployments start with an empty definition catalog. Apply
      connectorDefinitions before connectors; vski apply already orders
      definitions first.
    • OAuth providers need <server>/api/connectors/oauth/callback as their
      registered callback. Set PUBLIC_URL when the server is behind a proxy
      that does not forward the public host.
    • Accounts are connected interactively and are not IaC data. Their
      credentials can reference entries in the encrypted secrets store.
    • Connector management endpoints require an admin. The request proxy
      requires the workflows role; admins also pass.

    Definitions and IaC

    connectorDefinitions and connectors are global IaC kinds. vski plan,
    vski apply, and vski pull round-trip them, with definitions applied
    before connectors. vski pull writes connector-definitions.yaml and
    connectors.yaml.

    A definition declares:

    • api: base URL, exact allowed hosts, default headers, header fields,
      auth scheme, and read-only mode.
    • oauth: static client, authorization and token URLs, PKCE, token auth
      style, form or JSON token requests, scopes, extra auth and refresh params,
      token refresh, and the connect-time form.
    • key: credential label, connect-time form, and auth scheme.
    • account: profile URL and JSON paths for the remote account id, label,
      and avatar. It labels OAuth accounts after the grant completes.
    • name, description, icon, docs, and free-form metadata.

    api.baseUrl and api.allowedHosts accept {field} placeholders filled from
    connect-time values, so one definition can target per-account sites without
    giving up the host allow-list. api.readOnly rejects every method except
    GET and HEAD.

    Auth schemes cover bearer, header, query, basic, and none. A
    custom-header connector declares its api.headerFields; a basic connector
    names the form field used as the username.

    Definition ids match [a-z0-9][a-z0-9-]{0,63}. OAuth registration is static:
    the client id and secret belong to the connector config. Invalid definitions
    fail before anything is written.

    Accounts and secrets

    • OAuth: start a connection with POST /api/connectors/{id}/connect.
      The returned authorize URL carries a single-use state that expires after
      10 minutes. The callback exchanges the code, stores the encrypted grant,
      fetches the account profile, and returns to the dashboard. Access tokens
      refresh automatically when an expiry is known, a refresh token is present,
      and the definition enables refresh.
    • API key and basic password: connect with a token, a token secret, or
      form values. Basic auth combines the username form field with the token.
    • Custom headers: each declared header is collected per account and can
      come from a secret.
    • None: public APIs need no account.

    tokenSecret, valueSecrets, and headerSecrets name entries in the
    encrypted VSKI secrets store and resolve at request time. Rotating a stored
    secret updates every account that references it without rewriting the account
    or the connector config. Credential values never appear in account responses,
    connector configs, pipeline output, or run events.

    A connector can hold several accounts. Requests select one by id or name, or
    fall back to the first active account.

    Pipelines, plugins, and triggers

    Pipelines gain a connector step. It resolves account, path, url,
    headers, and body as templates, attaches the account credential
    server-side, and returns {status, headers, body}. Non-2xx provider responses
    fail the step so the usual retries and onError behavior apply. The timeout
    defaults to 30 seconds and is capped at 120.

    { "id": "projects", "type": "connector",
      "connector": "internal",
      "account": "{{input.account}}",
      "method": "GET",
      "path": "/v1/projects?limit=50" }
    

    Because account is templated, one pipeline definition can run against any
    connected account supplied as run input, including accounts for different
    sites of the same provider.

    Plugins and external systems use POST /api/connectors/{id}/request with an
    API key that carries the workflows role. The proxy applies the stored
    credential, enforces the host allow-list, and returns the provider status,
    headers, and body. The proxy call itself answers 200; a provider rejection
    is reported in status, and transport or configuration failures return
    4xx or 5xx with the reason.

    A connector is bound to its declared hosts. Only api.allowedHosts entries
    and the resolved base URL host are reachable, matched exactly with no
    wildcards. A single account at a site root can serve several APIs on that
    host, such as Confluence under /wiki/api/v2/... and Jira under
    /rest/api/3/....

    Connector dashboard and SDKs

    • The Connectors page lists configured connectors with auth mode, account
      count, status, base URL, and connect, accounts, test, and delete actions.
    • The add and edit wizard walks provider, auth, endpoints, credential rules,
      connect-time form, and review as separate steps. Every provider comes from
      the applied definition catalog or the custom definition; no provider list
      is hardcoded in the UI.
    • The account dialog renders the definition form and supports stored-secret
      references. The test dialog sends an authenticated provider path and shows
      the status and response.
    • Dialog actions sit in a sticky footer outside the scrolling body.

    SDK support:

    • TypeScript @vski/sdk 2.11.0 adds client.connectors with definitions,
      configs, accounts, OAuth start, and the request proxy.
    • Python vski-sdk 0.20.0 adds client.connectors and IaC loading, diffing,
      applying, and pulling for both connector kinds. The synchronous client
      exposes the same namespace.

    Agent-facing dashboard

    The dashboard can be handed a connection and opened on a single surface,
    which is what an agent does when a human has to act: approve a run, filter
    records, run a pipeline.

    • GET /connect?endpoint=...&token=...&path=... saves the deployment URL and
      its bearer token and forwards to path. Without a token it goes to login
      with the target kept as ?next=, so the user lands where they were headed.
      The endpoint is normalized to an http(s) origin, next stays
      dashboard-relative, and a stale saved API key is cleared: the dashboard's
      admin endpoints reject API keys and X-API-Key is checked before
      Authorization, so a leftover key would shadow the token.
    • /w/... routes render the same pages inside a chrome-less shell: no
      sidebar, connection badge, breadcrumbs, feature chips, or desktop titlebar,
      with dense padding. The page keeps its own actions — record create, edit,
      delete, run, save, approve, reject — because widget mode strips navigation,
      never capability.
    • Every surface has a widget route: /w/overview, /w/collections/:name
      (and /stats, /:id, /create), /w/workflows, /w/workflows/:runId,
      /w/signals, /w/pipelines (and /:name), /w/connectors, /w/cron,
      /w/gates, /w/webhooks, /w/api-keys, /w/secrets, /w/plugins,
      /w/migrations, /w/graphs (and /:name, /create), /w/logs, /w/sql,
      /w/settings (and /auth).
    • Widget query params are filter (list and collection tables), sort
      (-field for descending), page, perPage, and db (/w/collections,
      /w/graphs, /w/sql; database is accepted as an alias on the SQL
      console), plus status/collection for webhook logs. action=create
      opens the create dialog on a list page (or the record form on a collection),
      and action=run opens the run dialog in the pipeline editor. /w/sql also
      takes sql to pre-fill the query and execute=true to run it on load and
      show the results, so an agent can hand over a query and its outcome in one
      link. Links stay inside /w, and the surface name is the document title,
      so the browser tab identifies the widget.
    • The widget gains no leftover dashboard space: the SQL results modal fills
      the viewport instead of leaving the sidebar's column, and the pipeline
      canvas drops React Flow's minimap (it duplicated the canvas and took room
      from the step palette).
    • The Harness Account panel gains Open dashboard: Vski.dashboard
      (GET /api/vski/connections/:id/dashboard) builds a one-shot /connect
      URL signed with the selected slot's bearer token and opens it externally.
      An API-key slot yields a URL without a token, because the dashboard cannot
      be signed in with one.

    Signal approvals

    A run parked on a signal is a human decision waiting to happen, so the
    dashboard treats it as a first-class surface.

    • /signals (and the /w/signals widget) lists every waiting signal across
      the deployment: pipeline signal steps and code workflows parked in
      waitForSignal. Plugin-managed pipeline steps also park on a signal, but
      that is the plugin's wake-up, so the inbox hides it.
    • Each entry renders approve or reject with an optional reason and records
      the signed-in admin as by. Delivery uses a single-use workflow hook:
      create it with a random token, POST /api/workflows/hooks/:token (emits
      signal_received and requeues the run), then delete the hook. The payload
      is { ok, by, reason?, signalId? }. The run page renders the same form
      above its timeline.
    • A wait can carry a description: description on a pipeline signal
      step, or the option on waitForSignal / wait_for_signal. It is rendered
      with the run context when the step parks and carried in the signal_waiting
      payload, so the inbox and the run say what is being approved. Rendering is
      best-effort: an unresolved template path is kept as written rather than
      failing the run. The pipeline builder edits it in the signal step inspector.
    • Harness automation notifications carry the prompt as their body ahead of
      the run reference, and GET /repos/:name/signals returns it as
      description.

    Tests and docs

    • A new end-to-end suite covers definition storage, encrypted accounts, the
      authenticated proxy, a pipeline step with a templated account, stored
      secret references, and rejection of hosts outside the allow-list.
    • The connector engine and pipeline step have unit coverage for auth
      scheme application, OAuth refresh, stored secret resolution, templating,
      non-2xx failures, and host rejection.
    • Managed providers were validated end to end: Google over OAuth and
      Atlassian over one account token serving both Confluence and Jira.
    • Executor tests cover the signal description payload and its
      unresolved-template fallback; harness tests cover the description reaching
      Signal.Pending and the notification body.
    • The approval flow was validated against a running deployment: a parked
      pipeline run and a waitForSignal run both appear in the inbox, a response
      writes signal_received with {ok, by, reason, signalId}, the run resumes
      and completes, and the single-use hook is gone afterwards. Every widget
      route was opened against a running deployment — tables, editors, and config
      pages — checking for the compressed header, no sidebar or connection badge,
      the surface title in the tab, the filter and action params, and links that
      stay under /w; the normal routes were checked unchanged.
    • The wiki gains a Connectors page linked from the sidebar, and the IaC and
      Pipelines pages document the new kinds and step. The Workflows and Pipelines
      pages document signal descriptions.
    Downloads
  • v0.17.7 54ad6ccdac

    v0.17.7 Stable

    x released this 2026-11-08 19:53:10 +00:00 | 9 commits to main since this release

    VSKI v0.17.7: Durable replication, tango cluster proxy

    Replication is rebuilt around a durable, verifiable changeset log:
    every committed write is appended to an on-disk log before its
    transaction commits, replicas resume from an exact position, and a
    promoted replica serves its former siblings from the same log. Nodes
    switch between master and replica in-process, without a restart. On
    top of that ships tango, a new cluster proxy: it splits reads across
    replicas, journals every write to Redis, and fails over to the
    most-current replica. Acknowledged writes survive the failover, and none
    is applied twice.

    failover under load (200 writes/s + 400 reads/s, master stopped)
    acknowledged writes missing on the new master 0
    duplicated writes 0
    write outage (503 + Retry-After) 3.4 s
    old master rejoined as replica 2.0 s

    Upgrade notes:

    • Upgrade replicas before the master. A master on the new protocol
      answers older replicas with 426 Upgrade Required; they stay readable
      but stale until upgraded. Each replica re-downloads once after the
      upgrade.
    • workflow_events gains a seq column, added automatically on start.
    • Behavior changes, all listed below: replicas refuse more writes, the
      SQL console refuses transaction control, and transient write failures
      now return 503 instead of 403.
    • No SDK changes.

    Durable changeset log

    • Writes through the write queue are captured on the database's pinned
      connection and appended to DATA_DIR/replication-log/<db>.clog
      (CRC-checked, fsynced) before their batch commits. A failed commit
      removes its entry; crash recovery drops uncommitted tails.
    • Entries form a hash chain. A replica resumes with its last
      position plus the chain hash at that position. A position that is
      pruned, ahead of the master, or on a different history (a failover
      onto a node that missed writes) gets a full download instead of a
      mixed history.
    • Replicas mirror the master's log entry for entry, so after a promotion
      the remaining replicas catch up incrementally from the new master.
      A master restart costs replicas nothing.
    • Entries are kept for REPLICATION_LOG_RETENTION_HOURS (24); a replica
      offline longer re-downloads once. A database no replica has asked for
      is not logged, so standalone nodes pay nothing.
    • Full downloads use SQLite's backup API: snapshots include writes still
      in the WAL, and replicas restore in place, so open handles stay
      valid. Previously a replica's first full download could leave services
      on a closed database.
    • Writes from the SQL console and from migration scripts now go through
      the write queue, so they replicate. Inside them, transaction control
      (BEGIN, COMMIT, ROLLBACK, savepoints), ATTACH/DETACH and
      pragma assignments are refused: each request already runs in its own
      transaction.
    • A changeset carrying columns or tables a replica lacks is no longer
      silently skipped: the replica syncs the schema first.
    • Workflow events carry a per-run seq assigned when they commit, so a
      promoted replica hands workers the same event order as the old
      master.

    Roles, failover and replica safety

    • Hot promote/demote: POST /api/admin/promote and
      POST /api/admin/demote (superuser token) switch a node's role
      in-process: write path, syncer, changeset journal, pipeline dispatcher,
      cron, workflow gateway, webhooks, graph index.
    • Workflow execution is master-only: workflow gateway, scheduler,
      cron dispatch, webhooks and realtime run on the master alone.
      Replicas answer workflow writes and queue polls with 503.
    • workflows.db replicates by default as a passive copy, so a
      promoted replica starts with every run, queued job, timer and
      definition. Jobs the dead master had handed out are reclaimed within
      30–60 s and redelivered. REPLICA_INCLUDE_WORKFLOWS=false keeps it
      node-local.
    • Replicas no longer write locally. Every write to a replicated
      database is refused below the HTTP layer, so a replica never holds rows
      its master never saw. Logins still work on replicas. Registration,
      OAuth account linking, admin init and API key creation now return
      403 there and must go to the master; before, they silently created
      diverging rows.
    • Exactly-once replay: a write carrying X-Tango-Idempotency-Key
      records the key in the same transaction; a repeat answers
      200 {"duplicate":true} without writing. Keys replicate with the
      data and are pruned after IDEMPOTENCY_KEY_RETENTION (2h), which must
      exceed tango's journal retention.
    • Older replicas (pre-log protocol) get 426 Upgrade Required, logged
      once per replica every few minutes.
    • Security: /api/admin/replicas* now requires an admin.
      Previously any authenticated user could mint replica tokens.
    • Replicas report their positions to the master after every sync
      (POST /api/replica/sync-complete); every node exposes its own at
      GET /api/replica/local-status.

    Write path fixes

    • Transient failures are retryable: a write hitting a node that is
      shutting down, a full write queue, a replica, or a busy database now
      returns 503 with Retry-After instead of 403, which clients
      treated as final.
    • Write-queue batches take the write lock up front (BEGIN IMMEDIATE).
      A batch that read before writing could fail with "database is locked"
      when another connection committed in between; SQLite reports that
      without waiting.
    • Deleting a database released neither its replication session nor its
      connection handle, which could hang shutdown or crash the server.
      Sessions are now released before their connection closes.
    • The Docker images are built with SQLite sessions enabled. Without
      them a replica's first snapshot failed, so containers could not
      replicate.

    tango: the cluster proxy

    A new component (tango/): one endpoint in front of a vski master and
    its replicas.

    • Routing: writes go to the master. Reads go to the least-busy
      healthy node, except the workflow and automation plane (workflows,
      pipelines, cron, webhooks, plugins, secrets) and the realtime and
      workflow websockets, which stay on the master. Sessions (/sso,
      logins, /api/auth, /api/admins, /api/keys) and admin scripts
      (SQL console, migrations) go to the master and are never journaled.
    • Write journal: every write is appended to a Redis Stream before
      it is forwarded, with an idempotency key the master stores with the
      write. On failover tango replays, in order, every entry the new master
      may lack, including writes the old master acknowledged but never
      replicated. Writes whose client saw a failure are dropped, never
      replayed.
    • Failover: tango fences the old master, hot-promotes the replica
      with the most entries applied, then replays. In strict mode
      (default) writes get 503 + Retry-After during the window; in
      absorb mode they are journaled and answered 202.
    • Read freshness: a replica that falls behind the master by more
      than TANGO_MAX_READ_LAG (30s) leaves the read pool until it catches
      up. Optional read-your-writes (TANGO_READ_YOUR_WRITES) pins a
      client's reads to the master briefly after its write.
    • Actuators carry out role changes:
      • docker: Engine API, role files read by containers at start;
        replicas sync through tango, so promotions need no reconfiguration
      • kubernetes: role ConfigMap, the label the master Service selects
        on, and lease election between tango replicas
      • local: child processes, for development
      • manual
    • Management API (:8001): status, manual promotion, replay.

    Measured (tango/bench, three nodes on one host, k6 open model),
    p95 added by tango:

    500/s 1,000/s 2,000/s 4,000/s
    reads (point, list) +0.2 ms +0.2–0.3 ms +0.1–0.2 ms +0.1 ms
    record creates +0.1 ms +0.1 ms +5.7 ms +8.0 ms

    No errors at any rate. On writes, most of the cost is the idempotency
    key vski stores with each write; the benchmark breaks the cost down.

    Tests and docs

    • New end-to-end suites cover multi-replica convergence, master restarts,
      hot promotion and switchover, incremental catch-up after promotion,
      workflow state across failover, and divergence handling. vski's full
      end-to-end suite (47 tests) passes through tango, and tango's failover
      test proves acknowledged-but-unreplicated writes survive exactly once.
    • Replication.md documents the changeset log, upgrade order, replica
      read-only rules and the new settings; tango ships its own README and
      benchmark guide.
    Downloads
  • v0.17.6 40ec3f563f

    v0.17.6 Stable

    x released this 2026-11-08 00:05:13 +00:00 | 11 commits to main since this release

    VSKI v0.17.6 — Plugin gates, containers hardening

    Gates — the platform's named HTTP endpoints — can now target plugin
    functions natively
    (url: plugin://<instance>/<function>): the
    caller's JSON body is dispatched as the function's params over the warm
    plugin pipe and the result comes back verbatim. This replaces the
    credential-injecting HTTP bridge as the way to expose one plugin
    function to selected callers, and closes that pattern's hole where any
    caller passing the gate rule could invoke any function on the
    instance. The containers plugin gets the hardening pass from its
    official-plugin review.

    Safe upgrade: no schema migrations and no configuration changes; gates
    with http(s):// targets behave exactly as before. No SDK changes —
    the call side (client.gate(name, body)) was already generic.

    Gates: plugin:// targets

    A gate whose url is plugin://<instance>/<function> dispatches
    directly into the running plugin instance:

    • The request body becomes the function's params — it must be a
      JSON object (an empty body is an empty object); anything else is a
      400.
    • The function's return value is the response body, verbatim (200).
    • The function identity is fixed by the gate record — callers pick
      the payload, never the function. Anyone a gate rule lets through can
      do exactly one thing; no plugin admin key is involved at any point.
    • Dispatch rides the warm stdio pipe (no loopback HTTP hop) under the
      standard invoke budget (30 s default, 120 s hard cap). The instance
      must be running — 502 with the reason otherwise, as with invoke
      failures and disabled plugins.
    • The instance is looked up in the caller's database (x-dbname
      header, default default), and the gate's access rule is checked
      exactly as for HTTP targets before anything is dispatched.

    The web console's gate form accepts plugin:// URLs (validated, with
    an inline hint), and the platform wiki gained a dedicated Gates
    page — model, CRUD via the _gates records API, proxy semantics and
    their caveats (no timeout, no streaming, buffered bodies), plugin
    targets, and security notes. The MCP guide now leads with plugin://
    targets and keeps the old admin-key bridge documented as a compatibility
    pattern, with its caller-picks-the-function caveat spelled out.

    Containers plugin: official-plugin hardening

    • Adoption no longer resets the timeout. Docker reports container
      times as RFC3339Nano (up to nine fractional digits), which the
      timestamp parser rejected on older supported Pythons — an adopted
      running container silently got a fresh full timeout budget after every
      plugin restart. All timestamps now parse through one normalizing
      helper (naive values read as UTC).
    • Undeliverable completion signals give up. A completed job whose
      workflow run was deleted server-side used to retry its signal forever
      and pin its record against retention. Signals now retry for
      signal_retry_hours (new instance config, default 24; 0 = never
      retry), after which the record is marked signalGaveUp and becomes
      reclaimable. An attempt-count cap guards against tight retry loops.
    • The prune sweep never touches a live job's container. Containers
      whose job record is still queued or running are skipped regardless of
      age — previously a legitimately long job (a max_timeout above the
      sweep age) could be force-removed mid-run.
    • network: host and network: container:<id> are rejected — they
      defeat the isolation the plugin promises; user-defined Docker
      networks only.
    • Duplicate dispatches can no longer create two jobs — the
      find-or-create path of container.start is serialized.
    • Per-job runtime state (locks, cancel events) is released once a
      job completes and its signal settles, instead of accumulating for the
      process lifetime.
    • The distribution image pins vski-sdk 0.15.0 (Makefile venv and
      requirements floor alike), so the plugin's stats collector actually
      registers in the shipped image — with the previous 0.14.0 pin the
      feature silently no-oped.

    All of the above is covered by new unit tests (timestamp table,
    adoption-timeout with nanosecond stamps, give-up both ways, prune
    sparing live jobs, state release, concurrent start) and the full unit +
    end-to-end suites pass.

    Downloads
  • v0.17.5 2e362fb796

    v0.17.5 Stable

    x released this 2026-11-07 22:20:47 +00:00 | 12 commits to main since this release

    VSKI v0.17.5 — Plugin steps, plugin stats, WAL durability

    The plugin platform grows up in two directions: plugins can now
    define their own pipeline step types (a container step instead of
    the start → signal → check workaround) and report live stats — both
    process metrics the server collects itself and custom metrics the
    plugin publishes. Under the hood, the storage layer fixes a
    connection-pool race that could make pooled connections read through
    deleted WAL files (empty/stale reads, then database disk image is malformed on writes) — that class of failure is now structurally
    impossible from inside the server and blocked from outside it.

    Safe upgrade: no schema migrations; stats tables are created in
    stats.db on start and plugin catalogs are re-scanned automatically.
    Everything is on by default. SDKs: @vski/sdk 2.10.0, vski-sdk
    0.15.0.

    Plugin-defined pipeline step types

    A plugin manifest can now declare step types alongside its functions:

    steps:
      container:
        description: Build/run an ephemeral Docker container and report its exit code and output.
        params: containers/step.schema.json   # file path or inline JSON Schema
        functions:
          start: container.start              # required; must ack fast
          check: container.check              # optional, informational
    
    • Registry and validation. Scanning inlines the params schema into
      the catalog; step types resolve per database from plugins with an
      enabled instance. Type names must match ^[a-zA-Z0-9_-]{1,64}$ and
      collisions across plugins are rejected at validation. Param templates
      ({{...}}) are checked as usual; fully-resolved params are validated
      against the plugin's JSON Schema (draft 2020-12).
    • Asynchronous execution. The server dispatches the start function
      with the step params plus an injected context
      (runId, stepId, pipeline, db, signal, timeout, attempt)
      under a 30-second dispatch budget — the function acks and does its
      work in the background. The step parks on a deterministic per-step
      signal; the plugin completes it with
      client.workflow.send_signal(runId, signal, {ok, error, output}) —
      ok: false fails the step with error, anything else completes it
      with output. Completion is durable: signals that arrive before the
      park are buffered and replayed, runId/stepId give the plugin an
      idempotency key for re-dispatches, and the step timeout (1–86400 s)
      is passed through as the overall budget. Retries cover dispatch
      failures; a plugin-reported failure is terminal.
    • Builder. Plugin step types appear in the palette next to the
      builtins with their own icon and accent color, and the inspector
      renders a form from the params schema (typed inputs, enums as
      selects, JSON fallback for complex shapes).
    • The containers plugin ships a native container step — build/run an
      ephemeral container with resource caps, log capture and timeout
      enforcement — replacing the three-step workaround in a single node.

    Plugin stats: process and custom

    Every plugin instance gets a live stats view, from two sources merged
    by the 30-second stats collector:

    • Process stats (server-collected). Read from /proc per plugin
      child process: RSS, CPU % (utime+stime delta), open file
      descriptors, uptime — stored in _stats_plugin_processes.
    • Custom metrics (plugin-reported). A new stats JSON-RPC method,
      polled only from runtimes that advertise capabilities: ["stats"] in
      initialize (old runners are simply not polled). In py-sdk,
      collect metrics with @app.stats — multiple collectors share a 4 s
      budget and merge into a flat numeric map, stored one row per metric
      in _stats_plugin_metrics. Both tables are queryable system
      collections; retention is 7 days.
    • API. GET /api/plugin-instances/:name/stats?window=3600 returns
      bucketed process and per-metric time series plus the latest report's
      meta; GET /api/plugins/stats returns the latest snapshot of every
      instance with per-plugin rollups (instances, RSS/CPU/FD totals).
    • Console. The instance modal shows stat tiles (RSS, CPU %, FDs,
      uptime) with history charts and a custom-metric picker with a
      multi-line chart; the plugins page shows a totals strip per plugin.

    Storage: the server can no longer lose its WAL files

    Two failure paths could delete a live database's -wal/-shm while
    pooled connections still held them open: a shared-memory
    reference-counting race in the vendored SQLite fork under connection
    churn, and any external SQLite client closing the same file
    cross-process. Connections left on deleted inodes read silently
    stale/empty data (an API list suddenly returns []) and fail writes
    with database disk image is malformed until restart — the on-disk
    database was never actually damaged, but the server was effectively
    poisoned.

    All three layers are fixed:

    • The vendored fork never unlinks -wal/-shm — files left by a
      crash or external tool are recovered on the next open. (The fork's
      deliberate close-on-reset behavior that cured the benchmarked fd
      leak is untouched; only the unlink calls are gone.)
    • Each database holds a permanent guard connection, so no
      in-process close path can ever believe it is the last connection.
    • Out-of-SQLite sentinel file locks (shared lock over the database
      file's lock region and the shm dead-man byte), refreshed every
      second, make any external client's last-connection cleanup see a
      live holder and stop deleting.

    Regression tests cover the churn path and guard boundedness
    (internal/db/fdleak_test.go); the full suite is green.

    Console polish

    • Plugin instance configuration is now schema-driven forms — typed
      inputs, selects for enum fields, secrets with show/hide, JSON
      fallback — for both create and edit, instead of raw JSON textareas.
    • The plugin catalog moved into a modal behind the Scan button;
      the plugins page leads with stat cards and instances.
    • The sidebar can be hidden (choice remembered per browser), with
      a floating reveal button in the top-left corner.
    • Pipeline builder: combined Save + actions dropdown (Settings,
      Validate) and a truncated pipeline title with the full name on
      hover.
    • Consistent page headers and actions on Pipelines and Workflows
      (Refresh, then New), and the Authentication page uses the full
      content width.

    Documentation

    • Backup.md gains "Data directory access rules": the server is the
      only process that should touch DATA_DIR; -wal/-shm are normal,
      never hand-pruned; replicas sync over HTTP and never read the
      master's data directory; file-level backups require a stopped
      server or a replica.
    • Replication.md states the HTTP-sync model explicitly; Plugins.md
      and Pipelines.md document step-type declarations and the async
      step contract.
    Downloads
  • v0.17.0 5c48cc2fb5

    v0.17.0 Stable

    x released this 2026-11-06 17:29:28 +00:00 | 13 commits to main since this release

    VSKI v0.17.0 — Performance

    v0.17.0 is a planned, performance-focused release. The write path is
    rebuilt around a group-commit write queue, the storage layer's WAL
    lifecycle and defaults are tuned for sustained throughput, and the
    per-request costs around authentication, settings and realtime are
    reduced. The result: 10,000+ requests per second sustained across
    every benchmark scenario — reads, creates, updates and mixed traffic —
    with zero errors
    , and flat tail latency under load.

    scenario (10k rows, 60 s steps) sustained RPS p95 at 10k errors
    read-point 10,000+ 0.40 ms 0
    write-create (~150 B) 10,000+ 37 ms* 0
    write-update 10,000+ 8.6 ms 0
    mixed-80-20 10,000+ 0.67 ms 0

    * near-saturation batching (see below). Full methodology and the
    Postgres/MySQL reference comparison: benchmarks/BENCHMARK.md — at
    10k mixed, vski's 0.67 ms p95 also leads the Postgres (4.05 ms) and
    MySQL (23.0 ms) reference services on this rig.

    Safe upgrade: no schema migrations; API keys are re-hashed lazily on
    first use; every new behavior is on by default and individually
    disableable via environment variables.

    Write queue: group commit for record writes

    SQLite allows one writer at a time, so the server now makes every write
    cycle count: a per-database write queue funnels concurrent write
    requests through a single writer goroutine that commits them together in
    one transaction. N concurrent requests pay one commit cycle instead of
    N — and on a busy server that is the difference between the writer
    thrashing the write lock and staying quiet enough that readers never
    notice it (mixed 80/20 traffic sustains 10,000+ RPS at 0.67 ms p95,
    ahead of the Postgres reference service's 4.05 ms on the same rig).

    • Applies to record CRUD (single and bulk), graph node/edge writes, and
      vector embedding upserts.
    • Correct by construction: a request is acknowledged only after its
      batch commits, so a read following an acknowledged write always sees
      it. No eventual consistency anywhere.
    • A failing request (e.g. a UNIQUE violation) is rolled back to its
      savepoint and fails alone; concurrent requests in the same batch
      still commit.
    • A lone write under no load skips batching entirely and commits
      immediately — zero added latency at low traffic.
    • Under overload beyond capacity the queue applies backpressure:
      requests batch-wait (that is the ~37 ms p95 on pure write-create at
      10k) and fail loudly after WRITE_QUEUE_SUBMIT_WAIT_MS (5 s) instead
      of piling up unbounded. Errors stay at zero up to capacity.

    Configuration (all optional): WRITE_QUEUE_ENABLED (default true),
    WRITE_QUEUE_DEPTH (4096), WRITE_QUEUE_MAX_BATCH (128),
    WRITE_QUEUE_WINDOW_MS (2), WRITE_QUEUE_SUBMIT_WAIT_MS (5000).

    Storage layer: the server owns the WAL lifecycle

    • Auto-checkpointing is disabled on every connection; a background loop
      checkpoints databases whose WAL exceeds
      WAL_CHECKPOINT_THRESHOLD_MB (16 MB) — a PASSIVE checkpoint first
      (no write lock), then a TRUNCATE bounded by a 50 ms busy budget,
      retried every second. Under normal and mixed traffic the WAL stays
      in the low megabytes. Under extreme bulk-ingest bursts (10 KB records
      at 10k RPS ≈ 100 MB/s) it can grow into the gigabytes until the burst
      pauses and checkpoints reclaim the space automatically — size disk
      headroom accordingly for sustained bulk ingest, or tune the
      threshold.
    • Small WALs are periodically backfilled so slow synchronous=NORMAL
      trickles still get fsynced.
    • Pools are bounded (data 120 connections, aux 20) with idle pinned to
      the open cap; SYNCHRONOUS defaults to NORMAL — the recommended
      WAL-mode setting, and with group commit also the fast one.

    SQLite binding: pooled-connection hardening

    Pooled SQLite connections closed mid-run (idle trims, bad-connection
    discards) could leak file descriptors and crash under sustained load.
    The vendored go-sqlite3 fork fixes the shared WAL/shm handling, and
    the pinned-pool configuration removes the churn path entirely.

    Faster authentication and settings

    • API keys are stored as indexed SHA-256 hashes: key lookup is one
      indexed row read. Legacy bcrypt-hashed keys keep working and are
      transparently re-hashed the first time they are presented.
    • _system_settings reads are cached (short TTL): values such as
      the auth collection name were previously re-read and JSON-parsed on
      every authenticated request.

    Realtime and stats

    • The realtime gateway's per-topic event store enforces its size cap by
      actual serialized size with deterministic trimming; consumer-group
      offsets survive store compaction.
    • API hit statistics moved off the request path entirely: hits buffer in
      memory and flush as multi-row inserts (500 rows / 5 s), so stats
      collection never competes with user writes.

    Collections

    • Older collections get their automatic created index backfilled on
      start (consistent with newly created ones), keeping sorted and
      paginated list queries fast on upgraded deployments.

    Benchmarks

    The repository ships a dockerized k6 harness (benchmarks/) comparing
    vski against minimal Go APIs over Postgres and MySQL across
    read/write/mixed/auth scenarios with a rate ladder and early saturation
    stop. The v0.17.0 numbers above are reproducible with make vski and
    published via make publish (benchmarks/BENCHMARK.md, raw runs under
    benchmarks/results/).

    The report also includes measured read-side guidance for large
    collections: a selective filter on a 100k-row table goes from 1,250 to
    10,000 RPS with one index on the filtered column (collection index
    management), and skipping the per-request listing count with
    skipTotal=true doubles listing throughput at 100k records (5,000 →
    10,000 RPS).

    Downloads
  • v0.16.2 5c48cc2fb5

    v0.16.2 Stable

    x released this 2026-11-05 12:51:02 +00:00 | 13 commits to main since this release

    VSKI v0.16.2

    SSO with OAuth providers, storage statistics for the dashboard, and
    self-contained API docs. Safe upgrade from v0.16.0 — the one schema
    change (an optional scope column on _auth_providers) is applied
    automatically on first start; no configuration changes are required.

    SSO: hosted login for apps and CLIs

    Applications can now hand authentication to the server instead of
    building their own login screens. The flow is OAuth-like: the app sends
    the user to GET /sso/authorize?redirect_uri=...&state=..., the server
    renders a hosted login page, and on success redirects back with a
    one-time 60-second code that the app exchanges for a token via
    POST /api/auth/exchange — the same {token, user} shape as
    /api/auth/login, and the token never appears in a URL or browser
    history.

    • Both superusers and auth-collection users can sign in; superusers are
      checked first. A valid session skips the form on the next
      authorization (HttpOnly vski_sso cookie, 24h), and GET /sso/logout
      clears it.
    • Redirect targets are locked down: loopback addresses
      (127.0.0.1/localhost/::1, any port) and the vski:// scheme are
      always allowed; anything else must be listed exactly in the
      _system_settings key sso_allowed_redirects (JSON array).
    • The hosted pages are embedded in the binary and can be overridden
      per-file from TEMPLATES_DIR (default <DATA_DIR>/templates), with
      the template data documented in Authentication.md.
    • Login form fields are readable in both themes (explicit input colors
      with autofill overrides — previously nearly invisible in dark mode).

    OAuth providers on the hosted page

    The hosted login page shows a "continue with" row for every configured
    provider. The full server-side client flow lives at
    /sso/oauth/{provider} and /sso/oauth/{provider}/callback:
    server-bound state (10-minute, single-use), code exchange, user info
    resolution, external_auths linking with automatic registration on
    first sign-in, then back to the application with a fresh code.

    • Providers are configured as _auth_providers records — name, auth and
      token URLs, userinfo URL, client id/secret, enabled, and the new
      optional scope (a sensible default is used per known provider).
    • Fixed two latent bugs in the existing /api/auth/oauth2-login flow:
      the client secret is now actually sent in the token exchange, and
      providers that report the subject as sub (with no numeric id) no
      longer fail user resolution.
    • The callback URL is derived from the request (with PUBLIC_URL as an
      override) so it works behind proxies without extra configuration.

    vski login --browser (py-sdk)

    The CLI can now authenticate through the hosted page instead of a
    password prompt. The default method runs a local loopback listener
    (http://127.0.0.1:<port>/callback) and waits; --method deeplink
    uses the vski:// scheme instead, with --register-deeplink to
    register the handler with the OS (Linux .desktop, Windows registry).
    --no-browser prints the URL for headless use.

    Storage statistics

    GET /api/stats/storage (admin only — superusers and admin-role API
    keys) returns deployment-level numbers in bytes: deployment_size (the
    whole data directory), db_size (the selected database file including
    WAL sidecars, selected via x-dbname), and disk_available/
    disk_total for the disk holding the data directory.

    The dashboard's Databases page shows these as three cards under the
    SQL editor whenever no query results are displayed, refetching when the
    selected database changes. SDKs expose it as client.stats.storage(db?)
    (@vski/sdk 2.8.0, vski-sdk with the stats namespace).

    SQL console results modal

    Query results now open in a modal overlaying the SQL view instead of
    pushing the page down: full-area card with the same results header plus
    a close button, dismissible via the X, Escape, or a click outside;
    column headers stay pinned while scrolling. Dry-run status gets a
    64px success/error icon with the message.

    Self-contained API docs

    /openapi.html no longer depends on a public CDN: the ReDoc bundle
    (v2.5.1) is embedded in the binary and served from /openapi.js, so
    the docs render on offline and firewalled installations too.

    Downloads
  • v0.16.0 60143db7c2

    v0.16.0 Stable

    x released this 2026-11-04 17:16:32 +00:00 | 14 commits to main since this release

    VSKI v0.16.0

    Dashboard overhaul, collection stats, and sortable tables. Safe upgrade
    from v0.15.6 — no migrations, no configuration changes.

    The dashboard, redesigned

    The admin UI keeps its developer-tool character but stops shouting: the
    brutalist look (global monospace, weight-900 caps everywhere, square
    corners) is replaced by a calmer "refined technical" system.

    • Two rebuilt themes, light and dark, sharing one indigo accent and one
      sky secondary, with status colors tuned per theme. The toggle in the
      top bar works as before; your saved preference survives the upgrade.
    • Typography is split by role: the system sans stack for UI chrome, and
      monospace reserved for data — record ids, SQL, JSON, timestamps.
    • Consistent radii, borders, and focus rings across buttons, inputs,
      selects, modals, and cards; charts get themed tooltips, legends, and
      bar colors instead of hardcoded blues.
    • The sidebar is rebuilt as a flat, icon-labeled nav: Databases,
      Workflows, Pipelines and Cron on top, then Gates and Webhooks, then
      API Keys, Secrets and Authentication, with Plugins and Migrations last
      (used, but less often). Collection and graph trees live below it
      unchanged.

    Collection stats and feature indicators

    Collections had embeddings, FTS indexes, triggers and views configured
    on them, but the dashboard never showed any of it. Now it does.

    • Every collection view gets a Stats button opening a dashboard-like
      overview: tiles for records, API hits, errors, embeddings, realtime
      consumers and triggers; a 24-hour API activity chart with success and
      error bars; a features panel with on/off state and details (embedding
      model, dimensions and vector count, FTS fields, trigger definitions,
      index SQL); the schema field list; and created/updated timestamps with
      the API rule chips.
    • The collection header shows one small colored chip per enabled
      feature, each with a tooltip: embeddings (live from the vector table —
      pinned model, dimensions, vector count), full-text search, realtime
      subscribers, triggers (webhook and pipeline counts combined), SQL
      indexes, and views.
    • Empty charts everywhere — the home dashboard and the stats view — get
      a proper placeholder: a faded ghost bar chart over a dashed baseline
      with a hint, instead of bare "not enough data" text on white.

    Sortable collection tables

    • Clicking a column header cycles ascending → descending → off, with
      the direction shown by an arrow in the header. Sorting is server-side
      (the API's existing sort parameter), so it composes with pagination
      and filters.
    • On first load the table sorts by id descending when the collection
      has ids (same order as before), and by the first column otherwise.
    • The id column is only rendered when the table actually has one —
      collections without ids no longer show a phantom empty column.

    Fixes under the hood

    • Per-collection API stats were broken since the beginning: the stats
      middleware matched /collections/... while routes are mounted under
      /api/collections/..., so every hit was tagged with the collection
      name "api". Hits now record the real collection name, which is what
      the collection stats view and the home dashboard's API performance
      chart are built on.
    • Listing a collection whose table has no id column (tables adopted
      from raw SQL or migrations) failed the whole query with "no such
      column: id", because the implicit default order is id DESC. The
      record listing now detects the table's actual columns and falls back
      to ordering by the first column.

    Repository restructure

    • Release notes moved from docs/release/ to .release/.
    • Submodules renamed to match their packages: client → ts-sdk,
      vski-sdk → py-sdk; the emulator playground submodule was removed.
    Downloads
  • v0.15.6 5d2cca4d3a

    v0.15.6 Stable

    x released this 2026-11-03 21:33:17 +00:00 | 17 commits to main since this release

    VSKI v0.15.6

    Plugin process keys gain the workflows role. Safe upgrade from v0.15.5.

    Plugin keys can signal pipeline runs

    • The per-process API key the plugin supervisor mints for every instance
      now carries workflows alongside plugin. Plugin functions already
      wrote collections with their role-scoped key; now they can also call the
      workflow API, which is what an ack-fast plugin needs to report back:
      ctx.client.workflow.send_signal(run_id, name, data) completes a
      pipeline run's waiting signal step, so a pipeline that kicks a long
      background job can still finish (or fail) when the job actually lands.
      The docs MCP indexer uses exactly this loop: an index step acks in
      seconds, a signal step parks the run while indexing runs in the
      plugin, and a final step re-raises the error payload so a failed pass
      fails the run.
    • The role grants the workflow endpoints generally (runs, events, queue),
      not only signal delivery. Plugins are admin-installed supervised
      processes; the role keeps them off the admin surface.
    • Keys are minted fresh at every process start, so running instances pick
      the role up on their first restart after the upgrade. No migration, no
      configuration.
    Downloads
  • v0.15.5 fb811b3bcc

    v0.15.5 Stable

    x released this 2026-11-03 14:13:59 +00:00 | 19 commits to main since this release

    VSKI v0.15.5

    S3 file storage and file upload fixes. Safe upgrade from v0.15.4.

    S3 file storage

    • Setting S3_BUCKET now actually switches file storage to S3-compatible
      object storage (MinIO, AWS S3, or any S3 API). Files are stored as objects
      keyed {collection}/{recordId}/{filename}, the same layout local storage
      uses, and the /files/... download route streams objects directly from
      the bucket.
    • Configure with S3_ENDPOINT (host[:port], optional http(s):// scheme;
      empty means AWS S3), S3_REGION, S3_BUCKET, S3_ACCESS_KEY,
      S3_SECRET_KEY, and S3_FORCE_PATH_STYLE=true for MinIO. With S3
      enabled, the replica file journal stays off: the bucket is the shared
      storage, so replicas with the same S3 settings read files from it
      directly.
    • Note: the server now requires Go >= 1.26 to build from source, and the
      Docker images use the golang:1.26 builder.

    File upload fixes

    • Creating a record with a multipart file upload no longer fails validation
      on required file fields: the uploaded filenames are merged into the
      record before validation. Previously a required file field could never be
      satisfied on create.
    • The file e2e suites (TypeScript and Python) now perform a real multipart
      upload, download through /files/{collection}/{recordId}/{filename}, and
      verify that deleting a record removes its files. They previously asserted
      against a wrong URL prefix and effectively tested nothing.
    Downloads
  • v0.15.4 fb811b3bcc

    v0.15.4 Stable

    x released this 2026-11-03 02:31:04 +00:00 | 19 commits to main since this release

    VSKI v0.15.4

    Pipelines and validation fixes. Safe upgrade from v0.15.3; no behavior
    changes beyond the two fixes below.

    Pipelines and validation fixes

    • Pipeline http steps accept templated URLs (for example
      url: "{{input.source.url}}"): validation now checks the prefix only on
      literal URLs, matching what the executor always supported. The same
      applies to llm step baseUrl.
    • url fields accept local and port-bearing hosts
      (http://127.0.0.1:3000/..., http://localhost:5173/...) and query
      strings. Previously the validator required a dot-TLD hostname without a
      port, which silently rejected localhost URLs.
    Downloads