-
v0.15.3 Stable
released this
2026-11-03 00:37:39 +00:00 | 20 commits to main since this releaseVSKI v0.15.3
Security and reliability release. Upgrade as soon as possible: realtime
subscriptions now enforce collection access rules (a behavioral change for
service consumers, see below), and the workflow engine fixes address two
bugs that could lose or kill jobs.Realtime security
Subscriptions are now authorized like every other surface. The collection's
listrule is evaluated when subscribing and again for every delivered
event, per subscriber: records the caller may not read are never sent to
their socket, and closed collections (rules: null) reject non-admin
subscriptions outright. Admins bypass rules as elsewhere.Rule expressions using
x in @request.auth.rolesnow evaluate correctly;
previously such rules always denied.Behavior change: API-key consumers (plugin instances, service workers)
subscribing to a collection without alistrule will now be rejected.
Grant access explicitly, same as for REST:collections: notes: rules: list: "*" # or a role check: "'service' in @request.auth.roles"Workflow engine
- WebSocket writes are serialized per worker connection. Subscription
confirmations and job dispatches previously raced on one connection,
panicking the handler and surfacing as HTTP 500s on workflow triggers. - Nacked jobs are actually redelivered. Nack and the 30s visibility-timeout
requeue put the job back to pending but never cleared the in-memory
dispatch marker, so the job was skipped forever and its run hung in
running. Both paths now clear it.
SDK fixes (TypeScript)
- The workflow worker no longer drops jobs that arrive before it finishes
subscribing: they are nacked and redelivered after 5 seconds instead of
sitting stuck until the visibility timeout. - Execution timeouts above ~24.8 days no longer overflow the JavaScript
timer ceiling and instantly fail the run with a spurious
"CircuitBreaker: Execution timeout". - API keys are percent-encoded in WebSocket and hook URLs. Keys whose
base64 contains+previously failed authentication (the+decoded to
a space), which looked like random 401s depending on the generated key. - The Python SDK already encoded tokens correctly and is unaffected.
Testing
The e2e harness now uses the in-tree SDK by default (previous default
pointed at a stale sibling checkout), boots plugins with the SDK venv
python, and binds helper servers to ephemeral ports. The full suite passes
47/47, twice in a row.Downloads
-
Source code (ZIP)
2 downloads
-
Source code (TAR.GZ)
1 download
- WebSocket writes are serialized per worker connection. Subscription
-
v0.15.2 Stable
released this
2026-11-02 20:50:12 +00:00 | 22 commits to main since this releaseVSKI v0.15.2
vplugin.yaml: plugin manifests in YAML (vski-sdk 0.11.0)Plugins were declared in
vski-plugin.json. IaC made YAML the house
format, so manifests follow: a plugin directory now carries a
vplugin.yaml, same schema, YAML spelling.# plugins/hello/vplugin.yaml name: hello version: 0.1.0 title: Hello Plugin runtime: entry: hello.main:app # package.module:attribute → vski.plugin.Plugin python: ">=3.11" config: greeting: {type: string, default: hello} subscribe: collections: [notes] functions: echo: {description: Returns the greeting}- The server (catalog scan at boot and on
POST /api/plugins/scan)
discoversvplugin.yaml(andvplugin.yml); the legacy
vski-plugin.jsonis still read — first match wins — so existing
distributions keep working. New plugins should use the YAML name. - Whatever the source format, the API's
manifestfield stays canonical
JSON (the dashboard, TS SDK and e2e suites parse it as JSON; nothing
downstream changed). - The Python plugin runner (
python -m vski.plugin.runner) reads both
formats for its display-name fallback; PyYAML stays a CLI-extra
dependency — without it, YAML manifests simply degrade to an empty
fallback name instead of killing the process. examples/plugins/*and the app template's starter plugin are converted
tovplugin.yaml; the platform e2e suites run against them.
vski seed: seed files for dev and initial deploymentsIaC deliberately doesn't manage record data. Seeds fill that gap: sample
data for development and initial deployments, as YAML files in git —
typicallyseeds/, one file per seed, the file name is the seed name.# seeds/dev.yaml version: 1 database: default collections: notes: - id: seed-welcome # stable ids make re-runs idempotent title: Welcome body: Hello from a seed filevski seed seeds # augment (default): insert only missing records vski seed seeds --overwrite # re-apply: update existing records by id vski seed seeds --drop --yes # empty seeded collections first, then insert- Every applied file is recorded in a
_seedstable in the target
database (seed name + sha256 of the file); in the default mode an
unchanged seed is skipped, sovski seedis safe to re-run on every
deploy.--overwriteand--dropalways re-apply. - The three modes are mutually exclusive flags; augment is the default.
--dropdeletes existing records of the seeded collections (through the
records API, so realtime/webhooks/pipeline triggers fire), prompts in
interactive use and requires--yesunder--json. - Records without an
idare always inserted (the server assigns one);
give records ids when you want idempotent behavior. A manually deleted
seeded record stays deleted until the file changes (or--overwrite),
at which point it is re-inserted. - Seeding needs an admin API key (the
_seedstable is maintained through
the SQL endpoint) and the collections to exist — apply IaC first. Files
are strict like IaC: unknown keys, wrong versions and bad collection
names are load-time errors. vski seedis also a thin SDK surface:vski.seed.load_seeds()/
SeedRunnerfor programmatic use.
Template:
plugins/directory, seeds demo (app template)The app template matches the platform
conventions:src/is renamed toplugins/— what it always contained (plugin
source baked into the image at/app/plugins); the dev overlay now
mounts./plugins, the Dockerfile, Makefile, unit-test layout checks
and docs follow.- The starter plugin ships
vplugin.yaml; the layout unit tests parse
the YAML manifest (PyYAML rides in withvski-sdk[cli]). - A
seeds/dev.yamldemo plus amake seedtarget, and an e2e test
proving the loop: seed → records exist → re-run skipped by file hash. - Template pins move to
vski-sdk[cli]>=0.11,<0.12. - Fixed the template's unit suite against current pytest-asyncio (its
filterwarningsescalation crashed on the plugin's configure-time
deprecation;asyncio_default_fixture_loop_scopeis now set
explicitly).
Also in this release
- Wiki: Plugins and the
tutorial showvplugin.yaml; CLI
gains a Seeds section; IaC's
"not managed" list points record seeding atvski seed;
Project Boilerplate
reflects theplugins/layout. gopkg.in/yaml.v3becomes a direct dependency of the server (manifest
parsing; it was already in the module graph).
Tests
- Server: new unit tests for manifest parsing (YAML, legacy JSON,
canonical-JSON output, invalid inputs); the full services suite passes. - Python SDK: 10 new seed unit tests (loader strictness, hash
idempotency, augment/overwrite/drop semantics over a fake client) and a
seed e2e suite against a real server (apply →_seedshash row → skip →
drift +--overwrite→--dropcounts → destructive-guard). The full
e2e suite (130 tests) passes, including the plugins suites now running
against YAML manifests end-to-end. - Template: unit suite passes with the new layout; the seed e2e covers
apply + idempotent skip.
Compatibility
- Additive.
vski-plugin.jsonmanifests keep working (discovery order:
vplugin.yaml,vplugin.yml,vski-plugin.json); the catalog API is
unchanged — the storedmanifestis canonical JSON regardless of
source format. vski seedwrites one new table (_seeds) per seeded database,
created on first use through the admin SQL endpoint; no server changes
are required to use it against an existing deployment.- vski-sdk 0.11.0 (published as part of this release) keeps the 0.10 API;
the new surface isvski.seedand thevski seedCLI command. - Distributions copying the template should rename
src/→plugins/
and their manifests tovplugin.yamlat their leisure — the old layout
keeps working with the old manifest name.
Downloads
-
Source code (ZIP)
2 downloads
-
Source code (TAR.GZ)
2 downloads
- The server (catalog scan at boot and on
-
v0.15.1 Stable
released this
2026-11-01 16:20:54 +00:00 | 24 commits to main since this releaseVSKI v0.15.1
The
vskiCLI and declarative IaC (vski-sdk 0.9.0)v0.15.0 made pipelines, secrets and schedules first-class server citizens —
all of it managed so far through the dashboard and ad-hoc curl. v0.15.1 adds
the dev and devops standard on top: a Python CLI that covers the full
management API, and a versioned YAML format that converges a deployment from
files in git — collections, pipelines, cron jobs, webhooks, secrets, plugin
instances and graphs — with plan/apply/pull semantics and CI-friendly exit
codes.pip install "vski-sdk[cli]" \ --extra-index-url https://vski.sh/api/packages/x/pypi/simple/ export VSKI_URL=https://vski.example.com vski keys create dev --roles admin # raw key shown once export VSKI_API_KEY=vski_... vski status vski plan # diff ./vski against the deployment vski apply --yes # converge vski pull # export live state back to YAMLThe CLI
Every command resolves its target the same way — flags >
VSKI_URL/VSKI_API_KEY/VSKI_TOKEN/VSKI_DBenv > a.envfile >
~/.vski/config.json(written byvski login; same file the old Deno CLI
used, so workstations carry over). Admin-role API keys are full admins; a
workflows-role key scopes to pipelines and secrets.--jsonmakes every
command scriptable.Command surface:
login/logout/whoami/status,db,collections,
records(CRUD + search),sql/query,migrations
(list/status/generate/apply/rollback/upload/export),pipelines
(list/show/validate/run/tokens/trigger),runs(list/show/events/cancel
over workflow and pipeline runs alike),cron,webhooks,graphs,
secrets,plugins+plugins instances(start/stop/restart/invoke/logs),
keys,replicas— and the IaC verbsplan/apply/pull, both as
top-level commands and undervski iac.IaC v1 — the format
A deployment's desired state lives in a
vski/directory; every YAML file
under it is merged (recursively, in path order). The format is strict: each
file declaresversion: 1, and unknown keys at any level are errors, never
warnings — a typo never deploys half a resource.version: 1 database: default # scopes collections/graphs/plugins collections: notes: fields: - {name: title, type: text, required: true} rules: {list: "*", create: null} # null ≡ "" = admin-only indexes: [{name: idx_created, sql: "CREATE INDEX idx_created ON notes(created)"}] fts: {fields: [title, body]} pipelines: # global; the map key is the name note-intake: triggers: [{type: collection, collection: notes, actions: [create]}] steps: [{id: enrich, type: plugin, target: hello/echo, params: {message: "{{trigger.record.title}}"}}] cron: {nightly: {schedule: "0 2 * * *", type: http, url: "pipeline://note-intake"}} secrets: {openai_key: {fromEnv: OPENAI_API_KEY}} # values never in YAML plugins: {hello: {plugin: hello, config: {greeting: hi}}} graphs: {knowledge: {rules: {list: "*"}}} databases: {analytics: {description: Analytics}} # ensure-exists onlySemantics worth knowing:
- Sub-blocks are opt-in per resource. An omitted
rules:block is left
unmanaged; a present one is authoritative.fieldsis required when
creating a collection; modifying an existing field (type, required, ...)
is rejected with an explanation — migrations support add/drop only. - Collection changes always go through the migration system. The CLI
diffs desired vs live, renders an ordinary migration file (the same
format the dashboard's generator emits), uploads it via
POST /api/migrations/saveand applies it — schema changes keep their
audit trail, pre-migration snapshots and rollback. `--save-migrations ` also writes the generated YAML to disk for review artifacts. - Secrets are references, never values. Apply reads the env var at
execution time; a new secret whose env var is unset is a warning, not
drift, soplancan gate CI on convergence in environments that
legitimately don't hold the value. - Deletion is deliberate.
state: absentremoves one resource
(collections through a softdrop_tablemigration);--pruneopts into
deleting undeclared resources of the kinds your files manage. Databases
are never dropped by IaC. - Pipelines are validated server-side before anything is applied — an
invalid definition aborts the apply with the per-step issue list. planexits0clean,1on error,2on drift — a drop-in CI gate.pullexports live state (rules, indexes, views, FTS fields, pipeline
definitions, plugin configs) in the exact formatapplyconsumes; the
round trip is a no-op by construction and covered by tests. Secret values
come out asfromEnv: SECRET_<NAME>placeholders.
Template demo
The app template is the reference IaC
repository:vski/declares the notes collection, a hello plugin instance,
a note-intake pipeline and a nightlypipeline://kick;make iac-plan/iac-apply/iac-pullwrap the CLI, and the e2e suite proves the
loop end-to-end — a new note triggers the pipeline through the plugin's
warm function. New wiki pages: CLI
and IaC.Fix: FTS sync triggers broke all record writes on indexed collections
CreateFtsTableInDBgenerated its sync triggers with bare column
references (VALUES (new.id, title, body)— unresolvable inside the
trigger) and keyed the index on the TEXTidcolumn while FTS5 rowids are
INTEGER. Net effect: inserting or updating a record in any collection
with an FTS index failed (no such column: <field>/datatype mismatch). Triggers now readnew.*/old.*qualified columns and key on
the table's implicit integer rowid (content_rowid='rowid'), matching the
search path (rowid IN (SELECT rowid FROM <fts> ...)).Existing FTS indexes created by older builds still carry the broken
triggers — recreate them once (drop and re-add the FTS index in the
dashboard, or declarefts:in IaC and let apply drop/create it).Python SDK repairs (vski-sdk 0.9.0)
vski.api.secretswas unimportable since v0.15.0 (a broken merge left a
wrong base-class import and dead duplicated methods) — the secrets
namespace now loads and is exercised by tests.SyncVskiClientwas missing thepipelines,secretsandplugins
namespaces (the sync mirror had not kept up with the async client).settings.migrations.save(migration)added — posts a MigrationFile JSON
body toPOST /api/migrations/save(the upload endpoint round-trips a
file; save is what IaC apply uses)._version.pyresynced withpyproject.toml(was two versions behind).
The minimal Deno CLI in
@vski/sdk(login/db/migrate) is deprecated in
favor of the Python CLI; its README says so.Tests
- Python SDK: 30 new unit tests (config precedence, strict loader, the diff
engine including the pull→apply round-trip no-op property) and ten new
e2e tests across two suites — the CLI surface (login/keys/db/secrets/sql)
and the IaC lifecycle (plan drift, apply, convergence, field add/drop +
rule/pipeline mutations,state: absent, pull round-trip against a fresh
deployment, field-modification rejection). The full e2e suite — 117 tests
— passes. - Server: the repositories suite passes over the FTS trigger change; the
template's e2e covers an FTS-indexed collection receiving writes through
a pipeline — the case the old triggers broke.
Compatibility
- Additive: no API, storage or wire changes beyond the new SDK version and
the corrected FTS trigger DDL.vski-sdk 0.9.0keeps the 0.8 API; the
CLI arrives as an optional extra (pip install vski-sdk[cli]). - The FTS fix changes generated trigger DDL only; indexes created by older
binaries must be recreated once to pick up working triggers (see above). - IaC manages what its files declare: adopting it against an existing
deployment starts as a no-op —vski pullfirst, review, commit, then
letplangate your CI.
Downloads
-
Source code (ZIP)
1 download
-
Source code (TAR.GZ)
4 downloads
- Sub-blocks are opt-in per resource. An omitted
-
v0.15.0 Stable
released this
2026-10-28 11:39:32 +00:00 | 25 commits to main since this releaseVSKI v0.15.0
Pipelines v2: foreach fan-out, child waits, public trigger tokens, llm steps, encrypted secrets
v0.14.0 shipped pipelines with a deliberate "not yet" list. v0.15.0 clears
five of them: fan-out over arrays, waiting on spawned children, external
token triggers, a native LLM step, and an encrypted secrets store for
pipeline credentials. (The realtime canvas overlay was
since solved differently and stays out.)foreach— fan-out over arrays{ "id": "fan", "type": "foreach", "items": "{{trigger.record.tags}}", "run": { "type": "pipeline", "name": "tag-worker", "input": { "tag": "{{item}}" } } }One child run per element of the resolved array (template or literal, cap
1000). Each child's input is templated against the normal context plus
{{item}}(the element, type preserved) and{{itemIndex}}. By default the
step joins: it parks until every child settles and its output is
{ runIds, results }with one{ index, runId, status, output }entry per
item — downstream steps read
{{steps.fan.output.results[0].output.*}}."wait": falsequeues and moves
on. Any failed child fails the step (respectingonError). Fan-out is
at-least-once like every step: crash recovery replays the event log and
spawns only items without a child yet — a crashed fan-out never duplicates
children.wait: true— join on spawned childrenworkflowandpipelinesteps stay fire-and-forget by default; set
wait: trueand the step parks (children_waiting) until the spawned run
settles. Completion is pushed, not polled: when a child run reaches a
terminal status — pipeline settle, workerPATCH, cancellation — the
engine writes achild_completedevent onto the parent's log and requeues
it, so the wake-up survives restarts and works identically for worker
workflows and child pipelines. The step output is{ runId, status, output }
of the child; a failed or cancelled child fails the step with the child run
id in the error.Public trigger tokens
External systems can now start runs without any vski credentials:
curl -X POST https://vski.example/api/pipelines/note-intake/tokens \ -H "Authorization: Bearer $KEY" -d '{"name": "github-ci"}' # → { "id": "ptoken_...", "token": "ptk_..." } (raw token shown once) curl -X POST https://vski.example/api/pipelines/trigger/ptk_... -d '{...}' # → 202 { "runId": "wkf_..." } (no auth; token is the credential)The request body becomes the run input; the trigger context is
{ "type": "token", "token": "github-ci" }— the raw secret never lands in
the event log or run pages. Only a SHA-256 hash is stored; unknown and
revoked tokens both answer 404, disabled pipelines 409; the endpoint is
rate limited per IP (60/min per node). The builder's settings panel mints,
displays-once and revokes tokens.llm— native LLM step{ "id": "summarize", "type": "llm", "model": "gpt-4o-mini", "system": "You write one-line summaries.", "prompt": "Summarize: {{trigger.record.text}}", "jsonMode": true, "retries": 2 }One synchronous chat completion against any OpenAI-compatible endpoint —
OpenAI, Ollama, OpenRouter, vLLM, LM Studio.baseUrl/modelfall back to
theLLM_BASE_URL/LLM_MODELenv vars (then OpenAI defaults); the API key
resolves from the step'sapiKeySecret(a stored secret, see below) or the
server'sLLM_API_KEY— it never lives in a definition.
Output:{ content, parsed?, model, usage }— withjsonModethe reply is
parsed intooutput.parsed(a non-JSON reply fails the step, so downstream
paths are safe).messagesreplacespromptfor full threads; retries and
timeouts reuse the standard step machinery.Encrypted secrets store
Credentials for
httpheaders and LLM providers no longer need to sit in
definitions (which are snapshotted into run rows and rendered in the UI):curl -X PUT https://vski.example/api/secrets/openai_main \ -H "Authorization: Bearer $KEY" -d '{"value": "sk-..."}'Values are AES-256-GCM encrypted at rest (key:
SECRETS_KEY, or derived
with domain separation fromJWT_SECRET) and write-only — no endpoint
returns them. Storing and deleting secrets is admin-only (workflows-role
users can list names and reference secrets in definitions, same stance as
API keys and plugins). Runs decrypt them in memory only when the definition
references them:{{secrets.NAME}}in any templated field, or llm
apiKeySecret. A rotated key fails affected runs with a clear message
rather than returning junk. The dashboard gains Settings → Secrets, and
the llm inspector offers anapiKeySecretpicker fed by the store.Also in this release
- New run events on the pipeline timeline:
child_spawned
({ step, runId, index, kind, name }),children_waiting, and
child_completed({ step, runId, status, output }). - The builder canvas palette and inspector learn the two new step types,
thewaittoggle, a trigger-token manager, and an llmapiKeySecret
picker; a new Settings → Secrets page manages the store. - Validation covers the new surface: foreach
runspecs (type, name,
self-reference), llm field ranges andapiKeySecretnames, and their
template syntax.
Tests
test/e2e/55_pipelines_test.tsgrows to 14 steps: foreach fan-out with
join over three pipeline children (per-item templating, ordered results,
parentRunIdlinks),wait: trueon a workflow child unblocked by a
worker-style run PATCH, the full token lifecycle against the unauthenticated
endpoint (404 for unknown/revoked, 409 disabled, input passthrough,
non-leaking trigger context), anllmstep verified end-to-end against
an in-test OpenAI-compatible mock (request shape, system+user messages,
response_format, usage passthrough,parsedtemplating), and the secrets
store ({{secrets.*}}in http headers and set values, llmapiKeySecret
sent as the bearer, write-only listings, delete, and the admin-only guard:
a workflows-role API key gets 403 on store/delete but can still list). Go
unit tests
extend the executor suite (fan-out/park/join, no-respawn across requeues,
failed children, wait semantics, llm request shaping, secrets in the
template context, secrets not loaded for plain definitions) and add token,
child-completion-hook and secrets service tests over SQLite (encryption at
rest, key-rotation failure, llm key resolution order) plus a handler test
for the admin-only mutations.Compatibility
- Additive: two new step types, optional
waitfield, newpipeline_tokens
andsecretstables (workflows.db, registered as system collections),
new endpoints and SDK methods. Existing definitions and runs are
unaffected; in-flight runs keep their pinned snapshots. foreachjoins by default ("wait": falseopts out); spawn steps remain
fire-and-forget by default ("wait": trueopts in).- New env vars (all optional):
LLM_API_KEY,LLM_BASE_URL,LLM_MODEL,
SECRETS_KEY.llmsteps fail with a clear message when no key is
available against the default OpenAI endpoint. Secrets encrypt under
SECRETS_KEYor (derived)JWT_SECRET— rotating either invalidates
stored secrets, so setSECRETS_KEYexplicitly ifJWT_SECRETrotates. - JS SDK 2.5.0 and Python SDK 0.8.0 publishing is part of the release run
(client.pipelines.createTriggerToken/triggerByTokenand
client.secrets.set/list/remove, Python equivalents); the web builder
calls the token and secrets endpoints directly so it works with any
published SDK version.
What's next
Still open from the original list: compensation/rollback steps. Ideas under
consideration: per-foreach parallelism limits, cancellation propagation from
a failed parent to its children.Downloads
-
Source code (ZIP)
4 downloads
-
Source code (TAR.GZ)
3 downloads
- New run events on the pipeline timeline:
-
v0.14.0 Stable
released this
2026-10-24 13:49:27 +00:00 | 27 commits to main since this releaseVSKI v0.14.0
Pipelines: automations as data
Workflows have been VSKI's answer to "run this reliably" — but they live in
worker code you write, deploy and version yourself. v0.14.0 adds
pipelines: multi-step automations defined as JSON and edited on a visual
canvas, executed durably by the engine itself. No worker, no deployment —
the definition is the automation.A pipeline is a DAG of steps — call a plugin function, make an HTTP request,
compute values, wait on a timer, park for a human approval, spawn a child
workflow or pipeline — with per-step dependencies, conditions, retries,
timeouts and failure policies. Pipelines start from collection events
(record created/updated/deleted), cron schedules, or manually via API/UI.They are not a parallel universe: every pipeline run is an ordinary
workflow run namedpipeline.<name>, driven by the same event log, job
queues, waits and signals machinery. The run monitor, Gantt timeline, stats,
circuit breaker, execution timeouts and crash recovery all apply unchanged —
pipeline steps emit the standardstep_*events withcorrelationId=
step id, so the existing timeline renders them for free.The definition
{ "name": "note-intake", "triggers": [ { "type": "collection", "database": "default", "collection": "notes", "actions": ["create"] } ], "steps": [ { "id": "embed", "type": "plugin", "target": "indexer/reindex", "params": { "id": "{{trigger.record.id}}" }, "retries": 2 }, { "id": "approve", "type": "signal", "name": "approval", "dependsOn": ["embed"] }, { "id": "notify", "type": "http", "method": "POST", "url": "https://hooks.example.com/x", "body": { "id": "{{trigger.record.id}}", "by": "{{steps.approve.output.by}}" }, "onError": "continue", "dependsOn": ["approve"] } ], "output": { "indexed": "{{steps.embed.output.count}}" } }Step types:
plugin(warm in-process invoke,instance/function, timeout
1-120s),http(synchronous, every attempt in_webhook_logs, output
{status, body}),set(template evaluation into{{vars.*}}),delay
(durable wait — survives restarts, fires exactly at the resume time),
signal(approval gate — parks until a matching signal arrives, FIFO by
name, resumable viasendSignalor workflow hook tokens), and
workflow/pipelinespawn steps (fire-and-forget children linked via
parentRunId).Common step fields:
dependsOn(independent branches run in parallel,
bounded byoptions.concurrency),condition(data-only expression
language — comparisons,and/or/not, literals; sandboxed by construction),
retries(linear backoff),timeout,onError: "fail" | "continue"
(failed steps skip their transitive dependents while unrelated branches
continue;continuecaptures the error assteps.<id>.error).Templates:
{{trigger.*}},{{input.*}},{{steps.<id>.output|.status|.error}},
{{vars.*}},{{run.id}}- apply to any string in params/headers/body/input,
with array indexing; a whole-string template preserves the value's type.Execution semantics worth knowing
- Version pinning: each run snapshots the definition into its
executionContext. Editing or deleting a pipeline never mutates an
in-flight run. - At-least-once across crashes: the event log is the source of truth;
after a restart the dispatcher replays events and re-executes only steps
that hadn't completed (boot recovery re-pends orphaned jobs and requeues
active runs). - Per-run serialization: runs of one pipeline execute one at a time;
independent pipelines run concurrently. Steps of one run run in parallel
up tooptions.concurrency(default 4). - True suspension: while a run waits on a delay or signal it holds no
job; a failed branch doesn't fail the run while another branch still
waits. - Replicas stay cold: triggers are only registered and pipelines only
dispatched on the master.
Triggers
- Collection events: the pipeline subscribes from its definition, no
collection-options changes, no auth-header webhook wiring. Disabled
pipelines ignore events; the registry rebuilds on every definition change. - Cron:
{"type": "cron", "schedule": "0 9 * * 1-5"}inside the
definition registers an in-process named schedule (retired on
disable/update/delete, no cron-job rows). Ad-hoc: any cron job with
url: "pipeline://<name>"fires a pipeline. - Manual:
POST /api/pipelines/:name/runsor the UI Run dialog.
The builder
The dashboard grows a Pipelines section: a list (trigger chips, version,
last-run status) and a full-page visual builder on
React Flow - drag nodes to declaredependsOn,
per-type step inspectors parsed from live plugin catalogs (function lists
come from manifests), a JSON⇄Canvas toggle that round-trips the definition,
server-side Validate with per-step issues, and a Run dialog. A Runs
button polls the last 10 runs with a dot indicator (pulsing green while
anything is active), linking into the existing run views; the workflows page
accepts?workflowName=pipeline.<name>to pre-filter its list.API & SDKs
GET/POST /api/pipelines,GET/PATCH/DELETE /api/pipelines/:name
(definition replace bumpsversion; rename rejected),
POST /api/pipelines/validate(per-step issue maps: cycles, refs, cron,
durations, template/condition syntax, option ranges),
POST /api/pipelines/:name/runs. Auth: theworkflowsrole or admin, like
the rest of the workflow API.pipelinesis also queryable as a system
collection.- JS/TS (
@vski/sdk2.3.0):client.pipelines—list,get,
getDefinition,create,update,remove,validate,run; exports
Pipeline,PipelineDefinition,PipelineStep,PipelineTrigger,
ValidationIssuetypes. - Python (
vski-sdk0.7.0):client.pipelines—list,get,
get_definition,create,update,delete,validate,run.
Correctness fixes
plugin://cron jobs never worked — documented since the plugins
release, but the cron executor routed every command through the SQL
branch, soplugin://jobs failed as SQL syntax errors. Cronplugin://
commands now dispatch through the webhook retry/log pipeline over the
warm pipe, and replicas correctly skip all dispatch-type commands
(http/plugin/pipeline).ExecuteStandaloneWebhookdropped the job's database whenever a body was
set, breakingplugin://instance resolution for POST cron jobs.- The OpenAPI spec documented
POST/GET/PATCH/DELETE /workflows
definition CRUD endpoints that never existed (routes were never
implemented). The dead spec block is removed. - Workflows dashboard: pre-existing
'sdk' possibly nulltype errors fixed;
the run list prefills its filter from?workflowName=deep links.
Tests
test/e2e/55_pipelines_test.ts(9 steps): validation + CRUD, DAG with
parallel branches/join/condition skip/output templating, durable delay,
signal approval with templated payload, failure semantics (transitive
skips, unrelated branches complete,onError: continue), collection + cron
triggers (definition schedule andpipeline://command), spawn steps with
parentRunId, and replay-skip resume through the real dispatcher. The
engine gains its first Go unit-test suite (internal/pipelines): template
resolver, condition grammar, validator (cycles, per-type fields), and the
executor against a fake runtime (retries, suspension, resume-skip, vars
fold order, terminal idempotency), plus trigger-registry tests.Compatibility
- No breaking API or storage changes. One new table (
pipelinesin
workflows.db, registered as a system collection) and new, additive
endpoints/SDK namespaces. Existing workflows, triggers and cron jobs are
untouched. - Pipeline dispatch is master-only; upgrading a replica is a no-op for
pipelines. - The standalone Docker build now ships the SDK source beside the web app
(build/client), since the web app compiles against local SDK source. - JS SDK 2.3.0 publishing to vski.sh/npm is part of the release run;
external consumers need it forclient.pipelines.
What's next
Deliberately out of v1:
foreachfan-out over arrays, wait-for-completion
on spawned children (model it with asignalstep today), a public token
endpoint for external triggers, realtime status overlay on the canvas, and
a nativellmstep type. The Pipelines wiki
page documents all of it.Downloads
-
Source code (ZIP)
2 downloads
-
Source code (TAR.GZ)
2 downloads
- Version pinning: each run snapshots the definition into its
-
v0.13.0 Stable
released this
2026-10-22 14:11:52 +00:00 | 28 commits to main since this releaseVSKI v0.13.0
Light graphs: calendar-grade indexing
Light graphs landed in v0.10 as a relationship index for collections, and
the first heavy users all converged on the same ask: index events by
time. "What did this person do last week", "what changed on this date",
"how many hours accumulated on this edge" - the graph data model already
fit (nodes and edges carry labels, weights and free-form JSON properties),
but the ingest and query primitives were missing. v0.13.0 closes that gap
without growing the engine: no new tables, no migrations, one new index.Idempotent ingest:
upsert=trueIndexing pipelines re-run.
POST /graphs/:name/nodes|edges?upsert=true
(now alsograph.createNodes(data, { upsert: true })in the JS SDK)
replaces an existing id wholesale instead of erroring - the graph
equivalent ofMERGE, and the primitive every timetree ingester needs.
Upserts hitting an existing id additionally check the graph'supdateRule
(the item already passedcreateRulelike any create). The whole batch is
written in a single transaction: on failure nothing is persisted (batch
creates previously could persist a prefix of the batch).Atomic accumulation:
weightDeltaPATCH /graphs/:name/edges/:idacceptsweightDelta(mutually exclusive
withweight): the server reads the current weight under the graph's
write lock and adds to it, so concurrent increments cannot lose updates -
the classic "hours accumulated", "stock consumed" counter. Weights also
feed a new stats aggregate:GET .../statsnow reports
edgeWeightSumsper edge label (computed in SQL, where NULL weights are
distinguishable from 0) alongside a newnodeLabelshistogram.Property filters:
prop=path op valueThe JSON
propertiesmaps are no longer opaque. List endpoints accept
repeatablepropfilters -ref.collection=users,capacity>=5,
status!=draft,title~meeting,archived=null- compiled to SQLite
json_extractcomparisons with bound parameters (numeric-looking values
compare numerically,~is a substring match,nulltests for
absence). This makesproperties.refand capacity/stock metadata
first-class query targets. Nodes also acceptids=a,b,c(max 500) and
edgesource/targetaccept comma lists, so "everything touching last
week's day nodes" is one request.Node types in the index:
nodeLabelsNode labels are now interned in the in-memory topology index (alongside
edge labels and weights; ~4 bytes/node).neighbors,traverseand
subgraphacceptnodeLabels=a,bto restrict visited/returned nodes by
type - "documents linked to this day, not the people". Node label PATCHes
keep the index in sync; a SQL index onnodes(label)is created for new
graphs and backfilled idempotently for existing ones.Correctness fixes
- Edge weight PATCHes left the in-memory index stale on the master:
the index entry was rebuilt only when the label changed, and with the
pre-update weight - weighted shortest paths kept answering with the old
cost until restart. Weight and label changes now sync the index
immediately. - Shortest path ignored edge direction - it always relaxed both
adjacency lists, so directed graphs got undirected routes. The default
stays both-ways (documented behavior), butpathnow accepts
direction=out|in|both.
Calendar timetree, packaged
The Calendar Timetree recipe
documents the pattern end to end:week --includes_day--> daybackbone
nodes,record --on_day--> dayactivity edges with magnitudes on the
weight, deterministic ids so re-ingest is a no-op. The new
calendar-index example plugin automates it - subscribe it to a
collection with a date field and it maintains the tree on every
create/update/delete (record deletes cascade their edges; the calendar
backbone stays), withbackfillandweek_ofhelper functions. It uses
native upsert (hencevski-sdk >= 0.6.0) and needs read rules on the
source collection plus write rules on the target graph for its service
account (the recipe shows both).The server stays calendar-agnostic: any bucketing (sprints, fiscal
quarters) is just more backbone nodes with the same shape.SDKs
- JS/TS:
createNodes/createEdgestake an optional
{ upsert }second argument;updateEdgetakesweightDelta;
pathtakesdirection;getNodestakesidsandprop,
getEdgestakes arraysource/targetandprop;
neighbors/traverse/subgraphtakenodeLabels;GraphStatsadds
nodeLabelsandedgeWeightSums. All additive. - Python (
vski-sdkv0.6.0):create_nodes/create_edgestake
upsert=;update_edgetakesweight_delta=;get_nodestakesids=
andprop=,get_edgestakes list-valuedsource=/target=and
prop=;neighbors/traverse/subgraphtakenode_labels=;path
takesdirection=;GraphStatsaddsnodeLabelsandedgeWeightSums.
The sync wrapper mirrors everything automatically.
Tests
47_graph_test.tsgrows a "Calendar indexing primitives" step (upsert
replacement + duplicate rejection, weightDelta accumulation and
exclusivity, weight-patch freshness on weighted paths, directed path,
property/ids/multi-value filters, nodeLabels on neighbors and subgraph,
stats sums);wiki_graph_verify.tspins every new claim the wiki pages
make;graphdbunit tests coverSetEdgeWeight,SetNodeLabel,
directed paths and node-label filters. On the Python side, the SDK's
test_47_graphs.pymirrors the new surface andtest_53_calendar_index.py
exercises the calendar-index plugin end to end (tree build, re-ingest
idempotency, delete cascade,week_of,backfill). A pre-existing flaky
SchemaFromMaptest (map iteration order) is now order-independent.Compatibility
- No breaking API or storage changes; everything new is opt-in query/body
parameters. Path defaults to the historical both-ways walk; plain
POST stays insert-only. - No migrations: the node-label index is
CREATE INDEX IF NOT EXISTS,
applied on graph creation and backfilled lazily (master only) when an
existing graph's index loads. - Upsert replaces
label/properties(and edge endpoints/weight)
wholesale - it does not merge. Pipelines that relied on the
create-then-update fallback ingraph-synckeep working unchanged.
Downloads
-
Source code (ZIP)
5 downloads
-
Source code (TAR.GZ)
2 downloads
- Edge weight PATCHes left the in-memory index stale on the master:
-
v0.12.0 Stable
released this
2026-10-22 00:30:07 +00:00 | 29 commits to main since this releaseVSKI v0.12.0
Plugins: supervised Python beside the server (experimental)
Every real deployment grows a system around VSKI: an embeddings indexer,
a record-to-graph sync, a crawler, a chat bot. Until now those lived as
undocumented sidecars - cron entries, hand-rolled daemons, HTTP triggers
pointing at localhost - with no shared lifecycle, no configuration story
and no observability. v0.12.0 makes that pattern first-class: plugins are
Python processes supervised by the server, configured and observed through
the admin API and dashboard, running in the same container with zero
network hops.The design stance matters as much as the feature:
- Plugins are reusable source, not packages. Think shadcn components:
you copy one into your distribution, verify its dependencies against
the rest of your environment, tune its config schema, and build one
image that just works. The server never installs or manages Python
dependencies - the official images ship no Python at all, and
distributions that want plugins bring their own locked interpreter
(the wiki has the Dockerfile snippet). - An instance is data. Plugin + database + JSON config + enabled
flag. Changing an API key or cloning a bot for another chat group is a
PATCH or a POST that restarts exactly one process - never the server,
never the container. - Not a sandbox. Plugin code ships inside your image and is trusted,
exactly like workflow workers. There is no upload endpoint; code
arrives via the filesystem only.
Plugins are experimental and disabled by default
(PLUGINS_ENABLED=trueto switch on; when off, every plugin endpoint
answers 503 with setup instructions).How it works
A plugin is a directory under
PLUGINS_DIR(default
DATA_DIR/plugins) containing avski-plugin.jsonmanifest (name,
runtime entry, config schema withsecretfields, declared deps as
metadata) and a Python package. The supervisor spawns
$PLUGIN_PYTHON -m vski.plugin.runnerinside it -PLUGIN_PYTHONmust
havevski-sdk>=0.5.0(and the plugin's dependencies) installed by the
distribution.- Control plane: protocol frames on the process's stdin/stdout
(initialize / ping / invoke / shutdown), logs on stderr. A warm
invocation is a pipe write - no socket, no cold start. - Health: pinged every 10s; an unresponsive process is killed and
restarted. Crashes restart with exponential backoff (1s doubling to
60s, reset after 5 minutes of uptime); therestartscounter tracks
crash restarts.kill -9recovery is covered by the e2e suite. - Credentials: each process start mints a fresh API key with the
pluginrole and deletes it when the process exits - key material
exists only in server memory and the child environment. The key is a
normal authenticated principal: collection and graph access rules
apply to it. - Events: plugin handlers subscribe through the existing realtime
websocket using a per-instance consumer group - at-least-once delivery
with ACK-committed group offsets, so events that occur while a plugin
is down are redelivered when it comes back (see the realtime section
below). - Replicas never supervise plugins (like webhooks and realtime);
instance status stays readable through the synced registry.
API
All admin-only:
GET /api/plugins,POST /api/plugins/scan- catalog from manifestsGET/POST /api/plugin-instances(instances are unique per
(dbName, name)),GET/PATCH/DELETE /api/plugin-instances/:name?db=- a config change bumps
configVersionand restarts only that
instance's process
- a config change bumps
POST .../:name/start|stop|restartPOST .../:name/invoke-{function, params, timeoutSeconds?}
(default 30s, max 120s) over the warm pipeGET .../:name/logs?level=&limit=- captured stderr (newest first;
kept 7 days in_plugin_logsin stats.db)
Collection triggers and HTTP cron jobs accept
plugin://<instance>/<function>as their URL: the webhook engine
dispatches it in-process with the same retry policy and the same
_webhook_logspipeline - a zero-hop alternative to pointing a trigger
at localhost. Cron jobs resolve their own database for the instance
lookup.Dashboard
Settings → Plugins: catalog cards from the manifests, the instances
table with live status chips / restart counts / uptime, a config editor
generated from each manifest's schema (saving restarts just that
process), an invoke console (function picker + params + result), a
captured-log tail, start/stop/restart actions, and a disabled-state
banner pointing at the env flags. The dashboard now depends on
@vski/sdk@^2.2.0.Writing a plugin
from vski.plugin import Plugin app = Plugin() @app.on_event("documents") # realtime consumer group, at-least-once async def on_document(ctx, event): ... @app.function("reindex") # callable via /invoke and plugin:// triggers async def reindex(ctx, params): ... @app.cron("*/5 * * * *") # dependency-free 5-field cron async def sweep(ctx): ...ctxcarries a db-scopedclient, the instanceconfigand alog
whose output the server captures. Four working examples ship in
examples/plugins/:hello(every handler type, doubles as the e2e
fixture),note-bot(the wiki tutorial plugin),embeddings-indexer
(event-driven chunking/embedding via an OpenAI-compatible endpoint -
formalizes the Embeddings tutorial's "next steps") andgraph-sync
(mirrors a collection into a light graph withproperties.reflinks -
formalizes the Graphs tutorial's sync pattern).Realtime: durable consumer groups and fixes
Work on plugin event delivery hardened the realtime gateway for
everyone:- API keys are now accepted on
/api/realtime(?auth=<key>,
any enabled key) - service processes can subscribe like any client. - Consumer groups are durable. ACKs now advance a per-group
committed offset, and consumers joining later (or plugin processes
restarting) resume after it instead of tailing from "now" - events
that fired while nobody was connected are no longer skipped. Group
deliveries now carry theackIdfield both SDKs already expected
(neither ever ACKed before, because the server never sent it). - Fixed: a fresh group consumer received one stale event from the
head of the untrimmed event store (delivery position initialized
from zero instead of the subscription position). - Fixed: event frames carried the id of the first stored event
with a matching action rather than their own, so ACKs could commit
the wrong position. - Event ids are zero-padded (
<millis>-<seq>), making the id ordering
used throughout the gateway numerically correct.
Non-group subscriptions are unchanged. Delivery remains at-least-once
for groups.Supervisor hardening
Start/stop transitions are serialized per instance (a restart can no
longer race the reconciler into double-spawning an orphaned process),
shutdown waits are bounded, and no process is spawned once shutdown has
begun.SDKs
- TypeScript/JS (
@vski/sdkv2.2.0): newclient.plugins
namespace -list,scan,listInstances,getInstance,
createInstance,updateInstance,deleteInstance,
startInstance/stopInstance/restartInstance,invoke,logs-
with full types (PluginCatalogItem,PluginInstance,
PluginRuntimeInfo,PluginLogEntry). - Python (
vski-sdkv0.5.0): thevski.pluginmodule described
above, a matchingclient.pluginsmanagement namespace, and three
transport fixes: request paths with an embedded query string were
silently dropped (httpx replaces a URL's query when givenparams,
even empty), websocket auth tokens are now URL-quoted (raw-base64
API keys containing+previously failed as spaces), and group
subscriptions tracklastIdso a reconnecting consumer resumes
after the last seen event.
Documentation & tests
Wiki: Plugins reference
(architecture, manifest spec, env contract, API, security model,
distribution Dockerfile) and a hands-on
Plugins Tutorial
whose flows are covered end-to-end by the new suites:
48_plugins_test.ts(15 steps: catalog, lifecycle, warm invoke, event
delivery, captured logs, per-instance restart on config change,
plugin://trigger delivery, clone-into-second-db, crash recovery,
admin gating, disabled-503) andtest_48_plugins.py(instance
lifecycle, event delivery,plugin://triggers, admin gating,
including downtime redelivery). The e2e harnesses now start their
servers with plugins enabled and the examples directory mounted.Compatibility
- No breaking API or storage changes. Plugins are off by default; the
plugins/plugin_instancestables (default.db) and_plugin_logs
(stats.db) are created idempotently on boot. - Realtime group consumers start ACKing and resume from committed
offsets - strictly fewer lost events; clients that manually ACKed are
unaffected. The padded event-id format only matters within a single
server process (the event store is in-memory). - Official docker images are unchanged (no Python); plugin-enabled
distributions buildFROMthem per the wiki.
Downloads
-
Source code (ZIP)
2 downloads
-
Source code (TAR.GZ)
2 downloads
- Plugins are reusable source, not packages. Think shadcn components:
-
v0.11.1 Stable
released this
2026-10-21 16:19:43 +00:00 | 30 commits to main since this releaseVSKI v0.11.1
Fix: per-item access rules on graphs
Graph list and read endpoints evaluate their access rule twice: once as
a gate when the request arrives (no node or edge in hand yet), then
again per item. The gate ran the rule with no item data, so any rule
that referenced item fields - for examplelistRule: "label = 'public'"- failed evaluation at the gate and every non-admin request
errored out. The per-item pass documented in v0.10.0 (filtering,
scan-ahead,"filtered": trueresponses) never ran.The gate now defers to the per-item evaluation when a rule references
item fields:- empty rules still deny,
"*"and auth-only rules
(@request.auth.id != '') still decide at the gate, and malformed
rules still error; - single reads (
GET .../nodes/:nodeId,.../edges/:edgeId) apply
viewRuleper item, so a field-based view rule hides individual
items instead of failing the request; - list endpoints filter with the documented scan-ahead and report
"filtered": true, counting only visible items; - traversal endpoints (
neighbors,traverse,path,subgraph,
stats) remain strictly gated byviewRulewith no per-item pass,
unchanged.
The behaviour is covered end-to-end in both the Deno and Python graph
suites: a field-basedlistRule/viewRuleagainst a non-admin client,
including thefilteredflag and per-node denial.Documentation
The project wiki gained a
Graphs reference page and a
knowledge-graph tutorial -
linking collection records through a graph (works_on/authored/
depends_onedges), traversal queries, a sync pattern for record
changes, rules, and the dashboard explorer. The Collections page was
corrected along the way:selectfields take
options: { values: [...] }, not an array.Compatibility
- No API, storage or SDK changes:
@vski/sdkstays at 2.1.1,
vski-sdkat 0.4.1. - Rules that already worked behave identically; requests that
previously errored (field-referencing rules) now return filtered
results. - Full-suite runs still show the environment-flaky
gates/realtime/workflow-hook tests known from earlier releases -
failing sets vary between runs of the same binary, and they reproduce
on the committed v0.10.x baseline with clean data. The graph suite
passes consistently.
Downloads
-
Source code (ZIP)
0 downloads
-
Source code (TAR.GZ)
2 downloads
- empty rules still deny,
-
v0.11.0 Stable
released this
2026-10-21 14:55:49 +00:00 | 30 commits to main since this releaseVSKI v0.11.0
Graphs in the dashboard
v0.10.0 shipped light graphs with SDK-only access; this release adds the
follow-up web update. Graphs are now first-class in the dashboard: browse
them alongside collections, explore a subgraph interactively, and edit
nodes and edges without leaving the browser.What's in the UI
- Sidebar: a Graphs section lists every graph grouped by database,
live-updating as graphs are created or deleted. The collections-style
header links to the index page and the create form. - Index (
/graphs): all graphs across all databases with direction,
caps and creation date; delete with confirmation. - Create (
/graphs/create): name, target database, directed/undirected,
optional per-graphmaxNodes/maxEdges, and the five access-rule
fields - the same inputs the SDK accepts. - Explorer (
/graphs/:name?db=): the main page.- A stats strip (node/edge counts against effective caps with usage
bars, average and max degree) and clickable edge-label chips that
double as label filters for every query on the page. - A query bar: searchable center node, hops (1-5), result limit and
direction. The page auto-seeds from the most recent node so a graph
is never a blank screen. - The canvas renders the subgraph with Cytoscape.js and the fCoSE
layout: distinct seed node, per-label edge colors, directed arrows,
zoom/fit controls, and theme-aware colors for both light and dark. - Click a node or edge for a detail panel with properties and actions:
Center here re-seeds the view, Expand merges one-hop neighbours
into the canvas without a full reload, plus edit and delete. - A shortest-path finder: pick two nodes (same search), choose
hops or weight cost, and the path is highlighted on the canvas with
everything else dimmed. - Full node and edge CRUD from modals - including endpoints picked via
node search when creating an edge.
- A stats strip (node/edge counts against effective caps with usage
Cytoscape is lazy-loaded with the explorer page, so the main dashboard
bundle does not grow. The embedded dashboard in the standalone binary
ships the same pages.Substring search for nodes and edges
Listing endpoints only supported an exact-match
labelfilter - usable
for slicing by label, useless for finding a node in a graph that may hold
100k of them. Both list endpoints now acceptq:GET /api/graphs/:name/nodes?q=substring-matches node id and labelGET /api/graphs/:name/edges?q=substring-matches edge
id, source, target and label
qcombines with the existinglabel/source/targetfilters and
pagination, and the usual access rules still apply. The dashboard's node
pickers are backed by it (debounced, 20 results per keystroke).SDKs
- TypeScript/JS (
@vski/sdkv2.1.1):getNodes/getEdgesaccept a
qsubstring-search option. - Python (
vski-sdkv0.4.1):get_nodes/get_edgesaccept the
matchingq=keyword (mirrored by the sync wrapper). No other surface
changes.
Fixes
- Dashboard: edge-label chips moved below the canvas, next to the
legend they explain. - Dashboard: the canvas now renders correctly on both themes. daisyUI
5 serves theme colors asoklch()and Chromium preservesoklchin
computed styles, which Cytoscape's canvas cannot parse - every color is
now converted to sRGB through a canvas pixel readback, and theme flips
re-resolve node, edge and label colors live. - Dashboard: the sidebar footer is decluttered - the OpenAPI link
moved to the connection modal footer (left of Retry Connection),
leaving user actions alone in the sidebar.
Compatibility
- No breaking changes.
qis an additive query parameter ignored by
older clients; older servers ignore it from newer SDKs (results are
unfiltered). - Access rules, caps and realtime publishing behave exactly as in
v0.10.0; nothing about the storage or index changed. - Python e2e: the graph suite covers substring search; one workflow
websocket test is flaky under full-suite ordering (passes in
isolation, reproduces on the v0.10.0 backend as well - not a
regression from this release).
Downloads
-
Source code (ZIP)
0 downloads
-
Source code (TAR.GZ)
2 downloads
- Sidebar: a Graphs section lists every graph grouped by database,
-
v0.10.0 Stable
released this
2026-10-21 10:54:05 +00:00 | 31 commits to main since this releaseVSKI v0.10.0
Light Graphs: retiring the Neo4J sidecar
Most tenants that needed graph workloads ran Neo4J alongside VSKI.
Looking at actual usage, 99% of those use cases - agent graphs,
knowledge linking, dependency walking - never touched the advanced
features (Cypher, clustering, multi-hop pattern matching at scale) and
never exceeded 100k edges. What they did incur was the cost of running
and maintaining a separate database service: another deployment unit,
another backup/replication story, another source of truth to keep
consistent with VSKI data, another thing on-call has to know.Light graphs move that workload into VSKI itself. Graphs are now a
first-class entity type alongside collections - same
(dbName, name)namespacing, same access-rule model, same single
SQLite storage engine, same replication - so the graph and the content
it indexes live in one place. For the workloads tenants actually run,
this removes the Neo4J dependency and its engineering overhead
entirely; no new infrastructure ships with it.How it works
A graph is a named, db-scoped registry entry (same tombstone/revive
semantics as collections) whose data lives in two physical tables in
the tenant database:graph_<name>_nodesandgraph_<name>_edges
(labelled, directed edges with optional numeric weight, cascading
deletes on endpoints). SQLite stays the source of truth - no separate
service, nothing new to operate.Traversal performance comes from an in-memory index: on first access
each graph's topology (ids, edge labels, weights - never properties)
is loaded into a compact adjacency index with nodes and edges
interned touint32slots. A fully-warm 100k-node / 100k-edge graph
occupies ~20-25 MB and a worst-case BFS runs in single-digit
milliseconds. Mutations serialize per graph (check + SQL write +
index update under one lock), so size caps cannot be raced and the
index cannot drift from committed state. Cold graphs are evicted LRU
(GRAPH_INDEX_MAX_GRAPHS, default 32) and reload from SQLite on next
touch; replicas revalidate against the database epoch and reload
transparently.API
Registry (mirrors
/api/collections):GET/POST /api/graphs,
GET/PATCH/DELETE /api/graphs/:name(soft delete keeps data;
re-creating the same name revives the graph with its contents).Data & traversal:
GET/POST /api/graphs/:name/nodes,GET/PATCH/DELETE .../nodes/:nodeIdGET/POST /api/graphs/:name/edges,GET/PATCH/DELETE .../edges/:edgeIdGET .../nodes/:nodeId/neighbors?direction=out|in|both&labels=a,b&limit=GET .../nodes/:nodeId/traverse?depth=N&direction=&labels=&limit=(BFS, max depth 10)GET .../path?from=&to=&maxHops=&cost=hops|weight&labels=(BFS or Dijkstra over edge weights)GET .../subgraph?nodes=a,b&hops=N- neighborhood around seed nodesGET .../stats- node/edge counts, label histogram, degree stats, effective caps
Nodes and edges accept single objects or arrays on create; ids are
optional (15-char generated). Deleting a node cascades its edges.Rules
The five collection rule fields apply with collection semantics: empty
rule denies non-admins,"*"is public, admins bypass. List endpoints
evaluate rules per item (non-trivial rules scan ahead to fill the page;
the response carries"filtered": trueand counts only visible items
when that happened). Traversal endpoints are gated byviewRule.Caps
GRAPH_MAX_EDGES/GRAPH_MAX_NODES(default 100000 each) bound every
graph server-wide; a graph may declare a lower per-graph cap via
options.maxEdges/options.maxNodes. Exceeding the cap is a clean
400 (graph edge cap exceeded (...)) from the per-graph write lock.The caps exist to keep the memory story boring: every graph is bounded,
so the total index footprint is bounded by the number of warm graphs,
and nothing about the feature changes VSKI's single-binary, single-file
storage profile.SDKs
- TypeScript/JS (
@vski/sdkv2.1.0): newclient.graph(name, db?)
/client.db(name).graph(name)namespace (node/edge CRUD,
neighbors,traverse,path,subgraph,stats) and
client.settings.graphsregistry namespace. New types:
GraphConfig,GraphNode,GraphEdge,GraphStats, and the
traversal result types. - Python (
vski-sdkv0.4.0): matching surface -client.graph(name, db)/client.db(name).graph(name)(get_nodes/create_nodes/
get_node/update_node/delete_node, the same five for edges,
neighbors,traverse,path,subgraph,stats) plus the
client.settings.graphsregistry namespace andGraphConfig/
GraphNode/GraphEdge/GraphStatsTypedDicts. The sync wrapper
mirrors both (SyncVskiClient.graph(...)). The e2e harness (including
the replica suite) now honoursVSKI_BINARY/VSKI_DIRoverrides, so
it runs against any server checkout; the suite passes on pytest 8 and 9.
Fixes
- Collections: the drop index/view/FTS routes
(DELETE /api/collections/:name/indexes/:indexName,
.../views/:viewName,.../fts/:ftsIndexName) were documented and
implemented but never registered by the router - every call 404ed. They
are wired now; the SDKdrop_view/drop_index/drop_fts_index
methods work against them. - Python SDK docs:
set_viewswas documented as full-list
replacement, but the server only ever adds views - slugs missing
from the new list stay until dropped individually. The docstring now
states the additive semantics and points atdrop_view(which takes
the full materialised<collection>_view_<slug>name).
Compatibility
- No breaking changes. The
graphsregistry table and the unique index
uq_graphs_db_nameare created idempotently indefault.dbon boot;
tenant tables are created per graph on demand. - Existing Neo4J deployments are unaffected; migration to light graphs
is per-workload (create the graph, replay nodes/edges through the
SDK or a workflow) and can be done incrementally. - Graph mutations publish realtime events on the
db:<graph>topic
(subscription UI support follows in the dashboard update). - Dashboard: graph management pages are not part of this release; use
the SDK or the SQL console until the follow-up web update.
Downloads
-
Source code (ZIP)
0 downloads
-
Source code (TAR.GZ)
2 downloads