-
v0.15.0 Stable
released this
2026-10-28 11:39:32 +00:00 | 25 commits to main since this releaseVSKI v0.15.0
Pipelines v2: foreach fan-out, child waits, public trigger tokens, llm steps, encrypted secrets
v0.14.0 shipped pipelines with a deliberate "not yet" list. v0.15.0 clears
five of them: fan-out over arrays, waiting on spawned children, external
token triggers, a native LLM step, and an encrypted secrets store for
pipeline credentials. (The realtime canvas overlay was
since solved differently and stays out.)foreach— fan-out over arrays{ "id": "fan", "type": "foreach", "items": "{{trigger.record.tags}}", "run": { "type": "pipeline", "name": "tag-worker", "input": { "tag": "{{item}}" } } }One child run per element of the resolved array (template or literal, cap
1000). Each child's input is templated against the normal context plus
{{item}}(the element, type preserved) and{{itemIndex}}. By default the
step joins: it parks until every child settles and its output is
{ runIds, results }with one{ index, runId, status, output }entry per
item — downstream steps read
{{steps.fan.output.results[0].output.*}}."wait": falsequeues and moves
on. Any failed child fails the step (respectingonError). Fan-out is
at-least-once like every step: crash recovery replays the event log and
spawns only items without a child yet — a crashed fan-out never duplicates
children.wait: true— join on spawned childrenworkflowandpipelinesteps stay fire-and-forget by default; set
wait: trueand the step parks (children_waiting) until the spawned run
settles. Completion is pushed, not polled: when a child run reaches a
terminal status — pipeline settle, workerPATCH, cancellation — the
engine writes achild_completedevent onto the parent's log and requeues
it, so the wake-up survives restarts and works identically for worker
workflows and child pipelines. The step output is{ runId, status, output }
of the child; a failed or cancelled child fails the step with the child run
id in the error.Public trigger tokens
External systems can now start runs without any vski credentials:
curl -X POST https://vski.example/api/pipelines/note-intake/tokens \ -H "Authorization: Bearer $KEY" -d '{"name": "github-ci"}' # → { "id": "ptoken_...", "token": "ptk_..." } (raw token shown once) curl -X POST https://vski.example/api/pipelines/trigger/ptk_... -d '{...}' # → 202 { "runId": "wkf_..." } (no auth; token is the credential)The request body becomes the run input; the trigger context is
{ "type": "token", "token": "github-ci" }— the raw secret never lands in
the event log or run pages. Only a SHA-256 hash is stored; unknown and
revoked tokens both answer 404, disabled pipelines 409; the endpoint is
rate limited per IP (60/min per node). The builder's settings panel mints,
displays-once and revokes tokens.llm— native LLM step{ "id": "summarize", "type": "llm", "model": "gpt-4o-mini", "system": "You write one-line summaries.", "prompt": "Summarize: {{trigger.record.text}}", "jsonMode": true, "retries": 2 }One synchronous chat completion against any OpenAI-compatible endpoint —
OpenAI, Ollama, OpenRouter, vLLM, LM Studio.baseUrl/modelfall back to
theLLM_BASE_URL/LLM_MODELenv vars (then OpenAI defaults); the API key
resolves from the step'sapiKeySecret(a stored secret, see below) or the
server'sLLM_API_KEY— it never lives in a definition.
Output:{ content, parsed?, model, usage }— withjsonModethe reply is
parsed intooutput.parsed(a non-JSON reply fails the step, so downstream
paths are safe).messagesreplacespromptfor full threads; retries and
timeouts reuse the standard step machinery.Encrypted secrets store
Credentials for
httpheaders and LLM providers no longer need to sit in
definitions (which are snapshotted into run rows and rendered in the UI):curl -X PUT https://vski.example/api/secrets/openai_main \ -H "Authorization: Bearer $KEY" -d '{"value": "sk-..."}'Values are AES-256-GCM encrypted at rest (key:
SECRETS_KEY, or derived
with domain separation fromJWT_SECRET) and write-only — no endpoint
returns them. Storing and deleting secrets is admin-only (workflows-role
users can list names and reference secrets in definitions, same stance as
API keys and plugins). Runs decrypt them in memory only when the definition
references them:{{secrets.NAME}}in any templated field, or llm
apiKeySecret. A rotated key fails affected runs with a clear message
rather than returning junk. The dashboard gains Settings → Secrets, and
the llm inspector offers anapiKeySecretpicker fed by the store.Also in this release
- New run events on the pipeline timeline:
child_spawned
({ step, runId, index, kind, name }),children_waiting, and
child_completed({ step, runId, status, output }). - The builder canvas palette and inspector learn the two new step types,
thewaittoggle, a trigger-token manager, and an llmapiKeySecret
picker; a new Settings → Secrets page manages the store. - Validation covers the new surface: foreach
runspecs (type, name,
self-reference), llm field ranges andapiKeySecretnames, and their
template syntax.
Tests
test/e2e/55_pipelines_test.tsgrows to 14 steps: foreach fan-out with
join over three pipeline children (per-item templating, ordered results,
parentRunIdlinks),wait: trueon a workflow child unblocked by a
worker-style run PATCH, the full token lifecycle against the unauthenticated
endpoint (404 for unknown/revoked, 409 disabled, input passthrough,
non-leaking trigger context), anllmstep verified end-to-end against
an in-test OpenAI-compatible mock (request shape, system+user messages,
response_format, usage passthrough,parsedtemplating), and the secrets
store ({{secrets.*}}in http headers and set values, llmapiKeySecret
sent as the bearer, write-only listings, delete, and the admin-only guard:
a workflows-role API key gets 403 on store/delete but can still list). Go
unit tests
extend the executor suite (fan-out/park/join, no-respawn across requeues,
failed children, wait semantics, llm request shaping, secrets in the
template context, secrets not loaded for plain definitions) and add token,
child-completion-hook and secrets service tests over SQLite (encryption at
rest, key-rotation failure, llm key resolution order) plus a handler test
for the admin-only mutations.Compatibility
- Additive: two new step types, optional
waitfield, newpipeline_tokens
andsecretstables (workflows.db, registered as system collections),
new endpoints and SDK methods. Existing definitions and runs are
unaffected; in-flight runs keep their pinned snapshots. foreachjoins by default ("wait": falseopts out); spawn steps remain
fire-and-forget by default ("wait": trueopts in).- New env vars (all optional):
LLM_API_KEY,LLM_BASE_URL,LLM_MODEL,
SECRETS_KEY.llmsteps fail with a clear message when no key is
available against the default OpenAI endpoint. Secrets encrypt under
SECRETS_KEYor (derived)JWT_SECRET— rotating either invalidates
stored secrets, so setSECRETS_KEYexplicitly ifJWT_SECRETrotates. - JS SDK 2.5.0 and Python SDK 0.8.0 publishing is part of the release run
(client.pipelines.createTriggerToken/triggerByTokenand
client.secrets.set/list/remove, Python equivalents); the web builder
calls the token and secrets endpoints directly so it works with any
published SDK version.
What's next
Still open from the original list: compensation/rollback steps. Ideas under
consideration: per-foreach parallelism limits, cancellation propagation from
a failed parent to its children.Downloads
-
Source code (ZIP)
4 downloads
-
Source code (TAR.GZ)
3 downloads
- New run events on the pipeline timeline: