• v0.15.0 529bb0ed1f

    v0.15.0 Stable

    x released this 2026-10-28 11:39:32 +00:00 | 25 commits to main since this release

    VSKI v0.15.0

    Pipelines v2: foreach fan-out, child waits, public trigger tokens, llm steps, encrypted secrets

    v0.14.0 shipped pipelines with a deliberate "not yet" list. v0.15.0 clears
    five of them: fan-out over arrays, waiting on spawned children, external
    token triggers, a native LLM step, and an encrypted secrets store for
    pipeline credentials. (The realtime canvas overlay was
    since solved differently and stays out.)

    foreach — fan-out over arrays

    {
      "id": "fan", "type": "foreach",
      "items": "{{trigger.record.tags}}",
      "run": { "type": "pipeline", "name": "tag-worker", "input": { "tag": "{{item}}" } }
    }
    

    One child run per element of the resolved array (template or literal, cap
    1000). Each child's input is templated against the normal context plus
    {{item}} (the element, type preserved) and {{itemIndex}}. By default the
    step joins: it parks until every child settles and its output is
    { runIds, results } with one { index, runId, status, output } entry per
    item — downstream steps read
    {{steps.fan.output.results[0].output.*}}. "wait": false queues and moves
    on. Any failed child fails the step (respecting onError). Fan-out is
    at-least-once like every step: crash recovery replays the event log and
    spawns only items without a child yet — a crashed fan-out never duplicates
    children.

    wait: true — join on spawned children

    workflow and pipeline steps stay fire-and-forget by default; set
    wait: true and the step parks (children_waiting) until the spawned run
    settles. Completion is pushed, not polled: when a child run reaches a
    terminal status — pipeline settle, worker PATCH, cancellation — the
    engine writes a child_completed event onto the parent's log and requeues
    it, so the wake-up survives restarts and works identically for worker
    workflows and child pipelines. The step output is { runId, status, output }
    of the child; a failed or cancelled child fails the step with the child run
    id in the error.

    Public trigger tokens

    External systems can now start runs without any vski credentials:

    curl -X POST https://vski.example/api/pipelines/note-intake/tokens \
      -H "Authorization: Bearer $KEY" -d '{"name": "github-ci"}'
    # → { "id": "ptoken_...", "token": "ptk_..." }   (raw token shown once)
    
    curl -X POST https://vski.example/api/pipelines/trigger/ptk_... -d '{...}'
    # → 202 { "runId": "wkf_..." }                   (no auth; token is the credential)
    

    The request body becomes the run input; the trigger context is
    { "type": "token", "token": "github-ci" } — the raw secret never lands in
    the event log or run pages. Only a SHA-256 hash is stored; unknown and
    revoked tokens both answer 404, disabled pipelines 409; the endpoint is
    rate limited per IP (60/min per node). The builder's settings panel mints,
    displays-once and revokes tokens.

    llm — native LLM step

    {
      "id": "summarize", "type": "llm",
      "model": "gpt-4o-mini",
      "system": "You write one-line summaries.",
      "prompt": "Summarize: {{trigger.record.text}}",
      "jsonMode": true, "retries": 2
    }
    

    One synchronous chat completion against any OpenAI-compatible endpoint —
    OpenAI, Ollama, OpenRouter, vLLM, LM Studio. baseUrl/model fall back to
    the LLM_BASE_URL/LLM_MODEL env vars (then OpenAI defaults); the API key
    resolves from the step's apiKeySecret (a stored secret, see below) or the
    server's LLM_API_KEY — it never lives in a definition.
    Output: { content, parsed?, model, usage } — with jsonMode the reply is
    parsed into output.parsed (a non-JSON reply fails the step, so downstream
    paths are safe). messages replaces prompt for full threads; retries and
    timeouts reuse the standard step machinery.

    Encrypted secrets store

    Credentials for http headers and LLM providers no longer need to sit in
    definitions (which are snapshotted into run rows and rendered in the UI):

    curl -X PUT https://vski.example/api/secrets/openai_main \
      -H "Authorization: Bearer $KEY" -d '{"value": "sk-..."}'
    

    Values are AES-256-GCM encrypted at rest (key: SECRETS_KEY, or derived
    with domain separation from JWT_SECRET) and write-only — no endpoint
    returns them. Storing and deleting secrets is admin-only (workflows-role
    users can list names and reference secrets in definitions, same stance as
    API keys and plugins). Runs decrypt them in memory only when the definition
    references them: {{secrets.NAME}} in any templated field, or llm
    apiKeySecret. A rotated key fails affected runs with a clear message
    rather than returning junk. The dashboard gains Settings → Secrets, and
    the llm inspector offers an apiKeySecret picker fed by the store.

    Also in this release

    • New run events on the pipeline timeline: child_spawned
      ({ step, runId, index, kind, name }), children_waiting, and
      child_completed ({ step, runId, status, output }).
    • The builder canvas palette and inspector learn the two new step types,
      the wait toggle, a trigger-token manager, and an llm apiKeySecret
      picker; a new Settings → Secrets page manages the store.
    • Validation covers the new surface: foreach run specs (type, name,
      self-reference), llm field ranges and apiKeySecret names, and their
      template syntax.

    Tests

    test/e2e/55_pipelines_test.ts grows to 14 steps: foreach fan-out with
    join over three pipeline children (per-item templating, ordered results,
    parentRunId links), wait: true on a workflow child unblocked by a
    worker-style run PATCH, the full token lifecycle against the unauthenticated
    endpoint (404 for unknown/revoked, 409 disabled, input passthrough,
    non-leaking trigger context), an llm step verified end-to-end against
    an in-test OpenAI-compatible mock (request shape, system+user messages,
    response_format, usage passthrough, parsed templating), and the secrets
    store ({{secrets.*}} in http headers and set values, llm apiKeySecret
    sent as the bearer, write-only listings, delete, and the admin-only guard:
    a workflows-role API key gets 403 on store/delete but can still list). Go
    unit tests
    extend the executor suite (fan-out/park/join, no-respawn across requeues,
    failed children, wait semantics, llm request shaping, secrets in the
    template context, secrets not loaded for plain definitions) and add token,
    child-completion-hook and secrets service tests over SQLite (encryption at
    rest, key-rotation failure, llm key resolution order) plus a handler test
    for the admin-only mutations.

    Compatibility

    • Additive: two new step types, optional wait field, new pipeline_tokens
      and secrets tables (workflows.db, registered as system collections),
      new endpoints and SDK methods. Existing definitions and runs are
      unaffected; in-flight runs keep their pinned snapshots.
    • foreach joins by default ("wait": false opts out); spawn steps remain
      fire-and-forget by default ("wait": true opts in).
    • New env vars (all optional): LLM_API_KEY, LLM_BASE_URL, LLM_MODEL,
      SECRETS_KEY. llm steps fail with a clear message when no key is
      available against the default OpenAI endpoint. Secrets encrypt under
      SECRETS_KEY or (derived) JWT_SECRET — rotating either invalidates
      stored secrets, so set SECRETS_KEY explicitly if JWT_SECRET rotates.
    • JS SDK 2.5.0 and Python SDK 0.8.0 publishing is part of the release run
      (client.pipelines.createTriggerToken / triggerByToken and
      client.secrets.set/list/remove, Python equivalents); the web builder
      calls the token and secrets endpoints directly so it works with any
      published SDK version.

    What's next

    Still open from the original list: compensation/rollback steps. Ideas under
    consideration: per-foreach parallelism limits, cancellation propagation from
    a failed parent to its children.

    Downloads