2 Agent Tutorial
Anton Nesterov edited this page 2026-11-03 15:42:27 +01:00

Tutorial: Graph-Memory Agent

Build an agent out of platform primitives: a pipeline runs its steps, a graph holds its memory, a collection holds its output, and a trigger hands that output to a plugin that does something with it.

The agent is called scout. It keeps a watchlist of URLs, checks every one on a schedule, remembers every distinct state it has ever observed, and reports only what is new. Repeated scans publish nothing; a changed version publishes exactly one finding, and that finding is logged by a plugin the moment it lands.

Note: the complete, runnable project lives at vski.sh/x/template_agent. This tutorial walks through every file in it.

flowchart TD
    W["sources collection<br/>(the watchlist)"] --> S
    C["cron trigger<br/>every 5 min"] --> S["pipeline: scout-scan<br/>load watchlist, fan out"]
    S -->|"one child run per source"| P["pipeline: scout-probe<br/>fetch (http step), observe (plugin step)"]
    P --> G["graph 'memory'<br/>source and state nodes,<br/>observed and next edges"]
    G -->|"fingerprint seen before"| X["nothing published"]
    G -->|"new only"| F["findings collection"]
    F -->|"create trigger"| N["notifier plugin<br/>stands in for Slack, email, webhooks"]
    N --> L["logs/findings.log"]

Why this shape

An agent is a loop: check something, decide, act, remember. On this platform each part has a first-class home:

Agent concern Platform primitive
Running the steps, in order, on a schedule a pipeline with a cron trigger
Checking something an http step (no code) or a plugin function (code)
Remembering what was already explored a graph, because history and relationships are the query
Acting on a result a record in a collection
Reacting to the result without polling a collection trigger into a plugin

The graph is the interesting part. "Have I seen this before?" is a node lookup; "what changed and in what order?" is a walk along next edges. That is memory worth having, and it costs three upserts per observation.

Step 1: the schema, declaratively

Everything the agent needs is declared in vski/ and applied with vski apply (see IaC). Two collections first:

# vski/default/collections.yaml
version: 1
database: default

collections:
  sources:
    type: base
    fields:
      - {name: name, type: text, required: true}
      - {name: url, type: url, required: true}
      - {name: selector, type: text}
      - {name: active, type: bool}
    rules:
      list: "'plugin' in @request.auth.roles"
      view: "'plugin' in @request.auth.roles"
      create: null
      update: null
      delete: null

  findings:
    type: base
    fields:
      - {name: sourceId, type: text, required: true}
      - {name: sourceName, type: text}
      - {name: url, type: text}
      - {name: fingerprint, type: text, required: true}
      - {name: state, type: text}
      - {name: summary, type: text}
    rules:
      list: null
      view: null
      create: "'plugin' in @request.auth.roles"
      update: null
      delete: null
    options:
      triggers:
        - action: create
          url: "plugin://notifier/on_finding"
          method: POST

Notes on the shape:

  • sources.url is what the agent fetches; sources.selector is an optional dotted path into the JSON response that names a state (like version or info.version). Without a selector the whole payload is hashed, so any change still counts.
  • Every rule on both collections admits only the plugin role (the agent's own service key; see Plugins), and null means even that is denied. Superusers bypass rules, so you can always read your own agent's data with an admin key. See Rules and Access Control.
  • The findings trigger is the whole "react" story: on every created record the server invokes the on_finding function of the notifier plugin instance, with the record as the payload.

The memory graph is its own resource (graphs are not collections; see Graphs):

# vski/default/graphs.yaml
version: 1
database: default

graphs:
  memory:
    options:
      directed: true
    rules:
      list: "'plugin' in @request.auth.roles"
      view: "'plugin' in @request.auth.roles"
      create: "'plugin' in @request.auth.roles"
      update: "'plugin' in @request.auth.roles"
      delete: null

Step 2: the agent's execution plan

The agent's "code" for orchestration is data: two pipelines in vski/default/pipelines.yaml. The parent sweeps; the child probes one source.

# vski/default/pipelines.yaml
version: 1
database: default

pipelines:
  scout-scan:
    enabled: true
    triggers:
      - {type: cron, schedule: "*/5 * * * *"}
      - {type: collection, collection: sources, actions: [create]}
    steps:
      - id: load
        type: plugin
        target: scout/sources
      - id: probe
        type: foreach
        dependsOn: [load]
        items: "{{steps.load.output.items}}"
        run:
          type: pipeline
          name: scout-probe
          input: {source: "{{item}}"}
        onError: continue

  scout-probe:
    enabled: true
    steps:
      - id: fetch
        type: http
        method: GET
        url: "{{input.source.url}}"
        timeout: 10
        retries: 1
        onError: continue
      - id: observe
        type: plugin
        target: scout/observe
        dependsOn: [fetch]
        condition: "{{steps.fetch.status}} == 'completed'"
        timeout: 60
        params:
          source: "{{input.source}}"
          status: "{{steps.fetch.output.status}}"
          body: "{{steps.fetch.output.body}}"

Reading it like a program:

  • scout-scan fires every five minutes, and also the moment a source is created (the collection trigger), so the watchlist is live.
  • load calls the sources function on the scout plugin instance, which returns the active watchlist.
  • probe is a foreach: one scout-probe child run per source, each with {{item}} (the source record) templated into its input. dependsOn matters; without it probe would race load.
  • In the child, fetch is a plain http step: no code, retried once, ten second timeout. A dead source records a failure but does not sink the sweep (onError: continue), and observe's condition skips it for that run.
  • observe does the thinking, in the plugin. A whole-string template like "{{steps.fetch.output.body}}" preserves the JSON type, so the function receives the parsed body.

See Pipelines for the full step vocabulary (set, delay, signal, workflow, llm, child pipelines) and Cron for schedules.

Step 3: the brain (scout plugin)

Plugins are Python processes the server keeps warm; pipeline plugin steps and collection triggers call functions in them (see Plugins). The scout plugin has two functions. sources is plain reading:

# plugins/scout/scout/main.py
@app.function("sources")
async def sources(ctx, params):
    page = await ctx.client.collection("sources").get_list(1, 200, sort="created")
    items = [
        {"id": r["id"], "name": r.get("name") or r["id"],
         "url": r["url"], "selector": r.get("selector") or ""}
        for r in page.get("items", []) if r.get("active", True)
    ]
    return {"items": items, "count": len(items)}

ctx.client is a full SDK client authenticated as the plugin instance's service key (role plugin, scoped to its database). That is why the rules in Step 1 mention that role: the agent, and only the agent, can read its watchlist, write findings, and touch its memory.

observe is the memory. It fingerprints the observation, checks the graph for that state, and only when the state is new does it chain it into history and publish:

@app.function("observe")
async def observe(ctx, params):
    source = (params or {}).get("source") or {}
    fp, slug = fingerprint(params.get("body"), source.get("selector") or "")
    src_node = f"src:{source['id']}"
    state_node = f"state:{source['id']}:{slug}"

    # Recall: which states has this source already been observed in?
    try:
        res = await ctx.client.graph("memory").neighbors(
            src_node, direction="out", labels=["observed"])
        known_states = res.get("nodes", [])
    except Exception:
        known_states = []  # first scan: the source node does not exist yet
    known = any(n.get("id") == state_node for n in known_states)

    # Remember: upsert the source, the state, and the edges. All ids are
    # deterministic, so repeats overwrite instead of duplicating.
    await ctx.client.graph("memory").create_nodes([...], upsert=True)
    await ctx.client.graph("memory").create_edges([...], upsert=True)

    if known:
        return {"sourceId": source["id"], "fingerprint": fp,
                "state": state_node, "known": True, "findingId": None}

    finding = await ctx.client.collection("findings").create({...})
    return {"sourceId": source["id"], "fingerprint": fp,
            "state": state_node, "known": False, "findingId": finding["id"]}

The memory layout, per source:

flowchart LR
    S["(src:demo-local)<br/>label: source"]
    V1["(state:demo-local:v1)<br/>label: state<br/>properties: fingerprint v1, checkedAt"]
    V2["(state:demo-local:v2)<br/>label: state<br/>properties: fingerprint v2, checkedAt"]
    S -- observed --> V1 -- next --> V2
  • One source node per watched URL, linked back to its record with properties.ref (the convention from Graphs).
  • One state node per distinct observation. The id embeds the fingerprint slug, so "have I seen this?" is one node lookup.
  • observed edges from the source to each of its states; next edges from the previous latest state to the new one. Walking next out of any state gives you the change history in order.

Because every id is derived from data, scans are idempotent: run the same sweep twice and the second run upserts the same nodes, refreshes checkedAt, and publishes nothing.

Step 4: the hands (notifier plugin)

The findings collection's trigger from Step 1 points at plugin://notifier/on_finding. The whole plugin is one function that appends the record to a log file, standing in for the request you would really make:

# plugins/notifier/notifier/main.py
@app.function("on_finding")
async def on_finding(ctx, params):
    record = params or {}   # for a plugin:// trigger, params IS the record
    path = resolve_logfile(str(ctx.config.get("logfile") or "logs/findings.log"))
    path.parent.mkdir(parents=True, exist_ok=True)
    entry = {"ts": _now(), "event": "finding.created", "record": record}
    with open(path, "a", encoding="utf-8") as fh:
        fh.write(json.dumps(entry, default=str) + "\n")
    return {"logged": True, "id": record.get("id"), "path": str(path)}

Instance config comes from IaC, so the log path is deployment data, not code:

# vski/default/plugins.yaml
version: 1
database: default

plugins:
  scout:
    plugin: scout
    enabled: true
    config: {}
  notifier:
    plugin: notifier
    enabled: true
    config:
      logfile: logs/findings.log

Trigger delivery is durable: every dispatch is retried with backoff and logged in _webhook_logs, so a finding is not lost if the plugin is restarting (see Triggers).

Step 5: run it

The server must run with plugins enabled and pointed at the template (a plain server binary; see Getting Started):

PLUGINS_ENABLED=true \
PLUGINS_DIR=$(pwd)/plugins \
PLUGIN_PYTHON=$(pwd)/.venv/bin/python \
/path/to/vski serve --port 3000

Then, from the template checkout (its Makefile wraps each step):

make venv                  # CLI + SDK; also the PLUGIN_PYTHON above
vski keys create agent-dev --roles admin   # paste into .env

make iac-apply             # collections, graph, pipelines, plugin instances
make seed                  # two sample sources
make scan                  # one sweep now (cron also fires every 5 min)

The seed watchlist is one local source (the platform's own health endpoint, selector status) and one real one (the latest pip release on PyPI, selector info.version).

Step 6: watch it act

make findings              # what it published
make memory                # what it remembers
make log                   # what the notifier wrote
make runs                  # pipeline runs, step outputs and all

A first sweep publishes one finding per source and writes one log line per finding:

2026-11-03T01:19:28Z finding.created platform-health -> ok
2026-11-03T01:19:28Z finding.created pypi-pip -> 26.3

make memory prints each source and its observed history, reconstructed from the graph:

platform-health  (http://127.0.0.1:3000/api/health)
  1 state(s) observed
  history: ok

pypi-pip  (https://pypi.org/pypi/pip/json)
  1 state(s) observed
  history: 26.3

Now run make scan again. Nothing is published: the agent recognizes both states. Change what a source returns (bump the version your demo service serves) and the next sweep produces exactly one new finding, summary: demo-service: v1 -> v2, a fresh log line, and a longer chain:

demo-service  (http://127.0.0.1:3199/status.json)
  2 state(s) observed
  history: v1 -> v2

Add a broken URL to the watchlist and the sweep still completes: the fetch step continues on error and the observe step skips for that source, so one dead source never blinds the agent to the others.

Where the work went

Requirement Where it lives
Checks something http step in scout-probe
Executes steps as a pipeline scout-scan + scout-probe in vski/default/pipelines.yaml
Remembers what it explored memory graph, deterministic ids, observed/next edges
Publishes to a collection findings, written by the scout plugin
Reacts without polling findings create trigger to plugin://notifier/on_finding
Writes output to a log notifier plugin appends to logs/findings.log (gitignored)

Where to go next

  • Put a real notification in on_finding: the log line is the seam; POST to chat, email, or a deploy hook instead.
  • Fan out wider: foreach handles up to 1000 items, and options.concurrency controls how many probes run at once.
  • Make the agent smarter with an llm step in scout-probe that summarizes diffs between states; keep it after observe and feed it both fingerprints.
  • Watch authenticated endpoints by storing tokens in secrets and referencing them in the fetch step's headers.
  • The same loop with different senses: swap the http step for a plugin step that reads a mailbox, a queue, or another collection.