Mixture of Bandits — A Length-8 Bypass-Risk Gate on Top of the Gold Digger Planner

This is the in-browser port of sequential_recommender_moe.ipynb's v4 design (the working one — the notebook documents two failed earlier designs that this app deliberately does NOT ship).

What it does

See Mixture of Bandits and MoB Queen

The agent runs the validated Gold Digger loop: 30×30 grid, 8 directions × 4 leap lengths {1, 2, 4, 8} = 32 candidate actions per tick, linear-Q fit by frozen-target Fitted Q-Iteration. On top of that, the gate watches the agent's belief and reward history:

• For every length-8 leap the planner chooses, we log a labelled example:

  • Input (6 features, computed from the agent's own belief + reward history — no truth queries):

    #	feature	what it captures
    0	local_belief_mean	mean bGold over sensor radius
    1	local_belief_max	peak bGold in sensor radius
    2	local_minus_global	local mean − global mean (the regime signal)
    3	gold_last_3_steps	gold events in last 3 steps
    4	gold_last_5_steps	gold events in last 5 steps
    5	step_in_episode	t / T

  • Label (computed from the painted ground truth at label-collection time — training only, never inference): y = 1 iff the chosen length-8 leap collected zero gold on its path AND there was gold within sensor radius that was NOT on the path.

• Once ≥ 200 labels accumulate, the gate fits by IRLS (iteratively reweighted least squares) — each iteration is a weighted Cholesky solve reusing the existing linalg.ts:choleskySolve. Class balancing handles the ~15–25% positive-rate imbalance. The gate auto-enables on the first successful fit.

• At decision time: P(bypass | state) = σ(w · φ + b). If P > τ (0.5), the gate fires: length-8 actions (indices 24..31) are masked out of the planner's argmax and zeroed in the Q-buffer (so even selectAction's fully-masked fallback branch respects the ban). The planner re-picks among the 24 short-hop actions using its unchanged value function.

This is a soft mixture of action-selection rules, not a mixture of Q-functions. There's only one value function; what changes per state is which actions it's allowed to consider.

Quick start

npm install
npm run dev      # vite dev server on :5175
npm test         # headless validation — 12/12 baseline + 10/12 gate + knight test
npm run build    # production bundle

Open the dev URL, paint a cluster (the cluster button generates the demo's actual use case: a few dense gold patches with light 2–7% stones), or hit scatter / spiral for the generalisation regimes, then press Start. Watch:

1. First ~5 episodes: the gate state pill reads — collecting labels —. The planner is running; every length-8 leap it chooses gets logged with a bypass label.
2. First gate fit (auto, around 200 labels): the pill flips to ON, the training-samples count stabilises around 250–350, and the train-accuracy readout appears (typically 0.55–0.70).
3. From then on: watch the action bars in the side panel. When the gate fires (P(bypass) > 0.5), the length-8 row dims with a red strike-through and the agent gets a green halo on canvas. The gate-firing counter ticks up.
4. Toggle the gate off mid-run: the length-8 row lights back up and you can see the planner resume length-8 leaps — sometimes walking past clusters it would otherwise collect.

The Refit Q now button also forces a gate refit.

Variants

The sidebar's Model block at the top lets you switch between three variants at runtime:

• MoB Baseline — 32 leap actions (8 dirs × {1,2,4,8}) × 11 features. This is the validated v4 design from sequential_recommender_moe.ipynb.
• MoB Knight — 40 actions (32 leaps + 8 knight L-shapes) × 14 features (11 base + 3 enriched: destNeighborhoodGold, destLookaheadGold, isKnight). The knight actions reach 8 cells within Chebyshev-2 that NO straight leap can hit; the enriched features describe the cluster context around the destination. The validated Python prototype (_proto_knight_v2.py) showed this variant recovers ~13% of gold on clustered worlds that the baseline leaves on the table.
• MoB Global — 40 actions (32 leaps + 8 queen jumps) × 11 features. Each queen action jumps toward one of 8 directions; the target is the argmax-density cell along that direction (computed from the existing belief map — no new sensor). The Q-net learns when each direction pays off. The validated Python prototype (_proto_global_v3.py) showed this variant recovers ~15% of gold on random worlds where the baseline hits the step cap with gold still on the board. The queen move fires with a blue halo on the canvas. On clustered worlds the baseline already collects 100%, so this variant regresses nothing but improves nothing there.

Switching models mid-run discards training (Q weights, replay buffer, gate labels) and rebuilds the agent from scratch — there is no soft swap, because the action/feature dimensionality differs and buffers cannot be copied across. Press Start again to retrain.

The active variant is also controllable via URL param:

http://localhost:5175/?model=mob_knight       # load the knight variant
http://localhost:5175/?model=mob_global       # load the global variant
http://localhost:5175/?model=mob               # baseline (the default)

World-type selector

The World per episode row (in the Brush block) lets you freeze the per-episode layout instead of the default random rotation:

• Auto (default) — 60% cluster / 20% spiral / 20% scatter, the demo's standard mix.
• Cluster / Spiral / Scatter / Empty — freezes that world type across every episode. Useful for inspecting how a variant behaves on a single layout (e.g. freeze cluster + select MoB Knight to watch the L-shape actions fire on dense gold patches, or freeze random + select MoB Global to watch the queen jumps fire on sparse terrain with a blue halo).

URL param:

http://localhost:5175/?world=cluster           # freeze cluster worlds
http://localhost:5175/?world=random            # freeze scatter worlds (queen regime)
http://localhost:5175/?model=mob_global&world=random