Skip to content

Add Snake AI mode with tabular Q-learning + companion study note - #42

Merged
Alexdruso merged 2 commits into
masterfrom
claude/snake-ai-mode-tfwrg4
Jul 20, 2026
Merged

Add Snake AI mode with tabular Q-learning + companion study note#42
Alexdruso merged 2 commits into
masterfrom
claude/snake-ai-mode-tfwrg4

Conversation

@Alexdruso

Copy link
Copy Markdown
Owner

What's in this PR

Two related additions, vanilla JS only, no dependencies:

1. AI mode for the Snake game (docs/game.md, docs/style.scss)

  • Mode switch on /game/: "Play yourself" (unchanged gameplay) vs "Watch the AI learn". In AI mode, keyboard/swipe/d-pad input is ignored and a tabular Q-learning agent drives the same game loop and rendering.
  • Genuine Q-learning: compact relative state (danger straight/left/right, heading, food-direction signs → ~288 states), relative actions (turn left / straight / turn right, so reversing is impossible), rewards +10 food / −10 death or starvation, asymmetric distance shaping (+0.3 / −0.45) so circling is a net loss, ε-greedy with per-episode decay, α = 0.15, γ = 0.9.
  • Training UX: 1×/50×/500× speed (fast modes run many steps per frame, render once per frame), live stats (episodes, last/best score, rolling average, ε), and a sparkline of score-per-episode with a rolling-average learning curve.
  • Persistence: Q-table saved to localStorage (every 20 episodes + on pause/tab hide/close) with a confirm-guarded "Reset Brain" button.
  • Theming: board and sparkline read CSS custom properties at runtime and redraw via a MutationObserver on the body class. Light-mode board colors are pixel-identical to before; dark mode gets a proper dark board.

2. Study note: "Tabular Q-Learning, Explained by a Snake" (docs/_posts/2026-07-20-q-learning-snake.html)

An interactive study note teaching the algorithm behind the game's AI mode, following the docs/_posts/CLAUDE.md design system (--sn-* tokens, shared components, theme-aware JS-drawn graphics). Four demos:

  • A drivable state encoder: steer a snake with the three relative actions and watch the exact state the agent perceives, with its sensory cells highlighted.
  • A one-step TD-update calculator with scenario buttons and α/γ sliders.
  • An ε-decay schedule chart with an interactive decay constant.
  • A live in-page trainer: the full algorithm learning Snake from scratch on a 12×12 board with speed control, stats, and a learning-curve canvas.

Verification

  • Jekyll build clean (only pre-existing Sass deprecation warnings).
  • Automated browser checks (Playwright + Chromium) on the built site, both themes: human mode regression (start/pause/steer) passes; AI-mode input ignoring, localStorage persistence across reload, and live theme-toggle canvas recolor all pass.
  • Learning confirmed: game agent reached 1,385 episodes / best score 520 in 60 s at 500×; the note's in-page trainer went from random play to rolling average ~12 (best 30) in 25 s at 200×.
  • Study note demos verified numerically (TD calculator outputs exact expected values) and the note appears on the /study-notes/ index.

🤖 Generated with Claude Code

https://claude.ai/code/session_015VN9jJEzsRNxYQj3UqwUVw


Generated by Claude Code

claude added 2 commits July 20, 2026 19:59
- Mode switch on /game/: 'Play yourself' (unchanged human controls) vs
  'Watch the AI learn', where a dependency-free tabular Q-learning agent
  drives the same game loop and rendering
- Compact relative state (danger straight/left/right, heading, food
  direction sign) with relative actions (straight/left/right) so the
  agent can never reverse into itself
- Rewards: +10 food, -10 death/starvation, small shaping toward the food;
  eps-greedy with per-episode decay
- Training UX: 1x/50x/500x speed control (fast modes run many steps per
  frame, render once per frame), live stats (episodes, last/best score,
  rolling average, epsilon) and a score-per-episode sparkline
- Q-table persisted to localStorage with a Reset Brain button
- Board and sparkline colors now come from CSS custom properties read at
  runtime (with a MutationObserver on the theme class), so the canvas
  follows light/dark mode; light-mode board colors are unchanged

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VN9jJEzsRNxYQj3UqwUVw
Interactive note covering state design, reward shaping, the TD update,
and epsilon-greedy exploration, with four demos: a drivable state
encoder, a one-step update calculator, an epsilon-decay chart, and a
live in-page trainer that learns Snake from scratch on a 12x12 board.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VN9jJEzsRNxYQj3UqwUVw
@Alexdruso
Alexdruso merged commit 5e803e1 into master Jul 20, 2026
2 checks passed
@Alexdruso
Alexdruso deleted the claude/snake-ai-mode-tfwrg4 branch July 20, 2026 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants