Add Snake AI mode with tabular Q-learning + companion study note - #42
Merged
Conversation
- Mode switch on /game/: 'Play yourself' (unchanged human controls) vs 'Watch the AI learn', where a dependency-free tabular Q-learning agent drives the same game loop and rendering - Compact relative state (danger straight/left/right, heading, food direction sign) with relative actions (straight/left/right) so the agent can never reverse into itself - Rewards: +10 food, -10 death/starvation, small shaping toward the food; eps-greedy with per-episode decay - Training UX: 1x/50x/500x speed control (fast modes run many steps per frame, render once per frame), live stats (episodes, last/best score, rolling average, epsilon) and a score-per-episode sparkline - Q-table persisted to localStorage with a Reset Brain button - Board and sparkline colors now come from CSS custom properties read at runtime (with a MutationObserver on the theme class), so the canvas follows light/dark mode; light-mode board colors are unchanged Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VN9jJEzsRNxYQj3UqwUVw
Interactive note covering state design, reward shaping, the TD update, and epsilon-greedy exploration, with four demos: a drivable state encoder, a one-step update calculator, an epsilon-decay chart, and a live in-page trainer that learns Snake from scratch on a 12x12 board. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VN9jJEzsRNxYQj3UqwUVw
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What's in this PR
Two related additions, vanilla JS only, no dependencies:
1. AI mode for the Snake game (
docs/game.md,docs/style.scss)/game/: "Play yourself" (unchanged gameplay) vs "Watch the AI learn". In AI mode, keyboard/swipe/d-pad input is ignored and a tabular Q-learning agent drives the same game loop and rendering.2. Study note: "Tabular Q-Learning, Explained by a Snake" (
docs/_posts/2026-07-20-q-learning-snake.html)An interactive study note teaching the algorithm behind the game's AI mode, following the
docs/_posts/CLAUDE.mddesign system (--sn-*tokens, shared components, theme-aware JS-drawn graphics). Four demos:Verification
/study-notes/index.🤖 Generated with Claude Code
https://claude.ai/code/session_015VN9jJEzsRNxYQj3UqwUVw
Generated by Claude Code