MineOps is a safe, observable AI-agent orchestration layer that uses a live Minecraft server as its execution environment. An LLM (or a Stream Deck button, or the stock market) requests typed actions; a safety policy validates every parameter; and only allowlisted, rendered Minecraft commands ever reach the server over RCON.
The premise on stream is an absurd AI job interview — the model rewards or punishes the "candidate" based on their answers, chat can vote in zombie waves, and if Alibaba's stock moves, somebody gets creepers. The engineering underneath is played straight: provider abstraction, typed tool execution, command allowlisting, cooldowns, redacted diagnostics, a simulator mode, and 52 unit tests.
Giving a language model raw console access to a game server is the same problem as giving an agent raw shell access to production — just with lower stakes and better content. MineOps treats it that way:
- The model never writes commands. It can only request one of five typed
actions:
gift_item,spawn_mobs,random_teleport,apply_effect,send_message. SafetyPolicyvalidates everything before rendering: player names, Minecraft resource IDs, item counts, mob caps, teleport radii, effect strength, message content, and per-action cooldowns.- Renderers are the only place commands are built. A validated action maps to a fixed command template — there is no string path from model output to RCON.
- Failure-safe defaults: the mock LLM provider is the default, the HTTP
server binds to
127.0.0.1only, and diagnostics redact secrets.
Same architecture you'd want for any tool-using agent; Minecraft just makes the failure modes funnier.
trigger sources core outputs
─────────────── ──── ───────
LLM provider ─┐ ┌─ RCON → Minecraft
Stream Deck ─┤→ typed Action → SafetyPolicy → renderer ─┤
HTTP API ─┤ │ └─ [SIMULATOR] log
stock market ─┘ rejected + logged
Streamer.bot │
└→ event log → SSE → OBS/Meld overlays + audio
mineops/— Python package: CLI, local HTTP server, AI providers (mock / OpenAI-compatible / OpenRouter), safety policy, command renderers, event bus, idle watcher, stock triggers.overlays/— transparent browser-source overlays for alerts and a paper-market replay clock.scripts/— Stream Deck launchers, a tray controller for stream-day ops, Streamer.bot adapters, and Minecraft server management.tests/— 52 tests covering policy rejection, command rendering, config redaction, AI-output parsing, cooldowns, idle tracking, market triggers, and server security.
Simulator mode prints rendered commands instead of sending them:
python -m mineops doctor
python -m mineops once "reward the candidate for understanding sandboxing"
python -m mineops runStart the local API + overlay server:
python -m mineops serve --host 127.0.0.1 --port 8765Then open http://127.0.0.1:8765/overlay in a browser (or as a transparent
OBS/Meld browser source) and fire a rehearsal alert:
http://127.0.0.1:8765/trigger/test/reward
http://127.0.0.1:8765/trigger/test/punish
- Copy
.env.example→.env.local(gitignored — secrets never touch the repo). - Set
MINEOPS_MODE=rconplus your RCON host/port/password and player name. python -m mineops doctorto verify, then.\scripts\start-mineops.ps1.
A local rehearsal server is one script away
(.\scripts\start-minecraft-server.ps1, join localhost:25565), and the
tray controller (wscript .\scripts\Start-MineOpsTray.vbs) can start, stop
and inspect every long-running piece from the system tray.
AI providers: MINEOPS_LLM_PROVIDER=mock (default), openai_compatible
(point it at any local server), or openrouter. Config in .env.local.
- Stream Deck:
scripts/streamdeck/has ready-made launchers for server control, rewards, punishments, and the market replay — wire them to System → Open actions. - Streamer.bot:
scripts/streamerbot-command.ps1 -Command zombie-waveroutes chat/channel-point events through the same safety policy as everything else (the six-zombie cap is enforced inSafetyPolicy, not in the script). - Market triggers:
python -m mineops stock-babachecks the latest BABA daily close — down day rewards the player, up day punishes them (the in-universe logic involves a rival AI lab's budget). When markets are closed,/market-replay/baba-upruns a simulated trading day as a transparent overlay that fires deterministic replay events.
Local-only (127.0.0.1) server for overlays and stream controls:
| Group | Endpoints |
|---|---|
| State | GET /status (redacted config + events) · /status.js · GET /events (SSE) |
| Overlays | GET /overlay · GET /market-replay/baba-up (+ /arm, /pause, /reset, /state.js) |
| Rehearsal | GET /trigger/test/reward · /trigger/test/punish |
| Manual actions | GET /trigger/manual/{zombies, zombie-wave, teleport, message} · POST /trigger/gift |
| Market | GET /trigger/stock/baba · /trigger/stock/baba/replay/up?step=n |
python -m unittest discover -s tests52 tests run on every push via GitHub Actions — including
test_server_security.py and test_safety.py, because the safety cage is
the feature.
- Demo clips and overlay screenshots in this README.
- More market modes: baskets, fake paper portfolios, themed rivalries.
- Richer Streamer.bot adapter with visible caps and cooldowns.
- Creative Architect mode: the model as a creative-mode builder emitting
structured build plans, executed only through bounded, allowlisted tools
(
fill_box,make_road,write_sign, …) with area/height limits, block allowlists, rate limits, and an emergency stop. Same safety posture, bigger canvas — "build an alien museum explaining Earth incorrectly."
Designed, directed, and stream-tested by CodingWithPak; implemented in pair-programming sessions with AI assistants (Claude, GPT), with commits co-authored accordingly and reviewed via CodeRabbit. The safety-first architecture — typed actions, validation before rendering, allowlists over denylists — is the human part; that's the design decision everything else hangs off.
MIT for code and documentation. Alert audio in assets/ is for
demo use — swap in sounds you have rights to for your own stream.