choresdb

choresdb is an experimental macOS automation framework for repeatable, semantic desktop workflows.

The central design rule is:

A demonstration is evidence about an action; it is not the workflow itself.

A recorded click may help identify a Sign in button, but replay must resolve the current button from accessibility, text, visual, and contextual signals. It must not depend on the original screen coordinates.

Current status

This repository contains the first project foundation:

• a typed workflow graph model;
• structural and reachability validation;
• a bounded state-machine runtime;
• a provider-independent perception and target-resolution core;
• a dry-run executor for exercising predefined branches;
• the milestone-one Chrome workflow as data;
• architecture and delivery notes.

Native capture and input primitives are being added incrementally. The first Chrome login slice now uses macOS Accessibility and synthetic keyboard events; screenshot/CV providers are not yet connected.

Quick start

Python 3.11 or newer is required. No runtime dependencies are needed yet.

PYTHONPATH=src python3 -m choresdb validate workflows/milestone_01.json \
  --targets targets/milestone_01.json
PYTHONPATH=src python3 -m choresdb validate-targets targets/milestone_01.json

Exercise the already-authenticated path:

PYTHONPATH=src python3 -m choresdb run workflows/milestone_01.json \
  --dry-run \
  --state detect_auth_state=logged_in

Exercise the login path:

PYTHONPATH=src python3 -m choresdb run workflows/milestone_01.json \
  --dry-run \
  --state detect_auth_state=logged_out \
  --state detect_terms_state=unchecked

Run the tests:

PYTHONPATH=src python3 -m unittest discover -s tests -v

The first semantic target contracts live in targets/milestone_01.json. Candidate providers may use accessibility, OCR, local CV, or hosted vision, but resolution policy—not a provider—decides whether evidence is strong and unambiguous enough to proceed.

Native Chrome login slice

Build the native helper:

cd native/ChoresDBMacOS
swift build
cd ../..

Copy examples/browser_login.example.json into the ignored local configuration directory, then configure the URL, unique logged-in/logged-out URL or text evidence, federated provider, exact provider account, and explicit terms authorization for the target site. URL state rules match scheme, host, port, and path while ignoring query parameters, fragments, and a trailing slash.

mkdir -p .choresdb/local
cp examples/browser_login.example.json .choresdb/local/browser_login.json
PYTHONPATH=src python3 -m choresdb browser-login \
  .choresdb/local/browser_login.json

Add --debug to follow each node and the accessibility resolution decisions while the workflow runs:

PYTHONPATH=src python3 -m choresdb browser-login \
  .choresdb/local/browser_login.json --debug

Debug output includes state-poll counts, target IDs, confidence, selected element geometry, dispatched actions, and elapsed time. It does not print accessibility text dumps, the configured account identifier, or the full destination URL.

Authenticated text-entry slice

Copy examples/browser_text_entry.example.json into .choresdb/local/ and configure its destination URL, required URL identifier, positive logged-in evidence, and anchor text. The slice aborts unless authentication and URL identity are verified. It then makes at most max_scrolls downward scrolls, selects an unambiguous accessibility text field spatially below the anchor, enters message, verifies the field value, and ends without closing the browser.

Closing Chrome manually cancels an active browser workflow. Browser-targeted hotkeys, scrolling, and text entry are rejected once Chrome is no longer running, so input cannot fall through to another application.

mkdir -p .choresdb/local
cp examples/browser_text_entry.example.json \
  .choresdb/local/browser_text_entry.json
PYTHONPATH=src python3 -m choresdb browser-text-entry \
  .choresdb/local/browser_text_entry.json --debug

Local personal configuration

Files under .choresdb/local/ are ignored by Git and are the only intended place for personal runtime values such as account identifiers, private URLs, task IDs, and site-specific anchor text. Files under examples/ are sanitized, tracked templates; the CLI refuses to execute an .example.json file directly. Passwords remain forbidden in every configuration.

Before committing, verify a local file is ignored:

git check-ignore -v .choresdb/local/browser_text_entry.json

On the first run, macOS requests Accessibility permission for the native helper. The slice fails closed if both marker sets are visible, neither is visible, a target is ambiguous, checkbox state cannot be verified, the exact configured account is absent, or verification does not reach the logged-in state.

The active slice supports federated sign-in only, initially Google. It can accept an explicitly authorized agreement, activate the declared provider, and select the exact configured account. choresdb never reads, retrieves, stores, or types a password. Chrome, Google Password Manager, iCloud Passwords, or the user owns any password challenge. Signup, account creation, CAPTCHA, and recovery flows are intentionally unsupported.

Intended CLI

The project is working toward these user-facing operations:

choresdb record demo-login-button
choresdb run workflows/milestone_01.json --param url=https://example.com

Passwords are outside the choresdb execution boundary. The browser or password manager handles them; workflows only select a declared provider and account.

See the architecture and roadmap for the current boundaries and implementation sequence.

License

Licensed under the MIT License.