Quixi Math Data Generator

QuixiMath is brought to you by Eric Hartford and QuixiAI

https://huggingface.co/datasets/QuixiAI/QuixiMath-1B

Purpose

Quixi Math generates synthetic math problems with visible, step-by-step scratchpads. Each example includes the problem text, pipe-delimited solution steps, a canonical final answer, and curriculum metadata.

The dataset is designed for training and evaluating language models on multi-step mathematical reasoning. It can be used for SFT-style trace learning and RL-style answer/reasoning validation; generate separate datasets for those uses so syntax learning and reward optimization can be controlled separately.

The repo hosts two programs. quixi_math_datagen.py generates records where the generator is the solver — the scratchpad is a byproduct of computing the answer. quixi_proofs_datagen.py generates records where the generator is the prover — the derivation is the artifact (see below).

QuixiProofs

The second program (plans/proofs_plan.md) produces machine-checkable proofs: PREMISE/IH/STEP lines whose STEP|k|claim|warrant|refs grammar cites earlier lines by number and names a warrant from the closed registry in WARRANTS.md, closed by QED|<final_answer>. Unlike the math program's organic op-codes, a warrant is only legal if a checker function in proofs_common.py backs it: check_proof replays every derivation line — and any candidate derivation, which makes it an RL/eval grader that rewards the relation, not one reference path (derivation paths are sampled, never canonical).

Families span justified rewrite chains (algebra, exponents, logs, factoring, telescoping), number-theory certificates (gcd/Bezout, primality, congruence chains, CRT, divisibility rules with refutations), scoped induction with IH blocks the checker encloses and discharges (sum formulas, exponential divisibility, inequalities with generated thresholds, recurrence closed forms), invariant and termination games, graph certificates, inequality composition from axiom atoms, exact-surd trigonometry, angle chasing, and error-spotting critics whose answer is step <k>; <correct claim>.

uv run python quixi_proofs_datagen.py --sample
uv run python quixi_proofs_datagen.py -n 10000 -s 42

Every proof family passes a two-layer oracle: the checker replays each derivation (validity), and an independent route re-establishes the theorem from the problem text alone (truth).

The v2 proof pipeline adds a typed, theorem-bound checker, 19 connected native schemas, reusable lemmas, proof/completion/critique tasks, and theorem-grouped evaluation splits. See the generated worked recipe catalog and measured diversity report. The original v1 generators remain available. Native validation ports now cover all 38 original families and 92 variants, with direct typed generation.

uv run python quixi_proofs_datagen.py --schema-version 2 --profile connected \
  --task mixed -n 200 -s 7 -o /tmp/proofs-v2.jsonl

Sample

Current Inventory

The generated catalog is the source of truth: PROBLEM_TYPES.md.

Current repo snapshot:

• 689 problem-type entries in the generated catalog, one per registered generator class
• 712 registered generator instances; 711 are in the default pool
• 688 default sampled skills in dataset builds; MixedNumberOperationsRandom is an opt-in wrapper and is excluded from the default pool to avoid double-counting the four explicit mixed-number operation variants
• 3,104 distinct operation variant labels across the catalog
• 2,062 observed scratchpad op-codes in OPCODES.md
• Catalog grade-band distribution: 54 elementary, 104 middle, 208 high, 194 college, 129 graduate

The CLI samples equally per skill by default, not equally per generator instance. Variant instances of one class, such as FractionOpGenerator('+') and FractionOpGenerator('/'), share one skill slot unless explicitly weighted.

Coverage

Coverage now spans elementary through graduate-level topics:

• Elementary: whole-number algorithms, decimals, fractions, mixed numbers, conversions, factors/GCF/LCM, order of operations, number sense, unit conversions, elementary geometry, graph reading, simple statistics, and probability.
• Middle school: ratios and rates, proportional relationships, integer operations, equations and inequalities, exponent rules, scientific notation, geometry and measurement, compound probability, finance, physics formulas, base arithmetic, modular arithmetic, and calendar/manual computation.
• High school: Algebra 1/2, factoring, polynomial and rational expressions, systems, functions, sequences, conics, geometry, trigonometry, vectors, matrices, calculus, statistics, probability, finite math, and critic formats such as error spotting and fill-in-the-step records.
• College: multivariable calculus, linear algebra, differential equations, discrete math, graph algorithms, number theory, abstract algebra, complex analysis, numerical methods, distributions, optimization, signals, physics, chemistry, information theory, machine learning, and finance.
• Graduate: differential geometry, quantum mechanics and quantum information, Lie/group and tensor notation, relativity, particle/field physics, advanced probability/statistics, information theory, deep learning, kernel methods, transformer arithmetic, and quantitative finance.
• Foundations strand: 55 generator classes and 238 internal variants span concrete classification and correspondence, formal propositional and predicate logic, proof systems, sets, relations, functions, number constructions, ordinals and cardinals, type theory, ZF axiom identification, finite-structure isomorphism, and critic records. Every variant has 3–5 prompt phrasings and a prompt-only exact oracle test.
• Probability strand: 47 generator classes and 280 internal variants span likelihood language and finite experiments through conditional expectation, named distributions, limit bounds, random walks, finite Markov chains, sigma-algebras, martingales, optional stopping, and critic records. Every variant has 3–5 prompt phrasings and a prompt-only exact oracle test.
• Statistics strand: 46 generator classes and 297 internal variants span data displays and descriptive measures through sampling distributions, confidence intervals, hypothesis tests, study design, likelihood theory, estimator quality, and conjugate Bayesian updating. Every variant has 3–5 prompt phrasings and a problem-text-only oracle; required normal and critical values are supplied in the prompt, while other arithmetic remains exact.
• Applied strand: modeling word problems (work rate, mixture, motion, systems, quadratics, growth comparison, optimization, rate-of-change interpretation), number sense and measurement judgment (mental-math strategies, magnitude comparison, rounding effects, significant figures, measurement uncertainty), judgment tasks (missing information, method discrimination, assumption checking, qualitative reasoning, plausibility critiquing), quantitative literacy (risk communication, Simpson's paradox, statistical literacy, index and growth, decision under uncertainty), representation translation, formula derivation, spatial intuition (square-cube law, packing, spatial description), and multi-part scenarios that compose several of the above into one record with a skills list naming what was reused. The strand's defining rule is that the problem text never names the solution method; distractor, estimate_first, and with_model variants run throughout, including retrofitted onto several pre-existing generators (finance, annuities, related rates, optimization, percent problems, linear systems, Pythagorean word problems, similar figures, and permutation/combination word problems).
• Depth strand: 16 generator classes plus two retrofits (ExtendedEuclid, ContinuedFraction) teach endurance: serial dependency chains of 40–260 steps where each step consumes the previous step's result, one slip propagates to the final answer, and the state stays bounded by construction (modular orbits, exact-cents ledgers, digit processes, √d's (P, Q) recurrence, register machines, sorting rewrites, amortization schedules, and serial skill pipelines carrying a skills list). Depth tiers ride the operation suffix (_d50/_d100/_d200, chain-length floors 40/85/170, measured structurally — enumeration cannot masquerade as depth), MILESTONE rows recompute a running invariant every 10–15 links at d100+, one long-form critic class (LedgerAudit) hides a single propagated error in a tier-length claimed table, and every variant has 4 prompt phrasings with a prompt-only exact oracle.

Hugging Face dataset-card note

The hosted QuixiMath-1B dataset is a release artifact and may lag the current generator registry. Future dataset cards should pin the source commit and publish the generated inventory and observed grade/difficulty/operation distributions for that release. tools/build_hf_release.py's write_readme() already documents the foundations, probability, and statistics strands (exactness guarantees and supplied lookup values); regenerate the card after any change to those strands' answer format so the description stays accurate.

Signature behaviors:

• Every arithmetic action is explicit when it would naturally appear in a pencil-and-paper solution.
• CHECK, substitute-back, multiply-back, sign-chart, table, and theorem checks are emitted where natural.
• Trial paths can be visible through TRY, REJECT, and ACCEPT.
• Derived critic records keep the same JSONL schema while embedding the worked or partial scratchpad in the prompt text.
• Generated op-code and problem-type docs can be checked for freshness in CI.

Usage

Prefer uv run python ... so the repo environment is selected explicitly. If you are not using uv, activate the virtual environment first:

source .venv/bin/activate

Generate Samples

With no arguments, the CLI prints one sample from each registered generator instance:

uv run python quixi_math_datagen.py

The explicit form is:

uv run python quixi_math_datagen.py --sample

Use a seed for reproducible samples:

uv run python quixi_math_datagen.py --sample -s 7

Limit samples to specific generator classes:

uv run python quixi_math_datagen.py --sample \
  --generators MultiDigitAdditionGenerator,LongDivisionGenerator

Generate a Dataset

Generate JSONL with an explicit output path:

uv run python quixi_math_datagen.py -n 50000 -o quixi_math_50000.jsonl -s 123

If -o/--output is omitted, the output path defaults to quixi_math_<n>.jsonl:

uv run python quixi_math_datagen.py -n 50000 -s 123

Restrict a build to selected generator classes:

uv run python quixi_math_datagen.py -n 5000 -o subset.jsonl \
  --generators MultiDigitAdditionGenerator,DecimalMultGenerator

Omit -s/--seed for natural randomness. Provide a seed when byte-for-byte reproducibility matters.

Sampling, Weights, and Deduplication

Dataset builds sample equally per skill by default. Override individual skill weights with --weights; unlisted skills keep weight 1.0.

Inline weights:

uv run python quixi_math_datagen.py -n 10000 \
  --weights "QuadraticGenerator=3,MeanGenerator=0.5"

JSON file weights:

{
  "QuadraticGenerator": 3,
  "MeanGenerator": 0.5
}

uv run python quixi_math_datagen.py -n 10000 --weights weights.json

Exact (operation, problem) repeats are skipped by default. Pass --allow-duplicates to keep repeats, which is useful for very large datasets or intentionally small exact problem spaces.

Every dataset run prints a per-generator stats table with emitted counts, duplicate skips, and errors. If the selected problem space is exhausted before -n, generation stops early with a warning.

Output Format

Each JSONL line is one problem:

{
  "problem_id": "1f8b6be5-...",
  "operation": "long_division",
  "problem": "1834 / 5",
  "steps": ["D|18|5|3", "M|3|5|15", "S|18|15|3", "B|3|3|33", "Z|366 R4"],
  "final_answer": "366 R4",
  "grade_level": "elementary",
  "difficulty": 3
}

Required fields:

• problem_id: generated UUID
• operation: internal operation or variant label
• problem: human-readable prompt
• steps: visible scratchpad as CODE|field|field|... strings
• final_answer: canonical answer string
• grade_level: elementary, middle, high, college, or graduate
• difficulty: integer from 1 to 5, read relative to the grade band

Scenario and discrimination records may additionally include skills, an ordered list of the procedures they compose. JSONL and Hugging Face Parquet releases preserve this optional metadata; ordinary records leave it absent or null.

The final step must be exactly Z|<final_answer>. Metadata is stamped from curriculum.py after generation unless a generator intentionally overrides it.

Answer-format conventions live in DESIGN.md. Generated examples are structurally validated before being written.

Generated Docs

Three files are generated and should not be hand-edited:

• PROBLEM_TYPES.md: user-facing catalog with one worked example per problem type (math program)
• OPCODES.md: descriptive legend of observed scratchpad op-codes
• WARRANTS.md: the closed, authoritative warrant registry for the proofs program

Regenerate or check them with:

uv run python tools/gen_problem_types.py
uv run python tools/gen_problem_types.py --check

uv run python tools/gen_opcode_legend.py
uv run python tools/gen_opcode_legend.py --check

uv run python tools/gen_warrants.py
uv run python tools/gen_warrants.py --check

The op-code vocabulary is descriptive and organic. New op-codes are fine, but do not reuse an existing op-code with different field semantics.

Testing

Run the full unittest suite:

uv run python -m unittest discover tests

If the dev dependency group is installed, pytest is also available:

uv run pytest tests

Focused generator tests follow the module name:

uv run python -m unittest tests.test_quadratic_generator

Before handing off generator changes, also run:

uv run python tools/gen_opcode_legend.py --check
uv run python tools/gen_problem_types.py --check
uv run python quixi_math_datagen.py --sample --generators MyNewGenerator

For capacity checks, use:

uv run python tools/probe_generator_capacity.py

Dependencies

• Python 3.9+
• Runtime dependencies: none beyond the standard library
• Dev dependency group: pytest>=8.0

Project Structure

quixi-math/
├── quixi_math_datagen.py      # Main CLI, sampling, validation, JSONL build
├── base_generator.py            # ProblemGenerator contract
├── helpers.py                   # step formatter, seeded UUID helper, utilities
├── curriculum.py                # class -> grade_level/difficulty table
├── generators/                  # generator implementations
├── tests/                       # unittest coverage and oracle helpers
├── plans/                       # implementation and dataset release plans
├── tools/
│   ├── gen_opcode_legend.py     # regenerates OPCODES.md
│   ├── gen_problem_types.py     # regenerates PROBLEM_TYPES.md
│   └── probe_generator_capacity.py
├── DESIGN.md                    # architecture and answer conventions
├── OPCODES.md                   # generated op-code legend
├── PROBLEM_TYPES.md             # generated problem-type catalog
├── TODO.md                      # implementation follow-ups/history
├── AGENTS.md                    # coding-agent guidelines
└── pyproject.toml               # package metadata and dev dependencies

Generated datasets are written to the repo root unless -o points elsewhere. Avoid committing large JSONL files; use /tmp/... for local experiments.

Contributing

When adding a new generator:

1. Create generators/my_new_generator.py extending ProblemGenerator.
2. Create tests/test_my_new_generator.py with unittest coverage.
3. Include an oracle test that recomputes final_answer from the problem text alone, preferably by a route independent of the generator implementation.
4. Add an import and an instance to ALL_GENERATORS in quixi_math_datagen.py.
5. Add a curriculum.CURRICULUM entry for the class.
6. Regenerate OPCODES.md and PROBLEM_TYPES.md.
7. Run the focused test, a restricted seeded sample, and the full test suite.

Each generator must emit pipe-safe steps, use exact arithmetic when practical, and end with Z|<final_answer>.