quick_match

Language: Simplified Chinese

quick_match is a Go trading matching system designed for continuous auction style assets and account-balance settlement flows. The project is split into three runtime processes: order, match, and settle. The order process provides the unified order entry point, match processes are sharded by symbol, and settle processes asynchronously consume match results.

Scope

The current implementation can be used as a prototype for continuous auction matching, matching engine learning, or internal matching services. It covers a single-symbol in-memory order book, price-time priority matching, limit and market orders, common time-in-force behavior, order fund freezing, deal fund mutations, fee calculation, Kafka outbox delivery, and idempotent settlement.

It should not be described as a complete securities or stock market system. The code does not yet implement market calendars, complete open and close trading phases, formal auction matching, price limit rule sources, securities positions and sellable quantity checks, T+1/T+2 settlement, corporate actions, regulatory reporting, broker or exchange seats, or clearing member semantics. Securities or stock use cases should add those market rules, risk controls, clearing modules, and compliance modules around the existing matching core.

Features

• Unified order entry: quick_match order exposes submit, cancel, and amend APIs through gRPC.
• Asynchronous order stream: validated orders are written to Kafka first, then persisted through an asynchronous path.
• Single-symbol matching: each quick_match match --symbol process maintains one in-memory order book.
• Price-time priority: higher buy prices are prioritized, lower sell prices are prioritized, and orders at the same price currently use ascending order ID for deterministic ordering.
• Pure in-memory matching core: internal/matchcore does not depend on databases, networking, logging, or wall-clock time.
• Reliable match delivery: match results are written to an outbox and published to Kafka, with retry support for publish failures.
• Asynchronous fund settlement: quick_match settle --symbol consumes match results and applies idempotent fund mutations by deal ID.
• Integer fixed-point values: prices, quantities, and deal amount calculations do not use floating point numbers; deal amounts use large integer boundaries.
• Runtime robustness: command entry points include panic recovery, graceful shutdown, stable log fields, and external dependency failure handling.
• Containerized deployment: Dockerfile, Compose, and deploy.sh support deploying multiple symbol-specific match and settle processes.

Architecture

Runtime constraints:

• order is the unified entry point for all symbols and is not sharded by symbol.
• Each match process handles exactly one symbol. The same symbol should not have multiple active match processes consuming the order stream at the same time.
• Each settle process handles exactly one symbol. Fund mutations are idempotent through processed_settlements.deal_id.
• PostgreSQL is the source of truth for final order, deal, and fund state.
• Kafka messages may be duplicated but must not be silently lost. Consumers should commit offsets only after business processing succeeds.
• Redis is used only for rate limiting, caching, and rebuildable auxiliary state. It is not a source of truth for accounting.

Directory Guide

Path	Description
api/	Protobuf definitions and generated Go gRPC bindings. API changes should start from .proto files.
cmd/	Cobra commands and runtime dependency wiring.
config/	Viper-based configuration loading.
internal/matchcore/	Pure in-memory matching engine and order book logic.
internal/order/	Core order validation, normalization, and submission logic.
internal/matchservice/	Kafka order consumption, matching processing, outbox, and publish flow.
internal/settle/	Pure settlement plan generation.
internal/settlepersist/	Fund mutation and idempotent settlement persistence.
internal/kafkamq/, internal/messaging/	Kafka adapter and message contracts.
model/	GORM persistence models and migration entry point.
deploy/	Local load test and deployment helper files.

Requirements

• Go 1.24 or a compatible toolchain.
• Docker and Docker Compose for local dependencies and container deployment.
• PostgreSQL, Kafka, and Redis. deploy.sh starts local dependencies through Compose.

Configuration

The repository provides two example files:

• config/config.example.yaml: sample CLI config used by quick_match --config. It currently reads only the global log_level.
• .env.example: sample environment variables used by Docker Compose and deploy.sh.

For local deployment, copy .env.example to .env and adjust it as needed:

cp .env.example .env

Runtime parameters such as DB, Kafka, Redis, symbol, and ports are currently passed through command flags or Compose environment variables. They are not read from config.yaml.

Quick Start

Inspect commands:

go run . --help
go run . order --help
go run . match --help
go run . settle --help
go run . migrate --help

Run tests:

go test ./...
go test ./internal/matchcore -bench=.
go test ./internal/order -bench=.
go test ./internal/settle ./internal/settlepersist -bench=.

For a database that already has symbol configuration, deploy.sh reads symbols from the symbols table:

./deploy.sh up
./deploy.sh ps
./deploy.sh logs
./deploy.sh down

deploy.sh generates .deploy/docker-compose.symbols.yaml and runs it together with docker_compose.yaml. The default deployment includes:

• postgres
• redis
• kafka
• migrate
• order
• one match_<symbol> and one settle_<symbol> service for each symbol
• optional symbol_seed, generated only when SEED_SYMBOLS=true

By default, deploy.sh starts PostgreSQL, Kafka, and Redis, runs migrate, queries the symbols table for symbols with status = 'TRADING', and generates match and settle services from that list.

For bootstrapping an empty local database, explicitly set SYMBOLS and enable seeding. The seed inserts missing symbols only and does not overwrite existing configuration:

SYMBOLS=ABC-USD,XYZ-USD SEED_SYMBOLS=true ./deploy.sh up

Command Manual

Database Migration

quick_match migrate \
  --db-driver postgres \
  --db-dsn "host=postgres user=quick_match password=quick_match dbname=quick_match port=5432 sslmode=disable TimeZone=UTC"

migrate runs model.AutoMigrate. In Compose, business services wait for this command to succeed before starting.

Order Service

quick_match order \
  --grpc-addr 0.0.0.0:50051 \
  --kafka-brokers kafka:9092 \
  --db-driver postgres \
  --db-dsn "host=postgres user=quick_match password=quick_match dbname=quick_match port=5432 sslmode=disable TimeZone=UTC" \
  --redis-addr redis:6379 \
  --rate-limit-tps 0 \
  --snowflake-node-id 1

Common flags:

Flag	Description
--grpc-addr	gRPC listen address. If empty, only the skeleton runtime runs.
--kafka-brokers	Comma-separated Kafka broker list.
--db-driver	postgres or sqlite.
--db-dsn	Database DSN.
--redis-addr	Redis address, required when rate limiting is enabled.
--rate-limit-tps	Global order TPS limit. 0 disables Redis rate limiting.
--snowflake-node-id	Order ID node ID.

The gRPC API is defined in api/order/v1/order.proto and currently includes:

• SubmitLimitOrder
• CancelOrder
• AmendLimitOrder

Match Service

quick_match match \
  --symbol ABC-USD \
  --kafka-brokers kafka:9092 \
  --db-driver postgres \
  --db-dsn "host=postgres user=quick_match password=quick_match dbname=quick_match port=5432 sslmode=disable TimeZone=UTC" \
  --snowflake-node-id 1000

Notes:

• --symbol is required.
• One process maintains one symbol order book.
• For multi-symbol deployments, start multiple processes or containers.
• --snowflake-node-id should not conflict with other match processes that generate deal IDs.

Settle Service

quick_match settle \
  --symbol ABC-USD \
  --kafka-brokers kafka:9092 \
  --db-driver postgres \
  --db-dsn "host=postgres user=quick_match password=quick_match dbname=quick_match port=5432 sslmode=disable TimeZone=UTC"

Notes:

• --symbol is required.
• Each settle process applies match results for the current symbol only.
• Duplicate consumption relies on DB idempotency records to prevent repeated debits or credits.

Docker and Compose

Build an image:

VERSION="$(date -u +%Y%m%d%H%M%S)"
docker build -t "quick_match:${VERSION}" .

Deploy a specific version:

./deploy.sh --version "${VERSION}" up
./deploy.sh --version 20260716184630 --no-build up
VERSION=20260716184630 ./deploy.sh up

Render deployment configuration:

./deploy.sh render
./deploy.sh config
SYMBOLS=ABC-USD,XYZ-USD,DEF-USD ./deploy.sh config

render and config also query the database by default. If no database is available, temporarily set SYMBOLS as shown above for offline rendering.

Common environment variables:

Variable	Default	Description
SYMBOLS	empty	Optional symbol override. If empty, symbols are queried from the symbols table.
SYMBOL_STATUS_FILTER	TRADING	Comma-separated status filter used when querying symbols. all disables filtering.
IMAGE_NAME	quick_match	Image name used when QUICK_MATCH_IMAGE is not set.
VERSION	local	Image tag used when QUICK_MATCH_IMAGE is not set. CI defaults to a UTC timestamp.
QUICK_MATCH_IMAGE	empty	Exact business service image. Takes precedence over IMAGE_NAME:VERSION.
BUILD_IMAGE	true	Whether up or restart should build the business image. Set to false or use --no-build when deploying an existing image tag.
KAFKA_BROKERS	kafka:9092	Kafka broker address inside containers.
DB_DRIVER	postgres	Database driver.
DB_DSN	local Compose Postgres DSN	Database connection string.
LOG_LEVEL	info	Log level.
MATCH_NODE_ID_START	1000	Starting node ID for multi-symbol match processes.
SEED_SYMBOLS	false	Whether explicit SYMBOLS overrides should generate and run symbol seed SQL.
SYMBOL_PRICE_SCALE	2	Seed symbol price scale.
SYMBOL_QTY_SCALE	8	Seed symbol quantity scale.
SYMBOL_TICK_SIZE	1	Seed symbol tick size.
SYMBOL_LOT_SIZE	1	Seed symbol lot size.

Order Behavior

Supported time-in-force values:

• GTC: default policy. Unfilled remaining quantity from a limit order enters the order book.
• IOC: immediately matches the fillable quantity and expires the remainder.
• FOK: must be fully filled, otherwise the whole order expires without modifying the order book.
• GTD: requires expires_at_millis; expired orders expire immediately.

Other rules:

• post_only=true is allowed only for limit orders. If it would match immediately, the order expires.
• Symbol rules control price scale, quantity scale, tick size, lot size, quantity bounds, and notional bounds.
• Amounts, prices, and quantities use integer fixed-point values and do not use floating point numbers.
• Self-trade prevention supports cancel taker, cancel maker, and cancel both.

Reliability Principles

• A match result must enter a reliable publish or retry path before matching output is considered delivered.
• Kafka producers use all-replica acknowledgements and retry policies.
• Kafka consumers disable automatic commits and commit offsets only after business processing succeeds.
• Poison messages that cannot be processed are written to dead letter storage.
• Orders are idempotent by order_id; match results and settlements are idempotent by deal_id.
• PostgreSQL failures must not acknowledge final business state.
• Redis failures may affect only caching or rate limiting, not final order, deal, or fund correctness.

Performance Baseline

The current core benchmarks meet the first-stage targets:

Module	Target	Current core benchmark result
match core	100000 TPS+	Main pure in-memory real-time paths exceed the target.
order core	50000 TPS+	Pure in-memory order submission core path exceeds the target.
settle persist	2000 TPS+	SQLite in-memory transactional fund mutation exceeds the target.

These results do not represent real end-to-end throughput across Kafka, PostgreSQL, Redis, and gRPC. To re-run benchmarks:

go test ./internal/matchcore -bench=. -benchmem
go test ./internal/order -bench=. -benchmem
go test ./internal/settle ./internal/settlepersist -bench=. -benchmem

Documentation

• deploy/loadtest/README.md: local end-to-end load test environment.

License

This project is open sourced under the Apache License 2.0.

Development Guidelines

• Keep internal/matchcore pure and deterministic. Do not introduce I/O, logging, persistence, or wall-clock time.
• When changing price-time priority behavior, update same-price insertion, recovery ordering, crossing orders tests, and benchmarks.
• Treat Kafka, PostgreSQL, and Redis as dependencies that can fail. Service-layer changes must preserve idempotency, retries, and outbox semantics.
• API changes should start from .proto files. Do not manually edit generated *.pb.go files.
• Do not add global mutable state unless it is part of runtime wiring and has a clear lifecycle.