From 4838b0b2b94835b9961e7737545e617ae13762a0 Mon Sep 17 00:00:00 2001 From: xiaolei Date: Mon, 8 Dec 2025 21:31:26 -0500 Subject: [PATCH 1/2] update doc --- README.md | 54 +++++++++++------------------- benchmark/README.md | 81 +++++++-------------------------------------- 2 files changed, 31 insertions(+), 104 deletions(-) diff --git a/README.md b/README.md index c60adcc..ece4730 100644 --- a/README.md +++ b/README.md @@ -16,15 +16,10 @@ A PostgreSQL connection pooler written in Go that acts as a proxy between client - **PostgreSQL Wire Protocol**: Full implementation supporting 40+ message types - **Extended Query Support**: Parse, Bind, Execute, and Sync operations - **Prepared Statement Caching**: Triggers prepare implicitly if server connection misses preprared on new pairing -- **Copy Command Support**: COPY FROM/COPY TO operations +- **Comprehensive testing**: Integration testing and TPC-B full protocol benchmarking with other poolers ## Quick Start -### Prerequisites - -- Go 1.25 or later -- PostgreSQL server (tested with PostgreSQL 16) - ### Installation ```bash @@ -63,8 +58,6 @@ max_clients = 100 # Run with custom config ./pgpool --config /path/to/config.ini -# or shorthand -./pgpool -c /path/to/config.ini ``` ### Command-Line Flags @@ -128,60 +121,51 @@ pgpool supports multiple authentication methods for securing client connections. ### Trust +No password required. Use only in trusted environments. Postgres server instance has to support trust authentication for the same user. + ```ini auth_type = trust password = ``` -No password required. Use only in trusted environments. +## Testing -## Development - -### Running Tests +### Unit Tests ```bash # Unit tests with coverage go test -v -race -coverprofile=coverage.txt ./pool/... ./proto/... - -# Integration tests (requires PostgreSQL) -cd test -./run.sh ``` -### Integration Test Configuration +### Integration tests -The integration test suite supports these environment variables: +#### smoke test -- `NUM_CONNECTIONS`: Number of concurrent connections (default: 50) -- `QUERIES_PER_CONNECTION`: Queries per connection (default: 5) -- `DB_CONNECTION_STRING`: Connection string to pgpool (default: uses pgpool.conf) +The smoke suite tests SQL queries, transactions and prepared statements. -Example: ```bash -NUM_CONNECTIONS=100 QUERIES_PER_CONNECTION=10 ./run.sh -``` +./run_smoke.sh -### Building +``` -```bash -# Standard build -go build -v -o pgpool . +#### concurent test -# Build with race detector -go build -race -v -o pgpool . +The concurent suite tests the core pooling functions for multiple client connections -# Cross-compile for Linux -GOOS=linux GOARCH=amd64 go build -v -o pgpool-linux . +```bash +NUM_CONNECTIONS=100 QUERIES_PER_CONNECTION=10 ./run_concurrent.sh ``` -### Benchmark +### Pgbench Benchmark Testing with other poolers together + +Check [Readme](benchmark/README.md) and example benchmark [report](benchmark/report.md) -Check [benchmark](benchmark/README.md) ## TODO * SSL support * Session pooling +* Extended protocol performance optimization * Enrich Statistics & Output * Sharding? @@ -197,4 +181,4 @@ Contributions are welcome! Please feel free to submit a Pull Request. ## Acknowledgments -Built with Go's standard library and inspired by PostgreSQL's connection pooling needs. +Built with Go's standard library and inspired by Pgbouncer diff --git a/benchmark/README.md b/benchmark/README.md index eb30d7b..e2225ab 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -1,6 +1,6 @@ -# PostgreSQL Connection Pool Benchmark Suite +# Benchmark Suite -Comprehensive benchmarking suite for comparing **pgpool** vs **pgcat** vs **pgbouncer** vs **direct PostgreSQL** connections using the industry-standard `pgbench` tool. +Comprehensive benchmarking suite for comparing **pgpool**, **pgcat**, **pgbouncer** and **direct PostgreSQL** connections using the industry-standard `pgbench` tool. ## Overview @@ -10,49 +10,14 @@ This benchmark suite tests: - **Protocol performance** (simple, extended, prepared statements) - **Workload patterns** (read-only, read-write, complex queries) -## Directory Structure - -``` -benchmark/ -├── config/ # Configuration files -│ ├── pgpool.ini # pgpool configuration -│ ├── pgbouncer.ini # pgbouncer configuration -│ ├── pgcat.toml # pgcat configuration -│ └── userlist.txt # pgbouncer auth file -├── scripts/ # Benchmark scripts -│ ├── setup.sh # Environment setup -│ ├── run_benchmark.sh # Main benchmark runner -│ ├── generate_report.sh # Report gener - -4. **Connection Time** - Lower is better - - Overhead of establishing conator -│ └── cleanup.sh # Cleanup script -├── results/ # Benchmark results (auto-generated) -│ └── YYYYMMDD_HHMMSS/ # Timestamped results -└── README.md # This file -└── report.md # sample report -``` - ## Prerequisites -1. **PostgreSQL** (tested with PostgreSQL 16+) -2. **pgbench** (included with PostgreSQL client tools) +1. **PostgreSQL** (tested with PostgreSQL 16) +2. **pgbench** 3. **pgpool** (built from this repository) 4. **pgcat** ([built from git repository](https://github.com/postgresml/pgcat)) 5. **pgbouncer** (1.24.0) -### Installation - -#### macOS -```bash -brew install postgresql pgbouncer -``` - -#### Ubuntu/Debian -```bash -sudo apt-get install postgresql-client pgbouncer -``` - ## Quick Start ### 1. Setup Environment @@ -62,43 +27,21 @@ cd benchmark ./scripts/setup.sh ``` -This will: -- Check PostgreSQL connectivity -- Verify pgbench installation -- Build pgpool binary if needed -- Check pgbouncer availability - ### 2. Configure Poolers -#### pgpool Configuration - -Edit `config/pgpool.ini`: -```ini -[app] -addr = localhost:5433 - -[db] -id = 1 -name = postgres -host = localhost:5432 - -[user] -dbid = 1 -name = pgtest -auth_type = scram -password = test123 -max_conn = 20 -max_clients = 200 -``` +make sure all the poolers have the same max db connection setting + +#### pgpool -#### pgbouncer Configuration +Edit `config/pgpool.ini` with db and credential -Edit `config/pgbouncer.ini` and `config/userlist.txt` with your credentials. +#### pgbouncer +Edit `config/pgbouncer.ini` and `config/userlist.txt` -#### pgcat Configuration +#### pgcat -Edit `pgcat.toml` with test db and credentials. +Edit `pgcat.toml` with test db and credentials ### 3. Start Poolers From 2a02a243e31fb3cfe676c87ee91e8d6ac321d4e5 Mon Sep 17 00:00:00 2001 From: xiaolei Date: Mon, 8 Dec 2025 21:47:39 -0500 Subject: [PATCH 2/2] update rm --- README.md | 66 +++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 45 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index ece4730..39def70 100644 --- a/README.md +++ b/README.md @@ -95,33 +95,21 @@ max_clients = 100 | `dbid` | Link to database ID | `1` | | `name` | Username | `myuser` | | `auth_type` | Authentication method: `password`, `md5`, `scram`, or `trust` | `scram` | -| `password` | Password (clear password) | See authentication section | +| `password` | Password (clear password) | `test` | | `max_conn` | Maximum server connections | `20` | | `max_clients` | Maximum client connections | `100` | ## Authentication -pgpool supports multiple authentication methods for securing client connections. The authentication flow works in two stages: +### Password -1. **Client → pgpool**: Clients authenticate to pgpool using the configured method -2. **pgpool → PostgreSQL**: pgpool authenticates to PostgreSQL on behalf of the client - -**Important:** The PostgreSQL server must be configured to support the relevant authentication method. Check your PostgreSQL `pg_hba.conf` file to ensure the authentication method is enabled for the target database and user. - - -**Recommended authentication method** - SCRAM-SHA-256 (Salted Challenge Response Authentication Mechanism) is the most secure authentication method supported by PostgreSQL. - -**How it works:** -- Simply provide the plain password in the configuration file -- pgpool automatically generates SCRAM keys internally (salt, stored key, server key) -- Uses challenge-response authentication between: - - Client → pgpool (validates client connections) - - pgpool → PostgreSQL server (authenticates with the database) +**Clear Password, MD5 and SCRAM** are supported for client to pgpool connection and pgpool to PostgreSQL server connection. +SCRAM (Salted Challenge Response Authentication Mechanism) is the default authentication method supported by PostgreSQL 14+. ### Trust -No password required. Use only in trusted environments. Postgres server instance has to support trust authentication for the same user. +No password required. Use only in trusted environments. PostgreSQL server must be configured to support trust authentication for the same user. ```ini auth_type = trust @@ -133,7 +121,6 @@ password = ### Unit Tests ```bash -# Unit tests with coverage go test -v -race -coverprofile=coverage.txt ./pool/... ./proto/... ``` @@ -145,7 +132,6 @@ The smoke suite tests SQL queries, transactions and prepared statements. ```bash ./run_smoke.sh - ``` #### concurent test @@ -158,7 +144,45 @@ NUM_CONNECTIONS=100 QUERIES_PER_CONNECTION=10 ./run_concurrent.sh ### Pgbench Benchmark Testing with other poolers together -Check [Readme](benchmark/README.md) and example benchmark [report](benchmark/report.md) +Check [Readme](benchmark/README.md) and example [report](benchmark/report.md) + +**Report Snapshort** + +#### Protocol: extended + +##### Throughput (TPS) + +| Connections | direct | pgpool | pgcat | pgbouncer | +|-------------|-----------|-----------|-----------|-----------| +| 10 | 540.992544 | 629.573937 | 400.953194 | 508.470016 | +| 50 | 520.948243 | 513.098964 | 444.584482 | 719.135004 | + +##### Average Latency (ms) + +| Connections | direct | pgpool | pgcat | pgbouncer | +|-------------|-----------|-----------|-----------|-----------| +| 10 | 18.485 | 15.884 | 24.941 | 19.667 | +| 50 | 95.979 | 97.447 | 112.465 | 69.528 | + +--- + +#### Protocol: prepared + +##### Throughput (TPS) + +| Connections | direct | pgpool | pgcat | pgbouncer | +|-------------|-----------|-----------|-----------|-----------| +| 10 | 603.806235 | 671.575983 | NA | 484.158180 | +| 50 | 571.118586 | 819.022499 | NA | 774.326083 | + +##### Average Latency (ms) + +| Connections | direct | pgpool | pgcat | pgbouncer | +|-------------|-----------|-----------|-----------|-----------| +| 10 | 16.562 | 14.890 | NA | 20.654 | +| 50 | 87.547 | 61.048 | NA | 64.572 | + +--- ## TODO @@ -181,4 +205,4 @@ Contributions are welcome! Please feel free to submit a Pull Request. ## Acknowledgments -Built with Go's standard library and inspired by Pgbouncer +Built with Go's standard library and inspired by Pgbouncer.