diff --git a/README.md b/README.md index c60adcc..39def70 100644 --- a/README.md +++ b/README.md @@ -16,15 +16,10 @@ A PostgreSQL connection pooler written in Go that acts as a proxy between client - **PostgreSQL Wire Protocol**: Full implementation supporting 40+ message types - **Extended Query Support**: Parse, Bind, Execute, and Sync operations - **Prepared Statement Caching**: Triggers prepare implicitly if server connection misses preprared on new pairing -- **Copy Command Support**: COPY FROM/COPY TO operations +- **Comprehensive testing**: Integration testing and TPC-B full protocol benchmarking with other poolers ## Quick Start -### Prerequisites - -- Go 1.25 or later -- PostgreSQL server (tested with PostgreSQL 16) - ### Installation ```bash @@ -63,8 +58,6 @@ max_clients = 100 # Run with custom config ./pgpool --config /path/to/config.ini -# or shorthand -./pgpool -c /path/to/config.ini ``` ### Command-Line Flags @@ -102,86 +95,101 @@ max_clients = 100 | `dbid` | Link to database ID | `1` | | `name` | Username | `myuser` | | `auth_type` | Authentication method: `password`, `md5`, `scram`, or `trust` | `scram` | -| `password` | Password (clear password) | See authentication section | +| `password` | Password (clear password) | `test` | | `max_conn` | Maximum server connections | `20` | | `max_clients` | Maximum client connections | `100` | ## Authentication -pgpool supports multiple authentication methods for securing client connections. The authentication flow works in two stages: - -1. **Client → pgpool**: Clients authenticate to pgpool using the configured method -2. **pgpool → PostgreSQL**: pgpool authenticates to PostgreSQL on behalf of the client - -**Important:** The PostgreSQL server must be configured to support the relevant authentication method. Check your PostgreSQL `pg_hba.conf` file to ensure the authentication method is enabled for the target database and user. +### Password +**Clear Password, MD5 and SCRAM** are supported for client to pgpool connection and pgpool to PostgreSQL server connection. -**Recommended authentication method** - SCRAM-SHA-256 (Salted Challenge Response Authentication Mechanism) is the most secure authentication method supported by PostgreSQL. - -**How it works:** -- Simply provide the plain password in the configuration file -- pgpool automatically generates SCRAM keys internally (salt, stored key, server key) -- Uses challenge-response authentication between: - - Client → pgpool (validates client connections) - - pgpool → PostgreSQL server (authenticates with the database) - +SCRAM (Salted Challenge Response Authentication Mechanism) is the default authentication method supported by PostgreSQL 14+. ### Trust +No password required. Use only in trusted environments. PostgreSQL server must be configured to support trust authentication for the same user. + ```ini auth_type = trust password = ``` -No password required. Use only in trusted environments. +## Testing -## Development - -### Running Tests +### Unit Tests ```bash -# Unit tests with coverage go test -v -race -coverprofile=coverage.txt ./pool/... ./proto/... - -# Integration tests (requires PostgreSQL) -cd test -./run.sh ``` -### Integration Test Configuration +### Integration tests -The integration test suite supports these environment variables: +#### smoke test -- `NUM_CONNECTIONS`: Number of concurrent connections (default: 50) -- `QUERIES_PER_CONNECTION`: Queries per connection (default: 5) -- `DB_CONNECTION_STRING`: Connection string to pgpool (default: uses pgpool.conf) +The smoke suite tests SQL queries, transactions and prepared statements. -Example: ```bash -NUM_CONNECTIONS=100 QUERIES_PER_CONNECTION=10 ./run.sh +./run_smoke.sh ``` -### Building +#### concurent test + +The concurent suite tests the core pooling functions for multiple client connections ```bash -# Standard build -go build -v -o pgpool . +NUM_CONNECTIONS=100 QUERIES_PER_CONNECTION=10 ./run_concurrent.sh +``` -# Build with race detector -go build -race -v -o pgpool . +### Pgbench Benchmark Testing with other poolers together -# Cross-compile for Linux -GOOS=linux GOARCH=amd64 go build -v -o pgpool-linux . -``` +Check [Readme](benchmark/README.md) and example [report](benchmark/report.md) + +**Report Snapshort** + +#### Protocol: extended + +##### Throughput (TPS) + +| Connections | direct | pgpool | pgcat | pgbouncer | +|-------------|-----------|-----------|-----------|-----------| +| 10 | 540.992544 | 629.573937 | 400.953194 | 508.470016 | +| 50 | 520.948243 | 513.098964 | 444.584482 | 719.135004 | + +##### Average Latency (ms) + +| Connections | direct | pgpool | pgcat | pgbouncer | +|-------------|-----------|-----------|-----------|-----------| +| 10 | 18.485 | 15.884 | 24.941 | 19.667 | +| 50 | 95.979 | 97.447 | 112.465 | 69.528 | + +--- + +#### Protocol: prepared + +##### Throughput (TPS) + +| Connections | direct | pgpool | pgcat | pgbouncer | +|-------------|-----------|-----------|-----------|-----------| +| 10 | 603.806235 | 671.575983 | NA | 484.158180 | +| 50 | 571.118586 | 819.022499 | NA | 774.326083 | + +##### Average Latency (ms) + +| Connections | direct | pgpool | pgcat | pgbouncer | +|-------------|-----------|-----------|-----------|-----------| +| 10 | 16.562 | 14.890 | NA | 20.654 | +| 50 | 87.547 | 61.048 | NA | 64.572 | -### Benchmark +--- -Check [benchmark](benchmark/README.md) ## TODO * SSL support * Session pooling +* Extended protocol performance optimization * Enrich Statistics & Output * Sharding? @@ -197,4 +205,4 @@ Contributions are welcome! Please feel free to submit a Pull Request. ## Acknowledgments -Built with Go's standard library and inspired by PostgreSQL's connection pooling needs. +Built with Go's standard library and inspired by Pgbouncer. diff --git a/benchmark/README.md b/benchmark/README.md index eb30d7b..e2225ab 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -1,6 +1,6 @@ -# PostgreSQL Connection Pool Benchmark Suite +# Benchmark Suite -Comprehensive benchmarking suite for comparing **pgpool** vs **pgcat** vs **pgbouncer** vs **direct PostgreSQL** connections using the industry-standard `pgbench` tool. +Comprehensive benchmarking suite for comparing **pgpool**, **pgcat**, **pgbouncer** and **direct PostgreSQL** connections using the industry-standard `pgbench` tool. ## Overview @@ -10,49 +10,14 @@ This benchmark suite tests: - **Protocol performance** (simple, extended, prepared statements) - **Workload patterns** (read-only, read-write, complex queries) -## Directory Structure - -``` -benchmark/ -├── config/ # Configuration files -│ ├── pgpool.ini # pgpool configuration -│ ├── pgbouncer.ini # pgbouncer configuration -│ ├── pgcat.toml # pgcat configuration -│ └── userlist.txt # pgbouncer auth file -├── scripts/ # Benchmark scripts -│ ├── setup.sh # Environment setup -│ ├── run_benchmark.sh # Main benchmark runner -│ ├── generate_report.sh # Report gener - -4. **Connection Time** - Lower is better - - Overhead of establishing conator -│ └── cleanup.sh # Cleanup script -├── results/ # Benchmark results (auto-generated) -│ └── YYYYMMDD_HHMMSS/ # Timestamped results -└── README.md # This file -└── report.md # sample report -``` - ## Prerequisites -1. **PostgreSQL** (tested with PostgreSQL 16+) -2. **pgbench** (included with PostgreSQL client tools) +1. **PostgreSQL** (tested with PostgreSQL 16) +2. **pgbench** 3. **pgpool** (built from this repository) 4. **pgcat** ([built from git repository](https://github.com/postgresml/pgcat)) 5. **pgbouncer** (1.24.0) -### Installation - -#### macOS -```bash -brew install postgresql pgbouncer -``` - -#### Ubuntu/Debian -```bash -sudo apt-get install postgresql-client pgbouncer -``` - ## Quick Start ### 1. Setup Environment @@ -62,43 +27,21 @@ cd benchmark ./scripts/setup.sh ``` -This will: -- Check PostgreSQL connectivity -- Verify pgbench installation -- Build pgpool binary if needed -- Check pgbouncer availability - ### 2. Configure Poolers -#### pgpool Configuration - -Edit `config/pgpool.ini`: -```ini -[app] -addr = localhost:5433 - -[db] -id = 1 -name = postgres -host = localhost:5432 - -[user] -dbid = 1 -name = pgtest -auth_type = scram -password = test123 -max_conn = 20 -max_clients = 200 -``` +make sure all the poolers have the same max db connection setting + +#### pgpool -#### pgbouncer Configuration +Edit `config/pgpool.ini` with db and credential -Edit `config/pgbouncer.ini` and `config/userlist.txt` with your credentials. +#### pgbouncer +Edit `config/pgbouncer.ini` and `config/userlist.txt` -#### pgcat Configuration +#### pgcat -Edit `pgcat.toml` with test db and credentials. +Edit `pgcat.toml` with test db and credentials ### 3. Start Poolers