Skip to content

Repository files navigation

RadixIP

Go Reference Rust CI License: MIT Go Report Card

One engine. Zero allocations. Nanosecond decisions.

RadixIP is the foundation for infrastructure protectionβ€”IP routing, rate limiting, and access control at memory speed. Drop it in front of any web framework, any Kubernetes cluster, any API gateway.

Go ART: 72.6 ns/lookup - Rust ART: 60.9 ns/lookup - Patricia/RadixNode tree: 177.7-223.4 ns/lookup - FFI: native SIMD support in rust and Go through CGoΒ· SIMD-accelerated

Block attacks. Secure databases. Save $3.6M/year on geolocation APIs. Stop abuse before it hits your app.

🎯 What is RadixIP?

RadixIP is a production-grade IP subnet caching and infrastructure protection engine. It delivers:

  • 72.6 ns concurrent LPM lookups in Go ART β€” zero allocations
  • 60.9 ns concurrent LPM lookups in Rust ART β€” SIMD-accelerated
  • 177.7-223.4 ns Patricia/RadixNode tree lookups for simpler workloads

The Problem: Standard hash maps can't efficiently match IPs against dynamic IPv4 and IPv6 CIDR blocks (/8, /16, /24, /32 , /48, /64, /96, /128) at scale. Database ACLs, API gateways, and edge proxies need sub-microsecond lookups with zero GC pressure.

The Solution: A lock-free binary radix tree with L1 (in-memory) + L2 (Redis look-aside) architecture, enabling:

  • 72.6 ns concurrent LPM lookups in Go ART
  • 60.9 ns concurrent LPM lookups in Rust ART
  • 177.7 ns Rust Patricia/RadixNode-tree lookups and 223.4 ns Go Patricia/RadixNode-tree lookups
  • Zero heap allocations on the Go ART read path; allocation behavior varies by engine variant
  • Instant global sync via Redis Pub/Sub
  • Multi-language support through C-FFI bindings

But RadixIP is more than an IP router. It's the foundation for:

  • βœ… Web framework middleware (Express, Gin, Axum, FastAPI, Django, Fiber)
  • βœ… Distributed rate limiting (Token Bucket, Sliding Window, Fixed Window)
  • βœ… Configurable IP flagging and auto-banning
  • βœ… Kubernetes-native deployment (Operator, Helm, Redis HA)
  • βœ… Unified configuration β€” one file controls everything

Use it for: API gateways Β· database ACLs Β· DDoS mitigation Β· geolocation caching Β· fraud detection Β· rate limiting Β· access control

🎯 Quick Start Example

Block a DDoS attack in 3 lines of code

// 1. Detect attack from 192.168.1.0/24
radixEngine.Insert("192.168.1.0/24", "malicious")

// 2. All future requests from that subnet are blocked instantly
_, found := radixEngine.Match(netip.MustParseAddr("192.168.1.100"))
// found = true β†’ BLOCKED

// 3. Propagate to all nodes via Redis
redis.Publish("security:blocklist", "192.168.1.0/24")

RadixIP Documentation

This is where RadixIP's design decisions are explained in depth β€” not just what the library does, but why it's built this way. The main repo README stays short on purpose; this folder is where the engineering reasoning lives.

Core Concepts

Document notes
Architecture How the L1 (in-process) / L2 (Redis) layers fit together
Sharding Architecture Scaling with sharding
Radix Tree Design The data structure at the core of RadixIP, and why it beats a hashmap or standard trie for this problem
Longest Prefix Match The algorithm every IP router on the internet runs, explained from first principles
IPv4 vs IPv6 How address structure differs, and what that means for caching strategy
Cache Locality Why memory access patterns usually matter more than algorithmic complexity
How Routers Work The real-world context RadixIP borrows from
Benchmark Methodology Exactly how the numbers in the README were produced, so you can reproduce or challenge them
Go SIMD via Rust FFI Why the Go ART Node16 uses a Rust-backed SIMD shared library instead of Go's experimental native SIMD

Reading order

If you're new to networking data structures, read in this order:

  1. How Routers Work β€” the motivating context
  2. Longest Prefix Match β€” the algorithm
  3. Radix Tree Design β€” the data structure that makes it fast
  4. IPv4 vs IPv6 β€” how it changes across protocols
  5. Cache Locality β€” why the implementation is shaped the way it is
  6. Architecture β€” how it's wired into a real system
  7. Benchmark Methodology β€” how to verify all of the above

⚑ SIMD Acceleration (Node16)

The Go ART Node16 node accelerates its 16-slot key scan using SIMD via a thin CGo bridge to a Rust shared library (libnode16_simd_ffi).

Architecture Instruction set How
x86_64 AVX2 β†’ SSE4.1 β†’ scalar Runtime dispatch in Rust
aarch64 NEON Compile-time (mandatory baseline)
other Scalar Pure-Rust fallback

This approach was chosen because Go 1.26's native SIMD is experimental and limited to x86_64. The Rust path gives full multi-arch coverage today.

# Build the shared library, then compile Go with SIMD active:
make build-go-simd

# Run Go ART tests with SIMD:
make test-go-simd

πŸ“– Full rationale, build instructions, and memory-safety notes: docs/guides/go-simd-rust-ffi.md


🧠 Design Goals

RadixIP is designed around a few core principles:

  • ⚑ Zero-allocation ART read path, with allocation behavior tracked per engine variant
  • πŸ”’ Lock-free lookups for highly concurrent workloads
  • 🌳 Efficient longest-prefix matching (LPM)
  • πŸ’Ύ Cache-conscious memory layout
  • 🌐 Cross-language interoperability through C ABI
  • πŸ”„ Distributed cache synchronization via Redis Pub/Sub
  • πŸ“ˆ Predictable performance under heavy read workloads

🌲 Why Radix Trees?

IP subnet matching is fundamentally different from exact-key lookups.

Given an address like:

192.168.1.42

the engine must determine the longest matching prefix:

192.168.0.0/16
192.168.1.0/24
192.168.1.32/27

A traditional hash map can efficiently answer:

"Does this exact key exist?"

It cannot efficiently answer:

"Which CIDR prefix best matches this address?"

This makes radix trees a natural fit for routing tables, ACLs, reverse proxies, firewalls, and API gateways.


πŸ“Š Data Structure Comparison

Structure Exact Match Longest Prefix Match Memory Notes
Hash Map βœ… Excellent ❌ No Medium Best for exact keys
Standard Trie βœ… βœ… High Large number of nodes
Patricia / Radix Tree βœ… βœ… Low Path compression reduces memory
Binary Radix Tree βœ… βœ… Low Well suited for IPv4 bit traversal
Adaptive Radix Tree (ART) βœ… βœ… Very Low Dynamic node sizes, SIMD/cache-friendly, ultra-fast reads

πŸ’‘ Why Radix Trees Work Well

Every IPv4 address is only 32 bits.

Instead of hashing an address, RadixIP walks those bits directly.

IP Address

11000000 10101000 00000001 01101010
β”‚
β”œβ”€β”€ 1
β”‚   β”œβ”€β”€ 1
β”‚   β”‚   β”œβ”€β”€ 0
β”‚   β”‚   └── ...

Traversal is deterministic and naturally supports longest-prefix matching.

No additional indexing structure is required.


πŸ’Ύ Memory Locality

Modern processors spend far more time waiting for memory than executing instructions.

Approximate memory hierarchy:

Memory Typical Latency
CPU Register <1 ns
L1 Cache ~1 ns
L2 Cache ~4 ns
L3 Cache ~10–20 ns
Main Memory ~60–100 ns

Reducing cache misses often has a larger impact than reducing arithmetic operations.

For that reason, RadixIP focuses on:

  • compact node layouts
  • minimizing unnecessary indirection
  • reducing allocations
  • improving spatial locality
  • keeping frequently traversed nodes hot in cache where possible

πŸ”¬ Cache-Conscious Design

Rather than optimizing only algorithmic complexity, RadixIP also considers modern CPU behavior.

Examples include:

  • branch-compressed Patricia nodes
  • compact metadata storage
  • allocation-aware read path, with zero-allocation ART lookups in the Go CI benchmark
  • predictable traversal
  • cache-line friendly structures where practical

These optimizations reduce memory pressure and improve throughput on large routing tables.


πŸ”„ Why Redis?

Redis is not part of the lookup path.

Every lookup is performed entirely inside the local in-memory radix tree.

Redis is used only for:

  • distributing subnet updates
  • cache invalidation
  • Pub/Sub synchronization
  • persistence coordination

The request path remains:


Incoming Request
        β”‚
        β–Ό
 Local Radix Tree
        β”‚
        β–Ό
 Decision

Redis is only consulted when routing information changes.


βš™οΈ Complexity

Operation Complexity
Insert O(prefix length)
Delete O(prefix length)
Lookup O(address bits)
Memory O(number of prefixes)

Since IPv4 addresses are only 32 bits, lookup time is effectively bounded by a small constant.


πŸ“ˆ Benchmark Methodology

Benchmarks are executed by the GitHub Actions Benchmarks workflow on ubuntu-latest.

Current CI runner details from the Go benchmark log:

  • OS/arch: linux/amd64
  • CPU: INTEL(R) XEON(R) PLATINUM 8573C
  • Go command: go test -run=NONE -bench=. -benchmem -count=10 -v -tags simd_cgo ./...
  • Rust command: cargo bench --bench lookup_bench -- --output-format bencher

The benchmark source mapping is:

Runtime Source Benchmark functions
Rust lib/rust/benches/lookup_bench.rs bench_insert, bench_concurrent_lookup_uncompressed, bench_concurrent_lookup_compressed, bench_concurrent_lookup_art
Go engine lib/go/engine_test.go BenchmarkInsert_*, BenchmarkLookup_Hit_*, BenchmarkLookup_Miss_*, BenchmarkConcurrent_Lookup_*, Benchmark*_ART_*
Go ART tree lib/go/art/tree_test.go BenchmarkTree_Insert, BenchmarkTree_Match_Hit, BenchmarkTree_Match_Miss
Go integration tests lib/go/tests/engine_test.go Functional tests only; no benchmark functions

Rust Criterion reports whole-batch times for these benchmarks. Insert results below divide ns/iter by 5,000 prefixes, and Rust concurrent lookup results divide ns/iter by 4 * 25,000 = 100,000 lookups. Go batched hit/miss lookup results divide ns/op by the loop size in the benchmark source; Go RunParallel concurrent results are already per lookup.

Results should be interpreted as measurements for the tested hardware and compiler versions rather than universal performance guarantees.


πŸ“Š CI Benchmark Results

Representative CI results comparing three different tree families:

Tree family Rust implementation Go implementation What it measures
Binary trie NormalTrieNode NormalTrieNode Uncompressed bit-by-bit trie traversal
Patricia / RadixNode tree NormalRadixNode NormalRadixNode Compressed Patricia/radix traversal using RadixNode variants
Adaptive Radix Tree (ART) EngineVariant::ART NewARTEngineAdapter Adaptive node-size tree optimized for dense lookup paths
Runtime Structure Insert batch Insert / prefix Concurrent lookup / op Source benchmark
Rust Binary trie (NormalTrieNode) 25,275,481 ns / 5k 5,055 ns 728.6 ns bench_insert, bench_concurrent_lookup_uncompressed
Rust Patricia/RadixNode tree (NormalRadixNode) 11,754,339 ns / 5k 2,351 ns 177.7 ns bench_insert, bench_concurrent_lookup_compressed
Rust ART (EngineVariant::ART) 761,077 ns / 5k 152.2 ns 60.9 ns bench_insert, bench_concurrent_lookup_art
Go Binary trie (NormalTrieNode) 49,013,688 ns / 5k 9,803 ns 118.7 ns BenchmarkInsert_Uncompressed_5k_Normal, BenchmarkConcurrent_Lookup_Uncompressed_Normal
Go Patricia/RadixNode tree (NormalRadixNode) 12,021,442 ns / 5k 2,404 ns 223.4 ns BenchmarkInsert_Compressed_5k_Normal, BenchmarkConcurrent_Lookup_Compressed_Normal
Go ART (NewARTEngineAdapter) 2,393,602 ns / 10k 239.4 ns 72.6 ns BenchmarkInsert_ART_10k, BenchmarkConcurrent_Lookup_ART_50k

Sequential Go lookup batches from lib/go/engine_test.go:

Structure Hit workload Hit / lookup Miss workload Miss / lookup Allocations
Binary trie (NormalTrieNode) 1,717,544 ns / 25k 68.7 ns 1,726,589 ns / 25k 69.1 ns 24 B/op, 1 alloc/lookup
Patricia/RadixNode tree (NormalRadixNode) 8,349,666 ns / 50k 167.0 ns 3,077,318 ns / 50k 61.5 ns 24 B/op, 1 alloc/lookup
ART (NewARTEngineAdapter) 1,347,823 ns / 50k 27.0 ns 1,041,880 ns / 50k 20.8 ns 0 B/op, 0 allocs

Key takeaways from this CI run:

  • Rust ART is the fastest concurrent lookup path measured here at 60.9 ns/lookup.
  • Go ART is close at 72.6 ns/lookup and has 0 B/op on ART lookup benchmarks.
  • Patricia/RadixNode-tree insert is about 2.1x faster than Rust binary trie for the normal Rust nodes, and ART insert is about 15.4x faster than the Rust Patricia/RadixNode tree in the measured insert workload.
  • The old ~45 ns Go and ~12 ns Rust headline numbers are no longer claimed by this README; the tables above are derived directly from the current CI logs.

See the CI artifacts for complete benchmark logs and hardware information.


🎯 Why It Matters

Algorithmic complexity is only part of the performance story.

On modern processors, memory access patterns frequently dominate execution time.

A well-designed data structure not only performs fewer operationsβ€”it performs them in a way that works with the processor's cache hierarchy rather than against it.

For networking, routing, and access-control workloads, that difference is often more important than asymptotic complexity alone.

πŸ—οΈ Architecture

πŸ”„ The Hybrid L1/L2 Pipeline

Redis is completely removed from the critical lookup path. Every validation runs locally inside the application's process memory.

                                    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                                    β”‚           API Gateway / Proxy Layer          β”‚
                                    β”‚    HTTP β€’ gRPC β€’ REST β€’ Load Balancer        β”‚
                                    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                                       β”‚
                                                       β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                               L1: RadixIP Engine                                                   β”‚
β”‚                                                                                                    β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”   β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”   β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                    β”‚
β”‚  β”‚ Go Module    β”‚   β”‚ Rust Crate   β”‚   β”‚ C ABI / FFI Bindings                 β”‚                    β”‚
β”‚  β”‚ radixip-go   β”‚   β”‚ radixip-rs   β”‚   β”‚ Python β€’ Node.js β€’ C/C++ β€’ Zig       β”‚                    β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜   β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜   β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                    β”‚
β”‚                                                                                                    β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”‚
β”‚  β”‚ Lock-Free Binary Radix Tree                                                                  β”‚  β”‚
β”‚  β”‚ β€’ Allocation-aware read path                                                                 β”‚  β”‚
β”‚  β”‚ β€’ Branch-compressed Patricia trie                                                            β”‚  β”‚
β”‚  β”‚ β€’ Atomic pointer traversal                                                                   β”‚  β”‚
β”‚  β”‚ β€’ Read-Copy-Update (RCU) friendly                                                            β”‚  β”‚
β”‚  β”‚ β€’ Cache-aware node layout                                                                    β”‚  β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β”‚
β”‚                                                                                                    β”‚
β”‚  CPU Cache Optimizations                                                                          β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”‚
β”‚  β”‚ L1 Cache β”‚ Hot traversal nodes β”‚ Prefetched prefixes β”‚ Pointer locality                     β”‚  β”‚
β”‚  β”‚ L2 Cache β”‚ Frequently accessed subtrees β”‚ Branch predictor friendly                         β”‚  β”‚
β”‚  β”‚ L3 Cache β”‚ Shared radix segments across worker threads                                       β”‚  β”‚
β”‚  β”‚                                                                                              β”‚  β”‚
β”‚  β”‚ β€’ Cache-line aligned structures (64-byte alignment)                                          β”‚  β”‚
β”‚  β”‚ β€’ False-sharing avoidance                                                                     β”‚  β”‚
β”‚  β”‚ β€’ NUMA-aware memory allocation                                                                β”‚  β”‚
β”‚  β”‚ β€’ Software prefetching where beneficial                                                       β”‚  β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                              β”‚
                                              β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                                 L2: Redis Cache                                                   β”‚
β”‚                                                                                                    β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”‚
β”‚  β”‚ β€’ Subnet β†’ Metadata mappings                                                                  β”‚  β”‚
β”‚  β”‚ β€’ Distributed cache                                                                           β”‚  β”‚
β”‚  β”‚ β€’ Pub/Sub instant invalidation                                                                β”‚  β”‚
β”‚  β”‚ β€’ Persistent backing store                                                                    β”‚  β”‚
β”‚  β”‚ β€’ High availability                                                                           β”‚  β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                              β”‚
                                              β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                               Persistent Storage                                                  β”‚
β”‚                                                                                                    β”‚
β”‚ β€’ PostgreSQL                                                                                      β”‚
β”‚ β€’ MySQL                                                                                           β”‚
β”‚ β€’ SQLite                                                                                          β”‚
β”‚ β€’ Custom IP metadata providers                                                                    β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Memory Layout

                    CPU
                     β”‚
          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
          β”‚          β”‚          β”‚
         L1         L2         L3
      (32KB)     (512KB)    (Shared)
          β”‚          β”‚          β”‚
          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                     β”‚
      Cache-Line Optimized Radix Nodes
                     β”‚
      β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
      β”‚ Prefix                       β”‚
      β”‚ Left Pointer                 β”‚
      β”‚ Right Pointer                β”‚
      β”‚ Metadata Pointer             β”‚
      β”‚ Flags                        β”‚
      β”‚ Padding (64-byte aligned)    β”‚
      β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                     β”‚
             Main Memory (DRAM)

Performance Strategy

Layer Purpose Latency Target
L1 CPU Cache Hot radix nodes ~1 ns
L2 CPU Cache Frequently traversed branches ~4 ns
L3 CPU Cache Shared worker data ~12 ns
DRAM Cold nodes 60–100 ns
Redis Distributed metadata cache <1 ms
Database Persistent storage 5–20 ms

πŸ› οΈ Production Use Cases

1. Database Security & Dynamic ACLs πŸ›‘οΈ

Protect databases (PostgreSQL, MySQL, MongoDB) by validating client IPs against dynamic whitelists at the proxy layerβ€”before expensive authentication handshakes.

Why RadixIP? Standard firewall rule updates take seconds; RadixIP propagates new ACLs in milliseconds via Redis Pub/Sub.

2. High-Throughput API Gateways 🌐

Enforce enterprise security boundaries by filtering inbound requests against thousands of partner subnets (/16, /24) with zero perceivable overhead.

The Edge: Lock-free reads mean no mutex contention, even at 100k+ RPS.

3. Distributed DDoS Mitigation πŸ”₯

When an attack is detected from an offending IP block, inject the subnet into Redis. All running instances pull the block into their local Radix tree instantly, dropping malicious traffic at the edge.

The Result: Block entire attack vectors in milliseconds, not minutes.

4. Multi-Tenant Infrastructure Platforms ☁️

Allow tenants to define custom IP whitelists for their isolated environments. RadixIP handles per-tenant subnet matching at memory speeds.

The Scale: Handle thousands of tenant-specific ACLs simultaneously.

5. Real Geolocation API Cost Killer πŸ’°

Geolocation APIs are expensive.

Commercial cloud geolocation API billing models scale linearly with lookups, quickly growing to massive monthly operational expenses at scale. RadixIP operates an intelligent hierarchical edge cache structure:

At scale, they can cost $100,000+/month.

RadixIP solves this with intelligent IP caching:

[Incoming Request] ──> L1: Local Radix (memory-speed, FREE) ──[Hit]──> Return Metadata
                             β”‚
                          [Miss]
                             β–Ό
                     L2: Distributed Redis (1ms, FREE) ──[Hit]──> Hydrate L1 & Return
                             β”‚
                          [Miss]
                             β–Ό
                     L3: External Geo API ($$$) ──> Commit to Redis & Hydrate Tree

The Cache Hierarchy

  1. L1: RadixIP (local memory-speed lookup, FREE)

    • Caches exact IPs
    • Caches /24, /16 subnets
    • Caches ASN and country blocks
  2. L2: Redis (1-5ms, FREE)

    • Shared across nodes
    • TTL 24-72 hours
  3. L3: Geo API (10-100ms, $$$)

    • Only on cache miss
    • 90%+ request reduction

Caching Strategy & Cost Savings

By caching both individual IPs and entire structural subnet masks locally, RadixIP can eliminate up to 90%+ of external network lookups.

Inbound Metrics

Traffic Volume Traditional Costs With RadixIP Cache Strategy Monthly Opex Savings
1,000,000 reqs / day $100 / day $10 / day 90% Savings
10,000,000 reqs / day $1,000 / day $100 / day 90% Savings
100,000,000 reqs / day $10,000 / day $1,000 / day Annualized: ~$3.2M saved

πŸ“Š Annualized Savings: ~$3.2M

Key Insight: RadixIP's intelligent subnet-aware caching reduces external lookup costs by 90% across all traffic volumes, delivering predictable and scalable cost savings.

Why It Works

  • 80%+ of IPs are repeat visitors β†’ cache hit
  • Subnet caching β†’ entire blocks cached at once
  • Local memory-speed lookups β†’ no external API call on cache hit
  • Automatic TTL β†’ fresh data when needed

6. Rate Limiting & Abuse Prevention ⚑

Protect APIs from brute force attacks, scraping, and abuse. Define per-IP, per-API-key, or per-route limits with sliding window, token bucket, or fixed window algorithms. Auto-flag and ban IPs that exceed thresholds.

The Result: Stop bad actors before they reach your application.


πŸ“Š Performance Benchmarks

The current headline benchmark is the CI-measured concurrent lookup path:

Runtime Fastest measured structure Latency / lookup Throughput Allocations
Rust (radixip-rs) ART 60.9 ns 16.4M ops/sec not reported by Criterion
Go (radixip-go) ART 72.6 ns 13.8M ops/sec 0 B/op
Rust (radixip-rs) Patricia / RadixNode tree 177.7 ns 5.6M ops/sec not reported by Criterion
Go (radixip-go) Patricia / RadixNode tree 223.4 ns 4.5M ops/sec 24 B/op, 1 alloc/op

The fastest structure is not the same as the most general-purpose structure for every workload. The binary trie and Patricia/RadixNode tree implementations remain useful as simple, predictable LPM baselines; ART is the current high-throughput read path.

Why ART is leading in this run:

  • compact adaptive node sizes
  • fewer traversal steps for dense key ranges
  • SIMD-backed Node16 acceleration in the Go path
  • zero-allocation Go ART lookups

Audit the raw benchmark logs: Download from CI artifacts

πŸš€ Getting Started

Clone the repository

git clone https://github.com/Mwangi-Derrick/radixip.git
cd radixip

Generate the dataset

python scripts/generate_mock_data.py --subnets 10000 --lookups 100000

Run Go benchmarks

cd lib/go
go test -run=NONE -bench=. -benchmem -count=10 -v -tags simd_cgo ./...

Run Rust benchmarks

cd lib/rust
cargo bench --bench lookup_bench -- --output-format bencher

Use the Go library

import (
    "net"

    radixip "github.com/Mwangi-Derrick/radixip/lib/go"
)

func main() {
    engine := radixip.NewEngineWrapperWithTree(
        radixip.EngineConcurrent,
        radixip.NormalRadixNode,
        true,
    )

    _, prefix, _ := net.ParseCIDR("192.168.0.0/16")
    _ = engine.Insert(prefix, radixip.Metadata{
        Value: "allow",
        Attributes: map[string]string{
            "region": "Nairobi",
            "isp":    "Safaricom",
        },
    })

    result := engine.Lookup(net.ParseIP("192.168.1.100"))
    // result != nil; result.Value == "allow"
}

Use the Rust crate

use radixip_rs::Engine;

fn main() {
    let mut engine = Engine::new();
    engine.insert("192.168.0.0/16", metadata);
    
    if let Some(result) = engine.match_ip("192.168.1.100") {
        // result contains the metadata
    }
}

Use the FFI layer (C-compatible)

#include "radixip_rs.h"

RadixEngine* engine = radix_engine_new();
radix_engine_insert(engine, "192.168.0.0/16", metadata);
bool matched = radix_engine_match(engine, "192.168.1.100");
radix_engine_free(engine);

⚑ Quick Install

Go

go get github.com/Mwangi-Derrick/radixip/lib/go

Rust

// add to Cargo.toml
[dependencies]
radixip-rs = "0.1.0"

C/C++ (FFI)

Download pre-built shared library

curl -LO https://github.com/Mwangi-Derrick/radixip/releases/latest/libradixip.so

πŸ”„ CI/CD Pipeline

Every commit to main triggers our continuous benchmarking pipeline:

  1. Build the Rust SIMD FFI shared library used by Go's SIMD CGo path
  2. Run Go benchmarks (go test -run=NONE -bench=. -benchmem -count=10 -v -tags simd_cgo ./...)
  3. Run Rust benchmarks (cargo bench --bench lookup_bench -- --output-format bencher)
  4. Upload raw performance logs as artifacts
  5. Fail if performance regresses beyond 5%

View results: GitHub Actions tab

Download artifacts: Raw benchmark logs available for auditing.

🌲 Tree Type Selection

RadixIP provides three routing tree implementations. Choose based on your workload:

UncompressedTree (Binary Trie) CompressedTree (Binary Patricia Tree) Adaptive Radix Tree (ART)
Write throughput ⚑ Fastest (no splitting) 🟑 Moderate 🟑 Moderate (node upgrades)
Read throughput 🟑 Good ⚑ Fast ⚑⚑ Fastest (SIMD / cache-aligned)
Memory (500K routes) ~4–8 MB ~1–2 MB ~1.5–3 MB (dynamic node sizes)
Best for BGP control plane Packet forwarding / FIB Extremely high-performance routing
// Rust β€” choose at construction time, zero runtime overhead
let control = StandardEngine::new(UncompressedTree::new(NodeVariant::NormalTrieNode));
let fib     = StandardEngine::new(CompressedTree::new(NodeVariant::NormalRadixNode));
// Go β€” same API, different tree
control := NewStandardEngine(NewUncompressedTree(NodeNormal))
fib     := NewStandardEngine(NewCompressedTree(NodeNormal))
// Or via EngineWrapper with compressed flag
controlPlane := NewEngineWrapperWithTree(EngineStandard, NormalTrieNode, false) // uncompressed
dataPlane    := NewEngineWrapperWithTree(EngineConcurrent, AtomicRadixNode, true)  // compressed

See ARCHITECTURE.md for the full design rationale, hybrid Redis architecture, and performance reference.

πŸ—ΊοΈ Roadmap

Core Engine

  • Go implementation with lock-free reads
  • Rust port with zero-cost abstractions
  • C-FFI bindings for multi-language support
  • CI benchmarking pipeline
  • Uncompressed binary trie β€” control-plane optimized, O(prefix_len) writes
  • Compressed Patricia trie β€” data-plane optimized, O(k) reads, 4Γ— memory savings
  • Generic engines β€” any engine can use any tree via StandardEngine<T: RouteTree>
  • Redis state bus β€” boot-load, cache hydration, Pub/Sub sync
  • IPv6 full support (Patricia trie path)
  • Python bindings via PyO3
  • Node.js bindings via N-API
  • gRPC service layer
  • Prometheus metrics integration
  • Lock-free CompressedTree (CAS-based node splitting)
  • (Adaptive Radix Tree) ART implementation in Go & Rust
    • Node4 (1-4 children, ~36 bytes)
    • Node16 (5-16 children, SIMD support)
    • Node48 (17-48 children, index array)
    • Node256 (49-256 children, direct array)
    • Auto-upgrade/downgrade logic
    • SIMD acceleration (x86 SSE / ARM NEON)
    • Zero-alloc lookups
    • Lock-free concurrency support

Multi-language bindings

  • Publish v1.0.0 release of the rust crate to crates.io
  • Publish the NAPI-RS node bindings to npm
  • Publish the PyO3 bindings to PYPI

Configuration Engine πŸ”¨

  • YAML/TOML config parser
  • Hot-reload without restart
  • Per-route overrides
  • Validation + schema

Protection Layer πŸ”¨

  • Web framework middleware (Express, Gin, Axum, Actix, FastAPI, Django, Fiber, etc.)
  • Distributed rate limiting (Token Bucket, Sliding Window, Fixed Window)
  • Configurable IP flagging and auto-banning
  • Unified configuration format

Kubernetes & Operations πŸ”¨

  • Kubernetes Operator for automated deployment
  • Redis HA failover support
  • Prometheus metrics integration
  • gRPC service layer
  • Helm charts for one-command install

🀝 Contributing

We welcome contributions! Here's how to get started:

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing)
  3. Commit your changes (git commit -m 'Add amazing feature')
  4. Push to the branch (git push origin feature/amazing)
  5. Open a Pull Request

Guidelines:

  • Run benchmarks before/after to prove no regression
  • Update documentation for any API changes
  • Add tests for new functionality
  • Keep performance as the #1 priority

πŸ“„Distribution Registries

Primary Source Engine: github.com/Mwangi-Derrick/radixip

Infrastructure Target Mirror: github.com/resplix/radixip

πŸ“„ License

MIT License - See LICENSE for details.


Built with ❀️ by Derrick Mwangi and the team at Resplix.

High-performance IP subnet matching for modern infrastructure.

⭐ Star this repo if you find it useful!

About

High-performance IP subnet caching engine with Go & Rust implementations. Zero-allocation LPM lookups, Redis sync, and C-FFI bindings for database security, API gateways, and DDoS mitigation.

Topics

Resources

Contributing

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages