Skip to content

Feat/rust output length guard clean - #4104

Closed
msureshkumar88 wants to merge 29 commits into
mainfrom
feat/rust-output-length-guard-clean
Closed

Feat/rust output length guard clean#4104
msureshkumar88 wants to merge 29 commits into
mainfrom
feat/rust-output-length-guard-clean

Conversation

@msureshkumar88

@msureshkumar88 msureshkumar88 commented Apr 9, 2026

Copy link
Copy Markdown
Collaborator

🔗 Related Issue

Closes #

📝 Summary

Recreation of PR #3926 in a clean branch with updated configuration to match the Python version of the output_length_guard plugin.

This PR introduces a Rust-accelerated implementation of the output_length_guard plugin that provides significant performance improvements while maintaining full feature parity with the Python implementation.

Key Changes:

  • Rust-based output length guard plugin with PyO3 bindings
  • Configuration updated to match Python plugin behavior (max_chars, strategy, word_boundary, etc.)
  • Comprehensive test suite validating all features (10/10 tests passing)
  • Performance improvement: ~0.06ms average processing time for 23K character inputs
  • Automatic fallback to Python for MCP content structures
  • Full support for truncate/block strategies, word boundaries, nested structures

🏷️ Type of Change

  • Bug fix
  • Feature / Enhancement
  • Documentation
  • Refactor
  • Chore (deps, CI, tooling)
  • Other (describe below)

🧪 Verification

Check Command Status
Lint suite make lint ✅
Unit tests make test ✅
Coverage ≥ 80% make coverage ✅
Rust checks make rust-check ✅
Plugin tests python test_rust_output_length_guard.py ✅ 10/10

Test Results:

  • All 10 comprehensive tests passed
  • Rust engine availability confirmed
  • Character truncation, blocking, word boundaries validated
  • List/dict/nested structure processing verified
  • Performance benchmark: 0.0595ms avg (100 iterations, 23K chars)
  • MCP content fallback to Python working correctly

✅ Checklist

  • Code formatted (make black isort pre-commit)
  • Tests added/updated for changes
  • Documentation updated (if applicable)
  • No secrets or credentials committed
  • Rust code formatted and linted (cargo fmt, cargo clippy)
  • Plugin configuration matches Python version

📓 Notes

Performance Characteristics:

  • Average processing time: 0.0595ms per call
  • Test input: ~23,000 characters
  • 100 iterations benchmark completed successfully
  • Rust acceleration confirmed on all applicable operations

Configuration Alignment:
The plugin configuration has been updated to exactly match the Python version's behavior:

  • max_chars: Character limit enforcement
  • strategy: "truncate" or "block" modes
  • word_boundary: Smart truncation at word boundaries
  • min_length: Minimum content length enforcement
  • Automatic Python fallback for MCP content structures

Clean Branch Recreation:
This is a clean recreation of PR #3926 with all configuration issues resolved and comprehensive testing in place.

Suresh Kumar Moharajan and others added 28 commits March 24, 2026 17:44
Suresh Kumar Moharajan @msureshkumar88
fix content truncate issues and added support for list dict and neste…
bb8d324 
…d dicts

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan @msureshkumar88
add support to blocking list, dict and nested dict
9d7e474 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan @msureshkumar88
add support to truncate at word boundaries to avoid mid-word cuts
5e38956 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan @msureshkumar88
implement token size based output tokens
393a247 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan @msureshkumar88
add allow limit modes config to segregate character and token
502c881 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan @msureshkumar88
add test mcp tools for list, dicts, and nested dicts
b5c4e65 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan @msureshkumar88
clear debug logs
5214b19 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan @msureshkumar88
add comprehensive logging
784c093 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan @msureshkumar88
add exceptions handling
b81e939 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan
reformatting code
d6686ef 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan
add critical componant type supports
34d674f 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan
fix(output_length_guard): treat 0 as disabled for max limits
7c292ac 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan
add unittests
66c4370 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan
add configuraton, readme, reformatting
df367bc 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan
delete redundant test file
1b63aa3 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan
fix formatting
5e32976 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan
fix linting issues
c7dc7d5 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan
fix syntax error
fd80bba 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan
fix syntax issue
35f0cbc 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
@gandhipratik203
feat(plugin): add Rust acceleration for output length guard
ab0c747 
Add a PyO3-based Rust implementation of the output length guard's core
processing logic (truncation, word-boundary search, token estimation,
binary search cut-point, recursive container traversal). The Python
plugin auto-detects the Rust engine at init and delegates str, list,
dict, and nested structure processing to it, falling back to Python
when unavailable or for MCP content dicts that need structuredContent
priority logic.

- 47 Rust unit tests mirroring the Python test contract
- 331 existing Python tests pass with Rust engine active
- Clean clippy (-D warnings) and rustfmt

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
@gandhipratik203
feat(plugin): add performance comparison script for Rust output lengt…
ff1cae9 
…h guard

Benchmarks Python vs Rust across 7 scenario groups: single string
truncation, token-mode binary search, word-boundary truncation, list
processing, nested dict traversal, block-mode violation detection, and
under-limit passthrough. Rust shows 3-10x speedup on container
traversal (lists, nested dicts, passthrough) while single string
truncation is FFI-bound.

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
@gandhipratik203
perf(plugin): optimize Rust output length guard with O(limit) char co…
768e65c 
…unting

Three key optimizations that eliminate O(n) full-string scans:

1. count_chars_capped(): early-exit char counting that stops once the
   limit is exceeded, plus byte-length fast path for ASCII strings.
   Turns O(n) into O(min(n, limit)).

2. byte_offset_of_char() + direct slicing: replaces
   .chars().take(n).collect() with &value[..byte_offset] for zero-copy
   truncation.

3. PyString pre-check in process_container(): uses Python's O(1)
   str.__len__() to skip string extraction entirely for under-limit
   strings in truncate mode.

Results: 1MB string truncation dropped from 31us to 2.4us (constant
regardless of input size). Passthrough improved to 12x faster.
Deep/wide nested structures remain 5-7x faster.

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
@gandhipratik203
perf(plugin): use zero-copy PyString::to_str() in process_container
42d5e9c 
Replace extract::<String>() (full copy) with PyString::cast() +
to_str() (zero-copy borrow) for the string leaf path. Also skip
is_numeric_string() for strings > 50 bytes, and extend the O(1)
pre-check to both truncate and block modes.

Results vs previous commit:
  - Deep nested dict:  5.4x → 7.1x faster
  - Wide nested dict:  6.2x → 8.4x faster
  - List passthrough:  9.7x → 13.6x faster
  - Block mode 10KB:   4.5x → 5.0x faster
  - All 331 Python tests + 47 Rust tests pass

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
@gandhipratik203
perf(plugin): batch list processing and pre-sized String allocation
fefd3ac 
Two optimizations informed by the rate limiter PR (#3809) patterns:

1. Batch list processing: for all-string lists in truncate mode, extract
   all &str borrows in one pass, process in a tight Rust loop, build
   output PyList in a single pass. Better cache locality and avoids
   per-item path string formatting and interleaved append calls.

2. Pre-sized String::with_capacity(): eliminate reallocation during
   truncation by pre-computing body + ellipsis size.

Results:
  - Short list passthrough: 13.6x → 18.9x faster
  - List 10x10KB:           2.6x → 3.0x faster
  - Deep nested dict:       7.1x → 7.0x faster (stable)
  - Wide nested dict:       8.4x → 8.5x faster (stable)
  - 331 Python tests + 47 Rust tests pass
Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
Suresh Kumar Moharajan
improve performance for Single String Truncation List of 10 x 1KB Str…
a994f2c 
…ings Shallow Nested Dict and fix testcases

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan
fix performance issue and match rast version to python version
4c5a33a 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan
fix dictionary single key not setting in the content
a9c97de 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Suresh Kumar Moharajan
fix request blocking
a29a68f 
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
@lucarlig

lucarlig commented Apr 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

should this be moved to https://github.com/IBM/cpex-plugins? altough we have no good testing CI yet there

@msureshkumar88
msureshkumar88 force-pushed the feat/rust-output-length-guard-clean branch from 02d7261 to 8ba4faf Compare April 9, 2026 14:02
@jonpspri

jonpspri commented Apr 9, 2026

Copy link
Copy Markdown
Collaborator

@lucarlig @msureshkumar88 Please close here and move work to https://github.com/IBM/cpex-plugins . Let's introduce whatever CI with think is necessary as a part of this move. I'm switching to draft until we confirm the move.

@jonpspri
jonpspri marked this pull request as draft April 9, 2026 19:12
@msureshkumar88

msureshkumar88 commented Apr 10, 2026

Copy link
Copy Markdown
Collaborator Author

closing this pr as it is a duplicate of #3926

@msureshkumar88 msureshkumar88 mentioned this pull request Apr 10, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@crivetimihai crivetimihai Awaiting requested review from crivetimihai crivetimihai is a code owner
@kevalmahajan kevalmahajan Awaiting requested review from kevalmahajan
@madhav165 madhav165 Awaiting requested review from madhav165
@lucarlig lucarlig Awaiting requested review from lucarlig
@dima-zakharov dima-zakharov Awaiting requested review from dima-zakharov
@brian-hussey brian-hussey Awaiting requested review from brian-hussey brian-hussey will be requested when the pull request is marked ready for review brian-hussey is a code owner
@Lang-Akshay Lang-Akshay Awaiting requested review from Lang-Akshay Lang-Akshay will be requested when the pull request is marked ready for review Lang-Akshay is a code owner
@ja8zyjits ja8zyjits Awaiting requested review from ja8zyjits ja8zyjits will be requested when the pull request is marked ready for review ja8zyjits is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@msureshkumar88 @lucarlig @jonpspri @gandhipratik203