Skip to content

Adding --max_length parameter for embeddings - #4496

Open
przepeck wants to merge 8 commits into
mainfrom
przepeck/embeddings_max_len
Open

Adding --max_length parameter for embeddings#4496
przepeck wants to merge 8 commits into
mainfrom
przepeck/embeddings_max_len

Conversation

@przepeck

@przepeck przepeck commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

🛠 Summary

CVS-187555
Adding --max_length parameter for embeddings to speed up model loading and inference on NPU.

Model loading time:
unset: ~13min
200: 18s

🧪 Checklist

  • Unit tests added.
  • The documentation updated.
  • Change follows security best practices.
    ``

Copilot AI lite review requested due to automatic review settings September 1, 2026 08:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for configuring embeddings max_length (via CLI/graph export and MediaPipe node options) so model context length can be fixed explicitly—intended to improve loading/inference behavior on NPU—along with test coverage for the new setting.

Changes:

  • Add --max_length to embeddings graph-export CLI and propagate it into generated graph pbtxt (max_length node option).
  • Extend embeddings node initialization/servable flow to carry an optional configured max length and apply it during preprocessing.
  • Add/update unit and integration tests plus test graphs/configs covering max-length behavior.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/test/ovmsconfig_test.cpp Extends config parsing tests to cover --max_length for embeddings graph settings.
src/test/graph_export_test.cpp Updates graph export expectations and settings to include max_length.
src/test/embeddingsnode_test.cpp Adds HTTP-level tests verifying requests within/exceeding configured max_length.
src/test/embeddings/graph_ov_max_length.pbtxt Adds a new embeddings graph fixture with max_length set.
src/test/embeddings/config_embeddings.json Registers the new test model/graph in embeddings test config.
src/graph_export/graph_export.cpp Emits max_length into generated embeddings graph templates when configured.
src/graph_export/embeddings_graph_cli_parser.cpp Adds CLI option parsing for --max_length into embeddings graph settings.
src/embeddings/embeddings_servable.hpp Adds optional configured max-length member/plumbing into the servable.
src/embeddings/embeddings_servable.cpp Applies configured max length as an override during preprocessing.
src/embeddings/embeddings_node_initializer.cpp Reads max_length from node options and passes it into the servable.
src/embeddings/embeddings_calculator_ov.proto Extends calculator options proto with optional uint32 max_length.
src/capi_frontend/server_settings.hpp Extends embeddings graph settings struct with maxLength.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/embeddings/embeddings_servable.hpp
Comment thread src/graph_export/embeddings_graph_cli_parser.cpp
Comment thread src/embeddings/embeddings_node_initializer.cpp
@dtrawins

dtrawins commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

looks good. add also documentation and include new parameter in the embeddings demo with NPU

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants