Skip to content

feat(storage): add Milvus vector adapter - #3041

Open
zc277584121 wants to merge 5 commits into
volcengine:mainfrom
zc277584121:feat/milvus-adapter
Open

feat(storage): add Milvus vector adapter#3041
zc277584121 wants to merge 5 commits into
volcengine:mainfrom
zc277584121:feat/milvus-adapter

Conversation

@zc277584121

Copy link
Copy Markdown

Summary

  • Add a Milvus vector database adapter with Milvus Lite defaults and configurable URI, token, database, and consistency level.
  • Register the backend in the VectorDB adapter factory and add configuration/schema support.
  • Document Milvus Lite, self-hosted Milvus, and Zilliz Cloud configuration examples.

Testing

  • uvx ruff format --check openviking/storage/vectordb_adapters/milvus_adapter.py tests/storage/test_milvus_adapter.py openviking_cli/utils/config/vectordb_config.py
  • uvx ruff check openviking/storage/vectordb_adapters/milvus_adapter.py tests/storage/test_milvus_adapter.py openviking/storage/vectordb_adapters/factory.py openviking/storage/vectordb_adapters/__init__.py openviking_cli/utils/config/vectordb_config.py
  • python3.10 -m py_compile openviking/storage/vectordb_adapters/milvus_adapter.py tests/storage/test_milvus_adapter.py openviking_cli/utils/config/vectordb_config.py
  • git diff --check origin/main...HEAD
  • uv lock --check
  • PYTHONPATH=. uv run --no-project --python 3.10 --with pytest --with pydantic --with typing-extensions --with pyyaml --with httpx --with loguru --with 'pymilvus[milvus_lite]' --with apscheduler --with json-repair --with xxhash --with volcengine pytest --confcutdir=tests/storage -o addopts= tests/storage/test_milvus_adapter.py -q

@zc277584121
zc277584121 force-pushed the feat/milvus-adapter branch 2 times, most recently from 782b00c to 4871c23 Compare July 8, 2026 11:53
@zc277584121

Copy link
Copy Markdown
Author

Hi @qin-ctx @ZaynJarvis, could you take a look when you have a chance?

This PR adds a Milvus vector database adapter using the existing CollectionAdapter/factory structure. The branch has been rebased on the latest main and is mergeable again. Focused local checks are passing: py_compile, ruff check, ruff format --check, uv lock --check, and git diff --check.

@zc277584121

Copy link
Copy Markdown
Author

Rebased this branch onto the latest main and resolved the vector-backend conflicts against the current adapter architecture. The retired Qdrant/OpenGauss configuration was not reintroduced.

Additional fixes from server validation:

  • preserve Milvus COSINE similarity scores (higher is better) instead of treating them as distances
  • use a server-compatible 2D metadata vector
  • create and load an AUTOINDEX for the metadata collection, including repair of an existing unindexed metadata collection
  • keep the updated dependency lock and configuration documentation aligned with the current tree

Validation completed:

  • 11 focused Milvus tests passed, including the Milvus Lite create/upsert/filter/search/delete smoke test
  • Zilliz Cloud passed metadata persistence, create, reconnect, upsert, filtered search, count, and delete
  • Cloud smoke collections were verified cleaned up
  • ruff check and ruff format --check passed for all changed Python files
  • uv lock --check and git diff --check passed

A fresh local source build of the latest main currently segfaults while initializing the native vectordb engine (_new_schema) before any Milvus adapter code is reached. The focused Milvus suite was therefore also run with that unrelated native backend isolated; both the Lite and Cloud paths completed successfully.

Signed-off-by: Cheney Zhang <chen.zhang@zilliz.com>
Signed-off-by: Cheney Zhang <chen.zhang@zilliz.com>
Signed-off-by: Cheney Zhang <chen.zhang@zilliz.com>
Signed-off-by: Cheney Zhang <chen.zhang@zilliz.com>
Signed-off-by: Cheney Zhang <chen.zhang@zilliz.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

1 participant