Skip to content

Refactor/embedding service cleanup - #44

Merged
ktamaral merged 6 commits into
mainfrom
refactor/embedding-service-cleanup
May 28, 2026
Merged

Refactor/embedding service cleanup#44
ktamaral merged 6 commits into
mainfrom
refactor/embedding-service-cleanup

Conversation

@hinxcode

@hinxcode hinxcode commented May 8, 2026

Copy link
Copy Markdown
Owner

Description

Make the offline embedding script use the new BaseEmbeddingService (it was bypassing it), stop the factory
from loading a model at import time, and fix SigLIP's scoring formula.

Motivation and Context

  1. Abstraction not used in the offline path. generate_embeddings.py reached into service.model / .processor directly, so adding a new model would require touching the batch script. Now it calls service.encode_image().

  2. Module-level singleton in embedding_service_factory. Importing the module triggered model download and collided with the existing vector-store embedding_service singleton.

  3. SigLIP scoring used CLIP's formula (logit_scale * sim) instead of SigLIP's (sigmoid(logit_scale * sim + logit_bias)).

  4. transformers>=5.0.0 forced CLIP-only users to upgrade. SigLIP needs only >=4.42.0.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Code refactoring (no functional changes)
  • Performance improvement
  • Research contribution (new models, evaluation methods, etc.)
  • Other (dependency cleanup):

Component(s) Affected

  • Backend (Python/FastAPI)
  • Frontend - Photographs
  • Frontend - Maps
  • Frontend - Documents
  • CLIP/ML models
  • Configuration
  • Documentation
  • Tests
  • Build/deployment

Changes Made

  • Use BaseEmbeddingService.encode_image() in offline pipeline; drop src/models/clip/ layer
  • Remove module-level singleton from factory; search.py owns its instance
  • Add transform_score abstract method; SigLIP applies sigmoid(logit_scale * sim + logit_bias)
  • EmbeddingService.search() takes optional score_transform callable instead of scalar logit_scale
  • Stop re-exporting embedding_service from services/__init__.py (avoids name collision)
  • transformers>=4.42.0; restore config.json default to clip + cuda
  • Add pyproject.toml with black + isort config
  • Remove dead or "clip" fallback in factory
  • Update test_search_with_logit_scaletest_search_with_score_transform to match new callable API
  • Bump CI to Python 3.12 and remove continue-on-error from the pytest job (was masking failures)

Testing

How Has This Been Tested?

  • pytest tests/ passes
  • Manual: text and image search via FastAPI on both CLIP and SigLIP

Test Configuration

  • Collection type tested: photographs
  • Python version: 3.12
  • Node version: 22
  • OS: macOS
  • CLIP model: openai/clip-vit-base-patch32
  • SigLIP model: google/siglip-base-patch16-224

Screenshots (if applicable)

Checklist

Code Quality

  • My code follows the project's coding standards
  • I have run black . and isort . on Python code
  • I have run npm run lint on frontend code (if applicable)
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings or errors

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have tested this locally with actual data

Documentation

  • I have updated the documentation accordingly
  • I have updated the README if needed
  • I have added docstrings to new functions/classes
  • I have updated config.json documentation if config changes were made

Dependencies

  • I have updated requirements.txt (if Python dependencies changed)
  • I have updated package.json (if Node dependencies changed)
  • I have documented any new configuration options

Research (if applicable)

  • I have included references to relevant papers or research
  • I have shared evaluation results or benchmarks
  • I have included information about datasets used
  • I have documented model training procedures

Breaking Changes

Two internal-API changes; both updated in this PR:

  • EmbeddingService.search(): parameter logit_scale: Optional[float] replaced with score_transform: Optional[Callable[[Tensor], Tensor]].
  • services/__init__.py: no longer re-exports embedding_service (it pointed to the factory's encoder, conflicting with the vector-store singleton). Import from embedding_service_factory directly.

Additional Notes

The transform_score abstraction makes explicit that absolute scores differ across models. Benchmarking should rely on rank-based metrics rather than score thresholds.

Reviewers Checklist (for maintainers)

  • Code quality and style compliance
  • Test coverage adequate
  • Documentation complete
  • No security concerns
  • Performance implications acceptable
  • Breaking changes documented

@hinxcode
hinxcode force-pushed the refactor/embedding-service-cleanup branch from cd7b10d to 3f4cb14 Compare May 8, 2026 09:38
@hinxcode
hinxcode requested a review from ktamaral May 21, 2026 00:12
@ktamaral

Copy link
Copy Markdown
Collaborator

@hinxcode Thank you for making these updates, everything looks good. I was able to generate new embeddings and run the unit tests successfully.

@ktamaral
ktamaral merged commit ba3b651 into main May 28, 2026
6 checks passed
@hinxcode
hinxcode deleted the refactor/embedding-service-cleanup branch July 23, 2026 17:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants