diff --git a/AGENTS.md b/AGENTS.md index c2cbc8e9..a77a6dc7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,6 +15,8 @@ sdks/ python/sdk/ — Python SDK (PyPI: moss), Python 3.10+ javascript/sdk/ — JS/TS SDK (npm: @moss-dev/moss), ESM-only elixir/sdk/ — Elixir SDK (Hex: moss) + go/sdk/ — Go SDK (module: github.com/usemoss/moss/sdks/go/sdk) + ruby/sdk/ — Ruby SDK (RubyGems: moss), Ruby 3.0+ examples/ python/ — Standalone Python usage examples javascript/ — Standalone TS usage examples @@ -223,6 +225,29 @@ mix deps.get mix test ``` +### Ruby SDK (`sdks/ruby/sdk/`) + +```bash +cd sdks/ruby/sdk +bundle install +bundle exec rake test # unit tests (native/E2E tests auto-skip) +bundle exec rubocop # lint (shared config in sdks/ruby/.rubocop.yml) + +# Local search + E2E need the native libmoss runtime (download the c-sdk +# release) and credentials: +# MOSS_LIB_DIR=/path/to/libmoss/lib MOSS_PROJECT_ID=... MOSS_PROJECT_KEY=... +# +# End-to-end validation harness (reads creds from a repo-root .env, auto-fetches +# libmoss): +ruby sdks/ruby/sdk/scripts/validate.rb +``` + +The Ruby SDK is two gems: `sdks/ruby/sdk/` (`moss`, high-level client) over +`sdks/ruby/bindings/` (`moss-core`, FFI bindings over `libmoss`). Like the Go +SDK it links the prebuilt `libmoss` C SDK rather than a per-language native +package, and degrades to a cloud-query fallback / `BindingsUnavailableError` +when `libmoss` is absent. + ## Architecture: Two-Layer Design Every SDK has the same two-layer structure: @@ -236,10 +261,12 @@ SDK layer (pure language, open source) ↓ Native bindings (Rust, pre-compiled, published as separate package) └─ ManageClient / IndexManager — handles embedding, indexing, local search - └─ Imported as: moss-core (Python), @moss-dev/moss-core (JS), moss_core (Elixir) + └─ Imported as: moss-core (Python), @moss-dev/moss-core (JS), moss_core (Elixir), + moss-core (Ruby, FFI over libmoss). The Go and Ruby SDKs link the prebuilt + `libmoss` C SDK directly instead of a per-language Rust package. ``` -The Python `MossClient` in [sdks/python/sdk/src/moss/client/moss_client.py](sdks/python/sdk/src/moss/client/moss_client.py) re-exports types from `moss_core` and wraps `ManageClient` + `IndexManager` from the native layer. The JS SDK follows the same pattern in [sdks/javascript/sdk/src/client/](sdks/javascript/sdk/src/client/). +The Python `MossClient` in [sdks/python/sdk/src/moss/client/moss_client.py](sdks/python/sdk/src/moss/client/moss_client.py) re-exports types from `moss_core` and wraps `ManageClient` + `IndexManager` from the native layer. The JS SDK follows the same pattern in [sdks/javascript/sdk/src/client/](sdks/javascript/sdk/src/client/). The Ruby `Moss::Client` in [sdks/ruby/sdk/lib/moss/client.rb](sdks/ruby/sdk/lib/moss/client.rb) wraps `Moss::Core::ManageClient` + `Moss::Core::IndexManager` from the `moss-core` FFI bindings. **Key invariant:** Mutations (create/add/delete) go to the cloud via `ManageClient`. Queries use the local `IndexManager` when an index is loaded; otherwise fall back to the cloud query API. diff --git a/sdks/ruby/.gitignore b/sdks/ruby/.gitignore new file mode 100644 index 00000000..908903e5 --- /dev/null +++ b/sdks/ruby/.gitignore @@ -0,0 +1,12 @@ +# Vendored native libmoss C SDK (downloaded on demand by scripts/validate.rb) +.libmoss/ + +# Ruby / Bundler artifacts +*.gem +.bundle/ +vendor/bundle/ +Gemfile.lock +coverage/ +.yardoc/ +doc/ +.rspec_status diff --git a/sdks/ruby/.rubocop.yml b/sdks/ruby/.rubocop.yml new file mode 100644 index 00000000..a6a0996a --- /dev/null +++ b/sdks/ruby/.rubocop.yml @@ -0,0 +1,69 @@ +# Shared RuboCop configuration for the Moss Ruby SDK (both the `moss` gem in +# sdk/ and the `moss-core` bindings in bindings/). Each gem has a thin +# .rubocop.yml that inherits from this file. + +AllCops: + TargetRubyVersion: 3.0 + NewCops: enable + SuggestExtensions: false + Exclude: + - ".libmoss/**/*" + - "**/vendor/**/*" + +# Documentation is provided as prose (README) plus inline comments; per-class +# doc comments are not required. +Style/Documentation: + Enabled: false + +# This SDK standardises on double-quoted strings. +Style/StringLiterals: + EnforcedStyle: double_quotes +Style/StringLiteralsInInterpolation: + EnforcedStyle: double_quotes + +# has_index? mirrors the native API name; delete_index/create_index are actions, +# not predicates — the newer predicate-naming cops produce false positives here. +Naming/PredicatePrefix: + Enabled: false +Naming/PredicateMethod: + Enabled: false + +# The validation harness is a linear script, not library code. +Metrics/MethodLength: + Max: 45 + Exclude: + - "**/scripts/**/*" +Metrics/AbcSize: + Max: 45 + Exclude: + - "**/scripts/**/*" +Metrics/CyclomaticComplexity: + Max: 12 +Metrics/PerceivedComplexity: + Max: 12 +Metrics/ClassLength: + Max: 400 +Metrics/ModuleLength: + Max: 250 +Metrics/BlockLength: + Max: 30 + Exclude: + - "**/test/**/*" + - "**/*.gemspec" + +# Test files naturally group several small support classes together. +Style/OneClassPerFile: + Enabled: false + +# Development dependencies are declared in the gemspec (versioned with the gem), +# which is a valid and common convention. +Gemspec/DevelopmentDependencies: + Enabled: false +Metrics/ParameterLists: + Max: 8 + CountKeywordArgs: false + +Layout/LineLength: + Max: 120 + Exclude: + - "**/test/**/*" diff --git a/sdks/ruby/README.md b/sdks/ruby/README.md new file mode 100644 index 00000000..007ce4a1 --- /dev/null +++ b/sdks/ruby/README.md @@ -0,0 +1,60 @@ +# Moss Ruby SDK + +On-device semantic search for Ruby and Rails, powered by the +[Moss](https://docs.moss.dev/docs/start/what-is-moss) runtime. + +This directory contains two gems, following the same two-layer structure as the +other Moss SDKs: + +| Directory | Gem | Role | +| --- | --- | --- | +| [`sdk/`](sdk) | `moss` | Ergonomic, pure-Ruby client — start here | +| [`bindings/`](bindings) | `moss-core` | Native FFI bindings over the `libmoss` C SDK | + +## Getting started + +See [`sdk/README.md`](sdk/README.md) for installation, quick start, metadata +filtering, custom embeddings, and the full API. + +```ruby +require "moss" + +client = Moss::Client.new # creds from MOSS_PROJECT_ID / MOSS_PROJECT_KEY +client.create_index("support-docs", documents) +client.load_index("support-docs") +client.query("support-docs", "how long do refunds take?", top_k: 3) +``` + +## Requirements + +- Ruby >= 3.0 +- The native `libmoss` runtime for local indexing and search — download from the + [`c-sdk-v0.9.0` release](https://github.com/usemoss/moss/releases/tag/c-sdk-v0.9.0) + and point `MOSS_LIB_DIR` at its `lib/` directory. +- Moss project credentials from [moss.dev](https://moss.dev). + +## Layout + +```text +sdks/ruby/ +├── sdk/ # the `moss` gem (high-level client) +│ ├── lib/moss/ # client, models, cloud query fallback, sessions +│ ├── test/ # unit tests + env-gated integration test +│ ├── samples/ # runnable usage examples +│ └── scripts/ # live end-to-end validation harness +└── bindings/ # the `moss-core` gem (FFI over libmoss) + └── lib/moss/core/ +``` + +## Development + +```bash +# high-level SDK +cd sdks/ruby/sdk && bundle install && bundle exec rake test && bundle exec rubocop + +# bindings +cd sdks/ruby/bindings && ruby -Itest -Ilib test/library_test.rb +``` + +Local semantic search, metadata filtering, and E2E tests require `libmoss` and +credentials; they auto-skip gracefully when either is absent. diff --git a/sdks/ruby/bindings/.rubocop.yml b/sdks/ruby/bindings/.rubocop.yml new file mode 100644 index 00000000..fc2019d4 --- /dev/null +++ b/sdks/ruby/bindings/.rubocop.yml @@ -0,0 +1 @@ +inherit_from: ../.rubocop.yml diff --git a/sdks/ruby/bindings/CHANGELOG.md b/sdks/ruby/bindings/CHANGELOG.md new file mode 100644 index 00000000..a3061dea --- /dev/null +++ b/sdks/ruby/bindings/CHANGELOG.md @@ -0,0 +1,15 @@ +# Changelog + +All notable changes to the `moss-core` gem are documented here. + +## [0.9.0] - Unreleased + +### Added + +- Initial release of the Ruby FFI bindings over the `libmoss` C SDK (targets + libmoss `0.9.0`). +- `Moss::Core::ManageClient` — cloud mutations and reads. +- `Moss::Core::IndexManager` — local index load/unload/query/refresh. +- `Moss::Core::Session` — ephemeral in-memory index sessions. +- Lazy library resolution via `MOSS_LIBRARY_PATH` / `MOSS_LIB_DIR`, degrading to + `Moss::Core::BindingsUnavailableError` when `libmoss` is absent. diff --git a/sdks/ruby/bindings/Gemfile b/sdks/ruby/bindings/Gemfile new file mode 100644 index 00000000..ff732f15 --- /dev/null +++ b/sdks/ruby/bindings/Gemfile @@ -0,0 +1,7 @@ +# frozen_string_literal: true + +source "https://rubygems.org" + +gemspec + +gem "rubocop", "~> 1.60", require: false diff --git a/sdks/ruby/bindings/LICENSE b/sdks/ruby/bindings/LICENSE new file mode 100644 index 00000000..372ad0ad --- /dev/null +++ b/sdks/ruby/bindings/LICENSE @@ -0,0 +1,25 @@ +BSD 2-Clause License + +Copyright (c) 2026, Moss Team +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/sdks/ruby/bindings/README.md b/sdks/ruby/bindings/README.md new file mode 100644 index 00000000..18b46340 --- /dev/null +++ b/sdks/ruby/bindings/README.md @@ -0,0 +1,72 @@ +# moss-core — Ruby bindings for libmoss + +`moss-core` wraps the native `libmoss` runtime for Ruby via [FFI](https://github.com/ffi/ffi). + +It mirrors the role of the other language bindings packages in this repository: + +- native runtime access +- local index loading +- local query execution +- cloud-backed manage operations exposed through the native client +- ephemeral in-memory sessions + +Most users should depend on the higher-level [`moss`](../sdk) gem instead of +using these bindings directly. + +## Status + +The bindings attach to `libmoss` lazily, on first client construction. If the +library cannot be found, constructing a client raises +`Moss::Core::BindingsUnavailableError` with guidance rather than crashing at +`require` time. Use `Moss::Core.available?` to probe without raising. + +## Providing libmoss + +Download the matching `libmoss` C SDK release archive for your platform from: + +- + +Extract it so you have: + +```text +/ +├── include/libmoss.h +└── lib/libmoss.{dylib,so} +``` + +Then point the bindings at it with either environment variable: + +```bash +export MOSS_LIB_DIR="/lib" # directory containing the library +# or +export MOSS_LIBRARY_PATH="/lib/libmoss.dylib" # exact file +``` + +The bindings `dlopen` the library by absolute path, so on macOS you do **not** +need `DYLD_LIBRARY_PATH` for the prebuilt `libmoss.dylib`. + +## API surface + +```ruby +require "moss/core" + +Moss::Core.available? # => true / false +Moss::Core.libmoss_sdk_version # => "0.9.0" (or nil when unavailable) + +manage = Moss::Core::ManageClient.new(project_id, project_key) +manage.create_index("docs", [Moss::Core::DocumentInfo.new(id: "1", text: "hi")], "moss-minilm") +manage.list_indexes +manage.close + +index = Moss::Core::IndexManager.new(project_id, project_key) +index.load_index("docs") +index.query("docs", "hello", top_k: 5) +index.close +``` + +## Development + +```bash +cd sdks/ruby/bindings +ruby -Itest -Ilib test/library_test.rb # attach test auto-skips without libmoss +``` diff --git a/sdks/ruby/bindings/Rakefile b/sdks/ruby/bindings/Rakefile new file mode 100644 index 00000000..5ac3ec4f --- /dev/null +++ b/sdks/ruby/bindings/Rakefile @@ -0,0 +1,19 @@ +# frozen_string_literal: true + +require "rake/testtask" + +Rake::TestTask.new(:test) do |t| + t.libs << "test" + t.libs << "lib" + t.test_files = FileList["test/**/*_test.rb"] + t.warning = false +end + +begin + require "rubocop/rake_task" + RuboCop::RakeTask.new +rescue LoadError + # RuboCop is a development-only dependency; skip the task if it is absent. +end + +task default: :test diff --git a/sdks/ruby/bindings/lib/moss/core.rb b/sdks/ruby/bindings/lib/moss/core.rb new file mode 100644 index 00000000..b3068d8e --- /dev/null +++ b/sdks/ruby/bindings/lib/moss/core.rb @@ -0,0 +1,41 @@ +# frozen_string_literal: true + +require_relative "core/version" +require_relative "core/errors" +require_relative "core/models" +require_relative "core/ffi" +require_relative "core/library" +require_relative "core/marshalling" +require_relative "core/client_handle" +require_relative "core/manage_client" +require_relative "core/index_manager" +require_relative "core/session" + +module Moss + # Moss::Core is the native binding layer: a thin FFI wrapper over the prebuilt + # `libmoss` C SDK. It exposes ManageClient (cloud mutations + reads), + # IndexManager (local index runtime + query) and Session (ephemeral in-memory + # indexes). The high-level `moss` gem builds its ergonomic client on top of + # these primitives. + # + # Requiring this file never touches the filesystem — libmoss is attached + # lazily on first client construction (see Moss::Core::Library). Use + # Moss::Core.available? to check whether the native runtime is present without + # raising. + module Core + module_function + + # Returns the libmoss SDK version string reported by the loaded native + # library, or nil when libmoss is unavailable. + def libmoss_sdk_version + return nil unless Library.available? + + Marshalling.read_string(FFIBindings.moss_sdk_version) + end + + # True when libmoss can be loaded in this process. + def available? + Library.available? + end + end +end diff --git a/sdks/ruby/bindings/lib/moss/core/client_handle.rb b/sdks/ruby/bindings/lib/moss/core/client_handle.rb new file mode 100644 index 00000000..261505d5 --- /dev/null +++ b/sdks/ruby/bindings/lib/moss/core/client_handle.rb @@ -0,0 +1,73 @@ +# frozen_string_literal: true + +require "ffi" +require_relative "ffi" +require_relative "library" +require_relative "errors" +require_relative "marshalling" + +module Moss + module Core + # Owns a single native `MossClient*` handle and serialises access to it. + # + # libmoss exposes one client type that backs manage, local-index and session + # operations; ManageClient and IndexManager each wrap their own handle so + # their locks stay independent (matching the Go bindings' two-handle model). + # + # The handle is freed by an ObjectSpace finalizer as a safety net, but + # callers should prefer explicit #close for deterministic cleanup. The + # finalizer closes over a shared state Hash (never over `self`) so it can run + # without keeping the object alive. + class ClientHandle + def initialize(project_id, project_key) + Library.ensure_attached! + + out = ::FFI::MemoryPointer.new(:pointer) + Marshalling.check!( + FFIBindings.moss_client_new(project_id.to_s, project_key.to_s, out) + ) + + @state = { ptr: out.read_pointer } + @mutex = Mutex.new + ObjectSpace.define_finalizer(self, self.class.finalizer(@state)) + end + + # Frees the native handle. Idempotent and safe to call from multiple + # threads. + def close + @mutex.synchronize do + ptr = @state[:ptr] + return if ptr.nil? || ptr.null? + + FFIBindings.moss_client_free(ptr) + @state[:ptr] = nil + end + end + + def closed? + ptr = @state[:ptr] + ptr.nil? || ptr.null? + end + + # Runs the block with the raw handle under the mutex, raising if closed. + def with_handle + @mutex.synchronize do + ptr = @state[:ptr] + raise ClientClosedError if ptr.nil? || ptr.null? + + yield ptr + end + end + + def self.finalizer(state) + proc do + ptr = state[:ptr] + if ptr && !ptr.null? + FFIBindings.moss_client_free(ptr) + state[:ptr] = nil + end + end + end + end + end +end diff --git a/sdks/ruby/bindings/lib/moss/core/errors.rb b/sdks/ruby/bindings/lib/moss/core/errors.rb new file mode 100644 index 00000000..d741bc64 --- /dev/null +++ b/sdks/ruby/bindings/lib/moss/core/errors.rb @@ -0,0 +1,40 @@ +# frozen_string_literal: true + +module Moss + module Core + # Base class for every error raised by the native binding layer. + class Error < StandardError; end + + # Raised when `libmoss` cannot be located or attached. The high-level SDK + # treats this as a signal to fall back to the cloud query API, mirroring the + # Go SDK's `ErrBindingsUnavailable` behaviour. + class BindingsUnavailableError < Error + DEFAULT_MESSAGE = + "moss-core: libmoss is unavailable. Download the libmoss C SDK release " \ + "and point MOSS_LIB_DIR (or MOSS_LIBRARY_PATH) at it. See " \ + "https://github.com/usemoss/moss/releases (c-sdk)." + + def initialize(message = DEFAULT_MESSAGE) + super + end + end + + # Raised after a client/session handle has been freed. + class ClientClosedError < Error + def initialize(message = "moss-core: client is closed") + super + end + end + + # Raised when a `moss_*` call returns a non-OK result code. Carries the + # numeric code and the thread-local message from `moss_last_error`. + class NativeError < Error + attr_reader :code + + def initialize(message, code) + @code = code + super("moss-core: #{message} (code #{code})") + end + end + end +end diff --git a/sdks/ruby/bindings/lib/moss/core/ffi.rb b/sdks/ruby/bindings/lib/moss/core/ffi.rb new file mode 100644 index 00000000..3306894f --- /dev/null +++ b/sdks/ruby/bindings/lib/moss/core/ffi.rb @@ -0,0 +1,207 @@ +# frozen_string_literal: true + +require "ffi" + +module Moss + module Core + # Raw FFI mapping of the `libmoss` C ABI (see include/libmoss.h in the + # `c-sdk` release). This module is a 1:1 translation of the header and + # contains no higher-level logic — that lives in ManageClient / IndexManager + # / Session. The library is attached lazily by Moss::Core::Library so that a + # missing `libmoss` degrades to BindingsUnavailableError instead of a load + # crash at require time. + module FFIBindings + extend ::FFI::Library + + # Result codes returned by every fallible `moss_*` function. + module Result + OK = 0 + ERR_NULL_POINTER = -1 + ERR_INVALID_ARG = -2 + ERR_CLOUD = -3 + ERR_INDEX_NOT_FOUND = -4 + ERR_MODEL = -5 + ERR_IO = -6 + ERR_INTERNAL = -7 + end + + # typedef struct MossMetadataEntry { char *key; char *value; } + class MetadataEntry < ::FFI::Struct + layout :key, :pointer, + :value, :pointer + end + + # typedef struct MossDocumentInfo { ... } + class DocumentInfo < ::FFI::Struct + layout :id, :pointer, + :text, :pointer, + :metadata, :pointer, # MossMetadataEntry* + :metadata_count, :size_t, + :embedding, :pointer, # float* + :embedding_dim, :size_t + end + + # typedef struct MossMutationResult { ... } + class MutationResult < ::FFI::Struct + layout :job_id, :pointer, + :index_name, :pointer, + :doc_count, :size_t + end + + # typedef struct MossMutationOptions { bool upsert; } + class MutationOptions < ::FFI::Struct + layout :upsert, :bool + end + + # typedef struct MossModelRef { char *id; char *version; } + class ModelRef < ::FFI::Struct + layout :id, :pointer, + :version, :pointer + end + + # typedef struct MossIndexInfo { ... MossModelRef model; } + class IndexInfo < ::FFI::Struct + layout :id, :pointer, + :name, :pointer, + :version, :pointer, + :status, :pointer, + :doc_count, :size_t, + :created_at, :pointer, + :updated_at, :pointer, + :model, ModelRef # nested by value + end + + # typedef struct MossJobStatusResponse { ... } + class JobStatusResponse < ::FFI::Struct + layout :job_id, :pointer, + :status, :pointer, + :progress, :double, + :current_phase, :pointer, + :error, :pointer, + :created_at, :pointer, + :updated_at, :pointer, + :completed_at, :pointer + end + + # typedef struct MossLoadIndexOptions { bool auto_refresh; uint64_t polling_interval_secs; } + class LoadIndexOptions < ::FFI::Struct + layout :auto_refresh, :bool, + :polling_interval_secs, :uint64 + end + + # typedef struct MossQueryOptions { ... } + class QueryOptions < ::FFI::Struct + layout :top_k, :size_t, + :alpha, :float, + :filter_json, :pointer, # const char* + :embedding, :pointer, # const float* + :embedding_dim, :size_t + end + + # typedef struct MossQueryResultDoc { ... float score; } + class QueryResultDoc < ::FFI::Struct + layout :id, :pointer, + :text, :pointer, + :metadata, :pointer, # MossMetadataEntry* + :metadata_count, :size_t, + :score, :float + end + + # typedef struct MossSearchResult { ... } + class SearchResult < ::FFI::Struct + layout :docs, :pointer, # MossQueryResultDoc* + :doc_count, :size_t, + :query, :pointer, + :index_name, :pointer, + :time_taken_ms, :uint64 + end + + # typedef struct MossRefreshResult { ... } + class RefreshResult < ::FFI::Struct + layout :index_name, :pointer, + :previous_updated_at, :pointer, + :new_updated_at, :pointer, + :was_updated, :bool + end + + # typedef struct MossSessionOptions { const char *model_id; } + class SessionOptions < ::FFI::Struct + layout :model_id, :pointer + end + + # typedef struct MossAddDocsOptions { bool upsert; } + class AddDocsOptions < ::FFI::Struct + layout :upsert, :bool + end + + # typedef struct MossPushIndexResult { ... } + class PushIndexResult < ::FFI::Struct + layout :job_id, :pointer, + :index_name, :pointer, + :doc_count, :size_t, + :status, :pointer + end + + # Attaches every `moss_*` symbol against a resolved libmoss path. Called + # once by Moss::Core::Library. Kept as a method (rather than top-level + # attach_function calls) so that require'ing this file never touches the + # filesystem — attachment is deferred until a client is constructed. + def self.attach!(library_path) + ffi_lib library_path + + # --- lifecycle ------------------------------------------------------- + attach_function :moss_sdk_version, [], :pointer + attach_function :moss_last_error, [], :pointer + attach_function :moss_client_new, %i[string string pointer], :int + attach_function :moss_client_free, [:pointer], :void + + # --- manage (cloud mutations + reads) -------------------------------- + attach_function :moss_client_create_index, + %i[pointer string pointer size_t string pointer], :int + attach_function :moss_client_add_docs, + %i[pointer string pointer size_t pointer pointer], :int + attach_function :moss_client_delete_docs, + %i[pointer string pointer size_t pointer], :int + attach_function :moss_client_delete_index, %i[pointer string pointer], :int + attach_function :moss_client_get_index, %i[pointer string pointer], :int + attach_function :moss_client_list_indexes, %i[pointer pointer pointer], :int + attach_function :moss_client_get_docs, + %i[pointer string pointer size_t pointer pointer], :int + attach_function :moss_client_get_job_status, %i[pointer string pointer], :int + + # --- local index runtime -------------------------------------------- + attach_function :moss_client_load_index, %i[pointer string pointer pointer], :int + attach_function :moss_client_unload_index, %i[pointer string], :int + attach_function :moss_client_query, + %i[pointer string string pointer pointer], :int + attach_function :moss_client_refresh_index, %i[pointer string pointer], :int + + # --- sessions -------------------------------------------------------- + attach_function :moss_client_session, %i[pointer string pointer pointer], :int + attach_function :moss_session_free, [:pointer], :void + attach_function :moss_session_name, [:pointer], :pointer + attach_function :moss_session_doc_count, [:pointer], :size_t + attach_function :moss_session_add_docs, + %i[pointer pointer size_t pointer pointer pointer], :int + attach_function :moss_session_delete_docs, + %i[pointer pointer size_t pointer], :int + attach_function :moss_session_get_docs, + %i[pointer pointer size_t pointer pointer], :int + attach_function :moss_session_query, %i[pointer string pointer pointer], :int + attach_function :moss_session_load_index, %i[pointer string pointer], :int + attach_function :moss_session_push_index, %i[pointer pointer], :int + + # --- deallocators (must pair with the allocating call) --------------- + attach_function :moss_free_string, [:pointer], :void + attach_function :moss_free_documents, %i[pointer size_t], :void + attach_function :moss_free_search_result, [:pointer], :void + attach_function :moss_free_index_info, [:pointer], :void + attach_function :moss_free_index_info_list, %i[pointer size_t], :void + attach_function :moss_free_mutation_result, [:pointer], :void + attach_function :moss_free_push_index_result, [:pointer], :void + attach_function :moss_free_job_status_response, [:pointer], :void + attach_function :moss_free_refresh_result, [:pointer], :void + end + end + end +end diff --git a/sdks/ruby/bindings/lib/moss/core/index_manager.rb b/sdks/ruby/bindings/lib/moss/core/index_manager.rb new file mode 100644 index 00000000..b080277d --- /dev/null +++ b/sdks/ruby/bindings/lib/moss/core/index_manager.rb @@ -0,0 +1,116 @@ +# frozen_string_literal: true + +require "set" +require_relative "ffi" +require_relative "client_handle" +require_relative "marshalling" +require_relative "models" + +module Moss + module Core + # Native-backed manager for the local index runtime: load/unload indexes and + # run sub-10ms local queries against them. Mirrors the Go bindings' + # IndexManager, including the loaded-index bookkeeping used by the SDK to + # decide between a local query and the cloud fallback. + class IndexManager + DEFAULT_TOP_K = Core::DEFAULT_TOP_K + DEFAULT_ALPHA = Core::DEFAULT_ALPHA + + def initialize(project_id, project_key) + @handle = ClientHandle.new(project_id, project_key) + @loaded = Set.new + @loaded_mutex = Mutex.new + end + + def close + @handle.close + @loaded_mutex.synchronize { @loaded.clear } + end + + def load_index(index_name, options = nil) + opts = Marshalling.build_load_index_options(options) + out = ::FFI::MemoryPointer.new(:pointer) + + @handle.with_handle do |client| + Marshalling.check!( + FFIBindings.moss_client_load_index(client, index_name.to_s, opts.pointer, out) + ) + end + _retain(opts) + + struct = FFIBindings::IndexInfo.new(out.read_pointer) + info = Marshalling.read_index_info(struct) + FFIBindings.moss_free_index_info(struct) + + @loaded_mutex.synchronize { @loaded.add(index_name.to_s) } + info + end + + def unload_index(index_name) + @handle.with_handle do |client| + Marshalling.check!(FFIBindings.moss_client_unload_index(client, index_name.to_s)) + end + @loaded_mutex.synchronize { @loaded.delete(index_name.to_s) } + nil + end + + def has_index?(index_name) + @loaded_mutex.synchronize { @loaded.include?(index_name.to_s) } + end + + # libmoss loads bundled query models as part of load_index; kept for parity + # with SDKs that expose explicit model loading. + def load_query_model(_index_name) + nil + end + + def query(index_name, query_text, embedding: nil, top_k: DEFAULT_TOP_K, + alpha: DEFAULT_ALPHA, filter_json: nil) + opts = Marshalling.build_query_options( + top_k: top_k, alpha: alpha, filter_json: filter_json, embedding: embedding + ) + out = ::FFI::MemoryPointer.new(:pointer) + + @handle.with_handle do |client| + Marshalling.check!( + FFIBindings.moss_client_query(client, index_name.to_s, query_text.to_s, opts.pointer, out) + ) + end + _retain(opts) + + struct = FFIBindings::SearchResult.new(out.read_pointer) + result = Marshalling.read_search_result(struct) + FFIBindings.moss_free_search_result(struct) + result + end + + def refresh_index(index_name) + out = ::FFI::MemoryPointer.new(:pointer) + @handle.with_handle do |client| + Marshalling.check!(FFIBindings.moss_client_refresh_index(client, index_name.to_s, out)) + end + + struct = FFIBindings::RefreshResult.new(out.read_pointer) + result = Marshalling.read_refresh_result(struct) + FFIBindings.moss_free_refresh_result(struct) + result + end + + def get_index_info(index_name) + out = ::FFI::MemoryPointer.new(:pointer) + @handle.with_handle do |client| + Marshalling.check!(FFIBindings.moss_client_get_index(client, index_name.to_s, out)) + end + + struct = FFIBindings::IndexInfo.new(out.read_pointer) + info = Marshalling.read_index_info(struct) + FFIBindings.moss_free_index_info(struct) + info + end + + private + + def _retain(_allocation); end + end + end +end diff --git a/sdks/ruby/bindings/lib/moss/core/library.rb b/sdks/ruby/bindings/lib/moss/core/library.rb new file mode 100644 index 00000000..d340b5c0 --- /dev/null +++ b/sdks/ruby/bindings/lib/moss/core/library.rb @@ -0,0 +1,85 @@ +# frozen_string_literal: true + +require "ffi" +require_relative "ffi" +require_relative "errors" + +module Moss + module Core + # Locates and attaches the native `libmoss` shared library exactly once. + # + # Resolution order: + # 1. ENV["MOSS_LIBRARY_PATH"] — absolute path to the shared library file + # 2. ENV["MOSS_LIB_DIR"] — directory containing lib{moss}.{dylib,so,dll} + # 3. the platform default name — resolved via the system loader search path + # + # Attaching by absolute path (options 1 and 2) is preferred on macOS: the + # prebuilt `libmoss.dylib` ships with a build-server install name baked in, + # and dlopen'ing the file directly sidesteps the need for DYLD_LIBRARY_PATH. + module Library + module_function + + DEFAULT_BASENAME = "#{::FFI::Platform::LIBPREFIX}moss.#{::FFI::Platform::LIBSUFFIX}".freeze + + # Returns the resolved shared-library path (absolute when derived from env), + # or the bare platform basename to let the system loader search for it. + def resolved_path + explicit = env_value("MOSS_LIBRARY_PATH") + return explicit if explicit + + dir = env_value("MOSS_LIB_DIR") + if dir + candidate = File.join(dir, DEFAULT_BASENAME) + return candidate if File.exist?(candidate) + + # Fall back to any lib{moss}.* in the directory (e.g. versioned names). + match = Dir.glob(File.join(dir, "#{::FFI::Platform::LIBPREFIX}moss.*")).first + return match if match + end + + DEFAULT_BASENAME + end + + # Attaches libmoss on first call; memoized thereafter. Raises + # BindingsUnavailableError (not a raw FFI/LoadError) when the library is + # missing so callers can branch on a single, documented error type. + def ensure_attached! + return true if @attached + + @mutex ||= Mutex.new + @mutex.synchronize do + return true if @attached + + begin + FFIBindings.attach!(resolved_path) + @attached = true + rescue LoadError, ::FFI::NotFoundError => e + raise BindingsUnavailableError, "#{BindingsUnavailableError::DEFAULT_MESSAGE} (#{e.message})" + end + end + @attached + end + + # True when libmoss can be loaded in this process. Never raises — intended + # for the SDK's "fall back to cloud query" decision. + def available? + ensure_attached! + rescue BindingsUnavailableError + false + end + + # Test/reset hook: forget the memoized attachment state. + def reset! + @attached = false + end + + def env_value(key) + value = ENV.fetch(key, nil) + return nil if value.nil? + + stripped = value.strip + stripped.empty? ? nil : stripped + end + end + end +end diff --git a/sdks/ruby/bindings/lib/moss/core/manage_client.rb b/sdks/ruby/bindings/lib/moss/core/manage_client.rb new file mode 100644 index 00000000..f0af85dd --- /dev/null +++ b/sdks/ruby/bindings/lib/moss/core/manage_client.rb @@ -0,0 +1,173 @@ +# frozen_string_literal: true + +require_relative "ffi" +require_relative "client_handle" +require_relative "marshalling" +require_relative "models" +require_relative "session" + +module Moss + module Core + # Native-backed client for cloud mutations and reads. Every method routes + # through libmoss and returns the plain-Ruby value objects from + # Moss::Core::models. Raises Moss::Core::BindingsUnavailableError at + # construction if libmoss cannot be loaded. + class ManageClient + def initialize(project_id, project_key) + @handle = ClientHandle.new(project_id, project_key) + end + + def close + @handle.close + end + + def create_index(name, docs, model_id = nil) + input = Marshalling.build_documents(docs) + out = ::FFI::MemoryPointer.new(:pointer) + + @handle.with_handle do |client| + Marshalling.check!( + FFIBindings.moss_client_create_index( + client, name.to_s, input.pointer, docs.length, model_id, out + ) + ) + end + _retain(input) + + read_and_free_mutation_result(out) + end + + def add_docs(name, docs, options = nil) + input = Marshalling.build_documents(docs) + opts = Marshalling.build_mutation_options(options) + out = ::FFI::MemoryPointer.new(:pointer) + + @handle.with_handle do |client| + Marshalling.check!( + FFIBindings.moss_client_add_docs( + client, name.to_s, input.pointer, docs.length, opts.pointer, out + ) + ) + end + _retain(input) + _retain(opts) + + read_and_free_mutation_result(out) + end + + def delete_docs(name, doc_ids) + ids = Marshalling.build_string_array(doc_ids) + out = ::FFI::MemoryPointer.new(:pointer) + + @handle.with_handle do |client| + Marshalling.check!( + FFIBindings.moss_client_delete_docs( + client, name.to_s, ids.pointer, doc_ids.length, out + ) + ) + end + _retain(ids) + + read_and_free_mutation_result(out) + end + + def get_job_status(job_id) + out = ::FFI::MemoryPointer.new(:pointer) + @handle.with_handle do |client| + Marshalling.check!(FFIBindings.moss_client_get_job_status(client, job_id.to_s, out)) + end + + struct = FFIBindings::JobStatusResponse.new(out.read_pointer) + result = Marshalling.read_job_status(struct) + FFIBindings.moss_free_job_status_response(struct) + result + end + + def get_index(name) + out = ::FFI::MemoryPointer.new(:pointer) + @handle.with_handle do |client| + Marshalling.check!(FFIBindings.moss_client_get_index(client, name.to_s, out)) + end + read_and_free_index_info(out) + end + + def list_indexes + out = ::FFI::MemoryPointer.new(:pointer) + count_ptr = ::FFI::MemoryPointer.new(:size_t) + + @handle.with_handle do |client| + Marshalling.check!(FFIBindings.moss_client_list_indexes(client, out, count_ptr)) + end + + base = out.read_pointer + count = count_ptr.read(:size_t) + result = Marshalling.read_index_info_list(base, count) + FFIBindings.moss_free_index_info_list(base, count) unless base.null? + result + end + + def delete_index(name) + deleted = ::FFI::MemoryPointer.new(:bool) + @handle.with_handle do |client| + Marshalling.check!(FFIBindings.moss_client_delete_index(client, name.to_s, deleted)) + end + deleted.read(:bool) + end + + def get_docs(name, doc_ids = []) + ids = Marshalling.build_string_array(doc_ids) + out = ::FFI::MemoryPointer.new(:pointer) + count_ptr = ::FFI::MemoryPointer.new(:size_t) + + @handle.with_handle do |client| + Marshalling.check!( + FFIBindings.moss_client_get_docs( + client, name.to_s, ids.pointer, doc_ids.length, out, count_ptr + ) + ) + end + _retain(ids) + + base = out.read_pointer + count = count_ptr.read(:size_t) + result = Marshalling.read_documents(base, count) + FFIBindings.moss_free_documents(base, count) unless base.null? + result + end + + # Opens a session backed by this client. Keeps the ManageClient referenced + # so the underlying native handle outlives the session. + def session(name, options = nil) + opts = Marshalling.build_session_options(options) + out = ::FFI::MemoryPointer.new(:pointer) + + @handle.with_handle do |client| + Marshalling.check!(FFIBindings.moss_client_session(client, name.to_s, opts.pointer, out)) + end + _retain(opts) + + Session.new(out.read_pointer, owner: self) + end + + private + + def read_and_free_mutation_result(out) + struct = FFIBindings::MutationResult.new(out.read_pointer) + result = Marshalling.read_mutation_result(struct) + FFIBindings.moss_free_mutation_result(struct) + result + end + + def read_and_free_index_info(out) + struct = FFIBindings::IndexInfo.new(out.read_pointer) + result = Marshalling.read_index_info(struct) + FFIBindings.moss_free_index_info(struct) + result + end + + # No-op reference sink documenting that the input allocation must survive + # until the native call above has returned. + def _retain(_allocation); end + end + end +end diff --git a/sdks/ruby/bindings/lib/moss/core/marshalling.rb b/sdks/ruby/bindings/lib/moss/core/marshalling.rb new file mode 100644 index 00000000..79e1108c --- /dev/null +++ b/sdks/ruby/bindings/lib/moss/core/marshalling.rb @@ -0,0 +1,333 @@ +# frozen_string_literal: true + +require_relative "ffi" +require_relative "models" +require_relative "errors" + +module Moss + module Core + # Conversions between Ruby values and the native C structs, plus result-code + # checking. Split out from the client classes so ownership rules live in one + # place: + # + # * "build_*" methods allocate C memory for INPUT and return an Allocation + # whose #retained array must stay referenced until the native call + # returns (FFI frees MemoryPointers once they are unreachable). + # * "read_*" methods COPY native OUTPUT into Ruby objects; the caller is + # then responsible for invoking the matching moss_free_* deallocator. + module Marshalling + module_function + + # Holds a primary pointer plus every child allocation that backs it, so a + # single local reference keeps the whole graph alive across an FFI call. + Allocation = Struct.new(:pointer, :retained) + + NULL = ::FFI::Pointer::NULL + + # Raises NativeError unless the code is OK, attaching moss_last_error text. + def check!(code) + return if code == FFIBindings::Result::OK + + message = read_string(FFIBindings.moss_last_error) || "libmoss call failed" + raise NativeError.new(message, code) + end + + # ---- reads (native -> Ruby, copying) --------------------------------- + + def read_string(ptr) + return nil if ptr.nil? || ptr.null? + + ptr.read_string.force_encoding(Encoding::UTF_8) + end + + def read_metadata(ptr, count) + return nil if ptr.nil? || ptr.null? || count.zero? + + entry_size = FFIBindings::MetadataEntry.size + result = {} + count.times do |i| + entry = FFIBindings::MetadataEntry.new(ptr + (i * entry_size)) + key = read_string(entry[:key]) + result[key] = read_string(entry[:value]) unless key.nil? + end + result + end + + def read_embedding(ptr, dim) + return nil if ptr.nil? || ptr.null? || dim.zero? + + ptr.read_array_of_float(dim) + end + + def read_index_info(struct) + model = struct[:model] + Core::IndexInfo.new( + id: read_string(struct[:id]), + name: read_string(struct[:name]), + version: read_string(struct[:version]), + status: read_string(struct[:status]), + doc_count: struct[:doc_count], + created_at: read_string(struct[:created_at]), + updated_at: read_string(struct[:updated_at]), + model: Core::ModelRef.new( + id: read_string(model[:id]), + version: read_string(model[:version]) + ) + ) + end + + def read_documents(base_ptr, count) + return [] if base_ptr.nil? || base_ptr.null? || count.zero? + + struct_size = FFIBindings::DocumentInfo.size + Array.new(count) do |i| + struct = FFIBindings::DocumentInfo.new(base_ptr + (i * struct_size)) + Core::DocumentInfo.new( + id: read_string(struct[:id]), + text: read_string(struct[:text]), + metadata: read_metadata(struct[:metadata], struct[:metadata_count]), + embedding: read_embedding(struct[:embedding], struct[:embedding_dim]) + ) + end + end + + def read_index_info_list(base_ptr, count) + return [] if base_ptr.nil? || base_ptr.null? || count.zero? + + struct_size = FFIBindings::IndexInfo.size + Array.new(count) do |i| + read_index_info(FFIBindings::IndexInfo.new(base_ptr + (i * struct_size))) + end + end + + def read_search_result(struct) + docs = [] + base = struct[:docs] + count = struct[:doc_count] + unless base.null? || count.zero? + doc_size = FFIBindings::QueryResultDoc.size + count.times do |i| + doc = FFIBindings::QueryResultDoc.new(base + (i * doc_size)) + docs << Core::QueryResultDocument.new( + id: read_string(doc[:id]), + text: read_string(doc[:text]), + metadata: read_metadata(doc[:metadata], doc[:metadata_count]), + score: doc[:score] + ) + end + end + + Core::SearchResult.new( + docs: docs, + query: read_string(struct[:query]), + index_name: read_string(struct[:index_name]), + time_taken_ms: struct[:time_taken_ms] + ) + end + + def read_job_status(struct) + Core::JobStatusResponse.new( + job_id: read_string(struct[:job_id]), + status: read_string(struct[:status]), + progress: struct[:progress], + current_phase: read_string(struct[:current_phase]), + error: read_string(struct[:error]), + created_at: read_string(struct[:created_at]), + updated_at: read_string(struct[:updated_at]), + completed_at: read_string(struct[:completed_at]) + ) + end + + def read_mutation_result(struct) + Core::MutationResult.new( + job_id: read_string(struct[:job_id]), + index_name: read_string(struct[:index_name]), + doc_count: struct[:doc_count] + ) + end + + def read_refresh_result(struct) + Core::RefreshResult.new( + index_name: read_string(struct[:index_name]), + previous_updated_at: read_string(struct[:previous_updated_at]), + new_updated_at: read_string(struct[:new_updated_at]), + was_updated: struct[:was_updated] + ) + end + + def read_push_index_result(struct) + Core::PushIndexResult.new( + job_id: read_string(struct[:job_id]), + index_name: read_string(struct[:index_name]), + doc_count: struct[:doc_count], + status: read_string(struct[:status]) + ) + end + + # ---- builds (Ruby -> native, caller retains until the call returns) --- + + def mem_string(value) + bytes = value.to_s.b + ptr = ::FFI::MemoryPointer.new(:char, bytes.bytesize + 1) # zero-filled -> NUL terminator + ptr.put_bytes(0, bytes) + ptr + end + + def mem_floats(values) + ptr = ::FFI::MemoryPointer.new(:float, values.length) + ptr.write_array_of_float(values.map(&:to_f)) + ptr + end + + def build_documents(docs) + retained = [] + count = docs.length + return Allocation.new(NULL, retained) if count.zero? + + array = ::FFI::MemoryPointer.new(FFIBindings::DocumentInfo, count) + retained << array + struct_size = FFIBindings::DocumentInfo.size + + docs.each_with_index do |doc, i| + entry = FFIBindings::DocumentInfo.new(array + (i * struct_size)) + + id_ptr = mem_string(doc.id) + text_ptr = mem_string(doc.text) + retained << id_ptr << text_ptr + entry[:id] = id_ptr + entry[:text] = text_ptr + + apply_metadata(entry, doc.metadata, retained) + apply_embedding(entry, doc.embedding, retained) + end + + Allocation.new(array, retained) + end + + def build_string_array(values) + retained = [] + count = values.length + return Allocation.new(NULL, retained) if count.zero? + + array = ::FFI::MemoryPointer.new(:pointer, count) + retained << array + values.each_with_index do |value, i| + str_ptr = mem_string(value) + retained << str_ptr + array.put_pointer(i * ::FFI::Pointer.size, str_ptr) + end + + Allocation.new(array, retained) + end + + # Builds a MossQueryOptions struct. Returns an Allocation whose pointer is + # the struct (or NULL when no options are provided). + def build_query_options(top_k:, alpha:, filter_json:, embedding:) + retained = [] + opts = FFIBindings::QueryOptions.new + retained << opts + + opts[:top_k] = top_k + opts[:alpha] = alpha + + if filter_json + filter_ptr = mem_string(filter_json) + retained << filter_ptr + opts[:filter_json] = filter_ptr + else + opts[:filter_json] = NULL + end + + if embedding && !embedding.empty? + emb_ptr = mem_floats(embedding) + retained << emb_ptr + opts[:embedding] = emb_ptr + opts[:embedding_dim] = embedding.length + else + opts[:embedding] = NULL + opts[:embedding_dim] = 0 + end + + Allocation.new(opts.to_ptr, retained) + end + + # Option-struct builders. Each returns an Allocation whose #retained array + # keeps the FFI::Struct (and any backing strings) referenced for the + # duration of the native call, and whose #pointer is NULL when no options + # apply. Callers must keep the Allocation referenced until the call + # returns (see the _retain sinks in the client classes). + + def build_mutation_options(options) + return Allocation.new(NULL, []) if options.nil? || options.upsert.nil? + + struct = FFIBindings::MutationOptions.new + struct[:upsert] = options.upsert ? true : false + Allocation.new(struct.to_ptr, [struct]) + end + + def build_add_docs_options(options) + return Allocation.new(NULL, []) if options.nil? || options.upsert.nil? + + struct = FFIBindings::AddDocsOptions.new + struct[:upsert] = options.upsert ? true : false + Allocation.new(struct.to_ptr, [struct]) + end + + def build_load_index_options(options) + return Allocation.new(NULL, []) if options.nil? + + struct = FFIBindings::LoadIndexOptions.new + struct[:auto_refresh] = options.auto_refresh ? true : false + struct[:polling_interval_secs] = options.polling_interval_secs.to_i + Allocation.new(struct.to_ptr, [struct]) + end + + def build_session_options(options) + model_id = options&.model_id + return Allocation.new(NULL, []) if model_id.nil? + + model_ptr = mem_string(model_id) + struct = FFIBindings::SessionOptions.new + struct[:model_id] = model_ptr + Allocation.new(struct.to_ptr, [struct, model_ptr]) + end + + def apply_metadata(entry, metadata, retained) + if metadata.nil? || metadata.empty? + entry[:metadata] = NULL + entry[:metadata_count] = 0 + return + end + + array = ::FFI::MemoryPointer.new(FFIBindings::MetadataEntry, metadata.length) + retained << array + entry_size = FFIBindings::MetadataEntry.size + + metadata.each_with_index do |(key, value), j| + meta = FFIBindings::MetadataEntry.new(array + (j * entry_size)) + key_ptr = mem_string(key) + value_ptr = mem_string(value) + retained << key_ptr << value_ptr + meta[:key] = key_ptr + meta[:value] = value_ptr + end + + entry[:metadata] = array + entry[:metadata_count] = metadata.length + end + + def apply_embedding(entry, embedding, retained) + if embedding.nil? || embedding.empty? + entry[:embedding] = NULL + entry[:embedding_dim] = 0 + return + end + + emb_ptr = mem_floats(embedding) + retained << emb_ptr + entry[:embedding] = emb_ptr + entry[:embedding_dim] = embedding.length + end + end + end +end diff --git a/sdks/ruby/bindings/lib/moss/core/models.rb b/sdks/ruby/bindings/lib/moss/core/models.rb new file mode 100644 index 00000000..e9737e8b --- /dev/null +++ b/sdks/ruby/bindings/lib/moss/core/models.rb @@ -0,0 +1,62 @@ +# frozen_string_literal: true + +module Moss + module Core + # Default local-query parameters shared by IndexManager and Session. + DEFAULT_TOP_K = 5 + DEFAULT_ALPHA = 0.8 + + # Plain-Ruby value objects returned by the binding layer. These intentionally + # mirror the C structs in libmoss.h field-for-field; the high-level `moss` + # gem re-maps them onto its own richer models. Keyword-initialized Structs + # keep them immutable-ish and cheap to construct from marshalled native data. + + DocumentInfo = Struct.new(:id, :text, :metadata, :embedding, keyword_init: true) do + def initialize(id:, text:, metadata: nil, embedding: nil) + super + end + end + + ModelRef = Struct.new(:id, :version, keyword_init: true) + + IndexInfo = Struct.new( + :id, :name, :version, :status, :doc_count, + :created_at, :updated_at, :model, + keyword_init: true + ) + + MutationResult = Struct.new(:job_id, :index_name, :doc_count, keyword_init: true) + + JobStatusResponse = Struct.new( + :job_id, :status, :progress, :current_phase, :error, + :created_at, :updated_at, :completed_at, + keyword_init: true + ) + + QueryResultDocument = Struct.new(:id, :text, :metadata, :score, keyword_init: true) + + SearchResult = Struct.new( + :docs, :query, :index_name, :time_taken_ms, + keyword_init: true + ) + + RefreshResult = Struct.new( + :index_name, :previous_updated_at, :new_updated_at, :was_updated, + keyword_init: true + ) + + PushIndexResult = Struct.new( + :job_id, :index_name, :doc_count, :status, + keyword_init: true + ) + + # Counts returned by Session#add_docs (added vs. upserted-over documents). + SessionAddResult = Struct.new(:added, :updated, keyword_init: true) + + # Options accepted by the binding layer (distinct from the high-level SDK + # option objects). Nil fields mean "use the native default". + MutationOptions = Struct.new(:upsert, keyword_init: true) + LoadIndexOptions = Struct.new(:auto_refresh, :polling_interval_secs, keyword_init: true) + SessionOptions = Struct.new(:model_id, keyword_init: true) + end +end diff --git a/sdks/ruby/bindings/lib/moss/core/session.rb b/sdks/ruby/bindings/lib/moss/core/session.rb new file mode 100644 index 00000000..a6b669b8 --- /dev/null +++ b/sdks/ruby/bindings/lib/moss/core/session.rb @@ -0,0 +1,162 @@ +# frozen_string_literal: true + +require "ffi" +require_relative "ffi" +require_relative "marshalling" +require_relative "models" +require_relative "errors" + +module Moss + module Core + # Native-backed ephemeral session: build and query an index in memory, then + # push it to the cloud. Sessions are created via ManageClient#session and + # hold a reference to their owner so the underlying native client outlives + # them. The MossSession* handle is freed by #close or an ObjectSpace + # finalizer (which closes over a state Hash, never over `self`). + class Session + def initialize(ptr, owner:) + @owner = owner + @state = { ptr: ptr } + @mutex = Mutex.new + ObjectSpace.define_finalizer(self, self.class.finalizer(@state)) + end + + def name + with_handle do |session| + Marshalling.read_string(FFIBindings.moss_session_name(session)) + end + end + + def doc_count + with_handle { |session| FFIBindings.moss_session_doc_count(session) } + end + + def add_docs(docs, options = nil) + input = Marshalling.build_documents(docs) + opts = Marshalling.build_add_docs_options(options) + added = ::FFI::MemoryPointer.new(:size_t) + updated = ::FFI::MemoryPointer.new(:size_t) + + with_handle do |session| + Marshalling.check!( + FFIBindings.moss_session_add_docs( + session, input.pointer, docs.length, opts.pointer, added, updated + ) + ) + end + _retain(input) + _retain(opts) + + Core::SessionAddResult.new(added: added.read(:size_t), updated: updated.read(:size_t)) + end + + def delete_docs(doc_ids) + ids = Marshalling.build_string_array(doc_ids) + deleted = ::FFI::MemoryPointer.new(:size_t) + + with_handle do |session| + Marshalling.check!( + FFIBindings.moss_session_delete_docs(session, ids.pointer, doc_ids.length, deleted) + ) + end + _retain(ids) + + deleted.read(:size_t) + end + + def get_docs(doc_ids = []) + ids = Marshalling.build_string_array(doc_ids) + out = ::FFI::MemoryPointer.new(:pointer) + count_ptr = ::FFI::MemoryPointer.new(:size_t) + + with_handle do |session| + Marshalling.check!( + FFIBindings.moss_session_get_docs(session, ids.pointer, doc_ids.length, out, count_ptr) + ) + end + _retain(ids) + + base = out.read_pointer + count = count_ptr.read(:size_t) + result = Marshalling.read_documents(base, count) + FFIBindings.moss_free_documents(base, count) unless base.null? + result + end + + def query(query_text, embedding: nil, top_k: Core::DEFAULT_TOP_K, + alpha: Core::DEFAULT_ALPHA, filter_json: nil) + opts = Marshalling.build_query_options( + top_k: top_k, alpha: alpha, filter_json: filter_json, embedding: embedding + ) + out = ::FFI::MemoryPointer.new(:pointer) + + with_handle do |session| + Marshalling.check!( + FFIBindings.moss_session_query(session, query_text.to_s, opts.pointer, out) + ) + end + _retain(opts) + + struct = FFIBindings::SearchResult.new(out.read_pointer) + result = Marshalling.read_search_result(struct) + FFIBindings.moss_free_search_result(struct) + result + end + + def load_index(index_name) + count_ptr = ::FFI::MemoryPointer.new(:size_t) + with_handle do |session| + Marshalling.check!( + FFIBindings.moss_session_load_index(session, index_name.to_s, count_ptr) + ) + end + count_ptr.read(:size_t) + end + + def push_index + out = ::FFI::MemoryPointer.new(:pointer) + with_handle do |session| + Marshalling.check!(FFIBindings.moss_session_push_index(session, out)) + end + + struct = FFIBindings::PushIndexResult.new(out.read_pointer) + result = Marshalling.read_push_index_result(struct) + FFIBindings.moss_free_push_index_result(struct) + result + end + + def close + @mutex.synchronize do + ptr = @state[:ptr] + return if ptr.nil? || ptr.null? + + FFIBindings.moss_session_free(ptr) + @state[:ptr] = nil + end + end + + def self.finalizer(state) + proc do + ptr = state[:ptr] + if ptr && !ptr.null? + FFIBindings.moss_session_free(ptr) + state[:ptr] = nil + end + end + end + + private + + def with_handle + @mutex.synchronize do + ptr = @state[:ptr] + raise ClientClosedError, "moss-core: session is closed" if ptr.nil? || ptr.null? + + yield ptr + end + end + + def _retain(_allocation); end + end + end +end diff --git a/sdks/ruby/bindings/lib/moss/core/version.rb b/sdks/ruby/bindings/lib/moss/core/version.rb new file mode 100644 index 00000000..5c9d0f1a --- /dev/null +++ b/sdks/ruby/bindings/lib/moss/core/version.rb @@ -0,0 +1,12 @@ +# frozen_string_literal: true + +module Moss + module Core + # Tracks the libmoss C ABI this binding targets. Bump alongside the pinned + # `c-sdk` release in Moss::Core::LIBMOSS_VERSION. + VERSION = "0.9.0" + + # The libmoss C SDK release these bindings are generated against. + LIBMOSS_VERSION = "0.9.0" + end +end diff --git a/sdks/ruby/bindings/moss-core.gemspec b/sdks/ruby/bindings/moss-core.gemspec new file mode 100644 index 00000000..08f3d5aa --- /dev/null +++ b/sdks/ruby/bindings/moss-core.gemspec @@ -0,0 +1,40 @@ +# frozen_string_literal: true + +require_relative "lib/moss/core/version" + +Gem::Specification.new do |spec| + spec.name = "moss-core" + spec.version = Moss::Core::VERSION + spec.authors = ["Moss"] + spec.summary = "FFI bindings for the Moss semantic search engine (libmoss)." + spec.description = <<~DESC + Low-level Ruby bindings over the prebuilt libmoss C SDK. Provides + ManageClient, IndexManager and Session primitives used by the high-level + `moss` gem. Requires the libmoss shared library at runtime; download it from + the usemoss/moss c-sdk releases and point MOSS_LIB_DIR (or MOSS_LIBRARY_PATH) + at it. + DESC + spec.homepage = "https://github.com/usemoss/moss" + spec.license = "BSD-2-Clause" + + spec.required_ruby_version = ">= 3.0" + + spec.metadata = { + "homepage_uri" => spec.homepage, + "source_code_uri" => "https://github.com/usemoss/moss/tree/main/sdks/ruby/bindings", + "documentation_uri" => "https://docs.moss.dev", + "rubygems_mfa_required" => "true" + } + + spec.files = Dir[ + "lib/**/*.rb", + "README.md", + "LICENSE" + ] + spec.require_paths = ["lib"] + + spec.add_dependency "ffi", "~> 1.15" + + spec.add_development_dependency "minitest", "~> 5.0" + spec.add_development_dependency "rake", "~> 13.0" +end diff --git a/sdks/ruby/bindings/test/library_test.rb b/sdks/ruby/bindings/test/library_test.rb new file mode 100644 index 00000000..1d332500 --- /dev/null +++ b/sdks/ruby/bindings/test/library_test.rb @@ -0,0 +1,58 @@ +# frozen_string_literal: true + +require_relative "test_helper" + +class LibraryTest < Minitest::Test + def test_default_basename_is_platform_appropriate + assert_includes Moss::Core::Library::DEFAULT_BASENAME, "moss" + end + + def test_env_library_path_takes_precedence + with_env("MOSS_LIBRARY_PATH" => "/custom/path/libmoss.dylib") do + assert_equal "/custom/path/libmoss.dylib", Moss::Core::Library.resolved_path + end + end + + def test_blank_env_is_ignored + with_env("MOSS_LIBRARY_PATH" => " ", "MOSS_LIB_DIR" => "") do + assert_equal Moss::Core::Library::DEFAULT_BASENAME, Moss::Core::Library.resolved_path + end + end + + def test_value_objects_are_constructible + doc = Moss::Core::DocumentInfo.new(id: "1", text: "hi") + assert_equal "1", doc.id + assert_nil doc.embedding + + result = Moss::Core::MutationResult.new(job_id: "j", index_name: "idx", doc_count: 2) + assert_equal 2, result.doc_count + end + + # Verifies the whole FFI mapping attaches against a real libmoss when one is + # available (MOSS_LIB_DIR / MOSS_LIBRARY_PATH set). Skips otherwise so the + # suite is green on machines without the C SDK. + def test_attaches_and_reports_version_when_libmoss_present + skip("libmoss not available") unless Moss::Core.available? + + version = Moss::Core.libmoss_sdk_version + refute_nil version + assert_match(/\A\d+\.\d+/, version) + + client = Moss::Core::ManageClient.new("dummy-project", "dummy-key") + refute_nil client + client.close + end + + private + + def with_env(overrides) + original = {} + overrides.each do |key, value| + original[key] = ENV.fetch(key, nil) + ENV[key] = value + end + yield + ensure + original.each { |key, value| ENV[key] = value } + end +end diff --git a/sdks/ruby/bindings/test/test_helper.rb b/sdks/ruby/bindings/test/test_helper.rb new file mode 100644 index 00000000..ee126963 --- /dev/null +++ b/sdks/ruby/bindings/test/test_helper.rb @@ -0,0 +1,6 @@ +# frozen_string_literal: true + +$LOAD_PATH.unshift File.expand_path("../lib", __dir__) + +require "minitest/autorun" +require "moss/core" diff --git a/sdks/ruby/sdk/.rubocop.yml b/sdks/ruby/sdk/.rubocop.yml new file mode 100644 index 00000000..fc2019d4 --- /dev/null +++ b/sdks/ruby/sdk/.rubocop.yml @@ -0,0 +1 @@ +inherit_from: ../.rubocop.yml diff --git a/sdks/ruby/sdk/CHANGELOG.md b/sdks/ruby/sdk/CHANGELOG.md new file mode 100644 index 00000000..66e7d6f2 --- /dev/null +++ b/sdks/ruby/sdk/CHANGELOG.md @@ -0,0 +1,29 @@ +# Changelog + +All notable changes to the `moss` gem are documented here. The format is based +on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). + +## [0.1.0] - Unreleased + +### Added + +- Initial release of the Moss Ruby SDK. +- `Moss::Client` with parity on index management, search, and metadata + filtering: + - Index mutations with async job polling: `create_index`, `add_documents` + (`add_docs`), `delete_documents` (`delete_docs`), `get_job_status`. + - Index reads: `get_index`, `list_indexes`, `delete_index`, `get_documents` + (`get_docs`). + - Local runtime: `load_index`, `unload_index`, `refresh_index`, + `get_index_info`. + - `query` (aliased `search`) with sub-10ms local execution when an index is + loaded, plus a cloud query fallback; supports `top_k`, `alpha`, caller + embeddings, and metadata `filter`. + - `session` support (`Moss::Session`) for building and querying ephemeral + in-memory indexes and pushing them to the cloud. +- Custom-embedding indexes with automatic model inference and dimension + validation. +- Credentials resolved from constructor arguments or the `MOSS_PROJECT_ID` / + `MOSS_PROJECT_KEY` environment variables. +- Local semantic search powered by the native `libmoss` runtime through the + `moss-core` bindings gem. diff --git a/sdks/ruby/sdk/Gemfile b/sdks/ruby/sdk/Gemfile new file mode 100644 index 00000000..afff9d99 --- /dev/null +++ b/sdks/ruby/sdk/Gemfile @@ -0,0 +1,9 @@ +# frozen_string_literal: true + +source "https://rubygems.org" + +gemspec + +# During in-repo development the native bindings are resolved from the sibling +# directory rather than RubyGems. +gem "moss-core", path: "../bindings" diff --git a/sdks/ruby/sdk/LICENSE b/sdks/ruby/sdk/LICENSE new file mode 100644 index 00000000..372ad0ad --- /dev/null +++ b/sdks/ruby/sdk/LICENSE @@ -0,0 +1,25 @@ +BSD 2-Clause License + +Copyright (c) 2026, Moss Team +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/sdks/ruby/sdk/README.md b/sdks/ruby/sdk/README.md new file mode 100644 index 00000000..924ec09c --- /dev/null +++ b/sdks/ruby/sdk/README.md @@ -0,0 +1,224 @@ +# Moss Ruby SDK + +`moss` is the Ruby SDK for [Moss](https://docs.moss.dev/docs/start/what-is-moss), +a real-time semantic search runtime for AI agents. It gives Ruby and Rails +developers on-device semantic search — indexing, sub-10ms local search, and +metadata filtering — without leaving their stack. + +The SDK has two layers: + +- **`moss`** (this gem) — the ergonomic, pure-Ruby client. +- **[`moss-core`](../bindings)** — native FFI bindings over the `libmoss` C SDK + that power local indexing and search. + +Mutations go to Moss Cloud; queries run locally when an index is loaded and +otherwise fall back to the cloud query API. + +## Features + +- Typed client and value objects for indexes, documents, and search results +- Index creation and document mutation with async job polling +- Index and document reads +- Local index loading, metadata, and sub-10ms query via native bindings +- Cloud query fallback when an index is not loaded locally +- Metadata filtering (`$eq`, `$and`, `$or`, `$in`, `$near`, …) +- Optional caller-provided embeddings for custom indexes +- Ephemeral in-memory sessions +- Env-gated live integration tests + +## Installation + +Add to your `Gemfile`: + +```ruby +gem "moss" +``` + +Then install the native `libmoss` runtime (required for local indexing and +search). Download the archive for your platform from the +[`c-sdk-v0.9.0` release](https://github.com/usemoss/moss/releases/tag/c-sdk-v0.9.0) +and point the bindings at it: + +```bash +export MOSS_LIB_DIR="/path/to/libmoss/lib" +``` + +Get project credentials at [moss.dev](https://moss.dev) and export them: + +```bash +export MOSS_PROJECT_ID=... +export MOSS_PROJECT_KEY=... +``` + +## Quick start + +```ruby +require "moss" + +client = Moss::Client.new # reads MOSS_PROJECT_ID / MOSS_PROJECT_KEY from ENV + +documents = [ + Moss::DocumentInfo.new( + id: "doc-1", + text: "Refunds are processed within five to seven business days.", + metadata: { "topic" => "refunds" } + ), + Moss::DocumentInfo.new( + id: "doc-2", + text: "Orders can be tracked from the account dashboard.", + metadata: { "topic" => "shipping" } + ) +] + +client.create_index("support-docs", documents) +client.load_index("support-docs") + +result = client.query("support-docs", "how long do refunds take?", top_k: 3) +result.docs.each { |doc| puts "#{doc.id} #{format('%.3f', doc.score)}" } + +client.close +``` + +Credentials can also be passed explicitly: + +```ruby +client = Moss::Client.new(project_id: "…", project_key: "…") +``` + +## Metadata filtering + +Filters require a locally loaded index. The filter is passed to the engine +verbatim, using its filter schema (`field` + `condition`): + +```ruby +client.load_index("support-docs") + +client.query( + "support-docs", + "how long do refunds take?", + top_k: 3, + filter: { "field" => "topic", "condition" => { "$eq" => "shipping" } } +) + +# Compound filters: +client.query( + "products", + "running shoes", + filter: { + "$and" => [ + { "field" => "category", "condition" => { "$eq" => "shoes" } }, + { "field" => "price", "condition" => { "$lt" => "100" } } + ] + } +) +``` + +## Custom embeddings + +If your documents already have embeddings, omit `model_id` and the SDK infers +the `custom` model automatically. All documents in a batch must either all have +embeddings or none: + +```ruby +docs = [ + Moss::DocumentInfo.new(id: "doc-1", text: "…", embedding: [0.1, 0.2, 0.3, 0.4]), + Moss::DocumentInfo.new(id: "doc-2", text: "…", embedding: [0.5, 0.6, 0.7, 0.8]) +] + +client.create_index("custom-embeddings", docs) +client.load_index("custom-embeddings") + +client.query("custom-embeddings", "", embedding: [0.1, 0.2, 0.3, 0.4], top_k: 5) +``` + +## Progress callbacks + +Long-running mutations poll an async job until completion. Pass `on_progress` +to observe it: + +```ruby +client.create_index("support-docs", documents, on_progress: lambda { |p| + puts "#{p.status} #{(p.progress * 100).round}%" +}) +``` + +## Sessions + +Build and query an index in memory, then push it to the cloud (sessions require +an enterprise plan): + +```ruby +session = client.session("scratch") +session.add_documents([Moss::DocumentInfo.new(id: "1", text: "hello world")]) +session.query("greeting", top_k: 3) +session.push_index +session.close +``` + +## API overview + +| Method | Description | +| --- | --- | +| `create_index(name, docs, model_id:, on_progress:)` | Create an index (polls to completion) | +| `add_documents(name, docs, upsert:, on_progress:)` | Add/upsert documents (`add_docs`) | +| `delete_documents(name, ids, on_progress:)` | Delete documents by id (`delete_docs`) | +| `get_job_status(job_id)` | Fetch async job status | +| `get_index(name)` / `list_indexes` / `delete_index(name)` | Index metadata management | +| `get_documents(name, doc_ids:)` | Read stored documents (`get_docs`) | +| `load_index(name, …)` / `unload_index(name)` | Manage the local runtime | +| `refresh_index(name)` / `get_index_info(name)` | Local index metadata | +| `query(name, text, top_k:, alpha:, embedding:, filter:)` | Search (`search`); local, else cloud | +| `session(name, model_id:)` | Open an in-memory session | +| `close` | Release native runtime handles | + +## Configuration + +| Environment variable | Purpose | +| --- | --- | +| `MOSS_PROJECT_ID` | Project id (required) | +| `MOSS_PROJECT_KEY` | Project key (required) | +| `MOSS_LIB_DIR` / `MOSS_LIBRARY_PATH` | Location of the `libmoss` runtime | +| `MOSS_CLOUD_API_MANAGE_URL` | Override the manage endpoint | +| `MOSS_CLOUD_QUERY_URL` | Override the cloud query endpoint | + +## Development + +```bash +cd sdks/ruby/sdk +bundle install +bundle exec rake test # unit tests (native/E2E auto-skip) +bundle exec rubocop # lint +``` + +Run the standalone samples in [`samples/`](samples), e.g.: + +```bash +MOSS_LIB_DIR=/path/to/libmoss/lib \ +MOSS_PROJECT_ID=… MOSS_PROJECT_KEY=… \ +ruby -Ilib -I../bindings/lib samples/comprehensive_sample.rb +``` + +### Live validation + +An end-to-end validation harness exercises the whole stack against your Moss +project. It reads credentials at runtime from a repo-root `.env` file and +auto-provisions `libmoss`: + +```bash +ruby sdks/ruby/sdk/scripts/validate.rb +``` + +## Integration tests + +Live tests auto-skip unless `MOSS_PROJECT_ID` / `MOSS_PROJECT_KEY` are set and +`libmoss` is available: + +```bash +MOSS_LIB_DIR=/path/to/libmoss/lib \ +MOSS_PROJECT_ID=… MOSS_PROJECT_KEY=… \ +ruby -Itest -Ilib -I../bindings/lib test/integration_test.rb +``` + +## License + +BSD 2-Clause. See [LICENSE](LICENSE). diff --git a/sdks/ruby/sdk/Rakefile b/sdks/ruby/sdk/Rakefile new file mode 100644 index 00000000..5ac3ec4f --- /dev/null +++ b/sdks/ruby/sdk/Rakefile @@ -0,0 +1,19 @@ +# frozen_string_literal: true + +require "rake/testtask" + +Rake::TestTask.new(:test) do |t| + t.libs << "test" + t.libs << "lib" + t.test_files = FileList["test/**/*_test.rb"] + t.warning = false +end + +begin + require "rubocop/rake_task" + RuboCop::RakeTask.new +rescue LoadError + # RuboCop is a development-only dependency; skip the task if it is absent. +end + +task default: :test diff --git a/sdks/ruby/sdk/lib/moss.rb b/sdks/ruby/sdk/lib/moss.rb new file mode 100644 index 00000000..5bc9eba2 --- /dev/null +++ b/sdks/ruby/sdk/lib/moss.rb @@ -0,0 +1,36 @@ +# frozen_string_literal: true + +require_relative "moss/version" +require_relative "moss/errors" +require_relative "moss/models" +require_relative "moss/cloud_query" +require_relative "moss/session" +require_relative "moss/client" + +# Moss is a real-time semantic search runtime for AI agents. This gem is the +# Ruby SDK: an ergonomic client over the native `libmoss` runtime for indexing, +# local sub-10ms search, and metadata filtering, with a cloud query fallback. +# +# require "moss" +# +# client = Moss::Client.new(project_id: "...", project_key: "...") +# client.create_index("support-docs", [ +# Moss::DocumentInfo.new(id: "1", text: "Refunds take 5-7 business days.", +# metadata: { "topic" => "refunds" }) +# ]) +# client.load_index("support-docs") +# result = client.query("support-docs", "how long do refunds take?", top_k: 3) +# result.docs.each { |doc| puts "#{doc.id} #{doc.score}" } +module Moss + # Returns the libmoss C SDK version reported by the loaded native runtime, or + # nil when libmoss is unavailable. + def self.libmoss_version + Moss::Core.libmoss_sdk_version + end + + # True when the native libmoss runtime can be loaded (local index + query). + # When false, the SDK still supports cloud-backed queries. + def self.native_runtime_available? + Moss::Core.available? + end +end diff --git a/sdks/ruby/sdk/lib/moss/client.rb b/sdks/ruby/sdk/lib/moss/client.rb new file mode 100644 index 00000000..b306d5af --- /dev/null +++ b/sdks/ruby/sdk/lib/moss/client.rb @@ -0,0 +1,488 @@ +# frozen_string_literal: true + +require "json" +require "moss/core" +require_relative "errors" +require_relative "models" +require_relative "cloud_query" +require_relative "session" + +module Moss + # The high-level Moss client. Wraps the native binding layer (Moss::Core) for + # mutations, local index loading and sub-10ms local queries, and falls back to + # the cloud query API when an index is not loaded locally (or libmoss is + # unavailable). + # + # Credentials are read from the constructor or, when omitted, from the + # MOSS_PROJECT_ID / MOSS_PROJECT_KEY environment variables. + # + # client = Moss::Client.new # creds from ENV + # client.create_index("docs", documents) + # client.load_index("docs") + # client.query("docs", "how do refunds work?", top_k: 3) + class Client + DEFAULT_MANAGE_URL = "https://service.usemoss.dev/v1/manage" + DEFAULT_TOP_K = Moss::Core::DEFAULT_TOP_K + DEFAULT_ALPHA = Moss::Core::DEFAULT_ALPHA + + DEFAULT_POLL_INTERVAL_SECONDS = 2.0 + DEFAULT_MUTATION_TIMEOUT_SECONDS = 30 * 60 + MAX_CONSECUTIVE_POLL_ERRORS = 3 + + # Call-shaping options for #query. Mirrors the fields other SDKs accept. + # The :filter member intentionally shadows Struct#filter; this object is a + # plain value holder and is never used as an Enumerable. + QueryOptions = Struct.new(:embedding, :top_k, :alpha, :filter, keyword_init: true) # rubocop:disable Lint/StructNewOverride + + def initialize(project_id: nil, project_key: nil, manage_url: nil, query_url: nil, + http_open_timeout: 10, http_read_timeout: 60, + poll_interval_seconds: DEFAULT_POLL_INTERVAL_SECONDS, + manage_factory: nil, index_factory: nil) + @project_id = string_or_nil(project_id) || string_or_nil(ENV.fetch("MOSS_PROJECT_ID", nil)) + @project_key = string_or_nil(project_key) || string_or_nil(ENV.fetch("MOSS_PROJECT_KEY", nil)) + @manage_url = string_or_nil(manage_url) || string_or_nil(ENV.fetch("MOSS_CLOUD_API_MANAGE_URL", + nil)) || DEFAULT_MANAGE_URL + @query_url = resolve_query_url(query_url) + @http_open_timeout = http_open_timeout + @http_read_timeout = http_read_timeout + @poll_interval_seconds = poll_interval_seconds + + # Runtime factories are injectable so the SDK can be unit-tested without + # libmoss present (mirrors the Go SDK's manageFactory/indexFactory). + @manage_factory = manage_factory || ->(id, key) { Moss::Core::ManageClient.new(id, key) } + @index_factory = index_factory || ->(id, key) { Moss::Core::IndexManager.new(id, key) } + + @manage_mutex = Mutex.new + @index_mutex = Mutex.new + @manage_client = nil + @index_manager = nil + @index_manager_unavailable = false + end + + # ---- manage: mutations ------------------------------------------------ + + def create_index(index_name, documents, model_id: nil, on_progress: nil) + validate_manage_request(index_name) + raise ArgumentError, "moss: documents must not be empty" if documents.nil? || documents.empty? + + resolved_model = resolve_model_id(documents, model_id) + validate_embedding_dimensions(documents, resolved_model) + + manage = ensure_manage_client + result = manage.create_index(index_name, to_core_documents(documents), resolved_model) + poll_job_until_complete(result, on_progress) + end + + def add_documents(index_name, documents, upsert: nil, on_progress: nil) + validate_manage_request(index_name) + raise ArgumentError, "moss: documents must not be empty" if documents.nil? || documents.empty? + + options = upsert.nil? ? nil : Moss::Core::MutationOptions.new(upsert: upsert) + manage = ensure_manage_client + result = manage.add_docs(index_name, to_core_documents(documents), options) + poll_job_until_complete(result, on_progress) + end + alias add_docs add_documents + + def delete_documents(index_name, doc_ids, on_progress: nil) + validate_manage_request(index_name) + raise ArgumentError, "moss: document IDs must not be empty" if doc_ids.nil? || doc_ids.empty? + + manage = ensure_manage_client + result = manage.delete_docs(index_name, Array(doc_ids).map(&:to_s)) + poll_job_until_complete(result, on_progress) + end + alias delete_docs delete_documents + + def get_job_status(job_id) + validate_credentials + raise ArgumentError, "moss: job ID must not be empty" if string_or_nil(job_id).nil? + + to_job_status(ensure_manage_client.get_job_status(job_id)) + end + + # ---- manage: reads ---------------------------------------------------- + + def get_index(index_name) + validate_manage_request(index_name) + to_index_info(ensure_manage_client.get_index(index_name)) + end + + def list_indexes + validate_credentials + ensure_manage_client.list_indexes.map { |info| to_index_info(info) } + end + + def delete_index(index_name) + validate_manage_request(index_name) + ensure_manage_client.delete_index(index_name) + end + + def get_documents(index_name, doc_ids: nil) + validate_manage_request(index_name) + ids = doc_ids.nil? ? [] : Array(doc_ids).map(&:to_s) + ensure_manage_client.get_docs(index_name, ids).map { |doc| to_document(doc) } + end + alias get_docs get_documents + + # ---- local index runtime --------------------------------------------- + + def load_index(index_name, auto_refresh: false, polling_interval_in_seconds: 0, cache_path: nil) + validate_manage_request(index_name) + unless cache_path.nil? || cache_path.to_s.strip.empty? + raise ArgumentError, "moss: cache_path is not supported by the current libmoss bindings" + end + + options = Moss::Core::LoadIndexOptions.new( + auto_refresh: auto_refresh, + polling_interval_secs: polling_interval_in_seconds + ) + manager = require_index_manager + info = manager.load_index(index_name, options) + manager.load_query_model(index_name) if info.model && info.model.id != Model::CUSTOM + info.name && !info.name.empty? ? info.name : index_name + end + + def unload_index(index_name) + raise ArgumentError, "moss: index name must not be empty" if string_or_nil(index_name).nil? + + require_index_manager.unload_index(index_name) + nil + end + + def refresh_index(index_name) + validate_manage_request(index_name) + to_refresh_result(require_index_manager.refresh_index(index_name)) + end + + def get_index_info(index_name) + validate_manage_request(index_name) + to_index_info(require_index_manager.get_index_info(index_name)) + end + + # ---- query (local, else cloud fallback) ------------------------------- + + def query(index_name, query_text = "", embedding: nil, top_k: nil, alpha: nil, filter: nil) + validate_query_request(index_name) + options = QueryOptions.new(embedding: embedding, top_k: top_k, alpha: alpha, filter: filter) + + manager = ensure_index_manager + if manager&.has_index?(index_name) + query_local(manager, index_name, query_text.to_s, options) + else + query_cloud(index_name, query_text.to_s, options) + end + end + alias search query + + # ---- sessions --------------------------------------------------------- + + def session(index_name, model_id: nil) + validate_manage_request(index_name) + options = model_id.nil? ? nil : Moss::Core::SessionOptions.new(model_id: model_id) + core_session = ensure_manage_client.session(index_name, options) + Session.new(core_session) + end + + # Releases the lazily created native runtime handles. + def close + manage = nil + manager = nil + @manage_mutex.synchronize do + manage = @manage_client + @manage_client = nil + end + @index_mutex.synchronize do + manager = @index_manager + @index_manager = nil + end + manage&.close + manager&.close + nil + end + + private + + # ---- query helpers ---------------------------------------------------- + + def query_local(manager, index_name, query_text, options) + top_k = positive_or_default(options.top_k, DEFAULT_TOP_K) + alpha = options.alpha.nil? ? DEFAULT_ALPHA : options.alpha.to_f + filter_json = + (JSON.generate(options.filter) if options.filter && !options.filter.empty?) + + result = manager.query( + index_name, query_text, + embedding: options.embedding, top_k: top_k, alpha: alpha, filter_json: filter_json + ) + to_search_result(result) + end + + def query_cloud(index_name, query_text, options) + CloudQuery.execute( + query_url: @query_url, + project_id: @project_id, + project_key: @project_key, + index_name: index_name, + query: query_text, + options: options, + http_open_timeout: @http_open_timeout, + http_read_timeout: @http_read_timeout + ) + end + + # ---- job polling ------------------------------------------------------ + + def poll_job_until_complete(mutation_result, on_progress) + completed = to_mutation_result(mutation_result) + deadline = monotonic_now + DEFAULT_MUTATION_TIMEOUT_SECONDS + consecutive_errors = 0 + + loop do + begin + status = get_job_status(mutation_result.job_id) + consecutive_errors = 0 + + on_progress&.call( + JobProgress.new( + job_id: status.job_id, + status: status.status, + progress: status.progress, + current_phase: status.current_phase + ) + ) + + case status.status + when JobStatus::COMPLETED + return completed + when JobStatus::FAILED + message = status.error && !status.error.empty? ? "moss: job failed: #{status.error}" : "moss: job failed" + raise JobError, message + end + rescue JobError + raise + rescue StandardError => e + consecutive_errors += 1 + if consecutive_errors >= MAX_CONSECUTIVE_POLL_ERRORS + raise JobError, "moss: job status polling failed after " \ + "#{MAX_CONSECUTIVE_POLL_ERRORS} consecutive errors: #{e.message}" + end + end + + raise JobError, "moss: timed out waiting for job #{mutation_result.job_id}" if monotonic_now > deadline + + sleep(@poll_interval_seconds) + end + end + + def monotonic_now + Process.clock_gettime(Process::CLOCK_MONOTONIC) + end + + # ---- model / embedding resolution ------------------------------------ + + def resolve_model_id(documents, model_id) + resolved = string_or_nil(model_id) + return resolved if resolved + return Model::CUSTOM if documents.any? { |doc| non_empty_embedding?(doc) } + + Model::MOSS_MINILM + end + + def validate_embedding_dimensions(documents, model_id) + with_embeddings = documents.select { |doc| non_empty_embedding?(doc) } + without_embeddings = documents.length - with_embeddings.length + + if !with_embeddings.empty? && without_embeddings.positive? + raise ArgumentError, + "moss: all documents must either all have embeddings or none should have embeddings" + end + + if with_embeddings.empty? + if model_id == Model::CUSTOM + raise ArgumentError, "moss: cannot use model \"#{Model::CUSTOM}\" without pre-computed embeddings" + end + + return + end + + dimension = embedding_of(with_embeddings.first).length + with_embeddings.each do |doc| + actual = embedding_of(doc).length + next if actual == dimension + + raise ArgumentError, + "moss: document \"#{doc.id}\" has mismatched embedding dimension (expected #{dimension}, got #{actual})" + end + end + + def non_empty_embedding?(doc) + embedding = embedding_of(doc) + embedding && !embedding.empty? + end + + def embedding_of(doc) + doc.respond_to?(:embedding) ? doc.embedding : doc[:embedding] + end + + # ---- runtime lifecycle ------------------------------------------------ + + def ensure_manage_client + @manage_mutex.synchronize do + return @manage_client if @manage_client + + begin + @manage_client = @manage_factory.call(@project_id, @project_key) + rescue Moss::Core::BindingsUnavailableError => e + raise ConfigurationError, e.message + end + end + end + + # Returns the IndexManager, or nil if libmoss is unavailable (query falls + # back to the cloud in that case). + def ensure_index_manager + @index_mutex.synchronize do + return @index_manager if @index_manager + return nil if @index_manager_unavailable + + begin + @index_manager = @index_factory.call(@project_id, @project_key) + rescue Moss::Core::BindingsUnavailableError + @index_manager_unavailable = true + nil + end + end + end + + # Like ensure_index_manager but raises when libmoss is unavailable — used by + # operations that have no cloud fallback (load/unload/refresh/local info). + def require_index_manager + manager = ensure_index_manager + raise ConfigurationError, Moss::Core::BindingsUnavailableError::DEFAULT_MESSAGE unless manager + + manager + end + + # ---- validation ------------------------------------------------------- + + def validate_manage_request(index_name) + validate_credentials + raise ArgumentError, "moss: index name must not be empty" if string_or_nil(index_name).nil? + end + alias validate_query_request validate_manage_request + + def validate_credentials + raise ConfigurationError, "moss: missing project ID" if @project_id.nil? + raise ConfigurationError, "moss: missing project key" if @project_key.nil? + end + + def resolve_query_url(explicit) + value = string_or_nil(explicit) || string_or_nil(ENV.fetch("MOSS_CLOUD_QUERY_URL", nil)) + return value if value + return nil if @manage_url.nil? || @manage_url.empty? + + # Derive the query endpoint from the manage endpoint only when the + # substitution actually changes the URL. If manage_url was overridden to a + # value without "/v1/manage", return nil so the cloud fallback fails fast + # with a clear ConfigurationError instead of silently POSTing to the + # manage endpoint; callers can set query_url / MOSS_CLOUD_QUERY_URL. + derived = @manage_url.sub("/v1/manage", "/query") + derived == @manage_url ? nil : derived + end + + # ---- conversions (Core -> Moss) --------------------------------------- + + def to_core_documents(documents) + documents.map do |doc| + Moss::Core::DocumentInfo.new( + id: doc.id.to_s, + text: (doc.text || "").to_s, + metadata: normalize_metadata(doc.metadata), + embedding: embedding_of(doc) + ) + end + end + + def normalize_metadata(metadata) + return nil if metadata.nil? || metadata.empty? + + metadata.each_with_object({}) do |(key, value), acc| + acc[key.to_s] = value.to_s + end + end + + def to_index_info(core) + IndexInfo.new( + id: core.id, + name: core.name, + version: core.version, + status: core.status, + doc_count: core.doc_count, + created_at: core.created_at, + updated_at: core.updated_at, + model: ModelRef.new(id: core.model&.id, version: core.model&.version) + ) + end + + def to_document(core) + DocumentInfo.new( + id: core.id, + text: core.text, + metadata: core.metadata, + embedding: core.embedding + ) + end + + def to_mutation_result(core) + MutationResult.new(job_id: core.job_id, index_name: core.index_name, doc_count: core.doc_count) + end + + def to_search_result(core) + docs = core.docs.map do |doc| + QueryResultDocument.new(id: doc.id, text: doc.text, metadata: doc.metadata, score: doc.score) + end + SearchResult.new( + docs: docs, + query: core.query, + index_name: core.index_name, + time_taken_ms: core.time_taken_ms + ) + end + + def to_job_status(core) + JobStatusResponse.new( + job_id: core.job_id, + status: core.status, + progress: core.progress, + current_phase: core.current_phase, + error: core.error, + created_at: core.created_at, + updated_at: core.updated_at, + completed_at: core.completed_at + ) + end + + def to_refresh_result(core) + RefreshResult.new( + index_name: core.index_name, + previous_updated_at: core.previous_updated_at, + new_updated_at: core.new_updated_at, + was_updated: core.was_updated + ) + end + + # ---- misc ------------------------------------------------------------- + + def positive_or_default(value, default) + value&.to_i&.positive? ? value.to_i : default + end + + def string_or_nil(value) + return nil if value.nil? + + stripped = value.to_s.strip + stripped.empty? ? nil : stripped + end + end +end diff --git a/sdks/ruby/sdk/lib/moss/cloud_query.rb b/sdks/ruby/sdk/lib/moss/cloud_query.rb new file mode 100644 index 00000000..ab4beb85 --- /dev/null +++ b/sdks/ruby/sdk/lib/moss/cloud_query.rb @@ -0,0 +1,86 @@ +# frozen_string_literal: true + +require "json" +require "net/http" +require "uri" +require_relative "errors" +require_relative "models" + +module Moss + # Cloud query fallback used when an index is not loaded into the local runtime + # (or when libmoss is unavailable entirely). Mirrors the Go SDK's queryCloud: + # a single POST to the derived query endpoint. Alpha and metadata filters are + # local-only; requesting them here raises UnsupportedQueryError. + module CloudQuery + module_function + + DEFAULT_TOP_K = 10 + + def execute(query_url:, project_id:, project_key:, index_name:, query:, options:, http_open_timeout:, + http_read_timeout:) + raise ConfigurationError, "moss: query URL is not configured" if query_url.nil? || query_url.strip.empty? + + top_k = DEFAULT_TOP_K + embedding = nil + if options + if options.alpha || (options.filter && !options.filter.empty?) + raise UnsupportedQueryError, + "moss: alpha and filter query options require a locally loaded index; call load_index first" + end + top_k = options.top_k if options.top_k&.positive? + embedding = options.embedding if options.embedding && !options.embedding.empty? + end + + payload = { + query: query, + indexName: index_name, + projectId: project_id, + projectKey: project_key, + topK: top_k + } + payload[:queryEmbedding] = embedding if embedding + + response = post_json(query_url, payload, http_open_timeout, http_read_timeout) + parse_response(response, query) + end + + def post_json(url, payload, open_timeout, read_timeout) + uri = URI.parse(url) + http = Net::HTTP.new(uri.host, uri.port) + http.use_ssl = (uri.scheme == "https") + http.open_timeout = open_timeout + http.read_timeout = read_timeout + + request = Net::HTTP::Post.new(uri.request_uri) + request["Content-Type"] = "application/json" + request.body = JSON.generate(payload) + + http.request(request) + end + + def parse_response(response, query) + code = response.code.to_i + unless code >= 200 && code < 300 + body = response.body.to_s[0, 16 * 1024].strip + raise HTTPError.new(status_code: code, body: body) + end + + data = JSON.parse(response.body || "{}") + docs = Array(data["docs"]).map do |doc| + QueryResultDocument.new( + id: doc["id"], + text: doc["text"], + metadata: doc["metadata"], + score: doc["score"] + ) + end + + SearchResult.new( + docs: docs, + query: data.fetch("query", query), + index_name: data["indexName"], + time_taken_ms: data["timeTakenMs"] + ) + end + end +end diff --git a/sdks/ruby/sdk/lib/moss/errors.rb b/sdks/ruby/sdk/lib/moss/errors.rb new file mode 100644 index 00000000..4d497a30 --- /dev/null +++ b/sdks/ruby/sdk/lib/moss/errors.rb @@ -0,0 +1,34 @@ +# frozen_string_literal: true + +module Moss + # Base class for every error raised by the Moss Ruby SDK. + class Error < StandardError; end + + # Raised when the client is missing required configuration (project id/key or + # a query URL for the cloud fallback). + class ConfigurationError < Error; end + + # Raised for invalid arguments before a request is issued. + class ArgumentError < Error; end + + # Raised when a mutation job ends in a failed state, or polling exhausts its + # retry budget. + class JobError < Error; end + + # Raised when the cloud query fallback returns a non-2xx response. + class HTTPError < Error + attr_reader :status_code, :body + + def initialize(status_code:, body: nil) + @status_code = status_code + @body = body + message = "moss: cloud query failed with status #{status_code}" + message += ": #{body}" if body && !body.empty? + super(message) + end + end + + # Raised when a query uses local-only options (alpha/filter) but no index is + # loaded locally and the request would fall back to the cloud API. + class UnsupportedQueryError < Error; end +end diff --git a/sdks/ruby/sdk/lib/moss/models.rb b/sdks/ruby/sdk/lib/moss/models.rb new file mode 100644 index 00000000..cb0c7951 --- /dev/null +++ b/sdks/ruby/sdk/lib/moss/models.rb @@ -0,0 +1,70 @@ +# frozen_string_literal: true + +module Moss + # Embedding models that can back an index. + module Model + MOSS_MINILM = "moss-minilm" + MOSS_MEDIUMLM = "moss-mediumlm" + CUSTOM = "custom" + end + + # Lifecycle states reported for an index. + module IndexStatus + NOT_STARTED = "NotStarted" + BUILDING = "Building" + READY = "Ready" + FAILED = "Failed" + end + + # Lifecycle states reported for an async mutation job. + module JobStatus + PENDING_UPLOAD = "pending_upload" + UPLOADING = "uploading" + BUILDING = "building" + COMPLETED = "completed" + FAILED = "failed" + end + + # A document to index or one already stored. `metadata` is a String=>String + # map; `embedding` is an optional array of floats for custom-embedding indexes. + DocumentInfo = Struct.new(:id, :text, :metadata, :embedding, keyword_init: true) do + def initialize(id:, text: nil, metadata: nil, embedding: nil) + super + end + end + + # A single scored result from a query. + QueryResultDocument = Struct.new(:id, :text, :metadata, :score, keyword_init: true) + + # The response returned by Client#query / Client#search. + SearchResult = Struct.new(:docs, :query, :index_name, :time_taken_ms, keyword_init: true) + + # Points at the embedding model backing an index. + ModelRef = Struct.new(:id, :version, keyword_init: true) + + # Persisted metadata for an index. + IndexInfo = Struct.new( + :id, :name, :version, :status, :doc_count, + :created_at, :updated_at, :model, + keyword_init: true + ) + + # Returned when a mutation job completes. + MutationResult = Struct.new(:job_id, :index_name, :doc_count, keyword_init: true) + + # Emitted to an on_progress callback while a mutation job runs. + JobProgress = Struct.new(:job_id, :status, :progress, :current_phase, keyword_init: true) + + # Persisted status view for a mutation job. + JobStatusResponse = Struct.new( + :job_id, :status, :progress, :current_phase, :error, + :created_at, :updated_at, :completed_at, + keyword_init: true + ) + + # Outcome of a local RefreshIndex. + RefreshResult = Struct.new( + :index_name, :previous_updated_at, :new_updated_at, :was_updated, + keyword_init: true + ) +end diff --git a/sdks/ruby/sdk/lib/moss/session.rb b/sdks/ruby/sdk/lib/moss/session.rb new file mode 100644 index 00000000..f7cdf977 --- /dev/null +++ b/sdks/ruby/sdk/lib/moss/session.rb @@ -0,0 +1,99 @@ +# frozen_string_literal: true + +require_relative "models" + +module Moss + # High-level wrapper around a native Moss::Core::Session. Build an index in + # memory, query it locally, then push it to the cloud. Obtained via + # Client#session. + # + # session = client.session("scratch") + # session.add_documents([Moss::DocumentInfo.new(id: "1", text: "hello")]) + # session.query("greeting") + # session.push_index + # session.close + class Session + def initialize(core_session) + @core = core_session + end + + def name + @core.name + end + + def doc_count + @core.doc_count + end + + def add_documents(documents, upsert: nil) + options = upsert.nil? ? nil : Moss::Core::AddDocsOptions.new(upsert: upsert) + result = @core.add_docs(to_core_documents(documents), options) + { added: result.added, updated: result.updated } + end + alias add_docs add_documents + + def delete_documents(doc_ids) + @core.delete_docs(Array(doc_ids).map(&:to_s)) + end + alias delete_docs delete_documents + + def get_documents(doc_ids = nil) + ids = doc_ids.nil? ? [] : Array(doc_ids).map(&:to_s) + @core.get_docs(ids).map { |doc| to_document(doc) } + end + alias get_docs get_documents + + def query(query_text, embedding: nil, top_k: Moss::Core::DEFAULT_TOP_K, + alpha: Moss::Core::DEFAULT_ALPHA, filter: nil) + filter_json = filter && !filter.empty? ? JSON.generate(filter) : nil + result = @core.query( + query_text, embedding: embedding, top_k: top_k, alpha: alpha, filter_json: filter_json + ) + to_search_result(result) + end + alias search query + + def load_index(index_name) + @core.load_index(index_name) + end + + def push_index + result = @core.push_index + MutationResult.new(job_id: result.job_id, index_name: result.index_name, doc_count: result.doc_count) + end + + def close + @core.close + end + + private + + def to_core_documents(documents) + documents.map do |doc| + Moss::Core::DocumentInfo.new( + id: doc.id.to_s, + text: (doc.text || "").to_s, + metadata: normalize_metadata(doc.metadata), + embedding: doc.respond_to?(:embedding) ? doc.embedding : nil + ) + end + end + + def normalize_metadata(metadata) + return nil if metadata.nil? || metadata.empty? + + metadata.each_with_object({}) { |(k, v), acc| acc[k.to_s] = v.to_s } + end + + def to_document(core) + DocumentInfo.new(id: core.id, text: core.text, metadata: core.metadata, embedding: core.embedding) + end + + def to_search_result(core) + docs = core.docs.map do |doc| + QueryResultDocument.new(id: doc.id, text: doc.text, metadata: doc.metadata, score: doc.score) + end + SearchResult.new(docs: docs, query: core.query, index_name: core.index_name, time_taken_ms: core.time_taken_ms) + end + end +end diff --git a/sdks/ruby/sdk/lib/moss/version.rb b/sdks/ruby/sdk/lib/moss/version.rb new file mode 100644 index 00000000..37981ac4 --- /dev/null +++ b/sdks/ruby/sdk/lib/moss/version.rb @@ -0,0 +1,5 @@ +# frozen_string_literal: true + +module Moss + VERSION = "0.1.0" +end diff --git a/sdks/ruby/sdk/moss.gemspec b/sdks/ruby/sdk/moss.gemspec new file mode 100644 index 00000000..9c33a965 --- /dev/null +++ b/sdks/ruby/sdk/moss.gemspec @@ -0,0 +1,43 @@ +# frozen_string_literal: true + +require_relative "lib/moss/version" + +Gem::Specification.new do |spec| + spec.name = "moss" + spec.version = Moss::VERSION + spec.authors = ["Moss"] + spec.summary = "Ruby SDK for Moss — real-time on-device semantic search." + spec.description = <<~DESC + The Moss Ruby SDK provides an ergonomic client over the Moss runtime for + indexing, sub-10ms local semantic search, and metadata filtering, with a + cloud query fallback. Local search is powered by the native libmoss runtime + via the moss-core bindings; download libmoss from the usemoss/moss c-sdk + releases and point MOSS_LIB_DIR at it to enable local operations. + DESC + spec.homepage = "https://github.com/usemoss/moss" + spec.license = "BSD-2-Clause" + + spec.required_ruby_version = ">= 3.0" + + spec.metadata = { + "homepage_uri" => spec.homepage, + "source_code_uri" => "https://github.com/usemoss/moss/tree/main/sdks/ruby/sdk", + "documentation_uri" => "https://docs.moss.dev", + "changelog_uri" => "https://github.com/usemoss/moss/blob/main/sdks/ruby/sdk/CHANGELOG.md", + "rubygems_mfa_required" => "true" + } + + spec.files = Dir[ + "lib/**/*.rb", + "README.md", + "LICENSE", + "CHANGELOG.md" + ] + spec.require_paths = ["lib"] + + spec.add_dependency "moss-core", ">= 0.9", "< 1.0" + + spec.add_development_dependency "minitest", "~> 5.0" + spec.add_development_dependency "rake", "~> 13.0" + spec.add_development_dependency "rubocop", "~> 1.60" +end diff --git a/sdks/ruby/sdk/samples/.env.template b/sdks/ruby/sdk/samples/.env.template new file mode 100644 index 00000000..b4ecb687 --- /dev/null +++ b/sdks/ruby/sdk/samples/.env.template @@ -0,0 +1,7 @@ +# Copy to .env and fill in your Moss project credentials (from https://moss.dev). +MOSS_PROJECT_ID=your-project-id +MOSS_PROJECT_KEY=your-project-key + +# Location of the native libmoss runtime (download from the c-sdk release). +# Point this at the directory containing libmoss.dylib / libmoss.so. +MOSS_LIB_DIR=/path/to/libmoss/lib diff --git a/sdks/ruby/sdk/samples/comprehensive_sample.rb b/sdks/ruby/sdk/samples/comprehensive_sample.rb new file mode 100644 index 00000000..2e03c564 --- /dev/null +++ b/sdks/ruby/sdk/samples/comprehensive_sample.rb @@ -0,0 +1,68 @@ +# frozen_string_literal: true + +# Comprehensive Moss Ruby SDK sample: create an index, load it locally, run a +# semantic query, read documents, and clean up. +# +# Run from the repo: +# MOSS_LIB_DIR=/path/to/libmoss/lib \ +# MOSS_PROJECT_ID=... MOSS_PROJECT_KEY=... \ +# ruby sdks/ruby/sdk/samples/comprehensive_sample.rb + +$LOAD_PATH.unshift(File.expand_path("../lib", __dir__)) +$LOAD_PATH.unshift(File.expand_path("../../bindings/lib", __dir__)) + +require "moss" + +client = Moss::Client.new +index_name = "support-docs-sample" + +documents = [ + Moss::DocumentInfo.new( + id: "doc-1", + text: "Refunds are processed within five to seven business days.", + metadata: { "topic" => "refunds" } + ), + Moss::DocumentInfo.new( + id: "doc-2", + text: "Orders can be tracked from the account dashboard.", + metadata: { "topic" => "shipping" } + ), + Moss::DocumentInfo.new( + id: "doc-3", + text: "Contact support any time via live chat.", + metadata: { "topic" => "support" } + ) +] + +begin + puts "Creating index '#{index_name}'..." + result = client.create_index(index_name, documents, on_progress: lambda { |p| + puts " #{p.status} #{(p.progress * 100).round}%" + }) + puts "Created: job=#{result.job_id} docs=#{result.doc_count}" + + info = client.get_index(index_name) + puts "Index status: #{info.status} (model #{info.model.id})" + + puts "Loading index locally..." + client.load_index(index_name) + + puts "Querying: 'how long do refunds take?'" + search = client.query(index_name, "how long do refunds take?", top_k: 3) + search.docs.each do |doc| + puts " #{doc.id} score=#{format("%.3f", doc.score)} #{doc.text}" + end + puts " (#{search.time_taken_ms} ms)" + + puts "Stored documents:" + client.get_documents(index_name).each { |doc| puts " #{doc.id}: #{doc.metadata.inspect}" } +ensure + puts "Cleaning up..." + begin + client.unload_index(index_name) + rescue StandardError + # ignore + end + client.delete_index(index_name) + client.close +end diff --git a/sdks/ruby/sdk/samples/custom_embeddings_sample.rb b/sdks/ruby/sdk/samples/custom_embeddings_sample.rb new file mode 100644 index 00000000..cd20a4a3 --- /dev/null +++ b/sdks/ruby/sdk/samples/custom_embeddings_sample.rb @@ -0,0 +1,42 @@ +# frozen_string_literal: true + +# Custom embeddings sample: index documents that already carry vectors and query +# by a raw embedding. When documents provide embeddings, the SDK infers the +# `custom` model automatically. +# +# Run: +# MOSS_LIB_DIR=/path/to/libmoss/lib \ +# MOSS_PROJECT_ID=... MOSS_PROJECT_KEY=... \ +# ruby sdks/ruby/sdk/samples/custom_embeddings_sample.rb + +$LOAD_PATH.unshift(File.expand_path("../lib", __dir__)) +$LOAD_PATH.unshift(File.expand_path("../../bindings/lib", __dir__)) + +require "moss" + +client = Moss::Client.new +index_name = "custom-embeddings-sample" + +documents = [ + Moss::DocumentInfo.new(id: "doc-1", text: "First custom vector.", embedding: [1.0, 0.0, 0.0, 0.0]), + Moss::DocumentInfo.new(id: "doc-2", text: "Second custom vector.", embedding: [0.0, 1.0, 0.0, 0.0]), + Moss::DocumentInfo.new(id: "doc-3", text: "Third custom vector.", embedding: [0.0, 0.0, 1.0, 0.0]) +] + +begin + puts "Creating custom-embedding index (model inferred as 'custom')..." + client.create_index(index_name, documents) + client.load_index(index_name) + + puts "Querying by embedding [1, 0, 0, 0]..." + result = client.query(index_name, "", embedding: [1.0, 0.0, 0.0, 0.0], top_k: 3) + result.docs.each { |doc| puts " #{doc.id} score=#{format("%.3f", doc.score)}" } +ensure + begin + client.unload_index(index_name) + rescue StandardError + # ignore + end + client.delete_index(index_name) + client.close +end diff --git a/sdks/ruby/sdk/samples/metadata_filtering_sample.rb b/sdks/ruby/sdk/samples/metadata_filtering_sample.rb new file mode 100644 index 00000000..27676730 --- /dev/null +++ b/sdks/ruby/sdk/samples/metadata_filtering_sample.rb @@ -0,0 +1,60 @@ +# frozen_string_literal: true + +# Metadata filtering sample: create a small catalog, load it locally, and run +# queries filtered with $eq, $and, and $in operators. +# +# Filters require a locally loaded index and use the engine's filter schema: +# each leaf is { "field" => name, "condition" => { "$op" => value } }. +# +# Run: +# MOSS_LIB_DIR=/path/to/libmoss/lib \ +# MOSS_PROJECT_ID=... MOSS_PROJECT_KEY=... \ +# ruby sdks/ruby/sdk/samples/metadata_filtering_sample.rb + +$LOAD_PATH.unshift(File.expand_path("../lib", __dir__)) +$LOAD_PATH.unshift(File.expand_path("../../bindings/lib", __dir__)) + +require "moss" + +client = Moss::Client.new +index_name = "catalog-filter-sample" + +documents = [ + Moss::DocumentInfo.new(id: "p-1", text: "Trail running shoes with grippy soles.", + metadata: { "category" => "shoes", "city" => "portland", "price" => "80" }), + Moss::DocumentInfo.new(id: "p-2", text: "Waterproof hiking boots.", + metadata: { "category" => "shoes", "city" => "denver", "price" => "140" }), + Moss::DocumentInfo.new(id: "p-3", text: "Lightweight rain jacket.", + metadata: { "category" => "outerwear", "city" => "portland", "price" => "120" }) +] + +def show(label, result) + puts label + result.docs.each { |doc| puts " #{doc.id} score=#{format("%.3f", doc.score)} #{doc.text}" } +end + +eq_filter = { "field" => "category", "condition" => { "$eq" => "shoes" } } +and_filter = { + "$and" => [ + { "field" => "category", "condition" => { "$eq" => "shoes" } }, + { "field" => "price", "condition" => { "$lt" => "100" } } + ] +} +in_filter = { "field" => "city", "condition" => { "$in" => ["portland"] } } + +begin + client.create_index(index_name, documents) + client.load_index(index_name) + + show("$eq category == shoes:", client.query(index_name, "footwear", top_k: 5, filter: eq_filter)) + show("$and shoes AND price < 100:", client.query(index_name, "footwear", top_k: 5, filter: and_filter)) + show("$in city in [portland]:", client.query(index_name, "gear", top_k: 5, filter: in_filter)) +ensure + begin + client.unload_index(index_name) + rescue StandardError + # ignore + end + client.delete_index(index_name) + client.close +end diff --git a/sdks/ruby/sdk/samples/session_usage_sample.rb b/sdks/ruby/sdk/samples/session_usage_sample.rb new file mode 100644 index 00000000..b7d583ba --- /dev/null +++ b/sdks/ruby/sdk/samples/session_usage_sample.rb @@ -0,0 +1,41 @@ +# frozen_string_literal: true + +# Session sample: build an index in memory, query it locally, then push it to +# the cloud. Sessions require an enterprise plan. +# +# Run: +# MOSS_LIB_DIR=/path/to/libmoss/lib \ +# MOSS_PROJECT_ID=... MOSS_PROJECT_KEY=... \ +# ruby sdks/ruby/sdk/samples/session_usage_sample.rb + +$LOAD_PATH.unshift(File.expand_path("../lib", __dir__)) +$LOAD_PATH.unshift(File.expand_path("../../bindings/lib", __dir__)) + +require "moss" + +client = Moss::Client.new +session = client.session("scratch-session-sample") + +begin + session.add_documents([ + Moss::DocumentInfo.new(id: "1", text: "Cats are small domesticated carnivores.", + metadata: { "kind" => "cat" }), + Moss::DocumentInfo.new(id: "2", text: "Dogs are loyal companion animals.", + metadata: { "kind" => "dog" }) + ]) + puts "Session holds #{session.doc_count} documents." + + result = session.query("feline pet", top_k: 2) + result.docs.each { |doc| puts " #{doc.id} score=#{format("%.3f", doc.score)} #{doc.text}" } + + puts "Pushing session to the cloud..." + push = session.push_index + puts "Pushed: job=#{push.job_id} docs=#{push.doc_count}" +rescue Moss::Core::NativeError, Moss::Error => e + raise unless e.message.match?(/enterprise|plan not allowed/i) + + puts "Sessions require an enterprise plan; skipping." +ensure + session.close + client.close +end diff --git a/sdks/ruby/sdk/scripts/validate.rb b/sdks/ruby/sdk/scripts/validate.rb new file mode 100755 index 00000000..85f64ca9 --- /dev/null +++ b/sdks/ruby/sdk/scripts/validate.rb @@ -0,0 +1,352 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true + +# Live end-to-end validation harness for the Moss Ruby SDK. +# +# This is the ONLY component that touches real credentials, and it does so at +# runtime only: it reads MOSS_PROJECT_ID / MOSS_PROJECT_KEY from the repo-root +# `.env` file into the process environment, never printing or persisting them. +# Every line written to stdout/stderr is passed through a scrubber that redacts +# the secret values as a defense-in-depth measure. +# +# It also auto-provisions the native libmoss C SDK into a gitignored vendor dir +# so the whole round-trip (create -> load -> query -> filter -> delete) runs +# locally. +# +# Usage: +# ruby sdks/ruby/sdk/scripts/validate.rb +# +# Exit codes: 0 = all checks passed, 1 = a check failed, 2 = misconfiguration. + +require "fileutils" + +SCRIPT_DIR = __dir__ +REPO_ROOT = File.expand_path("../../../..", SCRIPT_DIR) +SDK_LIB = File.expand_path("../lib", SCRIPT_DIR) +BINDINGS_LIB = File.expand_path("../../bindings/lib", SCRIPT_DIR) +$LOAD_PATH.unshift(SDK_LIB, BINDINGS_LIB) + +LIBMOSS_VERSION = "0.9.0" +VENDOR_DIR = File.join(REPO_ROOT, "sdks", "ruby", ".libmoss") +ENV_FILE = File.join(REPO_ROOT, ".env") +SECRET_KEYS = %w[MOSS_PROJECT_ID MOSS_PROJECT_KEY].freeze + +# --- credential loading (runtime only; values never printed) --------------- + +def load_env_file(path) + return unless File.exist?(path) + + File.foreach(path) do |raw| + line = raw.strip + next if line.empty? || line.start_with?("#") + + key, sep, value = line.partition("=") + next if sep.empty? + + key = key.sub(/\Aexport\s+/, "").strip + next if key.empty? + + value = value.strip + if value.length >= 2 && + ((value.start_with?('"') && value.end_with?('"')) || + (value.start_with?("'") && value.end_with?("'"))) + value = value[1..-2] + end + ENV[key] ||= value + end +end + +# Redacts secret values from any text before it is displayed. +def build_scrubber + secrets = SECRET_KEYS.map { |k| ENV.fetch(k, nil) }.compact.map(&:to_s).reject(&:empty?) + lambda do |text| + result = text.to_s + secrets.each { |value| result = result.gsub(value, "[REDACTED]") } + result + end +end + +# --- libmoss provisioning --------------------------------------------------- + +def libmoss_target + case RUBY_PLATFORM + when /arm64-darwin/, /aarch64-darwin/ then "aarch64-apple-darwin" + when /x86_64-darwin/ then nil # no x86_64 macOS build in this release + when /x86_64-linux/ then "x86_64-unknown-linux-gnu" + when /aarch64-linux/, /arm64-linux/ then "aarch64-unknown-linux-gnu" + end +end + +def libmoss_filename + RUBY_PLATFORM.include?("darwin") ? "libmoss.dylib" : "libmoss.so" +end + +def ensure_libmoss(log) + return if ENV["MOSS_LIB_DIR"] || ENV["MOSS_LIBRARY_PATH"] + + target = libmoss_target + unless target + log.call("! No prebuilt libmoss for #{RUBY_PLATFORM}; set MOSS_LIB_DIR manually.") + return + end + + base = "libmoss-v#{LIBMOSS_VERSION}-#{target}" + lib_dir = File.join(VENDOR_DIR, base, "lib") + lib_file = File.join(lib_dir, libmoss_filename) + + unless File.exist?(lib_file) + FileUtils.mkdir_p(VENDOR_DIR) + archive = "#{base}.tar.gz" + url = "https://github.com/usemoss/moss/releases/download/c-sdk-v#{LIBMOSS_VERSION}/#{archive}" + dest = File.join(VENDOR_DIR, archive) + log.call("• Downloading libmoss (#{target})…") + unless system("curl", "-sSL", "--fail", "-o", dest, url) + log.call("! Failed to download libmoss from #{url}") + return + end + system("tar", "xzf", dest, "-C", VENDOR_DIR) + FileUtils.rm_f(dest) + end + + ENV["MOSS_LIB_DIR"] = lib_dir if File.exist?(lib_file) +end + +# --- validation harness ----------------------------------------------------- + +class Checks + def initialize(scrubber) + @scrubber = scrubber + @failures = 0 + end + + def log(message) + puts @scrubber.call(message) + end + + def check(label, condition) + if condition + log(" ✓ #{label}") + else + @failures += 1 + log(" ✗ #{label}") + end + end + + attr_reader :failures +end + +def run_validation(checks) + require "moss" + + checks.log("Moss Ruby SDK — live validation") + checks.log("• moss gem: v#{Moss::VERSION}") + checks.log("• native runtime available: #{Moss.native_runtime_available?}") + checks.check("libmoss loads and reports a version", !Moss.libmoss_version.nil?) + checks.log("• libmoss C SDK: v#{Moss.libmoss_version}") + + client = Moss::Client.new(poll_interval_seconds: 2) + index_name = "ruby-sdk-validate-#{Process.pid}-#{rand(100_000)}" + checks.log("• index: #{index_name}") + + documents = [ + Moss::DocumentInfo.new( + id: "doc-1", + text: "Refunds are processed within five to seven business days.", + metadata: { "topic" => "refunds" } + ), + Moss::DocumentInfo.new( + id: "doc-2", + text: "Orders can be tracked from the account dashboard.", + metadata: { "topic" => "shipping" } + ) + ] + + begin + checks.log("→ create_index (polls job to completion)…") + create = client.create_index(index_name, documents) + checks.check("create_index returned doc_count == 2", create.doc_count == 2) + + checks.log("→ get_index…") + info = client.get_index(index_name) + checks.check("get_index name matches", info.name == index_name) + + checks.log("→ load_index…") + loaded = client.load_index(index_name) + checks.check("load_index returned the index name", loaded == index_name) + + checks.log("→ local query…") + result = client.query(index_name, "how long do refunds take?", top_k: 3) + checks.check("query returned results", !result.docs.empty?) + checks.check("top hit is the refunds doc", result.docs.first&.id == "doc-1") + top = result.docs.first + checks.log(" top: id=#{top&.id} score=#{format("%.4f", top&.score.to_f)} (#{result.time_taken_ms}ms)") + + checks.log("→ metadata-filtered query ($eq topic=shipping)…") + filtered = client.query( + index_name, "how long do refunds take?", + top_k: 3, filter: { "field" => "topic", "condition" => { "$eq" => "shipping" } } + ) + only_shipping = filtered.docs.all? { |d| d.metadata.nil? || d.metadata["topic"] == "shipping" } + checks.check("filter restricted results to topic=shipping", only_shipping) + + checks.log("→ metadata-filtered query ($in topic in [shipping])…") + in_filtered = client.query( + index_name, "orders", + top_k: 3, filter: { "field" => "topic", "condition" => { "$in" => ["shipping"] } } + ) + in_only_shipping = in_filtered.docs.all? { |d| d.metadata.nil? || d.metadata["topic"] == "shipping" } + checks.check("$in filter restricted results to topic=shipping", in_only_shipping) + + checks.log("→ get_documents…") + fetched = client.get_documents(index_name) + checks.check("get_documents returned stored docs", fetched.length >= 2) + + checks.log("→ list_indexes…") + listed = client.list_indexes + checks.check("list_indexes includes the new index", listed.any? { |i| i.name == index_name }) + + validate_custom_embeddings(client, checks) + validate_session(client, checks) + ensure + checks.log("→ cleanup (unload + delete_index)…") + begin + client.unload_index(index_name) + rescue StandardError + # ignore + end + begin + deleted = client.delete_index(index_name) + checks.check("delete_index reported success", deleted == true) + rescue StandardError => e + checks.log(" ! delete_index error: #{e.class}") + end + client.close + end +end + +def wait_for_job(client, job_id, timeout: 180) + deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + timeout + loop do + status = client.get_job_status(job_id) + return status.status if %w[completed failed].include?(status.status) + return "timeout" if Process.clock_gettime(Process::CLOCK_MONOTONIC) > deadline + + sleep 2 + end +end + +def validate_custom_embeddings(client, checks) + index_name = "ruby-sdk-custom-#{Process.pid}-#{rand(100_000)}" + checks.log("→ custom embeddings: create #{index_name} (model inferred)…") + documents = [ + Moss::DocumentInfo.new(id: "c-1", text: "first vector", embedding: [1.0, 0.0, 0.0, 0.0]), + Moss::DocumentInfo.new(id: "c-2", text: "second vector", embedding: [0.0, 1.0, 0.0, 0.0]), + Moss::DocumentInfo.new(id: "c-3", text: "third vector", embedding: [0.0, 0.0, 1.0, 0.0]) + ] + + begin + client.create_index(index_name, documents) + info = client.get_index(index_name) + checks.check("custom index uses the custom model", info.model.id == Moss::Model::CUSTOM) + + client.load_index(index_name) + result = client.query(index_name, "", embedding: [1.0, 0.0, 0.0, 0.0], top_k: 3) + checks.check("embedding query returned results", !result.docs.empty?) + checks.check("embedding query top hit is c-1", result.docs.first&.id == "c-1") + ensure + begin + client.unload_index(index_name) + rescue StandardError + # ignore + end + begin + client.delete_index(index_name) + rescue StandardError + # ignore + end + end +end + +# Opens a session, or returns nil (logging a skip) when the account plan does +# not include sessions. Any non-plan error propagates. +def open_session_or_skip(client, session_index, checks) + client.session(session_index) +rescue Moss::Core::NativeError, Moss::Error => e + raise unless e.message.match?(/enterprise|plan not allowed/i) + + checks.log(" ⊘ sessions skipped (requires enterprise plan)") + nil +end + +def validate_session(client, checks) + session_index = "ruby-sdk-session-#{Process.pid}-#{rand(100_000)}" + checks.log("→ session: open #{session_index}…") + session = open_session_or_skip(client, session_index, checks) + return if session.nil? + + begin + session.add_documents( + [ + Moss::DocumentInfo.new(id: "s-1", text: "Cats are small domesticated carnivores.", + metadata: { "kind" => "cat" }), + Moss::DocumentInfo.new(id: "s-2", text: "Dogs are loyal companion animals.", + metadata: { "kind" => "dog" }) + ] + ) + checks.check("session doc_count == 2", session.doc_count == 2) + + sres = session.query("feline pet", top_k: 2) + checks.check("session local query returned results", !sres.docs.empty?) + checks.check("session top hit is the cat doc", sres.docs.first&.id == "s-1") + + fetched = session.get_documents(["s-1"]) + checks.check("session get_documents returns requested doc", fetched.first&.id == "s-1") + + checks.log("→ session: push_index to cloud…") + push = session.push_index + checks.check("session push_index returned a job id", !push.job_id.to_s.empty?) + checks.log(" push job status: #{wait_for_job(client, push.job_id)}") + ensure + session.close + begin + client.unload_index(session_index) + rescue StandardError + # not necessarily loaded + end + begin + client.delete_index(session_index) + rescue StandardError + # best-effort cleanup of the pushed index + end + end +end + +# --- main ------------------------------------------------------------------- + +load_env_file(ENV_FILE) +scrubber = build_scrubber + +if SECRET_KEYS.any? { |k| ENV[k].to_s.strip.empty? } + warn scrubber.call("Missing credentials: ensure MOSS_PROJECT_ID and MOSS_PROJECT_KEY are set in #{ENV_FILE}") + exit 2 +end + +checks = Checks.new(scrubber) +ensure_libmoss(->(m) { checks.log(m) }) + +begin + run_validation(checks) +rescue StandardError => e + # Scrub the message in case any credential value leaked into it. + checks.log("FATAL: #{e.class}: #{scrubber.call(e.message)}") + checks.log(scrubber.call(e.backtrace.first(5).join("\n"))) + exit 1 +end + +if checks.failures.zero? + checks.log("\nAll checks passed ✅") + exit 0 +else + checks.log("\n#{checks.failures} check(s) failed ❌") + exit 1 +end diff --git a/sdks/ruby/sdk/test/client_test.rb b/sdks/ruby/sdk/test/client_test.rb new file mode 100644 index 00000000..9f35d314 --- /dev/null +++ b/sdks/ruby/sdk/test/client_test.rb @@ -0,0 +1,241 @@ +# frozen_string_literal: true + +require_relative "test_helper" + +class ClientTest < Minitest::Test + def build_client(manage: nil, index: nil, **opts) + Moss::Client.new( + project_id: "pid", + project_key: "pkey", + manage_factory: manage ? ->(_id, _key) { manage } : nil, + index_factory: index ? ->(_id, _key) { index } : nil, + poll_interval_seconds: 0, + **opts + ) + end + + def sample_docs + [ + Moss::DocumentInfo.new(id: "1", text: "Refunds take 5-7 days.", metadata: { "topic" => "refunds" }), + Moss::DocumentInfo.new(id: "2", text: "Track orders in the dashboard.", metadata: { "topic" => "shipping" }) + ] + end + + # ---- credential / argument validation --------------------------------- + + def test_missing_credentials_raise_configuration_error + client = Moss::Client.new(project_id: nil, project_key: nil) + assert_raises(Moss::ConfigurationError) { client.list_indexes } + end + + def test_empty_index_name_raises_argument_error + client = build_client(manage: TestSupport::FakeManageClient.new) + assert_raises(Moss::ArgumentError) { client.get_index(" ") } + end + + def test_empty_documents_raise_argument_error + client = build_client(manage: TestSupport::FakeManageClient.new) + assert_raises(Moss::ArgumentError) { client.create_index("docs", []) } + end + + # ---- create_index: model + embedding resolution ----------------------- + + def test_create_index_defaults_to_minilm_and_polls_to_completion + manage = TestSupport::FakeManageClient.new + client = build_client(manage: manage) + + result = client.create_index("docs", sample_docs) + + create_call = manage.calls.find { |c| c[0] == :create_index } + assert_equal "moss-minilm", create_call[3] + assert_equal "job-1", result.job_id + assert_equal 2, result.doc_count + assert(manage.calls.any? { |c| c[0] == :get_job_status }) + end + + def test_create_index_infers_custom_model_from_embeddings + manage = TestSupport::FakeManageClient.new + client = build_client(manage: manage) + docs = [ + Moss::DocumentInfo.new(id: "1", text: "a", embedding: [0.1, 0.2, 0.3]), + Moss::DocumentInfo.new(id: "2", text: "b", embedding: [0.4, 0.5, 0.6]) + ] + + client.create_index("vec", docs) + + create_call = manage.calls.find { |c| c[0] == :create_index } + assert_equal "custom", create_call[3] + end + + def test_create_index_rejects_mixed_embeddings + client = build_client(manage: TestSupport::FakeManageClient.new) + docs = [ + Moss::DocumentInfo.new(id: "1", text: "a", embedding: [0.1]), + Moss::DocumentInfo.new(id: "2", text: "b") + ] + assert_raises(Moss::ArgumentError) { client.create_index("vec", docs) } + end + + def test_create_index_rejects_mismatched_embedding_dimensions + client = build_client(manage: TestSupport::FakeManageClient.new) + docs = [ + Moss::DocumentInfo.new(id: "1", text: "a", embedding: [0.1, 0.2]), + Moss::DocumentInfo.new(id: "2", text: "b", embedding: [0.3]) + ] + assert_raises(Moss::ArgumentError) { client.create_index("vec", docs) } + end + + def test_create_index_rejects_custom_model_without_embeddings + client = build_client(manage: TestSupport::FakeManageClient.new) + assert_raises(Moss::ArgumentError) do + client.create_index("vec", sample_docs, model_id: "custom") + end + end + + # ---- job polling ------------------------------------------------------ + + def test_failed_job_raises_job_error + manage = TestSupport::FakeManageClient.new(job_status_sequence: ["failed"]) + client = build_client(manage: manage) + assert_raises(Moss::JobError) { client.create_index("docs", sample_docs) } + end + + def test_on_progress_callback_receives_updates + manage = TestSupport::FakeManageClient.new(job_status_sequence: %w[building completed]) + client = build_client(manage: manage) + seen = [] + + client.create_index("docs", sample_docs, on_progress: ->(p) { seen << p.status }) + + assert_includes seen, "building" + assert_includes seen, "completed" + end + + # ---- metadata normalisation ------------------------------------------- + + def test_metadata_is_stringified_for_native_layer + manage = TestSupport::FakeManageClient.new + client = build_client(manage: manage) + docs = [Moss::DocumentInfo.new(id: "1", text: "a", metadata: { topic: :refunds, count: 3 })] + + client.create_index("docs", docs) + + core_docs = manage.calls.find { |c| c[0] == :create_index }[2] + assert_equal({ "topic" => "refunds", "count" => "3" }, core_docs.first.metadata) + end + + # ---- reads ------------------------------------------------------------ + + def test_list_indexes_maps_to_high_level_models + client = build_client(manage: TestSupport::FakeManageClient.new) + indexes = client.list_indexes + assert_kind_of Moss::IndexInfo, indexes.first + assert_equal "docs", indexes.first.name + assert_equal "moss-minilm", indexes.first.model.id + end + + def test_get_documents_maps_metadata + client = build_client(manage: TestSupport::FakeManageClient.new) + docs = client.get_documents("docs") + assert_kind_of Moss::DocumentInfo, docs.first + assert_equal({ "k" => "v" }, docs.first.metadata) + end + + # ---- query routing ---------------------------------------------------- + + def test_query_uses_local_manager_when_index_loaded + index = TestSupport::FakeIndexManager.new(loaded: ["docs"]) + client = build_client(manage: TestSupport::FakeManageClient.new, index: index) + + filter = { "field" => "topic", "condition" => { "$eq" => "refunds" } } + result = client.query("docs", "hello", top_k: 3, alpha: 0.5, filter: filter) + + assert_kind_of Moss::SearchResult, result + query_call = index.calls.find { |c| c[0] == :query } + assert_equal 3, query_call[4] # top_k + assert_in_delta 0.5, query_call[5], 0.0001 # alpha + assert_equal JSON.generate(filter), query_call[6] # filter_json passed through verbatim + assert_equal 0.9, result.docs.first.score + end + + def test_query_falls_back_to_cloud_when_index_not_loaded + index = TestSupport::FakeIndexManager.new(loaded: []) + client = build_client(manage: TestSupport::FakeManageClient.new, index: index) + + # Stub the cloud path so no network call happens. + captured = nil + replacement = lambda { |**kwargs| + captured = kwargs + Moss::SearchResult.new(docs: [], query: kwargs[:query], index_name: kwargs[:index_name], time_taken_ms: 1) + } + TestSupport.with_stubbed_singleton(Moss::CloudQuery, :execute, replacement) do + client.query("docs", "hello", top_k: 7) + end + + refute_nil captured + assert_equal "docs", captured[:index_name] + assert_equal 7, captured[:options].top_k + assert(index.calls.none? { |c| c[0] == :query }) + end + + def test_cloud_query_raises_when_manage_url_has_no_derivable_query_endpoint + # manage_url without "/v1/manage" and no explicit query_url => query URL is + # undecidable, so the cloud fallback must fail fast rather than POST to the + # manage endpoint. + index = TestSupport::FakeIndexManager.new(loaded: []) + client = build_client( + manage: TestSupport::FakeManageClient.new, index: index, + manage_url: "https://custom.example/api" + ) + assert_raises(Moss::ConfigurationError) { client.query("docs", "hello") } + end + + def test_cloud_query_uses_explicit_query_url_when_manage_url_is_custom + index = TestSupport::FakeIndexManager.new(loaded: []) + client = build_client( + manage: TestSupport::FakeManageClient.new, index: index, + manage_url: "https://custom.example/api", query_url: "https://custom.example/search" + ) + captured = nil + replacement = lambda { |**kwargs| + captured = kwargs + Moss::SearchResult.new(docs: [], query: kwargs[:query], index_name: kwargs[:index_name], time_taken_ms: 1) + } + TestSupport.with_stubbed_singleton(Moss::CloudQuery, :execute, replacement) do + client.query("docs", "hello") + end + assert_equal "https://custom.example/search", captured[:query_url] + end + + def test_search_is_an_alias_for_query + index = TestSupport::FakeIndexManager.new(loaded: ["docs"]) + client = build_client(manage: TestSupport::FakeManageClient.new, index: index) + assert_kind_of Moss::SearchResult, client.search("docs", "hello") + end + + # ---- load / unload ---------------------------------------------------- + + def test_load_index_returns_name_and_tracks_loaded_state + index = TestSupport::FakeIndexManager.new + client = build_client(manage: TestSupport::FakeManageClient.new, index: index) + + assert_equal "docs", client.load_index("docs") + assert index.has_index?("docs") + end + + def test_load_index_rejects_cache_path + index = TestSupport::FakeIndexManager.new + client = build_client(manage: TestSupport::FakeManageClient.new, index: index) + assert_raises(Moss::ArgumentError) { client.load_index("docs", cache_path: "/tmp/cache") } + end + + # ---- close ------------------------------------------------------------ + + def test_close_releases_manage_client + manage = TestSupport::FakeManageClient.new + client = build_client(manage: manage) + client.list_indexes # force lazy init + client.close + assert manage.closed? + end +end diff --git a/sdks/ruby/sdk/test/integration_test.rb b/sdks/ruby/sdk/test/integration_test.rb new file mode 100644 index 00000000..9c8042fe --- /dev/null +++ b/sdks/ruby/sdk/test/integration_test.rb @@ -0,0 +1,68 @@ +# frozen_string_literal: true + +require_relative "test_helper" + +# End-to-end tests that hit the live Moss cloud + native libmoss runtime. +# +# They auto-skip unless BOTH of these hold, mirroring the other SDKs' E2E suites: +# * MOSS_PROJECT_ID and MOSS_PROJECT_KEY are set +# * libmoss is loadable (MOSS_LIB_DIR / MOSS_LIBRARY_PATH points at the C SDK) +# +# Run with, e.g.: +# MOSS_LIB_DIR=/path/to/libmoss/lib \ +# DYLD_LIBRARY_PATH=/path/to/libmoss/lib \ +# MOSS_PROJECT_ID=... MOSS_PROJECT_KEY=... \ +# ruby -Itest -Ilib test/integration_test.rb +class IntegrationTest < Minitest::Test + def setup + unless credentials? && Moss::Core.available? + skip("integration test skipped: set MOSS_PROJECT_ID/MOSS_PROJECT_KEY and make libmoss available") + end + + @client = Moss::Client.new(poll_interval_seconds: 2) + @index_name = "ruby-sdk-it-#{Process.pid}-#{rand(100_000)}" + end + + def teardown + return unless defined?(@client) && @client + + @client.unload_index(@index_name) + @client.delete_index(@index_name) + rescue StandardError + # best-effort cleanup + ensure + @client&.close + end + + def test_index_load_query_and_metadata_filter_round_trip + docs = [ + Moss::DocumentInfo.new(id: "1", text: "Refunds are processed within five to seven business days.", + metadata: { "topic" => "refunds" }), + Moss::DocumentInfo.new(id: "2", text: "Orders can be tracked from the account dashboard.", + metadata: { "topic" => "shipping" }) + ] + + create = @client.create_index(@index_name, docs) + assert_equal 2, create.doc_count + + @client.load_index(@index_name) + + result = @client.query(@index_name, "how long do refunds take?", top_k: 3) + refute_empty result.docs + assert_equal "1", result.docs.first.id + + filtered = @client.query( + @index_name, "how long do refunds take?", + top_k: 3, filter: { "field" => "topic", "condition" => { "$eq" => "shipping" } } + ) + assert(filtered.docs.all? { |doc| doc.metadata.nil? || doc.metadata["topic"] == "shipping" }) + end + + private + + def credentials? + id = ENV["MOSS_PROJECT_ID"].to_s.strip + key = ENV["MOSS_PROJECT_KEY"].to_s.strip + !id.empty? && !key.empty? + end +end diff --git a/sdks/ruby/sdk/test/models_test.rb b/sdks/ruby/sdk/test/models_test.rb new file mode 100644 index 00000000..4ccf17ae --- /dev/null +++ b/sdks/ruby/sdk/test/models_test.rb @@ -0,0 +1,83 @@ +# frozen_string_literal: true + +require_relative "test_helper" + +class ModelsTest < Minitest::Test + def test_document_info_defaults + doc = Moss::DocumentInfo.new(id: "1") + assert_equal "1", doc.id + assert_nil doc.text + assert_nil doc.metadata + assert_nil doc.embedding + end + + def test_document_info_full + doc = Moss::DocumentInfo.new(id: "1", text: "hi", metadata: { "a" => "b" }, embedding: [0.1]) + assert_equal "hi", doc.text + assert_equal({ "a" => "b" }, doc.metadata) + assert_equal [0.1], doc.embedding + end + + def test_model_and_status_constants + assert_equal "moss-minilm", Moss::Model::MOSS_MINILM + assert_equal "custom", Moss::Model::CUSTOM + assert_equal "Ready", Moss::IndexStatus::READY + assert_equal "completed", Moss::JobStatus::COMPLETED + end +end + +class CloudQueryTest < Minitest::Test + def base_args + { + query_url: "https://example.test/query", + project_id: "pid", + project_key: "pkey", + index_name: "docs", + query: "hello", + http_open_timeout: 1, + http_read_timeout: 1 + } + end + + def test_missing_query_url_raises_configuration_error + assert_raises(Moss::ConfigurationError) do + Moss::CloudQuery.execute(**base_args, query_url: "", options: nil) + end + end + + def test_alpha_option_rejected_for_cloud_query + options = Moss::Client::QueryOptions.new(alpha: 0.5) + assert_raises(Moss::UnsupportedQueryError) do + Moss::CloudQuery.execute(**base_args, options: options) + end + end + + def test_filter_option_rejected_for_cloud_query + options = Moss::Client::QueryOptions.new(filter: { "topic" => "refunds" }) + assert_raises(Moss::UnsupportedQueryError) do + Moss::CloudQuery.execute(**base_args, options: options) + end + end + + def test_parse_response_maps_docs + body = JSON.generate( + "docs" => [{ "id" => "1", "text" => "hi", "metadata" => { "k" => "v" }, "score" => 0.42 }], + "query" => "hello", + "indexName" => "docs", + "timeTakenMs" => 5 + ) + response = Struct.new(:code, :body).new("200", body) + result = Moss::CloudQuery.parse_response(response, "hello") + + assert_equal 1, result.docs.length + assert_equal "1", result.docs.first.id + assert_in_delta 0.42, result.docs.first.score, 0.0001 + assert_equal 5, result.time_taken_ms + end + + def test_parse_response_raises_http_error_on_non_2xx + response = Struct.new(:code, :body).new("500", "boom") + error = assert_raises(Moss::HTTPError) { Moss::CloudQuery.parse_response(response, "hello") } + assert_equal 500, error.status_code + end +end diff --git a/sdks/ruby/sdk/test/test_helper.rb b/sdks/ruby/sdk/test/test_helper.rb new file mode 100644 index 00000000..a9d38386 --- /dev/null +++ b/sdks/ruby/sdk/test/test_helper.rb @@ -0,0 +1,134 @@ +# frozen_string_literal: true + +$LOAD_PATH.unshift File.expand_path("../lib", __dir__) +# Resolve the sibling bindings gem without Bundler so tests run via `ruby -Itest`. +$LOAD_PATH.unshift File.expand_path("../../bindings/lib", __dir__) + +require "minitest/autorun" +require "moss" + +module TestSupport + # Temporarily replaces a singleton method, restoring it afterward. Avoids a + # dependency on minitest/mock (not bundled with every minitest build). + def self.with_stubbed_singleton(mod, name, replacement) + original = mod.method(name) + mod.singleton_class.send(:define_method, name, replacement) + yield + ensure + mod.singleton_class.send(:define_method, name, original) + end +end + +module TestSupport + # A fake ManageClient recording calls and returning canned results, used to + # unit-test the high-level client without libmoss or live credentials. + class FakeManageClient + attr_reader :calls + + def initialize(job_status_sequence: nil) + @calls = [] + @job_status_sequence = job_status_sequence + @closed = false + end + + def create_index(name, docs, model_id = nil) + @calls << [:create_index, name, docs, model_id] + Moss::Core::MutationResult.new(job_id: "job-1", index_name: name, doc_count: docs.length) + end + + def add_docs(name, docs, options = nil) + @calls << [:add_docs, name, docs, options] + Moss::Core::MutationResult.new(job_id: "job-2", index_name: name, doc_count: docs.length) + end + + def delete_docs(name, doc_ids) + @calls << [:delete_docs, name, doc_ids] + Moss::Core::MutationResult.new(job_id: "job-3", index_name: name, doc_count: doc_ids.length) + end + + def get_job_status(job_id) + @calls << [:get_job_status, job_id] + status = @job_status_sequence&.shift || "completed" + Moss::Core::JobStatusResponse.new( + job_id: job_id, status: status, progress: 1.0, current_phase: nil, + error: nil, created_at: "t0", updated_at: "t1", completed_at: "t1" + ) + end + + def get_index(name) + @calls << [:get_index, name] + Moss::Core::IndexInfo.new( + id: "idx-1", name: name, version: "1", status: "Ready", doc_count: 2, + created_at: "t0", updated_at: "t1", + model: Moss::Core::ModelRef.new(id: "moss-minilm", version: "1") + ) + end + + def list_indexes + @calls << [:list_indexes] + [get_index("docs")] + end + + def delete_index(name) + @calls << [:delete_index, name] + true + end + + def get_docs(name, doc_ids = []) + @calls << [:get_docs, name, doc_ids] + [Moss::Core::DocumentInfo.new(id: "1", text: "hello", metadata: { "k" => "v" }, embedding: nil)] + end + + def close + @closed = true + end + + def closed? + @closed + end + end + + # A fake IndexManager with configurable loaded state. + class FakeIndexManager + attr_reader :calls + + def initialize(loaded: []) + @calls = [] + @loaded = loaded.map(&:to_s) + end + + def load_index(index_name, _options = nil) + @calls << [:load_index, index_name] + @loaded << index_name.to_s + Moss::Core::IndexInfo.new( + id: "idx", name: index_name.to_s, version: "1", status: "Ready", doc_count: 1, + created_at: nil, updated_at: nil, + model: Moss::Core::ModelRef.new(id: "moss-minilm", version: nil) + ) + end + + def unload_index(index_name) + @calls << [:unload_index, index_name] + @loaded.delete(index_name.to_s) + nil + end + + def has_index?(index_name) + @loaded.include?(index_name.to_s) + end + + def load_query_model(_index_name) + nil + end + + def query(index_name, query_text, embedding: nil, top_k: nil, alpha: nil, filter_json: nil) + @calls << [:query, index_name, query_text, embedding, top_k, alpha, filter_json] + Moss::Core::SearchResult.new( + docs: [Moss::Core::QueryResultDocument.new(id: "1", text: "hello", metadata: nil, score: 0.9)], + query: query_text, index_name: index_name.to_s, time_taken_ms: 3 + ) + end + + def close; end + end +end