HoloFi Oracle

Off-chain Fair Market Value (FMV) price oracle service for the HoloFi TCG (Trading Card Game) card-backed decentralized lending protocol. The oracle ingests verified physical market sales data, calculates manipulation-resistant, time-decay weighted FMV prices, and commits price updates on-chain to the HoloFiCardPriceFeed contract on Base (OP Stack L2) to drive Loan-to-Value (LTV), Health Factor calculations, and Dutch auction liquidations in HoloFiVaultLoanCore.

Reference Specification: For comprehensive architectural details, mathematical models, and security guarantees, refer to the HoloFi Oracle Product Specification (docs/oracle_spec.md) (Chinese Version (docs/docs_zh/oracle_spec.md)).

---

Table of Contents

• 1. Project Overview & Objectives
• 2. Technology Stack
• 3. Repository Directory Structure
• 4. System Architecture
• 5. Core Modules Specification
• 6. Configuration & Environment Variables
• 7. Build, Run & Test Workflows

---

1. Project Overview & Objectives

1.1 Overview

In decentralized collateralized lending, pricing illiquid Real World Assets (RWA)—such as graded physical Trading Card Game (TCG) cards—presents distinct challenges: fragmented secondary market trading venues (eBay, TCGPlayer, PWCC), low transaction frequencies, heterogeneous valuations based on grading agencies (PSA, BGS, CGC) and numerical sub-grades, and vulnerability to single-trade wash trading manipulation.

The HoloFi Oracle bridges off-chain secondary market sales data and on-chain EVM smart contracts. It implements a push-model architecture that periodically computes robust Fair Market Value (FMV) benchmarks and writes them directly to the on-chain Single Source of Truth (SSOT) price feed.

1.2 Core Objectives

• TCG Collateral FMV Feeds: Supply accurate, manipulation-resistant valuations for tokenized card collateral (ERC-721 NFTs in HoloFiVaultCard), enabling dynamic LTV evaluation, borrowing capacity enforcement, and Dutch auction liquidation triggers in HoloFiVaultLoanCore.
• On-Chain Single Source of Truth (SSOT): Write batch prices directly to HoloFiCardPriceFeed.setBatchPrices indexed by unique, deterministic cardTypeId identifiers (card model + grade).
• Source-Agnostic Time-Decay FMV Engine: Normalize external pricing data points into a unified schema and apply an exponential time-decay weighted moving average (TWAP-decay with 30-day half-life and 90-day window).
• Reliable Batch Submission with Gas Chunking: Split price submissions into manageable arrays (DEFAULT_CHUNK_SIZE = 100) with EVM transaction receipt event verification (PriceUpdated) to prevent block gas limit breaches.
• High Availability & Fault Tolerance: Provide automated API key rotation with rate-limiting backoff (handling HTTP 429 burst vs quota limits) and RPC redundancy via Ethers v6 FallbackProvider and NonceManager.
• Pre-Mint Price Preview & Registration: Provide secure REST API endpoints (/api/card-type/price and /api/card-type/save-price) to support pre-mint collateral valuation and immediate on-chain registration for protocol administrators.

---

2. Core Technology Stack

• Language & Runtime: TypeScript on Node.js (ES Modules)
• Blockchain Integration: Ethers.js v6 (FallbackProvider, NonceManager, EVM contract ABIs, ABI encoding)
• Target Network: Base (OP Stack L2) & Local EVM Testnet (Hardhat)
• Persistence Layer: MongoDB Atlas (Serverless document store for catalog index and price history)
• Serverless & Scheduling: Vercel Serverless Functions & Cron / Local Node.js daemon (node-cron)
• Testing Suite: Vitest & Mock Service Worker (MSW)

---

3. Repository Directory Structure

holofi_oracle/
├── api/                                # Vercel Serverless Function endpoints
│   ├── auth.ts                         # Bearer token & x-api-key authentication middleware
│   ├── cardTypeBody.ts                 # Request payload validation & printing enum whitelist
│   ├── cron.ts                         # GET /api/cron: Scheduled batch price update trigger
│   └── card-type/
│       ├── price.ts                    # POST /api/card-type/price: Pre-mint FMV preview (Cache-first)
│       └── save-price.ts               # POST /api/card-type/save-price: FMV commit & on-chain submit
├── src/                                # Core Oracle source code
│   ├── abi/                            # Typed smart contract ABI definitions (extracted from holofi_protocol)
│   │   ├── AccessControlManager.ts
│   │   ├── GradeEligibilityPolicy.ts
│   │   ├── HoloFiCardPriceFeed.ts
│   │   ├── HoloFiLendingPool.ts
│   │   ├── HoloFiLendingPoolFactory.ts
│   │   ├── HoloFiVaultCard.ts
│   │   ├── HoloFiVaultLoanCore.ts
│   │   └── MockERC20.ts
│   ├── adapters/                       # External marketplace & price data source adapters
│   │   ├── http.ts                     # RateLimitedFetcher with delay & exponential backoff
│   │   ├── normalize.ts                # Normalization utilities (cardNumber, pricing, wei conversion)
│   │   ├── pokemonPriceTracker.ts      # PokemonPriceTracker adapter with API key rotation pool
│   │   ├── registry.ts                 # Adapter registry factory
│   │   └── types.ts                    # IPriceAdapter, SourcePriceEntry, SourcePriceMap interfaces
│   ├── aggregator/                     # FMV calculation engine
│   │   ├── index.ts                    # Exponential time-decay weighted average algorithm
│   │   └── types.ts                    # FmvResult & calculation parameter types
│   ├── catalog/                        # Card type catalog and on-chain mapping
│   │   ├── CardTypeIndex.ts            # In-memory and file-based catalog index
│   │   ├── chainReader.ts              # Reads active cardTypeIds & prices from on-chain contracts
│   │   ├── encoding.ts                 # Deterministic cardTypeId computation (abi.encode + keccak256)
│   │   └── index.ts                    # Catalog module exports
│   ├── config/                         # System configuration loader
│   │   └── index.ts                    # loadConfig: Typed environment variable loader & key collector
│   ├── runner/                         # Pipeline orchestration and execution runners
│   │   ├── cardTypeService.ts          # Service layer for admin API price preview and save
│   │   ├── cli.ts                      # CLI entrypoint for manual/ad-hoc pipeline execution
│   │   ├── factory.ts                  # Dependency injection factory (Stores, Adapters, Submitters)
│   │   ├── runOnce.ts                  # End-to-end pipeline executor (Read -> Fetch -> Aggregate -> Persist -> Submit)
│   │   └── scheduler.ts                # Local cron runner daemon (npm run serve)
│   ├── store/                          # Persistent storage layer (Mongo & JSON)
│   │   ├── json.ts                     # File-based JSON store for local development
│   │   ├── mongo.ts                    # MongoDB Atlas driver (card_type_index & price_history collections)
│   │   └── types.ts                    # Storage interfaces (CardTypeIndexStore, PriceHistoryStore, NormRecord)
│   └── submitter/                      # On-chain transaction submitter
│       ├── batch.ts                    # Batch chunking helper (DEFAULT_CHUNK_SIZE = 100)
│       ├── oracleWallet.ts             # EOA Wallet with NonceManager & FallbackProvider
│       └── priceFeed.ts                # PriceFeedSubmitter: submits setBatchPrices & verifies receipt logs
├── test/                               # Test suite (Unit & Integration)
│   ├── fixtures/                       # Test fixtures, mock data, and deployed address mocks
│   ├── integration/                    # E2E integration tests against live hardhat node
│   │   └── push-e2e.test.ts
│   └── unit/                           # Unit tests for adapters, aggregator, catalog, runner, store, submitter
├── docs/                               # Architecture and technical specification documentation
│   ├── holofi-oracle-system.jpeg       # System architecture diagram
│   ├── oracle_spec.md                  # ASD-STE100 Product Specification (English)
│   └── docs_zh/                        # Synchronized Chinese documentation
│       └── oracle_spec.md              # Product Specification (Chinese)
├── package.json                        # Dependencies, scripts, engine specifications
├── tsconfig.json                       # TypeScript compiler configuration
└── vercel.json                         # Vercel Serverless Function & Cron configuration

---

4. System Architecture

The HoloFi Oracle is structured across distinct functional layers to guarantee complete decoupling of external data ingestion, valuation mathematics, persistent state management, and EVM blockchain interaction.

Layer Responsibilities

1. Interface Layer:
   • Vercel Cron Trigger (GET /api/cron): Invoked once daily (0 0 * * *) to execute scheduled batch price refresh workflows.
   • Admin REST API (POST /api/card-type/*): Secures pre-mint pricing previews and manual price initialization endpoints protected by ORACLE_API_KEY.
   • CLI / Local Scheduler (npm run runOnce / npm run serve): Provides local command-line execution and local node-cron daemon mode for development and standalone server deployments.
2. Orchestration Layer (PipelineRunner):
   • Manages end-to-end execution flow: loads indexed card types, dispatches data source adapters, coordinates outlier cleaning, triggers the FMV aggregation engine, persists daily records, and invokes the on-chain submitter.
3. Adapter Layer (IPriceAdapter):
   • Encapsulates external marketplace APIs (PokemonPriceTracker), manages rate limits via RateLimitedFetcher (3000ms delay, burst vs daily quota detection, exponential backoff), executes multi-key rotation across up to 9 API keys, and standardizes data points into SourcePriceEntry records.
4. Aggregation Layer (FMV Aggregator):
   • Calculates time-decay weighted moving average (TWAP-decay) using an exponential decay constant $\lambda = \frac{\ln 2}{\text{halfLifeDays}}$. Weights discrete historical sales over a 90-day observation window with a 30-day half-life. Employs integer scaling ($10^9$) to avoid precision loss, generating prices in 18-decimal Wei format.
5. Storage Layer (StoreFactory):
   • Provides pluggable persistence implementations: JsonSourceStore for zero-dependency local testing and MongoSourceStore for production MongoDB Atlas deployments (managing card_type_index and price_history collections with idempotent upserts).
6. Blockchain Layer (On-Chain Submitter):
   • Connects to Base (or local Hardhat testnet) via Ethers v6 FallbackProvider (primary RPC priority 0 with 10s stall timeout; backup RPC priority 1). Employs NonceManager for non-blocking transaction sequencing. Batches transactions into slices of 100 card models (setBatchPrices) and validates receipt logs for PriceUpdated events.

---

5. Core Modules Specification

Module / Component	Main File Path	Key Responsibilities
Configuration Manager	src/config/index.ts	Loads .env variables, validates required fields, collects and deduplicates numbered API keys (_1.._9), and exposes a strongly-typed OracleConfig.
Price Source Adapters	src/adapters/pokemonPriceTracker.ts src/adapters/http.ts src/adapters/normalize.ts	Fetches graded sold card sales; manages key rotation upon quota exhaustion; applies 3000ms rate limiting and exponential backoff; disambiguates cards via normalized card number and printing whitelist.
Card Type Catalog & Encoding	src/catalog/encoding.ts src/catalog/chainReader.ts src/catalog/CardTypeIndex.ts	Computes unique, deterministic cardTypeId using keccak256(abi.encode(attrs)) matching on-chain Solidity contracts; queries on-chain HoloFiCardPriceFeed to discover registered card models.
FMV Aggregation Engine	src/aggregator/index.ts	Ingests normalized historical and fresh sales data; executes exponential time-decay weighted averaging ($10^9$ integer weight scaling); outputs 18-decimal Wei price valuations.
Data Persistence Store	src/store/mongo.ts src/store/json.ts src/store/types.ts	Manages card_type_index (unique cardTypeId) and price_history (compound unique cardTypeId: 1, day: 1); performs idempotent daily upserts.
On-Chain Submitter	src/submitter/priceFeed.ts src/submitter/oracleWallet.ts src/submitter/batch.ts	Manages Oracle EOA wallet (ORACLE_ROLE); configures FallbackProvider and NonceManager; splits arrays into chunks of 100 items; broadcasts setBatchPrices and parses receipt logs for PriceUpdated.
Pipeline Runner & Services	src/runner/runOnce.ts src/runner/cardTypeService.ts src/runner/cli.ts	Orchestrates the end-to-end price update workflow; provides service methods for pre-mint preview (/api/card-type/price) and commit (/api/card-type/save-price).
Serverless API Layer	api/cron.ts api/auth.ts api/cardTypeBody.ts api/card-type/price.ts api/card-type/save-price.ts	Exposes Vercel serverless endpoints protected by CRON_SECRET and ORACLE_API_KEY; enforces printing enum whitelist on all request payloads.

---

6. Configuration & Environment Variables

Create a .env file in the project root by copying .env.example:

cp .env.example .env

Environment Variables Reference

Variable Name	Required	Default Value	Description
CHAIN_ID	No	31337	Target blockchain chain ID (e.g. 31337 for Localhost, 84532 for Base Sepolia, 8453 for Base Mainnet).
RPC_URL	Yes	—	Primary EVM JSON-RPC endpoint URL (e.g. http://localhost:8545 or Alchemy/Infura RPC).
RPC_URL_BACKUP	No	—	Secondary backup EVM JSON-RPC endpoint for automatic failover.
ORACLE_PRIVATE_KEY	Yes	—	64-character hexadecimal private key of the Oracle EOA possessing ORACLE_ROLE on HoloFiCardPriceFeed.
PRICE_FEED_ADDRESS	Yes	—	Target HoloFiCardPriceFeed smart contract address on the target network.
PRICE_SOURCE	No	pokemonpricetracker	Price source adapter identifier (pokemonpricetracker or mock).
POKEMONPRICETRACKER_API_KEY_1	Yes	—	Primary API key for PokemonPriceTracker.
POKEMONPRICETRACKER_API_KEY_2..9	No	—	Additional API keys for automatic key rotation upon daily quota exhaustion.
SOURCE_STORE_TYPE	No	json	Persistent store backend: json (local development) or mongodb (production).
MONGODB_URI	Prod: Yes	—	MongoDB connection string URI (e.g. mongodb+srv://...).
MONGODB_DB_NAME	No	holofi	Database name in MongoDB Atlas.
ORACLE_API_KEY	Yes (in API)	—	Shared secret token securing /api/card-type/* Admin API endpoints.
CRON_SECRET	Yes (in Cron)	—	Bearer authentication token protecting GET /api/cron.
FMV_HALFLIFE_DAYS	No	30	Half-life in days for time-decay weighting.
FMV_WINDOW_DAYS	No	90	Lookback observation window in days for historical sales data.
DRY_RUN	No	false	When set to true, computes FMV and logs results without broadcasting on-chain transactions.

---

7. Build, Run & Test Workflows

7.1 Prerequisites

• Node.js: Version >= 20.0.0
• npm: Version >= 9.0.0

7.2 Installation & Build

# Install project dependencies
npm install

# Run TypeScript static type check
npm run typecheck

7.3 Running the Oracle

# 1. Execute a single full pipeline run (CLI mode using current .env configuration)
npm run runOnce

# 2. Execute a dry-run calculation (computes FMV and logs updates without sending on-chain transactions)
DRY_RUN=true npm run runOnce

# 3. Start local daemon scheduler (executes background cron runs via node-cron)
npm run serve

7.4 Testing

The test suite includes both comprehensive unit tests (using MSW to mock external HTTP APIs) and full end-to-end integration tests with a live local Hardhat node.

# Run unit test suite (excluding integration tests)
npx vitest run --exclude 'test/integration/**'

# Run entire test suite (requires running local hardhat node with HoloFi protocol deployed)
npm test

# Run tests in interactive watch mode
npm run test:watch

Note

Integration Test Requirements: The integration test suite (test/integration/push-e2e.test.ts) connects to http://localhost:8545 and expects the protocol to be deployed with address mappings in holofi_protocol/ignition/deployments/chain-31337/deployed_addresses.json. It validates the end-to-end lifecycle: NFT minting $\rightarrow$ collateral vault deposit $\rightarrow$ oracle runOnce price push $\rightarrow$ CardPriceFeed update $\rightarrow$ getVaultFMV recalculation $\rightarrow$ loan borrowing.

---

License

Private / Confidential — HoloFi Protocol. All rights reserved.