Add reproducible research metrics and benchmark evidence

.github/workflows/benchmarks.yml

@@ -2,30 +2,56 @@ name: Performance Benchmarking
 
 on:
   push:
-    branches: [ main ]
+    branches: [main]
   pull_request:
-    branches: [ main ]
+    branches: [main]
+  workflow_dispatch:
+
+permissions:
+  contents: read
 
 jobs:
   benchmark:
-    name: Core Engine Latency Tracking
+    name: Seeded Drift Research Benchmark
     runs-on: ubuntu-latest
+    timeout-minutes: 10
     steps:
-      - name: ⬇️ Checkout Repository
+      - name: Checkout repository
         uses: actions/checkout@v4
 
-      - name: 🐍 Set up Python
+      - name: Set up Python
         uses: actions/setup-python@v5
         with:
-          python-version: '3.11'
-          cache: 'pip'
+          python-version: "3.11"
 
-      - name: 📦 Install Engine & Profiling Tools
+      - name: Run deterministic benchmark
         run: |
-          python -m pip install --upgrade pip
-          if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
-          pip install pytest pytest-benchmark
+          python benchmarks/run_benchmark.py \
+            --iterations 20000 \
+            --evaluation-samples 2000 \
+            --output benchmark-results.json
 
-      - name: ⏱️ Profile Code Execution Speeds
+      - name: Validate benchmark schema
         run: |
-          pytest --benchmark-only --benchmark-skip || echo "Performance profiles logged successfully."
+          python - <<'PY'
+          import json
+          from pathlib import Path
+
+          result = json.loads(Path("benchmark-results.json").read_text())
+          required = {"latency_us", "throughput_ops_s", "peak_tracemalloc_mib", "quality"}
+          missing = required - result.keys()
+          if missing:
+              raise SystemExit(f"missing benchmark fields: {sorted(missing)}")
+          if result["throughput_ops_s"] <= 0 or result["latency_us"]["p99"] <= 0:
+              raise SystemExit("benchmark produced invalid performance metrics")
+          if result["quality"]["f1"] < 0.95:
+              raise SystemExit("synthetic drift regression: F1 below 0.95")
+          PY
+
+      - name: Upload raw benchmark evidence
+        uses: actions/upload-artifact@v4
+        with:
+          name: sentinelai-benchmark-${{ github.sha }}
+          path: benchmark-results.json
+          if-no-files-found: error
+          retention-days: 30

README.md

@@ -2,32 +2,18 @@ SentinelAI
 <img width="1536" height="1024" alt="image" src="https://github.com/user-attachments/assets/33e7a5fb-be0a-495b-a950-7a14b9aedb4b" />
 
 # SentinelAI — Enterprise AI Reliability & Governance Platform
-![CI](https://img.shields.io/github/actions/workflow/status/Trojan3877/SentinelAI/ci.yml?branch=main)
-![C++](https://img.shields.io/badge/C++-DriftEngine-blue)
-![Go](https://img.shields.io/badge/Go-Ingestion-00ADD8)
-![Snowflake](https://img.shields.io/badge/Snowflake-Optional-29B5E8)
-![Kubernetes](https://img.shields.io/badge/Kubernetes-Orchestration-326CE5)
-![Terraform](https://img.shields.io/badge/Terraform-IaC-7B42BC)
-![MLflow](https://img.shields.io/badge/MLflow-ExperimentTracking-0194E2)
-![LangChain](https://img.shields.io/badge/LangChain-LLMIntegration-green)
-![Ollama](https://img.shields.io/badge/Ollama-LocalLLM-black)
-[![Streamlit App](https://static.streamlit.io/badges/streamlit_badge_black_white.svg)](https://share.streamlit.io/trojan3877/sentinelai/main/demo_app.py)
-![Build Status](https://github.com/Trojan3877/SentinelAI/workflows/CI/badge.svg)
-[![LLM Engine: GPT-4](https://img.shields.io/badge/LLM_Engine-GPT--4_Omni-4aa377.svg?logo=openai&logoColor=white)](https://openai.com/)
-[![Control Plane: Streamlit](https://img.shields.io/badge/Control_Plane-Streamlit-FF4B4B.svg?logo=streamlit&logoColor=white)](https://share.streamlit.io/)
-[![Guardrails: Semantic & PII](https://img.shields.io/badge/Guardrails-Semantic_%26_PII-orange.svg)](#)
-[![Continuous Integration](https://github.com/Trojan3877/SentinelAI/actions/workflows/ci-cd.yml/badge.svg)](https://github.com/Trojan3877/SentinelAI/actions/workflows/ci-cd.yml)
-[![Code Quality Assurance](https://github.com/Trojan3877/SentinelAI/actions/workflows/ci.yml/badge.svg)](https://github.com/Trojan3877/SentinelAI/actions/workflows/ci.yml)
-[![Security Analysis](https://github.com/Trojan3877/SentinelAI/actions/workflows/security.yml/badge.svg)](https://github.com/Trojan3877/SentinelAI/actions/workflows/security.yml)
-[![SAST Code Flaw Scan](https://github.com/Trojan3877/SentinelAI/actions/workflows/sast.yml/badge.svg)](https://github.com/Trojan3877/SentinelAI/actions/workflows/sast.yml)
-[![Performance Benchmarks](https://github.com/Trojan3877/SentinelAI/actions/workflows/benchmarks.yml/badge.svg)](https://github.com/Trojan3877/SentinelAI/actions/workflows/benchmarks.yml)
-[![Schema Validation](https://github.com/Trojan3877/SentinelAI/actions/workflows/data-validation.yml/badge.svg)](https://github.com/Trojan3877/SentinelAI/actions/workflows/data-validation.yml)
-[![Automated Release](https://github.com/Trojan3877/SentinelAI/actions/workflows/release.yml/badge.svg)](https://github.com/Trojan3877/SentinelAI/actions/workflows/release.yml)
-[![Python 3.11](https://img.shields.io/badge/python-3.11-blue.svg)](https://www.python.org/downloads/release/python-3110/)
-[![Framework: MLflow](https://img.shields.io/badge/Framework-MLflow-005b96.svg?logo=mlflow)](https://mlflow.org/)
-[![Code Style: Flake8](https://img.shields.io/badge/code%20style-flake8-black)](https://flake8.pycqa.org/)
-[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
+[![CI](https://github.com/CoreyLeath-code/SentinelAI/actions/workflows/ci-cd.yml/badge.svg)](https://github.com/CoreyLeath-code/SentinelAI/actions/workflows/ci-cd.yml)
+[![Benchmarks](https://github.com/CoreyLeath-code/SentinelAI/actions/workflows/benchmarks.yml/badge.svg)](https://github.com/CoreyLeath-code/SentinelAI/actions/workflows/benchmarks.yml)
+[![Security](https://github.com/CoreyLeath-code/SentinelAI/actions/workflows/security.yml/badge.svg)](https://github.com/CoreyLeath-code/SentinelAI/actions/workflows/security.yml)
+[![SAST](https://github.com/CoreyLeath-code/SentinelAI/actions/workflows/sast.yml/badge.svg)](https://github.com/CoreyLeath-code/SentinelAI/actions/workflows/sast.yml)
+[![Schema Validation](https://github.com/CoreyLeath-code/SentinelAI/actions/workflows/data-validation.yml/badge.svg)](https://github.com/CoreyLeath-code/SentinelAI/actions/workflows/data-validation.yml)
+[![Release](https://github.com/CoreyLeath-code/SentinelAI/actions/workflows/release.yml/badge.svg)](https://github.com/CoreyLeath-code/SentinelAI/actions/workflows/release.yml)
+[![Coverage](https://img.shields.io/badge/focused%20API%20coverage-24%25-red)](#test-and-evidence-status)
+[![Benchmark](https://img.shields.io/badge/reference%20p95-54.7%20%C2%B5s-6f42c1)](benchmarks/benchmark_report.md)
+[![Throughput](https://img.shields.io/badge/reference%20throughput-23.0k%20ops%2Fs-2ea44f)](benchmarks/benchmark_report.md)
+[![Python](https://img.shields.io/badge/Python-3.11-blue.svg)](https://www.python.org/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
 ---
 - Detect model drift
 - Monitor inference anomalies
@@ -149,38 +135,45 @@ User → Go Ingestion API (8080) → Postgres (local) / Snowflake (optional)
 
 ---
 
-## 📊 Research Benchmarks & Recorded Metrics
+## 📊 Research Metrics & Benchmarks
 
-Measured and normalized on 2026-07-12. Values are grouped by provenance so
-documented targets, local smoke benchmarks, and reproducibility metrics remain
-auditable.
+The committed baseline is generated by a seeded, dependency-free harness that mirrors the PSI/KS decision rule in the C++ drift engine. These numbers measure the Python reference implementation—not native C++ or end-to-end HTTP latency. See the [full methodology, interpretation, and limitations](benchmarks/benchmark_report.md) and [raw JSON evidence](benchmarks/latest.json).
 
-### Recorded Platform Benchmarks
+### Latest reproducible baseline
 
-| Component | Metric | Recorded Value | Provenance |
-|---|---:|---:|---|
-| C++ Drift Engine | PSI drift threshold | 0.20 | `drift-engine/drift_engine.cpp`, `Metrics.md` |
-| C++ Drift Engine | KS drift threshold | 0.10 | `drift-engine/drift_engine.cpp` |
-| C++ Drift Engine | Documented compute target | < 2 ms | `README.md`, `Metrics.md` |
-| Go Ingestion API | Documented p95 latency | 180 ms | `README.md`, `Metrics.md` |
-| Platform Throughput | Documented request rate | 150 RPS | `README.md`, `Metrics.md` |
-| LLM Guard | Documented summarization latency | ~1.2 s | `README.md`, `Metrics.md` |
-| Docker stack | Documented cold start | < 3 s | `Metrics.md` |
-| Local warehouse | Default backend | Postgres | `.env.example`, `README.md` |
-| Production warehouse | Optional backend | Snowflake | `README.md` |
-
-### Local Smoke Benchmarks
-
-| Benchmark | Input / Runs | Result | Notes |
-|---|---:|---:|---|
-| `SentinelModel` parameter count | 16 -> 32 -> 1 MLP | 577 trainable params | `api/core/model.py` |
-| `SentinelModel` forward pass | 1,000 CPU runs | 0.0160 ms avg | Synthetic tensor, no network or model I/O |
-| `SentinelModel` synthetic throughput | 1,000 CPU runs | 62,470.72 RPS | Local smoke benchmark |
-| PSI/KS drift calculation | 10,000 Python-equivalent runs | 0.001549 ms avg | Mirrors C++ formula for README sample payload |
-| README sample PSI | `[0.2,0.3,0.25,0.25]` -> `[0.1,0.35,0.30,0.25]` | 0.086138 | Below PSI drift threshold |
-| README sample KS statistic | Same sample payload | 0.100000 | At KS threshold boundary |
-| README sample drift flag | PSI > 0.20 or KS > 0.10 | false | Strict threshold comparison |
-| LLM fallback summary path | 10,000 rule-based runs | 0.000104 ms avg | Ollama unavailable path only |
+| Metric | Value | Protocol |
+|---|---:|---|
+| Timed evaluations | 20,000 | 100 warm-ups, 32 bins, seed `20260718` |
+| Mean latency | 41.706 µs | Per-decision reference latency |
+| Median latency | 39.700 µs | Per-decision reference latency |
+| P95 / P99 latency | 54.700 / 76.200 µs | Linear percentile interpolation |
+| Minimum / maximum | 23.700 / 319.200 µs | Observed range |
+| Throughput | 23,031.13 operations/s | Single-process CPython reference |
+| Peak traced memory | 0.623 MiB | Python `tracemalloc` |
+| Precision / recall / F1 | 1.000 / 1.000 / 1.000 | 2,000 balanced synthetic cases |
+| Confusion matrix | TP 1000 · TN 1000 · FP 0 · FN 0 | Controlled seeded classes |
+| Environment | CPython 3.12.13 · Windows 11 | Recorded 2026-07-18 |
+
+### Benchmark scope and reproducibility
+
+```bash
+python benchmarks/run_benchmark.py --output benchmarks/latest.json
+```
+
+CI reruns the benchmark on every pull request, validates its schema and F1 regression floor, and uploads raw evidence for 30 days. For comparable hosts, median or P95 increases above 15% require investigation and a documented baseline update.
+
+The perfect synthetic classification result is a regression signal for deliberately separated perturbation classes; it is **not** a production accuracy claim. Native C++, service concurrency, network, warehouse, GPU, and real-world labeled drift benchmarks remain future evaluation layers.
+
+### Test and evidence status
+
+| Evidence | Current state | Source |
+|---|---:|---|
+| Focused API tests | 4 passed | Existing repository audit |
+| Focused API coverage | 24% | Existing repository audit; below 90% target |
+| Benchmark raw data | Versioned JSON | `benchmarks/latest.json` |
+| Benchmark methodology | Versioned report | `benchmarks/benchmark_report.md` |
+| Benchmark CI | Required execution + artifact | `.github/workflows/benchmarks.yml` |
+| Drift thresholds | PSI 0.20 · KS 0.10 | `drift-engine/drift_engine.cpp` |
 
 ### Observability Metrics
 

benchmarks/benchmark_report.md

@@ -0,0 +1,53 @@
+# SentinelAI Research Benchmark Report
+
+## Executive summary
+
+This report records a deterministic evaluation of the PSI/KS drift-decision rule used by SentinelAI. The committed baseline is a portable Python reference implementation that mirrors the thresholds and equations in `drift-engine/drift_engine.cpp`. It is intentionally reported separately from native C++ and end-to-end HTTP latency.
+
+| Metric | Baseline |
+|---|---:|
+| Samples | 20,000 timed + 2,000 labeled |
+| Mean latency | 41.706 µs |
+| Median latency | 39.700 µs |
+| P95 latency | 54.700 µs |
+| P99 latency | 76.200 µs |
+| Min / max latency | 23.700 / 319.200 µs |
+| Throughput | 23,031.13 operations/s |
+| Peak traced Python memory | 0.623 MiB |
+| Precision / recall / F1 | 1.000 / 1.000 / 1.000 |
+| Confusion matrix | TP 1000, TN 1000, FP 0, FN 0 |
+
+## Protocol
+
+- Fixed seed: `20260718`
+- Runtime: CPython 3.12.13 on Windows 11
+- Workload: 32-bin normalized gamma distributions
+- Timing population: 20,000 evaluations after 100 warm-up evaluations
+- Quality population: 2,000 balanced synthetic examples
+- Positive class: strong seeded perturbation (`magnitude=1.8`)
+- Negative class: weak seeded perturbation (`magnitude=0.05`)
+- Decision rule: `PSI > 0.20 or KS > 0.10`
+- Percentiles: linear interpolation over ordered observations
+- Memory: peak allocations reported by Python `tracemalloc`
+
+Reproduce with:
+
+```bash
+python benchmarks/run_benchmark.py --output benchmarks/latest.json
+```
+
+## Interpretation
+
+The baseline demonstrates that the reference decision rule is inexpensive and perfectly separates the deliberately distinct synthetic classes in this controlled experiment. The quality result is a pipeline regression check, not a claim of production model accuracy. Native C++ service latency, network latency, database effects, concurrency, and real-world drift labels remain separate evaluation domains.
+
+## Regression policy
+
+CI fails when the benchmark cannot execute or its artifact is invalid. Performance changes are surfaced by the checked-in JSON diff and uploaded artifact. A performance regression should be investigated when median or P95 latency increases by more than 15% on comparable hardware and runtime; update the baseline only with an explanation of the environment and cause.
+
+## Evidence and limitations
+
+Raw machine-readable evidence is stored in [latest.json](latest.json). The harness and seed are versioned in [run_benchmark.py](run_benchmark.py).
+
+- The benchmark measures the Python reference implementation, not the native C++ binary or HTTP service.
+- Synthetic labels validate repeatability and decision behavior; they do not replace production ground truth.
+- Cross-host microbenchmark comparisons require comparable CPU, OS, Python version, power policy, and background load.

benchmarks/latest.json

@@ -0,0 +1,42 @@
+{
+  "schema_version": 1,
+  "benchmark": "python-reference-drift-decision",
+  "generated_at": "2026-07-18T20:15:54.020637+00:00",
+  "seed": 20260718,
+  "environment": {
+    "python": "3.12.13",
+    "platform": "Windows-11-10.0.26200-SP0"
+  },
+  "protocol": {
+    "iterations": 20000,
+    "warmup": 100,
+    "bins": 32,
+    "evaluation_samples": 2000
+  },
+  "latency_us": {
+    "mean": 41.705635,
+    "median": 39.7,
+    "p95": 54.7,
+    "p99": 76.2,
+    "min": 23.7,
+    "max": 319.2
+  },
+  "throughput_ops_s": 23031.134524050663,
+  "peak_tracemalloc_mib": 0.6227264404296875,
+  "quality": {
+    "precision": 1.0,
+    "recall": 1.0,
+    "f1": 1.0,
+    "confusion_matrix": {
+      "tp": 1000,
+      "tn": 1000,
+      "fp": 0,
+      "fn": 0
+    }
+  },
+  "limitations": [
+    "Measures the Python reference implementation, not native C++ or HTTP latency.",
+    "Quality labels use seeded synthetic perturbation classes, not production ground truth."
+  ]
+}
+