Aporia

Identifies potentially under-explored connections between research areas.

Give Aporia a topic. It fetches a real paper corpus, embeds and clusters it, builds the citation graph between the papers, and surfaces pairs of related research clusters that cite each other far less than expected — ranked by a signal that's been backtested against real citation history, not just asserted to be reasonable.

Live demo: showcase mode — pre-computed results from 8 real pipeline runs, no setup required. To run a live query on your own topic, run the project locally (see Getting started).

---

What it does

1. Fetches a paper corpus from Semantic Scholar (arXiv fallback), cached locally so repeat runs never re-hit the API.
2. Embeds each paper with SPECTER2, a citation-aware scientific-paper encoder, caching every embedding per paper so nothing is ever encoded twice.
3. Filters off-topic papers by semantic similarity to the topic itself.
4. Clusters the corpus (UMAP → HDBSCAN) into research sub-areas, each labeled with an LLM-generated human-readable name (with a deterministic fallback if no LLM key is set).
5. Builds the directed citation graph across the corpus.
6. Scores every pair of clusters for how much less they cite each other than a degree-normalized null model would predict, combined with "open problem" language mined from abstracts.
7. Explains the top gaps with an LLM-generated research question, hypothesis, and suggested method.

The result is a ranked list of research gaps, an interactive citation graph, and per-cluster publication-growth charts, all in a full-stack app (FastAPI + React).

No API keys are required to run the core pipeline — Semantic Scholar and arXiv both work unauthenticated. LLM features (cluster names, hypotheses, advice) are optional and degrade to a deterministic fallback when no key is set.

---

Key results

• The gap score beats a random baseline on real citation history. Backtested across 8 topics: cluster pairs the score flags get connected by a real, later citation 88.9% of the time, vs. 80.0% for an equally-related but unflagged pair (+8.9 points). The formula it replaced did worse than its own baseline on the same data (-9.1 points). See Does it work? — this is a real but modest-sample result, not a clean sweep.
• The score's weights are fitted, not guessed. A logistic regression against 153 labeled cluster pairs (10 topics, cross-validated AUC 0.755) replaced a hand-set 70/30 weight split with a fitted 21/79 one. A first attempt on a smaller sample failed its own adoption bar and was correctly rejected rather than shipped anyway.
• A dead signal was found and replaced. The original citation signal had a standard deviation of 0.01 across 21 real gaps — it wasn't discriminating between anything. Its replacement (a null-model comparison) has a standard deviation of 0.29–0.38 on the same data — roughly a 30–40x improvement.
• A GNN was built, measured, and removed. An earlier version trained a GraphSAGE link-prediction model per job. Measurement showed its output was indistinguishable from chance on corpora this size, so it was removed rather than kept for its own sake.

---

Public deployment: showcase-only

The public deployment (Render backend + Vercel frontend) serves showcase mode only — real, pre-saved results from 8 completed pipeline runs (data/showcase/*.json), rendered through the exact same UI a live run uses.

Live queries (POST /api/query) are disabled there (ENABLE_LIVE_QUERY=false, returns 503) because the pipeline loads SPECTER2 and UMAP/HDBSCAN into memory simultaneously during a run, which doesn't fit a free-tier RAM budget. This is a deliberate cost trade-off, not a hidden limitation — run it locally for a real live query (see Getting started).

---

What's inside

Module	What it does
Showcase	Pre-saved results from real completed runs — instant, no live compute.
Live query	Submit a topic, watch the pipeline run stage by stage, get ranked gaps. (Local only.)
Search	Semantic search across every paper ever cached, across all topics.
Compare	Cross-topic semantic overlap between two already-run topics.
History	Re-open any past run's full results.
How it works	A stage-by-stage walkthrough of the pipeline.
Methods	The real backtest numbers, cluster-quality stats, and limitations, in one page.

Plus a System dialog (cache/job stats) and a light/dark theme toggle.

---

How gap scoring works

A "gap" is a relationship between two clusters, never a single paper or cluster. For every pair of clusters related enough to be worth comparing, two signals combine:

Signal	What it measures
Under-connection	How far below a degree-preserving null-model expectation the observed cross-cluster citation count falls — a capped z-score, so a deficit against a well-cited pair and a deficit against a barely-cited pair aren't treated the same.
Future-work density	Fraction of papers on both sides whose abstracts contain "open problem" / "under-explored" language.

gap_score = 0.21 × (under_connection × semantic_similarity)
          + 0.79 × future_work_density

Both weights were fitted (see Does it work?), not hand-picked. Two more signals — temporal lag and cluster-size asymmetry — are still computed and shown as context but no longer influence the score; measurement showed neither reliably distinguished a real gap from noise.

A real example, from the large-language-models fixture: one flagged pair observed 7 citations crossing between its two clusters where the null model — "given how much each cluster cites and is cited overall, how many cross-cluster citations would we expect if nothing special were going on here?" — expected 18.1. That deficit is what under_connection measures. The app shows this exact sentence ("N citations observed where E would be expected") for every gap.

Pairs below a semantic-similarity floor are dropped entirely — a "gap" is only a meaningful claim between areas related enough for the claim to make sense.

Does it work?

A backtest (full methodology and results) checks whether gap-flagged cluster pairs actually get connected by a real citation more often than an equally-related, unflagged pair. On 8 cached topics (cutoff year 2021), the current formula's flagged pairs connected 88.9% of the time vs. 80.0% for the baseline (+8.9 points); the formula it replaced did worse than its own baseline on the same data (-9.1 points). Three of eight topics show a decisive win for the current formula, one shows a real regression, and four saturated to 100% either way (no discriminating signal). A real, positive, modest-sample result — not a settled one.

---

Limitations

• Citations are corpus-bounded. The citation graph only includes edges between papers already in the fetched corpus. A real citation to a paper outside that set is invisible — "under-connected in this corpus" isn't the same as "under-connected in the literature."
• Retrieval quality drives everything downstream. A narrow or ambiguous topic can return an off-target corpus that clusters and scores just as confidently as a well-targeted one — see the computational-paleography-of-historical-manuscripts showcase fixture for a real, unforced example.
• Clustering isn't perfectly reproducible run to run. A different random seed can move papers between clusters (measured ARI seed-stability as low as 0.25–0.63 on several real corpora). The seed is pinned so this pipeline's own runs are reproducible; the instability is about what a different seed would have found instead.
• Low citation connectivity ≠ "a research gap." A pair can be under-connected for mundane reasons (different subfields' citation habits, a genuinely early-stage area, retrieval noise). The backtest measures whether flagged pairs get bridged more often than similar unflagged pairs — a real, positive signal, not a causal claim about why a gap exists.

---

Architecture

aporia
├── core/                        # the pipeline, one package per stage
│   ├── ingestion/               # Semantic Scholar + arXiv clients, corpus cache
│   ├── embedding/                # SPECTER2 encoder + SQLite vector cache
│   ├── clustering/               # UMAP reduce → HDBSCAN → cluster labels
│   ├── graph/                    # directed citation graph + density metrics
│   ├── gap_detection/            # the two gap signals + the combiner
│   ├── analysis/                 # cross-topic comparison (Compare module)
│   ├── validation/               # offline backtest (gap signal vs. real citations)
│   └── llm/                      # LLM-backed advice, hypotheses, cluster names
│
├── backend/                     # FastAPI app wrapping the pipeline
├── frontend/                    # Vite + React + TypeScript
├── scripts/                     # run_pipeline.py (CLI), backtest.py, fit_gap_weights.py
├── tests/                       # pytest suite for the signal math
├── docker/                      # backend + frontend Dockerfiles
├── data/                        # gitignored, except data/showcase/*.json fixtures
└── render.yaml

Data flow:

Semantic Scholar / arXiv  →  corpus cache  →  SPECTER2 embeddings (cached)
    →  relevance filter  →  UMAP → HDBSCAN clusters  →  citation graph
    →  gap signals → gap_scorer → ranked gaps  →  LLM hypotheses
    →  FastAPI job API  →  React frontend

The pipeline runs two ways: as scripts/run_pipeline.py (offline, one stage at a time via --stage fetch|embed|cluster|graph|score, or --stage all), or as an async HTTP job through the FastAPI backend. Both call the same stage functions and share the same caches, so work done one way is reused by the other.

---

Getting started

Prerequisites: Python 3.11+, Node.js 18+, ~2–4 GB disk (SPECTER2 model weights + caches).

git clone https://github.com/yourusername/aporia.git
cd aporia

# Python
pip install -r requirements.txt
pip install torch==2.12.1 --index-url https://download.pytorch.org/whl/cpu

# Frontend
cd frontend && npm install && cd ..

Run the backend and frontend in separate terminals:

uvicorn backend.main:app --reload      # backend, http://localhost:8000
cd frontend && npm run dev             # frontend, http://localhost:5173

Or both together: docker-compose up --build.

Then submit a live query, either through the Live query tab in the UI, or:

curl -X POST http://127.0.0.1:8000/api/query -H "Content-Type: application/json" \
  -d '{"topic": "Adversarial Robustness in Deep Learning", "limit": 200}'
# -> {"job_id": "<uuid>"}

curl http://127.0.0.1:8000/api/status/<uuid>    # poll until "done" or "failed"
curl http://127.0.0.1:8000/api/results/<uuid>   # full result once done

The first run of a topic fetches and embeds the corpus (a minute or two, depending on limit); re-runs of the same topic are served almost entirely from cache.

Tests: pip install pytest && pytest — pure signal-math unit tests, no network/torch, run fast.

---

API

REST API under http://localhost:8000/api. Interactive docs at /docs while running.

Method & path	Purpose
POST /api/query	Submit a live pipeline job → { job_id }. Body: { topic, limit, refresh? }.
GET /api/status/{job_id}	Poll job status.
GET /api/results/{job_id}	Full results once done (202 while running).
GET /api/jobs	Paginated list of past jobs.
GET /api/showcase	List pre-saved showcase fixtures.
GET /api/showcase/{slug}	One fixture's full results.
GET /api/search?q=&limit=	Semantic search across every cached paper.
GET /api/compare?topic_a=&topic_b=	Cross-topic cluster overlap.
GET /api/gaps/{job_id}/{gap_id}/advice	LLM-backed suggestion for one gap.
GET /api/system/stats	Cache / job / embedding stats.
GET /api/health	Health check.

topic must be non-empty, ≤200 chars; limit must be in [50, 800]. Only one pipeline job runs at a time — a second concurrent query gets 429. Read-heavy routes are per-IP rate-limited (/api/search: 10/min; /api/compare, /api/system/*: 30/min).

---

Configuration

See .env.example for the full annotated list. The essentials:

Variable	Purpose
LLM_PROVIDER	gemini | groq | none (default gemini). Picks which backend powers gap advice, hypotheses, topic normalization, and cluster labels.
GEMINI_API_KEY / GROQ_API_KEY	Optional. Numbered keys (_1, _2, ...) enable rotation across quotas.
ENABLE_LIVE_QUERY	Defaults on. false disables POST /api/query (the showcase-only gate).
SEMANTIC_SCHOLAR_API_KEY	Optional. Raises the ingestion rate limit.
ALLOWED_ORIGINS	CORS allowlist — set to your deployed frontend URL in production.

LLM_PROVIDER=none disables the LLM layer entirely (no network call, no SDK import) — every feature falls back to a deterministic result. The public showcase deployment uses this: it never calls an LLM at request time, since every label/hypothesis on the live demo is already baked into the committed showcase fixtures.

---

Deployment

• Backend → Render, as a Docker service (docker/backend.Dockerfile), health-checked at /api/health. render.yaml sets ENABLE_LIVE_QUERY=false.
• Frontend → Vercel, native Vite build, VITE_API_BASE_URL pointed at the live Render origin at build time.

Because the public deployment is showcase-only, it needs no GPU, no live-pipeline RAM, and no paid tier.

---

Tech stack

Pipeline: SPECTER2 (transformers + adapters) · UMAP · HDBSCAN · NetworkX · PyTorch · SQLite · Gemini / Groq

Backend: Python · FastAPI · Uvicorn · SQLAlchemy · SQLite · httpx · slowapi

Frontend: React 19 · TypeScript · Vite · Sigma.js + graphology · Recharts · Radix UI · Tailwind