Gem Finder

A browser-based tool for discovering recent preprints on arXiv and chemRxiv for editorial acquisition, powered by the OpenAlex open scholarly metadata API.

Live: https://teowaits.github.io/gem_finder/

Given a topic, a title phrase, an author, or a list imported from a companion tool, the app surfaces preprints published within a configurable time window, displays publication-status indicators, shows on-demand author profiles, and exports selected results for review.

Features

Mode	What it does
Topic search	Autocomplete against OpenAlex's full topic taxonomy (Subfield + Topic levels). Multi-select with OR / AND boolean. Optional Refine by title filter. Pre-flight count check before fetching.
Title search	Search preprint titles by phrase (OpenAlex `title.search`, stemmed matching). Multi-phrase with OR / AND. Quick add presets for regularly used phrases (configured in `constants.js`). Pre-flight count check before fetching.
Author search	Name search with disambiguation card, or direct ORCID lookup. Toggle between repository-only preprints and all works. Optional Refine by title filter.
CSV import	Accepts flexible CSV column names from companion tools. All rows are imported; up to 25 can be toggled on/off for the search. Topics searched with OR; authors offer a Repository only / All works scope toggle. Optional Refine by title filter on both passes.
Author profiles	On-demand per result card: institution, career citations, publication count, "publishing since" year, and recent venue history — three API calls, cached for the session.
Publication status	Green / amber / grey indicator per card based on DOI prefix and source type. Tooltip: "Verify publication status before contacting".
CSV export	Exports selected results as two files: a full results CSV and a shared-schema authors CSV for import into companion tools.

Using your own OpenAlex key

OpenAlex is free but usage is metered. Without a key the shared free tier is enough for light use; Gem Finder's topic autocomplete, author search, and title-phrase search are search-tier calls ($0.001 each) and burn budget faster than list/filter calls.

Request type	Examples in Gem Finder	Cost per call
List + filter	Topic/source/date filters, work pagination	$0.0001
Search	`title.search`, topic/subfield autocomplete	$0.001

On the public page: open API key (gear) in the header, paste a free personal key from openalex.org/settings/api, and you're on your own $1/day budget (~1,000 search-tier calls). The key is validated immediately via OpenAlex's /rate-limit endpoint.

Where the key lives: only in this browser (localStorage), and it is sent only to OpenAlex. There is no Gem Finder server — nothing is uploaded to us. Use Clear key in the same popover to remove it and fall back to the shared free tier.

Public deployment rule: the GitHub Pages build must never bake in VITE_OPENALEX_API_KEY. A shared build-time key would be visible to every visitor and would share one budget across all public traffic. Visitors bring their own key in the UI instead. (The deploy workflow comments this as a hard constraint.)

Running Locally

Requirements: Node.js 18+ and npm.

# 1. Clone the repo
git clone https://github.com/teowaits/gem_finder.git
cd gem_finder

# 2. Install dependencies
npm install

# 3. (Optional, local only) Seed a key via env
cp .env.example .env.local
# Edit .env.local and set VITE_OPENALEX_API_KEY=…  (free at openalex.org/settings/api)

# 4. Start the dev server
npm run dev

Then open http://localhost:5173/gem_finder/ in your browser.

# Build for production
npm run build

# Preview the production build
npm run preview

For local development you can either:

Paste a key in the API key UI (same as the public page — stored in localStorage), or
Set VITE_OPENALEX_API_KEY in .env.local so every request starts keyed without pasting. Restart the dev server after changing the env file.

A user-entered key always takes precedence over the build-time env key. The public Pages build leaves the env var unset on purpose — do not add it as a GitHub Actions secret.

Title-phrase search and topic autocomplete are 10× more expensive per call than topic/source filtering. A title search uses at least two search-tier calls (pre-flight + fetch). Keep Refine by title collapsed when not in use — leftover phrases turn cheap topic queries into search-tier ones.

Preset title phrases

Edit PRESET_TITLE_PHRASES in src/constants.js to add click-to-add shortcuts (shown as Quick add in Title mode and Refine by title):

export const PRESET_TITLE_PHRASES = [
  'closed loop',
  { label: 'XC regularizers', phrase: 'Physical exact conditions as regularizers for exchange-correlation in solids and surface chemistry' },
];

Use a plain string for short phrases, or { label, phrase } when the button label should differ from the search text. Restart the dev server after editing.

How It Works

Every search mode goes through a single OpenAlex /works filter pipeline (buildWorksFilter in api.js). arXiv and chemRxiv are primary_location.source.id values — Gem Finder never calls those repositories directly. All API calls route through one withApiKey() helper; a runtime key (from the UI) or optional local env key is appended there — no per-call-site plumbing.

Source verification — at startup, the hardcoded arXiv and chemRxiv source IDs are verified against the OpenAlex /sources endpoint. A non-blocking warning banner appears if either ID has changed.
Topic autocomplete — queries the /topics and /subfields endpoints in parallel, merges results (subfields first), and presents a combined dropdown of up to 10 matches.
Title-phrase search — adds an OpenAlex title.search: filter clause (stemmed matching; e.g. "closed loop" also matches "closing loops"). Multiple phrases combine with OR (pipe-joined) or AND (separate clauses). Works as a standalone Title mode or as Refine by title in Topic, Author, and Import modes — one shared titlePhrases state, not separate code paths. Result cards show display-only Title match tags for phrases found as substrings in the returned title.
Pre-flight count check — before every topic, title, or import fetch, a single per_page=1 request reads meta.count. If the result set exceeds 250 the user is warned; if it is zero a broadening suggestion is shown. Not used in Author mode (naturally bounded by author ID).
Work fetching — paginated in batches of 200 with a 60 ms inter-request delay, capped at 250 results. Repository-scoped searches filter type:preprint (OpenAlex classifies arXiv/chemRxiv uploads as preprints, not articles). Author-only searches omit the type filter so "All works" can include journal articles; repository-only results are filtered client-side.
Author disambiguation — name searches return up to 8 candidates displayed with institution, top topics, and career stats. ORCID input bypasses the card and resolves directly.
Author profile — triggered per card on demand. Three sequential API calls fetch the career summary, earliest work date, and recent venue history; results are cached in a session Map.
CSV import — flexible column detection accepts headers from multiple companion-tool export formats (openalex_id, OpenAlex ID, Open Alex ID, Name, etc.; extra columns ignored). All rows are parsed without a hard cap; up to 25 can be selected via checkboxes (top 25 pre-ticked). Topic rows are fetched with OR (any matching topic area); author rows respect the Repository only / All works scope toggle. Passes are merged and deduplicated by OpenAlex Work ID. Title refinement, when active, narrows both topic and author passes.
Export — selected cards are written to two CSVs: a detailed results file and a shared-schema author file compatible with journal-overlap and journal-profile-analyser.

Title phrases are typed or quick-added each session — there is no CSV import/export for phrases (companion tools have no phrase concept upstream).

Practical limits

Cap	Value
Results per search	250
Title phrases per search	15
Topics / subfields per import	25
Authors per import	25
OpenAlex paging wall	10,000 results (50 pages × 200)
Inter-request delay	60 ms

CSV Import Format

Gem Finder accepts CSVs from companion tools with flexible column naming. A type column and an OpenAlex ID column are required; all other columns are optional and extras are ignored.

Topics / subfields (exported by journal-profile-analyser):

type,OpenAlex ID,display_name,notes
topic,T10104,Machine Learning,from journal-profiler
subfield,S2208,Artificial Intelligence,from journal-profiler

Authors (exported by journal-overlap):

type,Name,ORCID,Open Alex ID,Notes
author,Heather Piwowar,0000-0003-1613-5981,A5023888391,from journal-overlap

Mixed files containing both topics and authors are supported. Valid type values: topic, subfield, author. All rows are imported; use the checkboxes in the panel to select up to 25 for the search (top 25 pre-selected by default).

Export schema

Exports use the shared cross-tool schema:

type,openalex_id,display_name,notes,orcid
author,A5023888391,Heather Piwowar,gem-finder export,0000-0003-1613-5981

Companion Tools

Tool	Purpose
journal-overlap	Authorship overlap between two sets of journals
journal-profile-analyser	Topic and venue profiling for a journal (in development)

Data & Acknowledgements

All scholarly metadata is provided by OpenAlex — a fully open, free index of global research output maintained by OurResearch. OpenAlex data is released under the CC0 1.0 Universal public domain dedication.

Priem, J., Piwowar, H., & Orr, R. (2022). OpenAlex: A fully-open index of the world's research. arXiv. https://doi.org/10.48550/arXiv.2205.01833

Created By

teowaits

This tool was built with the assistance of Claude Sonnet 4.6 by Anthropic, following OpenAlex API best practices:

Source ID verification at startup rather than filtering by name
Pre-flight count checks before every paginated fetch
select= field filtering to minimise response payload
Client-side deduplication and status-indicator logic to avoid redundant API calls
AbortController cancellation on every in-flight request

Changelog

Version	Date	Changes
1.3.0	2026-07-31	Bring-your-own OpenAlex key — header settings popover, `localStorage` persistence, `/rate-limit` validation, runtime `setRuntimeApiKey` chokepoint; public GitHub Pages deploy with no baked-in key
1.2.0	2026-07-14	Title-phrase search — new Title mode and cross-mode Refine by title (OR/AND, stemmed `title.search`, sanitization warnings, result-card match tags); preset phrases via `PRESET_TITLE_PHRASES` quick-add; preprint type fix (`type:preprint` for repository searches); OpenAlex API key support via `VITE_OPENALEX_API_KEY`
1.1.0	2026-04-08	CSV import overhaul — flexible column detection accepts headers from multiple companion-tool formats; no row limit on import; checkbox selection UI (top 25 pre-ticked, 25-item cap enforced live); OR label for topics; Repository only / All works scope toggle for authors
1.0.0	2026-04-08	Initial release — topic search (OR/AND, subfield autocomplete, pre-flight check), author search (ORCID detection, disambiguation card, repository toggle), CSV import/export (shared schema), on-demand author profiles, publication status indicators, landing page with example topics

License

MIT

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
src		src
.env.example		.env.example
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
index.html		index.html
package-lock.json		package-lock.json
package.json		package.json
vite.config.js		vite.config.js

Navigation Menu

Repository files navigation

Gem Finder

Features

Using your own OpenAlex key

Running Locally

Preset title phrases

How It Works

Practical limits

CSV Import Format

Export schema

Companion Tools

Data & Acknowledgements

Created By

Changelog

License

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

Gem Finder

Features

Using your own OpenAlex key

Running Locally

Preset title phrases

How It Works

Practical limits

CSV Import Format

Export schema

Companion Tools

Data & Acknowledgements

Created By

Changelog

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages