Duskmeter

Corrected sky brightness estimates from fused satellite and ground-truth data.

Duskmeter reconciles VIIRS satellite radiance with citizen-science sky brightness observations to produce site-by-site corrected light pollution estimates. It quantifies the spectral divergence between what the satellite sees and what the human eye observes, attributing the gap to LED blue-light emission that falls outside the VIIRS sensor window.

---

Why this exists

Every public light pollution map (NOAA, Light Pollution Atlas, the World Atlas) uses the VIIRS Day/Night Band as its only data source. That sensor was calibrated for high-pressure sodium lamps, which emit almost all of their light at 589 nm inside the sensor's detection range of roughly 500 to 900 nm.

White and blue-rich LEDs (now the dominant streetlight type worldwide) emit strongly at around 450 nm, below the sensor's window. The satellite literally cannot see the fastest-growing source of light pollution. The result is that every public map systematically underestimates sky brightness wherever LEDs have replaced sodium lighting.

Duskmeter cross-references VIIRS data with ground-truth observations from Globe at Night and SQM meters to estimate what the sky actually looks like, not just what the satellite reports.

---

How it works

Duskmeter uses a per-site physical calibration model to relate satellite radiance to ground-level sky brightness:

predicted_brightness = α × VIIRS_radiance + β

• α (slope): The atmospheric scattering coefficient. How much upwelling light scatters back to the ground. Depends on elevation, humidity, and aerosol load.
• β (intercept): The natural sky background. Airglow, zodiacal light, and unresolved starlight.

For sites without calibration data, α and β come from published population-level priors (Falchi 2016, Duriscoe 2013, Kyba 2017) specific to each site type (observatory, dark-sky park, or urban). Sites with enough matched SQM and VIIRS readings can be fitted automatically using Bayesian linear regression.

The fusion pipeline:

1. Ingest: VIIRS yearly GeoTIFFs are pre-processed once into SQLite via POST /api/admin/ingest-viirs. Supports BigTIFF (64-bit offsets for files over 4 GB) and regular TIFF.
2. Calibrate: The system tries to fit a site-specific α and β from matched VIIRS and ground-truth readings. Falls back to population priors if fewer than 8 matched readings are available.
3. Predict: Each VIIRS reading is converted to predicted sky brightness using the calibration.
4. Compare: Residuals between predicted and observed brightness are computed. If residuals grow over time, that is the signature of LED spectral divergence.
5. Report: The corrected trend, its uncertainty, the spectral divergence fraction, and the calibration source are stored and displayed.

---

Tech stack

Layer	Technology
Framework	SvelteKit
3D globe	Three.js
Styling	Tailwind CSS v4
Maps	Leaflet
Charts	Canvas 2D API (custom trend charts)
Database	better-sqlite3
Statistics	Bayesian linear regression with conjugate priors
Language	TypeScript, end-to-end

---

Getting started

npm install
npm run dev

The development server runs at http://localhost:5173.

Commands

Command	Description
npm run dev	Start development server
npm run build	Build for production
npm run preview	Preview production build
npm run check	Run Svelte type checking
npm run test	Run unit tests (Vitest)

Deployment

Environment variables

Variable	Required	Default	Description
ADMIN_SECRET	Yes	—	Shared token to protect admin endpoints. Pass via Authorization: Bearer <token> header. Without it, sync and ingest endpoints return 401.
NODE_ENV	No	development	Set to production to disable the TIFF file reader fallback. In production, VIIRS data must be pre-ingested.
LLM_PROVIDER	No	ollama	LLM provider for site narrative generation. Options: ollama, openai, anthropic.
LLM_MODEL	No	llama3	Model name for the selected provider.
OPENAI_API_KEY	No	—	Required only when LLM_PROVIDER=openai.
ANTHROPIC_API_KEY	No	—	Required only when LLM_PROVIDER=anthropic.
OLLAMA_BASE_URL	No	http://localhost:11434	Ollama server URL.
ASSESS_INTERVAL_MINUTES	No	360	Minutes between scheduled Bayesian assessments.

Docker (recommended)

A prebuilt image is published to GitHub Container Registry on every release. Tags follow semantic versioning: latest, v1, v1.0, v1.0.0.

Prerequisites

• Docker and Docker Compose installed
• A VIIRS yearly file downloaded from EOG and placed at data/viirs/VNL_npp_{year}_global_vcmslcfg_v2_c*.average.dat.tif.gz
• A value for ADMIN_SECRET — set it in a .env file or export it as an environment variable

Understanding the database

The application uses SQLite with a single file as its database. When running in development (npm run dev), the file is named duskmeter-dev.sqlite. When running in production (Docker or node build), it uses duskmeter.sqlite. This separation prevents your development work from accidentally overwriting production data.

The database file is bind-mounted from your host into the container:

./duskmeter.sqlite (on your machine) → /app/duskmeter.sqlite (inside the container)

The container reads and writes directly to your host file. When the container restarts, is stopped, or is replaced with a newer image, the file stays on your machine — nothing is lost.

Option A: Fresh start (no existing data)

# 1. Ensure the VIIRS file is in place
ls data/viirs/VNL_npp_*.tif.gz

# 2. Start the container (creates an empty database, seeds 12 default sites)
docker compose up -d

# 3. Ingest the VIIRS GeoTIFF into the database (reads pixel values at each site)
curl -X POST "http://localhost:3000/api/admin/ingest-viirs" \
  -H "Authorization: Bearer your-secret"

# 4. Run the initial sync (fetches Globe at Night data, computes metrics)
curl -X POST "http://localhost:3000/api/sync?force=true" \
  -H "Authorization: Bearer your-secret"

# 5. Verify
curl http://localhost:3000/api/health
# {"status":"ok","db":"connected","sites":12}

# 6. Open the dashboard
open http://localhost:3000

Option B: Migrate from development

If you have been running npm run dev and already have real VIIRS data and metrics in your database:

# Rename the dev database for production use
cp duskmeter-dev.sqlite duskmeter.sqlite

# Start the container — no re-ingestion needed, data is already in the database
docker compose up -d

# Verify
curl http://localhost:3000/api/health
# {"status":"ok","db":"connected","sites":12}

How data flows

Component	Host location	Container location	What happens
VIIRS GeoTIFF files	./data/viirs/	/app/data/viirs/	Mounted read/write. Ingest decompresses the file in /tmp, reads pixel values, stores results in the database, and deletes the temp file
Globe at Night cache	./data/gan/	/app/data/gan/	Mounted read/write. The container fetches remote data from NOIRLab and caches it here for faster subsequent syncs
SQLite database	./duskmeter.sqlite	/app/duskmeter.sqlite	Bind-mounted. The container reads and writes directly to your host file. Survives container restarts, rebuilds, and image updates
Application logs	stdout	stdout	View with docker compose logs -f duskmeter. JSON lines with timestamps, levels, and context

Adding new VIIRS years

When a new yearly composite is released, download it and place it alongside existing files in data/viirs/. Then run ingestion and sync again — only the new year's data is processed:

curl -X POST "http://localhost:3000/api/admin/ingest-viirs" \
  -H "Authorization: Bearer your-secret"
curl -X POST "http://localhost:3000/api/sync?force=true" \
  -H "Authorization: Bearer your-secret"

Backups

The entire application state is in one file. Back it up with:

cp duskmeter.sqlite duskmeter.sqlite.$(date +%Y-%m-%d).bak

The VIIRS files and Globe at Night cache in ./data/ are reproducible — they can be re-downloaded from their respective sources. Only the database needs backing up.

Updating the container

docker compose pull        # Pull the latest published image
docker compose up -d       # Replace the container — database and data files are untouched

Verifying the setup

# Health check — confirms DB is connected, shows site count and last sync time
curl http://localhost:3000/api/health

# List all sites with their corrected trends
curl http://localhost:3000/api/sites | python3 -m json.tool | head -20

# Check for active alerts
curl http://localhost:3000/api/alerts | python3 -m json.tool | head -10

Standalone Node.js

npm install
npm run build

# Place your VIIRS yearly file
cp VNL_npp_2025_global_vcmslcfg_v2_c202604011200.average.dat.tif.gz data/viirs/

# Ingest and sync
curl -X POST "http://localhost:3000/api/admin/ingest-viirs" \
  -H "Authorization: Bearer your-secret"

curl -X POST "http://localhost:3000/api/sync?force=true" \
  -H "Authorization: Bearer your-secret"

# Start
NODE_ENV=production ADMIN_SECRET=your-secret node build

The production server listens on port 3000 by default. Set PORT to override.

Health check

curl http://localhost:3000/api/health
# {"status":"ok","version":"0.1.0","uptime":120,"db":"connected","sites":12,"lastSync":"..."}

CI/CD

Two GitHub Actions workflows run automatically:

Workflow	Trigger	What It Does
CI	Push to main, pull requests	Type check, run tests, verify production build
Release	GitHub Release published	Build multi-arch Docker image (linux/amd64, linux/arm64), push to ghcr.io/thelinuxguy-ssh/duskmeter with semver tags + latest

To create a release:

git tag v0.1.0
git push origin v0.1.0

Then create a release on GitHub from that tag. The workflow builds and publishes the container automatically.

---

Data sources

Duskmeter relies on two public datasets:

EOG Nighttime Lights (VIIRS/DNB)

Annual cloud-free composite nighttime lights at roughly 15 arc-second resolution, produced by the Earth Observation Group, Payne Institute for Public Policy, Colorado School of Mines. VIIRS/DNB data are in the public domain.

To use real satellite data:

1. Download yearly files from eogdata.mines.edu and place them in data/viirs/.
2. The minimum required file is the average radiance composite:

   VNL_npp_{year}_global_vcmslcfg_v2_c*.average.dat.tif.gz
   

3. Optionally, add the cloud-free coverage file to get observation counts per pixel:

   VNL_npp_{year}_global_vcmslcfg_v2_c*.cf_cvg.dat.tif.gz
   

4. Run ingestion: curl -X POST "http://localhost:3000/api/admin/ingest-viirs" -H "Authorization: Bearer your-secret"
5. The ingestion pipeline decompresses the gzipped TIFF, reads pixel values at each site location, stores them in the database, and cleans up the temporary file.

A deterministic synthetic fallback is used for any year without a local file. The ingest endpoint takes roughly one to two minutes per yearly file.

Globe at Night

Citizen-science measurements of naked-eye limiting magnitude and sky quality (SQM). Data copyright Globe at Night, licensed under CC BY 4.0, fetched and cached locally in data/gan/. Attribution is required per license terms.

Attribution

When distributing Duskmeter output, credit the Earth Observation Group (VIIRS/DNB) and Globe at Night contributors (CC BY 4.0).

---

Project structure

src/
├── app.css                  # Global styles, theme tokens, form controls
├── app.html                 # HTML shell
├── hooks.server.ts          # Server-side hooks
├── assets/                  # Static assets (favicon)
├── lib/
│   ├── components/
│   │   ├── Globe.svelte     # Three.js 3D Earth with heatmap-style light pollution
│   │   ├── MapView.svelte   # Leaflet map with site markers and zoom-to
│   │   ├── SiteList.svelte  # Sidebar with per-site metrics and calibration info
│   │   ├── SiteDetail.svelte# Modal with full metrics, charts, and Bayesian assessment
│   │   ├── TrendChart.svelte# Canvas 2D trend visualization
│   │   ├── Header.svelte    # App header with nav, theme toggle, and refresh
│   │   ├── FilterBar.svelte # Search and filter controls
│   │   └── ui/              # Reusable UI components (buttons, badges, cards, logo)
│   ├── data/
│   │   ├── types.ts         # Shared TypeScript interfaces
│   │   ├── mock.ts          # Client-side mock data fallback
│   │   └── world-rings.json # Continent outline polygons for the globe
│   ├── server/
│   │   ├── db.ts            # Lazy-init SQLite with schema and migrations
│   │   ├── seed.ts          # Initial site and data seeding
│   │   ├── recalculate.ts   # Orchestrates recalculation of site metrics
│   │   ├── assess.ts        # Bayesian assessment pipeline
│   │   ├── alerts.ts        # Alert generation engine
│   │   ├── scheduler.ts     # Background assessment scheduling
│   │   ├── llm.ts           # Optional LLM narrative (Ollama, OpenAI, Anthropic)
│   │   ├── export.ts        # CSV stringifier
│   │   └── data-sources/
│   │       ├── viirs.ts         # BigTIFF/GeoTIFF parser, pixel reader, synthetic fallback
│   │       ├── calibration.ts   # Site calibration model with Bayesian fitting and population priors
│   │       ├── calculator.ts    # Calibration-based fusion metrics
│   │       ├── bayesian.ts      # Bayesian linear regression with conjugate priors
│   │       ├── globe-at-night.ts# Globe at Night data fetcher and cache
│   │       ├── quality-engine.ts# Ground report quality filtering by moon/cloud
│   │       └── change-point.ts  # Sliding-window change point detection
│   └── stores/
│       ├── sites.ts         # Site data stores, filters, fetch/sync/assess functions
│       ├── theme.ts         # Theme mode store (light, dark, night-vision)
│       └── ui.ts            # UI state store
└── routes/
    ├── (marketing)/         # Landing page with hero, globe, and methodology sections
    ├── (app)/               # Dashboard, global overview, methodology, alerts
    └── api/                 # REST API for sites, sync, ingest, export, and calibration

---

API endpoints

Endpoint	Method	Purpose
/api/sites	GET	List all sites with metrics
/api/sites/{id}	GET	Site detail with readings, metrics, alerts, and assessment
/api/sites/{id}/refresh	POST	Refresh single site from data sources
/api/sites/{id}/assess	POST	Run Bayesian assessment for a site
/api/sites/{id}/calibration	POST	Upload SQM/limiting magnitude readings for calibration
/api/sites/{id}/export	GET	Export site data as CSV or JSON
/api/sync	POST	Full sync of all sites (VIIRS, ground, metrics)
/api/admin/ingest-viirs	POST	Ingest VIIRS yearly GeoTIFF into database
/api/assess-all	POST	Run Bayesian assessment for all sites
/api/alerts	GET	List active alerts
/api/alerts/{id}/acknowledge	POST	Acknowledge an alert
/api/export	GET	Export all site metrics as CSV or JSON

---

Known limitations

• Calibration data: Most sites currently use population-level priors rather than site-specific calibration because calibrated SQM readings matched to satellite overpasses are not yet available for all locations. The corrected trends show appropriately wide uncertainty bounds for population-calibrated sites.
• Single-year VIIRS data: The repository ships with synthetic data for years 2012 through 2024 and real VIIRS data for 2025 only. Corrected trends will become more stable as more real satellite years are ingested.
• Coverage files: Cloud-free observation counts default to 1 per site when the cf_cvg file is not placed alongside the average file. This reduces the precision of data quality scoring.
• Ground observations: Globe at Night provides sparse and noisy citizen-science reports. The quality engine filters reports with high moon illumination or cloud cover, but the data remains inherently less dense than the satellite record.

---

Contributing

Contributions are welcome. See the issues directory for a curated list of good first issues and medium-difficulty tasks, ranging from TypeScript bug fixes to new feature implementation.

In short:

1. Fork the repository.
2. Create a feature branch.
3. Run npm run check before committing.
4. Submit a pull request.

---

License

Apache 2.0