Skip to content

Latest commit

 

History

28 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ScrapeForensics

Incident response for self-healing web scrapers. When a target site changes and a scraper breaks, ScrapeForensics detects it, figures out why, triggers Bright Data's self-healing, verifies the fix actually worked, and keeps a forensic record of the whole incident.

BREAK → INVESTIGATE → EXPLAIN → HEAL → VERIFY → REMEMBER

Built for WeMakeDevs Into the Scrape-Verse (Bright Data, Aug 17–23 2026).

Structure

Folder Role Owns
collector/ Bright Data + Collector infrastructure Create/run/heal/approve wrappers, the incident store, verification
diagnosis/ Scraper Doctor + AI diagnosis INVESTIGATE + EXPLAIN — what broke, why, in plain language
chaos-testing/ Chaos testing + verification BREAK — controlled mutations + the Reliability Score

Each folder has its own README with setup and API details. diagnosis and chaos-testing both build on top of collector's exported functions (runCollector, healCollector, verifyHeal, store.*) rather than talking to Bright Data directly — keeps one place responsible for the CLI integration.

Setup

```bash npm install -g @brightdata/cli bdata login ```

Claim hackathon credits: sign up at brightdata.com, enter promo code `wemakedevs` (lowercase) in the billing section.

Then see each folder's README for its own install/run steps.

Judging alignment

  • Use of Scraper Studiocollector/ wraps bdata scraper create/run/heal/approve directly
  • Reliability and self-healing — the entire premise; chaos-testing/ turns it into a measured score
  • Technical excellence / presentation — the collector/demo.ts script runs the full break→heal→verify→record loop end to end for the live demo

About

Incident response for self-healing web scrapers — detects breakage, diagnoses it, triggers Bright Data's heal, and verifies the fix actually worked. Built for WeMakeDevs Scrape-Verse.

Resources

Stars

3 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages