Scar Feed docs

Web, Docker, local pipeline, Bright Data Scraper Studio, and Heal Court — same workspace chrome as the rest of the app.

Overview

Restock radar that will not cry wolf. Studio collectors scrape Adafruit, SparkFun, Pimoroni, and The Pi Hut. Heal Court decides release / repair / refuse before Scar Feed speaks.

Live: https://wolverine-scraper.vercel.app

Open overview → · GitHub →

Web / Vercel

Next.js app in web/. Vercel Root Directory = web. Builds need no Bright Data key — web/public/data/scar.json is committed and refreshed by CI after every scrape/heal.

Field Console on /app polls /data/scar.json every 15s. Run field scrape calls POST /api/field/refresh, which dispatches GitHub Actions wolverine-scrape when optional Vercel secret GITHUB_PAT is set. Bright Data stays in Actions only. Without the PAT, the button opens the workflow Run page.

/Marketing landing
/appProduct workspace + Field Console
/app/feedScar Feed
/app/courtHeal Court
/app/studioCollectors
/docsThis documentation

npx vercel login
npx vercel link --cwd web
npx vercel --prod --cwd web

After local pipeline changes: npm run scar:export. CI also exports + commits the snapshot automatically.

Docker

Image on Docker Hub. Same dashboard as the live demo.

docker pull manasdutta04/wolverine-dashboard:latest
docker run --rm -p 3000:3000 manasdutta04/wolverine-dashboard:latest

Then open http://localhost:3000

Local pipeline

Full contributor setup lives in the repo’s CONTRIBUTING.md. Short version: Node 20+, install at the repo root and under web/, export the scar snapshot, then start the Next app. Bright Data login is only needed when you actually scrape or heal — never on the public Vercel site.

git clone https://github.com/manasdutta04/wolverine-scraper.git
cd wolverine-scraper
npm install
cd web && npm install && cd ..

npm run scar:export
npm run web:dev

Also: npm run scrape · npm run check · npm run heal · npm test

Scraper Studio

Custom collectors — pin IDs, never recreate.

Adafruitc_msyvm0ar1gznj2dlrq
SparkFunc_msywbl7b18fsthmxn
Pimoronic_msywj65f19rulm4cua
The Pi Hutc_msyx5sb61lfwvvvspd

bdata scraper create <url> "<fields>"
bdata scraper run <collector_id>
bdata scraper heal <collector_id> "<what broke>"
bdata scraper approve <collector_id>

Heal Court

Empty/null price or stock → court verdict:

release — batch clear, signals live
repair — heal → approve → re-run
refuse — still broken → suppress signals

heal-log.md · sample-output.json · GitHub Actions

Repo map

scar/Match, diff, Heal Court, export, tests
scrapers/Store registry + bdata runner
pipeline/Scrape all → SQLite
heal/Red-flag check + heal loop
web/Next.js landing + /app
docs/Architecture + deploy notes
.github/workflows/Cron scrape/heal

Keep BRIGHTDATA_API_KEY out of git. See SECURITY.md.