Web Scraping with Burst
Run scrapers for hours without keeping your laptop on. No timeouts, no babysitting, no infrastructure to manage.
The Problem
Web scrapers often need to run for hours. Maybe you're collecting data from thousands of pages, respecting rate limits, or navigating complex pagination. The problem is:
- Your laptop can't stay open for 8 hours
- AWS Lambda times out after 15 minutes
- VPS requires setup, maintenance, and SSH sessions
- Your WiFi drops, your scraper dies
You need compute that runs in the background while you close your laptop and go live your life.
How Burst Solves It
With Burst, you submit your scraper and walk away. It runs in an isolated VM until completion, regardless of how long that takes.
$
burst run ./scraper.py
Uploading... done.
Running in vm-7k3x.
You can close this terminal.
Check back later to stream logs or download results:
$
burst logs
[14:23:41] Scraped 2,847 pages
[14:23:42] Writing to output.json...
Done. 3h 12m
Why Burst for Scraping
- No timeout limits — Run for minutes or hours
- Isolated execution — Each scraper gets its own VM
- Fire and forget — Close your laptop, get results later
- Real-time logs — Monitor progress with
burst logs - No setup — No Docker, no YAML, no cloud console
Comparison
| Feature | Burst | Lambda | VPS |
|---|---|---|---|
| Max runtime | Unlimited | 15 min | Unlimited |
| Setup time | 1 min | 30+ min | 1+ hour |
| Maintenance | None | IAM, configs | Full server |
| Cost model | Per job | Per request | Monthly |
Ready to run scrapers without babysitting?
Get early access