*The production problem
*
Simple HTTP scraping still works for a lot of pages. If a site returns fully formed HTML in the first response, an HTTP client plus a parser is often enough. You send the request, parse the response, extract fields, and move on. For static pages, lightweight crawlers are faster, cheaper, and easier to run than browser...