Booking.com does not want bots, and it shows. Every plain HTTP request, any client, any proxy, gets absorbed by AWS WAF with a 202 that never resolves into content. There is no getting around a real browser here, which makes this the most expensive scrape in my portfolio. What kept the project alive was discovering that the two hard parts, search and data extraction, both have quiet side doors.
Don't touch the search box
My first version drove the UI like a user: open the homepage, type the city, click search. It was a disaster. The autocomplete is timing-sensitive, the results panel loads lazily, and about a third of runs ended with zero hotels for no visible reason. Debugging flaky UI automation is the worst kind of debugging because every failure looks different.
The side door: Booking maintains SEO landing pages per city with stable URLs:
https://www.booking.com/city/{countryCode}/{citySlug}.{lang}.html
Navigate straight there and the page arrives populated with a hotel list. No typing, no clicking, no autocomplete race. Runs went from flaky to boring, which is the highest compliment a scraper can receive.
To build the URL from a free-text city name I geocode with Open-Meteo's public API (no key): city name in, country code and a sluggable name out.
The data is not in the DOM
Second surprise: scraping the rendered page gets you almost nothing coherent. Prices and hotel names live in visually adjacent but structurally unrelated containers, and the class names are minified soup.
The actual data source is Booking's Apollo GraphQL cache, embedded in a <script type="application/json"> block with a ROOT_QUERY key. Every hotel on the page is there as a typed object: name, review score, coordinates, and the price in three formats.
About those formats: roundedValue and value are display strings like "€ 128". The numeric price is at minPrice.AMOUNT. If you parse the display strings you inherit every currency symbol and thousands separator problem for free. Take the number the frontend itself uses.
Know what price you are looking at
One honesty note that shapes the whole product: city landing prices are indicative nightly rates, not date-specific quotes. Booking shows "from X per night" without you having picked dates. For market-level questions (median price level of a city, how hotel A sits against the market) that is exactly what you want. For "what does a room cost on August 14" you need a second navigation with dates, which is a different, more expensive product.
My feed stamps priceBasis: "indicative-nightly" on every response so nobody mistakes one for the other. Label your price basis; blurred numbers look precise and mean nothing.
Try it
The result is booking-hotel-price-analyzer on Apify. One call gives you a city's hotel market: median and percentile prices, supply depth, and the hotel list behind the numbers.
curl -s -X POST \
"https://api.apify.com/v2/acts/jdepablos~booking-hotel-price-analyzer/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{"location": "Madrid"}'
Browser-based, so it costs more per scan than my HTTP actors, and it is priced accordingly. A scan that returns no hotels is never charged.
MCP config for agents:
https://mcp.apify.com?tools=jdepablos/booking-hotel-price-analyzer
Takeaways
When a site fights HTTP clients, the browser is the entry fee, but you still choose how much of the UI you depend on. Direct navigation to stable URLs beats interactive flows every time you can find them. And before you fight the DOM, check for an embedded state blob: Apollo caches, __NEXT_DATA__, Redux snapshots. Frontends increasingly ship their data model in one script tag, cleaner than any HTML you will ever parse.
Hotel folks: if you need date-specific quotes or a market I have not covered, tell me in the comments and it goes on the roadmap.
SOCIAL SHARE CARD GENERATOR