From Browser cURL to Clean Python Requests Code, Step by Step

If you've ever opened your browser's DevTools, right-clicked a request in the Network tab, and hit Copy as cURL, you already know the good part: you've captured exactly what the browser sent — headers, cookies, auth, body and all.

The annoying part is what comes next. You wanted Python, and instead you're holding a 30-line shell command full of -H flags and escaped quotes.

This post walks through porting a real curl command to requests by hand, one flag at a time, so the mapping is clear. Do it once like this and you'll recognize every piece next time.

The raw cURL

Here's a representative command copied straight out of Chrome:

curl 'https://api.example.com/v2/search?lang=en&page=2' \
  -H 'accept: application/json' \
  -H 'authorization: Bearer eyJhbGciOiJIUzI1NiJ9.eyJ1aWQiOjQyfQ.sig' \
  -H 'content-type: application/json' \
  -H 'referer: https://app.example.com/dashboard' \
  -H 'user-agent: Mozilla/5.0' \
  -b 'session=abc123; theme=dark' \
  --data-raw '{"q":"nginx","filters":["open"]}'

Let's take it apart.

1. Method and URL

There is no -X flag here, but there is a body (--data-raw). That detail decides the method: --data / --data-raw makes curl use POST by default, unless you explicitly override the method with flags like -X GET. With -G, curl moves the data into the query string instead.

So don't read the missing -X as "GET" — the presence of a body is what matters.

The URL also carries a query string, ?lang=en&page=2. We'll handle that as real parameters in a moment.

import requests

url = "https://api.example.com/v2/search"

2. Headers → a dict

Every -H 'key: value' becomes one entry in a headers dict. The one rule that bites people: split each header on the first colon, not every colon.

Header values can legitimately contain colons. Look at the referer here, whose value is a full URL: https://app.example.com/dashboard. A blanket split on : would mangle it into pieces; split(":", 1) keeps the value intact.

headers = {
    "accept": "application/json",
    "authorization": "Bearer eyJhbGciOiJIUzI1NiJ9.eyJ1aWQiOjQyfQ.sig",
    "content-type": "application/json",
    "referer": "https://app.example.com/dashboard",
    "user-agent": "Mozilla/5.0",
}

3. Cookies (-b) → their own dict

The -b flag is a semicolon-separated cookie string. Don't jam it into the headers dict as a raw Cookie line — hand it to requests as a proper cookies argument and let the library do the encoding:

cookies = {"session": "abc123", "theme": "dark"}

4. The JSON body: json= vs data=

The content-type is application/json, so the body is JSON. In requests, that means you pass it as json= — not data=:

payload = {"q": "nginx", "filters": ["open"]}

Why the distinction matters:

• json=payload serializes the dict to a JSON string and sets Content-Type: application/json for you when you do not override it.
• data=payload form-encodes it as q=nginx&filters=open, which a JSON endpoint will reject or misread.

If the original request had been application/x-www-form-urlencoded, you'd flip to data=. Reading the content type tells you which one to use.

5. Query params

You could leave ?lang=en&page=2 glued onto the URL string, but pulling it into a params dict is cleaner and lets requests handle the encoding:

params = {"lang": "en", "page": "2"}

One caution: if the params in the original URL are already percent-encoded — you see %20, %3D, etc. — don't decode them and pass them through params as well, or you'll double-encode. Either keep the encoded values in the URL string, or pass the decoded values through params, but not both.

6. Putting it together

import requests

url = "https://api.example.com/v2/search"

params = {"lang": "en", "page": "2"}

headers = {
    "accept": "application/json",
    "authorization": "Bearer eyJhbGciOiJIUzI1NiJ9.eyJ1aWQiOjQyfQ.sig",
    "content-type": "application/json",
    "referer": "https://app.example.com/dashboard",
    "user-agent": "Mozilla/5.0",
}

cookies = {"session": "abc123", "theme": "dark"}

payload = {"q": "nginx", "filters": ["open"]}

resp = requests.post(
    url,
    params=params,
    headers=headers,
    cookies=cookies,
    json=payload,
)

resp.raise_for_status()
print(resp.json())

Because json= sets the content type for you when you do not override it, you could often drop the explicit content-type header and get the same request on the wire.

The gotchas that eat an afternoon

A few things that don't show up until they break:

• Assuming GET because there's no -X. If --data / --data-raw is present, curl uses POST by default unless you override it with -X GET or use -G to move data into the query string.
• json= vs data=. The single most common mistake. Match it to the content type.
• Multipart uploads (-F). These map to files=, not data=. Mixing them up gives you a malformed boundary and a baffling 400.
• Splitting headers on every colon. Split once; values like URLs and timestamps contain colons of their own.
• Double-encoding query params. Don't both decode the URL and pass the same values through params.
• Compression headers. requests handles common decompression for you, and brotli support depends on your installed stack, so you can usually drop accept-encoding unless you are testing compression behavior specifically.

When you just want it done

Doing this by hand once is worth it — now you know what each flag maps to. After that, it's mostly tax, especially for requests with twenty-plus headers.

For the repetitive cases I keep a small browser-based converter open: paste the cURL, get requests code back. It runs entirely client-side, so the command — and any live auth tokens in it — never leaves the browser:

curl to Python requests converter

If you're dropping the request into a crawler rather than a one-off script, there's also a low-key Feapder spider mode that emits a ready-to-run spider class.

What's the gnarliest cURL you've had to port to Python by hand? Mine had a multipart body and eleven cookies. Curious what trips other people up most — json= and data=, or something else?