urllib.request.urlopen()

from urllib import request, parse

# GET request
with request.urlopen(url) as res:
    data = res.read()

# POST request
data = parse.urlencode({'key': 'value'}).encode()
with request.urlopen(url, data=data) as res:
    result = res.read()

# URL encoding
encoded = parse.urlencode({'q': 'search keyword'})
quoted = parse.quote('hello world')

urllib_get.py

from urllib import request, parse
import json

url = 'https://jsonplaceholder.typicode.com/todos/1'
try:
    with request.urlopen(url, timeout=10) as res:
        data = res.read()
        status = res.status
        ctype = res.getheader('Content-Type')
        print(status)
        print(ctype)
        result = json.loads(data.decode('utf-8'))
        print(result)
except Exception as e:
    print(e)

params = {'q': 'Python', 'page': 1, 'limit': 10}
query = parse.urlencode(params)
url = 'https://wp-p.info/sandbox/api.php?' + query
print(url)

Running the code produces the following output:

python3 urllib_get.py
200
application/json; charset=utf-8
{'userId': 1, 'id': 1, 'title': 'delectus aut autem', 'completed': False}
https://wp-p.info/sandbox/api.php?q=Python&page=1&limit=10

urllib_post.py

from urllib import request, parse
import json

req = request.Request(
    'https://wp-p.info/sandbox/api.php',
    headers={
        'User-Agent': 'MyApp/1.0',
        'Accept':     'application/json',
    }
)
with request.urlopen(req, timeout=10) as res:
    body = res.read()
    print(res.status)
    print(body.decode('utf-8'))

post_data = parse.urlencode({'name': 'Ayanami Rei', 'org': 'NERV'}).encode('utf-8')
req = request.Request(
    'https://httpbin.org/post',
    data=post_data,
    headers={'Content-Type': 'application/x-www-form-urlencoded'},
)
with request.urlopen(req, timeout=10) as res:
    result = json.loads(res.read().decode('utf-8'))
    print(result['form'])

Running the code produces the following output:

python3 urllib_post.py
200
{"status": "ok"}
{'name': 'Ayanami Rei', 'org': 'NERV'}

urllib_parse.py

from urllib import parse

text = 'hello world'
encoded = parse.quote(text)
print(encoded)
decoded = parse.unquote(encoded)
print(decoded)

parsed = parse.urlparse('https://example.com:8080/path?key=val#section')
print(parsed.scheme)
print(parsed.netloc)
print(parsed.path)
print(parsed.query)
print(parsed.fragment)

Running the code produces the following output:

python3 urllib_parse.py
hello%20world
hello world
https
example.com:8080
/path
key=val
section

Common Mistake 1: Trying to get the response body without calling res.read()

urlopen() returns a response object, not the content itself. Without calling res.read(), you cannot get the bytes of the response body.

mistake_read_ng.py

from urllib import request

url = 'https://jsonplaceholder.typicode.com/todos/1'
with request.urlopen(url, timeout=10) as res:
    print(res)

Running the code produces the following output:

python3 mistake_read_ng.py
<http.client.HTTPResponse object at 0x...>

Call res.read() to get the response body. If you need a string rather than bytes, also call decode() on the result.

mistake_read_ok.py

from urllib import request

url = 'https://jsonplaceholder.typicode.com/todos/1'
with request.urlopen(url, timeout=10) as res:
    body = res.read().decode('utf-8')
    print(body)

Running the code produces the following output:

python3 mistake_read_ok.py
{
  "userId": 1,
  "id": 1,
  "title": "delectus aut autem",
  "completed": false
}

Common Mistake 2: Forgetting urlencode causes garbled non-ASCII parameters

Embedding non-ASCII characters directly into a query string can cause the server to misinterpret them. Build URLs with parse.urlencode() to properly encode all parameters.

mistake_encode_ng.py

from urllib import request

keyword = 'Python tutorial'
url = 'https://example.com/search?q=' + keyword
print(url)

Running the code produces the following output:

python3 mistake_encode_ng.py
https://example.com/search?q=Python tutorial

Use parse.urlencode() to safely build URLs that contain special characters.

mistake_encode_ok.py

from urllib import parse

keyword = 'Python tutorial'
params = {'q': keyword}
query = parse.urlencode(params)
url = 'https://example.com/search?' + query
print(url)

Running the code produces the following output:

python3 mistake_encode_ok.py
https://example.com/search?q=Python+tutorial

urllib.request.urlopen() supports HTTPS and validates SSL certificates by default. You can set a timeout in seconds using the timeout parameter — setting one prevents the program from hanging when a server does not respond.

The response body is returned as bytes, so decode it with the appropriate encoding (usually UTF-8) before treating it as a string. You can also retrieve the encoding from the Content-Type response header.

If you make frequent HTTP requests in production, using the requests library is one option. It makes session management, authentication, and retries straightforward, and results in more concise code. When you can install external packages, run pip install requests to get started.