In Python, how do I use urllib to see if a website is 404 or 200?

Question

How to get the code of the headers through urllib?

Nadia Alramli · Accepted Answer · 2009-11-13T00:55:40.980

178

The getcode() method (Added in python2.6) returns the HTTP status code that was sent with the response, or None if the URL is no HTTP URL.

>>> a=urllib.urlopen('http://www.google.com/asdfsf')
>>> a.getcode()
404
>>> a=urllib.urlopen('http://www.google.com/')
>>> a.getcode()
200

edited Nov 13 '09 at 00:55

answered Nov 13 '09 at 00:45

Nadia Alramli

99,188
32
168
151

To use in python 3, just use `from urllib.request import urlopen`. – Nathanael Farley May 15 '16 at 20:17
4

In python 3.4, if there is a 404, `urllib.request.urlopen` returns a `urllib.error.HTTPError`. – mcb May 10 '17 at 07:43
Doesn't work in python 2.7. If the HTTP returns 400, an exception is thrown – Nathan B Feb 13 '19 at 14:32

score 88 · Answer 2 · edited Apr 16 '15 at 06:27

88

You can use urllib2 as well:

import urllib2

req = urllib2.Request('http://www.python.org/fish.html')
try:
    resp = urllib2.urlopen(req)
except urllib2.HTTPError as e:
    if e.code == 404:
        # do something...
    else:
        # ...
except urllib2.URLError as e:
    # Not an HTTP-specific error (e.g. connection refused)
    # ...
else:
    # 200
    body = resp.read()

Note that HTTPError is a subclass of URLError which stores the HTTP status code.

edited Apr 16 '15 at 06:27

AndiDog

62,237
17
152
196

answered Nov 13 '09 at 00:46

Joe Holloway

25,314
12
77
90

Is the second `else` a mistake? – Samy Bencherif Aug 25 '13 at 19:27
@NadavB The exception object 'e' will look like a response object. That is, it's file-like and you can 'read' the payload from it. – Joe Holloway Feb 14 '19 at 16:23

score 46 · Answer 3 · edited Jan 18 '18 at 16:10

46

For Python 3:

import urllib.request, urllib.error

url = 'http://www.google.com/asdfsf'
try:
    conn = urllib.request.urlopen(url)
except urllib.error.HTTPError as e:
    # Return code error (e.g. 404, 501, ...)
    # ...
    print('HTTPError: {}'.format(e.code))
except urllib.error.URLError as e:
    # Not an HTTP-specific error (e.g. connection refused)
    # ...
    print('URLError: {}'.format(e.reason))
else:
    # 200
    # ...
    print('good')

edited Jan 18 '18 at 16:10

Arpad Horvath

1,233
10
29

answered Apr 27 '16 at 17:20

XavierCLL

971
9
10

For [URLError](https://docs.python.org/3.5/library/urllib.error.html) `print(e.reason)` could be used. – Gitnik Aug 04 '17 at 20:23
What about `http.client.HTTPException`? – CMCDragonkai Nov 09 '18 at 07:10
1

How can I check for a 301 or 302? – LuckyStrike Nov 04 '20 at 08:23

score 6 · Answer 4 · answered Jul 08 '11 at 19:25

6

import urllib2

try:
    fileHandle = urllib2.urlopen('http://www.python.org/fish.html')
    data = fileHandle.read()
    fileHandle.close()
except urllib2.URLError, e:
    print 'you got an error with the code', e

answered Jul 08 '11 at 19:25

mrme

77
1
1

6

TIMEX is interested in grabbing the http request code (200, 404, 500, etc) not a generic error thrown by urllib2. – Joshua Burns Jul 09 '12 at 15:34

In Python, how do I use urllib to see if a website is 404 or 200?

4 Answers4

Linked

Related