Skip to content

Instantly share code, notes, and snippets.

@dehowell
Created March 23, 2011 22:49
Show Gist options
  • Save dehowell/884204 to your computer and use it in GitHub Desktop.
Save dehowell/884204 to your computer and use it in GitHub Desktop.
Python function to test if a file at a URL exists.
import urllib2
def file_exists(location):
request = urllib2.Request(location)
request.get_method = lambda : 'HEAD'
try:
response = urllib2.urlopen(request)
return True
except urllib2.HTTPError:
return False
@HGStyle
Copy link

HGStyle commented Oct 29, 2024

For all the people saying "doesn't return anything" make sure to include a timeout in the function. I don't really know how to do it using Urllib but I know it is possible using Requests:

import requests

def is_online(url: str) -> bool:
    """
    Checks if the document at a providen URL is online.
    """
    try:
        return requests.head(url, timeout=3).status_code // 100 == 2
    except Exception:
        return False

If it still doesn't works, it might be the webserver: sometimes they simply reject HEAD requests because they are only used by bots. Instead use a GET request (replace "requests.head" to "requests.get" in the code I wrote) but note that it will also download the content body shouldn't be a problem in most cases but if its a file to download then you're kinda screwed because Python will have to load that file into memory which can take a lot of space (and that's why HEAD requests exists, but they're sometimes rejected as I said)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment