| /dev/null
Blog: PHP, Python, Linux, Web services & Continuous delivery


Downloading resources in python can be a tedious job unless you find the right library to help. For cases where you just want to grab a plain url then the standard library's urllib2 is sufficient but if you need any extras such as basic authentication, keep-alives, or connection pooling then there are better alternatives available.

The requests module is a comprehensive 'pythonic' library for dealing with HTTP. It's essentially a high level wrapper over the functionality included in both urllib2 and urllib3 but with a few very useful extras. It's clean, hassle free approach is best highlighted with some examples.

Read more

I created this script in order to try out the standard queue and thread libraries in Python. It doesn't do anything exciting but can be quite useful if you have a bunch of files to download. It accepts either a list of files as a command line parameter or a path to a JSON file, the format of which is outlined a bit further down.

Read about the python queue library in the docs here and the thread library here. It also requires the requests library to fetch the resources (see earlier post on Downloading HTTP resources in python using the requests library )

Read more