| /dev/null
Blog: PHP, Python, Linux, Web services & Continuous delivery

Tag: Python

Downloading resources in python can be a tedious job unless you find the right library to help. For cases where you just want to grab a plain url then the standard library's urllib2 is sufficient but if you need any extras such as basic authentication, keep-alives, or connection pooling then there are better alternatives available.

The requests module is a comprehensive 'pythonic' library for dealing with HTTP. It's essentially a high level wrapper over the functionality included in both urllib2 and urllib3 but with a few very useful extras. It's clean, hassle free approach is best highlighted with some examples.

Read more

If your trying to install the python lxml package (specifically on Debian but this probably applies to other distros too) using either pip or easy_install and get messages like the following

Read more

virtualenv is a great tool that you can use to create isolated python environments that contain their own installation directories, libraries and package managers. This isolation enables you to control package dependencies at the application level rather than the system level and helps avoid the situation where upgrading a system library for the benefit of one application would cause a different application to break. However, its important to note that it does still share the base system's python installation with the virtual environment.

Its stable enough for production and provides the opportunity to ship your application with all dependencies bundled in. This is how I've been using it for development, all examples are in the context of installation on Debian Linux.

Read more

I created this script in order to try out the standard queue and thread libraries in Python. It doesn't do anything exciting but can be quite useful if you have a bunch of files to download. It accepts either a list of files as a command line parameter or a path to a JSON file, the format of which is outlined a bit further down.

Read about the python queue library in the docs here and the thread library here. It also requires the requests library to fetch the resources (see earlier post on Downloading HTTP resources in python using the requests library )

Read more