Blog: PHP, Python, Linux, Web services & Continuous delivery

Automated Browser Acceptance Testing with Behat & Mink

This post accompanies a project I've put on Github that provides an example of how to use Behat & Mink to create automated browser based acceptance tests. By combining the concepts and tooling from Behavior Driven Development (BDD) with remote browser driver technologies, we will describe the desired behavior of a system and then transform these descriptions into test simulations running in real web browsers. The intention of both the demo project and the post is to highlight that:

  • Driving your acceptance testing process with BDD techniques is a valuable concept
  • Behat and Mink are a solid choice for implementing acceptance tests

The demo project consists of a web site that sells ascii art (The Ascii Art Marketplace) and a test suite that covers the features 'add to basket' and 'newsletter sign up'. It also contains a wrapper script that will run the tests in either phantomJS or Selenium, this can be useful when hooking things up with Continuous Integration (CI). For more info on using phantomJS with Mink & Behat see this section a bit further down.

In this post, I'm mainly providing some background and ramblings around BDD, why automating acceptance testing is so valuable and general information on Behat & Mink. If your already familiar with this stuff then you might want to skip straight to the github repo at https://github.com/pipe-devnull/mink-behat-acceptance-test-demo

A Little Background

The core idea of BDD is centered around the use of an ubiquitous language called Gherkin that is designed to express what the desired behavior of a system should be using simple, non-technical language. Gherkin enforces a uniform structure to the definition of behavior that makes it easy for a human to read and understand but also easy for a program to parse. Behavior definitions are organized by a system's features, within each feature behavior is further grouped into scenarios. Each line of a scenario begins with one of the verbs GIVEN, WHEN, THEN, AND and BUT. The easiest way to understand this is by looking at an example, the following snippet of Gherkin describes the expected behavior of a VIP discount rule on an e-commerce store

    SCENARIO: Allow VIP discounts
      GIVEN I am a VIP customer
      AND I add a discountable product to the shopping cart
      AND I add a non-discountable product to the shopping cart
      WHEN I view the shopping cart 
      THEN I should see a 50% discount on the discountable product
      BUT  the non-discountable product should have no discount

Gherkin has a bit more to offer than just whats in the simple example above (e.g. Scenario backgrounds & Scenario outlines) but it is a typical example of how Gherkin is used to express desired behavior. For more in-depth information on BDD and Domain Specific Languages (DSLs) such as Gherkin check out the original BDD article from Dan North and Martin Fowlers overview on DSLs

An interesting aspect to Gherkin is that you have a lot of freedom to decide how you abstract the definitions of behavior. You can keep things vague and let the code layer take care of all the low level detail or you can be more specific in the Gherkin, writing more granular descriptions of the desired behavior.

Beyond BDD

BDD is an excellent development methodology but I have found opportunities to use it can be rare in large organizations, especially when your core role is largely supporting / enhancing legacy applications in non-tech driven companies. BDD focuses on driving the code development process using up front specifications of desired behavior but I'm instead looking more at how we can use BDD techniques more as a stand alone testing tool and forget the development aspect (at least for now).

There is a huge amount of value in driving your acceptance testing process with BDD techniques

  • Gherkin can be understood by both technical and non-technical personnel making it much easier to share understanding of a system's capabilities between developers, testers and business stakeholders.
  • Stakeholders / Testers / Developers / any others can start writing test cases with minimal effort.
  • You can start to build a common approach to writing tests that transcends what platform a system is running on or what programming language it is written in.
  • A consistent and structured approach of authoring naturally leads into automation.
  • Your tests naturally become a reliable source of living documentation yet remain accessible and readable.
  • Future maintainers can quickly acquire detailed knowledge of the system they inherit.

If you need to document a legacy system why not describe that's system's behavior using Gherkin? Then turn that documentation into real tests that you can re-run on every change to catch regressions. The same reasoning can be used when working with a black box systems that you don't have source code for, or when looking to introduce acceptance tests for a new system.

Behat, Mink and the Mink extension

Before getting into the example project, lets have a quick overview of the tools we are using:

Behat - A BDD framework written in PHP

Mink - A web browser driver abstraction layer written in PHP that includes integrations with Goutte, Selenium, Selenium 2, sahi and Zombie JS out the box.

Mink Extension - An extension for Behat that enables the use pf the Mink browser abstraction layer within Behat

So, Behat provides us a the ability to turn Gherkin into code, Mink enables us to use code to drive a web browser and the Mink Extension glues Mink and Behat together. I've tried several solutions like this in Ruby, Python and JavaScript and while they are all similar, I can honestly say that the PHP solution is one of the best I tried. It stacks up really well in terms of usability, extendability and scalability against the likes of Lettuce_webdriver (Python), Waitr (Ruby) and ZombieJs & Jasmine (Javascript).

Behat & Mink are also the natural choice if your strongest skill set is in PHP, there are some examples of BDD frameworks for other languages at the foot of this post.

Browser Drivers

Mink supports two types of browser driver: headless web browser drivers and standard web browser drivers. A headless web browser driver is simply a browser emulator that can perform all the actions of a normal web browser but does not need to launch an actual browser to perform the emulation. Headless drivers require no UI to be installed on the host upon which they run and as such are ideal for Continuous Integration (CI) environments.

Different drivers have different capabilities and the one you choose will depend on what you need to test. A lot of the Behat & Mink examples I have come across focus on using Goutte (a headless driver written in PHP) as the main driver. Goutte is fast and suits simple test cases but its not an option as soon as you encounter a complex form or JavaScript, in most cases the tests I write require more than Goutte has to offer. After a fair amount of experimentation I have found the most effective solution is to use:

  • Selenium 2 with Firefox for running tests locally and watching whilst they run
  • PhantomJS for running tests as part of CI


Using PhantomJS with Behat+Mink

PhantomJS is not officially included as a driver with Mink but is pretty easy to setup. PhantomJS provides the option to start up in remote webdriver server mode where by it will start a server process that listens on a local port number (which is configurable) and wait for instructions to be sent to it. Clients such as Mink, that implement the webdriver API, connect to the remote web driver server and send it browser actions to emulate. The server then sends responses back to the client so the client can then perform assertions on the simulation.

This works perfectly with Mink ...

1 Start phantomJs in remote server mode

    phantomjs --webdriver=8643 

2 Change your behat.yml config file to use selemium2 (webdriver) and provide the location for the webdriver service.

    default:
      extensions:
        Behat\MinkExtension\Extension:
          default_session:                selenium2
          selenium2:
            wd_host:                      "http://localhost:8643/wd/hub"

3 Run your tests.

    bin/behat

NOTE: Why not use zombieJS? Mainly because it requires nodeJS to run and I don't want to install node everywhere right now. PhantomJS is a standalone application with no dependencies. I also had issues getting zombieJS up and running where as PhantomJS has always worked out the box where ever i have put it.


Try the sample project

Thats the brief intro over, give the demo project a try and the perhaps use it as a base for building your own acceptance test suites.

https://github.com/pipe-devnull/mink-behat-acceptance-test-demo




BDD Frameworks & Browser driver layers in other languages

LanguageFramework
RubyCucumber
PythonLettuce
C# (.NET)SpecFlow
JavaJBehave
JavaScriptJasmine
comments powered by Disqus