Blogs

SpringSource Blog

The future of functional web testing?

Posted on August 28th, 2010 by Peter Ledbrook in Groovy/Grails, Testing, Web.

Peter Ledbrook

The Groovy community is a productive bunch, which means there are a plethora of frameworks, libraries, and tools to make your life easier. The area of testing seems to be particularly fertile ground and I've recently been looking into a couple of tools that, when combined, promise a step change in your productivity when writing functional web tests.

Although my usual focus is Grails, you don't have to use Grails to reap the benefits of these tools: they will work with any web application and will integrate well with any Java-based project/build. As it happens they both have associated plugins that make using them from Grails pretty straightforward.

The first of the tools I want to talk about is Spock. It's based on the Behaviour Driven Development (BDD) paradigm which shifts the focus away from the tests themselves to thinking about your code in terms of expected behaviour. The test cases you write read as specifications, which not only makes them easier to read and understand but also to write. You can even integrate Spock into any Java project and run from your specifications from your IDE (as long as the IDE has Groovy support – the three major ones do).

The second tool is even newer. It's called Geb and it uses WebDriver to test web applications using either real browsers or the HtmlUnit library. What sets Geb apart from the competition is the jQuery-like syntax for querying your HTML pages and its built-in support for the Page Object pattern.

So why do I think these make a winning combination? Because they make writing functional web tests as easy as they can be! Let's look at the two of them in action.

A simple example

Imagine you have a simple login page that you want to test. It accepts a username and password and has a 'Sign in' button. The HTML looks something like:

<html>
<head>
  <title>Login</title>
</head>
<body>
  <form action="/wildcard-realm/auth/signIn" method="post" >
    <input type="hidden" name="targetUri" value="" />
    <table>
      <tbody>
        <tr>
          <td>Username:</td>
          <td><input type="text" name="username" value="" /></td>
        </tr>
        <tr>
          <td>Password:</td>
          <td><input type="password" name="password" value="" /></td>
        </tr>
        <tr>
          <td>Remember me?:</td>
          <td>
            <input type="hidden" name="_rememberMe" />
            <input type="checkbox" name="rememberMe" id="rememberMe"  />
          </td>
        </tr>
        <tr>
          <td />
          <td><input type="submit" value="Sign in" /></td>
        </tr>
      </tbody>
    </table>
  </form>
</body>
</html>

Now take a look at the following Spock specification and try to work out what behaviour it's testing for:

import geb.spock.GebReportingSpec
import pages.*

class MySpec extends GebReportingSpec {
    String getBaseUrl() { "http://localhost:8080/wildcard-realm" }
    File getReportDir() { new File("target/reports/geb") }

    def "Test invalid password"() {
        given: "I'm at the login page"
        to LoginPage

        when: "I enter an invalid password for 'admin'"
        loginForm.username = "admin"
        loginForm.password = "sdfkjhk"
        signIn.click()

        then: "I'm redirected back to the login page with the password field empty and an error message"
        at LoginPage
        loginForm.username == "admin"
        !loginForm.password
        message.text() == "Invalid username and/or password"
    }

    def "Test valid login"() {
        given: "I'm at the login page"
        to LoginPage

        when: "I enter a valid username and password"
        loginForm.username = "admin"
        loginForm.password = "admin"
        signIn.click(HomePage)

        then: "I'm redirected to the home page, which displays my username"
        at HomePage
        $().text().contains("Welcome back admin!")
    }
}

I don't know about you, but I find it pretty easy to work out what the test is trying to do. Even if you don't know at this stage where the variables are coming from, you can effectively read the specification as natural language. That ease of comprehension is one of the great benefits of a BDD tool like Spock.

Let's look at the specification in more detail. Each test method (or "feature" method as Spock likes to call them) breaks down into several sections. The first one, given, contains any setup code and gives you the starting state for the test. You then declare a when block that initiates some behaviour in whatever you are testing, for example by submitting a form. You finish by checking the result of the stimulus in the then block, which contains the conditions you need to fully verify the expected behaviour. Unlike in a JUnit test, you don't need explicit assertions inside the then section because each expression is an implicit assertion.

It's a simple concept, but once you get used to writing specifications, you'll find that Spock makes writing tests easier. It's something I can't really explain. My best guess is that the syntax and structure match the way you formulate tests in your mind, so there is little impedance between thinking about what to test and writing the physical test case. But rather than take my word for it, I urge you to try. You can use Spock for unit testing as well as functional testing, so it's easy to play with.

Pretty much everything else in the test is Geb, including the to() and at() methods. Both of these work on page objects, which you have to write yourself. Fortunately that's easy enough as you can tell from the LoginPage class:

package pages

import geb.Page

class LoginPage extends Page {
    static url = "auth/login"

    static at = { title == "Login" }

    static content = {
        loginForm { $("form") }
        message { $("div.message") }
        signIn { $("input", value: "Sign in") }
    }
}

Let's look at the static properties in this class individually:

  • url – the relative URL of the page; used by the to() method to determine which URL to send the HTTP request to.
  • at – a closure that indicates whether the current page is this one or not – called by the at() method; it should return a boolean, but you can also include assertions.
  • content – a description of the page content, allowing for easy access to the parts declared here.

So in the above example, you can see that the login page has a relative URL of "auth/login". Relative to what? To the test's baseUrl. Determining whether the current page is the login page, involves simply checking that the page's title is "Login" in the at closure. Finally, the content block provides direct access to the login form (which is the only form on the page), the info/error message "div", and the "Sign in" button – all via Geb's $() method.

If you look back at the test, you'll see that I was able to access the content elements such as loginForm as if they were properties of the test. This feature of Geb allows for very succinct and self-descriptive tests, but more importantly it promotes code reuse. Imagine that your HTML page changes and one of the expressions no longer matches what you want it to. If you didn't use page objects, you'd have to perform a potentially unreliable global search and replace. How much better it is to change that one reference in the page object instead!

The $() function isn't just limited to content blocks – you can use it directly from your test code if you want. Consider this test:

    ...
    def "Test authentication redirect with query string"() {
        when: "I access the book list page with a sort query string"
        login "admin", "admin", BookListPage, [sort: 'title', order: 'desc']

        then: "The list of books is displayed in the correct order"
        at BookListPage
        $("tbody tr").size() == 3
        $("tbody tr")*.find("td", 1)*.text() == [ "Misery", "Guns, Germs, and Steel", "Colossus" ]
    }
    ...
    /**
     * Logs into the application either via a target page that requires
     * authentication or by directly requesting the login page.
     */
    private login(username, password, targetPage = null, params = [:]) {
        if (targetPage) {
            to([*:params], targetPage)
            page LoginPage
        }
        else {
            to LoginPage
        }

        loginForm.username = username
        loginForm.password = password

        if (targetPage) signIn.click(targetPage)
        else signIn.click(HomePage)
    }
    ...

The jQuery-like syntax makes it very easy to match elements in an HTML page and extract attribute values and content. Its syntax is well covered in the Geb manual, as are many of Geb's other features.

The previous example also demonstrates how you can factor out code from your tests into reusable methods. Because Spock's syntax is quite alien at first, you might not think this would be possible. But specifications are classes in the end, so you can treat them as such.

I could go on and on about the features of Spock and Geb and how to use them, but this article isn't a tutorial. It's more a taster to get you interested. If you want to see the full Spock specification from which I pulled the snippets above, then check out the source code for it and the associated page objects.

What are you waiting for?

Spock and Geb are pretty young technologies at the moment (neither has reached 1.0), but they are already far enough along that people are actively using them on their projects. Even at this stage, they present a compelling case: functional web tests that are both relatively easy to write and comprehend.

This is no small matter. Automated functional tests are essential to ensure that web applications are behaving as they should, but current approaches (in the Java universe at least) are typically clunky and act as a disincentive to writing those tests. So teams end up relying on manual testing, which never gives you the reliability of coverage you really need.

Is it all roses? Of course not. But what we have here are two tools that make what should be easy to test, actually easy to test – no mean feat in the world of functional web testing. And they will continue to support you as the pages you need to test become more complicated. Perhaps the biggest issue is dealing with Javascript and triggering DOM events from your tests, but even there you'll find that Geb is evolving rapidly to help you out.

I haven't even mentioned Spock's built-in mocking framework or it's support for data-driven tests (check out the where clause in the project's documentation). Even the output from its assertions is a big bonus:

dateService.getMonthString(new Date().updated(month: month)) == expected
|           |              |          |              |       |  |
|           June           |          |              5       |  July
|                          |          |                      false
|                          |          |                      2 differences (50% similarity)
|                          |          |                      Ju(ne)
|                          |          |                      Ju(ly)
|                          |          Sun Jun 27 12:24:02 BST 2010
|                          Fri Aug 27 12:24:02 BST 2010
org.grails.util.DateService@527f58ef

On the Geb front, take a look at the modules feature, which allows you decompose a page object into reusable parts. Great for complex pages. And since it uses WebDriver under the hood, you can run the tests either with real browsers or in headless mode (via HtmlUnit).

Let me finish by reiterating that these are tools that will work just as well in Java projects as Groovy or Grails ones. Just because your web application is written in Java doesn't mean that your tests have to be as well. Also understand that Spock can be used for any type of Java project – for unit, integration and/or functional tests.

If there's one thing I've learned it's that writing tests has to be as easy as possible, otherwise they simply won't be written. That's why I think Geb and Spock are important developments in the field of testing and well worth investigating.

Similar Posts

Share this Post
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google Bookmarks
  • DZone
  • LinkedIn
  • Slashdot
  • Technorati
  • Twitter
 

16 responses

  1. Luke Daley says:

    Added on August 29th, 2010 at 6:00 pm

    Thank you for this very well written article.

    I just wanted to quickly point out that while Geb was very much written with Spock in mind and it's API was optimised for it, it also works with regular JUnit, EasyB and Cucumber.

    More information on using these alternative test frameworks can be found at:

    http://bamboo.ci.codehaus.org/browse/GEB-MASTERDEFAULTS/latest/artifact/Manual/index.html

  2. Twice says:

    Added on August 30th, 2010 at 2:16 am

    Peter, thanks again for another brilliant article. Talking about spock haven't you tried installing the grails plugin through sts? I am asking this because I have been running into dependency problems when installing through sts.

  3. Peter Niederwieser says:

    Added on August 30th, 2010 at 2:44 pm

    @Twice: Have you reported this problem to the STS team? It's probably not specific to the Spock plugin.

  4. Peter Ledbrook (blog author) says:

    Added on August 31st, 2010 at 4:48 am

    @Twice I've noticed that JUnitResultFormatter appears to be missing from the classpath. I don't know what needs it, but it's provided by $GRAILS_HOME/lib so there shouldn't be a problem. I've raised an issue for it:

    https://issuetracker.springsource.com/browse/STS-1195

    If you manually add ant-junit-1.7.1.jar to the build path, you then run into a bunch of other errors. These appear to result from the Spock plugin declaring its dependencies as 'build' ones, whereas it should probably declare them in 'test' scope. I'll talk to Luke about that. You can work around this in two ways:

    1. Use the Grails 1.3.x dependency DSL to add the Spock plugin as a 'test' dependency

    2. Edit the 'dependencies.groovy' file in the installed plugin and replace the 'build' scope declarations with 'test'.

  5. jaw crusher says:

    Added on September 1st, 2010 at 2:38 am

    Very useful article!

  6. Shahzeb says:

    Added on September 2nd, 2010 at 6:19 pm

    Nice . How do they sit with selenium?

  7. Luke Daley says:

    Added on September 2nd, 2010 at 7:14 pm

    @Shahzeb: WebDriver and Selenium are merging. Which means that this effectively works with the next version of Selenium.

    See: http://code.google.com/p/selenium/wiki/FrequentlyAskedQuestions#Q:_So,_is_it_like_?_Or_?

  8. Luke Daley says:

    Added on September 2nd, 2010 at 9:17 pm

    @Twice this STS issue has been worked around in the latest version of the Spock plugin for Grails.

    See: http://grails.1312388.n4.nabble.com/New-spock-plugin-snapshot-td2524970.html#a2524970

  9. acai energy says:

    Added on September 3rd, 2010 at 4:38 am

    Thanks peter for this wonderful article. You are very much right about Spock and Geb. These are the future of we testing. its really amazing tools and the best thing is its so simple and very easy to learn. Its use is also very flexible nad yet very powerful. I always like to use this tools.

  10. Chavan says:

    Added on September 14th, 2010 at 9:37 am

    Thanks for the post, we wanted to use this for our product built using grails,but ended up with this exception which i believe is a bug in Geb Code.. Did anyone saw this one? I also tried http://fbflex.wordpress.com/2010/08/25/geb-and-grails-tips-tricks-and-gotchas/ this instruction set..

    Caused by: groovy.lang.MissingPropertyException: No such property: tagName for class: org.openqa.selenium.htmlunit.HtmlUnitWebElement

    at geb.navigator.NonEmptyNavigator.setInputValue(NonEmptyNavigator.groovy:340)

    at geb.navigator.NonEmptyNavigator.this$3$setInputValue(NonEmptyNavigator.groovy)

    at geb.navigator.NonEmptyNavigator$_setInputValues_closure23.doCall(NonEmptyNavigator.groovy:335)

    at geb.navigator.NonEmptyNavigator.setInputValues(NonEmptyNavigator.groovy:334)

    at geb.navigator.NonEmptyNavigator.this$3$setInputValues(NonEmptyNavigator.groovy)

    at geb.navigator.NonEmptyNavigator.value(NonEmptyNavigator.groovy:209)

    at ConsoleScript2$_run_closure1.doCall(ConsoleScript2:5)

    at ConsoleScript2$_run_closure1.doCall(ConsoleScript2)

    at geb.Browser.doDrive(Browser.groovy:195)

  11. Luke Daley says:

    Added on September 21st, 2010 at 6:50 pm

    @Chavan: you have an old version of the HtmlUnitDriver. Make sure you have at least version 2.0a3.

  12. tek says:

    Added on October 15th, 2010 at 10:32 am

    Wonderful tools. Is possible to use Geb with ZK? There are a project ( http://code.google.com/p/zk-ztl/ ) who permits jQuery selector with ZK, for example: verifyTrue(jq("@window").exists()), maybe is possible to do something similar with Geb?

    Thanks!

  13. Gavin Kromhout says:

    Added on May 17th, 2011 at 10:21 pm

    Fantastic clear post, thanks Peter.
    I actually half hoped that "http://localhost:8080/wildcard-realm" was a magic symbol :-)
    So I created a base class:

    class GebReportingSpecBase extends GebReportingSpec {
    def appName = grails.util.Metadata.current.'app.name'
    String getBaseUrl() { "http://localhost:8080/$appName/" }
    File getReportDir() { new File("target/test-reports/geb-pages") }
    }

    Naturally the example above becomes:
    class MySpec extends GebReportingSpecBase { …

    Ah life is so tough developing with Grails :-)

  14. 360logica says:

    Added on August 14th, 2012 at 4:58 am

    As you have defined the entire concept, you made it so easy to understand functional web testing. There is no doubt that scope for this is very wide…

  15. Testing-whiz says:

    Added on March 6th, 2013 at 6:02 am

    The tools you mentioned for testing web applications look quite interesting due to their features. I don't know whether this tool has an integration with bug detection tools. Moreover I would like to know whether this tools support database handling.
    Test Automation Tool for Web Apps

  16. stareast says:

    Added on April 4th, 2013 at 8:27 pm

    Thank you for the brilliant post on testing. We all know that the future of automated testing is bright.This is really going to replace manual testing due to its unforeseen features that are continuously added to the tool.
    Register to schedule a meeting at StarEast 2013

2 trackbacks

Leave a Reply