Apr 18, 2013

Multi-Browser JavaScript Unit Testing with Sauce

[banner id=”java”]
“The frontend code”. It’s often prefixed with “A fix to the…” and suffixed with “… due to a bug in Internet Explorer”. Why is this? JavaScript unit testing is easy due to a plethora of testing frameworks. But how do we catch bugs in our JavaScript code due to browser differences? It’s too manual of a process to open the tests in every browser before each commit.

Enter Sauce, which provides Remove WebDriver Browsers as a Service, or RWDBaaS. I may have just made up that acronym, but I am not making up this service. It’s a great way to run automated Selenium-based tests against a large number of browsers. But automated cross-browser unit tests? Well, that’s just one step removed and I would like to show you that.

Jasmine unit tests

Managing Expectations

Like any good software project, let’s first define what this article is NOT about:

With that out of the way, let’s move on to the real meat of this thought experiment: using Sauce to run our JavaScript tests on a large variety of browsers.

The full source code for this article can be found in the sauce-unit-test-example Github repository.

Step 1: Write your unit test

Here is a simple Jasmine test to verify the splice() function works as expected:

describe("JavaScript Array", function() {
  it("should be able to splice", function() {
    myArray = ['a', 'b', 'c'];
    myArray.splice(1);

    expect(myArray.length).toEqual(1);
    expect(myArray[0]).toEqual('a');
  });
});

Not very complicated. Let’s move on.

Step 2: Write your browser test

We need a way to quickly verify that the unit tests passed. Let’s use Geb:

class JasmineSpec extends GebReportingSpec {
    def "all Jasmine specs should pass"() {
        when:
        go '/SpecRunner.html'

        then:
        title == 'Jasmine Spec Runner'
        $('.passingAlert')
    }
}

This is also quite simple – verify that the page title is “Jasmine Spec Runner” and that all the tests passed.

Here is a sample GebConfig you could use with this:

baseUrl = 'http://localhost:8080'

sauceUsername = System.getProperty('sauce.username')
sauceApiKey = System.getProperty('sauce.apiKey')

driver = { new FirefoxDriver() }
environments {

    'windows-ie-8' {

        driver = { 
            DesiredCapabilities capabilities = DesiredCapabilities.internetExplorer()
            capabilities.setPlatform(Platform.XP)
            capabilities.setVersion("8")

            new RemoteWebDriver(new URL("http://${sauceUsername}:${sauceApiKey}@ondemand.saucelabs.com:80/wd/hub"), capabilities) 
        }
    }

    'windows-ie-9' {

        driver = { 
            DesiredCapabilities capabilities = DesiredCapabilities.internetExplorer()
            capabilities.setCapability('platform', 'Windows 2008')
            capabilities.setVersion('9')

            new RemoteWebDriver(new URL("http://${sauceUsername}:${sauceApiKey}@ondemand.saucelabs.com:80/wd/hub"), capabilities) 
        }
    }

}

This allows us to run the tests using a local Firefox browser, or optionally using a specific version of IE on Sauce.

Step 3: Setup the build environment
We will be using Gradle to start up a local server and run the Geb tests. Here is what I whipped up:

apply plugin: "groovy"
apply plugin: "idea"
apply plugin: "war"
apply plugin: "jetty"

buildscript {
    repositories {
        mavenCentral()
        maven {
            url "https://repository-saucelabs.forge.cloudbees.com/release"
        }
    }
    dependencies {
        classpath "com.saucelabs:sauce-connect:3.0.24"
        classpath("com.saucelabs:ci-sauce:1.29") {
            transitive = false
        }

        classpath "commons-io:commons-io:1.4"
        classpath "commons-lang:commons-lang:2.6"
    }
}

def sauce = [
    username: System.getProperty('sauce.username'),
    apiKey: System.getProperty('sauce.apiKey')
]

jettyRun.contextPath = '/'
jettyRun.webAppSourceDirectory = new File('src/test/javascript')
stopPort = 8090
stopKey = '123'

repositories {
    mavenCentral()
}

dependencies {
    groovy "org.codehaus.groovy:groovy-all:2.1.1"

    testCompile "org.gebish:geb-spock:0.9.0-RC-1"
    testCompile "org.spockframework:spock-core:0.7-groovy-2.0"

    testCompile "org.seleniumhq.selenium:selenium-firefox-driver:2.26.0"
}

task gebTest(type: Test) {
    testReportDir = reporting.file("$name/tests")
    testResultsDir = file("$buildDir/test-results/$name")

    systemProperty 'geb.build.reportsDir', reporting.file("$name/geb")
    systemProperty 'geb.env', System.getProperty('geb.env')
    systemProperty 'sauce.username', sauce.username
    systemProperty 'sauce.apiKey', sauce.apiKey
}

task test(overwrite: true, dependsOn: [tasks['gebTest']])

task wrapper(type: Wrapper) {
    gradleVersion = '1.4'
}

gebTest.doFirst {
  jettyRun.daemon = true
  jettyRun.execute()

  if(sauce.username && sauce.apiKey) {
    sauce.connect = new com.saucelabs.ci.sauceconnect.SauceConnectTwoManager()
    sauce.connect.openConnection(sauce.username, sauce.apiKey, 4445, null, null, null, null)
  }
}

gebTest.doLast {
  tasks.jettyStop.execute()
  sauce.connect?.closeTunnelsForPlan(sauce.username, null)
}

This Gradle file does three main things:

  1. Launch a server running locally.
  2. Run Geb tests using either Firefox or another browser, specified using the geb.env property.
  3. Set up a secure tunnel to Sauce, using Sauce Connect. This is the secret sauce that allows the web browser running on a remote virtual machine to access your local web server.

Step 4: Run the tests

To run the tests with Firefox:
./gradlew clean test

This should briefly open Firefox on your machine, run the Jasmine specifications, and verify that everything passed. You should see “BUILD SUCCESSFUL” on the command line.

That’s great, but what about other browsers (Internet Explorer)? Go get yourself a free account on Sauce, and copy down your API key. Then run:
./gradlew clean test -Dgeb.env=windows-ie-8 -Dsauce.username=your_username -Dsauce.apiKey=your_api_key

With any luck, you should see BUILD FAILED on the command line. Congratulations – Sauce just caught my rookie JavaScript mistake!

Summary

Sauce tests in Jenkins

In this example, we wrote a simple Jasmine specification, wrote a Geb test to verify that the Jasmine tests passed, used Sauce Connect to establish a secure tunnel with Sauce, and then used Sauce itself as a remote WebDriver to run the tests against Internet Explorer.

Where could you go from here?

  • Add more browsers to GebConfig.
  • Set up a Jenkins job to run the tests against a large number of browsers.

I have an example Jenkins job, also in the sauce-unit-test-example Github repo. Go clone the repository and try it out for yourself! Good luck!

About the Author

David Norton profile.

David Norton

Director, Platform Engineering

Passionate about continuous delivery, cloud-native architecture, DevOps, and test-driven development.

  • Experienced in cloud infrastructure technologies such as Terraform, Kubernetes, Docker, AWS, and GCP.
  • Background heavy in enterprise JVM technologies such as Groovy, Spring, Spock, Gradle, JPA, Jenkins.
  • Focus on platform transformation, continuous delivery, building agile teams and high-scale applications.
Leave a Reply

Your email address will not be published. Required fields are marked *

Related Blog Posts
An Exploration in Rust: Musings From a Java/C++ Developer
Why Rust? It’s fast (runtime performance) It’s small (binary size) It’s safe (no memory leaks) It’s modern (build system, language features, etc) When Is It Worth It? Embedded systems (where it is implied that interpreted […]
Getting Started with CSS Container Queries
For as long as I’ve been working full-time on the front-end, I’ve heard about the promise of container queries and their potential to solve the majority of our responsive web design needs. And, for as […]
Simple improvements to making decisions in teams
Software development teams need to make a lot of decisions. Functional requirements, non-functional requirements, user experience, API contracts, tech stack, architecture, database schemas, cloud providers, deployment strategy, test strategy, security, and the list goes on. […]
JavaScript Bundle Optimization – Polyfills
If you are lucky enough to only support a small subset of browsers (for example, you are targeting a controlled set of users), feel free to move along. However, if your website is open to […]