Apr 18, 2013

Multi-Browser JavaScript Unit Testing with Sauce

[banner id=”java”]
“The frontend code”. It’s often prefixed with “A fix to the…” and suffixed with “… due to a bug in Internet Explorer”. Why is this? JavaScript unit testing is easy due to a plethora of testing frameworks. But how do we catch bugs in our JavaScript code due to browser differences? It’s too manual of a process to open the tests in every browser before each commit.

Enter Sauce, which provides Remove WebDriver Browsers as a Service, or RWDBaaS. I may have just made up that acronym, but I am not making up this service. It’s a great way to run automated Selenium-based tests against a large number of browsers. But automated cross-browser unit tests? Well, that’s just one step removed and I would like to show you that.

Jasmine unit tests

Managing Expectations

Like any good software project, let’s first define what this article is NOT about:

With that out of the way, let’s move on to the real meat of this thought experiment: using Sauce to run our JavaScript tests on a large variety of browsers.

The full source code for this article can be found in the sauce-unit-test-example Github repository.

Step 1: Write your unit test

Here is a simple Jasmine test to verify the splice() function works as expected:

describe("JavaScript Array", function() {
  it("should be able to splice", function() {
    myArray = ['a', 'b', 'c'];
    myArray.splice(1);

    expect(myArray.length).toEqual(1);
    expect(myArray[0]).toEqual('a');
  });
});

Not very complicated. Let’s move on.

Step 2: Write your browser test

We need a way to quickly verify that the unit tests passed. Let’s use Geb:

class JasmineSpec extends GebReportingSpec {
    def "all Jasmine specs should pass"() {
        when:
        go '/SpecRunner.html'

        then:
        title == 'Jasmine Spec Runner'
        $('.passingAlert')
    }
}

This is also quite simple – verify that the page title is “Jasmine Spec Runner” and that all the tests passed.

Here is a sample GebConfig you could use with this:

baseUrl = 'http://localhost:8080'

sauceUsername = System.getProperty('sauce.username')
sauceApiKey = System.getProperty('sauce.apiKey')

driver = { new FirefoxDriver() }
environments {

    'windows-ie-8' {

        driver = { 
            DesiredCapabilities capabilities = DesiredCapabilities.internetExplorer()
            capabilities.setPlatform(Platform.XP)
            capabilities.setVersion("8")

            new RemoteWebDriver(new URL("http://${sauceUsername}:${sauceApiKey}@ondemand.saucelabs.com:80/wd/hub"), capabilities) 
        }
    }

    'windows-ie-9' {

        driver = { 
            DesiredCapabilities capabilities = DesiredCapabilities.internetExplorer()
            capabilities.setCapability('platform', 'Windows 2008')
            capabilities.setVersion('9')

            new RemoteWebDriver(new URL("http://${sauceUsername}:${sauceApiKey}@ondemand.saucelabs.com:80/wd/hub"), capabilities) 
        }
    }

}

This allows us to run the tests using a local Firefox browser, or optionally using a specific version of IE on Sauce.

Step 3: Setup the build environment
We will be using Gradle to start up a local server and run the Geb tests. Here is what I whipped up:

apply plugin: "groovy"
apply plugin: "idea"
apply plugin: "war"
apply plugin: "jetty"

buildscript {
    repositories {
        mavenCentral()
        maven {
            url "https://repository-saucelabs.forge.cloudbees.com/release"
        }
    }
    dependencies {
        classpath "com.saucelabs:sauce-connect:3.0.24"
        classpath("com.saucelabs:ci-sauce:1.29") {
            transitive = false
        }

        classpath "commons-io:commons-io:1.4"
        classpath "commons-lang:commons-lang:2.6"
    }
}

def sauce = [
    username: System.getProperty('sauce.username'),
    apiKey: System.getProperty('sauce.apiKey')
]

jettyRun.contextPath = '/'
jettyRun.webAppSourceDirectory = new File('src/test/javascript')
stopPort = 8090
stopKey = '123'

repositories {
    mavenCentral()
}

dependencies {
    groovy "org.codehaus.groovy:groovy-all:2.1.1"

    testCompile "org.gebish:geb-spock:0.9.0-RC-1"
    testCompile "org.spockframework:spock-core:0.7-groovy-2.0"

    testCompile "org.seleniumhq.selenium:selenium-firefox-driver:2.26.0"
}

task gebTest(type: Test) {
    testReportDir = reporting.file("$name/tests")
    testResultsDir = file("$buildDir/test-results/$name")

    systemProperty 'geb.build.reportsDir', reporting.file("$name/geb")
    systemProperty 'geb.env', System.getProperty('geb.env')
    systemProperty 'sauce.username', sauce.username
    systemProperty 'sauce.apiKey', sauce.apiKey
}

task test(overwrite: true, dependsOn: [tasks['gebTest']])

task wrapper(type: Wrapper) {
    gradleVersion = '1.4'
}

gebTest.doFirst {
  jettyRun.daemon = true
  jettyRun.execute()

  if(sauce.username && sauce.apiKey) {
    sauce.connect = new com.saucelabs.ci.sauceconnect.SauceConnectTwoManager()
    sauce.connect.openConnection(sauce.username, sauce.apiKey, 4445, null, null, null, null)
  }
}

gebTest.doLast {
  tasks.jettyStop.execute()
  sauce.connect?.closeTunnelsForPlan(sauce.username, null)
}

This Gradle file does three main things:

  1. Launch a server running locally.
  2. Run Geb tests using either Firefox or another browser, specified using the geb.env property.
  3. Set up a secure tunnel to Sauce, using Sauce Connect. This is the secret sauce that allows the web browser running on a remote virtual machine to access your local web server.

Step 4: Run the tests

To run the tests with Firefox:
./gradlew clean test

This should briefly open Firefox on your machine, run the Jasmine specifications, and verify that everything passed. You should see “BUILD SUCCESSFUL” on the command line.

That’s great, but what about other browsers (Internet Explorer)? Go get yourself a free account on Sauce, and copy down your API key. Then run:
./gradlew clean test -Dgeb.env=windows-ie-8 -Dsauce.username=your_username -Dsauce.apiKey=your_api_key

With any luck, you should see BUILD FAILED on the command line. Congratulations – Sauce just caught my rookie JavaScript mistake!

Summary

Sauce tests in Jenkins

In this example, we wrote a simple Jasmine specification, wrote a Geb test to verify that the Jasmine tests passed, used Sauce Connect to establish a secure tunnel with Sauce, and then used Sauce itself as a remote WebDriver to run the tests against Internet Explorer.

Where could you go from here?

  • Add more browsers to GebConfig.
  • Set up a Jenkins job to run the tests against a large number of browsers.

I have an example Jenkins job, also in the sauce-unit-test-example Github repo. Go clone the repository and try it out for yourself! Good luck!

About the Author

David Norton profile.

David Norton

Director, Platform Engineering

Passionate about continuous delivery, cloud-native architecture, DevOps, and test-driven development.

  • Experienced in cloud infrastructure technologies such as Terraform, Kubernetes, Docker, AWS, and GCP.
  • Background heavy in enterprise JVM technologies such as Groovy, Spring, Spock, Gradle, JPA, Jenkins.
  • Focus on platform transformation, continuous delivery, building agile teams and high-scale applications.
Leave a Reply

Your email address will not be published. Required fields are marked *

Related Blog Posts
Rockstar Development
This originally appeared on Marty Henderson’s personal blog Or, how to use Gitpod and GitLab so that no one else has to care about your questionable coding language choices. A true rockstar has a good […]
Testing a Quarkus Kafka Application
Quarkus, a “Kubernetes Native Java stack,” enables lighter Java applications with faster startup times. In a recent post, I talked about scaling Kafka consumers in Kubernetes. Quarkus applications fit right into this picture because they […]
Gitpod and Hringvegurinn
Iceland Ever seen an advertisement for visiting Iceland? Have you noticed that they all mention Hringvegurinn or the Ring Road, as a good tour? (If you haven’t seen a tour ad for Iceland, Steindi Jr […]
Tacos and Steak, an Istio story
This post originally appear on Marty Henderson’s personal blog A brief history In the before times of 2017, Varun Talwar and Louis Ryan sat under a forgotten waterfall at the edge of the world and […]