Apr 18, 2013

Multi-Browser JavaScript Unit Testing with Sauce

[banner id=”java”]
“The frontend code”. It’s often prefixed with “A fix to the…” and suffixed with “… due to a bug in Internet Explorer”. Why is this? JavaScript unit testing is easy due to a plethora of testing frameworks. But how do we catch bugs in our JavaScript code due to browser differences? It’s too manual of a process to open the tests in every browser before each commit.

Enter Sauce, which provides Remove WebDriver Browsers as a Service, or RWDBaaS. I may have just made up that acronym, but I am not making up this service. It’s a great way to run automated Selenium-based tests against a large number of browsers. But automated cross-browser unit tests? Well, that’s just one step removed and I would like to show you that.

Jasmine unit tests

Managing Expectations

Like any good software project, let’s first define what this article is NOT about:

With that out of the way, let’s move on to the real meat of this thought experiment: using Sauce to run our JavaScript tests on a large variety of browsers.

The full source code for this article can be found in the sauce-unit-test-example Github repository.

Step 1: Write your unit test

Here is a simple Jasmine test to verify the splice() function works as expected:

describe("JavaScript Array", function() {
  it("should be able to splice", function() {
    myArray = ['a', 'b', 'c'];
    myArray.splice(1);

    expect(myArray.length).toEqual(1);
    expect(myArray[0]).toEqual('a');
  });
});

Not very complicated. Let’s move on.

Step 2: Write your browser test

We need a way to quickly verify that the unit tests passed. Let’s use Geb:

class JasmineSpec extends GebReportingSpec {
    def "all Jasmine specs should pass"() {
        when:
        go '/SpecRunner.html'

        then:
        title == 'Jasmine Spec Runner'
        $('.passingAlert')
    }
}

This is also quite simple – verify that the page title is “Jasmine Spec Runner” and that all the tests passed.

Here is a sample GebConfig you could use with this:

baseUrl = 'http://localhost:8080'

sauceUsername = System.getProperty('sauce.username')
sauceApiKey = System.getProperty('sauce.apiKey')

driver = { new FirefoxDriver() }
environments {

    'windows-ie-8' {

        driver = { 
            DesiredCapabilities capabilities = DesiredCapabilities.internetExplorer()
            capabilities.setPlatform(Platform.XP)
            capabilities.setVersion("8")

            new RemoteWebDriver(new URL("http://${sauceUsername}:${sauceApiKey}@ondemand.saucelabs.com:80/wd/hub"), capabilities) 
        }
    }

    'windows-ie-9' {

        driver = { 
            DesiredCapabilities capabilities = DesiredCapabilities.internetExplorer()
            capabilities.setCapability('platform', 'Windows 2008')
            capabilities.setVersion('9')

            new RemoteWebDriver(new URL("http://${sauceUsername}:${sauceApiKey}@ondemand.saucelabs.com:80/wd/hub"), capabilities) 
        }
    }

}

This allows us to run the tests using a local Firefox browser, or optionally using a specific version of IE on Sauce.

Step 3: Setup the build environment
We will be using Gradle to start up a local server and run the Geb tests. Here is what I whipped up:

apply plugin: "groovy"
apply plugin: "idea"
apply plugin: "war"
apply plugin: "jetty"

buildscript {
    repositories {
        mavenCentral()
        maven {
            url "https://repository-saucelabs.forge.cloudbees.com/release"
        }
    }
    dependencies {
        classpath "com.saucelabs:sauce-connect:3.0.24"
        classpath("com.saucelabs:ci-sauce:1.29") {
            transitive = false
        }

        classpath "commons-io:commons-io:1.4"
        classpath "commons-lang:commons-lang:2.6"
    }
}

def sauce = [
    username: System.getProperty('sauce.username'),
    apiKey: System.getProperty('sauce.apiKey')
]

jettyRun.contextPath = '/'
jettyRun.webAppSourceDirectory = new File('src/test/javascript')
stopPort = 8090
stopKey = '123'

repositories {
    mavenCentral()
}

dependencies {
    groovy "org.codehaus.groovy:groovy-all:2.1.1"

    testCompile "org.gebish:geb-spock:0.9.0-RC-1"
    testCompile "org.spockframework:spock-core:0.7-groovy-2.0"

    testCompile "org.seleniumhq.selenium:selenium-firefox-driver:2.26.0"
}

task gebTest(type: Test) {
    testReportDir = reporting.file("$name/tests")
    testResultsDir = file("$buildDir/test-results/$name")

    systemProperty 'geb.build.reportsDir', reporting.file("$name/geb")
    systemProperty 'geb.env', System.getProperty('geb.env')
    systemProperty 'sauce.username', sauce.username
    systemProperty 'sauce.apiKey', sauce.apiKey
}

task test(overwrite: true, dependsOn: [tasks['gebTest']])

task wrapper(type: Wrapper) {
    gradleVersion = '1.4'
}

gebTest.doFirst {
  jettyRun.daemon = true
  jettyRun.execute()

  if(sauce.username && sauce.apiKey) {
    sauce.connect = new com.saucelabs.ci.sauceconnect.SauceConnectTwoManager()
    sauce.connect.openConnection(sauce.username, sauce.apiKey, 4445, null, null, null, null)
  }
}

gebTest.doLast {
  tasks.jettyStop.execute()
  sauce.connect?.closeTunnelsForPlan(sauce.username, null)
}

This Gradle file does three main things:

  1. Launch a server running locally.
  2. Run Geb tests using either Firefox or another browser, specified using the geb.env property.
  3. Set up a secure tunnel to Sauce, using Sauce Connect. This is the secret sauce that allows the web browser running on a remote virtual machine to access your local web server.

Step 4: Run the tests

To run the tests with Firefox:
./gradlew clean test

This should briefly open Firefox on your machine, run the Jasmine specifications, and verify that everything passed. You should see “BUILD SUCCESSFUL” on the command line.

That’s great, but what about other browsers (Internet Explorer)? Go get yourself a free account on Sauce, and copy down your API key. Then run:
./gradlew clean test -Dgeb.env=windows-ie-8 -Dsauce.username=your_username -Dsauce.apiKey=your_api_key

With any luck, you should see BUILD FAILED on the command line. Congratulations – Sauce just caught my rookie JavaScript mistake!

Summary

Sauce tests in Jenkins

In this example, we wrote a simple Jasmine specification, wrote a Geb test to verify that the Jasmine tests passed, used Sauce Connect to establish a secure tunnel with Sauce, and then used Sauce itself as a remote WebDriver to run the tests against Internet Explorer.

Where could you go from here?

  • Add more browsers to GebConfig.
  • Set up a Jenkins job to run the tests against a large number of browsers.

I have an example Jenkins job, also in the sauce-unit-test-example Github repo. Go clone the repository and try it out for yourself! Good luck!

About the Author

Object Partners profile.
Leave a Reply

Your email address will not be published.

Related Blog Posts
Natively Compiled Java on Google App Engine
Google App Engine is a platform-as-a-service product that is marketed as a way to get your applications into the cloud without necessarily knowing all of the infrastructure bits and pieces to do so. Google App […]
Building Better Data Visualization Experiences: Part 2 of 2
If you don't have a Ph.D. in data science, the raw data might be difficult to comprehend. This is where data visualization comes in.
Unleashing Feature Flags onto Kafka Consumers
Feature flags are a tool to strategically enable or disable functionality at runtime. They are often used to drive different user experiences but can also be useful in real-time data systems. In this post, we’ll […]
A security model for developers
Software security is more important than ever, but developing secure applications is more confusing than ever. TLS, mTLS, RBAC, SAML, OAUTH, OWASP, GDPR, SASL, RSA, JWT, cookie, attack vector, DDoS, firewall, VPN, security groups, exploit, […]