Jul 16, 2013

Creating self-contained, executable Jars with Gradle and Shadow

Earlier this year, we started implementing a new backend REST architecture for our application based on Yammer/Coda Hale’s Dropwizard Framework. Since we don’t use Maven anywhere in our stack, we decided to use Gradle as our build tool. This gives us enormous flexibility as we built our new backend around the concept of “micro-services” – each service is isolated from the others and responsible for a specific piece of functionality.

Since these services aren’t WARs, we wanted a way to deploy a pre-packaged jar that we could simply execute on our servers. Maven has a nice tool for this – Apache Maven Shade Plugin. It repackages all of your application’s dependencies into a single JAR file and offers a variety of extension points for effecting the contents of the resulting file. Unfortunately, at the time, Gradle lacked a similar plugin (there are a variety of similar plugins but all had severe limitations for our application). Instead, we wrote a Gradle port of the Shade plugin – Gradle Shadow Plugin.

Shadow is a port of the Shade plugin including most of its extensions points. This makes it easier for converting existing Maven builds to Gradle. It is open sourced under the ALv2 and contributions are most welcome.

To Shadow enable your project, add the BinTray repository and the Shadow plugin dependency to your buildscript configuration like so:

By default, Shadow will bundle all our your compile and runtime dependencies into an additional JAR with the classifier ‘shadow’ appended to it. This JAR is directly executable by executing the JAR:

$ java -jar build/libs/shadow-blog-0.1-shadow.jar
 Hello World

Notice that trying to execute the output of the normal JAR tasks results in an error due to the Groovy library not being available on the classpath:

$ java -jar build/libs/shadow-blog-0.1.jar 
Exception in thread "main" java.lang.NoClassDefFoundError: groovy/lang/GroovyObject at 
java.lang.ClassLoader.defineClass1(Native Method) at 
java.lang.ClassLoader.defineClassCond(ClassLoader.java:631) at 
java.lang.ClassLoader.defineClass(ClassLoader.java:615) at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) at 
java.net.URLClassLoader.defineClass(URLClassLoader.java:283) at 
java.net.URLClassLoader.access$000(URLClassLoader.java:58) at 
java.net.URLClassLoader$1.run(URLClassLoader.java:197) at 
java.security.AccessController.doPrivileged(Native Method) at 
java.net.URLClassLoader.findClass(URLClassLoader.java:190) at 
java.lang.ClassLoader.loadClass(ClassLoader.java:306) at 
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at 
java.lang.ClassLoader.loadClass(ClassLoader.java:247) 
Caused by: java.lang.ClassNotFoundException: groovy.lang.GroovyObject at
java.net.URLClassLoader$1.run(URLClassLoader.java:202) at 
java.security.AccessController.doPrivileged(Native Method) at 
java.net.URLClassLoader.findClass(URLClassLoader.java:190) at 
java.lang.ClassLoader.loadClass(ClassLoader.java:306) at 
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at
java.lang.ClassLoader.loadClass(ClassLoader.java:247) 
... 12 more 

This simple example can be found on Github – Shadow Blog.
For more advanced usages of the Shadow plugin, refer to the README

About the Author

Object Partners profile.

One thought on “Creating self-contained, executable Jars with Gradle and Shadow

  1. Erik Pragt says:

    Hi John,

    Nice writeup! Could you tell me which limitations you ran into? I did the same thing; creating an executable jar using Gradle for my dropwizard project, and after some initial hickups, it works fine:

    https://github.com/bodiam/epub-organizer/blob/master/epub-organizer-dropwizard/epub-organizer-dropwizard.gradle

    Erik

    1. Arunav Sanyal says:

      Hey thanks man that helped quite a bit. I got my application to work with it.

  2. Tomas Lin says:

    Erik, I don’t think the jar task in gradle will include all the dependencies so you can’t run it standalone. It seems other folks were using the fatjar plugin to solve this problem – https://github.com/musketyr/gradle-fatjar-plugin . I wonder how shadow improves on fatjar. ( Dropwizard with fatjar — https://github.com/tomaslin/dropwizard-gradle-groovy/blob/master/build.gradle )

  3. Kyle says:

    Shadow has a larger feature set (ported from Maven’s shade plugin) and is MUCH faster than the fat-jar plugin. fat-jar was taking ~8 minutes for us to build and shadow builds in 20 or 30 seconds.

  4. John Engelman says:

    @Erik – we evaluated OneJar (https://github.com/rholder/gradle-one-jar) & FatJar before deciding to write Shadow. OneJar didn’t work for us because service decriptors didn’t work correctly and FatJar was both too slow & didn’t allow us to configure everything we wanted to.

    @Tomas – Our biggest complaint against FatJar was speed. It was taking minutes to execute the fatjar task whereas Shadow is in the 20 second range for a simple build. FatJar does its work by physically decompressing the Jars to a staging directory and then re-jarring it. Shadow uses Jar I/O Streams to read from the resolved libraries and write directly to the output file – no intermediate staging. Additionally, Shadow is a port of Apache Shade which is the predominant tool in the Maven world for doing this (referenced by the Dropwizard documentation itself). We thought a port of Shade to Gradle would be helpful with lowering the entry bar of converting complex builds from Maven to Gradle.

  5. Thank you. This seems to solve a problem loading static content as well.

  6. Jakub Holy says:

    This is wonderful! It would be great if you managed to make the plugin available in Maven Central to make it more accessible. (It should not be too complicated via Sonatype’s OSS offer).

    Regards, Jakub

    1. John Engelman says:

      Jakub –
      Thanks for taking a look. There is an outstanding issue in Github to add this feature as well.
      I plan in the next version (0.8) to release Shadown in the JCenter repository.
      JCenter is just as easily accessible from a Gradle build as MavenCentral and a bit easier for me to maintain and use.
      In addition, there will likely be a shift in the artifact’s group name to more appropriately locate the plugin.

      Thanks for your interest! If you have any other issues or suggestions when using Shadow, please raise them on the Project’s Github page.

      John

  7. Michal says:

    Hi I have a question. I use gradle with plugin shadow and all works well when I use ‘gradle shadowJar’. I have problem when I publish my artifacts to maven, e.g. ‘gradle publishToMavenLocal’ – the uber-jar is not created – my jars are without dependencies.
    How can I deal with it?

  8. James says:

    Love the look of the plugin.

    Any further thoughts on publishing to Maven Central? Having the option (esp I’m circumstances where repos are limited) would be a massive plus.

    Thanks for what looks to be a great plugin.

  9. EK says:

    Love the plugin. Can’t use the script creation though.

    In the case where someone needs to add to classpath (I do realize all JARs are supposed to be in the shadow jar, but….) in order to set a java agent (say jolokia) and have it found by the JVM, there is no way to do adequate JVM options and/or classpath modification.

Leave a Reply

Your email address will not be published.

Related Blog Posts
Natively Compiled Java on Google App Engine
Google App Engine is a platform-as-a-service product that is marketed as a way to get your applications into the cloud without necessarily knowing all of the infrastructure bits and pieces to do so. Google App […]
Building Better Data Visualization Experiences: Part 2 of 2
If you don't have a Ph.D. in data science, the raw data might be difficult to comprehend. This is where data visualization comes in.
Unleashing Feature Flags onto Kafka Consumers
Feature flags are a tool to strategically enable or disable functionality at runtime. They are often used to drive different user experiences but can also be useful in real-time data systems. In this post, we’ll […]
A security model for developers
Software security is more important than ever, but developing secure applications is more confusing than ever. TLS, mTLS, RBAC, SAML, OAUTH, OWASP, GDPR, SASL, RSA, JWT, cookie, attack vector, DDoS, firewall, VPN, security groups, exploit, […]