Aug 30, 2016

Monitoring Grails Applications The Easy Way

Grails allows you to create RESTful APIs or Web applications much more quickly than other JVM frameworks. You may be surprised to know, though, that Grails also provides a number of handy features to help you monitor the health and performance of your applications.

Some of the these features you get for free while others require a bit of work on your part. This post should give you a taste of what you can do with very little work on your part.

When you want to start monitoring your Grails applications, the first place to start should be the Spring Boot production-ready actuators. Despite the odd terminology, these actuators provide a lot of handy features right out of the box. Actuators provide data on your running application through JSON endpoints.

Some of the actuator endpoints can help with diagnosing configuration issues, such as /autoconfig, /configprops, and /env. These aren’t related to monitoring your applications but they can help if you are having issues, especially with Spring configuration.

The /configprops endpoint lists all the Spring configuration properties but doesn’t include all the Grails-specific properties. The /env endpoint provides more information including Grails configuration settings. I’ve used these actuators to help debug Spring Security configuration issues.

In addition, the /info endpoint provides some basic information on your application, such as the Grails version number and your application’s version number. You can extend the information presented by writing Spring beans that implement the InfoContributor interface.

The endpoints most useful for monitoring include /health and /metrics.

Checking Application Health

The /health endpoint is designed to tell you whether your application is healthy or not. By default, it just tells you whether the application is up or not, along with information on disk space.

You can see this in action by creating a new Grails application and running it.

Create the application (using Grails 3.1.9):

grails create-app monitor --profile=web

Then, you need to edit the grails-app/conf/application.yml file, since the endpoints are disabled by default. Change the enabled setting to true for the endpoints:


# Spring Actuator Endpoints are Disabled by Default
endpoints:
    enabled: true
    jmx:
        enabled: true

Now, run the application:

grails run-app

Go to the /health endpoint, by default mapped to /health in your application, http://localhost:8080/health.

You will see output like the following:


{
    "status":"UP",
    "diskSpace": {
        "status":"UP",
        "total":250140434432,
        "free":203297030144,
        "threshold":10485760
    }
}

Typically though, you’d want a health check to tell you if a back-end service connection is working, or report on other possible conditions with your application. You can do this by defining beans that implement the HealthIndicator interface. Spring Boot comes with some handy built in indicators, including checks for disk space, and connectivity to systems such as Cassandra, MongoDB, SOLR, and Rabbit MQ, as well as a configured Grails DataSource.

You can write your own as well, for example to report the health of a back-end accounting system, you can use the following as a guide:


package com.opi.monitor

import org.springframework.boot.actuate.health.Health
import org.springframework.boot.actuate.health.HealthIndicator

/**
 * Checks back-end accounting system's health.
 */
class AccountingHealthMonitor implements HealthIndicator {
    Health health() {
        if (isHealthy()) {
             return Health.up().build()
        }
        return Health.down().withDetail("Accounting System Status", "Unreachable").build();
    }

    /**
     * In real life this method would check the health of the back-end system.
     * @return
     */
    boolean isHealthy() {
        false
    }
}

Define this class as a Spring bean in your grails-app/conf/spring/resources.groovy:


import com.opi.monitor.AccountingHealthMonitor

beans = {
    accountingHealthMonitor(AccountingHealthMonitor)
}

Since this example fakes a down status, you will now see a /health endpoint response that looks something like the following:


{
    "status":"DOWN",
    "diskSpace":{
        "status":"UP",
        "total":250140434432,
        "free":202630787072,
        "threshold":10485760
    },
    "accountingHealthMonitor":{
        "status":"DOWN",
        "Accounting System Status":"Unreachable"
    }
}

Notice how since one health indicator shows a DOWN status, the overall application status is DOWN.

Viewing Application Metrics

The /metrics endpoint contains a lot of data about how your application is running, including information on memory, processors, threads and so on, along with DataSource and Tomcat session metrics. Spring Boot by default uses the concepts of gauges and counters. A gauge holds a value and a counter shows a change up or down over time.

You can view the metrics with the /metrics endpoint. Here’s an example for a simple Grails Web application with one domain class:


{
    "mem":836396,
    "mem.free":583908,
    "processors":8,
    "instance.uptime":475604,
    "uptime":482989,
    "systemload.average":0.78564453125,
    "heap.committed":748032,
    "heap.init":786432,
    "heap.used":164123,
    "heap":748032,
    "nonheap.committed":92160,
    "nonheap.init":2496,
    "nonheap.used":88366,
    "nonheap":0,
    "threads.peak":34,
    "threads.daemon":31,
    "threads.totalStarted":39,
    "threads":34,
    "classes":11753,
    "classes.loaded":11753,
    "classes.unloaded":0,
    "gc.ps_scavenge.count":22,
    "gc.ps_scavenge.time":232,
    "gc.ps_marksweep.count":3,
    "gc.ps_marksweep.time":534,
    "httpsessions.max":-1,
    "httpsessions.active":1,
    "gauge.response.assets.skin.database_edit.png":1.0,
    "gauge.response.health":262.0,
    "gauge.response.assets.main.css":4.0,
    "gauge.response.assets.skin.database_add.png":1.0,
    "gauge.response.assets.mobile.css":3.0,
    "gauge.response.unmapped":254.0,
    "gauge.response.assets.grails-cupsonly-logo-white.svg":3.0,
    "gauge.response.employee.create":530.0,
    "gauge.response.assets.application.css":3.0,
    "gauge.response.employee.show.1":309.0,
    "gauge.response.assets.skin.database_delete.png":1.0,
    "gauge.response.assets.spinner.gif":1.0,
    "gauge.response.assets.skin.information.png":1.0,
    "gauge.response.assets.jquery-2.2.0.min.js":4.0,
    "gauge.response.assets.bootstrap.js":4.0,
    "gauge.response.assets.application.js":3.0,
    "gauge.response.assets.skin.database_save.png":2.0,
    "gauge.response.assets.skin.database_table.png":2.0,
    "gauge.response.assets.bootstrap.css":6.0,
    "gauge.response.assets.grails.css":3.0,
    "gauge.response.assets.skin.house.png":1.0,
    "gauge.response.employee.index":1611.0,
    "counter.status.200.assets.application.css":3,
    "counter.status.200.assets.bootstrap.css":3,
    "counter.status.200.assets.skin.database_save.png":1,
    "counter.status.200.assets.mobile.css":3,
    "counter.status.200.assets.grails.css":3,
    "counter.status.200.assets.skin.database_edit.png":1,
    "counter.status.200.employee.create":1,
    "counter.status.200.assets.skin.information.png":1,
    "counter.status.200.assets.spinner.gif":3,
    "counter.status.200.employee.index":1,
    "counter.status.200.assets.skin.database_table.png":2,
    "counter.status.302.unmapped":1,
    "counter.status.200.assets.application.js":3,
    "counter.status.200.assets.grails-cupsonly-logo-white.svg":3,
    "counter.status.200.assets.skin.house.png":3,
    "counter.status.200.employee.show.1":1,
    "counter.status.503.health":1,
    "counter.status.200.assets.jquery-2.2.0.min.js":3,
    "counter.status.200.assets.skin.database_delete.png":1,
    "counter.status.200.assets.bootstrap.js":3,
    "counter.status.200.assets.skin.database_add.png":2,
    "counter.status.200.assets.main.css":3
}

You can extend the available metrics by implementing the PublicMetrics interface, as documented at http://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-metrics.html.

Instead of writing PublicMetrics classes, though, many prefer to use a add-on library called Dropwizard Metrics.

Dropwizard Metrics

The Dropwizard Metrics library provides a fairly detailed library to allow your application to capture metrics.

Dropwizard uses a bit different terminology than Spring Boot though. In Dropwizard, a meter holds the rate of events over time. A timer provides a histogram of an event type along with a meter of the rate.

The best feature though, is that Spring Boot’s metrics automatically works with the Dropwizard Metrics library. Spring Boot registers all its metrics with the Dropwizard MetricRegistry. The interaction goes both ways, too. Metrics you add to a MetricRegistry using Dropwizard will also appear in the /metrics endpoint.

You can find more information on this library at http://metrics.dropwizard.io/3.1.0/.

To use the Dropwizard Metrics library, you’ll want to install the handy Grails plugin.

Installing Dropwizard Metrics

Unfortunately, you won’t likely find information on the Grails 3 plugin portal at https://grails.org/plugins.html. Instead, as is common with so many Grails 3 plugins in a snapshot state, look on github at:

Documentation is at http://grails-plugins.github.io/grails-dropwizard-metrics/snapshot/index.html.

To install, add the following dependency to your build.gradle file:

compile "org.grails.plugins:dropwizard-metrics:1.0.0.BUILD-SNAPSHOT"

Using the Dropwizard Metrics Library

Once installed, you can use the library itself to record metrics in as detailed a format as you’d like. Since this post concentrates on using out-of-the-box features, you should look at the handy annotations.

The @Metered annotation records an event into a named meter. Similarly, the @Timed annotation starts a timer every time the method is called.

For meters, you can use the following example service as a guide:


import grails.plugin.dropwizard.metrics.meters.Metered
import grails.transaction.Transactional

@Transactional
class UsageService {

    @Metered('accounting')
    def triggerUsage() {
        // This method would access some service
    }
}

Every call to the triggerUsage() method will record an event to the accounting meter.

For timers, here is an example of a slow service that performs a lot of time-consuming work:


import grails.plugin.dropwizard.metrics.timers.Timed
import grails.transaction.Transactional

@Transactional
class SlowService {

    @Timed('slow')
    def performWork() {
        // Simulate slowness
        Thread.sleep(1000)
    }
}

Your services will have real code, obviously.

Using these annotations, you can see the current values with the /metrics endpoint (showing just part of the new output):


"accounting.oneMinuteRate":0.029887807318695676,
"accounting.fiveMinuteRate":0.12063495313768283,,
"accounting.fifteenMinuteRate":0.16853371667117334,
"accounting.snapshot.mean":1015,
"accounting.snapshot.median":1013,
"counter.status.200.accounting.work":3,
"gauge.response.accounting.work":1016.0,
"accounting.snapshot.min":1013,
"accounting.snapshot.stdDev":12,
"accounting.meanRate":0.01779413878674953,
"accounting.snapshot.75thPercentile":1013,
"accounting.snapshot.95thPercentile":1013,
"accounting.snapshot.98thPercentile":1075,
"accounting.snapshot.99thPercentile":1075,
"accounting.snapshot.999thPercentile":1075,

"slow.count":3,
"slow.snapshot.75thPercentile":1000,
"slow.oneMinuteRate":0.029887807318695676,
"slow.fiveMinuteRate":0.12063495313768283,
"slow.fifteenMinuteRate":0.16853371667117334,
"slow.snapshot.stdDev":1,
"slow.snapshot.median":1000,
"slow.snapshot.max":1006,
"slow.meanRate":0.017800364215211435,
"slow.snapshot.mean":1000,
"slow.snapshot.min":1000,
"slow.snapshot.95thPercentile":1000,
"slow.snapshot.98thPercentile":1006,
"slow.snapshot.99thPercentile":1006,"
"slow.snapshot.999thPercentile":1006,

As you can see, you get a lot of data from just using annotations. You can also, of course, make more use of the library to extract even more data.

Using the Actuator UI Plugin

The Spring Boot actuator endpoints all respond with data in JSON format. You can get a nicer look on top of this data by using the actuator-ui plugin.

To install the actuator-ui plugin, add the following to your dependencies in your build.gradle file.

compile 'org.grails.plugins:actuator-ui:0.2'

Note: Version 1.0 of the actuator-ui plugin was just released to support Grails 3.1.

Once installed, you can view the data in a more friendly format at /actuator/dashboard.

You can see a demo of this feature in a YouTube video at https://www.youtube.com/watch?v=huhC1LV5I8Q.

Note that due to the sensitivity of the data, you want to secure the actuator user interface for administrators. The documentation at https://grails.org/plugins.html#plugin/actuator-ui provides guidance on how to do this with Spring Security.

In addition to securing the actuator-ui, you also should secure the entire set of actuator endpoints.

Securing Actuator Endpoints

Allowing this information to get out in production can be a security issue. You can use the Spring Security Grails plugin to lock down these end points. Endpoints marked as sensitive in your configuration should get automatically protected by Spring Security. Be sure to read the documentation at http://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-monitoring.html to see what options are available to you.

Spring Boot allows a lot of customization. You can map these endpoint URLs to something different if you’d like. In addition, you can access the data via JMX if you’d prefer.

All of this shows a taste of the information you can extract from your Grails applications with minimal work. Yet another reason to use Grails.

About the Author

Eric Foster-Johnson profile.

Eric Foster-Johnson

Principal Technologist

Eric has decades of industry experience in designing and developing complex enterprise software, including designing and developing Grails and Java EE solutions to tough client problems. He has experience leading development teams, mentoring developers, and helping troublesome projects get back onto a success track. He has lead teams in both traditional and agile settings.

One thought on “Monitoring Grails Applications The Easy Way

  1. Mike Miller says:

    Looks great!

Leave a Reply

Your email address will not be published.

Related Blog Posts
Natively Compiled Java on Google App Engine
Google App Engine is a platform-as-a-service product that is marketed as a way to get your applications into the cloud without necessarily knowing all of the infrastructure bits and pieces to do so. Google App […]
Building Better Data Visualization Experiences: Part 2 of 2
If you don't have a Ph.D. in data science, the raw data might be difficult to comprehend. This is where data visualization comes in.
Unleashing Feature Flags onto Kafka Consumers
Feature flags are a tool to strategically enable or disable functionality at runtime. They are often used to drive different user experiences but can also be useful in real-time data systems. In this post, we’ll […]
A security model for developers
Software security is more important than ever, but developing secure applications is more confusing than ever. TLS, mTLS, RBAC, SAML, OAUTH, OWASP, GDPR, SASL, RSA, JWT, cookie, attack vector, DDoS, firewall, VPN, security groups, exploit, […]