Sep 3, 2020

Using Spring Beans in a Kafka Streams ExceptionHandler

There are many things to know before diving into Kafka Streams. If you haven’t already, check out these 5 things as a starting point. Bullet 2 mentions designing for exceptions. Ironically, this seems to be the most critical as well as most forgotten… until a SEV-1 (this is bad) issue rolls into your inbox and the frantic patch process is initiated.

Before the incident lands in your inbox, let’s look at an easy way to bridge your custom exception handler (deserialization or production) to Spring.

Why is this bridge needed? When Kafka Streams starts up, the configured exception handler(s) are instantiated via reflection deep inside the Stream’s internals. As a result, they have no awareness of Spring’s Application Context. The ‘configure’ method on the handler interface is passed the current stream’s configurations and nothing else. This likely won’t provide enough context to build robust error handling that can produce a record, dump something to the DB, call an ancillary service, fire off an alert, etc.

Bridge to Spring

To get around this design, we can create a simple bridge to our Application Context. The following Gist illustrates an approach that achieves this. Notice it implements ApplicationContextAware, which will hand off the current application context state that we can keep for later use. Two static methods are defined for bean lookup, one by Class and one by name and Class which can be used if you have multiple beans of the same Class.

Second, the new utility can now be tapped into during the configure method of your exception handler to pull out any beans that may be helpful in error handling. In this case, I’m showing an implementation of the ProductionExceptionHandler but the same pattern can be used for the deserialization exception handler.


That’s it! Have you found alternatives to this approach that work for your team? I’d love to hear about them in the comments.

About the Author

Matt Schroeder profile.

Matt Schroeder

Director, Real-Time Data

A wide range of professional experience and a Master’s Degree in Software Engineering have become the foundation that enables Matt to lead teams to the best solution for every problem.

Leave a Reply

Your email address will not be published.

Related Blog Posts
Natively Compiled Java on Google App Engine
Google App Engine is a platform-as-a-service product that is marketed as a way to get your applications into the cloud without necessarily knowing all of the infrastructure bits and pieces to do so. Google App […]
Building Better Data Visualization Experiences: Part 2 of 2
If you don't have a Ph.D. in data science, the raw data might be difficult to comprehend. This is where data visualization comes in.
Unleashing Feature Flags onto Kafka Consumers
Feature flags are a tool to strategically enable or disable functionality at runtime. They are often used to drive different user experiences but can also be useful in real-time data systems. In this post, we’ll […]
A security model for developers
Software security is more important than ever, but developing secure applications is more confusing than ever. TLS, mTLS, RBAC, SAML, OAUTH, OWASP, GDPR, SASL, RSA, JWT, cookie, attack vector, DDoS, firewall, VPN, security groups, exploit, […]