OPI Blog
Learn from experts in their fields

Browsing Tags: kafka

Unleashing Feature Flags onto Kafka Consumers
Feature flags are a tool to strategically enable or disable functionality at runtime. They are often used to drive different user experiences but can also be useful in real-time data systems. In this post, we’ll ...
May 10, 2022
Kafka Schema Evolution With Java Spring Boot and Protobuf
In this blog I will be demonstrating Kafka schema evolution with Java, Spring Boot and Protobuf.  This app is for tutorial purposes, so there will be instances where a refactor could happen. I tried to ...
Kafka & Kubernetes: Scaling Consumers
Kafka and Kubernetes (K8s) are a great match. Kafka has knobs to optimize throughput and Kubernetes scales to multiply that throughput. On the consumer side, there are a few ways to improve scalability. Resource & ...
Jan 28, 2021
Exploring the CloudEvents Landscape
The cloudevents specification provides a standardized way of describing events that is meant to be consistent, accessible, and portable. The idea of the specification is simple; lots of cloud systems and tools produce and consume ...
Sep 3, 2020
Using Spring Beans in a Kafka Streams ExceptionHandler
There are many things to know before diving into Kafka Streams. If you haven’t already, check out these 5 things as a starting point. Bullet 2 mentions designing for exceptions. Ironically, this seems to be ...
Concurrently Process a Single Kafka Partition
Concurrency in Kafka is defined by how many partitions make up a topic. For a consumer group, there can be as many consumers as there are partitions, with each consumer being assigned one or more ...
Migrating to an Event-Driven System
Unless you have been granted the golden ticket to greenfield development, there isn’t a hard and fast approach to migrating to an Event-Driven System. However, there are designs, technologies, best practices and approaches that will ...
5 Things to Know Before Using Kafka Streams
The Kafka Streams API has been around since Apache Kafka v0.10 and as the adoption of Kafka booms, so does Kafka Streams. The Streams library enables developers to create distributed processing applications while avoiding most ...
Fault Tolerant Distributed Tracing with Apache Kafka and Jaeger
Using Jaeger tracing, I’ve been able to answer an important question that nearly every Apache Kafka project that I’ve worked on posed: how is data flowing through my distributed system? I’ve started writing about this ...
Distributed Tracing with Apache Kafka and Jaeger
If you are using Apache Kafka, you are almost certainly dealing with many applications that need to work together to accomplish some big picture goal. In other words, you are working within a distributed system. ...