OPI Blog
Learn from experts in their fields

Browsing Tags: kafka

Why Is Everyone Talking About Kafka?
Fending off buzzwords can feel like a full-time job, so having a reliable approach to assessing technologies is a critical piece of an engineer’s toolkit. With the rise of Kafka, more and more companies are ...
Capturing missing events with Apache Kafka Streams
Introduction There is a lot going on with Kafka Streams. Understanding it takes time, and it always seems there is more you could learn. In this article, I will utilize Kafka Core and Streams for ...
Testing with Spring Kafka and MockSchemaRegistryClient
Spring Kafka provides a variety of testing utilities to make writing integration tests easier. Most notably, the @EmbeddedKafka annotation spins up an embedded broker (and zookeeper) available for tests. The address of the broker is set to ...
Aug 14, 2018
3 Kotlin Features to Improve Your Kafka Connect Development
Kafka Connect is the hub that connects your Kafka cluster to any other system. The framework aims to make it easy to pull data into Kafka as well as copy data out of Kafka. As ...
Aug 7, 2018
Apache Kafka and Reactive Spring Boot
This post is a continuation of the two part series exploring Apache Ignite, Apache Kafka, and Reactive Spring Boot concepts. This part covers the use of Reactive Kafka consumers to return live database events to ...
Configuring Kafka Topics with Spring Kafka
Here’s a common scenario, your engineering team decides to use Kafka and they start writing producers, consumers, connectors, streams, you name it. Before long, 6 engineers have manually created 20+ topics in the lower environment ...
Jul 26, 2018
Streaming Apache Ignite changes in Apache Kafka Topics
This is a two part series exploring Apache Ignite, Apache Kafka, and Reactive Spring Boot concepts. This part covers the setting up of Apache Ignite as a datasource and using Apache Kafka as a streaming ...
Kafka Testing using YAML
Integration testing can be difficult for distributed systems. I’ve found this to be true when testing Kafka to ensure data is produced correctly. I’ve tried using a consumer in my tests to verify data made ...
Building a Kafka Connector with Gradle
Kafka is an exciting space and what I have been finding most exciting is the data streaming aspect of it. Kafka Connectors are the bookends to a powerful streaming engine that quickly transform and analyze ...
Functional Testing With Embedded Kafka
Spring publishes a spring-kafka-test library that is promoted as a way to do some unit testing with Kafka. The project I’m currently working on reads messages from a JMS queue, does some transforming of that ...