OPI Blog
Learn from experts in their fields

Browsing Tags: kafka

Configuring Kafka Topics with Spring Kafka
Here’s a common scenario, your engineering team decides to use Kafka and they start writing producers, consumers, connectors, streams, you name it. Before long, 6 engineers have manually created 20+ topics in the lower environment ...
Jul 26, 2018
Streaming Apache Ignite changes in Apache Kafka Topics
This is a two part series exploring Apache Ignite, Apache Kafka, and Reactive Spring Boot concepts. This part covers the setting up of Apache Ignite as a datasource and using Apache Kafka as a streaming ...
Kafka Testing using YAML
Integration testing can be difficult for distributed systems. I’ve found this to be true when testing Kafka to ensure data is produced correctly. I’ve tried using a consumer in my tests to verify data made ...
Building a Kafka Connector with Gradle
Kafka is an exciting space and what I have been finding most exciting is the data streaming aspect of it. Kafka Connectors are the bookends to a powerful streaming engine that quickly transform and analyze ...
Oct 25, 2017
Functional Testing With Embedded Kafka
Spring publishes a spring-kafka-test library that is promoted as a way to do some unit testing with Kafka. The project I’m currently working on reads messages from a JMS queue, does some transforming of that ...
Oct 28, 2016
Simplifying Distributed Systems Using Apache Kafka
With the rising popularity of micro service architectures and typical scaling patterns used at enterprises, distributed systems are becoming more common and complex. What was once a simple web server connected to a database now ...
Oct 13, 2016
Analyzing Kafka data streams with Spark
This blog describes a Spark Streaming application which consumes event data from a Kafka topic to provide continuous, near real-time processing and analysis of the event data stream. The demonstration application, written in Java 8 ...
May 6, 2014
Setting up your own Apache Kafka cluster with Vagrant – Step by Step
Apache Kafka is a distributed publish-subscribe messaging system that aims to be fast, scalable, and durable. If you want to just get up and running quickly with a cluster of Vagrant virtual machines configured with ...