@jeqo

for a more updated version, check https://jeqo.github.io/posts/2019-03-26-importance-of-distributed-tracing-for-apache-kafka-based-applications/ Tracing is one of the hardest time in integration or microservice development: knowing how a request impact your different components, and if your components have behave as expected. This could be fairly easy if we have monolith where we have one database and with some queries or checking one log file you can validate everything went well. Once you introduce distributed components and asynchronous communication this starts to get more complex and tedious.

Presented at OUGN 2017

One of the most important features from Apache Kafka is how it manages Multiple Consumers. Each consumer group has a current offset, that determine at what point in a topic this consumer group has consume messages. So, each consumer group can manage its offset independently, by partition. This offers the possibility to rollback in time and reprocess messages from the beginning of a topic and regenerate the current status of the system. But how to do it (programmatically)?

In this post I will show how to use Docker containers to create and scale a Kafka cluster, and also how to create, scale and move topics inside the cluster.

I decided to implement a naive integration between Java EE applications and RxJava/Kafka/Avro, to publish and subscribe to events. You can go directly to that code, or check my approach:

I’ve had the opportunity to present a review of this book Applied SOA Patterns on the Oracle Platform.