Tag Archives: Apache Kafka

Before You Adopt Event-Driven Architecture: Prerequisites, Red Flags, and Partition Strategy

Most teams adopting event-driven architecture never stop to understand what it actually involves. Before committing to the operational overhead that transforms a three-component system into fifteen, verify you actually have the problems EDA exists to solve. This guide provides a decision framework, prerequisites checklist, and a deep dive into partition strategy — the most consequential design decision you’ll make in EDA.

An introduction to Apache Kafka & Amazon MSK (Managed Kafka Service)

Explore the fundamentals of Apache Kafka and Amazon MSK, the cutting-edge solutions for real-time data streaming and processing. Designed for developers and architects, this guide introduces you to the high-throughput, low-latency capabilities of Kafka and the ease of management provided by Amazon MSK. Learn about Kafka’s publisher-subscriber model, efficient data handling, and the seamless integration with AWS services. Discover how these technologies facilitate robust, scalable communication between data producers and consumers, ensuring high availability, security, and seamless data flow across distributed systems.

Why are asynchronous Event-driven microservices complex compared?

An example of the additional complexities which event-driven architecture brings. In this example, I use a combination of Apache Kafka (AWS MKS), Oracle, and Amazon Data Migration Service (AWA DMS) to change an existing synchronous request-driven microservice to a asynchronous event-driven solution.

How to delay reprocessing when consuming from a Apache Kafka (retry) topic

Have you watched the video How to handle message retries & failures in event driven-systems? Handling retires with Kafka? , but you need more information. Hopefully this post helps. Let’s say you have a consumer, a retry consumer, subscribing to a retry topic. When the retry consumer consumes the retry topic and retrieves message/s, the consumer must check… Read More »

Why not having a Schema registry Can Be a Fatal Mistake in Event-Driven Architecture

How to ensure events are processed in order when retrying events out of sequence (when using event platforms like Apache Kafka)

In the post, How to handle retry messages and failures when using event streaming platforms like Apache Kafka, I walked through how to retry events when using Apache Kafka. Now in this post, I will discuss an approach you can implement to ensure processors process events in order even when you’ve implemented a retry solution. YouTube video After… Read More »

How to handling retry messages and failures when using event streaming platforms like Apache Kafka

Message Brokers VS Event Brokers: Choosing the right message broker for an Event-Driven System (Apache Kafka vs RabbitMQ)

Intro So you have learned (see my prevoius posts/videos) about the pros and cons of event-driven architecture, and you are still are looking to start your journey by either incrementally strangling an existing system or using EDA for a greenfield project. In this post, I will introduce you to the concept of a broker and run you through… Read More »