An introduction to Apache Kafka & Amazon MSK (Managed Kafka Service)

Explore the fundamentals of Apache Kafka and Amazon MSK, the cutting-edge solutions for real-time data streaming and processing. Designed for developers and architects, this guide introduces you to the high-throughput, low-latency capabilities of Kafka and the ease of management provided by Amazon MSK. Learn about Kafka’s publisher-subscriber model, efficient data handling, and the seamless integration with AWS services. Discover how these technologies facilitate robust, scalable communication between data producers and consumers, ensuring high availability, security, and seamless data flow across distributed systems.

Amazon Kinesis: Part 1 – Introduction for Developers and Architects

Dive into the dynamic world of Amazon Kinesis, a premier AWS service for real-time data processing and analytics. Designed for developers and architects, this introduction covers the fundamentals of Kinesis, including its ability to handle high-volume data streams from thousands of sources with minimal latency. Learn about its applications in real-time analytics, IoT data processing, log collection, and more. Understand how Kinesis facilitates immediate data ingestion, processing, and analysis, enabling timely decision-making across diverse sectors like finance, social media, and IoT.

Amazon SNS: Part 1 – Introduction for Developers and Architects

Dive into the core functionalities of Amazon SNS (Simple Notification Service), a cornerstone for enabling real-time and event-driven architectures in cloud computing. Understand how SNS facilitates seamless communication between applications through its pub/sub messaging model, allowing for broad-scale message distribution to various subscribers. Explore the service’s key features, including multi-protocol support, high throughput, message filtering, and integration within the AWS ecosystem, and learn how it ensures secure, scalable, and efficient messaging across diverse applications.

AWS SQS: Part 1 – Introduction for Developers and Architects

Explore the fundamentals of Amazon SQS, a cornerstone of message queuing within the AWS ecosystem. Learn about its application in microservices, distributed systems, and serverless functions, including the types of queues available, message lifecycle, and the unique features and benefits that SQS offers. Discover how SQS ensures high throughput, system reliability, controlled message delivery, and robust security, making it an essential tool for scalable, reliable, and efficient message handling.

Overview of Communication Patterns: Event-Driven, Message-Driven and Request-Driven

Dive deep into the intricate world of communication patterns in system architecture. Explore the strengths and trade-offs of request-driven, message-driven, and event-driven models, and how they shape the interaction, scalability, and resilience of modern applications. Unravel the complexities and operational dynamics that can make or break your system’s architecture.

When to use AWS SQS: Strategically Selecting the Optimal Messaging Solution (Strategic Decision-Making: Part 2)

AWS Simple Queue Service (SQS) is a cornerstone in cloud computing, providing robust solutions for messaging and queuing challenges. This post takes a deep dive into the realms where SQS excels, from order processing systems to distributed transaction processing. However, understanding when to pivot towards alternatives like SNS, Kinesis, or Kafka is just as crucial. Uncover the scenarios where SQS is the optimal choice and the circumstances that call for a different strategy.

Part 1: Strategic Decision-Making: Understanding AWS Suite Of Messaging/Event Services

Understanding the diverse landscape of AWS Messaging Services is essential for architecting robust, scalable, and efficient cloud solutions. AWS offers a suite of messaging services, each designed with unique features and collaboration patterns to cater to specific architectural needs. From SQS’s reliable message queuing and SNS’s dynamic pub/sub messaging to Kinesis’s real-time data streaming and MSK’s high-throughput message handling, the choice of service can profoundly impact your system’s performance, reliability, and scalability. Whether it’s decoupling components with SQS, broadcasting notifications with SNS, processing vast data streams with Kinesis, or handling high-volume events with MSK, understanding these services’ capabilities and use cases is crucial for making informed and strategic architectural decisions.

Centralised Orchestration vs Decentralised Choreography in Event-Driven Serverless Systems

In this article, I will share my thoughts on using centralised versus decentralised orchestration when building event-driven serverless systems. I have over 15 years of experience in designing and developing distributed event-driven systems. I have spent years migrating from monolithic systems to microservices and now to serverless-powered systems. Event-driven architecture can help build highly loosely coupled, fault-tolerant, scalable,… Read More »