When to use AWS SQS: Strategically Selecting the Optimal Messaging Solution (Strategic Decision-Making: Part 2)

By | January 25, 2024

When to use SQS

AWS Simple Queue Service (SQS) is a highly versatile message queuing service that offers reliable, scalable, and secure ways to decouple and scale microservices, distributed systems, and serverless applications.
As decision-makers, understanding each service’s unique strengths and capabilities is critical when designing and architecting robust and reliable systems.

(Note: Before reading this article, ensure you have read the introduction to Amazon SQS, and also for the different application collaboration services (SNS, Kineses, Kafka as MSK) which Amazon provides, You can find them linked here)


Now, this post, part two, will dive deep into scenarios where SQS is the optimal solution and pinpoint times when an alternative AWS service might be more fitting.

If you have not read part one, then click on the link above and read that first.

When to Leverage AWS SQS

SQS excels in a variety of scenarios, making it a cornerstone for many applications:

1. Order Processing Systems

Reliable and sequential processing of customer orders is crucial in e-commerce platforms.

Why SQS

With FIFO queues, SQS ensures that every order is processed precisely once and in the exact order it was received, maintaining the integrity of transactional systems.

Why not SNS, Kinesis, or Kafka

SNS doesn’t guarantee message ordering, and while Kinesis and Kafka handle high-throughput data streams, their complexity may not be necessary for straightforward queuing needs.

2. Microservices Communication

Microservices architectures require a buffer to efficiently manage communication and workload spikes.

Why SQS

SQS, normal & FIFO queues decouple microservices, ensuring they operate independently and smoothly, even under varying workloads.

Why not SNS, Kinesis, or Kafka

SNS is optimised for notifications, not queuing. Kinesis and Kafka’s focus on data streaming introduces complexities that might not align with the basic queuing needs of microservices.

3. Buffering Requests During Traffic Spikes

Handling sudden increases in requests, such as during sales or promotions, without losing or delaying any request.

Why SQS

SQS functions as an efficient buffer, adeptly handling surges in requests during peak times or unexpected spikes in traffic. This capability prevents the consumer from being overloaded as the consumer pulls messages, therefore regulating the flow of requests SQS ensures that no request is lost or overlooked, providing a reliable and stable system even during periods of high demand.

Why not SNS, Kinesis, or Kafka

Kinesis and Kafka are more data-oriented and might complicate simple request buffering scenarios. SNS, primarily a notification service, isn’t designed to queue and manage request backlogs.

4. Delayed Job Execution

Jobs that must be executed after a certain delay, like scheduled reports or batch processes.

Why SQS

SQS’s delayed messages feature allows for the precise timing of message delivery, ensuring that messages are available to the consumer only when needed.


With standard queues, you have the option to use a delay queue that postpones the delivery of all messages and also to set an initial invisibility period for each message individually.

However, in the case of FIFO queues, while you can employ delay queues to delay all messages, setting invisibility timers for individual messages is not possible.

Why not SNS, Kinesis, or Kafka

SNS and Kinesis don’t inherently support delayed delivery, and while Kafka can be configured for this, it might not be as straightforward as using SQS’s built-in capabilities.

5. Handling Sporadic or Bursty Workloads

Applications with irregular workload patterns need a system that can smoothly handle periods of inactivity as well as sudden bursts of activity.

Why SQS

SQS’s ability to retain messages and its integration with auto-scaling make it ideal for managing such unpredictable workloads.

Why not SNS, Kinesis, or Kafka

While Kafka and Kinesis can manage high-throughput workloads, they may not be cost-effective or straightforward enough for handling sporadic or bursty traffic.

SNS’s immediate delivery model isn’t suited for queuing messages during idle periods.

6. Distributed Transaction Processing

Ensuring that multiple operations across distributed services succeed or fail together.

Why SQS

SQS, combined with AWS Lambda and databases, can help manage distributed transactions effectively with features like message retention and visibility timeouts.

Why not SNS, Kinesis, or Kafka

Distributed transaction management requires precise control over message processing, which SQS provides. SNS’s pub/sub model, Kinesis’s streaming approach, and Kafka’s high-throughput focus may not offer the same level of transactional integrity.

When AWS SQS Might Not Be the Best Fit

It’s also crucial to recognise when SQS might not be the optimal solution:

1. Multiple Consumer Scenarios

Limitation with SQS

It’s primarily a queue, not a pub/sub service, so one consumer typically consumes a message.

Alternative Solutions

SNS or Kafka (MSK), with their pub/sub-models, are better suited for scenarios where a message needs to reach multiple independent consumers.

2. Long-Term Data Retention

Limitation with SQS

It’s designed for immediate processing and doesn’t support long-term data retention or delayed processing beyond 15 minutes.

Alternative Solutions

Kinesis or Kafka (MSK), with their configurable retention periods, cater to use cases that require storing data for extended periods or delayed processing.

3. Replayability and Historical Data Access

Limitation with SQS

Once a message is consumed and deleted, it can’t be replayed or accessed by new consumers.

Alternative Solutions

Kinesis or Kafka (MSK) allow data replay and provide access to historical data, crucial for scenarios like event sourcing or integrating new consumers requiring historical context.

Conclusion

In conclusion, AWS SQS is a potent solution for many messaging and queuing challenges, particularly excelling in scenarios demanding reliability, order precision, and operational simplicity. However, recognising its limitations and understanding when other AWS services like SNS, Kinesis, or Kafka (MSK) might better align with your specific needs is critical to architecting robust, reliable, and optimised solutions for your unique operational landscape. By carefully weighing the strengths and capabilities of each service, you can ensure your architecture is built on a foundation of strategic, informed decisions.

Leave a Reply