Three techniques to prevent message loss
One – Publish events synchronously
the first technique you can take to prevent data loss. It is always to publish events synchronously, so you have to wait for the messaging platform to acknowledge that the message has been successfully published and persisted in the topic.
Do not publish events asynchronously. You do not want to fire and forget.
Two – Manually acknowledge messages
The second technique you need to use to prevent data loss. You need to disable auto-commit, or auto-acknowledgement and enforce that the consumers of topics or queues MUST acknowledge a message or commit the consuming offset, manually.
Auto-commit can cause message loss because the event consumer may fail to process after the commit has happened. Now in the Apache Kafka world, the consumer’s offset would be incremented.
But if you are using another messaging platform, they could delete acknowledged messages from the queue, and the message would be lost.
Three- Cluster and use persistent data storage
the third technique to prevent data loss. Which is to ensure that messaging platform is clustered and topics and queues are replicated across server nodes, and messages are persisted in case of corruption or server restarts.