Designing Idempotent Payment APIs
In the world of payments, reliability and consistency are crucial. Customers expect their transactions to be processed accurately and efficiently, without any errors or duplicates. However, in a distributed system, failures can occur, and the same request can be sent multiple times.
For example, consider a payment service API that transfers money from one user to another. No matter where the failure occurs, the system needs to ensure that one request is processed exactly once; leading us to build idempotent APIs.
Idempotent APIs
An idempotent API is an API that can be called multiple times, but the result will always be the same. If a request is made twice or more, the API will only perform the operation once, ensuring that there are no duplicates or errors.
However, designing an idempotent API is not a trivial task. There are many things to consider, such as handling errors, passing idempotency keys, and maintaining consistency.
Core idea implementation
The core idea behind item potency keys is straightforward. Before making an API call, the client talks to the server to generate a random ID, which serves as the idempotency key.
The client then passes this key in all future requests to the server. The server stores the key and the purpose of the request in a backend database.
When the server receives a request, it extracts the idempotency key and checks if it has already processed the request. If it has, the server ignores the request. If it hasn't, the server processes the request and deletes the idempotency key.
This gives us the much-needed exactly-once processing.
Handling Errors
When designing an idempotent API, we need to consider what to do in case of failure. Depending on the application usecase and the transaction state, we can choose to
ignore the failure,
pass the error to the user, or
retry the request.
Retrying the request is often preferred because it provides a better user experience. However, we need to be careful when retrying requests, as we could end up processing the same payment multiple times.
Passing the keys
There are several ways to pass the idempotency key to the server, such as using request headers or query parameters. Stripe, a popular payment processing company, requires clients to pass idempotency keys in request headers named Idempotency-Key.
Database decisions
It is recommended to use a separate database or table to store idempotency keys to ensure that the server can quickly validate the processing status and load isolation.
Conclusion
In conclusion, designing an idempotent API for payments is crucial for ensuring reliability and consistency. By using idempotency keys, we can avoid processing the same payment multiple times.
This approach can significantly improve the reliability and user experience of your API, making it a must-have for any critical API that needs to be executed exactly once, no matter what.
Here's the video of me explaining this in-depth 👇
Thank you so much for reading this edition of the newsletter 🔮 If you found it interesting, you will also love my
Members-only videos are more hands-on and cover some really interesting and essential topics that every engineer should know. So, support me by becoming a member of my YouTube channel.
ps: the references for this edition of the newsletter can be found in the description of the video attached (if any).
How do you build operation of generating and serving transaction key as an idempotent operation?
`The server stores the key and the purpose of the request in a backend database. `
If the purpose is to make the system idempotent and the key has to be deleted after transaction is processed (either success or failure), it's better to store this unique/random ID in cache.
As long as the key is present in cache, we can check for duplicate transaction and straightaway invalidate the request.