|
| 1 | += Introducing Transaction Idempotency into Fineract |
| 2 | + |
| 3 | +For banks, Apache Fineract is introducing transaction idempotency: |
| 4 | + |
| 5 | +Transaction idempotency is the scenario in DELETE, PATCH, POST, and PUT API requests when if the caller sends retry of a request multiple times, the result is equivalent to single request modification. If there is no transaction idempotency, the user retries the request N times, the result will end up having N resources with N different URIs created on server. |
| 6 | + |
| 7 | +In simple terms transaction idempotency is required in case of: |
| 8 | + |
| 9 | +* Retry of write operation in case of timeout due to: |
| 10 | + ** VM Crash |
| 11 | + ** JVM Crash |
| 12 | + ** Thread exception |
| 13 | + ** APIs Crash |
| 14 | +* Duplicate writes |
| 15 | +* Lock expired, failed, ex: db connection failed. |
| 16 | +
|
| 17 | +
|
| 18 | += Background |
| 19 | + |
| 20 | +Fineract is deoployed in many organisations where high-availability and fault |
| 21 | +tolerance is essential. In case of retry operations, even though a former |
| 22 | +executions has already completed, in result 2 executions of the same |
| 23 | +operations are generated. |
| 24 | + |
| 25 | +Imagine you do a Loan repayment for $1000 via the API and the API call crashes. |
| 26 | +The client retries the Loan repayment with the $1000 and results in a success. |
| 27 | +Now looking at the Loan balance, it’s been decreased by $2000 and 2 repayment |
| 28 | +transactions have been created; which is the incorrect behavior. |
| 29 | + |
| 30 | +The system needs to be able to detect these scenarios and prevent executing |
| 31 | +the same request multiple times. |
| 32 | + |
| 33 | +Also, the emitted external events Fineract is sending should provide a similar |
| 34 | +behavior to the above to detect events which are for the same operation. |
| 35 | + |
| 36 | +Previously, Fineract's POST/PUT API operation were not idempotent. Some clients |
| 37 | +made this a requirement and implemented idempotency but this was not available |
| 38 | +in the public branch. Mifos considered this a functional gap and something |
| 39 | +Fineract should support. |
| 40 | + |
| 41 | += Features |
| 42 | + |
| 43 | +Customers and developers can now: |
| 44 | + |
| 45 | +* Command execution subsystem: Fineract operates with a partial CQRS model. |
| 46 | + All the write APIs are creating Command objects to be executed using |
| 47 | + Command Handlers instead of the APIs directly invoking service classes for an |
| 48 | + exact functionality. |
| 49 | + The Commands are also stored in the database for further auditing purposes. |
| 50 | +* External events: The events which Fineract is emitting onto external message |
| 51 | + channels upon business operations. |
| 52 | + |
| 53 | + |
| 54 | += How to enable API idempotency as a functionality |
| 55 | + |
| 56 | +== API idempotency |
| 57 | + |
| 58 | +APIs which are resubmitted due to fault-tolerance purposes; read: retried; |
| 59 | +are not executed multiple times. One single execution is allowed out of |
| 60 | +these API calls. |
| 61 | + |
| 62 | +== API idempotency key supplied |
| 63 | + |
| 64 | +The header name that should be used for passing idempotency keys in an API is |
| 65 | +called **Idempotency-Key**. |
| 66 | + |
| 67 | +Whether or not an API request correspond to the same operation is decided on the |
| 68 | +client level. The client should have the ability to supply an idempotency key |
| 69 | +within the API requests as an HTTP header. |
| 70 | + |
| 71 | + |
| 72 | +```Idempotency-Key: 8e03978e-40d5-43e8-bc93-6894a57f9324``` |
| 73 | + |
| 74 | +== External event idempotency |
| 75 | + |
| 76 | +The external events should provide an idempotency key as an extra information so |
| 77 | +that consumers can keep up with the at-least-once delivery guarantee and decide |
| 78 | +whether an event has been read already. |
| 79 | + |
| 80 | +The reliable event framework already defines an ```idempotencyKey``` attribute |
| 81 | +in the message wrapper. |
| 82 | + |
| 83 | +== Command execution recap |
| 84 | + |
| 85 | +In Fineract, all non-GET APIs are executed via the command execution framework. |
| 86 | +Instead of directly calling a service level method from the API level in the code, |
| 87 | +the API request is rather put onto a command execution queue. The execution is |
| 88 | +synchronous and submitted commands can be handled by different handlers. |
| 89 | + |
| 90 | +There’s a centralized place where each and every non-read operation goes through. |
| 91 | +Fineract already utilizes this aspect by doing some audit logging on the commands |
| 92 | +that are executed, the ability do to 4-eyed operation checks (maker-checker). |
| 93 | + |
| 94 | +Therefore the executed commands are stored in the database within the |
| 95 | +```m_portfolio_command_source``` table (later referred to as the |
| 96 | +command table). |
| 97 | + |
| 98 | += Idempotency code examples |
| 99 | + |
| 100 | +== API idempotency key supplied |
| 101 | + |
| 102 | +``` |
| 103 | +``` |
| 104 | + |
| 105 | +== External event idempotency |
| 106 | + |
| 107 | +``` |
| 108 | +``` |
0 commit comments