Kai Moritz [Sun, 20 Jun 2021 17:51:13 +0000 (19:51 +0200)]
Switched from single-node (assign) to multi-instance (subscribe)
* TransferConsumer uses subscribe() instead of assign().
* The subscription happens during the instanciation of TransferConsumer.
* The restorage-process of the state happens in onPartitionsAssigned().
* To be able to reset all data, that belongs to a specific partition
after a rebalance, in order to avoid state errors because of handling
events several times, InMemoryTransferRepository must know, which
partition a transfer belongs to.
* To achieve this, the partitioning algorithm is made known explicitly.
* On each rebalance, a mapping from the partions to the currently assigned
instances is maintained.
* TransferController uses the explicitly known partitioning algorithm, to
look up the partion for a requested transfer and decide, if the data is
available locally.
* If not, it looks up the assigned instance in the maintained mapping and
redirects the request.
Kai Moritz [Tue, 15 Jun 2021 18:27:20 +0000 (20:27 +0200)]
Automatically rebuild the state after a crash / restart
* Turning of enable.auto.commit and setting auto.offset.reset to earliest
does the trick in this setup.
* Since no offset-commits are made, all events are re-read on startup.
* The TransferConsumer uses a special instance of TransferService, that
is initialized with a NoOpMessageService, to replay the restored
events without generating new events.
Kai Moritz [Sun, 13 Jun 2021 21:40:56 +0000 (23:40 +0200)]
TransferRepository does not need any synchronization
* Only TransferService stores data in the repository
* Since alle instances of Transfer, that are handled by TransferService
are received through a single topic, no synchronization is needed at
all in the repository.
* This is, because records, that are received from a topic are guaranteed
to be processed one after the other.
* The topic simply is the single source for processing requests and a
KafkaConsumer never handles multiple records in parallel.
* Note: This implementation is not ready to run on multiple threads or
nodes!