Kai Moritz [Sun, 31 Jan 2021 21:24:16 +0000 (22:24 +0100)]
Sent messages are deleted individually after a crash
* If all messages with a sequence-number lower than or equal than the
sequence number of a message already send are deleted, this can only
be done for the lowest sequence-number seen accross all partitions
without violating the at-least-once semantics.
* Deleting seen messages individually by specifying their id (aka
sequence-number) explicitly in the delete-command allows to delete
_all_ messages from the outbox, that have been seen on the topic,
hence, achieving exactly-once-semantics even in the case of an
unclean shutdown (aka crash) of the OutboxProducer
Kai Moritz [Tue, 20 Apr 2021 20:50:26 +0000 (22:50 +0200)]
WIP: Using assign instead of subscribe
This clearifies, that the current implementation does not scale.
Subscribe suggests, that one can run multiple instances of the service
concurrently. But doing so would break this version of the implementation.
Kai Moritz [Sat, 30 Jan 2021 21:39:41 +0000 (22:39 +0100)]
The OutboxProducer restores the sequence-number from the wirtten topic
* The OutboxProducer consumes the written messages on start-up, extracts the
send sequence-numbers and recalculates the seen watermarks and the maximal
sequence-number, for at-least-once semantics.
* This implementation will most probably resend some messages after a crash,
but it will never skip any unsend messages.
* Messages may be resend despite the recalculated watermarks, because only
messages with a sequence number lower than the lowest watermark can be
discarded safely from the outbox, if message-loss is not acceptable.
* Deactivated the integration-test for the loading of the context, because
it cannot work without an available Kafka-Cluster.
Kai Moritz [Sun, 1 Nov 2020 11:58:20 +0000 (12:58 +0100)]
Moved postage of messages into a reusable standalone implementation
* Renamed maven-module outbox into delivery
* Renamed app polling-outbox into outbox-polling-delivery
* Added new artifact outbox-postage as maven-module postage
* Added fully qualified names for the docker-images
* Adapted docker-compose.yml and README.sh
* TODO: Move flyway-scriptes for outbox into module postage
Kai Moritz [Wed, 28 Oct 2020 17:31:56 +0000 (18:31 +0100)]
FIX: Added missing annotation @EnableScheduling
* The application did not do anything, because scheduling was not activated
* Adding @EnableScheduling also keeps the application alive
* Therefore, waiting on the CountDownLatch is not needed any more