summary |
shortlog | log |
commit |
commitdiff |
tree
first ⋅ prev ⋅ next
Kai Moritz [Sun, 31 Jan 2021 21:24:16 +0000 (22:24 +0100)]
Send messages are deleted individually after a crash
* If all messages with a sequence-number lower than or equal than the
sequence number of a message already send are deleted, this can only
be done for the lowest sequence-number seen accross all partitions
without violating the at-least-once semantics.
* Deleting seen messages individually by specifying their id (aka
sequence-number) explicitly in the delete-command allows to delete
_all_ messages from the outbox, that have been seen on the topic,
hence, achieving exactly-once-semantics even in the case of an
unclean shutdown (aka crash) of the OutboxProducer
Kai Moritz [Sat, 30 Jan 2021 21:39:41 +0000 (22:39 +0100)]
The OutboxProducer restores the sequence-number from the wirtten topic
* The OutboxProducer consumes the written messages on start-up, extracts the
send sequence-numbers and recalculates the seen watermarks and the maximal
sequence-number, for at-least-once semantics.
* This implementation will most probably resend some messages after a crash,
but it will never skip any unsend messages.
* Messages may be resend despite the recalculated watermarks, because only
messages with a sequence number lower than the lowest watermark can be
discarded safely from the outbox, if message-loss is not acceptable.
* Deactivated the integration-test for the loading of the context, because
it cannot work without an available Kafka-Cluster.
Kai Moritz [Sat, 6 Feb 2021 10:14:15 +0000 (11:14 +0100)]
Switched submodule-path to publicly available sources
Kai Moritz [Sun, 31 Jan 2021 15:55:58 +0000 (16:55 +0100)]
Enabled logging-level DEBUG for the "outbox"-service
Kai Moritz [Sun, 31 Jan 2021 15:51:59 +0000 (16:51 +0100)]
Moved the name of the header for the sequence-number into a static field
Kai Moritz [Sat, 30 Jan 2021 15:45:34 +0000 (16:45 +0100)]
Enabled idempotence for the producer
Kai Moritz [Sat, 30 Jan 2021 15:37:31 +0000 (16:37 +0100)]
Replaced handwritten property-names by references
Kai Moritz [Sat, 30 Jan 2021 14:06:54 +0000 (15:06 +0100)]
Old entries are removed from the outbox-table in batches
Kai Moritz [Sat, 30 Jan 2021 13:43:04 +0000 (14:43 +0100)]
Improved example execution in README.sh
* The setup for Docker Compose uses a specific version of Kafka
* README.sh uses Docker to execute kafka-commands
* No error-message is issued, if the topic already exists
Kai Moritz [Sun, 1 Nov 2020 20:58:07 +0000 (21:58 +0100)]
Made the polling-interval configurable
Kai Moritz [Sun, 1 Nov 2020 16:41:06 +0000 (17:41 +0100)]
Added an executable README.sh, that demonstrates the example
Kai Moritz [Sun, 1 Nov 2020 11:58:20 +0000 (12:58 +0100)]
Moved postage of messages into a reusable standalone implementation
* Renamed maven-module outbox into delivery
* Renamed app polling-outbox into outbox-polling-delivery
* Added new artifact outbox-postage as maven-module postage
* Added fully qualified names for the docker-images
* TODO: Move flyway-scriptes for outbox into module postage
Kai Moritz [Fri, 30 Oct 2020 10:57:01 +0000 (11:57 +0100)]
Added Google Guava for byte-conversions
Kai Moritz [Wed, 28 Oct 2020 17:31:56 +0000 (18:31 +0100)]
Added missing annotation @EnableScheduling
* The application did not do anything, because scheduling was not activated
* Adding @EnableScheduling also keeps the application alive
* Therefore, waiting on the CountDownLatch is not needed any more
Kai Moritz [Sat, 31 Oct 2020 16:03:24 +0000 (17:03 +0100)]
Added a simple implementation of the polling outbox pattern
Kai Moritz [Sat, 31 Oct 2020 15:59:17 +0000 (16:59 +0100)]
Added data-jdbc as git submodule jdbc