--- /dev/null
+---
+_edit_last: "2"
+author: kai
+categories:
+ - demos
+ - explained
+ - java
+ - kafka
+ - spring
+ - spring-boot
+classic-editor-remember: classic-editor
+date: "2021-02-05T17:59:38+00:00"
+guid: http://juplo.de/?p=1201
+parent_post_id: null
+post_id: "1201"
+title: 'Implementing The Outbox-Pattern With Kafka - Part 0: The example'
+url: /implementing-the-outbox-pattern-with-kafka-part-0-the-example/
+
+---
+_This article is part of a Blog-Series_
+
+Based on a [very simple example-project](/implementing-the-outbox-pattern-with-kafka-part-0-the-example/)
+we will implemnt the [Outbox-Pattern](https://microservices.io/patterns/data/transactional-outbox.html) with [Kafka](https://kafka.apache.org/quickstart).
+
+- Part 0: The Example-Project
+- [Part 1: Writing In The Outbox-Table](/implementing-the-outbox-pattern-with-kafka-part-1-the-outbox-table/ "Jump to the explanation what has to be added, to enqueue messages in an outbox for successfully written transactions")
+
+## TL;DR
+
+In this part, a small example-project is introduced, that features a component, which has to inform another component upon every succsessfully completed operation.
+
+## The Plan
+
+In this mini-series I will implement the [Outbox-Pattern](https://microservices.io/patterns/data/transactional-outbox.html)
+as described on Chris Richardson's fabolous website [microservices.io](https://microservices.io/).
+
+The pattern enables you, to send a message as part of a database transaction in a reliable way, effectively turining the writing of the data
+to the database and the sending of the message into an **[atomic operation](https://en.wikipedia.org/wiki/Atomicity_(database_systems))**:
+either both operations are successful or neither.
+
+The pattern is well known and implementing it with [Kafka](https://kafka.apache.org/quickstart) looks like an easy straight forward job at first glance.
+However, there are many obstacles that easily lead to an incomplete or incorrect implementation.
+In this blog-series, we will circumnavigate these obstacles together step by step.
+
+## The Example Project
+
+To illustrate our implementation, we will use a simple example-project.
+It mimics a part of the registration process for an web application:
+a (very!) simplistic service takes registration orders for new users.
+
+- Successfull registration requests will return a 201 (Created), that carries the URI, under which the data of the newly registered user can be accessed in the `Location`-header:
+
+`echo peter | http :8080/users
+ HTTP/1.1 201
+ Content-Length: 0
+ Date: Fri, 05 Feb 2021 14:44:51 GMT
+ Location: http://localhost:8080/users/peter
+ `
+- Requests to registrate an already existing user will result in a 400 (Bad Request):
+
+`echo peter | http :8080/users
+ HTTP/1.1 400
+ Connection: close
+ Content-Length: 0
+ Date: Fri, 05 Feb 2021 14:44:53 GMT
+ `
+- Successfully registrated users can be listed:
+ `http :8080/users
+ HTTP/1.1 200
+ Content-Type: application/json;charset=UTF-8
+ Date: Fri, 05 Feb 2021 14:53:59 GMT
+ Transfer-Encoding: chunked
+ [
+ {
+ "created": "2021-02-05T10:38:32.301",
+ "loggedIn": false,
+ "username": "peter"
+ },
+ ...
+ ]
+ `
+
+## The Messaging Use-Case
+
+As our messaging use-case imagine, that there has to happen several processes after a successful registration of a new user.
+This may be the generation of an invoice, some business analytics or any other lengthy process that is best carried out asynchronously.
+Hence, we have to generate an event, that informs the responsible services about new registrations.
+
+Obviously, these events should only be generated, if the registration is completed successfully.
+The event must not be fired, if the registration is rejected, because a duplicate username.
+
+On the other hand, the publication of the event must happen reliably, because otherwise, the new might not be charged for the services, we offer...
+
+## The Transaction
+
+The users are stored in a database and the creation of a new user happens in a transaction.
+A "brilliant" colleague came up with the idea, to trigger an `IncorrectResultSizeDataAccessException` to detect duplicate usernames:
+
+`User user = new User(username);
+repository.save(user);
+// Triggers an Exception, if more than one entry is found
+repository.findByUsername(username);
+`
+
+The query for the user by its names triggers an `IncorrectResultSizeDataAccessException`, if more than one entry is found.
+The uncaught exception will mark the transaction for rollback, hence, canceling the requested registration.
+The 400-response is then generated by a corresponding `ExceptionHandler`:
+
+`@ExceptionHandler
+public ResponseEntity incorrectResultSizeDataAccessException(
+ IncorrectResultSizeDataAccessException e)
+{
+ LOG.info("User already exists!");
+ return ResponseEntity.badRequest().build();
+}
+`
+
+Please do not code this at home...
+
+But his weired implementation perfectly illustrates the requirements for our messaging use-case:
+The user is written into the database.
+But the registration is not successfully completed until the transaction is commited.
+If the transaction is rolled back, no message must be send, because no new user was registered.
+
+## Decoupling with Springs EventPublisher
+
+In the example implementation I am using an `EventPublisher` to decouple the business logic from the implementation of the messaging.
+The controller publishes an event, when a new user is registered:
+
+`publisher.publishEvent(new UserEvent(this, usernam));
+`
+
+A listener annotated with `@TransactionalEventListener` receives the events and handles the messaging:
+
+`@TransactionalEventListener
+public void onUserEvent(UserEvent event)
+{
+ // Sending the message happens here...
+}
+`
+
+In non-critical use-cases, it might be sufficient to actually send the message to Kafka right here.
+Spring ensures, that the message of the listener is only called, if the transaction completes successfully.
+But in the case of a failure this naive implementation can loose messages.
+If the application crashes, after the transaction has completed, but before the message could be send, the event would be lost.
+
+In the following blog posts, we will step by step implement a solution based on the Outbox-Pattern, that can guarantee Exactly-Once semantics for the send messages.
+
+## May The Source Be With You!
+
+The complete source code of the example-project can be cloned here:
+
+- `git clone /git/demos/spring/data-jdbc`
+- `git clone https://github.com/juplo/demos-spring-data-jdbc.git`
+
+It includes a [Setup for Docker Compose](https://github.com/juplo/demos-spring-data-jdbc/blob/master/docker-compose.yml), that can be run without compiling
+the project. And a runnable [README.sh](https://github.com/juplo/demos-spring-data-jdbc/blob/master/README.sh), that compiles and run the application and illustrates the example.
projects / website / commitdiff