demos/kafka/chat
8 months agorefactor: Simplified shutdown - channel-tasks were joined multiple times
Kai Moritz [Wed, 6 Mar 2024 09:07:53 +0000 (10:07 +0100)]
refactor: Simplified shutdown - channel-tasks were joined multiple times

* `KafkaServicesApplicationRunner` does not have to join the channel-tasks.
* The channel-tasks are already joined by `ChannelTaskExecutor.join()`
  automatically, because the method is annotated with `@PreDestroy`.
* Simplified the test-configuration accordingly.

8 months agofix: The shutdown of the application was blocked
Kai Moritz [Wed, 6 Mar 2024 07:26:03 +0000 (08:26 +0100)]
fix: The shutdown of the application was blocked

* The auto-configured bean `applicationTaskExecutor` must not block, while
  it is shutting down, because otherwise, it infinitly waits for the
  completion, of the channel-tasks, which are stopped in a _later_ phase
  of the "smart" lifecycle.
* The bean is destroyed first, becaus it is is associated with the lowest
  lifecyle-phase (``Integer.MAX_VALUE``), which apparently cannot be
  overruled by `@DependsOn` (although suggested by the spring-
  documentation)
* Joining the channel-tasks was blocking infinitly, because the tasks were
  waiting for the kafka-consumers to be closed, what only happens _after_
  the joining completes.

8 months agofix: Detection of the `READY`-state for `InfoChannel` was flawed
Kai Moritz [Tue, 5 Mar 2024 16:50:47 +0000 (17:50 +0100)]
fix: Detection of the `READY`-state for `InfoChannel` was flawed

8 months agofeat: Implemented and configured health-indicator for the ``Channel``s
Kai Moritz [Tue, 5 Mar 2024 08:46:55 +0000 (09:46 +0100)]
feat: Implemented and configured health-indicator for the ``Channel``s

8 months agorefactor: Refined channel-states, introduced `ChannelState` -- ALIGN
Kai Moritz [Mon, 4 Mar 2024 13:34:34 +0000 (14:34 +0100)]
refactor: Refined channel-states, introduced `ChannelState` -- ALIGN

* Renamed attributes and method-names according to the class-renames.
* Introduced interface `Channel` and `enum ChannelState`.
* `Data` - and `InfoChannel` maintain a `ChannelState`, instead just a
  plain boolean, that only reflects the loading-state.
* The `ChannelTaskRunner` waits, until both channels entered the State
  `ChannelState.SHUTTING_DOWN`.

8 months agorefactor: Refined channel-states, introduced `ChannelState` -- MOVE
Kai Moritz [Mon, 4 Mar 2024 13:32:14 +0000 (14:32 +0100)]
refactor: Refined channel-states, introduced `ChannelState` -- MOVE

* Renamed and moved `LoadInProgressException`
** Moved exception into implementation-specific package
** Renamed exception to `ChannelNotReadyException`
* Renamed `ConsumerTaskExecutor` into `ChannelTaskExecutor`
* Renamed `ConsumerTaskRunner` into `ChannelTaskRunner`

8 months agotest: HandoverIT-POC - Added assertions for the sent/received messages
Kai Moritz [Mon, 4 Mar 2024 08:29:19 +0000 (09:29 +0100)]
test: HandoverIT-POC - Added assertions for the sent/received messages

8 months agotest: HandoverIT-POC - fix for the blocking wait for `TestListener`
Kai Moritz [Sun, 3 Mar 2024 09:20:11 +0000 (10:20 +0100)]
test: HandoverIT-POC - fix for the blocking wait for `TestListener`

* Droped the waiting for `TestListener` alltogehter.
* The waiting can be droped, because waiting for the `TestWriter`-instances
  ensures, that all messages are send (and therefore very likely received)

8 months agotest: HandoverIT-POC - working fix: using `delayElements()`
Kai Moritz [Sun, 3 Mar 2024 09:08:36 +0000 (10:08 +0100)]
test: HandoverIT-POC - working fix: using `delayElements()`

* Switched from `Flux.flatMap(Mono.delay()..)` to
  `Flux.from(..).delayElements()`FIX:delay_vs_delayElements.
* This delays eache element of the `Flux` by the same amount.
* The requests are made, when the according element of the flux is
  executed - not when the `Flux` is created, as before.

8 months agotest: HandoverIT-POC - not working fix: using `delay()`
Kai Moritz [Sat, 2 Mar 2024 17:21:44 +0000 (18:21 +0100)]
test: HandoverIT-POC - not working fix: using `delay()`

* Switched from `Mono.from(..).delayElement()` to `Mono.delay().then()`.
* This does _not_ solve the problem, that all delays are calculated and
  scheduled, when the `Flux` is created.

8 months agotest: HandoverIT-POC - FIX: `take(n)`, does end the work after n elements
Kai Moritz [Sat, 2 Mar 2024 17:21:05 +0000 (18:21 +0100)]
test: HandoverIT-POC - FIX: `take(n)`, does end the work after n elements

* `take(n)` does end the execution.
* `limitRate(n)` limits the rate of requested elements.

8 months agotest: HandoverIT-POC - Executing the Flux/Mono asynchrounusly
Kai Moritz [Fri, 1 Mar 2024 19:40:18 +0000 (20:40 +0100)]
test: HandoverIT-POC - Executing the Flux/Mono asynchrounusly

8 months agotest: HandoverIT-POC - Listener/Writer remember received/sent messages
Kai Moritz [Fri, 1 Mar 2024 19:18:00 +0000 (20:18 +0100)]
test: HandoverIT-POC - Listener/Writer remember received/sent messages

8 months agotest: HandoverIT-POC - Refactored listening into class `TestListener`
Kai Moritz [Fri, 1 Mar 2024 14:08:06 +0000 (15:08 +0100)]
test: HandoverIT-POC - Refactored listening into class `TestListener`

8 months agotest: HandoverIT-POC - Refactored `TestWriter` to use a `Flux`
Kai Moritz [Fri, 1 Mar 2024 13:53:43 +0000 (14:53 +0100)]
test: HandoverIT-POC - Refactored `TestWriter` to use a `Flux`

8 months agotest: HandoverIT-POC - Each `TestWriter` writes only to one chat-room
Kai Moritz [Fri, 1 Mar 2024 13:05:11 +0000 (14:05 +0100)]
test: HandoverIT-POC - Each `TestWriter` writes only to one chat-room

8 months agotest: HandoverIT-POC - Refactored / Cleaned up
Kai Moritz [Fri, 1 Mar 2024 13:02:01 +0000 (14:02 +0100)]
test: HandoverIT-POC - Refactored / Cleaned up

8 months agotest: HandoverIT-POC - Renamed `TestClient` to `TestWriter` -- ALIGN
Kai Moritz [Fri, 1 Mar 2024 12:48:57 +0000 (13:48 +0100)]
test: HandoverIT-POC - Renamed `TestClient` to `TestWriter` -- ALIGN

8 months agotest: HandoverIT-POC - Renamed `TestClient` to `TestWriter` -- MOVE
Kai Moritz [Fri, 1 Mar 2024 12:46:54 +0000 (13:46 +0100)]
test: HandoverIT-POC - Renamed `TestClient` to `TestWriter` -- MOVE

8 months agotest: HandoverIT-POC - Fixed illegal client-naming (one character only)
Kai Moritz [Wed, 28 Feb 2024 21:43:28 +0000 (22:43 +0100)]
test: HandoverIT-POC - Fixed illegal client-naming (one character only)

8 months agotest: HandoverIT-POC - Fixed logging of failed client-requests
Kai Moritz [Wed, 28 Feb 2024 21:39:59 +0000 (22:39 +0100)]
test: HandoverIT-POC - Fixed logging of failed client-requests

8 months agofix: Fixed `ConcurrentModificationException` when accessing a chat-room
Kai Moritz [Wed, 28 Feb 2024 10:50:11 +0000 (11:50 +0100)]
fix: Fixed `ConcurrentModificationException` when accessing a chat-room

* If a new chat-room was created, `InfoChannel` only reacted with the
  creation of the according `ChatRoomInfo`-instance.
* The creation of the accompanying `ChatRoomData`-instance through
  `DataChannel` was posponed until the new chat-room was accessed the
  first time.
* That way, `InfoChannel` did not need to know `DataChannel`, so that a
  cyclic dependency could be avoided.
* As a downside, this approach was open to a race-condition: if several
  accesses to the newly created chat-room happend in parallel, a
  `ConcurrentModificationException` was thrown, since the instance of
  `ChatRoomData` was created multiple times in parallel.
* To circumvent the locking, that would be necesarry to evade this race
  condition, the approach was refactored, so that `InfoChannel` now
  explicitly triggers the creation of the `ChatRoomData`-instance.
* To do so without introducing a cyclic dependency, the class
  `ChannelMediator` was introduced, so that `InfoChannel` and `DataChannel`
  need not to know each other.

8 months agorefactor: Introduced `ChannelMediator`
Kai Moritz [Wed, 28 Feb 2024 10:14:32 +0000 (11:14 +0100)]
refactor: Introduced `ChannelMediator`

* `InfoChannel` and `DataChannel` must not know each other directly.
* This is necessary, to prevent a cyclic dependency, that would otherwise
  be introduced, if `InfoChannel` also has to communicate with
  `DataChannel`.

8 months agotest: HandoverIT-POC - clients are stopped after some time
Kai Moritz [Wed, 28 Feb 2024 09:09:56 +0000 (10:09 +0100)]
test: HandoverIT-POC - clients are stopped after some time

8 months agotest: HandoverIT-POC - running multiple clients in parallel
Kai Moritz [Wed, 28 Feb 2024 08:59:20 +0000 (09:59 +0100)]
test: HandoverIT-POC - running multiple clients in parallel

8 months agotest: HandoverIT-POC - added a delay to the sending-loop
Kai Moritz [Tue, 27 Feb 2024 22:53:08 +0000 (23:53 +0100)]
test: HandoverIT-POC - added a delay to the sending-loop

8 months agotest: HandoverIT-POC - remodeled sending into a for-loop
Kai Moritz [Tue, 27 Feb 2024 22:42:27 +0000 (23:42 +0100)]
test: HandoverIT-POC - remodeled sending into a for-loop

8 months agotest: HandoverIT-POC - messages are written to 23 chat-rooms instead of 1
Kai Moritz [Tue, 27 Feb 2024 20:56:45 +0000 (21:56 +0100)]
test: HandoverIT-POC - messages are written to 23 chat-rooms instead of 1

8 months agotest: HandoverIT-POC - splitted up code into smaller classes -- ALIGN
Kai Moritz [Tue, 27 Feb 2024 08:39:56 +0000 (09:39 +0100)]
test: HandoverIT-POC - splitted up code into smaller classes -- ALIGN

8 months agotest: HandoverIT-POC - splitted up code into smaller classes -- MOVE
Kai Moritz [Tue, 27 Feb 2024 08:31:51 +0000 (09:31 +0100)]
test: HandoverIT-POC - splitted up code into smaller classes -- MOVE

8 months agotest: HandoverIT-POC - Moved general container-defs into base-class
Kai Moritz [Tue, 27 Feb 2024 08:24:41 +0000 (09:24 +0100)]
test: HandoverIT-POC - Moved general container-defs into base-class

8 months agotest: HandoverIT-POC - setup without static containers
Kai Moritz [Tue, 27 Feb 2024 04:14:09 +0000 (05:14 +0100)]
test: HandoverIT-POC - setup without static containers

8 months agotest: HandoverIT-POC - first working setup for the planned test
Kai Moritz [Sat, 3 Feb 2024 07:29:06 +0000 (08:29 +0100)]
test: HandoverIT-POC - first working setup for the planned test

* First working POC of the setup for the planned test.
* This version blocks endlessly, because the SSE-flux never completes.

8 months agofix: Errors during shard-publishing should not kill the instance
Kai Moritz [Mon, 26 Feb 2024 18:55:19 +0000 (19:55 +0100)]
fix: Errors during shard-publishing should not kill the instance

* `HaproxyShardingPublisherStrategy` has to transform any exception into
  a `Mono.error()`.
* `DataChannel.onPartitionsAssigned(..)` has to log and swallow errors
  during the propagation of the shard-ownership.

8 months agofix: Sensible value for sink-buffer
Kai Moritz [Sun, 25 Feb 2024 20:44:16 +0000 (21:44 +0100)]
fix: Sensible value for sink-buffer

8 months agofeat: Made polling-interval for Kafka configurable
Kai Moritz [Sun, 25 Feb 2024 19:50:21 +0000 (20:50 +0100)]
feat: Made polling-interval for Kafka configurable

8 months agofix: The number of chards is the number of partitions of the `data_channel`
Kai Moritz [Sat, 24 Feb 2024 11:56:24 +0000 (12:56 +0100)]
fix: The number of chards is the number of partitions of the `data_channel`

8 months agofix: Disabled mongodb-autoconfig, if mongodb is not used
Kai Moritz [Fri, 23 Feb 2024 10:39:26 +0000 (11:39 +0100)]
fix: Disabled mongodb-autoconfig, if mongodb is not used

9 months agofeat: Switched to `spring-boot-starter-data-mongodb-reactive`
Kai Moritz [Sat, 3 Feb 2024 23:24:07 +0000 (00:24 +0100)]
feat: Switched to `spring-boot-starter-data-mongodb-reactive`

9 months agofeat: Added logging for io.projectreactor to `MongoDbStorageStrategy`
Kai Moritz [Tue, 20 Feb 2024 16:35:14 +0000 (17:35 +0100)]
feat: Added logging for io.projectreactor to `MongoDbStorageStrategy`

9 months agorefactor: DRY for logging-category from io.projectreactor
Kai Moritz [Tue, 20 Feb 2024 16:32:54 +0000 (17:32 +0100)]
refactor: DRY for logging-category from io.projectreactor

9 months agofeat: Added counting of restored instances
Kai Moritz [Tue, 20 Feb 2024 15:14:22 +0000 (16:14 +0100)]
feat: Added counting of restored instances

9 months agotest: `StorageStrategy`-IT are restoring instead of recreating
Kai Moritz [Thu, 22 Feb 2024 15:46:27 +0000 (16:46 +0100)]
test: `StorageStrategy`-IT are restoring instead of recreating

9 months agofix: GREEN - Fixed the restore-mechanism
Kai Moritz [Thu, 22 Feb 2024 15:03:08 +0000 (16:03 +0100)]
fix: GREEN - Fixed the restore-mechanism

* The code of a reactive flow _must not_ call blocking functions.
* In order to solve this, the restore-process is triggered explicitly
  after the creation of the classes.

9 months agorefactor: RED - Refined success/error-handling for restore-operations
Kai Moritz [Tue, 20 Feb 2024 15:12:00 +0000 (16:12 +0100)]
refactor: RED - Refined success/error-handling for restore-operations

* This innocent little change discloses a severe missconception in the
  implementation of the storage strategies.
* The call to `Mono.block()`, though not really changing the behaviour
  during the restore-process, triggers a sanity-check from
  io.projectractor.

9 months agotest: Simplified the integration-tests for `StorageStrategy`
Kai Moritz [Thu, 22 Feb 2024 14:35:39 +0000 (15:35 +0100)]
test: Simplified the integration-tests for `StorageStrategy`

9 months agotest: Simplified `InMemoryWithMongoDbStorageIT`
Kai Moritz [Thu, 22 Feb 2024 13:49:46 +0000 (14:49 +0100)]
test: Simplified `InMemoryWithMongoDbStorageIT`

* Switched to `@ServiceConnection` instead of hand-coded initializer.

9 months agotest: Added IT for `ChatRoomRepository` and `MessageRepository`
Kai Moritz [Wed, 21 Feb 2024 17:30:01 +0000 (18:30 +0100)]
test: Added IT for `ChatRoomRepository` and `MessageRepository`

9 months agotest: Simplified & Unified the Kafka-tests
Kai Moritz [Thu, 22 Feb 2024 13:39:30 +0000 (14:39 +0100)]
test: Simplified & Unified the Kafka-tests

9 months agotest: Simplified `ChatHomeServiceTest`
Kai Moritz [Thu, 22 Feb 2024 13:24:28 +0000 (14:24 +0100)]
test: Simplified `ChatHomeServiceTest`

* Removed the unnecessary explicit instantiation of the `ObjectMapper`.

9 months agotest: Simplified & unified test-setup for ``ChatHomeServiceTest``s
Kai Moritz [Thu, 22 Feb 2024 12:10:09 +0000 (13:10 +0100)]
test: Simplified & unified test-setup for ``ChatHomeServiceTest``s

* Using existing configuration-classes instead of hand-coded configuartion.
* Aligned the `KafkaChatHomeServiceTest` with the `in-memory`-tests.

9 months agorefactor: Cleaned up code
Kai Moritz [Tue, 20 Feb 2024 12:05:21 +0000 (13:05 +0100)]
refactor: Cleaned up code

9 months agofeat: Introduced counting of stored instances in `StorageStrategy`
Kai Moritz [Tue, 20 Feb 2024 12:02:09 +0000 (13:02 +0100)]
feat: Introduced counting of stored instances in `StorageStrategy`

9 months agofeat: Introduced config-parameters for the `io.projectreactor`-logging
Kai Moritz [Tue, 20 Feb 2024 11:48:40 +0000 (12:48 +0100)]
feat: Introduced config-parameters for the `io.projectreactor`-logging

9 months agorefactor: Set logging-level for io.projectreactor to `DEBUG`
Kai Moritz [Tue, 20 Feb 2024 10:43:00 +0000 (11:43 +0100)]
refactor: Set logging-level for io.projectreactor to `DEBUG`

9 months agorefactor: Refined stream-definition in `StorageStrategy#write`
Kai Moritz [Tue, 20 Feb 2024 10:28:22 +0000 (11:28 +0100)]
refactor: Refined stream-definition in `StorageStrategy#write`

* Changed the stream-definition to a more natural order.
* As a result, the stored `ChatRoomInfo`-instances do not have to be
  handed clumsily to the following stream.

9 months agorefactor: Refined return-type of `StorageStrategy#write`
Kai Moritz [Tue, 20 Feb 2024 10:14:43 +0000 (11:14 +0100)]
refactor: Refined return-type of `StorageStrategy#write`

9 months agorefactor: One stream -> using `flatMap` instead of an inner `subscribe`
Kai Moritz [Tue, 20 Feb 2024 07:50:56 +0000 (08:50 +0100)]
refactor: One stream -> using `flatMap` instead of an inner `subscribe`

9 months agorefactor: Moved succes/error-logging to outer stream-definitions
Kai Moritz [Tue, 20 Feb 2024 07:23:13 +0000 (08:23 +0100)]
refactor: Moved succes/error-logging to outer stream-definitions

9 months agorefactor: Simplified `StorageStrategy`
Kai Moritz [Tue, 20 Feb 2024 06:47:22 +0000 (07:47 +0100)]
refactor: Simplified `StorageStrategy`

* Reconfigurable success/error-logging was introduced for
  `NoStorageStorageStrategy`.
* But as it turns out, this strategy can simply apply its logging in the
  overwritten method, that disables the whole storing-logic.
* Hence, the interface was greatly simplified again, by removing this
  ununsed mechanism.

9 months agorefactor: Moved extracted the `subscribe()`-call from `StorageStrategy`
Kai Moritz [Mon, 19 Feb 2024 14:01:58 +0000 (15:01 +0100)]
refactor: Moved extracted the `subscribe()`-call from `StorageStrategy`

* The subscription does no more happen inside the implementations of the
  interface `StorageStrategy`.
* Instead, the methods, that are defined in `StorageStrategy` return the
  created `Flux`.
* The call to `subscribe()` happens in the code, that uses this methods.
* This faciliates feature refinements concerning the asynchronous handling
  of success- and error-cases and so forth.

9 months agorefactor: Extracted subscription into a separate method-call
Kai Moritz [Sat, 3 Feb 2024 23:37:23 +0000 (00:37 +0100)]
refactor: Extracted subscription into a separate method-call

* This is in preparation to a planed refinement.
* The refinment will move the `subscribe()`-call out of the implementation
  of the `StorageStrategy`.

9 months agotest: Added missing JSR-310 module to not Spring-Boot-based tests
Kai Moritz [Mon, 19 Feb 2024 13:50:34 +0000 (14:50 +0100)]
test: Added missing JSR-310 module to not Spring-Boot-based tests

9 months agofeat: Added log message for disabled storage
Kai Moritz [Mon, 19 Feb 2024 13:31:21 +0000 (14:31 +0100)]
feat: Added log message for disabled storage

9 months agofix: Without `@DirtiesContext` the app is not teared down correctly
Kai Moritz [Sun, 18 Feb 2024 19:12:19 +0000 (20:12 +0100)]
fix: Without `@DirtiesContext` the app is not teared down correctly

* `StorageStrategy` depends on Spring to call the method
  `ChatBackendApplication#onExit()`, that is annotated with `@PreDestroy`.
* If this method is not called, the strategy is not applied, which leads
  to errors in the integration-tests.
* This happens, if all tests are run through Maven, because Spring
  recycles the application-context and only tears down the context after
  all tests have run.
* The addition of `@DirtiesContext` on those tests forces Spring to tear
  down the application completely after each so annotated test.
* Hence, the method is called as expected, which fixes the described
  errors.

9 months agotest: Introduced integration-tests for `NoStorageStorageStrategy`
Kai Moritz [Mon, 19 Feb 2024 10:05:17 +0000 (11:05 +0100)]
test: Introduced integration-tests for `NoStorageStorageStrategy`
--
ALIGN

9 months agotest: Introduced integration-tests for `NoStorageStorageStrategy`
Kai Moritz [Mon, 19 Feb 2024 10:01:13 +0000 (11:01 +0100)]
test: Introduced integration-tests for `NoStorageStorageStrategy`
--
COPY

9 months agorefactor: Extracted annonymous class into `NoStorageStorageStrategy`
Kai Moritz [Mon, 19 Feb 2024 12:07:35 +0000 (13:07 +0100)]
refactor: Extracted annonymous class into `NoStorageStorageStrategy`
--
ALIGN

9 months agorefactor: Extracted annonymous class into `NoStorageStorageStrategy`
Kai Moritz [Mon, 19 Feb 2024 12:05:51 +0000 (13:05 +0100)]
refactor: Extracted annonymous class into `NoStorageStorageStrategy`
--
COPY

9 months agorefactor: Added success- and failure-callbacks for `ChatHomeService`
Kai Moritz [Sun, 18 Feb 2024 20:46:56 +0000 (21:46 +0100)]
refactor: Added success- and failure-callbacks for `ChatHomeService`

9 months agorefactor: Added success- and failure-callbacks to `StorageStrategy`
Kai Moritz [Sat, 3 Feb 2024 21:44:02 +0000 (22:44 +0100)]
refactor: Added success- and failure-callbacks to `StorageStrategy`

9 months agorefactor: `KafkaServicesApplicationRunner` aufgeräumt
Kai Moritz [Sat, 3 Feb 2024 19:43:29 +0000 (20:43 +0100)]
refactor: `KafkaServicesApplicationRunner` aufgeräumt

9 months agofix: Config-ITs do not fail, if run multiple times
Kai Moritz [Sat, 3 Feb 2024 11:08:15 +0000 (12:08 +0100)]
fix: Config-ITs do not fail, if run multiple times

9 months agofix: Removed manual version-management for testcontainers.org
Kai Moritz [Sat, 3 Feb 2024 07:35:38 +0000 (08:35 +0100)]
fix: Removed manual version-management for testcontainers.org

9 months agofeat: Upgraded Spring Boot 3.1.3 -> 3.2.2
Kai Moritz [Sat, 3 Feb 2024 07:32:40 +0000 (08:32 +0100)]
feat: Upgraded Spring Boot 3.1.3 -> 3.2.2

9 months agofix: `getChatRoomInfo()` thrwos `LoadInProgressException` when loading
Kai Moritz [Sat, 3 Feb 2024 16:25:45 +0000 (17:25 +0100)]
fix: `getChatRoomInfo()` thrwos `LoadInProgressException` when loading

- The method `InfoChannel.getChatRoomInfo(UUID)` has to check, if loading
  is in process.
- Otherwise, an existing chat-room might erronously not be found, if it is
  requested, while `InfoChannel` is loading, because it is not yet loaded.

9 months agorefactor: compute `loadInProgress` on offset-change
Kai Moritz [Sat, 3 Feb 2024 14:32:25 +0000 (15:32 +0100)]
refactor: compute `loadInProgress` on offset-change

9 months agorefactor: separated message- and record- (aka offset-) handling
Kai Moritz [Sat, 3 Feb 2024 14:20:25 +0000 (15:20 +0100)]
refactor: separated message- and record- (aka offset-) handling

9 months agorefactor: Renamed method in `InfoChannel` according to conventions
Kai Moritz [Sat, 3 Feb 2024 14:13:09 +0000 (15:13 +0100)]
refactor: Renamed method in `InfoChannel` according to conventions

9 months agorefactor: Made only locally used method in `KafkaChatHomeService` private
Kai Moritz [Sat, 3 Feb 2024 14:11:50 +0000 (15:11 +0100)]
refactor: Made only locally used method in `KafkaChatHomeService` private

9 months agofeat: The position of the last seen messages is stored on a revoke
Kai Moritz [Sun, 24 Sep 2023 19:42:22 +0000 (21:42 +0200)]
feat: The position of the last seen messages is stored on a revoke

9 months agofix: The actual position has to be requested from the consumer
Kai Moritz [Sun, 24 Sep 2023 19:39:01 +0000 (21:39 +0200)]
fix: The actual position has to be requested from the consumer

* If the last seen offset and the current offset differ, although the
  partition did not contain any messages between this offsets, the loading
  process got stuck, because the position never advanced.
* Therefore, the actual position, that is compared against the read
  end-offset, has to be requested from the consumer.

9 months agorefactor: Added logging of loaded messages
Kai Moritz [Sun, 24 Sep 2023 19:38:46 +0000 (21:38 +0200)]
refactor: Added logging of loaded messages

9 months agorefactor: Refined success/error-handling when publishing shard-ownership
Kai Moritz [Tue, 20 Feb 2024 09:26:25 +0000 (10:26 +0100)]
refactor: Refined success/error-handling when publishing shard-ownership

9 months agofeat: Implemented `HaproxyShardingPublisherStrategy`
Kai Moritz [Sun, 17 Sep 2023 09:31:22 +0000 (11:31 +0200)]
feat: Implemented `HaproxyShardingPublisherStrategy`

* Implemented a first simple `ShardingPublisherStrategy`, that uses the
 https://www.haproxy.com/documentation/haproxy-runtime-api/[HAProxy Runntime API]
 to publish changed ownerships.
* Added configuration-properties `kafka.haproxyRuntimeApi` and
  `kafka.haproxyMap` to configure the strategy.

9 months agofeat: Introduced interface `ShardingPublisherStrategy`
Kai Moritz [Tue, 20 Feb 2024 08:53:10 +0000 (09:53 +0100)]
feat: Introduced interface `ShardingPublisherStrategy`

* The interface is used by `DataChannel` to publish the changed ownership
  each time, a new partition is assigned to the consumer-group.
* Added a dummy-implementation in `KafkaServicesConfiguration`.

9 months agofeat: Introduced a configurable instance-id
Kai Moritz [Tue, 20 Feb 2024 08:48:03 +0000 (09:48 +0100)]
feat: Introduced a configurable instance-id

9 months agofix: `ConsumerTaskRunner` waits until the data-loading is finished
Kai Moritz [Fri, 22 Sep 2023 16:20:31 +0000 (18:20 +0200)]
fix: `ConsumerTaskRunner` waits until the data-loading is finished

9 months agofeat: Introduced events that are send, if a shard is assigned/revoked
Kai Moritz [Sat, 16 Sep 2023 19:40:45 +0000 (21:40 +0200)]
feat: Introduced events that are send, if a shard is assigned/revoked

* In order to redirect requests to the appropriate instances, each
  instance must know the mapping from  shard-IDs to instance-URIs.
** For the static `in-memory`-implementation `ShardedChatHomeService`,
   this is a static mapping, that can be configured and read on start-up.
** For the dynamic `KafkaChatHomeService`, this mapping has to be
   propageted to all instances each time the partition-assignment of the
   consumer-group changes.
* Changes for `ShardedChatHomeService`
** Introduced `ChatBackendProperties.shardOwners` (of type `URI[]`).
** Each instance reads the static mapping from shard-ID to instance-URI
   on start-up.
* Changes for `KafkaChatHomeService`
** Introduced `KafkaServicesProperties.instanceUri` (of type `URI`).
** Each instance reads its URI on start-up.
** `DataChannel` sends an event for each assigned/removed partition,
   when the partition-assignment of the consumer-group changes.
** These events propagete the changed mapping from partition-ID to
   instance-URI.
** `InfoChannel` receives these events and updates the dynamic mapping
   from partition-ID (aka:shard-ID) to instance-URI.
* The shards, that are owned by an instance, can be queried via `/shards`.

9 months agofix: `metadata` is never `null`
Kai Moritz [Sun, 28 Jan 2024 17:50:33 +0000 (18:50 +0100)]
fix: `metadata` is never `null`

9 months agorefactor: DRY for computation of new `ChatRoomData` instances
Kai Moritz [Fri, 15 Sep 2023 19:35:17 +0000 (21:35 +0200)]
refactor: DRY for computation of new `ChatRoomData` instances

9 months agofix: GREEN - `DataChannel` creates entries for existent chat-rooms
Kai Moritz [Fri, 15 Sep 2023 18:56:14 +0000 (20:56 +0200)]
fix: GREEN - `DataChannel` creates entries for existent chat-rooms

9 months agotest: RED - Putting a message in a newly created chat-room
Kai Moritz [Fri, 15 Sep 2023 18:04:01 +0000 (20:04 +0200)]
test: RED - Putting a message in a newly created chat-room

9 months agorefactor: Splitted test in `AbstractConfigurationIT` into smaler pieces
Kai Moritz [Fri, 15 Sep 2023 15:29:05 +0000 (17:29 +0200)]
refactor: Splitted test in `AbstractConfigurationIT` into smaler pieces

9 months agorefactor: Moved common test-code into helper-class `KafkaTestUtils` - ALIGN
Kai Moritz [Fri, 15 Sep 2023 15:08:33 +0000 (17:08 +0200)]
refactor: Moved common test-code into helper-class `KafkaTestUtils` - ALIGN

9 months agorefactor: Moved common test-code into helper-class `KafkaTestUtils` - COPY
Kai Moritz [Fri, 15 Sep 2023 08:57:55 +0000 (10:57 +0200)]
refactor: Moved common test-code into helper-class `KafkaTestUtils` - COPY

9 months agorefactor: Extracted interface `WorkAssignor` into separate file
Kai Moritz [Fri, 15 Sep 2023 08:48:28 +0000 (10:48 +0200)]
refactor: Extracted interface `WorkAssignor` into separate file

9 months agorefactor: Separated channels for data and info -- Refactored/aligned code
Kai Moritz [Tue, 12 Sep 2023 21:33:59 +0000 (23:33 +0200)]
refactor: Separated channels for data and info -- Refactored/aligned code

* Split `ChatRoomChannel` into `InfoChannel` and `DataChannel`
** `DataChannel` manages only data for chat-messages
** `InfoChannel` manages all info-data (at the moment only
   `EventChatRoomCreated`)
* Aligned test-setup for kafka-related tests

9 months agorefactor: Separated channels for data and info -- Moved/copied classes
Kai Moritz [Mon, 11 Sep 2023 16:36:05 +0000 (18:36 +0200)]
refactor: Separated channels for data and info -- Moved/copied classes

* Split `ChatRoomChannel` into `InfoChannel` and `DataChannel`
** `DataChannel` manages only data for chat-messages
** `InfoChannel` manages all info-data (at the moment only
   `EventChatRoomCreated`)
* Aligned test-setup for kafka-related tests

9 months agorefactor: Introduced `ConsumerTaskExecutor` -- Aligned code
Kai Moritz [Thu, 14 Sep 2023 21:42:59 +0000 (23:42 +0200)]
refactor: Introduced `ConsumerTaskExecutor` -- Aligned code

* Moved startup-logic in a separate class (`ConsumerTaskExecutor`), so
  that it is reusable more easily in test scenarios.
* `KafkaServicesApplicationRunner` is instanciated via `@Component`, so
  that it is _not_ instanciated automatically, if the configuration is
  triggered explicitly via `KafkaServicesConfiguration` in test scenarios.