Kai Moritz [Tue, 27 Feb 2024 10:48:49 +0000 (11:48 +0100)]
WIP:test: HandoverIT-POC - splitted up code into smaller classes -- ALIGN
Kai Moritz [Tue, 27 Feb 2024 10:48:26 +0000 (11:48 +0100)]
WIP:test: HandoverIT-POC - splitted up code into smaller classes -- ALIGN
Kai Moritz [Tue, 27 Feb 2024 10:42:17 +0000 (11:42 +0100)]
WIP:test: HandoverIT-POC - splitted up code into smaller classes -- ALIGN
Kai Moritz [Tue, 27 Feb 2024 10:40:59 +0000 (11:40 +0100)]
WIP:test: HandoverIT-POC - splitted up code into smaller classes -- ALIGN
Kai Moritz [Tue, 27 Feb 2024 10:40:01 +0000 (11:40 +0100)]
WIP:test: HandoverIT-POC - splitted up code into smaller classes -- ALIGN
Kai Moritz [Tue, 27 Feb 2024 10:28:47 +0000 (11:28 +0100)]
WIP:test: HandoverIT-POC - splitted up code into smaller classes -- ALIGN
Kai Moritz [Tue, 27 Feb 2024 10:24:52 +0000 (11:24 +0100)]
WIP:test: HandoverIT-POC - splitted up code into smaller classes -- ALIGN
Kai Moritz [Tue, 27 Feb 2024 10:13:19 +0000 (11:13 +0100)]
WIP:test: HandoverIT-POC - splitted up code into smaller classes -- ALIGN
Kai Moritz [Tue, 27 Feb 2024 09:13:47 +0000 (10:13 +0100)]
WIP:test: HandoverIT-POC - splitted up code into smaller classes -- ALIGN
Kai Moritz [Tue, 27 Feb 2024 08:55:06 +0000 (09:55 +0100)]
WIP:test: HandoverIT-POC - splitted up code into smaller classes -- ALIGN
Kai Moritz [Tue, 27 Feb 2024 08:39:56 +0000 (09:39 +0100)]
test: HandoverIT-POC - splitted up code into smaller classes -- ALIGN
Kai Moritz [Tue, 27 Feb 2024 10:41:46 +0000 (11:41 +0100)]
WIP:test: HandoverIT-POC - splitted up code into smaller classes -- MOVE
Kai Moritz [Tue, 27 Feb 2024 10:37:57 +0000 (11:37 +0100)]
WIP:test: HandoverIT-POC - splitted up code into smaller classes -- MOVE
Kai Moritz [Tue, 27 Feb 2024 08:56:39 +0000 (09:56 +0100)]
WIP:test: HandoverIT-POC - splitted up code into smaller classes -- MOVE
Kai Moritz [Tue, 27 Feb 2024 08:41:11 +0000 (09:41 +0100)]
WIP:test: HandoverIT-POC - splitted up code into smaller classes -- MOVE
Kai Moritz [Tue, 27 Feb 2024 08:31:51 +0000 (09:31 +0100)]
test: HandoverIT-POC - splitted up code into smaller classes -- MOVE
Kai Moritz [Tue, 27 Feb 2024 08:24:41 +0000 (09:24 +0100)]
test: HandoverIT-POC - Moved general container-defs into base-class
Kai Moritz [Tue, 27 Feb 2024 04:14:09 +0000 (05:14 +0100)]
test: HandoverIT-POC - setup without static containers
Kai Moritz [Sat, 3 Feb 2024 07:29:06 +0000 (08:29 +0100)]
test: HandoverIT-POC - first working setup for the planned test
* First working POC of the setup for the planned test.
* This version blocks endlessly, because the SSE-flux never completes.
Kai Moritz [Mon, 26 Feb 2024 18:55:19 +0000 (19:55 +0100)]
fix: Errors during shard-publishing should not kill the instance
* `HaproxyShardingPublisherStrategy` has to transform any exception into
a `Mono.error()`.
* `DataChannel.onPartitionsAssigned(..)` has to log and swallow errors
during the propagation of the shard-ownership.
Kai Moritz [Sun, 25 Feb 2024 20:44:16 +0000 (21:44 +0100)]
fix: Sensible value for sink-buffer
Kai Moritz [Sun, 25 Feb 2024 19:50:21 +0000 (20:50 +0100)]
feat: Made polling-interval for Kafka configurable
Kai Moritz [Sat, 24 Feb 2024 11:56:24 +0000 (12:56 +0100)]
fix: The number of chards is the number of partitions of the `data_channel`
Kai Moritz [Fri, 23 Feb 2024 10:39:26 +0000 (11:39 +0100)]
fix: Disabled mongodb-autoconfig, if mongodb is not used
Kai Moritz [Sat, 3 Feb 2024 23:24:07 +0000 (00:24 +0100)]
feat: Switched to `spring-boot-starter-data-mongodb-reactive`
Kai Moritz [Tue, 20 Feb 2024 16:35:14 +0000 (17:35 +0100)]
feat: Added logging for io.projectreactor to `MongoDbStorageStrategy`
Kai Moritz [Tue, 20 Feb 2024 16:32:54 +0000 (17:32 +0100)]
refactor: DRY for logging-category from io.projectreactor
Kai Moritz [Tue, 20 Feb 2024 15:14:22 +0000 (16:14 +0100)]
feat: Added counting of restored instances
Kai Moritz [Thu, 22 Feb 2024 15:46:27 +0000 (16:46 +0100)]
test: `StorageStrategy`-IT are restoring instead of recreating
Kai Moritz [Thu, 22 Feb 2024 15:03:08 +0000 (16:03 +0100)]
fix: GREEN - Fixed the restore-mechanism
* The code of a reactive flow _must not_ call blocking functions.
* In order to solve this, the restore-process is triggered explicitly
after the creation of the classes.
Kai Moritz [Tue, 20 Feb 2024 15:12:00 +0000 (16:12 +0100)]
refactor: RED - Refined success/error-handling for restore-operations
* This innocent little change discloses a severe missconception in the
implementation of the storage strategies.
* The call to `Mono.block()`, though not really changing the behaviour
during the restore-process, triggers a sanity-check from
io.projectractor.
Kai Moritz [Thu, 22 Feb 2024 14:35:39 +0000 (15:35 +0100)]
test: Simplified the integration-tests for `StorageStrategy`
Kai Moritz [Thu, 22 Feb 2024 13:49:46 +0000 (14:49 +0100)]
test: Simplified `InMemoryWithMongoDbStorageIT`
* Switched to `@ServiceConnection` instead of hand-coded initializer.
Kai Moritz [Wed, 21 Feb 2024 17:30:01 +0000 (18:30 +0100)]
test: Added IT for `ChatRoomRepository` and `MessageRepository`
Kai Moritz [Thu, 22 Feb 2024 13:39:30 +0000 (14:39 +0100)]
test: Simplified & Unified the Kafka-tests
Kai Moritz [Thu, 22 Feb 2024 13:24:28 +0000 (14:24 +0100)]
test: Simplified `ChatHomeServiceTest`
* Removed the unnecessary explicit instantiation of the `ObjectMapper`.
Kai Moritz [Thu, 22 Feb 2024 12:10:09 +0000 (13:10 +0100)]
test: Simplified & unified test-setup for ``ChatHomeServiceTest``s
* Using existing configuration-classes instead of hand-coded configuartion.
* Aligned the `KafkaChatHomeServiceTest` with the `in-memory`-tests.
Kai Moritz [Tue, 20 Feb 2024 12:05:21 +0000 (13:05 +0100)]
refactor: Cleaned up code
Kai Moritz [Tue, 20 Feb 2024 12:02:09 +0000 (13:02 +0100)]
feat: Introduced counting of stored instances in `StorageStrategy`
Kai Moritz [Tue, 20 Feb 2024 11:48:40 +0000 (12:48 +0100)]
feat: Introduced config-parameters for the `io.projectreactor`-logging
Kai Moritz [Tue, 20 Feb 2024 10:43:00 +0000 (11:43 +0100)]
refactor: Set logging-level for io.projectreactor to `DEBUG`
Kai Moritz [Tue, 20 Feb 2024 10:28:22 +0000 (11:28 +0100)]
refactor: Refined stream-definition in `StorageStrategy#write`
* Changed the stream-definition to a more natural order.
* As a result, the stored `ChatRoomInfo`-instances do not have to be
handed clumsily to the following stream.
Kai Moritz [Tue, 20 Feb 2024 10:14:43 +0000 (11:14 +0100)]
refactor: Refined return-type of `StorageStrategy#write`
Kai Moritz [Tue, 20 Feb 2024 07:50:56 +0000 (08:50 +0100)]
refactor: One stream -> using `flatMap` instead of an inner `subscribe`
Kai Moritz [Tue, 20 Feb 2024 07:23:13 +0000 (08:23 +0100)]
refactor: Moved succes/error-logging to outer stream-definitions
Kai Moritz [Tue, 20 Feb 2024 06:47:22 +0000 (07:47 +0100)]
refactor: Simplified `StorageStrategy`
* Reconfigurable success/error-logging was introduced for
`NoStorageStorageStrategy`.
* But as it turns out, this strategy can simply apply its logging in the
overwritten method, that disables the whole storing-logic.
* Hence, the interface was greatly simplified again, by removing this
ununsed mechanism.
Kai Moritz [Mon, 19 Feb 2024 14:01:58 +0000 (15:01 +0100)]
refactor: Moved extracted the `subscribe()`-call from `StorageStrategy`
* The subscription does no more happen inside the implementations of the
interface `StorageStrategy`.
* Instead, the methods, that are defined in `StorageStrategy` return the
created `Flux`.
* The call to `subscribe()` happens in the code, that uses this methods.
* This faciliates feature refinements concerning the asynchronous handling
of success- and error-cases and so forth.
Kai Moritz [Sat, 3 Feb 2024 23:37:23 +0000 (00:37 +0100)]
refactor: Extracted subscription into a separate method-call
* This is in preparation to a planed refinement.
* The refinment will move the `subscribe()`-call out of the implementation
of the `StorageStrategy`.
Kai Moritz [Mon, 19 Feb 2024 13:50:34 +0000 (14:50 +0100)]
test: Added missing JSR-310 module to not Spring-Boot-based tests
Kai Moritz [Mon, 19 Feb 2024 13:31:21 +0000 (14:31 +0100)]
feat: Added log message for disabled storage
Kai Moritz [Sun, 18 Feb 2024 19:12:19 +0000 (20:12 +0100)]
fix: Without `@DirtiesContext` the app is not teared down correctly
* `StorageStrategy` depends on Spring to call the method
`ChatBackendApplication#onExit()`, that is annotated with `@PreDestroy`.
* If this method is not called, the strategy is not applied, which leads
to errors in the integration-tests.
* This happens, if all tests are run through Maven, because Spring
recycles the application-context and only tears down the context after
all tests have run.
* The addition of `@DirtiesContext` on those tests forces Spring to tear
down the application completely after each so annotated test.
* Hence, the method is called as expected, which fixes the described
errors.
Kai Moritz [Mon, 19 Feb 2024 10:05:17 +0000 (11:05 +0100)]
test: Introduced integration-tests for `NoStorageStorageStrategy`
--
ALIGN
Kai Moritz [Mon, 19 Feb 2024 10:01:13 +0000 (11:01 +0100)]
test: Introduced integration-tests for `NoStorageStorageStrategy`
--
COPY
Kai Moritz [Mon, 19 Feb 2024 12:07:35 +0000 (13:07 +0100)]
refactor: Extracted annonymous class into `NoStorageStorageStrategy`
--
ALIGN
Kai Moritz [Mon, 19 Feb 2024 12:05:51 +0000 (13:05 +0100)]
refactor: Extracted annonymous class into `NoStorageStorageStrategy`
--
COPY
Kai Moritz [Sun, 18 Feb 2024 20:46:56 +0000 (21:46 +0100)]
refactor: Added success- and failure-callbacks for `ChatHomeService`
Kai Moritz [Sat, 3 Feb 2024 21:44:02 +0000 (22:44 +0100)]
refactor: Added success- and failure-callbacks to `StorageStrategy`
Kai Moritz [Sat, 3 Feb 2024 19:43:29 +0000 (20:43 +0100)]
refactor: `KafkaServicesApplicationRunner` aufgeräumt
Kai Moritz [Sat, 3 Feb 2024 11:08:15 +0000 (12:08 +0100)]
fix: Config-ITs do not fail, if run multiple times
Kai Moritz [Sat, 3 Feb 2024 07:35:38 +0000 (08:35 +0100)]
fix: Removed manual version-management for testcontainers.org
Kai Moritz [Sat, 3 Feb 2024 07:32:40 +0000 (08:32 +0100)]
feat: Upgraded Spring Boot 3.1.3 -> 3.2.2
Kai Moritz [Sat, 3 Feb 2024 16:25:45 +0000 (17:25 +0100)]
fix: `getChatRoomInfo()` thrwos `LoadInProgressException` when loading
- The method `InfoChannel.getChatRoomInfo(UUID)` has to check, if loading
is in process.
- Otherwise, an existing chat-room might erronously not be found, if it is
requested, while `InfoChannel` is loading, because it is not yet loaded.
Kai Moritz [Sat, 3 Feb 2024 14:32:25 +0000 (15:32 +0100)]
refactor: compute `loadInProgress` on offset-change
Kai Moritz [Sat, 3 Feb 2024 14:20:25 +0000 (15:20 +0100)]
refactor: separated message- and record- (aka offset-) handling
Kai Moritz [Sat, 3 Feb 2024 14:13:09 +0000 (15:13 +0100)]
refactor: Renamed method in `InfoChannel` according to conventions
Kai Moritz [Sat, 3 Feb 2024 14:11:50 +0000 (15:11 +0100)]
refactor: Made only locally used method in `KafkaChatHomeService` private
Kai Moritz [Sun, 24 Sep 2023 19:42:22 +0000 (21:42 +0200)]
feat: The position of the last seen messages is stored on a revoke
Kai Moritz [Sun, 24 Sep 2023 19:39:01 +0000 (21:39 +0200)]
fix: The actual position has to be requested from the consumer
* If the last seen offset and the current offset differ, although the
partition did not contain any messages between this offsets, the loading
process got stuck, because the position never advanced.
* Therefore, the actual position, that is compared against the read
end-offset, has to be requested from the consumer.
Kai Moritz [Sun, 24 Sep 2023 19:38:46 +0000 (21:38 +0200)]
refactor: Added logging of loaded messages
Kai Moritz [Tue, 20 Feb 2024 09:26:25 +0000 (10:26 +0100)]
refactor: Refined success/error-handling when publishing shard-ownership
Kai Moritz [Sun, 17 Sep 2023 09:31:22 +0000 (11:31 +0200)]
feat: Implemented `HaproxyShardingPublisherStrategy`
* Implemented a first simple `ShardingPublisherStrategy`, that uses the
https://www.haproxy.com/documentation/haproxy-runtime-api/[HAProxy Runntime API]
to publish changed ownerships.
* Added configuration-properties `kafka.haproxyRuntimeApi` and
`kafka.haproxyMap` to configure the strategy.
Kai Moritz [Tue, 20 Feb 2024 08:53:10 +0000 (09:53 +0100)]
feat: Introduced interface `ShardingPublisherStrategy`
* The interface is used by `DataChannel` to publish the changed ownership
each time, a new partition is assigned to the consumer-group.
* Added a dummy-implementation in `KafkaServicesConfiguration`.
Kai Moritz [Tue, 20 Feb 2024 08:48:03 +0000 (09:48 +0100)]
feat: Introduced a configurable instance-id
Kai Moritz [Fri, 22 Sep 2023 16:20:31 +0000 (18:20 +0200)]
fix: `ConsumerTaskRunner` waits until the data-loading is finished
Kai Moritz [Sat, 16 Sep 2023 19:40:45 +0000 (21:40 +0200)]
feat: Introduced events that are send, if a shard is assigned/revoked
* In order to redirect requests to the appropriate instances, each
instance must know the mapping from shard-IDs to instance-URIs.
** For the static `in-memory`-implementation `ShardedChatHomeService`,
this is a static mapping, that can be configured and read on start-up.
** For the dynamic `KafkaChatHomeService`, this mapping has to be
propageted to all instances each time the partition-assignment of the
consumer-group changes.
* Changes for `ShardedChatHomeService`
** Introduced `ChatBackendProperties.shardOwners` (of type `URI[]`).
** Each instance reads the static mapping from shard-ID to instance-URI
on start-up.
* Changes for `KafkaChatHomeService`
** Introduced `KafkaServicesProperties.instanceUri` (of type `URI`).
** Each instance reads its URI on start-up.
** `DataChannel` sends an event for each assigned/removed partition,
when the partition-assignment of the consumer-group changes.
** These events propagete the changed mapping from partition-ID to
instance-URI.
** `InfoChannel` receives these events and updates the dynamic mapping
from partition-ID (aka:shard-ID) to instance-URI.
* The shards, that are owned by an instance, can be queried via `/shards`.
Kai Moritz [Sun, 28 Jan 2024 17:50:33 +0000 (18:50 +0100)]
fix: `metadata` is never `null`
Kai Moritz [Fri, 15 Sep 2023 19:35:17 +0000 (21:35 +0200)]
refactor: DRY for computation of new `ChatRoomData` instances
Kai Moritz [Fri, 15 Sep 2023 18:56:14 +0000 (20:56 +0200)]
fix: GREEN - `DataChannel` creates entries for existent chat-rooms
Kai Moritz [Fri, 15 Sep 2023 18:04:01 +0000 (20:04 +0200)]
test: RED - Putting a message in a newly created chat-room
Kai Moritz [Fri, 15 Sep 2023 15:29:05 +0000 (17:29 +0200)]
refactor: Splitted test in `AbstractConfigurationIT` into smaler pieces
Kai Moritz [Fri, 15 Sep 2023 15:08:33 +0000 (17:08 +0200)]
refactor: Moved common test-code into helper-class `KafkaTestUtils` - ALIGN
Kai Moritz [Fri, 15 Sep 2023 08:57:55 +0000 (10:57 +0200)]
refactor: Moved common test-code into helper-class `KafkaTestUtils` - COPY
Kai Moritz [Fri, 15 Sep 2023 08:48:28 +0000 (10:48 +0200)]
refactor: Extracted interface `WorkAssignor` into separate file
Kai Moritz [Tue, 12 Sep 2023 21:33:59 +0000 (23:33 +0200)]
refactor: Separated channels for data and info -- Refactored/aligned code
* Split `ChatRoomChannel` into `InfoChannel` and `DataChannel`
** `DataChannel` manages only data for chat-messages
** `InfoChannel` manages all info-data (at the moment only
`EventChatRoomCreated`)
* Aligned test-setup for kafka-related tests
Kai Moritz [Mon, 11 Sep 2023 16:36:05 +0000 (18:36 +0200)]
refactor: Separated channels for data and info -- Moved/copied classes
* Split `ChatRoomChannel` into `InfoChannel` and `DataChannel`
** `DataChannel` manages only data for chat-messages
** `InfoChannel` manages all info-data (at the moment only
`EventChatRoomCreated`)
* Aligned test-setup for kafka-related tests
Kai Moritz [Thu, 14 Sep 2023 21:42:59 +0000 (23:42 +0200)]
refactor: Introduced `ConsumerTaskExecutor` -- Aligned code
* Moved startup-logic in a separate class (`ConsumerTaskExecutor`), so
that it is reusable more easily in test scenarios.
* `KafkaServicesApplicationRunner` is instanciated via `@Component`, so
that it is _not_ instanciated automatically, if the configuration is
triggered explicitly via `KafkaServicesConfiguration` in test scenarios.
Kai Moritz [Thu, 14 Sep 2023 21:41:46 +0000 (23:41 +0200)]
refactor: Introduced `ConsumerTaskExecutor` -- Copied class
Kai Moritz [Thu, 14 Sep 2023 21:17:48 +0000 (23:17 +0200)]
refactor: `KafkaChatHomeServiceTest` reuses regular startup-logic
Kai Moritz [Wed, 13 Sep 2023 19:20:39 +0000 (21:20 +0200)]
refactor: `KafkaServicesApplicationRunner` without autowireing
Kai Moritz [Mon, 11 Sep 2023 14:19:22 +0000 (16:19 +0200)]
refactor: Renamed `ChatRoomService` into `ChatMessageService` -- ALIGNE
Kai Moritz [Mon, 11 Sep 2023 14:18:42 +0000 (16:18 +0200)]
refactor: Renamed `ChatRoomService` into `ChatMessageService` -- MOVE
Kai Moritz [Mon, 11 Sep 2023 14:10:22 +0000 (16:10 +0200)]
refactor: Removed unused methods
Kai Moritz [Wed, 6 Sep 2023 21:48:50 +0000 (23:48 +0200)]
refactor: Renamed `persistence` into `implementation` - Aligned code
Kai Moritz [Wed, 6 Sep 2023 21:48:39 +0000 (23:48 +0200)]
refactor: Renamed `persistence` into `implementation` - Moved classes
Kai Moritz [Wed, 6 Sep 2023 21:46:36 +0000 (23:46 +0200)]
refactor: `storage` is not a sub-package of `persistence` - Aligned code
Kai Moritz [Wed, 6 Sep 2023 21:46:29 +0000 (23:46 +0200)]
refactor: `storage` is not a sub-package of `persistence` - Moved classes
Kai Moritz [Wed, 6 Sep 2023 21:41:37 +0000 (23:41 +0200)]
refactor: `ChatHome` is a service - Aligned code
Kai Moritz [Wed, 6 Sep 2023 21:41:27 +0000 (23:41 +0200)]
refactor: `ChatHome` is a service - Moved classes
Kai Moritz [Wed, 6 Sep 2023 21:38:28 +0000 (23:38 +0200)]
refactor: Moved exceptions into package `exceptions` - Aligned Code
Kai Moritz [Wed, 6 Sep 2023 21:38:15 +0000 (23:38 +0200)]
refactor: Moved exceptions into package `exceptions` - Moved classes