demos/kafka/wordcount
4 months agoForced emidiate emitt of join-results
Kai Moritz [Fri, 5 Jul 2024 22:25:17 +0000 (00:25 +0200)]
Forced emidiate emitt of join-results

4 months agoRefined log messages
Kai Moritz [Sat, 6 Jul 2024 07:21:45 +0000 (09:21 +0200)]
Refined log messages

4 months agoAdded log-message, when sending a record
Kai Moritz [Sat, 6 Jul 2024 07:10:54 +0000 (09:10 +0200)]
Added log-message, when sending a record

4 months agoAdded forgotten input record from original example
Kai Moritz [Fri, 5 Jul 2024 21:54:30 +0000 (23:54 +0200)]
Added forgotten input record from original example

4 months agoConfigured logback for shorter log-lines
Kai Moritz [Fri, 5 Jul 2024 21:51:26 +0000 (23:51 +0200)]
Configured logback for shorter log-lines

4 months agoFixed demo: The order looked only random, because of the wrong input time
Kai Moritz [Fri, 5 Jul 2024 21:31:39 +0000 (23:31 +0200)]
Fixed demo: The order looked only random, because of the wrong input time

4 months agoSimplified code: DRY for assertions
Kai Moritz [Fri, 5 Jul 2024 21:30:56 +0000 (23:30 +0200)]
Simplified code: DRY for assertions

4 months agoCompacted & clearified code: records are created "in place"
Kai Moritz [Fri, 5 Jul 2024 21:27:11 +0000 (23:27 +0200)]
Compacted & clearified code: records are created "in place"

4 months agoFixed some misconceptions in the stream/stream-demo
Kai Moritz [Fri, 5 Jul 2024 21:10:02 +0000 (23:10 +0200)]
Fixed some misconceptions in the stream/stream-demo

4 months agoSimplified demo-code: removed unnecessary keywords and performed DRY
Kai Moritz [Fri, 5 Jul 2024 21:06:32 +0000 (23:06 +0200)]
Simplified demo-code: removed unnecessary keywords and performed DRY

4 months agoSeparated the Demo for an Inner Stream/Stream-Join
Kai Moritz [Fri, 5 Jul 2024 20:51:44 +0000 (22:51 +0200)]
Separated the Demo for an Inner Stream/Stream-Join

4 months agoSeparated Demo-Tests from Example-Tests demos
Kai Moritz [Fri, 5 Jul 2024 20:45:58 +0000 (22:45 +0200)]
Separated Demo-Tests from Example-Tests

4 months agoSeparated Example-Tests from `counter` examples
Kai Moritz [Fri, 5 Jul 2024 20:44:23 +0000 (22:44 +0200)]
Separated Example-Tests from `counter`

4 months agoDemonstration of the outcome of an Stream/Stream-Inner-Join
Kai Moritz [Fri, 5 Jul 2024 18:58:21 +0000 (20:58 +0200)]
Demonstration of the outcome of an Stream/Stream-Inner-Join

4 months agoExamples für `@SpringBootTest` und `@SpringJunitConfig`
Kai Moritz [Fri, 5 Jul 2024 17:23:20 +0000 (19:23 +0200)]
Examples für `@SpringBootTest` und `@SpringJunitConfig`

5 months agocounter: 1.4.2 - RocksDB does nor work in Alpine-Linux counter counter-1.4.2
Kai Moritz [Sun, 23 Jun 2024 09:43:46 +0000 (11:43 +0200)]
counter: 1.4.2 - RocksDB does nor work in Alpine-Linux

5 months agocounter: 1.4.1 - Upgraded JDK-version, Spring Boot & Docker counter-1.4.1
Kai Moritz [Sun, 23 Jun 2024 08:43:00 +0000 (10:43 +0200)]
counter: 1.4.1 - Upgraded JDK-version, Spring Boot & Docker

5 months agocounter: 1.4.0 - Refined output JSON to match the new general stats-format counter-1.4.0
Kai Moritz [Sat, 22 Jun 2024 12:22:35 +0000 (14:22 +0200)]
counter: 1.4.0 - Refined output JSON to match the new general stats-format

* Changed the type-mapping for `Word` from `word` to `key`.
* Refined the class `Word`, that defines the JSON for the output key.
** Added attribute `type` with fixed value `POPULAR`.
** Renamed attribute `user` to `channel`.
** Renamed attribute `word` to `key`.
* Refined the class `WordCounter`, that defines the JSON for the output
  value.
** Renamed attribute `word` to `key`.
* Adapted test-classes and -cases accordingly.

5 months agocounter: 1.3.1 - Splitted up test in `CounterStreamProcessorTopologyTest`
Kai Moritz [Sun, 16 Jun 2024 20:33:52 +0000 (22:33 +0200)]
counter: 1.3.1 - Splitted up test in `CounterStreamProcessorTopologyTest`

5 months agocounter: 1.3.1 - Refined `CounterStreamProcessor` (DRY for type-mapping) counter-1.3.1
Kai Moritz [Sun, 16 Jun 2024 19:17:27 +0000 (21:17 +0200)]
counter: 1.3.1 - Refined `CounterStreamProcessor` (DRY for type-mapping)

5 months agocounter: 1.3.1 - Refined `CounterStreamProcessor` (serde-config)
Kai Moritz [Sun, 16 Jun 2024 19:08:01 +0000 (21:08 +0200)]
counter: 1.3.1 - Refined `CounterStreamProcessor` (serde-config)

* Refactored the creation of the ``JsonSerde``s, that are used to consume
  the incomming messages.
* All special ``Serdes``, that are used for incomming and outgoing messages,
  are created in separted methods now.
* Removed unnecessary operatorx in the ``Materialized``-configuration for
  the state store (the operator is not necessary, because no headers are
  present, when deserializing from a store).

5 months agocounter: 1.3.1 - Refined/Simplified the type-mapping
Kai Moritz [Sun, 16 Jun 2024 17:57:23 +0000 (19:57 +0200)]
counter: 1.3.1 - Refined/Simplified the type-mapping

* Removed all explicit type-mappings for the internally used types.
* This greatly simplifies the configuration, because it is sufficient to
  configure the trusted package to serialize/deserialize all internally
  used types.
* To make this possible, the type-mappings for the outgoing messages are
  specified with `Produced.with()` in the ``to()``-operation.

5 months agocounter: 1.3.1 - Refined `CounterStreamProcessorTopologyTest`
Kai Moritz [Sun, 16 Jun 2024 17:38:39 +0000 (19:38 +0200)]
counter: 1.3.1 - Refined `CounterStreamProcessorTopologyTest`

* `CounterStreamProcessorTopologyTest` uses the type-headers to determine
  the correct type for the deserialization of the output-data.
* Beforehand, the used types were hard-coded in the test.

5 months agocounter: 1.3.1 - Removed the defaults for serialization/deserialization
Kai Moritz [Sun, 16 Jun 2024 17:18:43 +0000 (19:18 +0200)]
counter: 1.3.1 - Removed the defaults for serialization/deserialization

5 months agocounter: 1.3.1 - Cleand code/setup for tests
Kai Moritz [Sun, 16 Jun 2024 16:26:47 +0000 (18:26 +0200)]
counter: 1.3.1 - Cleand code/setup for tests

5 months agocounter: 1.3.0 - Fixed possible NPE in `Counter10ApplicationIT` counter-1.3.0
Kai Moritz [Sat, 8 Jun 2024 17:39:00 +0000 (19:39 +0200)]
counter: 1.3.0 - Fixed possible NPE in `Counter10ApplicationIT`

5 months agocounter: 1.3.0 - (GREEN) Fixed the typing for the state-store
Kai Moritz [Wed, 5 Jun 2024 21:57:32 +0000 (23:57 +0200)]
counter: 1.3.0 - (GREEN) Fixed the typing for the state-store

5 months agocounter: 1.3.0 - (RED) Made `CounterApplicationIT` fail too
Kai Moritz [Wed, 5 Jun 2024 19:44:26 +0000 (21:44 +0200)]
counter: 1.3.0 - (RED) Made `CounterApplicationIT` fail too

* Validate with: `mvn test -Dtest=CounterApplicationI`
* Turning on the caching forces the application, to serialize and
  deserialize _every_ message, hence, revealing the bug, that was only
  detected by the `CounterStreamProcessorToplogyTest` before, that always
  behaves like this.

5 months agocounter: 1.3.0 - (RED) Introduced domain-class `User` as key
Kai Moritz [Wed, 5 Jun 2024 19:30:17 +0000 (21:30 +0200)]
counter: 1.3.0 - (RED) Introduced domain-class `User` as key

* _GREEN:_ The `CounterApplicationIT` does _not_ reveal the bug!
* _RED:_ The `CounterStreamProcessorToplogyTest` fails with an exception,
  that gives a hint for the cause of the bug.
* The bug is caused by missing type-specifications for the operation
  ``cout()``.
* Before the introduction of the domain-class `User` everything worked as
  expected, because the class `Word` could be specified as default for
  the deserialization of the key.
** With the introduction of the domain-class `User` as key of the incoming
   messages, the default for the key has to switched to this class, to
   enable the application to deserialize incomming keys despite the missing
   type mapping.
** Beforehand, the default `Word` covered the missing type information
   for the ``count()``-operator.

5 months agocounter: 1.2.15 - Inlined the asserting ``Consumer``s counter-1.2.15
Kai Moritz [Sat, 8 Jun 2024 10:26:10 +0000 (12:26 +0200)]
counter: 1.2.15 - Inlined the asserting ``Consumer``s

5 months agocounter: 1.2.15 - Added assertion for the expected state
Kai Moritz [Sat, 8 Jun 2024 09:33:53 +0000 (11:33 +0200)]
counter: 1.2.15 - Added assertion for the expected state

5 months agocounter: 1.2.15 - Added assertion for the expected final output messages
Kai Moritz [Sat, 8 Jun 2024 09:01:22 +0000 (11:01 +0200)]
counter: 1.2.15 - Added assertion for the expected final output messages

5 months agocounter: 1.2.15 - Added assertion for the expected number of messages
Kai Moritz [Sat, 8 Jun 2024 08:35:34 +0000 (10:35 +0200)]
counter: 1.2.15 - Added assertion for the expected number of messages

5 months agocounter: 1.2.15 - Refined `CounterApplicationIT`
Kai Moritz [Sat, 8 Jun 2024 07:36:43 +0000 (09:36 +0200)]
counter: 1.2.15 - Refined `CounterApplicationIT`

* Preparations for the addition of new tests
* The messages are send only once during `@BeforeAll`

5 months agocounter: 1.2.15 - DRY for test-data
Kai Moritz [Sat, 8 Jun 2024 07:09:37 +0000 (09:09 +0200)]
counter: 1.2.15 - DRY for test-data

5 months agocounter: 1.2.15 - Fixed race-condition in `CounterApplicationIT`
Kai Moritz [Thu, 6 Jun 2024 17:47:52 +0000 (19:47 +0200)]
counter: 1.2.15 - Fixed race-condition in `CounterApplicationIT`

5 months agocounter: 1.2.15 - Refined `TestData` (explicit key in input-data)
Kai Moritz [Wed, 5 Jun 2024 19:00:15 +0000 (21:00 +0200)]
counter: 1.2.15 - Refined `TestData` (explicit key in input-data)

5 months agocounter: 1.2.15 - Fixed deprecated parameter for cache-config
Kai Moritz [Wed, 5 Jun 2024 18:33:31 +0000 (20:33 +0200)]
counter: 1.2.15 - Fixed deprecated parameter for cache-config

5 months agocounter: 1.2.15 - Removed logging of type-headers in tests
Kai Moritz [Wed, 5 Jun 2024 18:29:37 +0000 (20:29 +0200)]
counter: 1.2.15 - Removed logging of type-headers in tests

5 months agocounter: 1.2.15 - Cleaned up the serialization configuration
Kai Moritz [Wed, 5 Jun 2024 16:29:24 +0000 (18:29 +0200)]
counter: 1.2.15 - Cleaned up the serialization configuration

5 months agocounter: 1.2.15 - Separated serialization-config into a static method
Kai Moritz [Wed, 5 Jun 2024 15:06:30 +0000 (17:06 +0200)]
counter: 1.2.15 - Separated serialization-config into a static method

5 months agocounter: 1.2.15 - `TestData` only holds and asserts the test-data
Kai Moritz [Mon, 27 May 2024 20:28:28 +0000 (22:28 +0200)]
counter: 1.2.15 - `TestData` only holds and asserts the test-data

5 months agocounter: 1.2.15 - `ToplogyTestDriver.close` must always be called
Kai Moritz [Mon, 27 May 2024 19:25:55 +0000 (21:25 +0200)]
counter: 1.2.15 - `ToplogyTestDriver.close` must always be called

5 months agocounter: 1.2.15 - Made in/out messages in `TestData` immutable
Kai Moritz [Mon, 27 May 2024 20:23:58 +0000 (22:23 +0200)]
counter: 1.2.15 - Made in/out messages in `TestData` immutable

5 months agocounter: 1.2.15 - Refined method-naming in `TestData`
Kai Moritz [Mon, 27 May 2024 19:39:22 +0000 (21:39 +0200)]
counter: 1.2.15 - Refined method-naming in `TestData`

5 months agocounter: 1.2.15 - Refined `TestData.writeInputData(..)`
Kai Moritz [Mon, 27 May 2024 19:53:26 +0000 (21:53 +0200)]
counter: 1.2.15 - Refined `TestData.writeInputData(..)`

5 months agocounter: 1.2.15 - Refined `TestData.assertExpectedResult(..)`
Kai Moritz [Sun, 26 May 2024 20:59:26 +0000 (22:59 +0200)]
counter: 1.2.15 - Refined `TestData.assertExpectedResult(..)`

5 months agocounter: 1.2.15 - Refined `CounterApplicationIT` (simplified setup)
Kai Moritz [Sun, 26 May 2024 19:59:55 +0000 (21:59 +0200)]
counter: 1.2.15 - Refined `CounterApplicationIT` (simplified setup)

6 months agocounter: 1.2.14 - Set up type-mappings for JSON-Deserialization counter-1.2.14
Kai Moritz [Tue, 14 May 2024 20:25:51 +0000 (22:25 +0200)]
counter: 1.2.14 - Set up type-mappings for JSON-Deserialization

6 months agocounter: 1.2.13 - The tests print out the type-mapping headers counter-1.2.13
Kai Moritz [Mon, 13 May 2024 20:02:57 +0000 (22:02 +0200)]
counter: 1.2.13 - The tests print out the type-mapping headers

6 months agocounter: 1.2.12 - Renamed `WordCount` into `WordCounter` -- ALIGN counter-1.2.12
Kai Moritz [Tue, 14 May 2024 19:54:28 +0000 (21:54 +0200)]
counter: 1.2.12 - Renamed `WordCount` into `WordCounter` -- ALIGN

6 months agocounter: 1.2.12 - Renamed `WordCount` into `WordCounter` -- MOVE
Kai Moritz [Tue, 14 May 2024 19:54:20 +0000 (21:54 +0200)]
counter: 1.2.12 - Renamed `WordCount` into `WordCounter` -- MOVE

6 months agocounter: 1.2.11 - Refactored test-classes (DRY for test-config) counter-1.2.11
Kai Moritz [Mon, 13 May 2024 19:11:38 +0000 (21:11 +0200)]
counter: 1.2.11 - Refactored test-classes (DRY for test-config)

6 months agocounter: 1.2.10 - Replaced helper-class `Message` with `KeyValue` counter-1.2.10
Kai Moritz [Mon, 13 May 2024 17:14:07 +0000 (19:14 +0200)]
counter: 1.2.10 - Replaced helper-class `Message` with `KeyValue`

6 months agocounter: 1.2.9 - Reconfigured tests to receive data as domain-instances counter-1.2.9
Kai Moritz [Sun, 12 May 2024 21:30:04 +0000 (23:30 +0200)]
counter: 1.2.9 - Reconfigured tests to receive data as domain-instances

6 months agocounter: 1.2.8 - Reconfigured tests to send data as domain-instances counter-1.2.8
Kai Moritz [Sun, 12 May 2024 16:06:09 +0000 (18:06 +0200)]
counter: 1.2.8 - Reconfigured tests to send data as domain-instances

6 months agocounter: 1.2.7 - Refined bean-naming in `CounterApplicationConfiguration` counter-1.2.7
Kai Moritz [Mon, 13 May 2024 19:04:56 +0000 (21:04 +0200)]
counter: 1.2.7 - Refined bean-naming in `CounterApplicationConfiguration`

6 months agocounter: 1.2.6 - Removed unused explicit references to `ObjectMapper` counter-1.2.6
Kai Moritz [Sun, 12 May 2024 15:43:29 +0000 (17:43 +0200)]
counter: 1.2.6 - Removed unused explicit references to `ObjectMapper`

6 months agocounter: 1.2.5 - Updated Spring Boot to `3.2.5` counter-1.2.5
Kai Moritz [Sun, 5 May 2024 09:08:27 +0000 (11:08 +0200)]
counter: 1.2.5 - Updated Spring Boot to `3.2.5`

6 months agocounter: 1.2.4 - Headers for type-mapping are not necessary counter-1.2.4
Kai Moritz [Mon, 13 May 2024 18:34:31 +0000 (20:34 +0200)]
counter: 1.2.4 - Headers for type-mapping are not necessary

6 months agocoutner: 1.2.3 - Simplified topology, using default-serdes for all steps counter-1.2.3
Kai Moritz [Fri, 17 Feb 2023 13:48:23 +0000 (14:48 +0100)]
coutner: 1.2.3 - Simplified topology, using default-serdes for all steps

- Defined `JsonSerde` as default for keys and values.
- Configured the `JsonDeserializer` for header-based typing.
- Removed the configuration of specific serdes from all steps of the
  processor-topology, expect the initial step, that needs a `StringSerde`
  for the key.

21 months agocounter: 1.2.2. - Simplified topology - better readability counter-1.2.2
Kai Moritz [Thu, 16 Feb 2023 16:51:24 +0000 (17:51 +0100)]
counter: 1.2.2. - Simplified topology - better readability

21 months agocounter: 1.2.1 - Simplified topology (no serdes needed for materialization) counter-1.2.1
Kai Moritz [Thu, 16 Feb 2023 16:48:52 +0000 (17:48 +0100)]
counter: 1.2.1 - Simplified topology (no serdes needed for materialization)

21 months agocounter: 1.2.0 - Service reads/writes using `JsonSerde` counter-1.2.0
Kai Moritz [Wed, 15 Feb 2023 07:19:09 +0000 (08:19 +0100)]
counter: 1.2.0 - Service reads/writes using `JsonSerde`

- First working version of a refined topology, that reads and writes
  key and value with the `JsonSerde`.
- This topology expects and writes all data as JSON, except the key
  of the incomming messages.
- All used Serdes are directly configured in the topology.

21 months agocounter: 1.1.11 - Added a test, that is based on `TopologyTestDriver` counter-1.1.11
Kai Moritz [Sat, 11 Feb 2023 12:02:44 +0000 (13:02 +0100)]
counter: 1.1.11 - Added a test, that is based on `TopologyTestDriver`

- The test reuses `TestData` to asserts the exact same assumptions, as
  `CounterApplicationIT`.
- The only difference is, that the message processing is carried out by
  the `ToplogyTestDriver` instead of a real Kafka cluster, that is started
  along the test-code in the same JVM, as in `CounterApplicationIT`.

21 months agocounter: 1.1.10 - Refactored the configuration inito a separated class counter-1.1.10
Kai Moritz [Wed, 15 Feb 2023 06:51:46 +0000 (07:51 +0100)]
counter: 1.1.10 - Refactored the configuration inito a separated class

21 months agocounter: 1.1.9 - Refactored creation of `Properties` as a separate bean counter-1.1.9
Kai Moritz [Tue, 14 Feb 2023 19:17:12 +0000 (20:17 +0100)]
counter: 1.1.9 - Refactored creation of `Properties` as a separate bean

21 months agocounter: 1.1.8 - Fixed a `ConcurentModificationException`-bug counter-1.1.8
Kai Moritz [Sat, 11 Feb 2023 13:27:15 +0000 (14:27 +0100)]
counter: 1.1.8 - Fixed a `ConcurentModificationException`-bug

21 months agocounter: 1.1.7 - Fixed a bug in the assertion-logic for the expected state counter-1.1.7
Kai Moritz [Tue, 14 Feb 2023 19:03:08 +0000 (20:03 +0100)]
counter: 1.1.7 - Fixed a bug in the assertion-logic for the expected state

- Each combination of a user and a word is a uniq key.
- Since the ordering is only guaranteed for messages, with the same key,
  no expectations can be made according the ordering of messages with
  different keys!
- This also simplifies the test-logic, because no deserialization is needed
  any more to pick the username out of the JSON-data.

21 months agocounter: 1.1.6 - Refactored the test-data into a separated class counter-1.1.6
Kai Moritz [Mon, 13 Feb 2023 18:00:53 +0000 (19:00 +0100)]
counter: 1.1.6 - Refactored the test-data into a separated class

21 months agocounter: 1.1.5 - Fixed a bug in the integration-test `CounterApplicationIT` counter-1.1.5
Kai Moritz [Sat, 11 Feb 2023 14:26:42 +0000 (15:26 +0100)]
counter: 1.1.5 - Fixed a bug in the integration-test `CounterApplicationIT`

- The default store-type creates state, that is stored on disk.
- Hence, only the first run of the test succseeded.
- The bug was fixed by providing an in-memory store-type.

21 months agocounter: 1.1.4 - Introduced parameters for the config needed by the test counter-1.1.4
Kai Moritz [Sat, 11 Feb 2023 14:38:03 +0000 (15:38 +0100)]
counter: 1.1.4 - Introduced parameters for the config needed by the test

21 months agocounter: 1.1.3 - Implemented an integration test for the old message-format counter-1.1.3
Kai Moritz [Sat, 11 Feb 2023 06:09:45 +0000 (07:09 +0100)]
counter: 1.1.3 - Implemented an integration test for the old message-format

21 months agocounter: 1.1.2 - Moved configuration into `CounterApplication` counter-1.1.2
Kai Moritz [Sat, 11 Feb 2023 05:45:40 +0000 (06:45 +0100)]
counter: 1.1.2 - Moved configuration into `CounterApplication`

21 months agocounter: 1.1.1 - Upgraded Spring Boot 2.5.4 -> 3.0.2 counter-1.1.1
Kai Moritz [Sun, 5 Feb 2023 13:31:47 +0000 (14:31 +0100)]
counter: 1.1.1 - Upgraded Spring Boot 2.5.4 -> 3.0.2

3 years agocounter: 1.1.0 - Only counts words (splitting is done separately now) counter-1.1.0
Kai Moritz [Thu, 21 Oct 2021 19:49:51 +0000 (21:49 +0200)]
counter: 1.1.0 - Only counts words (splitting is done separately now)

3 years agocounter: 1.0.1 - changed default-port and exposed all actuator-endpoints counter-1.0.1
Kai Moritz [Sat, 16 Oct 2021 14:41:09 +0000 (16:41 +0200)]
counter: 1.0.1 - changed default-port and exposed all actuator-endpoints

3 years agowordcount:1.0.0 - counts words, keyed by username and word counter-1.0.0
Kai Moritz [Wed, 1 Sep 2021 20:31:01 +0000 (22:31 +0200)]
wordcount:1.0.0 - counts words, keyed by username and word