feat: Move out concrete working implementation and add working modes #307

georgi-l95 · 2024-10-24T16:59:37Z

Description:
This PR aims mainly to make the maintanence and using of simulator (as test driver) easier. It moves away the implementation of the working modes and exposes only abstractions. So that all concrete implementations can be refactored (if needed), without the main application, knowing or caring about it.
Main simulator app should only be responsible for starting the processes and nothing more. Wherther the simulator will work in publisher or consumer mode, or both is not responsibility of the main class.

Also adds E2E test for data persistence.

Related issue(s):

Fixes #239

Notes for reviewer:

Checklist

Documented (Code comments, README, etc.)
Tested (unit, integration, etc.)

Signed-off-by: georgi-l95 <[email protected]>

codecov · 2024-10-25T13:51:41Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.63%. Comparing base (8d10540) to head (c3a6b5c).
Report is 1 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main     #307      +/-   ##
============================================
+ Coverage     99.62%   99.63%   +0.01%     
- Complexity      288      300      +12     
============================================
  Files            57       61       +4     
  Lines          1060     1101      +41     
  Branches         78       80       +2     
============================================
+ Hits           1056     1097      +41     
  Misses            3        3              
  Partials          1        1

Files with missing lines	Coverage Δ
...edera/block/simulator/BlockStreamSimulatorApp.java	`100.00% <100.00%> (ø)`
...block/simulator/config/data/BlockStreamConfig.java	`100.00% <100.00%> (ø)`
...ra/block/simulator/config/types/SimulatorMode.java	`100.00% <100.00%> (ø)`
...ck/simulator/grpc/PublishStreamGrpcClientImpl.java	`100.00% <100.00%> (ø)`
...dera/block/simulator/mode/CombinedModeHandler.java	`100.00% <100.00%> (ø)`
...dera/block/simulator/mode/ConsumerModeHandler.java	`100.00% <100.00%> (ø)`
...era/block/simulator/mode/PublisherModeHandler.java	`100.00% <100.00%> (ø)`

mattp-swirldslabs

I like the new structural changes. Try out the thread management changes commented on and see if you can utilize that ExecutorService API.

mattp-swirldslabs · 2024-10-25T19:10:16Z

simulator/src/main/java/com/hedera/block/simulator/mode/PublisherModeHandler.java

+
+    /**
+     * Constructs a new {@code PublisherModeHandler} with the specified block stream configuration and publisher client.
+     *


The build is showing: warning: no @param for publishStreamGrpcClient. You just need to add it to the javadoc here

mattp-swirldslabs · 2024-10-25T19:13:33Z

simulator/src/main/java/com/hedera/block/simulator/BlockStreamSimulatorApp.java

+            simulatorModeHandler = new ConsumerModeHandler(blockStreamConfig);
+        } else {
+            simulatorModeHandler = new CombinedModeHandler(blockStreamConfig);
+        }


You might leverage a switch statement here like:

switch (simulatorMode) { case PUBLISHER: // code break; case CONSUMER: // code break; default: // code break; }

mattp-swirldslabs · 2024-10-25T19:19:30Z

simulator/src/main/java/com/hedera/block/simulator/grpc/PublishStreamGrpcClientImpl.java

@@ -37,9 +37,11 @@
 */
 public class PublishStreamGrpcClientImpl implements PublishStreamGrpcClient {

-    private final BlockStreamServiceGrpc.BlockStreamServiceStub stub;
-    private final StreamObserver<PublishStreamRequest> requestStreamObserver;
+    private BlockStreamServiceGrpc.BlockStreamServiceStub stub;


I think you can move this declaration of stub to the init() method (doesn't need to be a member variable).

mattp-swirldslabs · 2024-10-25T19:20:54Z

simulator/src/main/java/com/hedera/block/simulator/grpc/PublishStreamGrpcClient.java

@@ -24,6 +24,11 @@
 * The PublishStreamGrpcClient interface provides the methods to stream the block and block item.
 */
 public interface PublishStreamGrpcClient {


I like the new init() and shutdown() methods 👍

mattp-swirldslabs · 2024-10-25T19:24:25Z

simulator/src/main/java/com/hedera/block/simulator/mode/CombinedModeHandler.java

+     * @param blockStreamConfig the configuration data for managing block streams
+     */
+    public CombinedModeHandler(@NonNull final BlockStreamConfig blockStreamConfig) {
+        requireNonNull(blockStreamConfig);


Does @NonNull in the parameter section give you the same safeguards as requireNonNull()? You might be able to remove that line ^^

mattp-swirldslabs · 2024-10-25T19:32:31Z

simulator/src/main/java/com/hedera/block/simulator/mode/PublisherModeHandler.java

+    private final StreamingMode streamingMode;
+    private final int delayBetweenBlockItems;
+    private final int millisecondsPerBlock;
+    private final int NANOS_PER_MILLI = 1_000_000;


You could promote this helpful math shortcut ^^ to the simulator Constants class

mattp-swirldslabs · 2024-10-25T19:33:17Z

simulator/src/main/java/com/hedera/block/simulator/mode/PublisherModeHandler.java

+            @NonNull final BlockStreamConfig blockStreamConfig,
+            @NonNull PublishStreamGrpcClient publishStreamGrpcClient) {
+        requireNonNull(blockStreamConfig);
+        requireNonNull(publishStreamGrpcClient);


I think you can remove these requireNonNull() checks give the @NonNull annotations

mattp-swirldslabs · 2024-10-25T19:50:00Z

suites/src/main/java/com/hedera/block/suites/BaseSuite.java

@@ -116,16 +107,19 @@ public static void teardown() {
     *
     * @return a configured {@link GenericContainer} instance for the Block Node server
     */
-    public static GenericContainer<?> getConfiguration() {
+    protected static GenericContainer<?> getContainer() {


Perhaps change the name of this method to createContainer()

mattp-swirldslabs · 2024-10-25T19:50:52Z

suites/src/main/java/com/hedera/block/suites/grpc/negative/NegativeServerAvailabilityTests.java

@@ -45,13 +45,13 @@ public NegativeServerAvailabilityTests() {}
     * Clean up method executed after each test.
     *
     * <p>This method stops the running container, resets the container configuration by retrieving
-     * a new one through {@link BaseSuite#getConfiguration()}, and then starts the Block Node
+     * a new one through {@link BaseSuite#getContainer()} ()}, and then starts the Block Node
     * container again.
     */
    @AfterEach
    public void cleanUp() {


Maybe change this method name to restartContainer()

mattp-swirldslabs · 2024-10-25T20:01:44Z

...src/main/java/com/hedera/block/suites/persistence/positive/PositiveDataPersistenceTests.java

+        assertFalse(savedBlocksFolderAfter.isEmpty());
+        assertTrue(savedBlocksCountAfter > savedBlocksCountBefore);
+    }
+


Rather than spin up new Threads individually (expensive). Consider using private ExecutorService executorService = Executors.newFixedThreadPool(8) and managing simulator instances from that API with something like: executorService.execute(blockStreamSimulatorAppInstance::start);. The ExecutorService gives you a number of thread lifecycle management methods for free.

feat: abstract away the publisher logic

55be112

Signed-off-by: georgi-l95 <[email protected]>

georgi-l95 added the Simulator Issue related to Block Stream Simulator label Oct 24, 2024

georgi-l95 added this to the 0.2.0 milestone Oct 24, 2024

georgi-l95 self-assigned this Oct 24, 2024

georgi-l95 mentioned this pull request Oct 24, 2024

feat: Move out concrete working implementation and add working modes #284

Closed

2 tasks

georgi-l95 changed the title ~~feat: Move out concrete working implementation and add working mode~~ feat: Move out concrete working implementation and add working modes Oct 24, 2024

chore: spotless apply and cleanup

889c464

Signed-off-by: georgi-l95 <[email protected]>

georgi-l95 added the Tests issue related to enhancing the tests label Oct 24, 2024

georgi-l95 added 2 commits October 25, 2024 15:47

test: add additional unit tests

c23693c

Signed-off-by: georgi-l95 <[email protected]>

test: add more unit tests

c3a6b5c

Signed-off-by: georgi-l95 <[email protected]>

georgi-l95 marked this pull request as ready for review October 25, 2024 13:50

georgi-l95 requested a review from a team as a code owner October 25, 2024 13:50

mattp-swirldslabs reviewed Oct 25, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Move out concrete working implementation and add working modes #307

feat: Move out concrete working implementation and add working modes #307

georgi-l95 commented Oct 24, 2024 •

edited

Loading

codecov bot commented Oct 25, 2024

mattp-swirldslabs left a comment

mattp-swirldslabs Oct 25, 2024

mattp-swirldslabs Oct 25, 2024

mattp-swirldslabs Oct 25, 2024

mattp-swirldslabs Oct 25, 2024

mattp-swirldslabs Oct 25, 2024

mattp-swirldslabs Oct 25, 2024

mattp-swirldslabs Oct 25, 2024

mattp-swirldslabs Oct 25, 2024

mattp-swirldslabs Oct 25, 2024

mattp-swirldslabs Oct 25, 2024

feat: Move out concrete working implementation and add working modes #307

Are you sure you want to change the base?

feat: Move out concrete working implementation and add working modes #307

Conversation

georgi-l95 commented Oct 24, 2024 • edited Loading

codecov bot commented Oct 25, 2024

Codecov Report

mattp-swirldslabs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

georgi-l95 commented Oct 24, 2024 •

edited

Loading