Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modbus Bridge: optimize execution of Tasks #1976

Merged
merged 53 commits into from
Aug 2, 2023

Conversation

sfeilmeier
Copy link
Contributor

@sfeilmeier sfeilmeier commented Nov 10, 2022

In its core this is a rewrite of the 'ModbusWorker', that schedules the execution of modbus Tasks. See readme below for details on the new version:

Modbus

Modbus is a widely used standard for fieldbus communications. It is used by all kinds of hardware devices like photovoltaics inverters, electric meters, and so on.

Modbus/TCP

https://github.com/OpenEMS/openems/blob/develop/io.openems.edge.bridge.modbus/src/io/openems/edge/bridge/modbus/BridgeModbusTcpImpl.java[Bridge Modbus/RTU Serial] for fieldbus communication via TCP/IP network.

Modbus/RTU

https://github.com/OpenEMS/openems/blob/develop/io.openems.edge.bridge.modbus/src/io/openems/edge/bridge/modbus/BridgeModbusSerialImpl.java[Bridge Modbus/TCP] for fieldbus communication via RS485 serial bus.

Implementation details

OpenEMS Components that use Modbus communication, must implement the ModbusComponent interface and provide a ModbusProtocol. A protocol uses the notion of a Task to define an individual Modbus Read or Write request that can cover multiple Modbus Registers or Coils depending on the Modbus function code. It is possible to add or remove tasks to/from a protocol at runtime or to change the execution Priority. The Modbus Bridge (Bridge Modbus/RTU Serial or Bridge Modbus/TCP) collects all protocols and manages the execution of Tasks.

Execution of Modbus Tasks

Execution of Modbus Tasks is managed by the ModbusWorker. It...

  • executes Write-Tasks as early as possible (directly after the EXECUTE_WRITE event)
  • executes Read-Tasks as late as possible to have values available exactly when they are needed (i.e. just before the BEFORE_PROCESS_IMAGE event). To achieve this, the ModbusWorker evaluates all execution times and 'learns' an ideal delay time, that is applied on every Cycle - the 'CycleDelay'
  • handles defective ModbusComponents (i.e. ones where tasks have repeatedly failed) and delays reading from/writing to those components in order to avoid having defective components block the entire communication bus. Maximum delay is 5 minutes for read from defective components. ModbusComponents can trigger a retry from a defective Component by calling the retryModbusCommunication() method.

Priority

Read-Tasks can have two different priorities, that are defined in the ModbusProtocol definition:

  • HIGH: the task is executed once every Cycle
  • LOW: only one task of all defined LOW priority tasks of all components registered on the same bridge is executed per Cycle
    Write-Tasks always have HIGH priority, i.e. a set-point is always executed as-soon-as-possible - as long as the Component is not marked as defective

Channels

Each Modbus Bridge provides Channels for more detailed information:

  • CycleTimeIsTooShort: the configured global Cycle-Time is too short to execute all planned tasks in one Cycle
  • CycleDelay: see 'CycleDelay' in the 'ModbusWorker' description above

Logging

Often it is useful to print detailed logging information on the Console for debugging purposes. Logging can be enabled on Task level in the definition of the ModbusProtocol by adding .debug() or globally per Modbus Bridge via the LogVerbosity configuration parameter:

  • NONE: Show no logs
  • DEBUG_LOG: Shows basic logging information via the Controller.Debug.Log
  • READS_AND_WRITES: Show logs for all read and write requests
  • READS_AND_WRITES_VERBOSE: Show logs for all read and write requests, including actual hex or binary values of request and response
  • READS_AND_WRITES_DURATION: Show logs for all read and write requests, including actual duration time per request
  • READS_AND_WRITES_DURATION_TRACE_EVENTS: Show logs for all read and write requests, including actual duration time per request & trace the internal Event-based State-Machine

The log level via configuration parameter may be changed at any time during runtime without side-effects on the communication.


See also the discussion in OpenEMS Community: https://community.openems.io/t/zeitlicher-ablauf-der-modbus-tasks/734

The idea is to try to execute all Write-Tasks as early as possible (directly after the EXECUTE_WRITE event) and all Read-Tasks as late as possible to have values available exactly when they are needed (i.e. at the BEFORE_PROCESS_IMAGE event).
A WaitHandler is introduced, that is responsible for calculating a 'wait' time before execution of Modbus ReadTasks.
It uses an internal queue of waiting times that would have been possible in previous Cycles. The minimum value of this queue is taken as next waiting time.

See discussion at OpenEMS Community: https://community.openems.io/t/zeitlicher-ablauf-der-modbus-tasks/734
@sfeilmeier
Copy link
Contributor Author

sfeilmeier commented Feb 1, 2023

TODO: consider introducing an IMMEDIATE priority for Registers (or via special Write-Channels). See discussion at https://community.openems.io/t/power-distribution-between-battery-and-pv-inverter/1184/6

Update: In fact writing as early as possible after EXECUTE_WRITE is very close to an IMMEDIATE priority, so this has not been explicitely implemented.

@sfeilmeier sfeilmeier marked this pull request as draft May 14, 2023 21:48
@sfeilmeier
Copy link
Contributor Author

sfeilmeier commented May 14, 2023

Planned State-Machine for CycleTasksManager:

graph TD
ON_BEFORE_PROCESS_IMAGE>ON_BEFORE_PROCESS_IMAGE]
ON_EXECUTE_WRITE>ON_EXECUTE_WRITE]

ON_EXECUTE_WRITE ==> WRITE
INITIAL_WAIT -->|sleep| READ_BEFORE_WRITE
INITIAL_WAIT -.- ON_EXECUTE_WRITE
READ_BEFORE_WRITE -->|read finished early| WAIT_FOR_WRITE
READ_BEFORE_WRITE -.- ON_EXECUTE_WRITE
WAIT_FOR_WRITE -.- ON_EXECUTE_WRITE

WRITE -->|write finished| WAIT_BEFORE_READ
WAIT_BEFORE_READ -->|sleep| READ_AFTER_WRITE
READ_AFTER_WRITE -->|read finished| FINISHED

ON_BEFORE_PROCESS_IMAGE ==> INITIAL_WAIT
FINISHED -.-o ON_BEFORE_PROCESS_IMAGE
Loading

- Implement CycleTasksSupplier with DefectiveComponents logic
- Make Task interface sealed
- JUnit tests
- DefectiveComponents: add public method to trigger retry (used by Fenecon Home Battery)
- Fix Checkstyle, activate new ModbusWorker, fix JUnit tests
…k-execution

# Conflicts:
#	io.openems.edge.battery.fenecon.home/src/io/openems/edge/battery/fenecon/home/BatteryFeneconHome.java
#	io.openems.edge.bridge.modbus/src/io/openems/edge/bridge/modbus/BridgeModbusSerialImpl.java
#	io.openems.edge.bridge.modbus/src/io/openems/edge/bridge/modbus/BridgeModbusTcpImpl.java
@github-actions
Copy link

Code Coverage

@github-actions
Copy link

Code Coverage

@github-actions
Copy link

Code Coverage

@github-actions
Copy link

Code Coverage

@github-actions
Copy link

Code Coverage

@sfeilmeier
Copy link
Contributor Author

@sebastianasen, @huseyinsaht Please test this implementation on your test systems if possible. From my side it's very close to being released.

@sfeilmeier
Copy link
Contributor Author

@sfeilmeier
Copy link
Contributor Author

Followup refactoring & cleanup for Modbus Elements: #2273

sfeilmeier added a commit that referenced this pull request Jul 25, 2023
This PR adds JUnit tests for all Modbus Elements (BitsWordElement, FloatQuadruplewordElement, etc.), to make sure that future PRs (e.g. #1976 and #2273) do not add regressions to the current state
…k-execution

# Conflicts:
#	io.openems.edge.bridge.modbus/test/io/openems/edge/bridge/modbus/DummyModbusComponent.java
@github-actions
Copy link

Code Coverage

@github-actions
Copy link

Code Coverage

@github-actions
Copy link

github-actions bot commented Aug 2, 2023

Code Coverage

@sfeilmeier sfeilmeier merged commit a37da1a into develop Aug 2, 2023
3 checks passed
@sfeilmeier sfeilmeier deleted the feature/modbus-task-execution branch August 2, 2023 11:25
sfeilmeier added a commit that referenced this pull request Aug 3, 2023
This is a follow-up PR to #1976. 

The initial intention for this PR was a bug that existed previously: When the Modbus Bridge detects a 'defective device', i.e. it is not possible to read from an external device connected via modbus, e.g. because it is turned off, somebody unplugged the cable, etc., all Channels are 'invalidated'. This invalidation did not work properly for individual Bits in `BitsWordElements`.

When looking at the previous code it becomes obvious how such a bug could happen. The structure and inheritance was complicated and @OVERRIDES were often unclear. Also there were hardly any JUnit tests to validate the function.

Because of this I had added JUnit tests in #2283, then refactored all Modbus Element classes and revalidated that they behave exactly the same way as before with the JUnit tests.

This implementation is tested on a couple of production systems, including my own private system.
sfeilmeier added a commit that referenced this pull request Sep 1, 2023
- Modbus Bridge: optimize execution of Tasks (#1976)
- split TimestampedDatanotification into AggregatedDataNotification, TimestampedDataNotification and ResendDataNotification. Added/enhanced some utilities classes. (#2297)
- Modbus Bridge: refactor & cleanup Elements (#2273)
- Bump @types/uuid from 9.0.1 to 9.0.2 in /ui (#2225)
- Bump com.squareup.okio:okio-jvm from 3.4.0 to 3.5.0 in /cnf (#2307)
- Bump org.checkerframework:checker-qual from 3.36.0 to 3.37.0 in /cnf (#2308)
- Backend Timedata: prepare new data notification types and utilities (#2298)
- Docs: Fix typing error in backend-to-backend section (#2315)
- Add Backend Timedata AggregatedInflux (#2313)
- Bump info.faljse:SDNotify from 1.3 to 1.5 in /cnf (#2328)
- Bump org.rrd4j:rrd4j from 3.8.2 to 3.9 in /cnf (#2327)
- Bump org.apache.felix:org.apache.felix.http.jetty from 5.0.4 to 5.0.6 in /cnf (#2326)
- ModbusBridge: Fix ClassCastException to Register[] (#2320)
- Update Gradle to 8.3 (#2330)
- Bump compare-versions from 6.0.0 to 6.1.0 in /ui (#2319)
- SunSpecCodeGenerator: refactor, add 7xx models, use JSON input files (#2324)
- Docs: Update Getting Started & Implementing a Device (#2331)
- CI: build artifacts on tag/release event (#2332)
- Bump org.apache.felix:org.apache.felix.http.jetty from 5.0.6 to 5.1.0 in /cnf (#2334)
- Bump org.jetbrains.kotlin:kotlin-osgi-bundle from 1.9.0 to 1.9.10 in /cnf (#2335)
- Bump io.reactivex.rxjava3:rxjava from 3.1.6 to 3.1.7 in /cnf (#2333)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant