Ethernet Data Logger (DLS-L) Support

[DOSprojects]

Hi

When trying the register_scan.py I get following error (verbose=1):

RECD: a5 17 00 10 45 03 00 XX XX
frame_len does not match payload_len.

I played a bit with the code line:
modbus = PySolarmanV5("192.168.xxx.xxx", 1XXXXXXXXX, port=8899, mb_slave_id=1, verbose=1)
Changing IP address and port number gives me other errors, so I presume these are OK.
Modifying serial number (added one) only changes the second last byte in the response
Modifying mb_slave_id doesn't change anything

Kind regards
LDos

[jmccrohan]

Hi @LDossche,

That error likely indicates that two frames are being concatenated. I have seen this issue myself on my own device.

You may have some success by enabling error_correction mode. You'll need to be using the latest commit on master i.e. 888ae19, rather than 2.3.0 from PyPI.

modbus = PySolarmanV5("192.168.xxx.xxx", 1XXXXXXXXX, port=8899, mb_slave_id=1, verbose=1, error_correction=1)

Regards,
Jon

[DOSprojects]

Hi Jon

Thx for the response. I added the error correction, but this gives me an error
I added some debug print statements to notice that line 219
frame_len = frame_len_without_payload_len + payload_len
causes the index out of range. The response I get seems to short?

Scanning input registers
SENT: a5 17 00 10 45 00 00 c5 2d 61 72 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 04 75 46 00 01 ca 13 d1 15
RECD: a5 17 00 10 45 03 00 c5 02
frame: b'\xa5\x17\x00\x10E\x03\x00\xc5\x02'
framelen: 9
payloadlen: 23
frame_len does not match payload_len.
corrected framelen: 36
Traceback (most recent call last):
  File "D:\_prj\solis\solismon3\register_scan.py", line 27, in <module>
    main()
  File "D:\_prj\solis\solismon3\register_scan.py", line 11, in main
    val = modbus.read_input_registers(register_addr=x, quantity=1)[0]
  File "D:\_prj\solis\solismon3\pysolarmanv5.py", line 310, in read_input_registers
    modbus_values = self._get_modbus_response(mb_request_frame)
  File "D:\_prj\solis\solismon3\pysolarmanv5.py", line 264, in _get_modbus_response
    mb_response_frame = self._send_receive_modbus_frame(mb_request_frame)
  File "D:\_prj\solis\solismon3\pysolarmanv5.py", line 259, in _send_receive_modbus_frame
    mb_response_frame = self._v5_frame_decoder(v5_response_frame)
  File "D:\_prj\solis\solismon3\pysolarmanv5.py", line 224, in _v5_frame_decoder
    v5_frame[frame_len - 1] != int.from_bytes(self.v5_end, byteorder="big")
IndexError: index out of range

What is strange: a random serial number for the logger gives me the same output. Nevertheless I checked the number several times with the one on the soliscloud site and on the stick itself.

the stick is a wired one:
model: Solis-DLS-L
FCC ID: 2AWEB-LAN-STICK

the umodbus library is 1.0.4

Kind regards
Lieven

[LucidityCrash]

I've just discovered this :)

I got an error with struct.error: unpack requires a buffer of 2 bytes

SENT: a5 17 00 10 45 00 00 71 19 8e f3 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 04 77 2c 00 01 eb b7 c4 15
RECD: a5 db 00 10 15 00 84 71 19 8e f3 02 01 53 c7 0b 00 19 22 00 00 c1 36 96 62 01 04 c8 00 00 00 00 00 00 00 00 2a f8 03 e8 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 07 01 00 00 00 00 00 00 00 00 00 27 34 e0 09 7c 03 37 00 00 06 fa 00 21 01 fd 00 02 00 01 00 6b 09 75 00 5a 00 64 00 63 13 b3 00 00 00 00 00 fa 00 00 00 00 01 58 00 28 00 00 00 0a 00 00 08 52 00 00 00 00 00 00 00 00 00 dd 00 00 00 00 00 01 00 00 01 6e 00 18 00 1b 00 00 01 d1 00 07 00 23 00 00 06 00 00 18 00 40 00 00 04 08 00 5d 00 10 00 00 0b 13 00 3e 00 8a 00 00 00 00 08 35 00 00 00 46 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1d 7d e6 15
Traceback (most recent call last):
  File "register_scan.py", line 27, in <module>
    main()
  File "register_scan.py", line 11, in main
    val = modbus.read_input_registers(register_addr=x, quantity=1)[0]
  File "/home/barney/.local/lib/python3.8/site-packages/pysolarmanv5/pysolarmanv5.py", line 224, in read_input_registers
    modbus_values = self._get_modbus_response(mb_request_frame)
  File "/home/barney/.local/lib/python3.8/site-packages/pysolarmanv5/pysolarmanv5.py", line 179, in _get_modbus_response
    modbus_values = rtu.parse_response_adu(mb_response_frame, mb_request_frame)
  File "/home/barney/.local/lib/python3.8/site-packages/umodbus/client/serial/rtu.py", line 190, in parse_response_adu
    function = create_function_from_response_pdu(resp_pdu, req_pdu)
  File "/home/barney/.local/lib/python3.8/site-packages/umodbus/functions.py", line 138, in create_function_from_response_pdu
    return function.create_from_response_pdu(resp_pdu, req_pdu)
  File "/home/barney/.local/lib/python3.8/site-packages/umodbus/functions.py", line 911, in create_from_response_pdu
    read_input_registers.data = list(struct.unpack(fmt, resp_pdu[2:]))
struct.error: unpack requires a buffer of 2 bytes

[wdullaer]

I'm going to tack on to this issue, because I have the same error as the OP:

The serial number of my logger starts with 19xxxxxxxx
The settings in /config_hide.html exactly match the expected state mentioned in other issues.

The scan.py correctly discovers my logger when ran from the same subnet:

root@wdullaer-XPS-13-9380:/# python scan.py 
{'ipaddress': '192.168.xxx.xxx', 'mac': '34xxxxxxxxxx', 'serial': '19xxxxxxxxxx'}

The firmware reported by the html status page is: ME_0C_0501_5.08

When I run the basic client I get the following error:

root@fe5ea366462e:/# python basic.py
DEBUG:pysolarmanv5.pysolarmanv5:SENT: a5 17 00 10 45 00 00 5e a2 58 72 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 04 80 fe 00 06 38 38 31 15
DEBUG:pysolarmanv5.pysolarmanv5:RECD: a5 17 00 10 45 03 00 de 02
DEBUG:pysolarmanv5.pysolarmanv5:frame_len does not match payload_len.
Traceback (most recent call last):
...

Essentially the first 5 bytes are an echo of the input. The 03 00 feels like an error code and the de 02 bytes like a frame footer.

When I intercept the data that gets sent to the solarman platform, the data does seem to follow the format from other solarmanv5 loggers.
If I replay some of the data the solarman server sends to the logger, it will echo more bytes back before it starts with the error code.

My intuition tells me some of the 'signing' or 'preamble' of the messages the 8899 server expects are slightly different. Because it does respond to the discovery messages, I'm guessing the changes will be rather small.
Maybe that means support for this logger is outside of the scope of this library, but if you have any resources to share I'd be very grateful. For example: how did you find the discovery UDP protocol?

[jmccrohan]

Hi @wdullaer,

I have documented the protocol as best as I understand it here. The response frame frame should start with A5 and end with 15. The second and third bytes denote the payload length (0x0017 in this case, little endian).

I am really not sure what is going on with the response you have received because it makes no sense. Even if the frame length matched the reported payload length, the control code field is also wrong. I suspect the issue is with the data logger version. My data logger is running firmware MW_08_512_0501_1.82 (S/N starting with 405xxxxxxx), and it works. Perhaps contact Solis or Solarman to see if there is a firmware update available for your data logging stick?

As for the scan utility, this is a feature of the HF-A11 SOC used by Solarman data loggers. I first came across it here.

Regards,
Jon

[wdullaer]

Thanks for the links.
After refreshing my memory on modbus again, the 03 in that frame is likely the error code and the two bytes after the crc16.
However, it does indeed feel like the response payloads are weirdly truncated. It might be possible the server code on this firmware is bugged. I'll see if an updated version exists.
If it's all the same, I'll report my progress here: more people are likely to see it here than if I were to put it up on my blog.

[jmccrohan]

@wdullaer Are you using a DLS-W (WiFi) or DLS-L (ethernet)?

[wdullaer]

I have a DLS-L (ethernet).
The firmware has all the wifi options as well, I never tried them out.

[jmccrohan]

@wdullaer Might be worthwhile chasing Solis for an updated firmware so. StephanJoubert/home_assistant_solarman#87 seems to indicate that it is possible to use the DLS-L with the Solarman V5 protocol.

ME-121001-V1.0.6(202109061500) appears to be available on the Solis support portal, but no sign of the version that is mention in the linked issue.

[DOSprojects]

I got a new LAN stick (DLS-L)
Device serial number 1920XXXXXX
Firmware version ME_0C_0501_5.08
config_hide:

Working mode Data collection
Server A Setting
IP address 47.88.8.200
Domain name data1.solarmanpv.com
Port 10000
Connection TCP
Optional Server Setting
IP address 115.29.186.234
Domain name data2.solarmanpv.com
Port 10000
Connection TCP
Serial port parameters setting
Baud rate 9600
Data bit 8
Parity bit None
Stop bit 1
CTSRTS Disable
Internal server parameters setting
Protocol TCP-Server
Port 8899
Server address 10.10.100.254
TCP time out setting(no more than 600s) 300

A small check brewed from jmccrohan's code:

""" check comm """
from pysolarmanv5 import PySolarmanV5, V5FrameError
from umodbus.client.serial import rtu
import umodbus.exceptions

def main():
    modbus = PySolarmanV5("192.168.0.198", 1920XXXXXX, port=8899, mb_slave_id=1, verbose=1, error_correction=0)
    mb_request_frame = rtu.read_input_registers(1, 30022, 1)

    print("SEND")
    mb_response_frame = modbus._send_receive_modbus_frame(mb_request_frame)

if __name__ == "__main__":
    main()

gives me bottom line the same error:

(solis) D:\_prj\solis\solismon3>python check_dsl.py
SEND
frameout: SENT: a5 17 00 10 45 00 ..... 13 c3 15
RECD: a5 17 00 10 45 03 00 82 02
frame_len does not match payload_len.
Traceback (most recent call last):
  File "D:\_prj\solis\solismon3\check_dsl.py", line 20, in <module>
    main()
  File "D:\_prj\solis\solismon3\check_dsl.py", line 11, in main
    mb_response_frame = modbus._send_receive_modbus_frame(mb_request_frame)
  File "D:\_prj\solis\solismon3\pysolarmanv5.py", line 260, in _send_receive_modbus_frame
    mb_response_frame = self._v5_frame_decoder(v5_response_frame)
  File "D:\_prj\solis\solismon3\pysolarmanv5.py", line 227, in _v5_frame_decoder
    raise V5FrameError("V5 frame contains invalid start or end values")
pysolarmanv5.V5FrameError: V5 frame contains invalid start or end values

So I must agree with @wdullaer than the frame we send is invalid.
Is there anyone who has succeeded with a lan stick?

[wdullaer]

That's the same firmware version my stick is currently running.
I am going to try to flash the latest firmware found on the solis support portal when I get some free time later this week.

[jmccrohan]

Is there any chance that the DLS-L doesn't actually require Solarman V5 encapsulation at all? Have you tried querying the device on port 8899 using Modbus TCP?

@drsmarsden Seemed to get pysolarmanv5 working on his DLS-L. See issue #7

[drsmarsden]

Hi
I found my DLS-L would not accept TCP connections from HA , so I botched the code to stop the server initiating connections , just to wait for the DLS-L to initiate the connection.

I monitored the traffic using Wireshark and found that the DLS-L would send the server unsolicited messages (some short) on a regular basis. I think one is a keep alive and one a status message

I think what you are seeing is one of those which will not decode. I just ignored short messages , you then get the reply to the request which does decode. The problem I then had was deciphering the data, some values where correct other garbage. Ginlong will not release the packet definitions, so I gave up.

I switched to using https://github.com/hultenvp/solis-sensor which talks to soliscloud.com
it worked 1st time , and populates the energy panel ok.

I have fixed a couple of issues with that code (in my copy) , but they have not been integrated back yet.

The HA guys are now working on a version of Solis support for /core.

I have just looked, I did not keep the DLS-L frig that I did

Stuart

[jmccrohan]

The HA guys are now working on a version of Solis support for /core.

Do you have any more info on this?

[DOSprojects]

Is there any chance that the DLS-L doesn't actually require Solarman V5 encapsulation at all? Have you tried querying the device on port 8899 using Modbus TCP?

@drsmarsden Seemed to get pysolarmanv5 working on his DLS-L. See issue #7

This did it, thx!

import socket
from umodbus import conf
from umodbus.client import tcp

# configuration
CFG_IP   = 'xxx.xxx.xxx.xxx'
CFG_PORT = 8899

# address + length
MSG_ADDR = 33022
MSG_LEN  = 10        # max recomended data frame 100 bytes (50 registers)

# Enable values to be signed (default is False).
conf.SIGNED_VALUES = True
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((CFG_IP, CFG_PORT))
# Return a message or Application Data Unit (ADU) specific for doing Modbus TCP/IP.
message = tcp.read_input_registers(slave_id=1, starting_address=MSG_ADDR, quantity=MSG_LEN)
s = list(message)
adu = {"msglen": len(message), "transaction ID": s[:2], "protocol ID": s[2:4], "length": s[4:6], "unit ID": s[6:7], "PDU": s[7:] }
#print(adu)
## prints: {'msglen': 12, 'transaction ID': [237, 3], 'protocol ID': [0, 0], 'length': [0, 6], 'unit ID': [1], 'PDU': [4, 130, 220, 0, 100]}
## In words: Request with transaction ID xxx for slave 1. The request uses Protocol ID 0, which is the Modbus protocol
## The length of the bytes after the Length field is 6 bytes. These 6 bytes are Unit Identifier (1 byte) + PDU (5 bytes)

response = tcp.send_message(message, sock)
print( response);
# prints [22, 9, 8, 14, 11, 31, 0, 0, 2491, 0]
# year, month, day, hour, minute, secons, ...
sock.close()

So I can get the timestamp from the inverter. This does not match an address table I have, but now I can go further...

[DOSprojects]

Found an address list: https://www.scss.tcd.ie/Brian.Coghlan/Elios4you/RS485_MODBUS-Hybrid-BACoghlan-201811228-1854.pdf
My readings seem to be OK
Thanks everyone for the help - next step: implement this on nodered :)

[wdullaer]

I just wanted to confirm that my logger also works with modbus-tcp. No wrapping of the frames necessary.
Both the code snippet from @LDossche and the home-assistant modbus integration can read it.

I'd like to thank everyone here for there suggestions, it was all invaluable in figuring this out.

As for the register index: the values from this file are working for my Solis hybrid inverter: https://github.com/StephanJoubert/home_assistant_solarman/blob/main/custom_components/solarman/inverter_definitions/solis_hybrid.yaml

[jmccrohan]

@LDossche @wdullaer Thanks for this. I'll update the documentation referencing this issue.

[Bignose3]

Thanks for all the hard work done here

I was wondering if there is a way to set/send the settings to the inverter?

Funny how HA works well with the Ethernet connector and has trouble with the WiFi version.

I have HA to read all settings from my Solis hybrid inverter & I can also SET thing like battery start & end charge times but I would like to do all this from python.

Manage to get python to read from the inverter thanks to the above code, not tested too much as really need to set the values.

CFG_IP = '192.168.2.240'
CFG_PORT = 8899

#43143 to 43146 start hour & mins & end hour & mins

address + length

MSG_ADDR = 43143
MSG_LEN = 10 # max recomended data frame 100 bytes (50 registers)

Enable values to be signed (default is False).

conf.SIGNED_VALUES = True
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((CFG_IP, CFG_PORT))

Return a message or Application Data Unit (ADU) specific for doing Modbus TCP/IP.

message = tcp.read_holding_registers(slave_id=1, starting_address=MSG_ADDR, quantity=MSG_LEN)
s = list(message)
adu = {"msglen": len(message), "transaction ID": s[:2], "protocol ID": s[2:4], "length": s[4:6], "unit ID": s[6:7], "PDU": s[7:] }

response = tcp.send_message(message, sock)
print( response);

[3, 1, 5, 58, 0, 0, 0, 0, 0, 0]

start 3:10 & end 5:58 - get some odd results if I shorten the msg-len but investigate later

sock.close()`