quick and dirty hoymles grid profile parser

_hgp_qd.py

#!/usr/bin/python3
#
# hoymiles grid profile parser
# quick and dirty
# dyn table/version
#
# param 1: grid profile
# f.ex. python3 _hgp_qd.py 0a002001000c08fc07a3000f09e2001e064a00140a5500140ac8000a09e21003138812c0001413ec0014128e000514500005200000013003025809e207a3139c1356400007d0001050010001139c0190001000006000000109e20a5a021580010000085b012c08b70941099d012c006490000000005fb000000001f4005f700200012710a00200000000cbfe00
# 

import sys

hex_string = sys.argv[1]

table_types={ 0x00: "Voltage (H/LVRT)", 0x10: "Frequency (H/LFRT)", 0x20: "Island Detection (ID)", 0x30: "Reconnection (RT)", 0x40:"Ramp Rates (RR)", 0x50: "Frequency Watt (FW)", 0x60: "Volt Watt (VW)", 0x70: "Active Power Control (APC)", 0x80:"Volt Var (VV)", 0x90: "Specified Power Factor (SPF)", 0xA0: "Reactive Power Control (RPC)", 0xB0: "Watt Power Factor (WPF)" }

### country / regulations or simple internal DB ID ? ;)
# 02 00 - austria ???? ( empty list ) / IEEE 1547 240V (?)
# 03 00 - germany - DE_VDE4105_2018 (2.0.x)
# 03 01 - unknown
# 0a 00 - European - EN 50549-1:2019
# 0c 00 - Austria - 2.0.x EU_EN50438
# 0d 04 - france - ???
# 12 00 - Poland - 2.0.x (EU_EN50438)
# 37 00 - swiss - 2.0.x (CH_NA EEA-NE7–CH2020)

table_regs={ 
		0x02: { 0x00: "US - IEEE 1547 240V (?)"}, 
	    	0x03: { 0x00: "Germany - DE_VDE4105_2018"}, 
		0x0a: { 0x00: "European - EN 50549-1:2019"},
	 	0x0c: { 0x00: "AT Tor - EU_EN50438" },
		0x0d: { 0x04: "France" }, 
                0x12: { 0x00: "Poland - EU_EN50438" },
		0x37: { 0x00: "Swiss - CH_NA EEA-NE7-CH2020"},
	}

### { table_nr : { table_ver: [["name","unit",divider]] }}
tables_struct = {
	0x00 : { 
		# Germany - DE_VDE4105_2018
		0x0A: [
			["Nominale Voltage (NV)","V",10],
               		["Low Voltage 1 (LV1)","V",10],
	                ["LV1 Maximum Trip Time (MTT)","s",10],
			["High Voltage 1 (HV1)","V",10],
                	["HV1 Maximum Trip Time (MTT)","s",10],
                	["Low Voltage 2 (LV2)","V",10],
                	["LV2 Maximum Trip Time (MTT)","s",10],
                	["10mins Average High Voltage (AHV)","V",10]
		],
		# AT Tor - EU_EN50438
                0x0B: [
                        ["Nominale Voltage (NV)","V",10],
                        ["Low Voltage 1 (LV1)","V",10],
                        ["LV1 Maximum Trip Time (MTT)","s",10],
                        ["High Voltage 1 (HV1)","V",10],
                        ["HV1 Maximum Trip Time (MTT)","s",10],
                        ["Low Voltage 2 (LV2)","V",10],
                        ["LV2 Maximum Trip Time (MTT)","s",10],
			["High Voltage 2 (HV2)","V",10],
			["HV2 Maximum Trip Time (MTT)","s",10],
                        ["10mins Average High Voltage (AHV)","V",10]
                ],
		# Poland - EU_EN50438
                0x00: [
                        ["Nominale Voltage (NV)","V",10],
                        ["Low Voltage 1 (LV1)","V",10],
                        ["LV1 Maximum Trip Time (MTT)","s",10],
                        ["High Voltage 1 (HV1)","V",10],
                        ["HV1 Maximum Trip Time (MTT)","s",10],
                ],
		# Swiss - CH_NA EEA-NE7-CH2020
                0x03: [
                        ["Nominale Voltage (NV)","V",10],
                        ["Low Voltage 1 (LV1)","V",10],
                        ["LV1 Maximum Trip Time (MTT)","s",10],
                        ["High Voltage 1 (HV1)","V",10],
                        ["HV1 Maximum Trip Time (MTT)","s",10],
                        ["Low Voltage 2 (LV2)","V",10],
                        ["LV2 Maximum Trip Time (MTT)","s",10],
                        ["High Voltage 2 (HV2)","V",10],
                        ["HV2 Maximum Trip Time (MTT)","s",10],
                ],
		# European - EN 50549-1:2019
                0x0C: [
                        ["Nominale Voltage (NV)","V",10],
                        ["Low Voltage 1 (LV1)","V",10],
                        ["LV1 Maximum Trip Time (MTT)","s",10],
                        ["High Voltage 1 (HV1)","V",10],
                        ["HV1 Maximum Trip Time (MTT)","s",10],
                        ["Low Voltage 2 (LV2)","V",10],
                        ["LV2 Maximum Trip Time (MTT)","s",10],
                        ["High Voltage 2 (HV2)","V",10],
                        ["HV2 Maximum Trip Time (MTT)","s",10],
                        ["High Voltage 3 (HV3)","V",10],
                        ["HV3 Maximum Trip Time (MTT)","s",10],
                        ["10mins Average High Voltage (AHV)","V",10],
		],
		# unknown 03 01
                0x08: [
                        ["Nominale Voltage (NV)","V",10],
                        ["Low Voltage 1 (LV1)","V",10],
                        ["LV1 Maximum Trip Time (MTT)","s",10],
                        ["High Voltage 1 (HV1)","V",10],
                        ["HV1 Maximum Trip Time (MTT)","s",10],
                        ["? High Voltage 2 (HV2)","V",10],
                ],
		# IEEE 1547 240V (?)
                0x35: [
                        ["Nominale Voltage (NV)","V",10],
                        ["Low Voltage 1 (LV1)","V",10],
                        ["LV1 Maximum Trip Time (MTT)","s",10],
                        ["High Voltage 1 (HV1)","V",10],
                        ["HV1 Maximum Trip Time (MTT)","s",10],
                        ["Low Voltage 2 (LV2)","V",10],
                        ["LV2 Maximum Trip Time (MTT)","s",10],
                        ["High Voltage 2 (HV2)","V",10],
                        ["HV2 Maximum Trip Time (MTT)","s",10],
                        ["?","",1],
                        ["?","",1],
                        ["?","",1],
                        ["?","",1],
		]



	},
	0x10: {
		0x00: [
                	["Nominal Frequency","Hz",100],
	                ["Low Frequency 1 (LF1)","Hz",100],
	                ["LF1 Maximum Trip Time (MTT)","s",10],
	                ["High Frequency 1 (HF1)","Hz",100],
	                ["HF1 Maximum Trip time (MTT)","s",10]
		],
                0x03: [
                        ["Nominal Frequency","Hz",100],
                        ["Low Frequency 1 (LF1)","Hz",100],
                        ["LF1 Maximum Trip Time (MTT)","s",10],
                        ["High Frequency 1 (HF1)","Hz",100],
                        ["HF1 Maximum Trip time (MTT)","s",10],
                        ["Low Frequency 2 (LF2)","Hz",100],
                        ["LF2 Maximum Trip Time (MTT)","s",10],
                        ["High Frequency 2 (HF2)","Hz",100],
                        ["HF2 Maximum Trip time (MTT)","s",10],
                ]

	},
	0x20: { 0x00: [ ["ID Function Activated","bool",1]] },
	
	0x30: { 
		0x03: [
                        ["Reconnect Time (RT)","s",10],
                        ["Reconnect High Voltage (RHV)","V",10],
                        ["Reconnect Low Voltage (RLV)","V",10],
                        ["Reconnect High Frequency (RHF)","Hz",100],
                        ["Reconnect Low Frequency (RLF)","Hz",100]
                ],
                0x07: [
                        ["Reconnect Time (RT)","s",10],
                        ["Reconnect High Voltage (RHV)","V",10],
                        ["Reconnect Low Voltage (RLV)","V",10],
                        ["Reconnect High Frequency (RHF)","Hz",100],
                        ["Reconnect Low Frequency (RLF)","Hz",100],
                        [" ??? ","?",1],
                        [" ??? ","?",1],
                ],
	},
	0x40: { 0x00: [
                	["Normal Ramp up Rate(RUR_NM)","Rated%/s",100],
                	["Soft Start Ramp up Rate (RUR_SS)","Rated%/s",100]
		],


	},
	0x50: { 0x08: [
                	["FW Function Activated","bool",1],
                	["Start of Frequency Watt Droop (Fstart)","Hz",100],
                	["FW Droop Slope (Kpower_Freq)","Pn%/Hz",10],
                	["Recovery Ramp Rate (RRR)","Pn%/s",100],
                	["Recovery High Frequency (RVHF)","Hz",100],
                	["Recovery Low Frequency (RVLF)","Hz",100]
		],
       		0x00: [      
                        ["FW Function Activated","bool",1],
                        ["Start of Frequency Watt Droop (Fstart)","Hz",100],
                        ["FW Droop Slope (Kpower_Freq)","Pn%/Hz",10],
                        ["Recovery Ramp Rate (RRR)","Pn%/s",100],
                ],
                0x01: [    
                        ["FW Function Activated","bool",1],
                        ["Start of Frequency Watt Droop (Fstart)","Hz",100],
                        ["FW Droop Slope (Kpower_Freq)","Pn%/Hz",10],
                        ["Recovery Ramp Rate (RRR)","Pn%/s",100],
			["Recovery High Frequency (RVHF)","Hz",100],
                ],
		# ieee
                0x11: [
                        ["FW Function Activated","bool",1],
                        ["Start of Frequency Watt Droop (Fstart)","Hz",100],
                        ["FW Droop Slope (Kpower_Freq)","Pn%/Hz",10],
                        ["Recovery Ramp Rate (RRR)","Pn%/s",100],
                        ["Recovery High Frequency (RVHF)","Hz",100],
                ],
	},
	0x60: { 0x00: [ 
                        ["VW Function Activated","bool",1],
                        ["Start of Voltage Watt Droop (Vstart)","V",10],
                        ["End of Voltage Watt Droop (Vend)","V",10],
                        ["Droop Slope (Kpower_Volt)","Pn%/V",100]
		],
                0x04: [
                        ["VW Function Activated","bool",1],
                        ["Start of Voltage Watt Droop (Vstart)","V",10],
                        ["End of Voltage Watt Droop (Vend)","V",10],
                        ["Droop Slope (Kpower_Volt)","Pn%/V",100]
                ],
	},
	0x70: { 0x02: [
                	["APC Function Activated","bool",1],
                	["Power Ramp Rate (PRR)","Pn%/s",100]
		],
		0x00: [
                        ["APC Function Activated","bool",1]
		]

	},
	0x80: { 0x00: [
                	["VV Function Activated","bool",1],
                	["Voltage Set Point V1","V",10],
                	["Reactive Set Point Q1","%Pn",10],
                	["Voltage Set Point V2","V",10],
                	["Voltage Set Point V3","V",10],
                	["Voltage Set Point V4","V",10],
                	["Reactive Set Point Q4","%Pn",10]
		],
		0x01: [
                        ["VV Function Activated","bool",1],
                        ["Voltage Set Point V1","V",10],
                        ["Reactive Set Point Q1","%Pn",10],
                        ["Voltage Set Point V2","V",10],
                        ["Voltage Set Point V3","V",10],
                        ["Voltage Set Point V4","V",10],
                        ["Reactive Set Point Q4","%Pn",10],
			["Setting Time (Tr)","s",10]
                ],

	},
	0x90: { 0x00: [ 
			["SPF Function Activated","bool",1],
			["Power Factor (PF)","",100]
		]
	},
	0xA0: { 0x02: [
                	["RPC Function Activated","bool",1],
                	["Reactive Power (VAR)","%Sn",1]
		]
	},
	0xB0: { 0x00: [
                	["WPF Function Activated","bool",1],
                	["Start of Power of WPF (Pstart)","%Pn",10],
                	["Power Factor ar Rated Power (PFRP)","",100]
		]
	}
}

def modbusCrc(msg:str) -> int:
    crc = 0xFFFF
    for n in range(len(msg)):
        crc ^= msg[n]
        for i in range(8):
            if crc & 1:
                crc >>= 1
                crc ^= 0xA001
            else:
                crc >>= 1
    return crc


binary_string = bytes.fromhex(hex_string)
binary_length = len(binary_string)

### internal DB ID ( country/regulations ) ???
str_header1 = binary_string[0]
str_header2 = binary_string[1]
### version ( major.minor / rev ) ???
str_version1 = binary_string[2]
str_version2 = binary_string[3]

try:
	print("Grid Profile: %s " % (table_regs[str_header1][str_header2]))
except:
	print("Grid Profile: unknown ( plz report to https://github.com/tbnobody/OpenDTU/wiki/Grid-Profile-Parser )" )

print ("Version: %s.%s.%s" % (((str_version1 >> 4) & 0x0F),(str_version1 & 0x0F),str_version2))

position=4
while (position < binary_length):
	str_table_n = binary_string[position]
	str_table_v = binary_string[position+1]
	try:
		print("Table Type: " , table_types[str_table_n])
	except:
		pass

	try:
		tables_diz=tables_struct[str_table_n][str_table_v]
		table_length = len(tables_diz)
	except:
		crc=bytearray(binary_string[position:position+2]).hex()
		crc2 = modbusCrc(binary_string[0:position])
		crcc = crc2.to_bytes(2, byteorder='big').hex()

		if crc == crcc:
			print("CRC (ok): ",crcc)
		else:
			print("CRC (?): ",crc)
			print("CRC calced: %s " % (crcc))
			print(" - possible unknown table (module), plz report to https://github.com/tbnobody/OpenDTU/wiki/Grid-Profile-Parser")

		print("end")
		break

	table_pos=0
	#print("str_table_n: %s, str_table_v: %s, table_length: %s" % (str_table_n,str_table_v,table_length))
	position += 2
	for x in range(table_length):
		try:	
			table_diz=tables_diz[table_pos]
		except:
			table_diz=["","",1]
		table_pos += 1

		str_work = binary_string[position:position+2]
		str_hex = str_work.hex()
		str_int = int(str_hex,16) 
		str_val = str_int / table_diz[2]

		print("position: %s \t: %s \t %s \t %s\t[%s]\t\t[%s]" % (position,str_hex,str_int,str_val,table_diz[1],table_diz[0]))
		position += 2

updated gist with the corrections
btw. somehow i mixes something up with my local edits ... hopefully, its still ok although
i finished this "walkthrough" already some time ago and my head is already on the next adventure :-)

Thanks I transferred your code into the PR tbnobody/OpenDTU#1527

Please take a look at my guess work in tbnobody/OpenDTU#987 (comment) regarding the Leading flag.
I still have a 0.01 (one-off) in my calculation.

Regarding the fix for RVHF I do agree, it looks like they added some special versions of the Hoymiles Firmware.
Also see tbnobody/OpenDTU#987 for some reports about upgrades to the Firmware due to Grid Profile oddities.

updates gist ( 2 profiles added / incomplete )
need more informations from ( printout from s-miles cloud )

search for "Technical-Note-How-to-set-Hoymiles-3Gen-Grid-Profile", that is the base i'm worked from.

position: 96 : ffa1 65441 654.41 [] [Power Factor (PF)] -> in CRC 9c27 you've a point ... didnt recognize the leading selection. must be analyzed further.

The document you rightfully referred to here https://www.hoymiles.com/wp-content/uploads/2022/11/Technical-Note-How-to-set-Hoymiles-3Gen-Grid-Profile-V1.1.pdf has it in the fine print:

3.9 Specified Power Factor (SPF)
The specified power factor mode is required in some situations by the electric utility to
meet the local requirements. The minimum range of the setting of the fixed power factor is
0.8 leading to 0.8 lagging.
Note1: Power factor is ratio of the absolute value of the active power P to the apparent power S under
periodic conditions.
Note2: Lagging power factor is defined to be when the inverter acts as an inductive load from the
perspective of the grid. Leading power factor is defined to be when the inverter acts as a capacitive load
from the perspective of the grid.
Note3: As shown in the table below, if the input value of power factor is positive, the leading power factor
will be set while if the input value of power factor is negative, the lagging power factor will be set.

Table 9 Specified Power Factor Module

Specified Power Factor (SPF)	-	-	-
Parameter	Value	Unit	Range(min-max)
Function Activated	0	/	0: Disable
			1: Enable
Power Factor (PF)	0.95	/	-0.9~0.9

So Leading is positive and Lagging is negative, hence -0 = 65536 -> 65536 - 95 = 65441 = 0xffa1 as expected.

The same is stated for Power Factor at Rated Power (PFRP) under 3.10 Watt Power Factor (WPF)

Oh by the way, would you consider it worthwhile to sort the tables_struct by the table_nr and table_ver ?

I did clean up the code in my fork of the gist you opened and I find it quite telling that "Poland - EU_EN54038" has table_ver 0x00, whereas Germany has 0x0a and AT_TOR 0x0b respectively. They are likely based on later changes within local law / regulations based on the original EU_EN54038 norm.

@noone2k tbnobody already implemented your parser in the latest OpenDTU release in commits tbnobody/OpenDTU@06651f3 and tbnobody/OpenDTU@00bc631

@stefan123t , @noone2k : I made some changes to parse IEEE 1547 240V correctly in this fork. How to interpret tables / sub-tables and divider taken from here. In case there is some doubt about the divider (e.g. 1.6s or 0.16s), here some nice tabular overviews of the permitted parameter. BTW I am not sure what exactly the difference between IEEE 1547 240V and California Rule21 240V is (If there is any...maybe rule 21 is just interpreting IEEE1547...or rule 21 is making it mandatory to use the IEEE 1547 240V grid profile).

thx ... i'll try to put everything together , once i find some time