glowmqtt.md

⚠️ There is now an official way to connect Glow to Home Assistant via MQTT, I would suggest using that instead of the method described below. Here are the sensor templates for that (replace GLOW_DEVICE_ID with the relevant information for your device):

mqtt:
  sensor:
    - name: "Home Power"
      state_topic: "glow/GLOW_DEVICE_ID/SENSOR/electricitymeter"
      unit_of_measurement: 'W'
      value_template: "{{ (value_json['electricitymeter']['power']['value'] * 1000) | int }}"
      icon: 'mdi:flash'

    - name: "Home Energy"
      state_topic: "glow/GLOW_DEVICE_ID/SENSOR/electricitymeter"
      unit_of_measurement: 'kWh'
      state_class: 'total_increasing'
      device_class: 'energy'
      unique_id: 'home_energy'
      value_template: "{{ value_json['electricitymeter']['energy']['import']['cumulative'] }}"

    - name: "Home Daily Energy"
      state_topic: "glow/GLOW_DEVICE_ID/SENSOR/electricitymeter"
      unit_of_measurement: 'kWh'
      value_template: "{{ value_json['electricitymeter']['energy']['import']['day'] }}"
      icon: 'mdi:flash'

    - name: "Home Gas"
      state_topic: "glow/GLOW_DEVICE_ID/SENSOR/gasmeter"
      unit_of_measurement: 'm³'
      state_class: 'total_increasing'
      device_class: 'gas'
      unique_id: 'home_gas'
      value_template: "{{ value_json['gasmeter']['energy']['import']['cumulativevol'] }}"

    - name: "Home Daily Gas"
      state_topic: "glow/GLOW_DEVICE_ID/SENSOR/gasmeter"
      unit_of_measurement: 'kWh'
      value_template: "{{ value_json['gasmeter']['energy']['import']['day'] }}"
      icon: 'mdi:flash'

Here is a way to expose most of the Glow Display MQTT information to home assistant through templates. Please make sure you ask Hildebrand support to enable your account for MQTT access before setting everything up.

The first step is to connect Home Assistant to MQTT. If you don't have your own broker hooked up yet, go ahead and set it up (I would suggest using the Integrations UI) and specify your Glow username and password as well as glowmqtt.energyhive.com as the server and 8883 as the port to make sure we're establishing a secure SSL connection.

If you already have your own MQTT server hooked up to Home Assistant, like I do, you will have to set up a bridge between your MQTT server and the Glow one, after that your server will mirror the MQTT messages from the Glow server. You have to do this because Home Assistant does not support connecting to more than one MQTT server.

I will assume you are using Mosquitto as an MQTT server. Start by creating a file named /etc/mosquitto/conf.d/glow.conf with the following content (replace GLOW_USERNAME, GLOW_PASSWORD and GLOW_DEVICE_ID with your relevant information):

connection glowmqtt
address glowmqtt.energyhive.com:8883
remote_username GLOW_USERNAME
remote_password GLOW_PASSWORD
bridge_capath /etc/ssl/certs/
topic SMART/HILD/GLOW_DEVICE_ID in 0

Once that is done, just restart Mosquitto to reload its configuration:

sudo systemctl restart mosquitto

Now you should be able to see your Glow messages over MQTT by using your client of choice.

If that works all right, you can open the Home Assistant configuration.yaml file (for me it is located at /home/homeassistant/.homeassistant/configuration.yaml) and add these new sensors under the sensor section (replace GLOW_DEVICE_ID with the relevant information for your device):

mqtt:
  sensor:
    - name: "Home Power"
      state_topic: "SMART/HILD/GLOW_DEVICE_ID"
      unit_of_measurement: 'W'
      value_template: "{{ value_json['elecMtr']['0702']['04']['00']|int(base=16) }}"
      icon: 'mdi:flash'

    - name: "Home Energy"
      state_topic: "SMART/HILD/GLOW_DEVICE_ID"
      unit_of_measurement: 'kWh'
      state_class: 'total_increasing'
      device_class: 'energy'
      unique_id: 'home_energy'
      value_template: "{{ value_json['elecMtr']['0702']['00']['00']|int(base=16) * value_json['elecMtr']['0702']['03']['01']|int(base=16) / value_json['elecMtr']['0702']['03']['02']|int(base=16) }}"

    - name: "Home Daily Energy"
      state_topic: "SMART/HILD/GLOW_DEVICE_ID"
      unit_of_measurement: 'kWh'
      value_template: "{{ value_json['elecMtr']['0702']['04']['01']|int(base=16) * value_json['elecMtr']['0702']['03']['01']|int(base=16) / value_json['elecMtr']['0702']['03']['02']|int(base=16) }}"
      icon: 'mdi:flash'

    - name: "Home Weekly Energy"
      state_topic: "SMART/HILD/GLOW_DEVICE_ID"
      unit_of_measurement: 'kWh'
      value_template: "{{ value_json['elecMtr']['0702']['04']['30']|int(base=16) * value_json['elecMtr']['0702']['03']['01']|int(base=16) / value_json['elecMtr']['0702']['03']['02']|int(base=16) }}"
      icon: 'mdi:flash'

    - name: "Home Monthly Energy"
      state_topic: "SMART/HILD/GLOW_DEVICE_ID"
      unit_of_measurement: 'kWh'
      value_template: "{{ value_json['elecMtr']['0702']['04']['40']|int(base=16) * value_json['elecMtr']['0702']['03']['01']|int(base=16) / value_json['elecMtr']['0702']['03']['02']|int(base=16) }}"
      icon: 'mdi:flash'

    - name: "Home Gas"
      state_topic: "SMART/HILD/GLOW_DEVICE_ID"
      unit_of_measurement: 'm³'
      state_class: 'total_increasing'
      device_class: 'gas'
      unique_id: 'home_gas'
      value_template: "{{ value_json['gasMtr']['0702']['00']['00']|int(base=16) * value_json['gasMtr']['0702']['03']['01']|int(base=16) / value_json['gasMtr']['0702']['03']['02']|int(base=16) }}"

    - name: "Home Daily Gas"
      state_topic: "SMART/HILD/GLOW_DEVICE_ID"
      unit_of_measurement: 'kWh'
      value_template: "{{ value_json['gasMtr']['0702']['0C']['01']|int(base=16)  * value_json['gasMtr']['0702']['03']['01']|int(base=16) / value_json['gasMtr']['0702']['03']['02']|int(base=16) }}"
      icon: 'mdi:flash'

    - name: "Home Weekly Gas"
      state_topic: "SMART/HILD/GLOW_DEVICE_ID"
      unit_of_measurement: 'kWh'
      value_template: "{{ value_json['gasMtr']['0702']['0C']['30']|int(base=16)  * value_json['gasMtr']['0702']['03']['01']|int(base=16) / value_json['gasMtr']['0702']['03']['02']|int(base=16) }}"
      icon: 'mdi:flash'

    - name: "Home Monthly Gas"
      state_topic: "SMART/HILD/GLOW_DEVICE_ID"
      unit_of_measurement: 'kWh'
      value_template: "{{ value_json['gasMtr']['0702']['0C']['40']|int(base=16)  * value_json['gasMtr']['0702']['03']['01']|int(base=16) / value_json['gasMtr']['0702']['03']['02']|int(base=16) }}"
      icon: 'mdi:flash'

If you would like to set up the energy dashboard, pick Home Energy for electricity and Home Gas for gas.

Now restart your Home Assistant instance, and you should be good to go! All the sensors should appear in the developer tools menu, for instance sensor.home_instant_energy, and update in real time.

If you'd like to play with the MQTT API, you can also check out my python code to connect and fetch data directly from the MQTT server.

Are you able to extract the cost information from MQTT also ? I have the gas and electric working in HA now via Hildebrand MQTT but the cost info isn't jumping out at me

@m3t4r0n, the MQTT feed doesn't include cost data as far as I know (apart from pre-payment meters). In Home Assistant I have a sensor which stores the kWh unit rate (e.g. 0.1654) with unit of measurement set to "EUR/kWh". This could be static and easily done with a template sensor or in my case it is pulled from Octopus API as the price is dynamic. I then use this with the new "Energy" platform in Home Assistant in order for it to calculate costs for me based on the kWh usage. You can set this in "Grid Consumption" and "Use an entity with current price" in the energy configuration page. I am not sure if the REST API from Glowmarkt has support for retrieving cost data.

@scooter1556 thank you for the GitHub link. Just installed it into Portainer on HA and it’s working well 👍🙂

Interesting....
`

• platform: mqtt
  name: "Home Gas Meter M3"
  state_topic: "SMART/HILD/############"
  last_reset_topic: "SMART/HILD/############"
  last_reset_value_template: homeassistant.util.dt.utc_from_timestamp(0)
  unit_of_measurement: m³
  value_template: "{{ value_json['gasMtr']['0702']['00']['00']|int(base=16) * value_json['gasMtr']['0702']['03']['01']|int(base=16) / value_json['gasMtr']['0702']['03']['02']|int(base=16) }}"
  icon: 'mdi:counter'
  device_class: gas
  state_class: total_increasing
  `
  Works for me....but on checking the numbers, the value the above gives me is total Kwh of gas, not m3 of gas....

I've used the below and although the electricity works, the gas is reporting the register value rather than the incremental/usage;
` - platform: mqtt
name: "Home Electricity Import Meter"
state_topic: "SMART/HILD/############"
last_reset_topic: "SMART/HILD/############"
last_reset_value_template: homeassistant.util.dt.utc_from_timestamp(0)
unit_of_measurement: 'kWh'
device_class: energy
state_class: total_increasing
value_template: "{{ value_json['elecMtr']['0702']['00']['00']|int(base=16) * value_json['elecMtr']['0702']['03']['01']|int(base=16) / value_json['elecMtr']['0702']['03']['02']|int(base=16) }}"
icon: 'mdi:counter'

• platform: mqtt
  name: "Home Electricity Export Meter"
  state_topic: "SMART/HILD/############"
  last_reset_topic: "SMART/HILD/############"
  last_reset_value_template: homeassistant.util.dt.utc_from_timestamp(0)
  unit_of_measurement: 'kWh'
  device_class: energy
  state_class: total_increasing
  icon: 'mdi:counter'
  value_template: "{{ (value_json['elecMtr']['0702']['00']['01']|int(base=16) * value_json['elecMtr']['0702']['03']['01']|int(base=16) / value_json['elecMtr']['0702']['03']['02']|int(base=16)) }}"

• platform: mqtt
  name: "Home Gas Meter M3"
  state_topic: "SMART/HILD/############"
  last_reset_topic: "SMART/HILD/############"
  last_reset_value_template: homeassistant.util.dt.utc_from_timestamp(0)
  unit_of_measurement: m³
  value_template: "{{ value_json['gasMtr']['0702']['00']['00']|int(base=16) * value_json['gasMtr']['0702']['03']['01']|int(base=16) / value_json['gasMtr']['0702']['03']['02']|int(base=16) }}"
  icon: 'mdi:counter'
  device_class: gas
  state_class: total_increasing`

Any idea's?

Just need to give it time to accumulate data I think - gas meters are read less frequently than electricity. You may not have a in-day delta to record yet. Your first delta is 0 -> 83,492

Add this for Gas kWh (change the formula to the one from your last gas bill) HA doesn't like 'gas' with a unit of 'kWh'

• platform: mqtt
  name: "Home Gas Meter kWh"
  state_topic: "SMART/HILD/XXXXXXXXXX"
  unit_of_measurement: 'kWh'
  value_template: "{{ ( ( states('sensor.home_gas_meter_m3') | int ) * 1.02264 * 38.9 / 3.6 ) | round(1) }}"
  icon: 'mdi:counter'
  device_class: energy
  state_class: total_increasing

I also added these. Outside the dedicated Energy dashboard, I graph the below to give a daily cycle/comparator

utility_meter:
electric_import:
source: sensor.home_electricity_import_meter
cycle: daily

electric_export:
source: sensor.home_electricity_export_meter
cycle: daily

gas_meter_m3:
source: sensor.home_gas_meter_m3
cycle: daily

gas_meter_kwh:
source: sensor.home_gas_meter_kwh
cycle: daily

I am clearly doing something wrong as I cannot get this to work.

What unit is the data coming from the meter in? kWh or m3? I'm assuming m3 as that is how the units are measured at the meter.

If that's the case, I cannot get the kWh readings into HA - if I use the m3 readings, they look nice on the graph but they are not feeding into the overall energy usage for the house (i.e. the total is only equal to the electricity readings). If I try and convert the m3 to kWh with a template sensor, every reading is for the full value rather than HA calculating incremental consumption as I showed in my previous screenshot - it doesn't matter how long you wait, the data doesn't show small increments, just the consumption added to whatever the first value was (which is 900,000kWh).

@norfolkmustard how is your gas sensor setup in the energy module? Which entity is it tracking?

Using your sensor above (also copied below) should this work with the energy module?

• platform: mqtt
  name: "Yet Another Home Gas Meter kWh"
  state_topic: "SMART/HILD/XXXXXXXXXX"
  unit_of_measurement: 'kWh'
  value_template: "{{ ( ( states('sensor.another_home_gas_meter') | int ) * 1.02264 * 38.9 / 3.6 ) | round(1) }}"
  icon: 'mdi:counter'
  device_class: energy
  state_class: total_increasing

I have the following block showing the values of the sensors I've configured, the bottom one is your code which is tracking the m3 sensor - the m3 sensor "works" to a certain degree, as in it shows in the graph and the values are in line with my expectations, but 1) the consumption doesn't feed into the overall household energy usage (I assume as there's a mismatch between kWh and m3) and I can't track costs without doing a convoluted calculation to work out the cost per m3.

The Energy dashboard needs the accumulating m³ sensor. For me that’s “sensor.home_gas_meter_m3”

Check DeveloperTools > Statistics for any issues with that sensor

I’m not tracking costs.

It seems that the gas readings coming from my meter (can they be supplier specific? I'm with e.on) are in kWh - the [0702][03][00] is set to [00] which from what I can gather means kWh (01 would be m3) so I'm probably on a hiding to nothing trying to get this to work until Home Assistant sort out allowing different units of measure.

It seems I can use both kWh and m3 (the latter makes no sense as the meter data is in kWh) but the reads seem to stop working at midnight and the cost data is screwed (and only works for kWh in any event).

Anyone else experiencing an mqtt disconnection from server?

Yeah, yesterday 1000 to 1900 GMT.

Also not getting any data back from api.glowmarkt.com this morning

Update: ah, api is working but a very slow response. Had a 2 sec timeout set and it wasn’t responding. Upped it to 20 secs and it’s responding now.

https://forum.glowmarkt.com/index.php?p=%2Fdiscussion%2Fcomment%2F442#Comment_442

Still turned off I think - no mqtt access at all for me.

Yeah, off again. Was working overnight, lost it around 0540

The "Home Gas" sensor seems to be returning kWh and not M3. Any ideas?

@BenHarris it will depend on what unit of measurement your gas meter is sending. Check line 21 of the following and compare to your MQTT values:
https://gist.github.com/ndfred/b373eeafc4f5b0870c1b8857041289a9
I have the opposite issue; meter reports in M3 but whenever I convert m3 to kWh everything goes screwy

Right, Table D-25 on page 236 of the smart energy profile spec (https://zigbeealliance.org/wp-content/uploads/2019/12/docs-07-5356-19-0zse-zigbee-smart-energy-profile-specification.pdf) says 0207 0x0300 shows the unit of measure. Mine is "00", which when referenced to Table D-26 shows 0x00 = kWh

@BenHarris so is the problem you want to store M3 or that the sensor is setup to capture in M3 but you want to correct it to capture in kWh?
If the latter, amend the sensor code to the following:

unit_of_measurement: 'kWh'
device_class: energy
state_class: total_increasing

I really just want to get the data into the energy dashboard. I deleted the m3 sensor, changed it to kWh as you said. This allowed me to add it to the gas part of the energy dashboard, although it has now been 24 hours, and no data has appeared. Wondering if the rogue m3 data is still in there somewhere causing problems

Check in Developer Tools > Statistics to see if there are issues it knows about

@BenHarris have you checked your sensor is using the correct register value for historic readings? For example the sensors listed above use [0C] however my own sensors need to use [04]. [0C] is the AlternativeHistoric
Eg above shows "{{ value_json['gasMtr']['0702']['0C']['01']|int(base=16) but my sensor has to be "{{ value_json['gasMtr']['0702']['04']['01']|int(base=16)

Also, another gotcha that I seem to be encountering is after a reboot my gas and electric sensor values go nuts. I’m trying to find out how to stop the reboot taking a zero value and then adding the meter value to old value eg gas was 0.4m3 before reboot. After reboot and allow the hour to pass it went to 24,000 m3.

@townsmcp I've double-checked the register values and they are definitely correct. I can also see the correct value in Developer Tools.

There are no issues listed under the statistics section for the gas sensor. I checked the db, and the old m3 statistics were still in there though (even though I had deleted the sensor), so I manually deleted those and restarted HA. I'll give it a couple of hours and see if that sorts it!

Also, another gotcha that I seem to be encountering is after a reboot my gas and electric sensor values go nuts. I’m trying to find out how to stop the reboot taking a zero value and then adding the meter value to old value eg gas was 0.4m3 before reboot. After reboot and allow the hour to pass it went to 24,000 m3.

I suspect this issue relates to using total_increasing, and a value not being available on the mqtt server yet. The announcement about the total_increasing state class says "a decreasing value is interpreted as the start of a new meter cycle or the replacement of the meter". I think using total instead might be a better option and prevent this from happening.

@BenHarris thanks for the advice. After deleting the M3 sensor and rebooting/ restarting manually configured mqtt sensors, did you check the statistics in dev, see the rogue long term stats, choose to delete the data, then add you new sensor with old sensor name? Maybe try a completely different sensor name?

Unfortunately I’m already using total in the sensor and still get the issue:

sensor:
  - platform: mqtt
    name: "Gas Meter M3"
    state_topic: !secret smart_hild_state_topic
    last_reset_value_template: homeassistant.util.dt.utc_from_timestamp(0)
    unit_of_measurement: m³
    value_template: "{{ value_json['gasMtr']['0702']['00']['00']|int(base=16) * value_json['gasMtr']['0702']['03']['01']|int(base=16) / value_json['gasMtr']['0702']['03']['02']|int(base=16) }}"
    icon: 'mdi:counter'
    device_class: gas
    state_class: total

Does any know how to drop specific value readings that come through in mqtt? I have been talking with Hildebrand about the spikes I keep getting, specifically in gas meter readings, and they have come back with:

`We confirm that the meter has returned a value of "FFFFFFFFFFFF", which for the data type means it's an invalid value.

The ingress to the Hildebrand database detects these invalid values and discards them so they don't appear in our APIs (and therefore Bright); we assume they do appear in the MQTT feed and so will need to be detected and discarded by MQTT users.`

it certainly explains why I’m getting gas readings at various times even though gas should only come through every half hour

I bring in the raw reading and then use a template as below to remove the zeros:

gas_meter_reading:
  friendly_name: "Gas Meter Reading"
  unit_of_measurement: 'kWh'
  value_template: >-
      {% if states("sensor.gas_meter_raw") | float(0) > 0 %}
      {{ states("sensor.gas_meter_raw") }}
      {%- else -%}
      {{ states("sensor.gas_meter_reading") }}
      {% endif %}

@pg-nut thanks. Being as "FFFFFFFFFFFF" in hex converts to 281474976710655 in decimal and Hildebrand have said they exclude "FFFFFFFFFFFF" from any readings, would the following then work (so that I am excluding that as well as 0)? Or would the register value be a signed 2's complement and therefore convert as n/a? If n/a, would that be a 0 value registered in HA?

gas_meter_reading:
  friendly_name: "Gas Meter Reading"
  unit_of_measurement: 'kWh'
  value_template: >-
      {% if states("sensor.gas_meter_raw") | float(0) > 0 %}
      {{ states("sensor.gas_meter_raw") }}
      {% elif states("sensor.gas_meter_raw") | float(0) == 281474976710655 %}
      {{ states("sensor.gas_meter_reading") }}
      {%- else -%}
      {{ states("sensor.gas_meter_reading") }}
      {% endif %}

When I was having problems, I looked at my “raw” sensor history and could see that every night it spiked down to zero and that was why I setup my template that way. Maybe you could see what your rogue value is in the same way and adjust the template accordingly?

It’s been a while since I have checked in here, nice to see people are still fixing issues!

I’ll refresh my config soon and expect to make the following changes, let me know if there is anything else you’d like to see or stuff I should know about:

• share my energy dashboard configuration which took WAY too long to set up properly
• leverage the new local MQTT connection to hopefully improve privacy / latency / resilience
• filter out spikes and invalid values in the gas measurements in particular