Unavailable Sensor Detection and Notification

package_unavailable_entities.yaml

#######################################################################################################################
# The Unavailable Entities Sensor Package has been moved to it's own repository!
# https://github.com/jazzyisj/unavailable-entities-sensor
#######################################################################################################################

Hey guys, thanks for this awesome package and for all of your contributions! I've been looking for something similar that fits my needs better, so I've built a similar package that has some cool benefits and wanted to share it here, since there are many people with the same goal in the comments section.

Pros:

1. The best part: A group of whitelisted entities that can be specifically monitored. If you want an entity to be monitored, all you need to do it add it to the group!
2. The template trigger for the automation is smart: it only watches the entities that are in the whitelist group, and not all of the entities in HA. This also has a benefit in performance, because when a trigger listens to all entities or an entire domain, HA throttles the automation (to something like once per minute or 30 seconds). This does not happen here - notifications could be virtually immediate, if you want them to be.
3. Has a built-in 10 seconds delay so that if multiple entities go unavailable in a matter of seconds - only 1 notification will be sent.
4. Creates a text summary of entities that have recently gone online or offline (and sends it as a notification via telegram). The friendly name of the entities is used then building the summary.
5. If there are still unavailable entities from before, they are added to the notification so that it reminds you they're still offline.
6. Creates an "unknown entities" sensor so that you could use it in your UI, if you'd like.

Cons:

1. It's internally based on a template sensor, and HA limits its state to 255 chars. So if you have many entities going unavailable at one time, it will be truncated and you won't see the full list. This is a small price to pay, because if your entire house goes down, it's probably not a specific device that went offline, but probably a power outage.

The package is available here:
https://github.com/shlomki/HomeAssistant-Config/blob/main/packages/Automations/notify_when_entities_unavailable.yaml

I'd love to get some feedback and know if you can think of any improvements!

This is great and what I was looking for, thanks! Is there anyway to modify it to only show if an entity has been unavailable for XX seconds/minutes? I have a couple of entities that often go unavailable on and off for a few seconds (ex. robotic vacuum when docked) which is fine, but I'd want to know if it became persistently unavailable.

@Snuffy2 - That was actually a really good suggestion that also solved the template loop problem. See updated sensor.

Thank you for this great sensor template :)

Two feature requests:

1. Is there a way to ignore a device? For example, I have a Wemos D1 with which I test various ESPHome components, and I'd love to ignore all the entities it creates. I know its possible to {{ expand(name_of_group) }} to get a list of entities in that group; if it were possible, to supply a list of devices to ignore, and pull all entities associated with that device.
2. Wildcard support?... Is there a startswith filter in Jinja? As in:

ignored_entities:
  entities:
    - sensor.test*

and then using something like | rejectattr('entity_id','match',...)

Thanks!

I'm happy to find this template, its almost exactly what i wanted..
I cant figure out, is there any way to invert the working scheme of this template ? I have lot of esphome sensors, if a controller goes down, the sensors which are connected to this hardware are going down as well. So i need notifications for few sensor groups.

@amitkeret
There is no way to use globs or wildards in the group. You can however modify the template.
Try this, modifying it for your specific needs.

        {% set ignore_time = 15 %}
        {% set unavail = namespace(entities=[]) %}
        {% set entities = states|rejectattr('domain','eq','group')
          |rejectattr('entity_id','in',state_attr('group.ignored_entities','entity_id'))
          |selectattr('state','in',['unavailable','unknown','none'])|map(attribute='entity_id')|list %}
        {% for item in entities %}
          {% if states[item].last_changed.timestamp() < now().timestamp() - ignore_time
              and 'sensor.test' not in item 
              and 'esphome' not in item %}
            {% set unavail.entities = unavail.entities + [item] %}
          {% endif %}
        {% endfor %}
        {{ unavail.entities }}

@ant7511 Are you asking for a way to ignore all the associated esphome sensors if a controller goes down? If so see my reply to amitkeret.
If you are not, I'm not getting what you are trying to do, can you clarify?

It goes through all entities, except group.ignored_entities.
Is there any way to scan a specific group , so ignore all, except a specific group ?
So an inverted function. I see, i have to modify the rejectattr and selectattr area but i cant figure out how.

@ant7511

Create a group with the entities you want to monitor and change rejectattr to selectattr.

            {% set ignore_time = 15 %}
            {% set unavail = namespace(entities=[]) %}
            {% set entities = states|rejectattr('domain','eq','group')
              |selectattr('entity_id','in',state_attr('group.<NEW_GROUP_NAME>','entity_id'))
              |selectattr('state','in',['unavailable','unknown','none'])|map(attribute='entity_id')|list %}
            {% for item in entities %}
              {% if states[item].last_changed.timestamp() < now().timestamp() - ignore_time %}
                {% set unavail.entities = unavail.entities + [item] %}
              {% endif %}
            {% endfor %}
            {{ unavail.entities }}

It works like a charm :) thank you :)

Great project and it works great. I am getting template log warnings and notice in the code the ignore_time does not appear and the comments say that if it is set to less than 5 secs we will see these errors. Where should I add the ignore_time. Also just a heads up i was also getting a warning from unavailable_entities_alert sensor regarding int value and found modifying the line to include a default value of zero stopped the error as below
sensors:
unavailable_entities_alert:
friendly_name: Unavailable Entities Alert
delay_on:
seconds: 30 # delay before activating alert
value_template: "{{ states('sensor.unavailable_entities')|int(default=0) > 0 }}"
Once again thanks for a great project

@Bytelink5616 I'm getting the same warning. I've tried several things, but I could not get it to go away.

@Bytelink5616 I think you are using an old version of the template. Try the current version in the gist at the top of this page. If you're still having issues with this version let me know.

@pmd5700 I just started getting the warnings again also, but only when I reload my templates. Something must have changed, I'm looking into it. However, these are just warnings and they do not affect the sensor. You can use the log filter code given in the package to prevent the warnings from showing up in the home assistant log.

Are you using the most recent version?

@jazzyisj Here is the full warning. I just want to make sure it doesn't break in 2022.1. Yes, I actually just updated to the most recent version. I don't recall if I was getting this warning prior to that.

Template warning: 'int' got invalid input 'unknown' when rendering template '{{ states('sensor.unavailable_entities')|int > 0 }}' but no default was specified. Currently 'int' will return '0', however this template will fail to render in Home Assistant core 2022.1

Template warning: 'int' got invalid input '' when rendering template '{{ states('sensor.unavailable_entities')|int > 0 }}' but no default was specified. Currently 'int' will return '0', however this template will fail to render in Home Assistant core 2022.1

@pmd5700 - The code you are referring to does not exist in the most current version. Where are you copying the code from?

@jazzyisj I'm using this to trigger my automation:

binary_sensor:
  - platform: template
    sensors:
     unavailable_entities_alert:
        delay_on:
          seconds: 60 # delay before activating alert
        value_template: "{{ states('sensor.unavailable_entities')|int > 0 }}"

@pmd5700

That is not part of the package in this gist. That does look suspiciously like some template sensors I have in my personal config though..lol.
Anyway yes, you can just add a default value for int there. That should only happen on startup or when you're reloading templates.

value_template: "{{ states('sensor.unavailable_entities')|int(0) > 0 }}"

I use alerts for my notifications instead of an automation which is why I had to create the binary sensors in addition to the unavailable entities sensor. You could just use a numeric_state trigger if you're using an automation, you don't need the binary sensor template.

trigger:
 - platform: numeric_state
   entity_id: sensor.unavailable_entities
   above: 0

As for the template loop warnings - I'm not seeing them anymore again.
What is happening when you see that warning in your logs? Startup? Reloading templates?

@jazzyisj Thanks! I think I misread Bytelink5616's comment above. I was only getting the errors I posted above about the Int. I added the (0) and they seem to have gone away.

It should look familiar, I think you help me set it up ~a year ago lol. I tried the numeric state trigger, but I wanted to put a delay in the binary sensor so it didn't trigger all the time.

@pmd5700 Too funny..lol

This will give you your delay. Also if you're using the most current version you can increase the ignore_sec value to accomplish pretty much the same thing.

trigger:
 - platform: numeric_state
   entity_id: sensor.unavailable_entities
   above: 0
   for: 60

The Unavailable Entities Sensor has been moved to it's own package.

Please ask any further questions or report issues here.
https://github.com/jazzyisj/unavailable-entities-sensor

@massive

I have a similar issue as some others: I have a few devices that expose multiple sensors. I thought I could extend the sensor in a way that it would allow excluding sensors using regular expression. However, it seems that there is no elegant way to do that with HA and Jinja:

This feature is now available via the search test. The README in the new repository explains how to utilize it. I remembered your comment because it was the basis for my first failed attempt at accomplishing it with custom attributes so I thought I'd point it out to you.

@amitkeret

| rejectattr('entity_id','match',...)

YES! Home Assistant finally supports this! I finally got around to implementing it on this sensor (well, more like documenting it).

@Snuffy2

Is there anyway to modify it to only show if an entity has been unavailable for XX seconds/minutes?

I've updated the sensor and this is now accomplished with a much more elegant method. I recommend updating to the new method.

Jazzy, Just wanted to let you know that the package has been working like a charm and I wanted to thank you for making it available to all of us. I put in place the light only entities as you explained to me and it works like a charm. I have been reading the notes which are very detailed but because I am still learning about HA as I transitioned from smartthings and a quick question id I can. My unavailable entities show the lights and their groups as well as all the others as you would expect. I know I can enter them all one by one in the ignored_unavailable_entities group but is there a way to enter them using a wild card like approach foer example light.* or light.group_*. Anyway I really just wanted to thank you for al your help Ian This message is confidential. It may also be privileged or otherwise protected by work product immunity or other legal rules. If you have received it by mistake, please let us know by e-mail reply and delete it from your system; you may not copy this message or disclose its contents to anyone. The integrity and security of this message cannot be guaranteed on the Internet.

@Bytelink5616 My pleasure! I'm just paying forward all the help I've received along the way :-)

Can you visit the new repository for the sensor and check out the READ.ME? I've added quite a bit of documentation there.
If you still have a question then feel free to open an issue there so others can benefit.

https://github.com/jazzyisj/unavailable-entities-sensor