Datapoint Aggregator Example

datapoint-aggregator-example.markdown

Aggregating Network Throughput

The Calculated Performance (ZenPacks.zenoss.CalculatedPerformance) ZenPack adds two new datasources types to Zenoss: Calculated Performance, and Datapoint Aggregator.

The Calculated Performance datasource type allows datapoints to be collected that are derived from numeric model properties and the most recent value of other datapoints on the same device or component. This means that only model properties and datapoint values for the device or component the template is bound to can be used to calculated the derived datapoint value.

Example usages of the Calculated Performance datasource type:

• (cpuUser + cpuSystem) = cpuUsage
• (usedBytes / totalBytes) * 100 = usedPercent

The Datapoint Aggregator datasource type addresses the single device or component restriction of the Calculated Performance datasource type. It allows the following aggregation operations such as count, sum, avg, min, and max to be used on many device or components that have a datapoint by the same name.

Example usages of the Datapoint Aggregator datasource type:

• sum(ifInOctets on all interfaces)
• sum(ifInOctets on some interfaces)

Example: Total Network Throughput on All Interfaces

Let's walk through an example of using a Datapoint Aggregator datasource to collect and graph the total network throughput on all of a device's interfaces. We'll use a Cisco C2960 as an example.

1. Create a new monitoring template named Network SNMP in the /Network device class.

2. Add a Datapoint Aggregator datasource named ifHCInOctets.

   1. Edit the ifHCInOctets datasource, set the following fields, then save.

      • Method: os.interfaces
      • Datasource: ifHCInOctets
      • Datapoint: ifHCInOctets

   2. Add a datapoint named sum to the ifHCInOctets datasource.

   3. Edit the ifHCInOctets.sum datapoint, set the following fields, then save.

      • Operation: sum

3. Repeat step 2 replacing ifHCInOctets with ifHCOutOctets.

4. Add a graph named Total Network Throughput.

   1. Edit the Total Network Throughput graph, set the following fields, then save.

      • Units: bits/sec
      • Min Y: 0

   2. Choose Manage Graph Points from the Graph Definitions gear menu.

   3. Click the + to add both datapoints to the graph.

   4. Edit each graph point, set the following fields, then save.

      • Name: Inbound (or Outbound for ifHCOutOctets_sum)
      • Format: %7.2lf%s
      • RPN: 8,*

      The RPN multiplies the value by 8 to convert from octets (bytes) to the bits that we want to display.

   5. Click Save on the Manage Graph Points dialog.

5. Bind the Network SNMP template to an appropriate device or device class.

   For testing, I'll just bind it to a single device. Once it's working the way we want on our one device we might want to bind it to an entire device class.

Example: Total Network Throughput on Some Interfaces

Aggregating network throughput across all interfaces was easy enough, but what if we only wanted to get the total throughput of some interfaces we know to be used for a specific purpose. Perhaps just uplink interfaces, or something like that. It can be done, but it's a bit more involved.

The only thing that's different for aggregating some interfaces instead of all interfaces is the method defined in our datapoint aggregator datasource. Referring to all interfaces was easy, we just used "os.interfaces". Refering to a group of potentially arbitrary interfaces will require a way for us to define which interfaces are part of the group. We'll also have to create a script that returns just those interfaces instead of all interfaces. This script will become the method for our datapoint aggregator datasource.

Create a Custom Property that Defines an Interface Group

1. Create the new custom property.

   1. Navigate to the Infrastructure page.

   2. Click the Details button at the top of the left pane with the root DEVICES selected.

   3. Click Custom Schema on the left pane.

   4. Add a property with the following fields set.

      • Label: Interface Group 1
      • Name: cInterfaceGroup1
      • Type: lines

2. Set a value for the custom property on a device.

   1. Navigate to your test device and click Custom Properties on the left pane.

   2. Set the value of Interface Group 1 to something like the following.

      GigabitEthernet0_1
      GigabitEthernet0_2
      

      Note that it is one interface id per line. You'll note that the interface id is slightly different than its name. For example, the forward slash (/) has been replaced with an underscore (_). The best way to make sure you have the id correct is to navigate to the interface in the web interface, then look at the end of the brower's URL bar.

Create a Custom Script that Returns Interfaces in Interface Group 1

1. Navigate to http://zenoss.example.com/zport/dmd/Devices/manage.

   The Manager role is required to do this.

2. Choose Script (Python) from the Add drop-down near the top-right.

3. Set Id to in_cInterfaceGroup1 then click Add and Edit.

4. Replace the contents of the big text area with the following.

   interfaces = []
   
   names = getattr(context, 'cInterfaceGroup1', None)
   if names:
       for name in names:
           if name:
               try:
                   interfaces.append(context.getObjByPath('os/interfaces/{}'.format(name)))
               except Exception:
                   pass
   
   return interfaces

5. Click Save Changes.

6. You can test that the script works in zendmd as follows.

   find("device-id").in_cInterfaceGroup1()

Update the Datapoint Aggregator Datasources

Now the only thing left to do is to update the datasources in our Network SNMP monitoring template to use in_cInterfaceGroup1 as their method instead of os.interfaces. You might also want to change the graph's name from Total Network Throughput to something like Total Uplink Network Throughput.

To aggregate across devices use a script that looks something like:

interfaces=[]

dmd=context.getDmd()
for device in dmd.Devices.Network.Cisco.getSubDevices():
  interfaces.extend(device.in_cInterfaceGroup1())
return interfaces

Obviously I've used the same cProp, and have selected everything under the /Devices/Network/Cisco branch - both of those can be changed according to your needs.

Hi!
Is there any way to refer to the current (calling) device.
I would like to write the dmd-script to address the deviceClass of the aggregating device rather than one hard coded DeviceClass.
Thanks in advance,
g

Is there any way to refer to the current (calling) device.
I would like to write the dmd-script to address the deviceClass of the aggregating device rather than one hard coded DeviceClass.

@gekr: context is the calling device in this case of this script. To get the calling device's device class you would use context.deviceClass().primaryAq().

Hi,

Is this part accessible with Zenoss Cloud?

Navigate to http://zenoss.example.com/zport/dmd/Devices/manage.

I tried to access on our instance but getting an "This XML file does not appear to have any style information associated with it. The document tree is shown below." message.

I have CZ admin permissions

Thanks
Matt (Gamma)