Skip to content

Instantly share code, notes, and snippets.

@rbellamy
Last active August 29, 2015 14:19
Show Gist options
  • Save rbellamy/3f7388c56af6bc266769 to your computer and use it in GitHub Desktop.
Save rbellamy/3f7388c56af6bc266769 to your computer and use it in GitHub Desktop.
IPMI troubleshooting and threshold configuration

Run the command

$ ipmitool sensor list all  
CPU1 Temp        | 0x0        | discrete   | 0x0000| na        | na        | na        | na        | na        | na        
CPU2 Temp        | 0x0        | discrete   | 0x0000| na        | na        | na        | na        | na        | na        
System Temp      | 38.000     | degrees C  | ok    | -9.000    | -7.000    | -5.000    | 75.000    | 77.000    | 79.000    
CPU1 Vcore       | 0.976      | Volts      | ok    | 0.808     | 0.816     | 0.824     | 1.432     | 1.440     | 1.448     
CPU2 Vcore       | 0.920      | Volts      | ok    | 0.808     | 0.816     | 0.824     | 1.432     | 1.440     | 1.448     
CPU1 DIMM        | 1.520      | Volts      | ok    | 1.288     | 1.296     | 1.304     | 1.656     | 1.664     | 1.672     
CPU2 DIMM        | 1.520      | Volts      | ok    | 1.288     | 1.296     | 1.304     | 1.656     | 1.664     | 1.672     
+1.1 V           | 1.104      | Volts      | ok    | 0.960     | 0.968     | 0.976     | 1.216     | 1.224     | 1.232     
+1.8 V           | 1.832      | Volts      | ok    | 1.592     | 1.600     | 1.608     | 1.976     | 1.984     | 1.992     
+5 V             | 4.992      | Volts      | ok    | 4.416     | 4.448     | 4.480     | 5.536     | 5.568     | 5.600     
+12 V            | 12.190     | Volts      | ok    | 10.600    | 10.653    | 10.706    | 13.250    | 13.303    | 13.356    
VTT              | 1.144      | Volts      | ok    | 0.808     | 0.816     | 0.824     | 1.512     | 1.520     | 1.528     
+3.3 V           | 3.264      | Volts      | ok    | 2.880     | 2.904     | 2.928     | 3.648     | 3.672     | 3.696     
+3.3VSB          | 3.336      | Volts      | ok    | 2.880     | 2.904     | 2.928     | 3.648     | 3.672     | 3.696     
VBAT             | 3.168      | Volts      | ok    | 2.448     | 2.472     | 2.496     | 3.648     | 3.672     | 3.696     
FAN 1            | 576.000    | RPM        | cr    | 400.000   | 576.000   | 784.000   | 33856.000 | 34225.000 | 34596.000 
FAN 2            | 484.000    | RPM        | cr    | 400.000   | 576.000   | 784.000   | 33856.000 | 34225.000 | 34596.000 
FAN 3            | 576.000    | RPM        | cr    | 400.000   | 576.000   | 784.000   | 33856.000 | 34225.000 | 34596.000 
FAN 4            | 625.000    | RPM        | nc    | 400.000   | 576.000   | 784.000   | 33856.000 | 34225.000 | 34596.000 
FAN 5            | 625.000    | RPM        | nc    | 400.000   | 576.000   | 784.000   | 33856.000 | 34225.000 | 34596.000 
FAN 6            | 576.000    | RPM        | cr    | 400.000   | 576.000   | 784.000   | 33856.000 | 34225.000 | 34596.000 
FAN 7            | 900.000    | RPM        | ok    | 400.000   | 576.000   | 784.000   | 33856.000 | 34225.000 | 34596.000 
FAN 8            | 1024.000   | RPM        | ok    | 400.000   | 576.000   | 784.000   | 33856.000 | 34225.000 | 34596.000 
Intrusion        | 0x1        | discrete   | 0x0100| na        | na        | na        | na        | na        | na        
PS Status        | 0x1        | discrete   | 0x01ff| na        | na        | na        | na        | na        | na        

The thresholds listed are, in order: lnr, lcr, lnc, unc, ucr, unr

These are acronyms for:

Lower Non-Recoverable

Lower Critical

Lower Non-Critical

Upper Non-Critical

Upper Critical

Upper Non-Recoverable

{ ipmitool sensor thresh FAN 1 lower 780.0 785.0 789.0 2>&1 && ipmitool sensor get FAN 1 2>&1 }
Locating sensor record 'FAN 1'...
Setting sensor "FAN 1" Lower Non-Recoverable threshold to 780.000
Setting sensor "FAN 1" Lower Critical threshold to 785.000
Setting sensor "FAN 1" Lower Non-Critical threshold to 789.000
Locating sensor record...
Sensor ID              : FAN 1 (0xf)
 Entity ID             : 29.1 (Fan Device)
 Sensor Type (Threshold)  : Fan (0x04)
 Sensor Reading        : 3969 (+/- 0) RPM
 Status                : ok
 Nominal Reading       : 3969.000
 Normal Minimum        : 1024.000
 Normal Maximum        : 33124.000
 Upper non-recoverable : 34596.000
 Upper critical        : 34225.000
 Upper non-critical    : 33856.000
 Lower non-recoverable : 144.000
 Lower critical        : 289.000
 Lower non-critical    : 441.000
 Positive Hysteresis   : 1.000
 Negative Hysteresis   : 1.000
 Minimum sensor range  : Unspecified
 Maximum sensor range  : Unspecified
 Event Message Control : Per-threshold
 Readable Thresholds   : lnr lcr lnc unc ucr unr 
 Settable Thresholds   : lnr lcr lnc unc ucr unr 
 Threshold Read Mask   : lnr lcr lnc unc ucr unr 
 Assertion Events      : 
 Assertions Enabled    : lnc- lcr- lnr- unc+ ucr+ unr+ 
 Deassertions Enabled  : lnc- lcr- lnr- unc+ ucr+ unr+

What is going on here? Clearly the threshold settings are not being accepted as-is.

{ ipmitool sensor thresh FAN 1 lnc 780.0 2>&1 && ipmitool sensor get FAN 1 2>&1 }
Locating sensor record 'FAN 1'...
Setting sensor "FAN 1" Lower Non-Critical threshold to 780.000
Locating sensor record...
Sensor ID              : FAN 1 (0xf)
 Entity ID             : 29.1 (Fan Device)
 Sensor Type (Threshold)  : Fan (0x04)
 Sensor Reading        : 3969 (+/- 0) RPM
 Status                : ok
 Nominal Reading       : 3969.000
 Normal Minimum        : 1024.000
 Normal Maximum        : 33124.000
 Upper non-recoverable : 34596.000
 Upper critical        : 34225.000
 Upper non-critical    : 33856.000
 Lower non-recoverable : 144.000
 Lower critical        : 289.000
 Lower non-critical    : 144.000
 Positive Hysteresis   : 1.000
 Negative Hysteresis   : 1.000
 Minimum sensor range  : Unspecified
 Maximum sensor range  : Unspecified
 Event Message Control : Per-threshold
 Readable Thresholds   : lnr lcr lnc unc ucr unr 
 Settable Thresholds   : lnr lcr lnc unc ucr unr 
 Threshold Read Mask   : lnr lcr lnc unc ucr unr 
 Assertion Events      : 
 Assertions Enabled    : lnc- lcr- lnr- unc+ ucr+ unr+ 
 Deassertions Enabled  : lnc- lcr- lnr- unc+ ucr+ unr+ 
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment