IPMI data to SNMP
I’ve been monitoring CPU, Ram, HDD, and network usage using cacti for a while, but recently I’ve wanted to monitor temperature and fan data. A couple of servers have IPMI on them, which I can use to access the temperature and fan data, however there doesn’t seem to be an easy way of getting this into cacti.
I started off following IPMI Sensor Data on Dell 1850s and 2850s via SNMP and Cacti by Barry O’Donovan; however that is now a bit old, and it seems that a couple of things seem to have changed.
I use Ubuntu (10.04 LTS) on my servers, but the main idea should be pretty transferable.
Firstly you need the ipmi packages installed, there are multiple guides for this, but the basic process is install the packages, and then enable the required modules.
Once you have ipmi working, and $ ipmitool sensor outputting some data, we can continue.
The first bit is basically the same, but with some additions
Set up a cron job to put the ipmi data in a file /etc/cron.d/ipmitool; This is done every 5 mins, matching Cacti
*/5 * * * * root /usr/bin/ipmitool sensor >/tmp/ipmisensor-snmp
Note If ipmitool sensor runs quite slowly, you may want to modifiy the cron job to run at an offset time, I had to do this on one of my servers:
2,7,12,17,22,27,32,37,42,47,52,57 * * * * root /usr/bin/ipmitool sensor >/tmp/ipmisensor-snmp
next write a couple of scripts to get the appropriate data from that file. I have had to make a couple of modifications, and had to make a couple of extra files, I’ll explain why in a moment;
/usr/local/sbin/ipmi-fan-stats
#! /bin/sh PATH=/usr/bin:/bin STATS=/tmp/ipmisensor-snmp printf "%fn" `cat $STATS | grep Fan | cut -s -d "|" -f 2` 2> /dev/null
/usr/local/sbin/ipmi-fan-descs
#! /bin/sh PATH=/usr/bin:/bin STATS=/tmp/ipmisensor-snmp printf "%sn" `cat $STATS | grep Fan | sed 's/ //g' | cut -n -d "|" -f 1` 2> /dev/null
/usr/local/sbin/ipmi-fan-id
#! /bin/sh echo "1n2n3n4";
You may be asking why there is a script that just echos 1,2,3,4 on new lines. I’ll explain why in a moment
Once you have those in place (I’ve also got duplicates for temperature, I just changed what was being grep’d for (degrees instead of Fan) we need to tell SNMP about it. Again this is slightly different;
/etc/snmp/snmpd.conf
extend ipmitemp /usr/local/sbin/ipmi-temp-stats extend ipmitempdesc /usr/local/sbin/ipmi-temp-descs extend ipmitempid /usr/local/sbin/ipmi-temp-id extend ipmifan /usr/local/sbin/ipmi-fan-stats extend ipmifandesc /usr/local/sbin/ipmi-fan-descs extend ipmifanid /usr/local/sbin/ipmi-fan-id
restart snmpd ($ sudo service snmpd restart), and assuming the cron job has run at least once we can check if we can get the data via snmp
$ snmpwalk -c <community> -v <version> <address> 'NET-SNMP-EXTEND-MIB::nsExtendOutLine' NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifan".1 = STRING: 2178.649000 NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifan".2 = STRING: 0.000000 NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifan".3 = STRING: 1167.134000 NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifan".4 = STRING: 1083.306000 NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifanid".1 = STRING: 1 NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifanid".2 = STRING: 2 NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifanid".3 = STRING: 3 NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifanid".4 = STRING: 4 NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifandesc".1 = STRING: CPU0Fan NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifandesc".2 = STRING: CPU1Fan NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifandesc".3 = STRING: PCIFan NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifandesc".4 = STRING: RearFan .....
Why the extra scripts?
Cacti doesn’t appear to like reading data in without an id for that data. At first I tried something that gets the id from the OID, however that didn’t work, so I ‘spoofed’ some ids to make it work. The important thing is to make sure the number of ids you echo match the number of fan/temperature readings you have. In my case I have 4 fans, and 13 tempurature sensors, so I output 1 to 4 for the fans, and 1 to 13 for the tempuratures.
The descriptions (descs) scripts are really optional, I just included them to make it easier to identify the readings in cacti.
SNMP data to cacti graph
First of all you need to make a snmp query interface. Below is mine
/usr/share/cacti/resource/snmp_queries/ipmifan.xml
<interface>
<name>Get ipmifan Information</name>
<description>Get SNMP based ipmifan</description>
<oid_index>NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifanid"</oid_index>
<index_order>ipmifanDesc:ipmifanIndex</index_order>
<index_order_type>numeric</index_order_type>
<index_title_format>|chosen_order_field|</index_title_format>
<fields>
<ipmifanIndex>
<name>Index</name>
<method>walk</method>
<source>value</source>
<direction>input</direction>
<oid>NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifanid"</oid>
</ipmifanIndex>
<ipmifanDesc>
<name>Description</name>
<source>value</source>
<direction>input</direction>
<method>walk</method>
<oid>NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifandesc"</oid>
</ipmifanDesc>
<ipmifanValue>
<name>Value</name>
<method>walk</method>
<source>value</source>
<direction>input</direction>
<oid>NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifan"</oid>
</ipmifanValue>
<ipmifanOut>
<name>fan</name>
<method>walk</method>
<source>value</source>
<direction>output</direction>
<oid>NET-SNMP-EXTEND-MIB::nsExtendOutLine."ipmifan"</oid>
</ipmifanOut>
</fields>
</interface>
The rest we can now do in the cacti web interface
Rather than list all the steps here you can now follow http://docs.cacti.net/manual:087:3a_advanced_topics.3a_snmp_data_queries#snmp_data_queries, replacing where appropriate.
Hopefully you should now have some pretty graphs like this!
Here are my graph templates if they are any help for anyone, but I preferred making my own graphs