The new consolidated Asterisk machine has been running well, but a recent outage shows the potential folly of (still) using Gentoo as a server base.
The new machine was not in any of the Cacti graphing, so I decided to set it up on Friday night. net-snmp had been installed, but I must have removed it in an overly-agressive cleanup (when I transferred the desktop built onto it). So I rebuilt net-snmp, configured and started it, added the box to Cacti, and was happy.
The following night, Susan told me that her mum had tried to phone in the morning and it rang out. At first we figured we were actually out, but the time was wrong. Looking in the Asterisk CDR log, I saw a call of 0:00 duration, marked as “NO ANSWER”. Picked up the mobile and dialled home, and sure enough no ringing. What was worse was that I could not even dial between extensions. Hmmm.
Googling turned up little response, and neither did turning up the log level. I could find that a curious AGI script called “dialparties.agi” was telling the dialplan that all available extensions were busy (which they certainly were not). By the output, it looked like the script was not running correctly… I found that there was an update to FreePBX available, and thinking that I might have tickled some bug all of a sudden I tried to install it. The installation script produced a cryptic error, something like the following:
/usr/bin/php: libnetsnmp.so.5: cannot open shared object file: No such file or directory
Suddenly the soup got clearer! Rebuilding net-snmp probably brought in a version upgrade; PHP was linked against net-snmp, and the library was gone. But the whole FreePBX interface is PHP code, and it was working fine… it would seem that Apache mod_php was either rebuilt after the net-snmp build or does not link against net-snmp, because any PHP code in web pages was fine. And sure enough, the dialparties.agi script is command-line PHP.
So rebuilding PHP fixed the problem! Now I know some more about FreePBX — it is totally broken without a functioning command-line PHP. PHP in turn is dependent on net-snmp… and who knows where the chain goes after that…
I probably need to put this kind of stuff on my Wiki, as it will likely catch me out again. Maybe I need to get in the habit of running revdep-rebuild a bit more often (or compile a list of those packages that revdep-rebuild should *always* be run after).
EDIT: Zoinks! revdep-rebuild has a list of about 50 packages that are broken… Sigh 🙂
Crossed Wires 