[rtg] sol table insert woes

[Najarian, Michael]

Have been running rtg for > 3 years now. About a month ago, data for a
few specific interfaces on a few specific routers would no longer get
inserted into mysql tables. Example: I'm not getting updates on
ifInOctets on ifIndex.12 on router 312. But I am getting
if[In|Out]UcastPkts, ifInErrors, and in some cases even ifOutOctets on
interface #12, and I'm getting all 5 OIDs on interfaces 1-11 and 13-15.
The tables are fine, because they'll take inserts on all the other
interfaces on those routers/switches. I'm pretty sure the poller is
coming back with data on the OIDs that arent inserting because a) I can
walk them by hand and get the counters, and b) I've run separate
non-inserting, spit-to-a-file instances of rtgpoll against the devices
and get valid data back on all OIDs for all interfaces. 

I've myisamchk'd the affected tables. I've even pruned some of 3 year
old data and reindexed them. I can insert data by hand. If there were
problems with the tables, or some odd disk problem, I'd expect the whole
table to be b0rked and not take inserts on ANY interfaces. But it's
always the same OIDs on the same interfaces.

Restarted poller, mysql, even rebooted the sun (985 day uptime,
*sniff*). When I restart mysql, I sometimes will get a couple inserts on
the affected interfaces, sometimes not. But only a few inserts. Which is
why I'm tempted to blame mysql.

Load on the box hovers around .35, data partition is well under 90%, and
some of the affected tables are relatively tiny (15Mb). solaris 8 +
mysql 3.23.56 + rtg 0.7.4 

Any ideas?

Mike

Mike Najarian, MTS
US LEC
voice / data / Internet
(484) 893-1606 
mnajarian at uslec.com
www.uslec.com