[rtg] rtg status?

[Matt Simerson]

On Jan 7, 2009, at 10:18 AM, Drew Weaver wrote:

> The poller is pretty great for RTG, I just wish the target maker was  
> a little smarter.
>
> It would be highly beneficial if you could do something like this  
> with the targetmaker:
>
> When a ports data is 'no longer needed' a flag would be set on the  
> row in the table for that port which tells rtargmkr the next time it  
> runs to move the data for that port into another table or  
> ([archiving] database altogether) then after the data is archived,  
> if the target maker sees that the port comes 'back' it just creates  
> a new entry for it. Which would be a good way of cleaning data which  
> is obviously useless out of the database.
>
> Like Matt said, the biggest problem we've had with rtg is "what to  
> do with all that data" as deleting hundreds of thousands of rows a  
> few times a day in MySQL is not terribly efficient.
>
> -Drew

I had considered doing something quite similar. Periodically, taking  
the RTG data out of the tables and into compressed read-only MyISAM  
tables.  That would dramatically reduce the disk space required by old  
data, while keeping it all 'available'. However, that also creates a  
number of problems. First, it only removes a small portion of the  
data. You still have the "gobs and gobs of data" problem.

How would you view bandwidth graphs for old data? The graphing apps  
only understand one DSN.  The reports, graphing scripts, and other  
tools would all need to be updated to support fetching data for  
various periods from their respective databases. That's a major PITA.

By consolidating ALL older data, it becomes fairly cheap to have many,  
many months of data available. Graphs continue to work normally,  
reports can be re-run on older data, etc.

Matt



> -----Original Message-----
> From: rtg-bounces at lists.grdata.com [mailto:rtg-bounces at lists.grdata.com 
> ] On Behalf Of Matt Simerson
> Sent: Wednesday, January 07, 2009 1:05 PM
> To: rtg at lists.grdata.com
> Subject: Re: [rtg] rtg status?
>
>
> Or they are just running the latest version of RTG and are quite
> content with the way it works. When I was a Layered Tech, we monitored
> several thousand switch ports with it. We use it at Spry as well, but
> only as a reference. I only have a few problems with the released
> version of RTG:
>
>    1. if the RTG poller loses the connection to the DB, it does not
> reconnect. Ever.
>    2. the perl scripts for reporting sucked.
>    3. databases get absolutely enormous after a few months worth of
> polling N thousand ports
>
> The solution for 1 is easy.  The application could be fixed to test
> the DB connection and reconnect after failures, or it can worked
> around. Just write a script that checks the last time a DB insert was
> made. If it has been more than 2X polling interval, then restart the
> polling daemon.
>
> Solving #2 required a bit more work but I published the solution here:
> 	http://search.cpan.org/~msimerson/RTG-Report-1.16/
>
> #3 is also solved by the record_consolidator.pl script in the
> RTG::Report dist.
>
> Of course, other people have had different problems with RTG and
> developed their own solutions. But you might want to just give the
> stock RTG a whirl. It may just do everything you need as is.
>
> Matt
>
>
> On Jan 7, 2009, at 9:37 AM, Adrian Chadd wrote:
>
>> People are running their own hacked up versiosn of RTG, and noone's
>> really stepped up to create an RTG fork to include all the various
>> patches, stuff from the Y!rtg, etc.
>>
>> Anyone up to the challenge? :)
>>
>>
>>
>> Adrian
>>
>> On Tue, Jan 06, 2009, Jeremy Guarini wrote:
>>> been using version 7 myself as I didn't see anything firm for
>>> version 8 or 9
>>>
>>> --Jeremy
>>>
>>> Quoting John Center <john.center at villanova.edu>:
>>>
>>>> Hi,
>>>>
>>>> What is the status of this project?  We've been using an older
>>>> version of rtg.  I saw a discussion about various patches floating
>>>> around, but it doesn't look like there has been any further
>>>> development.  Could someone please update the list?
>>>>
>>>> Thanks.
>>>>
>>>> 	-John
>>>>
>>>> -- 
>>>> John Center
>>>> Villanova University
>>>> _______________________________________________
>>>> RTG mailing list
>>>> RTG at lists.grdata.com
>>>> http://lists.grdata.com/mailman/listinfo/rtg
>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> RTG mailing list
>>> RTG at lists.grdata.com
>>> http://lists.grdata.com/mailman/listinfo/rtg
>> _______________________________________________
>> RTG mailing list
>> RTG at lists.grdata.com
>> http://lists.grdata.com/mailman/listinfo/rtg
>
> _______________________________________________
> RTG mailing list
> RTG at lists.grdata.com
> http://lists.grdata.com/mailman/listinfo/rtg