[rtg] Does anyone find this interesting?

[Brian T. O'Neill]

On the issue you're having with more then 15 minute intervals, are you
using the same rtg.conf for the rtgplot application as you are with
rtgpoll? The defaults will throw out data that is too far skewed from
the time period listed in rtg.conf.

Also, with the 15min time period not being a problem, it will be if you
are not using 64bit counters and you have a high speed link. A 32bit
counter can easily. A fully loaded 100Mbps link will wrap in 5.75
minutes, a 1Gbps link can wrap in 35 seconds. If you're polling every 15
minutes, you could wrap twice on a 100Mbps link, but RTG will only see
it as one wrap, and you will grossly underreport your bandwidth. As you
mention that you can find a port upgraded from 100M to gigabit, if you
are reporting on gigabit links with 32bit, you could miss 25 counter
wraps in that 15 minute poll.

Brian


Quoting Matt Simerson (matt at layeredtech.com) from  :
> 
> I'm looking for feedback right now, questions, and concerns...
> 
> 
> ~/scripts/rtg % cat README
> 
> RTG is "a flexible, scalable, high-performance SNMP statistics  
> monitoring system" available at http://rtg.sourceforge.net/  If you  
> need to measure bandwidth based on router/switch port traffic, then  
> RTG is an excellent tool that performs the task of polling the network  
> devices via SNMP and storing the traffic data in a SQL RDBMS.
> 
> Layered Technologies uses RTG to monitor tens of thousands of network  
> ports, feeding the bandwidth data from each of our networks into MySQL  
> databases. Each of the scripts included in RTG::Reporter serve a  
> particular need. We appreciate the generosity of RTGs authors and it  
> is our hope that the RTG community finds this software useful as well.
> 
> billing_report.pl - Generate a bandwidth billing report. The script  
> includes a slew of options for customizing the report to provide as  
> much or as little data as is needed. The report exports a CSV file and  
> emails it (as an attachment) to a list of email addresses specified in  
> the config file. The report includes the following fields: network  
> name, router, Interface ID, Interface Name, Port Speed, Description,  
> and In/Out bytes. It can optionally include the Avg In/Out (bit/s),  
> Peak In/Out (bit/s), and 95th In/Out (bits/s). For each network  
> defined, a set of 4 summary records are included in the CSV data  
> (uplink, downlink, isp_uplink, and total). An example CSV is included  
> in examples/billing_report.csv.
> 
> uplink_report.pl - Generates an uplink report that shows how much  
> traffic each ISP uplink port is utilizing. We run this report weekly  
> and it gives our operations team and executives a high level overview  
> of our network traffic. It also gives us handy reference points to  
> compare and validate the usage our ISPs bill us for.
> 
> record_consolidator.pl - Consolidates RTG data into summary records.  
> We monitor tens of thousands of network ports and have hundreds of  
> gigabytes of RTG data. When disk space or performance becomes an  
> issue, the RTG supplied tool is a pruning script that simply deletes  
> old data. Rather than deleting that data, I wanted to condense it,  
> preserving the essence of the data. I borrowed a concept from Tobias  
> Otiker and his RRD databases--rather than discard old data, reduce its  
> granularity.
> 
> In testing, I discovered that 15 minute averages still work perfectly  
> with unaltered versions of the RTG CGI application. Any more than 15  
> minute averages and graphs stop rendering. Dropping from 5 to 15  
> minute averages reduces the disk space required by old data by 66%.  
> The consolidator script is controlled by two settings in the config  
> file:
> 
>  [Pruning]
>  days     = 190
>  interval = 60
> 
> Days controls what data you want to alter. A setting of 190 as shown  
> will only condense data older than 190 days. Interval controls what  
> interval the consolidated records will be. A setting of 15 will  
> condense the data to 15 minute averages (66% reduction). A setting of  
> 60 as shown will reduce the data to 60 minute averages for a 91%  
> reduction in disk space.
> 
> How you use the consolidator script will be dictated by your business  
> needs. I suggest using the consolidator script with two sets of  
> settings. Old data that is no longer needed for graph publication  
> should be pruned to 60 minute averages. A setting of 190/60 will do  
> this. Then have another tier of pruning where you consolidate data  
> older than 2-3 months to 15 minute averages. A setting of 90/15 would  
> do that.
> 
> rtg_target_maker.pl - not released yet...
> 
> A re-write of rtgtargetmkr.pl. Fixes a few notable problems with the  
> existing script. Notably, it corrects errors in the RTG database. For  
> example, say you see that a customer is utilizing 250% of his port  
> speed in the billing report. Being gifted in math, you deduce that is  
> not possible and figure out it's because the port was upgraded from  
> 100Mbit to Gigabit. This target maker script will automatically update  
> port speeds and port descriptions so that the data in the database  
> accurately reflects the configuration of the network device. It also  
> sends a HUP signal to the rtgpoll process so it will re-read the  
> routers file.
> 
> 
> 
> RTG::Reporter is written by Matt Simerson @ Layered Technologies, Inc.  
> Layered Technologies is a hosting provider that you should consider  
> for your web hosting needs: http://www.layeredtech.com/
> 
> 
> 
> Matt Simerson
> Unix Automation Developer
> Email: matt at layeredtech.com
> Phone: 214.564.6085
> 
> Layered Technologies, Inc.
> On-Demand Utility Computing & Hosting Solutions
> Learn more>> http://www.layeredtech.com
> 
> 
> 

> _______________________________________________
> RTG mailing list
> RTG at lists.grdata.com
> http://lists.grdata.com/mailman/listinfo/rtg


-- 
btoneill at misplaced.net

****************************************************************************
UNIX is simple and coherent, but it takes a genius (or at any rate a 
programmer) to understand and appreciate the simplicity."  - Dennis Ritchie
****************************************************************************