CarrierIn - a FlowScan module for reporting on carrier or ISP input traffic
$ flowscan CarrierIn
or in flowscan.cf:
CarrierIn is a general flowscan report for reporting on flows of input traffic for a carrier or ISP. It does this by processing flows reported by one or more routers at the network border. The carrier is thought to have an Autonomous System (AS) and BGP protocol running on the Netflow exporting routers.
CarrierIn relies on the fact that NetFlow is turned on at inbound interfaces only.
CarrierIn is based on CampusIO.pm written by Dave Plonka.
flowscan will run the CarrierIn report if you configure this in
The difference to Dave Plonka's CampusIO.pm is as follows:
CarrierIn's configuration file is CarrierIn.cf. This configuration file is located in the directory in which the flowscan script resides.
Configuration directives removed from CampusIO.pm:
New configuration directives are:
The CarrierIn configuration directives include:
# OutputDir /var/local/flows/graphs OutputDir graphs
# SubnetFiles our_subnets.boulder SubnetFiles bin/our_subnets.boulder
Each file contains network definitions in Boulder format. For each subnet you can specify optional name and level. The name is used as the symbolic representation of this subnet and will be used for RRD database file names.
SUBNET=184.108.40.206/16 NAME=whole_62_2 LEVEL=0 = SUBNET=220.127.116.11/22 NAME=my_favorite_customer LEVEL=1 = SUBNET=18.104.22.168/21 NAME=dialin_pool LEVEL=1 = SUBNET=22.214.171.124/20 NAME=dialin_pool LEVEL=1 =
You need to specify levels if you want to collect statistics on nested subnets, like in the eample above. Each level consists of a separate Patricia tree, thus allowing for nested counters. If the level is not specified, the subnet is put into Level 0.
Several subnets can have the same names. In such case, they will have common counters. This is useful when you have non-contiguous address pools for some common purposes.
However, if you need the collected data for raw traffic overview only, this option might be useful for multi-gigabit Internet upstream.
# TCPServices ftp-data, ftp, smtp, nntp, http, 7070, 554 TCPServices ftp-data, ftp, smtp, nntp, http, 7070, 554
# UDPServices domain, snmp, snmp-trap
# Protocols icmp, tcp, udp Protocols icmp, tcp, udp
# source_AS:destination_AS, e.g.: # ASPairs 0:0 ASPairs 0:0
Note that the effect of setting ASPairs will be different based on whether you specified ``peer-as'' or ``origin-as'' when you configured your Cisco. This option was intended to be used when ``peer-as'' is configured.
BGPDumpFile directive for other AS-related features.
flowscanmore verbose with respect to messages and warnings. Currently the values
2are understood, the higher value causing more messages to be produced. E.g.:
# Verbose (OPTIONAL, non-zero = true) Verbose 1
TopNis the number of entries to show in the tables that will be generated in HTML top reports. E.g.:
# TopN (OPTIONAL) TopN 10
If you'd prefer to see hostnames rather than IP addresses in your top reports, use the ip2hostname script. E.g.:
$ ip2hostname -I *.*.*.*_*.html
strftime(3)format specifiers in the value, and it may also specify sub-directories. If not set, the prefix defaults to the null string, which means that, every five minutes, subsequent reports will overwrite the previous. E.g.:
# Preserve one day of HTML reports using the time of day as the dir name: ReportPrefixFormat html/CarrierIn/%H:%M/
# Preserve one month by using the day of month in the dir name (like sar(1)): ReportPrefixFormat html/CarrierIn/%d/%H:%M_
ASNFileit causes FlowScan to produce ``Top ASN'' reports which show the ``top'' Autonomous Systems with which your site exchanges traffic.
BGPDumpFile requires the
ParseBGPDump perl module by Sean
McCreary, which is supplied with CAIDA's CoralReef Package:
Unfortunately, CoralReef is governed by a different license than FlowScan itself. The Copyright file says this:
Permission to use, copy, modify and distribute any part of this CoralReef software package for educational, research and non-profit purposes, without fee, and without a written agreement is hereby granted, provided that the above copyright notice, this paragraph and the following paragraphs appear in all copies. [...]
The CoralReef software package is developed by the CoralReef development team at the University of California, San Diego under the Cooperative Association for Internet Data Analysis (CAIDA) Program. Support for this effort is provided by the CAIDA grant NCR-9711092, and by CAIDA members.
After fetching the
coral release from:
ParseBGPDump.pm in FlowScan's perl include path, such as in
$ cd /tmp $ gunzip -c coral-3.4.1-public.tar.gz |tar x coral-3.4.1-public/./libsrc/misc-perl/ParseBGPDump.pm $ mv coral-3.4.1-public/./libsrc/misc-perl/ParseBGPDump.pm $PREFIX/bin/ParseBGPDump.pm
Also you must specify
TopN to be greater than zero, e.g. 10, and the
HTML::Table perl module is required if you do so.
BGPDumpFile value is the name of a file containing the output of
show ip bgp from a Cisco router, ideally from the router that is
exporting flows. If this option is used, and the specified file
exists, it will cause the ``originAS'' and ``pathAS'' reports to be
TopN 10 BGPDumpFile etc/router.our.domain.bgp
One way to create the file itself, is to set up rsh access to your Cisco, e.g.:
ip rcmd rsh-enable ip rcmd remote-host username 10.10.42.69 username
Then do something like this:
$ cd $PREFIX $ mkdir etc $ echo show ip bgp >etc/router.our.domain.bgp # required by ParseBGPDump.pm $ time rsh router.our.domain "show ip bgp" >>etc/router.our.domain.bgp 65.65s real 0.01s user 0.05s system $ wc -l /tmp/router.our.domain.bgp 197883 /tmp/router.our.domain.bgp
flowscan is up and running with
BGPDumpFile configured, it
will reload that file if its timestamp indicates that it has been
modified. This allows you to ``freshen'' the image of the routing table
without having to restart
BGPDumpFile option causes
FlowScan to use much more
memory than usual. This memory is used to store a
trie containing a node for every prefix in the BGP routing table. For
instance, on my system it caused the
FlowScan process to grow to
over 50MB, compared to less than 10MB without
BGPDumpFile. If specified, this directive will cause the AS names rather than just their numbers to appear in the Top ASN HTML reports. Its value should be the path to a file having the format of the file downloaded from this URL:
TopN 10 BGPDumpFile etc/router.our.domain.bgp ASNfile etc/asn.txt
flowscan is up and running with
ASNFile configured, it will
reload the file if its timestamp indicates that it has been modified.
This module provides no public methods. It is a report module meant
only for use by
flowscan. Please see the
documentation for information on how to write a FlowScan report
perl(1), FlowScan, CampusIO, SubNetIO, flowscan(1), Net::Patricia.
See CampusIO.pm bugs.
Dave Plonka <firstname.lastname@example.org> Stanislav Sinyagin <email@example.com>
Copyright (C) 1998-2001 Dave Plonka. Copyright (C) 2002 Cablecom GmbH
This program source is based on CampusIO.pm. It was developed by the order of Cablecom GmbH (www.cablecom.ch).
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
The version number is the module file RCS revision number ($Revision: 1.6 $) with the minor number printed right justified with leading zeroes to 3 decimal places. For instance, RCS revision 1.1 would yield a package version number of 1.001.
This is so that revision 1.10 (which is version 1.010), for example, will test greater than revision 1.2 (which is version 1.002) when you want to require a minimum version of this module.