Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
rwtotal(1)			SiLK Tool Suite			    rwtotal(1)

NAME
       rwtotal - Count how much	traffic	matched	specific keys

SYNOPSIS
	 rwtotal {--sip-first-8	| --sip-first-16 | --sip-first-24 |
	       --sip-last-8 | --sip-last-16 | --dip-first-8 |
	       --dip-first-16 |	--dip-first-24 | --dip-last-8 |
	       --dip-last-16 | --sport | --dport | --proto | --packets |
	       --bytes | --duration | --icmp-code}
	       [--summation] [--min-bytes=COUNT] [--max-bytes=COUNT]
	       [--min-packets=COUNT] [--max-packets=COUNT]
	       [--min-records=COUNT] [--max-records=COUNT] [--skip-zeroes]
	       [--no-titles] [--no-columns] [--column-separator=CHAR]
	       [--no-final-delimiter] [{--delimited | --delimited=CHAR}]
	       [--print-filenames] [--copy-input=PATH] [--output-path=PATH]
	       [--pager=PAGER_PROG] [--site-config-file=FILENAME]
	       {[--xargs] | [--xargs=FILENAME] | [FILE [FILE ...]]}

	 rwtotal --help

	 rwtotal --version

DESCRIPTION
       rwtotal reads SiLK Flow records,	bins those records by the user-
       specified specified key,	computes the volume per	bin (record count and
       sums of packets and bytes), and prints the bins and their volumes.

       rwtotal reads SiLK Flow records from the	files named on the command
       line or from the	standard input when no file names are specified	and
       --xargs is not present.	To read	the standard input in addition to the
       named files, use	"-" or "stdin" as a file name.	If an input file name
       ends in ".gz", the file is uncompressed as it is	read.  When the
       --xargs switch is provided, rwtotal reads the names of the files	to
       process from the	named text file	or from	the standard input if no file
       name argument is	provided to the	switch.	 The input to --xargs must
       contain one file	name per line.

       By default, rwtotal prints a bin	for every possible key,	even when the
       volume for that bin is zero.  Use the --skip-zeroes switch to suppress
       the printing of these empty bins.

       Use the --summation switch to include a row giving the volume for all
       flow records.

       The maximum key value that rwtotal supports is 16,777,215.  When	the
       key field is --bytes or --packets, rwtotal will create a	bin for	all
       unique values up	to 16,777,214.	The final bin (16,777,215) will
       consist of all values greater than 16,777,214.

OPTIONS
       Option names may	be abbreviated if the abbreviation is unique or	is an
       exact match for an option.  A parameter to an option may	be specified
       as --arg=param or --arg param, though the first form is required	for
       options that take optional parameters.

       One and only one	of the following counting keys is required:

       --sip-first-8
	   Key on the first 8 bits of the source IP address

       --sip-first-16
	   Key on the first 16 bits of the source IP address

       --sip-first-24
	   Key on the first 24 bits of the source IP address

       --sip-last-8
	   Key on the last 8 bits of the source	IP address

       --sip-last-16
	   Key on the last 16 bits of the source IP address

       --dip-first-8
	   Key on the first 8 bits of the destination IP address

       --dip-first-16
	   Key on the first 16 bits of the destination IP address

       --dip-first-24
	   Key on the first 24 bits of the destination IP address

       --dip-last-8
	   Key on the last 8 bits of the destination IP	address

       --dip-last-16
	   Key on the last 16 bits of the destination IP address

       --sport
	   Key on the source port.

       --dport
	   Key on the destination port.

       --proto
	   Key on the protocol.

       --packets
	   Key on the number of	packets	in the record

       --bytes
	   Key on the number of	bytes in the record

       --duration
	   Key on the duration of the record.

       --icmp-code
	   Key on the ICMP type	and code.  This	switch will assume that	all
	   incoming records are	ICMP.

       The following options affect the	output:

       --summation
	   Print as the	final row a total of the values	in each	column.

       --min-bytes=COUNT
	   Disable printing of bins with fewer than COUNT bytes.  By default,
	   all bins are	printed.

       --max-bytes=COUNT
	   Disable printing of bins with more than COUNT bytes.	 By default,
	   all bins are	printed.

       --min-packets=COUNT
	   Disable printing of bins with fewer than COUNT packets.  By
	   default, all	bins are printed.

       --max-packets=COUNT
	   Disable printing of bins with more than COUNT packets.  By default,
	   all bins are	printed.

       --min-records=COUNT
	   Disable printing of bins with fewer than COUNT flow records.	 By
	   default, all	bins are printed.

       --max-records=COUNT
	   Disable printing of bins with more than COUNT flow records.	By
	   default, all	bins are printed.

       --skip-zeroes
	   Disable printing of bins with no traffic.  By default, all bins are
	   printed.

       --no-titles
	   Turn	off column titles.  By default,	titles are printed.

       --no-columns
	   Disable fixed-width columnar	output.

       --column-separator=C
	   Use specified character between columns and after the final column.
	   When	this switch is not specified, the default of '|' is used.

       --no-final-delimiter
	   Do not print	the column separator after the final column.  Normally
	   a delimiter is printed.

       --delimited
       --delimited=C
	   Run as if --no-columns --no-final-delimiter --column-sep=C had been
	   specified.  That is,	disable	fixed-width columnar output; if
	   character C is provided, it is used as the delimiter	between
	   columns instead of the default '|'.

       --print-filenames
	   Print to the	standard error the names of input files	as they	are
	   opened.

       --copy-input=PATH
	   Copy	all binary SiLK	Flow records read as input to the specified
	   file	or named pipe.	PATH may be "stdout" or	"-" to write flows to
	   the standard	output as long as the --output-path switch is
	   specified to	redirect rwtotal's textual output to a different
	   location.

       --output-path=PATH
	   Write the textual output to PATH, where PATH	is a filename, a named
	   pipe, the keyword "stderr" to write the output to the standard
	   error, or the keyword "stdout" or "-" to write the output to	the
	   standard output (and	bypass the paging program).  If	PATH names an
	   existing file, rwtotal exits	with an	error unless the SILK_CLOBBER
	   environment variable	is set,	in which case PATH is overwritten.  If
	   this	switch is not given, the output	is either sent to the pager or
	   written to the standard output.

       --pager=PAGER_PROG
	   When	output is to a terminal, invoke	the program PAGER_PROG to view
	   the output one screen full at a time.  This switch overrides	the
	   SILK_PAGER environment variable, which in turn overrides the	PAGER
	   variable.  If the --output-path switch is given or if the value of
	   the pager is	determined to be the empty string, no paging is
	   performed and all output is written to the terminal.

       --site-config-file=FILENAME
	   Read	the SiLK site configuration from the named file	FILENAME.
	   When	this switch is not provided, rwtotal searches for the site
	   configuration file in the locations specified in the	"FILES"
	   section.

       --xargs
       --xargs=FILENAME
	   Read	the names of the input files from FILENAME or from the
	   standard input if FILENAME is not provided.	The input is expected
	   to have one filename	per line.  rwtotal opens each named file in
	   turn	and reads records from it as if	the filenames had been listed
	   on the command line.

       --help
	   Print the available options and exit.

       --version
	   Print the version number and	information about how SiLK was
	   configured, then exit the application.

EXAMPLES
       In the following	examples, the dollar sign ("$")	represents the shell
       prompt.	The text after the dollar sign represents the command line.
       Lines have been wrapped for improved readability, and the back slash
       ("\") is	used to	indicate a wrapped line.

   Group by the	protocol
       Group all incoming data for the first hour of March 1, 2003 by
       protocol.

	$ rwfilter --start-date=2003/03/01:00 --end-date=2003/03/01:00 \
	       --all-destination=stdout				       \
	  | rwtotal --proto --skip-zero
	   protocol|	    Records|		   Bytes|	   Packets|
		  1|	      15622|		10695328|	    147084|
		  6|	     330726|	    120536195111|	 144254362|
		 17|	     155528|		24500079|	    155528|

       To get the same result with rwuniq(1), use:

	$ rwfilter ... --pass=stdout				       \
	  | rwuniq --fields=proto --values=records,bytes,packets       \
	       --sort-output
	pro|   Records|		      Bytes|	    Packets|
	  1|	 15622|		   10695328|	     147084|
	  6|	330726|	       120536195111|	  144254362|
	 17|	155528|		   24500079|	     155528|

   Group by the	source Class A addresses
	$ rwfilter --start-date=2003/03/01:00 --end-date=2003/03/01:00 \
	       --all-destination=stdout				       \
	  | rwtotal --sip-first-8 --skip-zero
	 sIP_First8|	    Records|		   Bytes|	   Packets|
		 10|	     173164|	     59950837766|	  72201390|
		172|	      77764|		17553593|	     77764|
		192|	     250948|	     60602999159|	  72277820|

       Use rwnetmask(1)	and rwuniq(1) to get a similar result:

	$ rwfilter ... --pass=stdout				       \
	  | rwnetmask --4sip-prefix=8				       \
	  | rwuniq --fields=sip	--values=records,bytes,packets	       \
	       --sort-output --ipv6-policy=ignore
		    sIP|   Records|		  Bytes|	Packets|
	       10.0.0.0|    173164|	    59950837766|       72201390|
	      172.0.0.0|     77764|	       17553593|	  77764|
	      192.0.0.0|    250948|	    60602999159|       72277820|

   Group by the	final IPv4 octet
	$ rwfilter --start-date=2003/03/01:00 --end-date=2003/03/01:00	   \
	       --proto=6 --pass=stdout --daddress=192.168.x.x		   \
	  | rwtotal --dip-last-16 --skip-zero |	head -5
	 dIP_Last16|	    Records|		   Bytes|	   Packets|
	      0. 38|		  6|		 4862678|	      4016|
	      1. 14|		  1|		   32844|	       452|
	     18.146|		  1|		    4226|		12|
	     21.  4|		  6|		 5462032|	      4521|

       One way to accomplish this with rwuniq is to create a new field using
       PySiLK (see pysilk(3)) and the PySiLK plug-in capability	(see
       silkpython(3).  The invocation is:

	$ rwfilter ... --pass=stdout					  \
	  | rwuniq --python=/tmp/dip16.py --fields=dip-last-16		  \
	       --values=flows,bytes,packets --sort-output | head -5
	    dip-last-16|   Records|		  Bytes|	Packets|
	       0.0.0.38|	 6|		4862678|	   4016|
	       0.0.1.14|	 1|		  32844|	    452|
	     0.0.18.146|	 1|		   4226|	     12|
	       0.0.21.4|	 6|		5462032|	   4521|

       where the definition of the "dip-last-16" field is given	in the file
       tmp/dip16.py:

	import silk
	mask = silk.IPAddr("0.0.255.255")
	def mask_dip(r):
	    return r.dip.mask(mask)

	register_ipv4_field("dip-last-16", mask_dip)

ENVIRONMENT
       SILK_PAGER
	   When	set to a non-empty string, rwtotal automatically invokes this
	   program to display its output a screen at a time.  If set to	an
	   empty string, rwtotal does not automatically	page its output.

       PAGER
	   When	set and	SILK_PAGER is not set, rwtotal automatically invokes
	   this	program	to display its output a	screen at a time.

       SILK_CLOBBER
	   The SiLK tools normally refuse to overwrite existing	files.
	   Setting SILK_CLOBBER	to a non-empty value removes this restriction.

       SILK_CONFIG_FILE
	   This	environment variable is	used as	the value for the
	   --site-config-file when that	switch is not provided.

       SILK_DATA_ROOTDIR
	   This	environment variable specifies the root	directory of data
	   repository.	As described in	the "FILES" section, rwtotal may use
	   this	environment variable when searching for	the SiLK site
	   configuration file.

       SILK_PATH
	   This	environment variable gives the root of the install tree.  When
	   searching for configuration files, rwtotal may use this environment
	   variable.  See the "FILES" section for details.

FILES
       ${SILK_CONFIG_FILE}
       ${SILK_DATA_ROOTDIR}/silk.conf
       /data/silk.conf
       ${SILK_PATH}/share/silk/silk.conf
       ${SILK_PATH}/share/silk.conf
       /usr/local/share/silk/silk.conf
       /usr/local/share/silk.conf
	   Possible locations for the SiLK site	configuration file which are
	   checked when	the --site-config-file switch is not provided.

SEE ALSO
       rwaddrcount(1), rwnetmask(1), rwstats(1), rwuniq(1), pysilk(3),
       silkpython(3), silk(7)

BUGS
       rwtotal replicates some functionality in	rwuniq(1) (most	notably	when
       rwuniq checks by	port or	protocol), but the implementations differ:
       rwtotal uses an array instead of	a hash-table, so access	is faster, the
       output is always	sorted,	and the	output includes	keys with a value of
       zero.  The use of an array prevents rwtotal from	using the complete IP
       address the way rwuniq does, but	it also	ensures	that rwtotal will not
       run out of memory.

       When used in an IPv6 environment, rwtotal will process every record as
       long as the IP address is not part of the key.  When aggregating	by the
       IP address, rwtotal converts IPv6 flow records that contain addresses
       in the ::ffff:0:0/96 prefix to IPv4 and processes them.	IPv6 records
       having addresses	outside	of that	prefix are silently ignored.  rwtotal
       will not	be modified to support IPv6 addresses; instead,	users should
       use rwuniq(1) (maybe combined with rwnetmask(1)).

       rwtotal is also similar to rwaddrcount(1) and rwstats(1).

SiLK 3.15.0			  2017-07-02			    rwtotal(1)

NAME | SYNOPSIS | DESCRIPTION | OPTIONS | EXAMPLES | ENVIRONMENT | FILES | SEE ALSO | BUGS

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=rwtotal&sektion=1&manpath=FreeBSD+12.0-RELEASE+and+Ports>

home | help