Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
rwpackchecker(8)		SiLK Tool Suite		      rwpackchecker(8)

NAME
       rwpackchecker - Find unusual patterns that may indicate a corrupt file

SYNOPSIS
	 rwpackchecker [--value=TEST=VALUE] [--allowable-count=TEST=ALLOWED]
	       [--print-all]
	       {[--xargs] | [--xargs=FILENAME] | [FILE [FILE ...]]}

	 rwpackchecker --help

	 rwpackchecker --version

DESCRIPTION
       rwpackchecker reads SiLK	Flow records and checks	for "unusual" patterns
       that may	indicate the file has been corrupted.

       rwpackchecker has a default series of tests it runs on every flow
       record in an input file.	 Each default test has two numbers associated
       with it:	a value	threshold and an allowed count threshold.  A test
       compares	a value	on the flow record to the value	threshold, and if the
       value violates the threshold, a counter for that	test is	incremented.
       In addition, if the flow	record violates	the value threshold for	any
       test, a global counter is incremented to	denote a suspect record.

       Once rwpackchecker finishes processing a	file, it determines whether
       the file	appears	to be valid.  A	file is	considered valid if either

       o   the global counter of suspect records is 0, or

       o   no test has a counter that exceeds the test's allowed count
	   threshold

       If rwpackchecker	determines that	all files are valid, it	does not print
       any output by default.  If rwpackchecker	does find an invalid file, it
       will print the name of the input	file, the global number	of suspect
       records it found, and information for those tests where the counter
       exceeds the allowed count threshold.

       As an example, if there are 10 tests and	the count threshold for	each
       is 10, it is possible for the global suspect counter to be 90 and for
       rwpackchecker to	consider the file valid.

       To force	rwpackchecker to print the results for all tests and for all
       input files, specify the	--print-all switch.

       Some of the tests that run by default include checking the number of
       packets,	the bytes per second ratio, the	bytes per packet ratio,	and
       the bytes per packet ratio for a	particular protocol (TCP, UDP, and
       ICMP).

       The --value and --allowable-count switches modify the value threshold
       and allowed count threshold for a test, respectively.  The argument to
       the switch is the test name and the threshold, separated	by a equals
       sign ("=").  Repeat the switches	to set multiple	thresholds.  For
       example,	to change the value thresholds for the max-bytes test to 20000
       and for the max-packets test to 15000, specify the following:

	rwpackchecker --value=max-bytes=20000 --value max-packets=15000	...

       To get the most value from rwpackchecker, one should customize it for
       the particular site where it is being used, since the default value for
       a threshold may or may not be unusual for a particular installation.
       For example, a router that has Ethernet connections should have no more
       than 1500 bytes per packet, since that is the Ethernet MTU; however,
       the default value for that ratio	is 16384.  In some cases the default
       value is	the largest value that a SiLK IPv4 hourly repository file can
       store, making it	impossible for a record	to violate the threshold.

       rwpackchecker supports additional tests which are not run by default.
       Representative tests include checking whether an	IP is (not) in an
       IPset or	whether	a port is (not)	in an integer list.  To	run an
       additional test,	specify	the name of the	test using the --value switch
       and provide an argument for the test.

       To see all of the tests that rwpackchecker supports as well as the
       value threshold and allowed count threshold for each test, see the
       "OPTIONS" section below,	or run rwpackchecker with the --help switch.

       rwpackchecker reads SiLK	Flow records from the files named on the
       command line or from the	standard input when no file names are
       specified and --xargs is	not present.  To read the standard input in
       addition	to the named files, use	"-" or "stdin" as a file name.	If an
       input file name ends in ".gz", the file is uncompressed as it is	read.
       When the	--xargs	switch is provided, rwpackchecker reads	the names of
       the files to process from the named text	file or	from the standard
       input if	no file	name argument is provided to the switch.  The input to
       --xargs must contain one	file name per line.

OPTIONS
       Option names may	be abbreviated if the abbreviation is unique or	is an
       exact match for an option.  A parameter to an option may	be specified
       as --arg=param or --arg param, though the first form is required	for
       options that take optional parameters.

       --value=TEST=VALUE
	   Set the value of TEST to the	specified VALUE; separate the test
	   name	from value by "=".  The	available TESTs	are given below; the
	   test	name can be shortened to the shortest unique prefix.  The form
	   of VALUE depends on the type	of TEST:

	   o   If TEST expects a minimum or maximum, VALUE should be a number.

	   o   If TEST expects a list of IPs, VALUE should the name of a file
	       containing an IPset (see	rwsetbuild(1)).

	   o   If TEST expects a list of numbers (for example, ports or
	       protocols), VALUE should	contain	a comma	separated list of
	       integers	and integer-ranges where a range is two	integers
	       separated by a hyphen ("-").

	   Repeat this switch for each value that you wish to set.

       --allowable-count=TEST=ALLOWED
	   Allow the named TEST	to be violated ALLOWED of times	before
	   treating it as "unusual".  ALLOWED is an integer value. Separate
	   the test name from the allowed count	by "=".	 Repeat	this switch
	   for each allowable count you	wish to	set.

       --print-all
	   Print the result of all tests for all input files.  Normally	only
	   tests that are deemed "unusual" are printed.

       --xargs
       --xargs=FILENAME
	   Read	the names of the input files from FILENAME or from the
	   standard input if FILENAME is not provided.	The input is expected
	   to have one filename	per line.  rwpackchecker opens each named file
	   in turn and reads records from it as	if the filenames had been
	   listed on the command line.

       --help
	   Print the available options and exit.

       --version
	   Print the version number and	information about how SiLK was
	   configured, then exit the application.

       The following tests are always run:

       min-bpp-ratio=NUMBER
	   Byte-per-packet ratio is less than NUMBER.  Default value: 1.
	   Allowed count: 0.

       max-bpp-ratio=NUMBER
	   Byte-per-packet ratio is greater than NUMBER.  Default value:
	   16384.  Allowed count: 0.

       min-bps-ratio=NUMBER
	   Byte-per-second ratio is less than NUMBER.  Default value: 0.
	   Allowed count: 0.

       max-bps-ratio=NUMBER
	   Byte-per-second ratio is greater than NUMBER.  Default value:
	   4294967295.	Allowed	count: 0.

       min-packets=NUMBER
	   Packet count	is less	than NUMBER.  Default value: 1.	 Allowed
	   count: 0.

       max-packets=NUMBER
	   Packet count	is greater than	NUMBER.	 Default value:	67108864.
	   Allowed count: 0.

       min-bytes=NUMBER
	   Byte	count is less than NUMBER.  Default value: 1.  Allowed count:
	   0.

       max-bytes=NUMBER
	   Byte	count is greater than NUMBER.  Default value: 4294967295.
	   Allowed count: 0.

       min-tcp-bpp-ratio=NUMBER
	   TCP byte-per-packet ratio is	less than NUMBER.  Default value: 1.
	   Allowed count: 0.

       max-tcp-bpp-ratio=NUMBER
	   TCP byte-per-packet ratio is	greater	than NUMBER.  Default value:
	   16384.  Allowed count: 0.

       min-udp-bpp-ratio=NUMBER
	   UDP byte-per-packet ratio is	less than NUMBER.  Default value: 1.
	   Allowed count: 0.

       max-udp-bpp-ratio=NUMBER
	   UDP byte-per-packet ratio is	greater	than NUMBER.  Default value:
	   16384.  Allowed count: 0.

       min-icmp-bpp-ratio=NUMBER
	   ICMP	byte-per-packet	ratio is less than NUMBER.  Default value: 1.
	   Allowed count: 0.

       max-icmp-bpp-ratio=NUMBER
	   ICMP	byte-per-packet	ratio is greater than NUMBER.  Default value:
	   16384.  Allowed count: 0.

       The following tests are only run	when the --value switch	is used	to
       specify a value for the test.

       match-protocol=LIST
	   Protocol is present in LIST.	 No default.  Allowed count: 0.

       nomatch-protocol=LIST
	   Protocol is not present in LIST.  No	default.  Allowed count: 0.

       match-flags=LIST
	   TCP Flag Combination	is present in LIST.  No	default.  Allowed
	   count: 0.

       nomatch-flags=LIST
	   TCP Flag Combination	is not present in LIST.	 No default.  Allowed
	   count: 0.

       match-sip=IPSET_FILE
	   Source IP is	present	in IPSET_FILE.	No default.  Allowed count: 0.

       nomatch-sip=IPSET_FILE
	   Source IP is	not present in IPSET_FILE.  No default.	 Allowed
	   count: 0.

       match-dip=IPSET_FILE
	   Destination IP is present in	IPSET_FILE.  No	default.  Allowed
	   count: 0.

       nomatch-dip=IPSET_FILE
	   Destination IP is not present in IPSET_FILE.	 No default.  Allowed
	   count: 0.

       match-sport=LIST
	   Source Port is present in LIST.  No default.	 Allowed count:	0.

       nomatch-sport=LIST
	   Source Port is not present in LIST.	No default.  Allowed count: 0.

       match-dport=LIST
	   Destination Port is present in LIST.	 No default.  Allowed count:
	   0.

       nomatch-dport=LIST
	   Destination Port is not present in LIST.  No	default.  Allowed
	   count: 0.

       match-nhip=IPSET_FILE
	   Next	Hop IP is present in IPSET_FILE.  No default.  Allowed count:
	   0.

       nomatch-nhip=IPSET_FILE
	   Next	Hop IP is not present in IPSET_FILE.  No default.  Allowed
	   count: 0.

       match-input=LIST
	   SNMP	Input is present in LIST.  No default.	Allowed	count: 0.

       nomatch-input=LIST
	   SNMP	Input is not present in	LIST.  No default.  Allowed count: 0.

       match-output=LIST
	   SNMP	Output is present in LIST.  No default.	 Allowed count:	0.

       nomatch-output=LIST
	   SNMP	Output is not present in LIST.	No default.  Allowed count: 0.

EXAMPLES
       In these	examples, the dollar sign ("$")	represents the shell prompt
       and a backslash ("\") is	used to	continue a line	for better
       readability.  The examples do not use the optional "=" between the
       --value switch and the switch's argument	for better readability.

       Given the SiLK Flow file	data.rw	where the number of flows with various
       byte and	packet counts are described by this table:

	Number of flows	   bytes <= 2000000    bytes > 2000000	   TOTAL
	packets	<= 500	       379303		     308	  379611
	packets	>  500	       119586		    2679	  122265
	TOTAL		       498889		    2987	  501876

       Running rwpackchecker:

	$ rwpackchecker	--value	max-bytes=2000000	     \
	       --value max-packets=500	data.rw
	data.rw:
		     122573/501876 flows are bad or unusual
		     122265 flows where	Packet Count > 500
		       2987 flows where	Byte Count > 2000000

       The counts for the individual tests are greater than the	overall	total
       since 2679 records fall into both categories.

       To see the effect of the	--allowable-count switch:

	$ rwpackchecker	--value	max-packets=500				     \
	       --value max-bytes=2000000 --allowable max-bytes=3000 data.rw
	data.rw:
		     122573/501876 flows are bad or unusual
		     122265 flows where	Packet Count > 500

	$ rwpackchecker	--value	max-bytes=2000000			     \
	       --value max-packets=500 --allowable max-packets=150000 data.rw
	data.rw:
		     122573/501876 flows are bad or unusual
		       2987 flows where	Byte Count > 2000000

       In each case the	total number of	unusual	flows did not change; the
       violation of the	other limit is still noted, even when the test is not
       printed since the test's	allowed	count threshold	was not	reached.

       When the	allowed	count thresholds for none of the tests are reached,
       rwpackchecker produces no output	by default:

	$ rwpackchecker	--value	max-bytes=2000000 --allowable max-bytes=3000 \
	       --value max-packets=500 --allowable max-packets=150000 data.rw
	$

       Specify the --print-all switch to print the results:

	$ rwpackchecker	--value	max-bytes=2000000 --allowable max-bytes=3000 \
	       --value max-packets=500 --allowable max-packets=15000	     \
	       --print-all data.rw
	data.rw:
		     122573/501876 flows are bad or unusual
			  0 flows where	BPP Calculation	is incorrect
			  0 flows where	Elapsed	Time > 4096
			  0 flows where	Byte/Packet Ratio < 1
			  0 flows where	Byte/Packet Ratio > 16384
			  0 flows where	Byte/Second Ratio < 0
			  0 flows where	Byte/Second Ratio > 4294967295
			  0 flows where	Packet Count < 1
		     122265 flows where	Packet Count > 500
			  0 flows where	Byte Count < 1
		       2987 flows where	Byte Count > 2000000
			  0 flows where	TCP Byte/Packet	Ratio <	1
			  0 flows where	TCP Byte/Packet	Ratio >	16384
			  0 flows where	UDP Byte/Packet	Ratio <	1
			  0 flows where	UDP Byte/Packet	Ratio >	16384
			  0 flows where	ICMP Byte/Packet Ratio < 1
			  0 flows where	ICMP Byte/Packet Ratio > 16384

SEE ALSO
       rwflowpack(8), rwsetbuild(1), silk(7)

SiLK 3.19.1			  2020-08-27		      rwpackchecker(8)

NAME | SYNOPSIS | DESCRIPTION | OPTIONS | EXAMPLES | SEE ALSO

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=rwpackchecker&sektion=8&manpath=FreeBSD+12.2-RELEASE+and+Ports>

home | help