Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
nfdump(1)							     nfdump(1)

NAME
       nfdump -	netflow	display	and analyze program

SYNOPSIS
       nfdump [options]	[filter]

DESCRIPTION
       nfdump  is the netflow display and analyzing program of the nfdump tool
       set.  It	reads the netflow data from files stored by  nfcapd  and  pro-
       cesses the flows	according the options given. The filter	syntax is com-
       parable to tcpdump and extended for netflow data. Nfdump	can also  dis-
       play many different top N flow and flow element statistics.

OPTIONS
       -r inputfile
	  Read input data from inputfile. Default is read from stdin.

       -R expr
	  Read	input from a sequence of files in the same directory. expr may
	  be one of:
	   /any/dir	     Read recursively all files	in directory dir.
	   /dir/file	     Read all files beginning with file.
	   /dir/file1:file2  Read all files from file1 to file2.

	   When	using in combination with a sub	hierarchy:
	   /dir/sub1/sub2/file1:sub3/sub4/file2
	   Read	all files from sub1/sub2/file1 sub3/sub4/file2 iterating  over
	   all required	hierarchy levels.

	   Note: files are read	in alphabetical	sequence.

       -M expr
	  Read	 input	 from	multiple   directories.	  expr	 looks	 like:
	  /any/path/to/dir1:dir2:dir3 etc. and will be expanded	to the	direc-
	  tories:  /any/path/to/dir1,  /any/path/to/dir2 and /any/path/to/dir3
	  Any number of	colon separated	directories may	be given. The files to
	  read	are specified by -r or -R and are expected to exist in all the
	  given	directories.  The options -r and -R must not contain  any  di-
	  rectory part when used in conjunction	with -M.

       -m Sort	the netflow records according the date first seen. This	option
	  is usually only useful in conjunction	with -M, when netflow  records
	  are read from	different sources, which are not necessarily sorted.

       -w outputfile
	  If specified writes binary netflow records to	outputfile ready to be
	  processed again with nfdump. The default output is ASCII on stdout.

       -f filterfile
	  Reads	the filter syntax from filterfile. Note: Any filter  specified
	  directly on the command line takes precedence	over -f.

       -t timewin
	  Process  only	 flows,	 which	fall in	the time window	timewin, where
	  timewin is YYYY/MM/dd.hh:mm:ss[-YYYY/MM/dd.hh:mm:ss].	Any  parts  of
	  the	time   spec   may   be	 omitted  e.g  YYYY/MM/dd  expands  to
	  YYYY/MM/dd.00:00:00-YYYY/MM/dd.23:59:59 and processes	all flow  from
	  a given day. The time	window may also	be specified as	+/- n. In this
	  case it is relativ to	the beginning or end of	all flows.  +10	 means
	  the  first 10	seconds	of all flows, -10 means	the last 10 seconds of
	  all flows.

       -c num
	  Limit	number of records to process to	the first num flows.

       -a Aggregate netflow data. Aggregation is done at connection level.

       -A fields[/netmask]
	  Aggregate netflow data using the specified fields, where fields is a
	  ','  separated  list	out of proto srcip dstip srcport dstport srcas
	  dstas. The  default  is  using  all  fields:	proto,srcip,dstip,src-
	  port,dstport.	An additional netmask may be given. In that case flows
	  from the same	subnets	are aggregated.	In order to do proper aggrega-
	  tion,	 the  IP  version  is  important,  for which the mask applies.
	  Therefore the	IP protocol version must be given in the form of:  sr-
	  cip4/24  for	IPv4  or srcip6/64 for IPv6 address aggregation. Apply
	  the protocol version for dstip respectively.

       -I Print	flow statistics	from file specified by -r, or timeslot	speci-
	  fied	by  -R/-M.   The printed information corresponds to pre	nfdump
	  1.5 nfcapd stat files.

       -S Compatibility	 option	 with  pre  1.4	 nfdump.  Is   equal   to   -s
	  record/packets/bytes.

       -s statistic[:p][/orderby]
	  Generate the Top N flow or flow element statistic. statistic can be:
	    record  Statistic about arregated netflow records.
	    srcip   Statistic about source IP addresses
	    dstip   Statistic about destination	IP addresses
	    ip	    Statistic about any	(source	or destination)	IP addresses
	    srcport Statistic about source ports
	    dstport Statistic about destination	ports
	    port    Statistic about any	(source	or destination)	ports
	    tos	    Statistic about type of service
	    srcas   Statistic about source AS numbers
	    dstas   Statistic about destination	AS numbers
	    as	    Statistic about any	(source	or destination)	AS numbers
	    inif    Statistic about input interface
	    outif   Statistic about output interface
	    proto   Statistic about IP protocols
	  By  adding  :p  to  the  statistic  name, the	resulting statistic is
	  splitted up into transport layer  protocols.	Default	 is  transport
	  protocol independant statistics.
	  orderby  is optional and specifies the order by which	the statistics
	  is ordered and can be	flows, packets,	bytes, pps, bps	 or  bpp.  You
	  may  specify more than one orderby which results in the same statis-
	  tic but ordered differently. If no orderby is	given, statistics  are
	  ordered  by  flows.  You can specify as many -s flow element statis-
	  tics on the command line for the same	run.
	  Example:  -s	srcip  -s  ip/flows  -s	 dstport/pps/packets/bytes  -s
	  record/bytes

       -O orderby
	  Specifies  the default orderby for flow element statistics -s, which
	  applies when no orderby is given at -s. orderby can be flows,	 pack-
	  ets, bytes, pps, bps or bpp. Defaults	to flows.

       -l [+/-]packet_num
	  Limit	 statistics  output  to	 those	records	 above	or  below  the
	  packet_num limit. packet_num accepts positive	 or  negative  numbers
	  followed  by 'K' , 'M' or 'G'	10E3, 10E6 or 10E9 flows respectively.
	  See also note	at -L

       -L [+/-]byte_num
	  Limit	statistics output to those records above or below the byte_num
	  limit. byte_num accepts positive or negative numbers followed	by 'K'
	  , 'M'	or 'G' 10E3, 10E6 or 10E9 bytes	respectively. Note: These lim-
	  its  only  apply  to the statistics and aggregated outputs generated
	  with -a -s or	-S.  To	filter netflow records by packets  and	bytes,
	  use the filter syntax	'packets' and 'bytes' described	below.

       -n num
	  Define  the number for the Top N statistics. Defaults	to 10. If 0 is
	  specified the	number is unlimited.

       -o format
	  Selects the output format to print flows or flow  record  statistics
	  (-s record). The following formats are available:
	    raw	     Print each	file flow record on multiple lines.
	    line     Print each	flow on	one line. Default format.
	    long     Print each	flow on	one line with more details
	    extended Print each	flow on	one line with even more	details.
	    pipe     Machine readable format: Print all	fields '|' separated.
	    fmt:format User defined output format.
	  For  each  defined output format except -o fmt:<format> an IPv6 long
	  output format	exists.	 line6,	long6 and extended6. See output	formts
	  below	for more information.

       -K key
	  Anonymize  all  IP addresses using the CryptoPAn (Cryptography-based
	  Prefix-preserving Anonymization) module. The key is used to initial-
	  ize  the  Rijndael cipher. key is either a 32	character string, or a
	  64 hex digit string starting with 0x.	Anonymizing takes place	 after
	  applying  the	 flow  filter, but before printing the flow or writing
	  the flow to a	file.

	  See http://www.cc.gatech.edu/computing/Telecomm/cryptopan/ for  more
	  information about CryptoPAn.

       -q Suppress the header line and the statistics at the bottom.

       -N Print	the numbers in the summary line	as plain numbers. Better pars-
	  ing.

       -z Zero flows. Do not dump flows	into the output	 file,	but  only  the
	  statistics record.

       -Z Check	filter syntax and exit.	Sets the return	value accordingly.

       -X Compiles  the	filer syntax and dumps the filter engine table to std-
	  out.	This is	for debugging purpose only.

       -V Print	nfdump version and exit.

       -h Print	help text on stdout with all options and exit.

RETURN VALUE
       Returns
	   0   No error.
	   255 Initialization failed.
	   254 Error in	filter syntax.
	   250 Internal	error.

OUTPUT FORMATS
       The output format raw prints each flow record on	 multiple  lines,  in-
       cluding	all  information available in the record. This is the most de-
       tailed view on a	flow.

       Other output formats print each flow on a single	line. Predefined  out-
       put  formats  are line, long and	extended The output format line	is the
       default output format when no format is specified.  It limits  the  im-
       formation to the	connection details as well as number of	packets, bytes
       and flows.

       The output format long is identical to the format  line,	 and  includes
       additional information such as TCP flags	and Type of Service.

       The  output  format  extended  is identical to the format long, and in-
       cludes additional computed information such as pps, bps and bpp.

       Fields:

	  Date flow start: Start time flow first seen. ISO 8601	format includ-
	  ing miliseconds.

	  Duration: Duration of	the flow in seconds and	miliseconds.  If flows
	  are aggregated, duration is the time span over the entire periode of
	  time from first seen to last seen.

	  Proto: Protocol used in the connection.

	  Src IP Addr:Port: Source IP address and source port.

	  Dst  IP  Addr:Port: Destination IP address and destination port.  In
	  case of ICMP,	port is	decodes	as type.code.

	  Flags: TCP flags ORed	of the connection.

	  Tos: Type of service.

	  Packets: The number of packets in this flow.	If  flows  are	aggre-
	  gated, the packets are summed	up.

	  Bytes:  The  number  of bytes	in this	flow. If flows are aggregated,
	  the bytes are	summed up.

	  pps: The calculated packets per second: number of  packets  /	 dura-
	  tion.	  If flows are aggregated this results in the average pps dur-
	  ing this periode of time.

	  bps: The calculated bits per second: 8 * number of bytes / duration.
	  If  flows are	aggregated this	results	in the average bps during this
	  periode of time.

	  Bpp: The calculated bytes per	packet:	number of bytes	 /  number  of
	  packets.  If	flows  are  aggregated this results in the average bpp
	  during this periode of time.

	  Flows: Number	of flows. If flows are listed only, this number	is al-
	  wasy 1. If flows are aggregated, this	shows the number of aggregated
	  flows	to one record.

       Numbers larger than 1048576 (1024*1024),	are scaled to 4	digits and one
       decimal	digit  including the scaling factor M, G or T for cleaner out-
       put, e.g. 923.4 M

       To make the output more readable, IPv6 addresses	are shrinked  down  to
       16 characters. The seven	most and seven least digits connected with two
       dots '..' are displayed in any normal output formats.  To  display  the
       full IPv6 address, use the appropriate long format, which is the	format
       name followed by	a 6.

       Example:	-o line	displays an IPv6 address as 2001:23..80:d01e where  as
       the   format  -o	 line6	displays  the  IPv6  address  in  full	length
       2001:234:aabb::211:24ff:fe80:d01e.  The combination of -o  line	-6  is
       equivalent to -o	line6.

       The  pipe  output format	is intended to be read by another programm for
       further processing.  Values are separated by a '|'.  IP	addresses  are
       printed as 4 consecutive	32bit numbers.	Output sequence:

	  Address family  PF_INET or PF_INET6
	  Time first seen UNIX time seconds
	  msec first seen Mili seconds first seen
	  Time last seen  UNIX time seconds
	  msec last seen  Mili seconds first seen
	  Protocol	  Protocol
	  Src address	  Src address as 4 consecutive 32bit numbers.
	  Src port	  Src port
	  Dst address	  Dst address as 4 consecutive 32bit numbers.
	  Dst port	  Dst port
	  Src AS	  Src AS number
	  Dst AS	  Dst AS number
	  Input	IF	  Input	Interface
	  Output IF	  Output Interface
	  TCP Flags	  TCP Flags
				000001 FIN.
				000010 SYN
				000100 RESET
				001000 PUSH
				010000 ACK
				100000 URGENT
				e.g. 6 => SYN +	RESET
	  Tos		  Type of Service
	  Packets	  Packets
	  Bytes		  Bytes
       For  IPv4 addresses only	the last 32bit integer is used.	All others are
       set to zero.

       The output format fmt:<format> allows you to  define  your  own	output
       format.	A format description format consists of	a single line contain-
       ing arbitrary strings and format	specifier as described below

	  %ts	Start Time - first seen
	  %te	End Time - last	seen
	  %td	Duration
	  %pr	Protocol
	  %sa	Source Address
	  %da	Destination Address
	  %sap	Source Address:Port
	  %dap	Destination Address:Port
	  %sp	Source Port
	  %dp	Destination Port
	  %sas	Source AS
	  %das	Destination AS
	  %in	Input Interface	num
	  %out	Output Interface num
	  %pkt	Packets
	  %byt	Bytes
	  %fl	Flows
	  %pkt	Packets
	  %flg	TCP Flags
	  %tos	Tos
	  %bps	bps - bits per second
	  %pps	pps - packets per second
	  %bpp	bps - Bytes per	package

       For example the standard	output format long can be created as

       -o "fmt:%ts %td %pr %sap	-> %dap	%flg %tos %pkt %byt %fl"

       You may also define your	own output format and have  it	compiled  into
       nfdump.	See nfdump.c around line 100 for more details.

FILTER
       The  filter  syntax  is	similar	to the well known pcap library used by
       tcpdump.	 The filter can	be either specified on the command line	 after
       all  options or in a separate file. It can span several lines. Anything
       after a '#' is treated as a comment and ignored to the end of the line.
       There is	virtually no limit in the length of the	filter expression. All
       keywords	are case independent.

       Any filter consists of one or more expressions expr. Any	number of expr
       can be linked together:

       expr and	expr, expr or expr, not	expr and ( expr	).

       Expr can	be one of the following	filter primitives:

       protocol	version
	   inet	for IPv4 and inet6 for IPv6

       protocol
	   proto  <protocol>  where protocol can be any	known protocol such as
	   TCP,	UDP, ICMP, ICMP6 GRE, ESP, AH, or a valid protocol number.

       IP address
	   [SourceDestination] IP <ipaddr> or
	   [SourceDestination] HOST <ipaddr> with _ipaddr_ as any  valid  IPv4
	   or IPv6 address.  SourceDestination may be omitted.

       IP in [ _iplist_	]
	   [SourceDestination] IP in [<iplist>]
	   [SourceDestination] host in [<iplist>]
	   iplist space	separated list of individual <ipaddr>

       SourceDestination
	   defines  the	 IP  address to	be selected and	can be SRC, DST	or any
	   combination of SRC and|or DST. Ommiting SourceDestination is	equiv-
	   alent to SRC	or DST.

       inout
	   defines the interface to be selected	and can	be IN or OUT.

       network
	   [SourceDestination]	NET  a.b.c.d m.n.r.s. for IPv4 with m.n.r.s as
	   netmask.
	   [SourceDestination] NET _net_ / num with _net_ as a valid  IPv4  or
	   IPv6	 network  and  num  as	maskbits. The number of	mask bits must
	   match the appropriate address familiy IPv4 or IPv6. Networks	may be
	   abreviated such as 172.16/16	if they	are unambiguous.

       Port
	   [SourceDestination]	 PORT [comp] num with num as a valid port num-
	   ber.	 If comp is omitted, '=' is assumed.

       Interface
	   [inout]  IF num with	num as an interface number.

       Flags
	   flags tcpflags with tcpflags	as a combination of:
	   A	ACK.
	   S	SYN.
	   F	FIN.
	   R	Reset.
	   P	Push.
	   U	Urgent.
	   X	All flags on.
       The ordering of the flags is not	 relevant.  Flags  not	mentioned  are
       treated	as  don't care.	 In order to get those flows with only the SYN
       flag set, use the syntax	'flags S and not flags AFRPU'.

       TOS Type	of service: tos	value with value 0..255.

       Packets
	   packets [comp] num [scale] to specify the packet count in the  net-
	   flow	record.

       Bytes
	   bytes  [comp]  num [scale] to specify the byte count	in the netflow
	   record.

       Packets per second: Calculated value.
	   pps [comp] num [scale] to specify the pps of	the flow.

       Duration: Calculated value
	   duration [comp] num to specify the duration in miliseconds  of  the
	   flow.

       Bits per	second:	Calculated value.
	   bps [comp] num [scale] to specify the bps of	the flow.

       Bytes per packet: Calculated value.
	   bpp [comp] num [scale] to specify the bpp of	the flow.

       AS  [SourceDestination]	AS num with num	as a valid AS number.

       scale scaling factor. Maybe k m g. Factor is 1024

       comp The	following comparators are supported:
	   =, ==, >, <,	EQ, LT,	GT .  If comp is omitted, '=' is assumed.

EXAMPLES
       nfdump  -r  /and/dir/nfcapd.200407110845	 -c  100  'tcp	and  (	src ip
       172.16.17.18 or dst ip 172.16.17.19 )'  Dumps  the  first  100  netflow
       records which match the given filter:

       nfdump	 -R   /and/dir/nfcapd.200407110845:nfcapd.200407110945	 'host
       192.168.1.2' Dumps all netflow records of host 192.168.1.2 from July 11
       08:45 - 09:45

       nfdump  -M /to/and/dir1:dir2 -R nfcapd.200407110845:nfcapd.200407110945
       -S -n 20	Generates the Top 20 statistics	from 08:45  to	09:45  from  3
       sources

       nfdump  -r  /and/dir/nfcapd.200407110845	-S -n 20 -o extended Generates
       the Top 20 statistics, extended output format

       nfdump -r /and/dir/nfcapd.200407110845 -S -n 20 'in if 5	and bps	> 10k'
       Generates the Top 20 statistics from flows comming from interface 5

       nfdump  -r /and/dir/nfcapd.200407110845 'inet6 and tcp and ( src	port >
       1024 and	dst port 80 ) Dumps all	port 80	IPv6 connections  to  any  web
       server.

NOTES
       Generating  the	statistics  for	 data  files of	a few hundred MB is no
       problem.	However	be careful if you want to create statistics of several
       GB  of  data.  This  may	 consume a lot of memory and can take a	while.
       Also, anonymizing IP addresses is time consuming	and uses a lot of  CPU
       power,  which  reduces  the  number  of	flows  per  second.  Therefore
       anonymizing takes place only, when flow records are printed or  written
       to  files.  Any internal	flow processing	takes place using the original
       IP addresses.

SEE ALSO
       nfcapd(1), nfprofile(1),	nfreplay(1)

BUGS
       There is	still the famous last bug. Please report them -	all  the  last
       bugs - back to me.

				  2005-08-19			     nfdump(1)

NAME | SYNOPSIS | DESCRIPTION | OPTIONS | RETURN VALUE | OUTPUT FORMATS | FILTER | EXAMPLES | NOTES | SEE ALSO | BUGS

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=nfdump&sektion=1&manpath=FreeBSD+8.2-RELEASE+and+Ports>

home | help