Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
YAFMETA2PCAP(1)		     Yet Another Flowmeter	       YAFMETA2PCAP(1)

NAME
       yafMeta2Pcap - YAF PCAP Metadata	File Parser and	PCAP Creator

SYNOPSIS
	   yafMeta2Pcap	[--pcap-meta-file META_FILE_PATTERN]
			[--pcap	PCAP_FILE_PATTERN]
			[--caplist CAP_FILE_NAME]
			[--metalist META_FILE_NAME]
			[--out OUTPUT_SPECIFIER]
			[--hash	FLOW_KEY_HASH]
			[--time	FLOW_START_MILLISECONDS]
			[--etime FLOW_END_MILLISECONDS]
			[--packets NUM_PACKETS]
			[--yaf-program PATH_TO_YAF]
			[--verbose]

DESCRIPTION
       yaf has several PCAP export options available. yaf will create a	pipe-
       delimited file that contains information	to assemble a PCAP file	for
       any particular flow. yafMeta2Pcap parses	the metadata file created by
       yaf and creates a PCAP file for the given flow identified by the	flow
       key hash	and start time.

       yafMeta2Pcap has	the ability to read IPFIX from getFlowKeyHash on stdin
       to avoid	typing the flow	key hash and start time	on the command line.
       yafMeta2Pcap will accept	the first 2 records from getFlowKeyHash.  If
       the first 2 records have	the same start time, yafMeta2Pcap will sum the
       values for packetTotalCount to determine	the total number of packets to
       search for (to account for uniflows).  yafMeta2Pcap will	only search
       for the first hash and start time it receives.  See below for examples
       for using IPFIX input.

OPTIONS
       The following options are required for yafMeta2Pcap to create the PCAP
       file.

       --pcap-meta-file	META_FILE_PATTERN
	   META_FILE_NAME is the path to the pcap-meta-file as created by yaf.
	   META_FILE_NAME is a file glob pattern, which	must be	escaped	or
	   quoted to prevent the shell expansion if using wildcards.  Files
	   that	match this pattern will	be processed by	yafMeta2Pcap.
	   Alternatively, you can use --metalist to provide a text file	with
	   an ordered list of pcap-meta-files created by yaf.  Alternatively,
	   you can set the YAF_PCAP_META_FILE environment variable to the full
	   path	to the pcap-meta-file created by yaf to	avoid including	it to
	   the command line.

       --pcap PCAP_FILE_NAME
	   PCAP_FILE_NAME is the path to the PCAP file that was	given to yaf
	   to create the pcap-meta-file.  PCAP_FILE_NAME is a file glob
	   pattern, which must be escaped or quoted to prevent shell expansion
	   if using wildcards.	The option is only required if full path is
	   not available in the	PCAP metadata file. yafMeta2Pcap will open
	   subsequent PCAPs only when the third	column in the PCAP-meta-file
	   changes.  For example, if the third column is 0, yafMeta2Pcap will
	   open	the first PCAP given to	--pcap.	Alternatively, you can use
	   --caplist to	give a text file with an ordered list of PCAP files.

       --caplist CAP_FILE_NAME
	   Instead of listing multiple PCAP files, use CAP_FILE_NAME as	an
	   ordered newline-delimited list of pathnames to PCAP dumpfiles as
	   read	by yaf.	 Alternatively,	you can	set the	YAF_CAPLIST
	   environment variable	to the full path of the	CAP_FILE_NAME to avoid
	   adding it to	the command line.

       --metalist META_FILE_NAME
	   Instead of listing multiple PCAP metadata files, use	META_FILE_NAME
	   as an ordered newline-delimited list	of pathnames to	PCAP metadata
	   files as created by yaf.  Alternatively, you	can set	the
	   YAF_PCAP_META_LIST environment variable to the META_FILE_NAME to
	   avoid adding	it to the command line.

       --out OUTPUT_SPECIFIER
	   The full path of the	PCAP file that will be created containing just
	   the packets from the	given flow.  If	this option is not present,
	   yafMeta2Pcap	will simply write the filename(s) of the PCAP(s) that
	   contained the flow with the given FLOW_KEY_HASH. yafMeta2Pcap will
	   overwrite any existing file at OUTPUT_SPECIFIER.

       --hash FLOW_KEY_HASH
	   The flow key	hash of	the flow as created by getFlowKeyHash.	Use
	   getFlowKeyHash with the flow's 5-tuple and vlan to calculate	the
	   flow	key hash.  If this option is not present, yafMeta2Pcap assumes
	   IPFIX will be present on stdin.

       --time FLOW_START_MILLISECONDS
	   The milliseconds since epoch	time of	the start time of the flow.
	   getFlowKeyHash can calculate	the milliseconds since epoch given the
	   date	and time.  Not required, but suggested.	 If
	   FLOW_START_MILLISECONDS is not provided, all	packets	matching the
	   FLOW_KEY_HASH will be given,	which may include multiple flows.

       --etime FLOW_END_MILLISECONDS
	   The milliseconds since epoch	time of	the end	time of	the flow.
	   This	value is exported by getFlowKeyHash if present and accepted by
	   yafMeta2Pcap.  Not required,	but suggested. Including this value
	   will	likely speed up	search times.  This argument overrides any end
	   time	information collected via IPFIX	from getFlowKeyHash/

       --packets NUM_PACKETS
	   NUM_PACKETS is the total number of packets for the bi-flow.	This
	   optional argument may speed up processing because it	tells
	   yafMeta2Pcap	when it	has found the entire flow.  This can also be
	   used	if you only want the first NUM_PACKETS of the flow instead of
	   the entire flow.  This argument overrides any packet	count
	   information collected via IPFIX from	getFlowKeyHash.

       --yaf-program PATH_TO_YAF
	   PATH_TO_YAF is the location of the yaf program to use to create the
	   PCAP	file if	OUTPUT_SPECIFIER is present and	the pcap-meta-file has
	   the format:

	   hash	| stime	| path_to_pcap

	   If not specified, yafMeta2Pcap assumes there	is program yaf on your
	   $PATH.

       --verbose
	   If present, print debug information to stdout.

Examples
       In the following	examples, the dollar sign ("$")	represents the shell
       prompt.	The text after the dollar sign represents the command line.
       Lines have been wrapped for improved readability, and the back slash
       ("\") is	used to	indicate a wrapped line.

	The following example shows how	to run yaf, getFlowKeyHash, and
	yafMeta2Pcap to	create a PCAP for a given flow.

	 $ yaf --in /data/some_large_file.pcap --out /data/ipfix.yaf \
	      --pcap-meta-file /tmp/pcap-info

	 $ getFlowKeyHash --sip4 10.10.25.11 --dip4 10.10.26.12	--sport	1586 \
			--dport	80 --protocol 6	--vlan 3  --date 2011-03-21 \
			--time 16:15:54.256

		     sIP|	     dIP|sPort|dPort|pro| vlan|	     hash|		    ms
	     10.10.25.11|    10.10.26.12| 1586|	  80|  6|    3|	100819735|	 1300724154256

	 FILE PATH: 256/100819735-2011321161554_0.pcap

	 $ yafMeta2Pcap	--pcap-meta-file /tmp/pcap-info20150107141533_00000.meta \
			--pcap /data/some_large_file.pcap \
		       --out /data/my-new-pcap.pcap \
		       --hash 100819735	--time 1300724154

	The following example shows how	to run getFlowKeyHash directly to
	yafMeta2Pcap and use environment variables.

	  $ export YAF_PCAP_META_FILE=/tmp/pcap-info20150107141533_00000.meta
	  $ export YAF_CAPLIST=/tmp/yafcaplist.txt
	  $ getFlowKeyHash --sip4 10.10.25.11 --dip4 10.10.26.12 --sport 1586 \
			--dport	80 --protocol 6	--vlan 3  --date 2011-03-21 \
			--time 16:15:54.256 -I | yafMeta2Pcap -o /data/my-new-pcap -n 10

	The following example demonstrates how to use SiLK tools with
	getFlowKeyHash and yafMeta2Pcap.  This assumes the appropriate
	environment variables are set.

	  $ rwfilter --application=80 --pass-dest=stdout --max-pass-records=2 file.rw |	\
	       rwsilk2ipfix | getFlowKeyHash -I	| yafMeta2Pcap -o /data/mynewpcap.pcap

	The following example shows how	to configure yaf to create rolling
	PCAP files and use yafMeta2Pcap	to create a PCAP for a specific	flow.

	 $ yaf --in eth0 --out localhost --ipfix tcp --ipfix-port 18000	\
	       --pcap-meta-file	/tmp/pcap-info --pcap /data/pcap \
	       --live pcap --index-pcap

	 $ getFlowKeyHash --sip4 10.10.25.11 --dip4 10.10.26.12	--sport	1586 \
			  --dport 80 --protocol	6 --vlan 3  --date 2011-03-21 \
			  --time 16:15:54.256

		    sIP|	    dIP|sPort|dPort|pro| vlan|	    hash|		   ms
	    10.10.25.11|    10.10.26.12| 1586|	 80|  6|    3| 100819735|	1300724154256

	 FILE PATH: 256/100819735-2011321161554_0.pcap

	 $ yafMeta2Pcap	--pcap-meta-file /tmp/pcap-info20150107141533_00000.meta \
			--out /data/my-new-pcap.pcap \
			--hash 100819735 --time	1300724154

	The following example shows how	to run yaf creating rolling PCAP files
	and use	yafMeta2Pcap to	create a PCAP file for a particular flow.

	 $ yaf --in eth0 --out localhost --ipfix tcp --ipfix-port 18000	\
	       --pcap-meta-file	/tmp/pcap-info --pcap /data/pcap \
	       --live pcap

	 $ yafMeta2Pcap	--pcap-meta-file="/tmp/pcap-info*"\
			-o /data/new.pcap \
			--hash 100819735 --time	1300724154

	The following example shows how	to run yaf over	a list of pcap files
	and use	yafMeta2Pcap to	create a PCAP for a specific flow

	 $ yaf --in /tmp/caplist.txt --out /tmp/yaffile.yaf \
	       --pcap-meta-file	/tmp/pcap-info --caplist

	 $ yafMeta2Pcap	--caplist /tmp/caplist.txt -o flow.pcap	\
			--hash 100819735 --time	1300724154 \
			--pcap-meta-file "/tmp/pcap-info*" -v

	 Found 1 packets that match criteria.

Known Issues
       If the pcap-meta-file was generated in a	way that it has	the form:

       hash | stime | path_to_pcap

       and an OUTPUT_SPECIFIER is provided to --out, yafMeta2Pcap creates a
       temporary file with the PCAP file names that contain the	flow it	is
       searching for.  This temporary file is placed in	/tmp by	default, but
       the location can	be configured through the use of the TMPDIR
       environment variable.  The temporary file is deleted upon termination
       of yafMeta2Pcap.

       Bug reports may be sent directly	to the Network Situational Awareness
       team at <netsa-help@cert.org>.

AUTHORS
       Emily Sarneso and the CERT/CC Engineering Team,
       <http://www.cert.org/netsa>.

SEE ALSO
       yaf(1), getFlowKeyHash(1)

2.8.4				   3-Jul-2017		       YAFMETA2PCAP(1)

NAME | SYNOPSIS | DESCRIPTION | OPTIONS | Examples | Known Issues | AUTHORS | SEE ALSO

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=yafMeta2Pcap&sektion=1&manpath=FreeBSD+12.0-RELEASE+and+Ports>

home | help