Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
silk-plugin(3)			SiLK Tool Suite			silk-plugin(3)

NAME
       silk-plugin - Creating a	SiLK run-time plug-in using C

SYNOPSIS
	sk_cc=`silk_config --compiler`
	sk_cflags=`silk_config --cflags`
	$sk_cc $sk_cflags -shared -o FILENAME.so FILENAME.c

	rwfilter --plugin=FILENAME.so [--plugin=FILENAME.so ...] ...

	rwcut --plugin=FILENAME.so [--plugin=FILENAME.so ...]
	      --fields=FIELDS ...

	rwgroup	--plugin=FILENAME.so [--plugin=FILENAME.so ...]
	      --id-fields=FIELDS ...

	rwsort --plugin=FILENAME.so [--plugin=FILENAME.so ...]
	      --fields=FIELDS ...

	rwstats	--plugin=FILENAME.so [--plugin=FILENAME.so ...]
	      --fields=FIELDS --values=VALUES ...

	rwuniq --plugin=FILENAME.so [--plugin=FILENAME.so ...]
	      --fields=FIELDS --values=VALUES ...

DESCRIPTION
       Several of the SiLK analysis tools allow	the user to augment the	tools'
       functionality through the use of	plug-ins that get loaded at run-time.
       These tools are:

       rwfilter(1)
	   Supports adding new switches	to determine whether each SiLK Flow
	   record should be written in the --pass or the --fail	output stream.

       rwcut(1)
	   Supports adding new output fields that, when	selected using the
	   --fields switch, appear as a	column in the output.

       rwsort(1)
	   Supports adding new key fields that,	when selected using the
	   --fields switch, are	used to	determine the order in which records
	   are sorted.

       rwgroup(1)
	   Supports adding new key fields that,	when selected using the
	   --id-fields switch, are used	to determine how records are grouped.

       rwuniq(1)
	   Supports adding new key fields that,	when selected using the
	   --fields switch, are	used to	bin (i.e., group) the records.	In
	   addition, rwuniq supports adding new	aggregate value	fields that,
	   when	selected using the --values switch, will be computed for each
	   bin.	 The key and value fields will appear in the output.

       rwstats(1)
	   Supports adding new key fields that,	when selected using the
	   --fields switch, are	used to	bin (i.e., group) the records.	In
	   addition, rwstats supports adding new aggregate value fields	that,
	   when	selected using the --values switch, will be computed for each
	   bin and can be used to determine the	top-N (or bottom-N) bins.  The
	   key and value fields	will appear in the output for bins that	meet
	   the top-N threshold.

       rwptoflow(1)
	   Supports adding functionality to ignore packets in the pcap(3)
	   input stream	or to modify the SiLK Flow records as the records are
	   generated.

       In addition, all	of the above tools support adding new command line
       switches	that can be used to initialize the plug-in itself (for
       example,	to load	an auxiliary file that the plug-in requires).

       The plug-ins for	all tools except rwptoflow can be written in either C
       or using	PySiLK (the SiLK Python	extension, see pysilk(3)).  Although
       the execution time for PySiLK plug-ins is slower	than for C plug-ins,
       we encourage you	to use PySiLK for your plug-ins	since the time-to-
       result can be faster for	PySiLK:	The faster development time in Python
       typically more than compensates for the slower execution	time.  Once
       you find	that your PySiLK plug-in is seeing a great deal	of use,	or
       that PySiLK is just too slow for	the amount of data you are processing,
       then re-write the plug-in using C.  Even	when you intend	to write a
       plug-in using C,	it can be helpful to prototype your plug-in using
       PySiLK.

       The remainder of	this document explains how to create a plug-in for the
       SiLK analysis tools (except rwptoflow) using the	C programming
       language.  For information on creating a	plug-in	using PySiLK, see
       silkpython(3).

       A template file for plug-ins is included	in the SiLK source tree, in
       the silk-VERSION/src/template/c-plugin.c	file.

   The setup function
       When you	provide	"--plugin=my-plugin.so"	on the command line to an
       application, the	application loads the my-plugin.so file	and calls a
       setup function in that file to determine	the new	switches and/or	fields
       that "my-plugin.so" provides.

       This setup function is called with three	arguments: the first two
       describe	the version of the plug-in API,	and the	third is a pointer
       that is currently unused.

	skplugin_err_t SKPLUGIN_SETUP_FN(
	    uint16_t	major_version,
	    uint16_t	minor_version,
	    void       *plug_in_data)
	{
	    ...
	}

       There are several tasks this setup function may do: (1) check the API
       version,	(2) register new command line switches (if any), (3) register
       new filters (if any), and (4) register new fields (if any).  Let's
       describe	these in more detail.

       (1) Check the API version

       The setup function should ensure	that the plug-in and the application
       agree on	the API	to use.	 This provides protection in case the SiLK API
       to plug-ins changes in the future.  To make this	determination, call
       the skpinSimpleCheckVersion() function.	A typical invocation is	shown
       here, where the "major_version" and "minor_version" were	passed into
       the SKPLUGIN_SETUP_FN, and "PLUGIN_API_VERSION_MAJOR" and
       "PLUGIN_API_VERSION_MINOR" are macros defined in	the template file to
       the current version of the API.

	#define	PLUGIN_API_VERSION_MAJOR 1
	#define	PLUGIN_API_VERSION_MINOR 0

	/* Check the plug-in API version */
	rv = skpinSimpleCheckVersion(major_version, minor_version,
				     PLUGIN_API_VERSION_MAJOR,
				     PLUGIN_API_VERSION_MINOR,
				     skAppPrintErr);
	if (rv != SKPLUGIN_OK) {
	    return rv;
	}

       (2) Register command line switches

       If the plug-in wants to define new command line switches, those
       switches	must be	registered in the setup	function.  A typical use of a
       command line switch is to allow the user	to configure the plug-in; for
       example,	the switch may allow the user to specify the location of an
       auxiliary input file that the plug-in requires, or to set a parameter
       used by the plug-in.

       A second	use for	a command line switch is more subtle.  When creating a
       plug-in for rwfilter, you may want your plug-in to provide several
       similar features, and only enable each feature when the user requests
       it via a	command	line switch.  For this case, you want to delay
       registering the filter until the	command	line switch is seen, in	which
       case the	filter registration function should be invoked in the switch's
       callback	function.

       Information on registering a command line switch	is available below
       ("Registering command line switches").

       (3) Register filters

       You only	need to	register filters when the plug-in will be used by
       rwfilter(1).  You may choose to register	the filters in the setup
       function; if you	do, the	filter will always be used when	the plug-in is
       loaded by rwfilter.  If you the plug-in provides	several	filtering
       functions that the user may choose from via command line	switches, you
       should call the filter registration function in the callback function
       for the command line switch.

       See "Registering	filter functions" for details on registering a
       function	to use with rwfilter.

       (4) Register fields

       If you want your	plug-in	to create a new	printable field	for rwcut(1),
       a new sorting field for rwsort(1), a new	grouping field for rwgroup(1),
       rwstats(1), or rwuniq(1), or a new aggregate value field	for rwstats or
       rwuniq, you should register those fields	in the setup function.	(While
       you can register	the fields in a	switch's callback function, there is
       usually little reason to	do so.)

       There are two interfaces	to registering a new field:

       1.  The advanced	interface provides complete control over how the field
	   is defined, and allows (or forces) you to specify exactly how to
	   map from a SiLK Flow	record to a binary representation to a textual
	   representation.  To use the advanced	interface you will need	to
	   define several functions and	fill in	a C structure with pointers to
	   those functions.  This interface is described in the	"Advanced
	   field registration function"	section	below.

       2.  The simple interface	can be used to define fields that map to an
	   integer value, an IP	address, or text that is index by an integer
	   value.  To use this interface, you need to define only one or two
	   functions.  The simple interface should handle many common cases,
	   and it is described in "Simple field	registration functions".

   Registering command line switches
       When you	register a switch, the two important pieces of information you
       must provide are	a name for the switch and a callback function.	When
       the application encounters the command line switch registered by	your
       plug-in,	the application	will invoke the	callback function with the
       parameter that the user provided	(if any) to the	command	line switch.

       To register a command line switch, call the skpinRegOption2() function:

	skplugin_err_t skpinRegOption2(
	    const char		 *option_name,
	    skplugin_arg_mode_t	  mode,
	    const char		 *option_help_string,
	    skplugin_help_fn_t	  option_help_fn,
	    skplugin_option_fn_t  opt_process_fn,
	    void		 *opt_callback_data,
	    int			  num_fn_mask,
	    ...);  /* list of skplugin_fn_mask_t */

       The parameters are

       "option_name"
	   Specifies the command line switch to	create.	 Do not	include	the
	   leading "--"	characters in the name.

       "mode"
	   Determines whether the switch takes an argument.  It	should be one
	   of

	   "NO_ARG"
	       when the	command	line option acts as an on/off switch

	   "OPTIONAL_ARG"
	       when the	command	line option has	a default value, or

	   "REQUIRED_ARG"
	       when the	user of	the plug-in must provide an argument to	the
	       command line option.

       "option_help_string"
	   This	parameter specifies the	usage string to	print when the user
	   requests --help from	the application.  This parameter may be	NULL.
	   Alternatively, you may instruct the application to generate a help
	   string by invoking a	callback function your plug-in provides, as
	   described next.

       "option_help_fn"
	   This	parameter specifies a pointer to a function that the
	   application will to call to print a help message for	the command
	   line	switch when the	user requests --help from the application.
	   This	parameter may be NULL; if it is	not NULL, the
	   "option_help_string"	value is ignored.  The signature of the
	   function to provide is

	    void option_help_fn(
		FILE		    *file_handle,
		const struct option *option,
		void		    *opt_callback_data);

	   The "file_handle" argument is where the function should print its
	   help	message.  The "opt_callback_data" is the value provided	to
	   skpinRegOption2() when the option was registered.  The "struct
	   option" parameter has two members of	interest: "name" contains the
	   number used to register the option, and "has_arg" contains the
	   "mode" that was used	when the option	was specified.

       "opt_process_fn"
	   Specifies the callback function, whose signature is

	    skplugin_err_t opt_process_fn(
		const char *opt_arg,
		void	   *opt_callback_data);

	   The application will	call
	   "opt_process_fn(opt_arg,opt_callback_data)" when --option_name is
	   seen	as a command line argument.  "opt_arg" will be the parameter
	   the user passed to the switch, or it	will be	NULL if	no parameter
	   was given.

       "opt_callback_data"
	   Will	be passed back unchanged to the	plug-in	as a parameter in the
	   opt_process_fn() and	option_help_fn() callback functions.

       "num_fn_mask"
	   Specifies the number	of "skplugin_fn_mask_t"	values specified as
	   the final argument(s) to skpinRegOption2().

       "..."
	   Specifies a list of "skplugin_fn_mask_t" values.  The length	of
	   this	list must be specified in the "num_fn_mask" parameter.	A
	   plug-in file	(e.g., my-plugin.so) can be loaded into	any SiLK tool
	   that	supports plug-ins, but you may want a command line switch to
	   appear only in certain applications.	 For example, the flowrate(3)
	   plug-in can be used in both rwfilter	and rwcut.  When used by
	   rwfilter, flowrate provides a --bytes-per-second switch; when used
	   by rwcut, that switch is not	available, and instead the bytes/sec
	   field becomes available.  This list determines in which
	   applications	the switch gets	defined, and the list should contain
	   the "SKPLUGIN_FN_*" or "SKPLUGIN_APP_*" macros defined in
	   skplugin.h.	To make	the switch available in	all applications,
	   specify "SKPLUGIN_FN_ANY".  When skpinRegOption2() is called	in an
	   the application that	does not match a value in this list, the
	   function returns SKPLUGIN_ERR_DID_NOT_REGISTER, indicating that
	   this	option is not applicable to the	application.

   Registering filter functions
       When you	register a filter function, you	are specifying a function that
       rwfilter	will call for every SiLK Flow record that rwfilter reads from
       its input files.	 If the	function returns "SKPLUGIN_FILTER_PASS",
       rwfilter	writes the record into the stream(s) specified by --pass.  The
       record goes to the --fail streams if the	function returns
       "SKPLUGIN_FILTER_FAIL".

       (The previous paragraph is true only when the plug-in is	the only
       filtering predicate.  When multiple tests are specified on the rwfilter
       command line, rwfilter will put the record into the fail	destination as
       soon as any test	fails.	If there are multiple tests, your plug-in
       function	will only see records that have	not yet	failed a test.	If a
       plug-in filter function follows your function, it may fail a record
       that your filter	function passed.)

       To register a filter function, call the following function:

	skplugin_err_t skpinRegFilter(
	    skplugin_filter_t	      **filter_handle,
	    const skplugin_callbacks_t *regdata,
	    void		       *cbdata);

       "filter_handle"
	   When	this parameter is not NULL, skpinRegFilter() will set the
	   location it references to the newly created filter.	Currently, no
	   other function accepts the "skplugin_filter_t" as an	argument.

       "cbdata"
	   This	parameter will be passed back unchanged	to the plug-in as a
	   parameter in	the various callback functions.	 It may	be NULL.

       "regdata"
	   This	structure has a	member for every possible callback function
	   the SiLK plug-in API	supports.  When	used by	skpinRegFilter(), the
	   following members are supported.

	   "filter"
	       rwfilter	invokes	this function for each SiLK flow record.  If
	       the function returns SKPLUGIN_FILTER_PASS, the record is
	       accepted; if it returns SKPLUGIN_FILTER_FAIL, the record	is
	       rejected.  The type of the function is a
	       "skplugin_filter_fn_t", and its signature is:

		skplugin_err_t filter(
		    const rwRec	 *rec,
		    void	 *cbdata,
		    void	**extra);

	       where "rec" is the SiLK Flow record, "cbdata" is	the "cbdata"
	       specified in skpinRegFilter(), and "extra" will likely be
	       unused.

	   "init"
	       rwfilter	invokes	this function for all registered filter
	       predicates.  It is called after argument	processing and before
	       reading records.	 The function's	type is
	       "skplugin_callback_fn_t"	and the	function pointer may be	NULL.
	       The callback's signature	is

		skplugin_err_t init(
		    void *cbdata);

	   "cleanup"
	       When this function pointer is non-NULL, rwfilter	calls this
	       function	after all records have been processed.	This function
	       has the same type and signature as the "init" function.

       The function's return value will	be "SKPLUGIN_OK" unless	the "filter"
       member of the "regdata" structure is NULL.

       If your plug-in registers a filter function and the plug-in is used in
       an application other that rwfilter, the call to skpinRegFilter()	is a
       no-op.

   Simple field	registration functions
       Using a plug-in,	you can	augment	the keys available in the --fields
       switch on rwcut(1), rwgroup(1), rwsort(1), rwstats(1), and rwuniq(1),
       and provide new aggregate value fields for the --values switch on
       rwstats and rwuniq.

       The standard field registration function, skpinRegField(), is
       powerful---for example, you can control exactly how the value you
       compute will be printed.	 However, that power comes with	complexity.
       Many times, all your plug-in needs to do	is to compute a	value, and
       having to write a function to print a number is work with little
       reward.	The functions in this section handle the registration of
       common field types.

       All of these functions require a	name for the new field.	 The name is
       used as one of the arguments to the --fields or --values	switch,	and
       the name	will also be used as the title when the	field is printed (as
       in rwcut).  Field names are case	insensitive, and all field names must
       be unique within	an application.	 You will get a	run-time error if you
       attempt to create a field whose name already exists.  (In rwuniq	and
       rwstats,	you may	have a --fields	key and	a --values aggregate value
       with the	same name.)

       The callback functions dealing with integers use	"uint64_t" for
       convenience, but	internally the value will be stored in a smaller
       integer field if	possible.  Specifying the "max"	parameter to the
       largest value you actually use may allow	SiLK to	use a smaller integer
       field.

       The functions in	this section return "SKPLUGIN_OK" unless the callback
       function	is NULL.

       Integer key field

       The following function is used to register a key	field whose value is
       an unsigned 64 bit integer.

	skplugin_err_t skpinRegIntField(
	    const char		    *name,
	    uint64_t		     min,
	    uint64_t		     max,
	    skplugin_int_field_fn_t  rec_to_int,
	    size_t		     width);

       "name"
	   The name of the new key field.

       "min"
	   A number representing the minimum integer value for the field.

       "max"
	   A number representing the maximum integer value for the field.  If
	   "max" is 0, a value of UINT64_MAX is	used instead.

       "rec_to_int"
	   A callback function that accepts a SiLK Flow	record as its sole
	   argument, and returns an unsigned integer (in host byte order)
	   which represents the	value of the "name" field for the given
	   record.  The	signature is

	    uint64_t rec_to_int(
		const rwRec *rec);

       "width"
	   The column width to use when	displaying the field.  If "width" is
	   0, it will be computed to be	the number of digits necessary to
	   display the integer "max".

       IPv4 key	field

       The following function registers	a new key field	whose value is an IPv4
       address.

	skplugin_err_t skpinRegIPv4Field(
	    const char		     *name,
	    skplugin_ipv4_field_fn_t  rec_to_ipv4,
	    size_t		      width);

       "name"
	   The name of the new key field.

       "rec_to_ipv4"
	   A callback function that accepts a SiLK Flow	record as its sole
	   argument, and returns a 32 bit integer (in host byte	order) which
	   represents the IPv4 addresses for the "name"	field for the given
	   record.  The	signature is

	    uint32_t rec_to_ipv4(
		const rwRec *rec);

       "width"
	   The column width to use when	displaying the field.  If "width" is
	   0, it will be set to	15.

       IP key field

       The following function is used to register a key	field whose value is
       any IP address (an "skipaddr_t").

	skplugin_err_t skpinRegIPAddressField(
	    const char		   *name,
	    skplugin_ip_field_fn_t  rec_to_ipaddr,
	    size_t		    width);

       "name"
	   The name of the new key field.

       "rec_to_ipaddr"
	   A callback function that accepts a SiLK Flow	record and an
	   "skipaddr_t"	as arguments.  The function should fill	in the IP
	   address as required for the "name" field.  The signature is

	    void rec_to_ipaddr(
		skipaddr_t  *dest,
		const rwRec *rec);

       "width"
	   The column width to use when	displaying the field.  If "width" is
	   0, it will be set to	39 when	SiLK has support for IPv6 addresses,
	   or 15 otherwise.

       Text key	field (from an integer)

       The following function is used to register a key	field whose value is
       an unsigned 64 bit integer (similar to skpinRegIntField()), but where
       the printed representation of the field is determined by	a second
       callback	function.  This	allows the plug-in to create arbitrary text
       for the field.

	skplugin_err_t skpinRegTextField(
	    const char		     *name,
	    uint64_t		      min,
	    uint64_t		      max,
	    skplugin_int_field_fn_t   value_fn,
	    skplugin_text_field_fn_t  text_fn,
	    size_t		      width);

       "name"
	   The name of the new key field.

       "min"
	   A number representing the minimum integer value for the field.

       "max"
	   A number representing the maximum integer value for the field.  If
	   "max" is 0, a value of UINT64_MAX is	used instead.

       "value_fn"
	   A callback function that accepts a SiLK Flow	record as its sole
	   argument, and returns an unsigned integer (in host byte order)
	   which represents the	value of the "name" field for the given
	   record.  The	signature is

	    uint64_t rec_to_int(
		const rwRec *rec);

       "text_fn"
	   A callback function that provides the textual representation	of the
	   value returned by "value_fn".  The function's signature is

	    void text_fn(
		char	 *dest,
		size_t	  dest_len,
		uint64_t  val);

	   The callback	should fill the	character array	"dest" with the
	   printable representation of "val".  The number of characters	in
	   "dest" is given by "dest_len".  Note	that "dest_len"	may be
	   different than the parameter	"width"	passed to skpinRegTextField(),
	   and "text_fn" must NUL-terminate the	string.

       "width"
	   The column width to use when	displaying the field.

       Text key	field (from a list)

       The following function is used to register a field whose	value is one
       of a list of strings.  The plug-in provides the list of strings and a
       callback	that takes a SiLK Flow record and returns an index into	the
       list of strings.

	skplugin_err_t skpinRegStringListField(
	    const char		     *name,
	    const char		    **list,
	    size_t		      entries,
	    const char		     *default_value,
	    skplugin_int_field_fn_t   rec_to_index,
	    size_t		      width);

       "name"
	   The name of the new key field.

       "list"
	   List	is the list of strings.	 The list should either	be NULL
	   terminated, or "entries" should have	a non-zero value.

       "entries"
	   The number of entries in "list".  If	"entries" is 0,	SiLK
	   determines the number of entries by traversing "list" until it
	   finds a element whose value is NULL.

       "default_value"
	   The value to	use when "rec_to_index"	returns	an invalid value.

       "rec_to_index"
	   A callback function that accepts a SiLK Flow	record as its sole
	   argument, and returns an unsigned integer (in host byte order)
	   which represents an index into "list".  If the return value is
	   beyond the end of "list", "default_value" will be used instead.
	   The signature of this callback function is

	    uint64_t rec_to_int(
		const rwRec *rec);

       "width"
	   The column width to use when	displaying the field.  If "width" is
	   0, it is defaulted to the width of the longest string in "list" and
	   "default_value".

       Integer sum aggregate value field

       The following function registers	an aggregate value field that
       maintains a running unsigned integer sum.  That is, the values returned
       by the callback are summed for every SiLK Flow record that matches a
       bin's key.  The sum is printed when the bin is printed.

	skplugin_err_t skpinRegIntSumAggregator(
	    const char		    *name,
	    uint64_t		     max,
	    skplugin_int_field_fn_t  rec_to_int,
	    size_t		     width);

       "name"
	   The name of the new aggregate value field.

       "max"
	   A number representing the maximum integer value for the field.  If
	   "max" is 0, a value of UINT64_MAX is	used instead.

       "rec_to_int"
	   A callback function that accepts a SiLK Flow	record as its sole
	   argument, and returns an unsigned integer (in host byte order)
	   which represents the	value of the "name" value field	for the	given
	   record.  The	signature is

	    uint64_t rec_to_int(
		const rwRec *rec);

       "width"
	   The column width to use when	displaying the value.  If "width" is
	   0, it will be computed to be	the number of digits necessary to
	   display the integer "max".

       Integer minimum or maximum aggregate value field

       The following function registers	an aggregate value field that
       maintains the minimum integer value seen	among all values returned by
       the callback function.

	skplugin_err_t skpinRegIntMinAggregator(
	    const char		    *name,
	    uint64_t		     max,
	    skplugin_int_field_fn_t  rec_to_int,
	    size_t		     width);

       This function is	similar, except	it maintains the maximum value.

	skplugin_err_t skpinRegIntMaxAggregator(
	    const char		    *name,
	    uint64_t		     max,
	    skplugin_int_field_fn_t  rec_to_int,
	    size_t		     width);

       "name"
	   The name of the new aggregate value field.

       "max"
	   A number representing the maximum integer value for the field.  If
	   "max" is 0, a value of UINT64_MAX is	used instead.

       "rec_to_int"
	   A callback function that accepts a SiLK Flow	record as its sole
	   argument, and returns an unsigned integer (in host byte order)
	   which represents the	value of the "name" value field	for the	given
	   record.  The	signature is

	    uint64_t rec_to_int(
		const rwRec *rec);

       "width"
	   The column width to use when	displaying the value.  If "width" is
	   0, it will be computed to be	the number of digits necessary to
	   display the integer "max".

       Unsigned	integer	aggregate value	field

       The following function registers	an aggregate value field that can be
       represented by a	64 bit integer.	 The plug-in must register two
       callback	functions.  The	first takes a SiLK Flow	record and returns an
       integer value; the second takes two integer values (as returned by the
       first callback function)	and combines them to form a new	aggregate
       value.

	skplugin_err_t skpinRegIntAggregator(
	    const char		    *name,
	    uint64_t		     max,
	    skplugin_int_field_fn_t  rec_to_int,
	    skplugin_agg_fn_t	     agg,
	    uint64_t		     initial,
	    size_t		     width);

       "name"
	   The name of the new aggregate value field.

       "max"
	   A number representing the maximum integer value for the field.  If
	   "max" is 0, a value of UINT64_MAX is	used instead.

       "rec_to_int"
	   A callback function that accepts a SiLK Flow	record as its sole
	   argument, and returns an unsigned integer (in host byte order)
	   which represents the	value of the "name" value field	for the	given
	   record.  The	signature is

	    uint64_t rec_to_int(
		const rwRec *rec);

       "agg"
	   A callback function that combines (aggregates) two values.  For
	   example, if you wanted to create a new aggregate value that
	   contained a bit-wise	OR of the TCP flags seen on every packet, your
	   "agg" function would	OR the values.	The signature is

	    uint64_t agg(
		uint64_t current,
		uint64_t operand);

       "initial"
	   Specifies the initial value for the aggregate value.	 The first
	   time	the "agg" function is called on	a bin, "operand" will be the
	   value returned by "rec_to_int", and "current" will be the value
	   given in "initial".	The value in "initial" must be less than or
	   equal to the	value in "max".

       "width"
	   The column width to use when	displaying the value.  If "width" is
	   0, it will be computed to be	the number of digits necessary to
	   display the integer "max".

   Advanced field registration function
       When the	simple field registration functions do not provide what	you
       need, you can use the skpinRegField() function that gives you complete
       control over the	field.

       skpinRegField() registers a new derived field for record	processing.
       The plug-in must	supply the name	of the new field.  The name is used as
       one of the arguments to the --fields switch (for	key fields) or
       --values	switch (for aggregate value fields).  Field names are case
       insensitive, and	all field names	must be	unique within an application.
       You will	get a run-time error if	you attempt to create a	field whose
       name already exists.  (In rwuniq	and rwstats, you may have a --fields
       key and a --values aggregate value with the same	name.)

       The skpinRegField() function requires you initialize and	pass in	a
       structure.  In this structure you will specify the callback functions
       that the	application will call, as well as additional information
       required	by some	applications.  Although	the structure is complex, not
       all applications	use all	members.

       If the plug-in is loaded	by an application that does not	support	fields
       (such as	rwfilter), the function	is a no-op.

       The advanced field registration function	is

	skplugin_err_t skpinRegField(
	    skplugin_field_t	      **return_field,
	    const char		       *name,
	    const char		       *description,
	    const skplugin_callbacks_t *regdata,
	    void		       *cbdata);

       "return_field"
	   When	this value is not NULL,	skpinRegField()	will set the location
	   it references to the	newly created field.

       "name"
	   This	sets the primary name of the field, and	by default will	be the
	   title used when printing the	field.

       "description"
	   The "description" provides a	textual	description of the field.
	   Currently this is unused.

       "regdata"
	   The "regdata" structure provides the	application with the callback
	   functions and additional information	it needs to use	the plug-in.
	   The members that must be set	vary by	application.  It is described
	   in more detail below.

       "cbdata"
	   This	parameter will be passed back unchanged	to the plug-in as a
	   parameter in	the various callback functions.	 It may	be NULL.

       The structure used by the skpinRegField() (and skpinRegFilter())
       functions to specify callback functions is shown	here:

	typedef	struct skplugin_callbacks_st {
	    skplugin_callback_fn_t     init;
	    skplugin_callback_fn_t     cleanup;
	    size_t		       column_width;
	    size_t		       bin_bytes;
	    skplugin_text_fn_t	       rec_to_text;
	    skplugin_bin_fn_t	       rec_to_bin;
	    skplugin_bin_fn_t	       add_rec_to_bin;
	    skplugin_bin_to_text_fn_t  bin_to_text;
	    skplugin_bin_merge_fn_t    bin_merge;
	    skplugin_bin_cmp_fn_t      bin_compare;
	    skplugin_filter_fn_t       filter;
	    skplugin_transform_fn_t    transform;
	    const uint8_t	      *initial;
	    const char		     **extra;
	} skplugin_callbacks_t;

       All of the callback functions reference in this structure take "cbdata"
       as a parameter, which is	the value that was specified in	the call to
       skpinRegField().	 The "extra" parameter to the callback functions is
       used in complex plug-ins	and can	be ignored.

       The members of the structure are:

       "init"
	   This	specifies a callback function which the	application will call
	   when	it has determined this field will be used.  (In	the case of
	   skpinRegFilter(), the function is called for	all registered
	   filters.)  The application calls the	function before	processing
	   data.  It may be NULL; the signature	of the callback	function is

	    skplugin_err_t init(
		void *cbdata);

       "cleanup"
	   When	this callback function is not NULL, the	application will call
	   it after all	records	have been processed.  It has the same
	   signature as	the "init" function.

       "column_width"
	   The number of characters (not including trailing NUL) required to
	   hold	a string representation	of the longest value of	the field.
	   This	value can be 0 if not used (e.g., rwsort does not print
	   fields), or if it will be set later using skpinSetFieldWidths().

       "bin_bytes"
	   The number of bytes (octets)	required to hold a binary
	   representation of a value of	the field.  This value can be 0	if not
	   used	(e.g., rwcut does not use binary values), or if	it will	be set
	   later using skpinSetFieldWidths().

       "rec_to_text"
	   The rwcut application uses this callback function to	fetch the
	   textual value for the field given a SiLK Flow record.  The
	   signature of	this function is

	    skplugin_err_t rec_to_text(
		const rwRec  *rec,
		char	     *dest,
		size_t	      width,
		void	     *cbdata,
		void	    **extra);

	   The callback	function should	fill the character array "dest"	with
	   the textual value, and the value should be NUL-terminated.  "width"
	   specifies the overall size of "dest", and it	may not	have the same
	   value as specified by the "column_width" member.  For proper
	   formatting, the callback function should write no more than
	   "column_width" characters into "dest".  Note	that if	an application
	   requires a "rec_to_bin" function and	"rec_to_bin" is	NULL, the
	   application will use	"rec_to_text" if it is provided.  The
	   application will use	"column_width" as the width for	binary values
	   (zeroing out	the destination	area before it is written to).

       "rec_to_bin"
	   This	callback function is used by the application to	fetch the
	   binary value	for this field given the SiLK Flow record.  The
	   signature of	this function is:

	    skplugin_err_t rec_to_bin(
		const rwRec  *rec,
		uint8_t	     *dest,
		void	     *cbdata,
		void	    **extra);

	   The callback	function should	write exactly "bin_bytes" of data into
	   "dest" (where "bin_bytes" was specified in the call to
	   skpinRegField() or skpinSetFieldWidths()).  See also	the
	   "rec_to_text" member.

       "add_rec_to_bin"
	   This	callback function is used by rwuniq and	rwstats	when computing
	   aggregate value fields.  The	application expects this function to
	   get the binary value	for this field from the	SiLK Flow record and
	   merge it (e.g., add it) to the current value.  That is, the
	   function should update the value in "current_and_new_value" with
	   the value that comes	from the current "rec".	 The signature is:

	    skplugin_err_t add_rec_to_bin(
		const rwRec  *rec,
		uint8_t	     *current_and_new_value,
		void	     *cbdata,
		void	    **extra);

	   The callback	function should	write exactly "bin_bytes" of data into
	   "current_and_new_value".

       "bin_to_text"
	   This	callback function is used to get a textual representation of a
	   binary value	that was set by	a prior	call to	the "rec_to_bin" or
	   "add_rec_to_bin" functions.	The function signature is

	    skplugin_err_t bin_to_text(
		const uint8_t *bin,
		char	      *dest,
		size_t	       width,
		void	      *cbdata);

	   The binary input value is in	"bin", and it is exactly "bin_bytes"
	   in length.  The textual output must be written to "dest".  The
	   overall size	of "dest" is given by "width", which may be different
	   than	the "column_width" value that was previously specified.	 For
	   proper formatting, the callback function should write no more than
	   "column_width" characters into "dest".

       "bin_merge"
	   When	rwstats	and rwuniq are unable to store all values in memory,
	   the applications write their	current	state to temporary files on
	   disk.  Once all input data has been processed, the temporary	files
	   are combined	to produce the output.	When a key appears in multiple
	   temporary files, the	aggregate values must be merged	(for example,
	   the byte count for two keys would be	added).	 This callback
	   function is used to merge aggregate value fields defined by the
	   plug-in.  The function signature is below.  The "src1_and_dest"
	   parameter will contain a binary aggregate value from	one of the
	   files, and the "src2" parameter a value from	the other.  These
	   should be combined and the (binary) result written to
	   "src1_and_dest".  The byte length of	both parameters	is
	   "bin_bytes".

	    skplugin_err_t bin_merge(
		uint8_t	       *src1_and_dest,
		const uint8_t  *src2,
		void	       *cbdata);

       "bin_compare"
	   This	callback function is used by rwstats when determining the top-
	   N (or bottom-N) bins	based on the binary aggregate values.  The
	   function accepts two	binary values, "value_a" and "value_b",	each
	   of length "bin_bytes".  The function	must set "cmp_result" to an
	   integer less	than 0,	equal 0, or greater than 0 to indicate whether
	   "value_a" is	less than, equal to, or	greater	than "value_b",
	   respectively.  If this function is NULL, memcmp() will be used on
	   the binary values instead.

	    skplugin_err_t bin_compare(
		int	       *cmp_result,
		const uint8_t  *value_a,
		const uint8_t  *value_b,
		void	       *cbdata);

       "filter"
	   This	callback function is only required when	the plug-in will be
	   used	by rwfilter, as	described above.  When defining	a field,
	   "filter" is ignored.

       "transform"
	   This	callback function is only required when	the plug-in will be
	   used	by rwptoflow.  This callback allows the	plug-in	to modify the
	   SiLK	Flow record, "rec", before it is written to the	output.	 The
	   callback function should modify "rec" in place; the signature is

	    skplugin_err_t transform(
		rwRec  *rec,
		void   *cbdata,
		void  **extra);

       "initial"
	   When	the "initial" member is	not NULL, it should point to a value
	   containing at least "bin_bytes" bytes.  These bytes will be used to
	   initialize the binary aggregate value.  As an example use case,
	   when	the plug-in is computing a minimum, it may choose to
	   initialize the field	to contain the maximum value.  When "initial"
	   is NULL, binary aggregate values are	initialized using bzero().

       "extra"
	   This	member is usually NULL.	 When not NULL,	it points to a NULL-
	   terminated constant array of	strings	representing "extra
	   arguments".	These are not often used, and they will	not be
	   discussed in	this manual page.

       Once a field is registered, you may make	changes	to it by calling the
       additional functions described below.  In each of these functions, the
       "field" parameter is the	handle returned	when the field was registered.

       By default, the "name" will also	be used	as the field's title.  To
       specify a different title, the plug-in may call

	skplugin_err_t skpinSetFieldTitle(
	    skplugin_field_t	field,
	    const char		title);

       To create an alternate name for the field (that is, a name that can be
       used in the --fields or --values	switches) call

	skplugin_err_t skpinAddFieldAlias(
	    skplugin_field_t	field,
	    const char		alias);

       To set or modify	the textual and	binary widths for a field, use the
       following function.  This function should called	in the field's "init"
       callback	function.

	skplugin_err_t skpinSetFieldWidths(
	    skplugin_field_t	field,
	    size_t		field_width_text,
	    size_t		field_width_bin);

       The following table shows when a	member of the "skplugin_callbacks_t"
       structure is required or	optional.  (Where the table shows
       "column_width" and "bin_bytes" as required, the values can be set in
       the structure or	via the	skpinSetFieldWidths() function.)

		       rwfilter	rwcut rwgroup rwsort rwstats rwuniq rwptoflow
	init		  r	  f	 f	f      f,a    f,a	r
	cleanup		  r	  f	 f	f      f,a    f,a	r
	column_width	  .	  F	 .	.      F,A    F,A	.
	bin_bytes	  .	  .	 F	F      F,A    F,A	.
	rec_to_text	  .	  F	 .	.	.      .	.
	rec_to_bin	  .	  .	 F	F	F      F	.
	add_rec_to_bin	  .	  .	 .	.	A      A	.
	bin_to_text	  .	  .	 .	.      F,A    F,A	.
	bin_merge	  .	  .	 .	.	A      A	.
	bin_compare	  .	  .	 .	.	A      .	.
	initial		  .	  .	 .	.	a      a	.
	filter		  R	  .	 .	.	.      .	.
	transform	  .	  .	 .	.	.      .	R
	extra		  r	  f	 f	f      f,a    f,a	r

       The legend is

       F   required for	a key field

       A   required for	an aggregate value field

       R   required for	a non-field application	(e.g., rwfilter)

       f   optional for	a key field

       a   optional for	an aggregate value field

       r   optional for	a non-field application

       .   ignored

   Miscellaneous functions
       The following registers a cleanup function for the plug-in.  This
       function	will be	called by the application after	any field- or filter-
       specific	cleanup	functions are called.  Specifically, this is the last
       callback	that the application will invoke on a plug-in.

	skplugin_err_t skpinRegCleanup(
	    skplugin_cleanup_fn_t cleanup);

       The signature of	the "cleanup" function is:

	void cleanup(void);

       The plug-in author should invoke	the following function to tell
       rwfilter	that this plug-in is not thread	safe.  Calling this function
       causes rwfilter not use multiple	threads; as such, this function	should
       only be called when the plug-in has registered an active	filter
       function.

	void skpinSetThreadNonSafe(void);

   Compiling the plug-in
       Once you	have finished writing the C code for the plug-in, save it in a
       file.  The following uses the name my-plugin.c for the name of this
       file.

       In the following, the leading dollar sign ("$") followed	by a space
       represents the shell prompt.  The text after the	dollar sign represents
       the command line.  Lines	have been wrapped for improved readability,
       and the back slash ("\")	is used	to indicate a wrapped line.

       When compiling a	plug-in, you should use	the same compiler and
       compiler-options	as when	SiLK was compiled.  The	silk_config(1) utility
       can be used to obtain that information.	To store the compiler used to
       compile SiLK into the variable "sk_cc", specify the following at	a
       shell prompt (note that those are backquotes, and this assumes a
       Bourne-compatible shell):

	$ sk_cc=`silk_config --compiler`

       To get the compiler flags used to compile SiLK:

	$ sk_cflags=`silk_config --cflags`

       Using those two variables, you can now compile the plug-in.  The
       following will work on Linux and	Mac OS X:

	$ $sk_cc $sk_cflags -shared -o my-plugin.so my-plugin.c

       For Mac OS X:

	$ $sk_cc $sk_cflags -bundle -flat_namespace -undefined suppress	   \
	       -o my-plugin.so my-plugin.c

       If there	are compilation	errors,	fix them and compile again.

       Notes: The preceding assumed you	were building the plug-in after	having
       installed SiLK.	The paths given	by silk_config do not work if SiLK has
       not been	installed.  To compile the plug-in, you	must have access to
       the SiLK	header files.  (If you are using an RPM	installation of	SiLK,
       ensure that the "silk-devel" RPM	is installed.)

       Once you	have created the my-plugin.so file, you	can load it into an
       application by using the	--plugin switch	on the application as shown in
       the "SYNOPSIS".	When loading a plug-in from the	current	directly, it
       is best to prefix the filename with "./":

	$ rwcut	--plugin=./my-plugin.so	...

       If there	are problems loading the plug-in into the application, you can
       trace the actions the application is doing by setting the
       SILK_PLUGIN_DEBUG environment variable:

	$ SILK_PLUGIN_DEBUG=1  rwcut --plugin=./my-plugin.so ...

EXAMPLES
   rwfilter
       Suppose you want	to find	traffic	destined to a particular host,
       10.0.0.23, that is either ICMP or coming	from 1434/udp.	If you attempt
       to use:

	$ rwfilter --daddr=10.0.0.23 --proto=1,17 --sport=1434	   \
	       --pass=outfile.rw  flowrec.rw

       the --sport option will not match any of	the ICMP traffic, and your
       result will not contain ICMP records.  To avoid having to use two
       invocations of rwfilter,	you can	create the following plug-in to	do the
       entire check in a single	pass:

	#include <silk/silk.h>
	#include <silk/rwrec.h>
	#include <silk/skipaddr.h>
	#include <silk/skplugin.h>
	#include <silk/utils.h>

	/* These variables specify the version of the SiLK plug-in API.	*/
	#define	PLUGIN_API_VERSION_MAJOR 1
	#define	PLUGIN_API_VERSION_MINOR 0

	/* ip to search	for */
	static skipaddr_t ipaddr;

	/*
	 *  status = filter(rwrec, reg_data, extra);
	 *
	 *    The function should examine the SiLK flow	record and return
	 *    SKPLUGIN_FILTER_PASS to write the	rwRec to the
	 *    pass-destination(s) or SKPLUGIN_FILTER_FAIL to write it to the
	 *    fail-destination(s).
	 */
	static skplugin_err_t filter(
	    const rwRec	   *rwrec,
	    void	   *reg_data,
	    void	  **extra)
	{
	    skipaddr_t dip;

	    rwRecMemGetDIP(rwrec, &dip);
	    if (0 == skipaddrCompare(&dip, &ipaddr)
		&& (rwRecGetProto(rwrec) == 1
		    || (rwRecGetProto(rwrec) ==	17
			&& rwRecGetSPort(rwrec)	== 1434)))
	    {
		return SKPLUGIN_FILTER_PASS;
	    }
	    return SKPLUGIN_FILTER_FAIL;
	}

	/* The set-up function that the	application will call. */
	skplugin_err_t SKPLUGIN_SETUP_FN(
	    uint16_t	major_version,
	    uint16_t	minor_version,
	    void       *plug_in_data)
	{
	    uint32_t ipv4;
	    skplugin_err_t rv;
	    skplugin_callbacks_t regdata;

	    /* Check the plug-in API version */
	    rv = skpinSimpleCheckVersion(major_version,	minor_version,
					 PLUGIN_API_VERSION_MAJOR,
					 PLUGIN_API_VERSION_MINOR,
					 skAppPrintErr);
	    if (rv != SKPLUGIN_OK) {
		return rv;
	    }

	    /* set global ipaddr */
	    ipv4 =  ((10 << 24)	| 23);
	    skipaddrSetV4(&ipaddr, &ipv4);

	    /* register	the filter */
	    memset(&regdata, 0,	sizeof(regdata));
	    regdata.filter = filter;
	    return skpinRegFilter(NULL,	&regdata, NULL);
	}

       Once this file is created and compiled, you can use it from rwfilter as
       shown here:

	$ rwfilter --plugin=./my-plugin.so --pass=outfile.rw  flowrec.rw

   Additional examples
       For additional examples,	see the	source files in
       silk-VERSION/src/plugins.

ENVIRONMENT
       SILK_PATH
	   This	environment variable gives the root of the install tree.  When
	   searching for plug-ins, a SiLK application may use this environment
	   variable.  See the "FILES" section for details.

       SILK_PLUGIN_DEBUG
	   When	set to 1, the SiLK applications	print status messages to the
	   standard error as they attempt to find and open each	plug-in.  In
	   addition, when an attempt to	register a field fails,	the
	   application prints a	message	specifying the additional function(s)
	   that	must be	defined	to register the	field in the application.  Be
	   aware that the output can be	rather verbose.

FILES
       ${SILK_PATH}/lib64/silk/
       ${SILK_PATH}/lib64/
       ${SILK_PATH}/lib/silk/
       ${SILK_PATH}/lib/
       /usr/local/lib64/silk/
       /usr/local/lib64/
       /usr/local/lib/silk/
       /usr/local/lib/
	   Directories that a SiLK application checks when attempting to load
	   a plug-in.

SEE ALSO
       rwfilter(1), rwcut(1), rwgroup(1), rwsort(1), rwstats(1), rwuniq(1),
       silk_config(1), rwptoflow(1), pysilk(3),	silkpython(3), flowrate(3),
       silk(7),	pcap(3)

SiLK 3.19.1			  2021-02-28			silk-plugin(3)

NAME | SYNOPSIS | DESCRIPTION | EXAMPLES | ENVIRONMENT | FILES | SEE ALSO

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=silk-plugin&sektion=3&manpath=FreeBSD+13.0-RELEASE+and+Ports>

home | help