Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
LOGSURFER(1)		    General Commands Manual		  LOGSURFER(1)

       logsurfer - process logfiles and	perform	certain	actions

       logsurfer  [  -c	configfile ] [ -d dumpfile ] [ -D ] [ -f ] [ -F	] [ -l
       start_linenum ] [ -p pidfile ] [	-r start_regex ] [ -s ]	[ -t ] [ -e  ]
       [ filename ]

       Logsurfer  is  a	 program that works on single textlines	from a logfile
       (or from	standard input). It compares each line against regular expres-
       sions of	a set of "rules" and performs certain actions that are defined
       for this	match.

       The logsurfer uses "rules"  to  define  actions	for  messages  (single
       lines).	Messages  may  be  collected in	"contexts" which are stored in
       memory.	Those contexts have their own default actions (e.g. for	 time-
       outs). At starttime you have to define a	set of rules in	the configura-
       tion file.  While processing logmessages	the logsurfer is able  to  dy-
       namically  add  or delete rules,	create or destroy contexts, invoke ex-
       ternal programs for alerting people or to perform further automated ac-
       tions etc.

       A  rule	starts with a set of regular expressions. The messageline that
       is being	checked	has to match against the first regular expression  but
       must not	match the second expression. The string	"-" can	be used	as the
       second regex if you don't need this "match not" expression. If the mes-
       sageline	 meets these requirements the associated action	is taken. Cur-
       rently the following actions are	allowed: ignore	this message,  execute
       an  external program, pass this messageline as standard input to	an ex-
       ternal program, create or delete	a context, generate a "report" or  dy-
       namically add a new rule. In addition to	these actions rules may	have a
       configuration to	delete themself	if another message (again specified by
       two  regular  expressions) is being processed by	the logsurfer or if an
       optional	timeout	has occured. Matching of messagelines against rules is
       usually	done until the first match occurs. This	can be circumvented by
       specifying a "continue" keyword in the rule. In this case  the  program
       will continue to	search for another match in the	rule definitions.

       A  context  is  a  set  of messagelines that have been read by the log-
       surfer. Again the messagelines are specified  by	 two  regular  expres-
       sions.	Contexts  can  be used in "reports" to present a collection of
       messages	associated with	a specific action (e.g.	all messagelines  from
       one  ftp	session). Every	context	has a default action which is launched
       if either a specified timeout has occured or  a	configured  limit  (of
       linenumbers)  has  been	reached. Default actions may be: ignore	(drop)
       this context, execute an	external program, feed all  collected  logmes-
       sages as	standard input to an external program or generate a "report".

       A  report  summarizes a number of contexts and pipes the	result into an
       external	program.

       The normal procedure for	each logline is: Store this line in all	match-
       ing contexts (as	defined	by their regular expressions), if the self-de-
       stroy regex matches, too, then the default action of the	context	is ex-
       ecuted and the context is deleted, find the first matching rule and ex-
       ecute the given action (continue	rule matching if the rule has the key-
       word "continue" --- see logsurfer.conf(4) manpage), check for the stop-
       pattern of the rule and in case of a match delete the rule,  check  for
       timeouts	 of  rules  and	 delete	 the affected rules, check for context
       timeouts	and execute their default action.

       Errors (like broken configuration statements or	memory	problems)  are
       reported	on standard error. The syslog mechanism	is currently not being
       used because this might result in a loop	if the logsurfer is processing
       those syslog-messages. It is suggested to start the logsurfer and redi-
       rect the	output (incl. error output) to another file. If	necessary  you
       might  want  to start a second logsurfer	that is	watching the output of
       the first logsurfer.

       If the program exits the	following steps	are executed:

       1)    The status	of all active rules and	 contexts  is  dumped  to  the

       2)    If	 contexts have timeout values (other than 0) then a timeout is
	     send to those functions. This will	result in  launching  the  de-
	     fault actions of the affected contexts.

       3)    Rules  and	 contexts  are	deleted	from memory (cleanup). This is
	     only important for	debugging purposes and should have no effect.

       4)    If	there are still	external programs running the  logsurfer  will
	     wait up to	20 seconds to catch them.

       5)    The logsurfer exits with a	return value of	zero (0).

       -c configfile
	      Specify  another	configuration  file for	the logsurfer. The de-
	      fault is "/usr/local/etc/logsurfer/logsurfer.conf".

       -d dumpfile
	      Specify another dumpfile for the logsurfer. The logsurfer	writes
	      the  internal  information about rules and contexts to this file
	      if the program exists or	a  special  signal  was	 sent  to  the
	      process (see NOTES). The default is "/dev/null".

       -D     Daemonize. Fork and run the logsurfer process in the background.
	      Any output sent to stdin,	stdout and stderr  are	redirected  to
	      the bitbucket.

       -f     Normally	the  logsurfer will process the	input once and exit at
	      the end (after processing	all the	input  lines).	By  using  the
	      "-f"  option  the	 program will enter a special "follow" mode if
	      the end of file has been reached.	In this	case the program  will
	      sleep  for one second before trying to read a new	line. If a new
	      line has been added to the end of	the input, then	this  line  is
	      processed.  Otherwise  the program will sleep again for one sec-
	      ond. This	mode is	similar	to the "-f" option of the tail(1) com-

       -F     Acts  like the "-f" option, but follows the logfile across rota-
	      tions by checking	if the inode of	the file has changed  and  re-
	      opening the log.

       -l start_linenum
	      Start logfile processing at the given linenumber.

       -p pidfile
	      Write  the processid of the logsurfer process to the given file-
	      name. This can be	used to	simplify sending signals to  the  log-
	      surfer program (for example after	shifting logfiles).

       -r start_regex
	      Start  logfile  processing  with the first line that matches the
	      given regular expression.

       -s     Normally the logsurfer prints some information to	stderr if  the
	      program  exists.	If  you	specify	the "-s" (silent) option, then
	      those messages are not printed.

       -t     When logsurfer exits, the	-t option will instruct	it to  explic-
	      itly  expire  all	contexts which have either a relative or abso-
	      lute timeout configured.	Without	the -t option,	contexts  will
	      not be timed-out and will	be removed silently without triggering
	      their actions.

       -e     When Logsurfer starts reading the	log file this option will  in-
	      struct it	to seek	to the end of the file before scanning starts.
	      Without this option, Logsurfer will start	scanning at the	begin-
	      ning of the file.

				       default configuration file
       /dev/null		       dump of the rules and contexts

       logsurfer.conf(4), swatch(8)

       This  program  may  consume a lot of memory. Resource control should be
       done by the operating system (for example by setting limits). The  pro-
       gram was	designed to fall back to a defined state if some resources are
       not available.

       Please do not run the logsurfer with a privileged userid	 (like	root).
       You  usually do not need	the extra privileges. If your logfiles are not
       world-readable change them to be	group readable and start the logsurfer
       with  a	non-privileged account that is a member	of that	group. Keep in
       mind, that the logsurfer	might execute external programs	(depending  on
       your  configuration  file)  which  are  not designed to run with	higher
       privileges. The logsurfer program will print a warning message if  it's
       started	with  root  privilege (this feature can	be disabled at compile

       Be careful with external	programs. The logsurfer	is able	to process any
       input  (for  example  arbitrary long lines) but some other programs may
       crash or	produce	other inpredictable results if feeded with long	lines.

       The following signals have the specified	effect when sent to  the  log-
       surfer program using the	kill(1)	command.

       SIGHUP	 Reopens  the logfile (to be analyzed).	Processing starts with
		 the first line	in this	file. This can be used	after  "shift-
		 ing" logfiles and creating new	logfiles.

       SIGUSR1	 Dump the internal state of all	active rules and the collected
		 contexts to the dumpfile "/dev/null".

       SIGTERM	 Dumps the state to the	dumpfile (see above) and exits.

       Quoting of regular expressions isn't that easy. If  you	use  variables
       ($0 - $9) in dynamic rules, then	you have to keep in mind, that certain
       characters (matching your regular  expresion  for  that	variable)  may
       again  contain special chars, that are interpreted as regex-parts.  Ex-
       ample: If you use one regular expresion to match	a hostname and you are
       using  this  again  as  part of a regular expression, then the dots "."
       separating parts	of the domainname will match any char  (not  only  the

       If  the	logsurfer has been compiled with the "sendmail flush hack" and
       the logsurfer exists, then there	might still be some mails in the local
       sendmail	queue (not flushed). Delivery of such mail maybe delayed until
       the next	time sendmail flushes the queue.

				Thu Oct	19 1995			  LOGSURFER(1)


Want to link to this manual page? Use this URL:

home | help