Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
SWISH-CONFIG(1)		     SWISH-E Documentation	       SWISH-CONFIG(1)

NAME
       SWISH-CONFIG - Configuration File Directives

OVERVIEW
       This document lists the available configuration directives available in
       Swish-e.

CONFIGURATION FILE
       What files Swish-e indexes and how they are indexed, and	where the in-
       dex is written can be controlled	by a configuration file.

       The configuration file is a text	file composed of comments, blank
       lines, and configuration	directives.  The order of the directives is
       not important.  Some directives may be used more	than once in the con-
       figuration file,	while others can only be used once (e.g. additional
       directives will overwrite preceding directives).	 Case of the directive
       is not important	-- you may use upper, lower, or	mixed case.

       Comments	are any	line that begin	with a "#".

	   # This is a comment

       As of 2.4.3 lines may be	continued by placing a backslas	as the last
       character on the	line:

	   IgnoreWords \
	       am \
	       the \
	       foo

       Directives may take more	than one parameter.  Enclose single parameters
       that include whitespace in quotes (single or double).  Inside of	quotes
       the backslash escapes the next character.

	   ReplaceRules	append "foo bar"   <- define "foo bar" as a single parameter

       If you need to include a	quote character	in the value either use	a
       backslash to escape it, or enclose it in	quotes of the other type.

       Backslashes also	have special meaning in	regular	expressions.

	   FileFilterMatch pdftotext "'%p' -" /\.pdf$/

       This says that the dot is a real	dot (instead of	matching any charac-
       ter).  If you place the regular expression in quotes then you must use
       double-backslashes.

	   FileFilterMatch pdftotext "'%p' -" "/\\.pdf$/"

       Swish-e will convert the	double backslash into a	single backslash be-
       fore passing the	parameter to the regular expression compiler.

       Commented example configuration files are included in the conf direc-
       tory of the Swish-e distribution.

       Some command line arguments can override	directives specified in	the
       configuration file.  Please see also the	SWISH-RUN for instructions on
       running Swish-e,	and the	SWISH-SEARCH page for information and examples
       on how to search	your index.

       The configuration file is specified to Swish-e by the "-c" switch.  For
       example,

	   swish-e -c myconfig.conf

       You may also split your directives up into different configuration
       files.  This allows you to have a master	configuration file used	for
       many different indexes, and smaller configuration files for each	sepa-
       rate index.  You	can specify the	different configuration	files when
       running from the	command	line with the "-c" switch (see SWISH-RUN), or
       you may include other Configuration file	with the IncludeConfigFile di-
       rective below.

       Typically, in a configuration file the directives are grouped together
       in some logical order --	that is, directives that control the source of
       the documents would be grouped together first, and directives that con-
       trol how	each document is filtered or its words index in	another	group
       of directives. (The directives listed below are grouped in this order).

       The configuration file directives are listed below in these groups:

       o   "Administrative Headers Directives" -- You may add administrative
	   information to the header of	the index file.

       o   "Document Source Directives"	-- Directives for selecting the	source
	   documents and the location of the index file.

       o   "Document Contents Directives" -- Directives	that control how a
	   document content is indexed.

       o   "Directives for the File Access method only"	-- These directives
	   are only applicable to the File Access indexing method.

       o   "Directives for the HTTP Access Method Only"	-- Likewise, these
	   only	apply to the HTTP Access method.

       o   "Directives for the prog Access Method Only"	-- These only apply to
	   the prog Access method.

       o   "Document Filter Directives"	-- This	is a special section that de-
	   scribes using document filters with Swish-e.

       Alphabetical Listing of Directives

       o   AbsoluteLinks [yes|NO]

       o   BeginCharacters *string of characters*

       o   BumpPositionCounterCharacters *string*

       o   Buzzwords [*list of buzzwords*|File:	path]

       o   CompressPositions  [yes|NO]

       o   ConvertHTMLEntities [YES|no]

       o   DefaultContents [TXT|HTML|XML|TXT2|HTML2|XML2|TXT*|HTML*|XML*]

       o   Delay *seconds*

       o   DontBumpPositionOnEndTags *list of names*

       o   DontBumpPositionOnStartTags *list of	names*

       o   EnableAltSearchSyntax  [yes|NO]

       o   EndCharacters *string of characters*

       o   EquivalentServer *server alias*

       o   ExtractPath *metaname* [replace|remove|prepend|append|regex]

       o   FileFilter *suffix* *program* [options]

       o   FileFilterMatch *program* *options* *regex* [*regex*	...]

       o   FileInfoCompression [yes|NO]

       o   FileMatch [contains|is|regex] *regular expression*

       o   FileRules [contains|is|regex] *regular expression*

       o   FuzzyIndexingMode [NONE|Stemming|Soundex|Metaphone|DoubleMetaphone]

       o   FollowSymLinks [yes|NO]

       o   HTMLLinksMetaName *metaname*

       o   IgnoreFirstChar *string of characters*

       o   IgnoreLastChar *string of characters*

       o   IgnoreLimit *integer	integer*

       o   IgnoreMetaTags *list	of names*

       o   IgnoreNumberChars *list of characters*

       o   IgnoreTotalWordCountWhenRanking [YES|no]

       o   IgnoreWords [*list of stop words*|File: path]

       o   ImageLinksMetaName *metaname*

       o   IncludeConfigFile

       o   IndexAdmin *text*

       o   IndexAltTagMetaName *tagname*|as-text

       o   IndexComments [yes|NO]

       o   IndexContents [TXT|HTML|XML|TXT2|HTML2|XML2|TXT*|HTML*|XML*]	 *file
	   extensions*

       o   IndexDescription *text*

       o   IndexDir [URL|directories or	files]

       o   IndexFile *path*

       o   IndexName *text*

       o   IndexOnly *list of file suffixes*

       o   IndexPointer	*text*

       o   IndexReport [0|1|2|3]

       o   MaxDepth *integer*

       o   MaxWordLimit	*integer*

       o   MetaNameAlias *meta name* *list of aliases*

       o   MetaNames *list of names*

       o   MinWordLimit	*integer*

       o   NoContents *list of file suffixes*

       o   obeyRobotsNoIndex [yes|NO]

       o   ParserWarnLevel [0|1|2|3]

       o   PreSortedIndex *list	of property names*

       o   PropCompressionLevel	[0-9]

       o   PropertyNameAlias *property name* *list of aliases*

       o   PropertyNames *list of meta names*

       o   PropertyNamesCompareCase *list of meta names*

       o   PropertyNamesIgnoreCase *list of meta names*

       o   PropertyNamesNoStripChars *list of meta names*

       o   PropertyNamesDate *list of meta names*

       o   PropertyNamesNumeric	*list of meta names*

       o   PropertyNamesMaxLength integer *list	of meta	names*

       o   PropertyNamesSortKeyLength integer *list of meta names*

       o   ReplaceRules	[replace|remove|prepend|append|regex]

       o   ResultExtFormatName	name -x	format string

       o   SpiderDirectory *path*

       o   StoreDescription [XML <tag>|HTML <meta>|TXT size]

       o   "SwishProgParameters	*list of parameters*

       o   SwishSearchDefaultRule   [<AND-WORD>|<or-word>]

       o   TmpDir *path*

       o   TranslateCharacters [*string1 string2*|:ascii7:]

       o   TruncateDocSize *number of characters*

       o   UndefinedMetaTags [error|ignore|INDEX|auto]

       o   UndefinedXMLAttributes [DISABLE|error|ignore|index|auto]

       o   UseStemming [yes|NO]

       o   UseSoundex [yes|NO]

       o   UseWords [*list of words*|File: path]

       o   WordCharacters *string of characters*

       o   XMLClassAttributes *list of XML attribute names*

       Directives that Control Swish

       These configuration directives control the general behavior of Swish-e.

       IncludeConfigFile *path to config file*
	   This	directive can be used to include configuration directives lo-
	   cated in another file.

	       IncludeConfigFile /usr/local/swish/conf/site_config.config

       IndexReport [0|1|2|3]
	   This	is how detailed	you want reporting while indexing. You can
	   specify numbers 0 to	3.  0 is totally silent, 3 is the most ver-
	   bose.   The default is 1.

	   This	may be overridden from the command line	via the	"-v" switch
	   (see	SWISH-RUN).

       ParserWarnLevel [0|1|2|3]
	   Sets	the error level	when using the libxml2 parser for XML and
	   HTML.  libxml2 will point out structural errors in your documents.

	       0 = no report
	       1 = fatal errors
	       2 = errors
	       3 = warnings

	   Currently (as of 2.4.4 - early 2005)	libxml2	only reports errors at
	   level 2.  The default as of 2.4.4 is	"2" which should report	any
	   errors that might indicate a	problem	parsing	a document.

	   The exception to this is UTF-8 to Latin-1 conversion	errors are re-
	   ported at level 3 (changed from 1 in	2.4.4).	 Although these	errors
	   indicate a problem indexing text, they are only reported at level 3
	   because they	can be very common.

	   It is recommended that you index at ParserWarnLevel 3 when first
	   starting out	to see what errors and warnings	are reported.  Then
	   reduce the level when you understand	what documents are causing
	   parsing problems and	why.

       IndexFile *path*
	   Index file specifies	the location of	the generated index file.  If
	   not specified, Swish-e will create the file index.swish-e in	the
	   current directory.

	       IndexFile /usr/local/swish/site.index

       obeyRobotsNoIndex [yes|NO]
	   When	enabled, Swish-e will not index	any HTML file that contains:

	       <meta name="robots" content="noindex">

	   The default is to ignore these meta tags and	index the document.
	   This	tag is described at http://www.robotstxt.org/wc/exclu-
	   sion.html.

	   Note: This feature is only available	with the libxml2 HTML parser.

	   Also, if you	are using the libxml2 parser (HTML2 and	XML2) then you
	   can use the following comments in your documents to prevent index-
	   ing:

		  <!-- SwishCommand noindex -->
		  <!-- SwishCommand index -->

	   and/or these	may be used also:

		  <!-- noindex -->
		  <!-- index -->

	   For example,	these are very helpful to prevent indexing of common
	   headers, footers, and menus.

       NOTE: This following items are currently	not available.	These items
       require Swish-e to parse	the configuration file while searching.

       EnableAltSearchSyntax [yes|NO]
	   NOTE: This following	item is	currently not available.

	   Enable alternate search syntax.  Allows the usage of	a basic	"Al-
	   tavista(c)",	"Lycos(c)", etc. like search syntax.  This means a
	   search query	can contain "+"	and "-"	as syntax parameter.

	   Example:

	       swish-e -w "+word1 +word2 -word3	 word4 word5"
	       "+"  = following	word has to be in all found documents
	       "-"  = following	word may not be	in any document	found
	       " "  = following	word will be searched in documents

       SwishSearhOperators <and-word> <or-word>	<not-word>
	   NOTE: This following	item is	currently not available.

	   Using this config directive you can change the boolean search oper-
	   ators of Swish-e, e.g. to adapt these to your language.  The	de-
	   fault is:	AND  OR	 NOT

	   Example (german):

	       SwishSearchOperators   UND  ODER	 NICHT

       SwishSearchDefaultRule	[<AND-WORD>|<or-word>]
	   NOTE: This following	item is	currently not available.

	   "SwishSearchDefaultRule" defines the	default	Boolean	operator to
	   use if none is specified between words or phrases.  The default is
	   "AND".

	   The word you	specify	must match one of the available	"SwishSearch-
	   Operators".

	   Example:

	       SwishSearchOperators   UND  ODER	 NICHT
	       # Make it act like a web	search engine
	       SwishSearchDefaultRule ODER

       ResultExtFormatName name	-x format string
	   NOTE: This following	item is	currently not available.

	   The output of Swish-e can be	defined	by specifying a	format string
	   with	the "-x" command line argument.	 Using "ResultExtFormatName"
	   you can assign a predefined format string to	a name.

	   Examples:

	       ResultExtFormatName  moreinfo   "%c|%r|%t|%p|<author>|<publishyear>\n"

	   Then	when searching you can specify the format string's name

	       swish-e	 ...  -x moreinfo  ...

	   See the "-x"	switch in SWISH-RUN for	more information about output
	   formats.

       Administrative Headers Directives

       Swish-e stores configuration information	in the header of the index
       file.  This information can be retrieved	while searching	or by func-
       tions in	the Swish-e C library.	There are a number of fields available
       for your	own use.  None of these	fields are required:

       IndexName *text*
       IndexDescription	*text*
       IndexPointer *text*
       IndexAdmin *text*
	   These variables specify information that goes into index files to
	   help	users and administrators.  IndexName should be the name	of
	   your	index, like a book title.  IndexDescription is a short de-
	   scription of	the index or a URL pointing to a more full descrip-
	   tion.  IndexPointer should be a pointer to the original informa-
	   tion, most likely a URL.  IndexAdmin	should be the name of the in-
	   dex maintainer and can include name and email information.  These
	   values should not be	more than 70 or	so characters and should be
	   contained in	quotes.	 Note that the automatically generated date in
	   index files is in D/M/Y and 24-hour format.

	   Examples:

	       IndexName "Linux	Documentation"
	       IndexDescription	"This is an index of /usr/doc on our Linux machine."
	       IndexPointer http://localhost/swish/linux/index.html
	       IndexAdmin webmaster

       Document	Source Directives

       These directives	control	what documents are indexed and how they	are
       accessed.  See also Directives for the File Access method only and Di-
       rectives	for the	HTTP Access Method Only	for directives that are	spe-
       cific to	those access methods.

       IndexDir	[directories or	files|URL|external program]
	   IndexDir defines the	source of the documents	for Swish-e.  Swish-e
	   currently supports three file access	methods: File system, HTTP
	   (also called	spidering), and	prog for reading files from an exter-
	   nal program.

	   The "-S" command line argument is used to select the	file access
	   method.

	       swish-e -c swish.config -S fs	- file system
	       swish-e -c swish.config -S http	- internal http	spider
	       swish-e -c swish.config -S prog	- external program of any type

	   For the fs method of	access IndexDir	is a space-separated list of
	   files and directories to index.  Use	a forward slash	as the path
	   separator in	MS Windows.

	   For the http	method the IndexDir setting is a list of space-sepa-
	   rated URLs.

	   For the prog	method the IndexDir setting is a list of space-sepa-
	   rated programs to run (which	generate documents for swish to	in-
	   dex).

	   You may specify more	than one IndexDir directive.

	   Any sub-directories of any listed directory will also be indexed.

	   Note: While processing directories, Swish-e will ignore any files
	   or directories that begin with a dot	(".").	You may	index files or
	   directories that begin with a dot by	specifying their name with
	   "IndexDir" or "-i".

	   Examples:

	       # Index this directory an any subdirectories
	       IndexDir	/usr/local/home/http

	       # Index the docs	directory in current directory
	       IndexDir	./docs

	       # Index these files in the current directory
	       IndexDir	./index.html ./page1.html ./page2.html
	       # and index this	directory, too
	       IndexDir	../public_html

	   For the HTTP	method of access specify the URL's from	which you want
	   the spidering to begin.

	   Example:

	       IndexDir	http://www.my-site.com/index.html
	       IndexDir	http://localhost/index.html

	   Obviously, using the	HTTP method to index is	much slower than in-
	   dexing local	files.	Be well	aware that some	sites do not appreci-
	   ate spidering and may block your IP address.	 You may wish to con-
	   tact	the remote site	before spidering their web site.  More infor-
	   mation about	spidering can be found in Directives for the HTTP Ac-
	   cess	Method Only below.

	   For the prog	method of access IndexDir specifies the	path to	the
	   program(s) to execute.  The external	program	must correctly format
	   the documents being passed back to Swish-e.	Examples of external
	   programs are	provided in the	prog-bin directory.

	       IndexDir	./myprogram.pl

	   See prog for	details.

	   Note: Not all directives work with all methods.

       NoContents *list	of file	suffixes*
	   Files with these suffixes will not have their contents indexed, but
	   will	have their path	name (file name) indexed instead.

	   If the file's type is HTML or HTML2 (as set by "IndexContents" or
	   "DefaultContents") then the file will be parsed for a HTML title
	   and that title will be indexed.  Note that you must set the file's
	   type	with "IndexContents" or	"DefaultContents": ".html" and ".htm"
	   are NOT type	HTML by	default.  For example:

	      IndexContents HTML* .htm .html

	   If a	title is found,	it will	still be checked for "FileRules	ti-
	   tle", and the file will be skipped if a match is found.  See
	   "FileRules".

	   If the file's type is not HTML, or it is HTML and no	title is
	   found, then the file's path will be indexed.

	   For example,	this will allow	searching by image file	name.

	       NoContents .gif .xbm .au	.mov .mpg .pdf .ps

	   Note: Using this directive will not cause files with	those suffixes
	   to be indexed.  That	is, if you use "IndexOnly" to limit the	types
	   of files that are indexed, then you must specify in "IndexOnly" the
	   same	suffixes listed	in "NoContents".

	   This	does not work:

	       # Wrong!
	       IndexOnly .htm .html
	       NoContents .gif .xbm .au	.mov .mpg .pdf .ps

	   A "-S prog" program may set the "No-Contents:" header to enable
	   this	feature	for a specific document	(although it would be smarter
	   for the "-S prog" program to	simply only send the pathname or title
	   to be indexed.

       ReplaceRules [replace|remove|prepend|append|regex]
	   ReplaceRules	allows you to make changes to file pathnames before
	   they're indexed.  These changed file	names or URLs will be returned
	   in search results.

	   For example,	you may	index your files locally (with the File	system
	   indexing method), yet return	a URL in search	results.  This direc-
	   tive	can be used to map the file names to their respective URLs on
	   your	web server.

	   There are five operations you can specify: replace, append, remove,
	   prepend, and	regex They will	parse the pathname in the order	you've
	   typed these commands.

	   This	directive uses C library regex.h regular expressions.

	      replace "the string you want replaced" "what to change it	to"
	      remove "a	string to remove"
	      prepend "a string	to add before the result"
	      append "a	string to add after the	result"
	      regex  "/search string/replace string/options"

	   Remember, quotes are	needed if an expression	contains white space,
	   and backslashes have	special	meaning.

	   Regex is an Extended	Regular	Expression.  The first character found
	   is the delimiter (but it's not smart	enough to use matched chars
	   such	as [], (), and {}).

	   The replace string may use substitution variables:

	       $0      the entire matched (sub)string
	       $1-$9   returns patterns	captured in "("	")" pairs
	       $`      the string before the matched pattern
	       $'      the string after	the matched pattern

	   The options change the behavior of expression:

	       i       ignore the case when matching
	       g       repeat the substitution for the entire pattern

	   Examples:

	       ReplaceRules replace testdir/ anotherdir/
	       ReplaceRules replace [a-z_0-9]*_m.*\.html index.html

	       ReplaceRules remove testdir/

	       ReplaceRules prepend http://localhost/
	       ReplaceRules append .html

	       ReplaceRules regex  !^/web/(.+)/!http://$1.domain.com/!
	       replaces	a file path:
		   /web/search/foo/index.html
	       with
		   http://search.domain.com/foo/index.html

	       ReplaceRules regex  #^#http://localhost/www#
	       ReplaceRules prepend http://localhost/www  (same	thing)

	       # Remove	all extensions from C source files
	       ReplaceRules remove .c	  # ERROR! That	"." is *any char*
	       ReplaceRules remove \.c	  # much better...

	       ReplaceRules remove "\\.c" # if in quotes you need double-backslash!
	       ReplaceRules remove "\.c"  # ERROR! "\."	-> "." and is *any char*

       IndexContents [TXT|HTML|XML|TXT2|HTML2|XML2|TXT*|HTML*|XML*]  *file ex-
       tensions*
	   The "IndexContents" directive assigns one of	Swish-e's document
	   parsers to a	document, based	on the its extension.  Swish-e cur-
	   rently knows	how to parse TXT, HTML,	and XML	documents.

	   The XML2, HTML2, and	TXT2 parsers are currently only	available when
	   Swish-e is configured to use	libxml2.

	   You may use XML*, HTML*, and	TXT* to	select the parser automati-
	   cally.  If libxml2 is installed then	it will	be used	to parse the
	   content.  Otherwise,	Swish-e's internal parsers will	be used.

	   Documents that are not assigned a parser with "IndexContents" will,
	   by default, use the HTML2 parser if libxml2 is installed, otherwise
	   will	use Swish-e's internal HTML parser.  The "DefaultContents" di-
	   rective may be used to assign a parser to documents that do not
	   match a file	extension defined with the "IndexContents" directive.

	   Example:

	       IndexContents HTML* .htm	.html .shtml
	       IndexContents TXT*  .txt	.log .text
	       IndexContents XML*  .xml

	   HTML* is the	default	type for all files, unless otherwise specified
	   (and	this default can be changed by the DefaultContents directive.
	   Swish-e parses titles from HTML files, if available,	and keeps
	   track of the	context	of the text for	context	searching (see "-t" in
	   SWISH-RUN).

	   If using filters (with the "FileFilter" directive) to convert docu-
	   ments you should include those extensions, too.  For	example, if
	   using a filter to convert .pdf to .html, you	need to	tell Swish-e
	   that	.pdf should be indexed by the internal HTML parser:

	       FileFilter  .pdf	  pdf2html
	       IndexContent  HTML  .pdf

	   See also Document Filter Directives.

	   Note: Some of this may be changed in	the future to use content-
	   types instead of file extensions.  See SWISH-3.0

       DefaultContents [TXT|HTML|XML|TXT2|HTML2|XML2|TXT*|HTML*|XML*]
	   This	sets the default parser	for documents that are not specified
	   in IndexContents. If	not specified the default is HTML.

	   The XML2, HTML2, and	TXT2 parsers are currently only	available when
	   Swish-e is configured to use	libxml2.

	   You may use XML*, HTML*, and	TXT* to	select the parser automati-
	   cally.  If libxml2 is installed then	it will	be used	to parse the
	   content.  Otherwise,	Swish-e's internal parsers will	be used.

	   Example:

	       DefaultContents HTML

	   The "DefaultContents" directive should be used when spidering, as
	   HTML	files may be returned without a	file extension (such as	when
	   requesting a	directory and the default index.html is	returned).

       FileInfoCompression [yes|NO]
	   ** This directive is	currently not supported	**

	   Setting FileInfoCompression to "yes"	will compress the index	file
	   to save disk	space.	This may result	in longer indexing times.  The
	   default is "no".

	   Also	see the	"-e" switch in SWISH-RUN for saving RAM	during index-
	   ing.

       Document	Contents Directives

       These directives	control	what information is extracted from your	source
       documents, and how that information is made available during searching.

       ConvertHTMLEntities [YES|no]
	   ASCII entities can be converted automatically while indexing	docu-
	   ments of type HTML (not for HTML2).	For performance	reasons	you
	   may wish to set this	to "no"	if your	documents do not contain HTML
	   entities.  The default is "yes".

	   If "ConvertHTMLEntities" is set "no"	the entities will be indexed
	   without conversion.

	   NOTE: Entities within XML files and files parsed with libxml2
	   (HTML2) are converted regardless of this setting.

       MetaNames *list of names*
	   META	names are a way	to define "fields" in your XML and HTML	docu-
	   ments.  You can use the META	names in your queries to limit the
	   search to just the words contained in that META name	of your	docu-
	   ment.  For example, you might have a	META tagged field in your doc-
	   uments called "subjects" and	then you can search your documents for
	   the word "foo" but only return documents where "foo"	is within the
	   "subjects" META tag.

	       swish-e -w subjects=foo

	   (See	also the "-t" switch in	SWISH-RUN for information about	con-
	   text	searching in HTML documents.)

	   The MetaNames directive is a	space separated	list.  For example:

	       MetaNames meta1 meta2 keywords subjects

	   You may also	use "UndefinedMetaTags"	to specify automatic extrac-
	   tion	of meta	names from your	HTML and XML documents,	and also to
	   ignore indexing content of meta tags.

	   META	tags can have two formats in your HTML source documents:

	       <META NAME="meta1" CONTENT="some	content">

	   and (if using the HTML2/libxml2 parser)

	       <meta1>
		   some	content
	       </meta1>

	   But this second version is invalid HTML, and	will generate a	warn-
	   ing if ParserWarningLevel is	set (libxml2 only).

	   And in XML documents, use the format:

	       <meta1>
		   Some	Content
	       </meta1>

	   Then	you can	limit your search to just META meta1 like this:

	       swish-e -w 'meta1=(apples or oranges)'

	   You may nest	the XML	and the	start/end tag versions:

	       <keywords>
		   <tag1>
		       some content
		   </tag1>
		   <tag2>
		       some other content
		   </tag2>
	       <keywords>

	   Then	you can	search in both tag2 and	tag2 with:

	       swish-e -w 'keywords=(query words)'

	   Swish-e indexes all text as some metaname.  The default is
	   "swishdefault", so these two	queries	are the	same:

	       swish-e -w foo
	       swish-e -w swishdefault=foo

	   When	indexing HTML Swish-e indexes the HTML title as	default	text,
	   so when searching Swish-e will find matches in both the HTML	body
	   and the HTML	title.	Swish also, by default,	indexes	content	of
	   meta	tags.  So:

	       swish-e -w foo

	   will	find "foo" in the body,	the title, or any meta tags.

	   Currently, there's no way to	prevent	Swish-e	from indexing the ti-
	   tle contents	along with the body contents, but see "Undefined-
	   MetaTags" for how to	control	the indexing of	meta tags.

	   If you would	like to	search just the	title text, you	may use:

	       MetaNames swishtitle

	   This	will index the title text separately under the built-in	swish
	   internal meta name "swishtitle".  You may then search like

	       swish-e -w foo  -- search for "foo" in title, body (and undefined meta tags)
	       swish-e -w swishtitle=foo -- search for "foo" in	title only

	   In addition to swishtitle, you can limit searches to	documents'
	   path	with:

	      MetaNames	swishdocpath

	   Then	to search for "foo" but	also limit searches to documents that
	   include "manual" or "tutorial" in their path:

	      swish-e -w foo swishdocpath=(manual or tutorial)

	   See also "ExtractPath".

       MetaNameAlias *meta name* *list of aliases*
	   MetaNameAlias assigns aliases for a meta name.  For example,	if
	   your	documents contain meta tags "description", "summary", and
	   "overview" that all give a summary of your documents	you could do
	   this:

	       MetaNames summary
	       MetaNameAlias summary description overview

	   Then	all three tags will get	indexed	as meta	tag "summary".	You
	   can then search all the fields as:

	       -w summary=foo

	   The Alias work at search time, too.	So these will also limit the
	   search to the "summary" meta	name.

	       -w description=foo
	       -w overview=foo

       MetaNamesRank integer *list of meta names*
	   You can assign a bias to metanames that will	affect how ranking is
	   calculated.	The range of values is from -10	to +10,	with zero be-
	   ing no bias.

	       MetaNamesRank 4 subject
	       MetaNamesRank 3 swishdefault
	       MetaNamesRank 2 author publisher
	       MetaNamesRank -5	wrongwords

	   This	feature	is still considered experimental. If you use it,
	   please send feedback	to the discussion list.

       HTMLLinksMetaName *metaname*
	   Allows indexing of HTML links.  Normally, HTML links	(href tags)
	   are not indexed by Swish-e.	This directive defines a metaname, and
	   links will be indexed under this meta name.

	   Example:

	       HTMLLinksMetaName links

	   Now,	to limit searches to files with	a link to "home.html" do this:

	       -w links='"home.html"'

	   The double quotes force a phrase search.

	   To make Swish-e index links as normal text, you may use:

	       HTMLLinksMetaName swishdefault

	   This	feature	is only	available with the libxml2 HTML	parser.

       ImageLinksMetaName *metaname*
	   Allows indexing of image links under	a metaname.  Normally, image
	   URLs	are not	indexed.

	   Example:

	       ImagesLinksMetaName images

	   Now,	if you would like to find pages	that include a nice image of a
	   beach:

	       -w images='beach'

	   To make Swish-e index links as normal text, you may use:

	       ImageLinksMetaName swishdefault

	   This	feature	is only	available with the libxml2 HTML	parser.

       IndexAltTagMetaName *tagname*|as-text
	   Allows indexing of images <IMG> ALT tag text.  Specify either a tag
	   name	which will be used as a	metaname, or the special text
	   "as-text" which says	to index the ALT text as if it were plain text
	   at the current location.

	   For example,	by specifying a	tag name:

	      IndexAltTagMetaName bar

	   would make this markup:

	       <foo>
		   <img	src="/someimage.png" alt="Alt text here">
	       </foo>

	   appear like

	       <foo>
		   <bar>Alt text here</bar>
	       </foo>

	   Then	the normal rules ("MetaNames" and "PropertyNames") apply to
	   how that text is indexed.

	   If you use the special tag "as-text"	then

	       <foo>
		   <img	src="/someimage.png" alt="Alt text here">
	       </foo>

	   simply becomes

	       <foo>
		   Alt text here
	       </foo>

	   This	feature	is only	available when using the libxml2 parser	(HTML2
	   and XML2).

       AbsoluteLinks [yes|NO]
	   If this is set true then Swish-e will attempt to convert relative
	   URIs	extracted from HTML documents for use with "HTMLLinksMetaName"
	   and "ImageLinksMetaName" into absolute URIs.	 Swish-e will use any
	   <BASE> tag found in the document, otherwise it will use the file's
	   pathname.  The pathname used	will be	the pathname *after* "Re-
	   placeRules" has been	applied	to the document's pathname.

	   For example,	say you	wish to	index image links under	the metaname
	   "images".

	       ImageLinksMetaName images

	   If an image is located in http://localhost/vacations/france/in-
	   dex.html and	"AbsoluteLinks"	is set to no, then a image within that
	   document:

		<img src="beach.jpeg">

	   will	only index "beach.jpeg".

	   But,	if you want more detail	when searching,	you can	enable "Abso-
	   luteLinks" and Swish-e will index "http://localhost/vaca-
	   tions/france/beach.jpeg".  You can then look	for images of beaches,
	   but only in France:

	       -w images=(beach	and france)

	   This	also means you can search for any images within	France:

	       -w images=(france)

	   This	feature	is only	available with the libxml2 HTML	parser.

       UndefinedMetaTags [error|ignore|INDEX|auto]
	   This	directive defines the behavior of Swish-e during indexing when
	   a meta name is found	but is not listed in MetaNames.	 There are
	   four	choices:

	   error
	     If	a meta name is found that is not listed	in MetaNames then in-
	     dexing will be halted and an error	reported.

	   ignore
	     The contents of the meta tag are ignored and not indexed unless a
	     metaname has been defined with the	"MetaNames" directive.

	   index
	     The contents of the meta tag are indexed, but placed in the main
	     index unless there's an enclosing metatag already in force. This
	     is	the default.

	   auto
	     This method create	meta tags automatically	for HTML meta names
	     and XML elements.	Using this is the same as specifying all the
	     meta names	explicitly in a	MetaNames directive.

       UndefinedXMLAttributes [DISABLE|error|ignore|index|auto]
	   This	is similar to "UndefinedMetaTags", but only applies to XML
	   documents (parsed with libxml2).  This allows indexing of attribute
	   content, and	provides a way to index	the content under a metaname.
	   For example,	"UndefinedXMLAttributes" can make

	       <person age="23">
		     John Doe
	       </person>

	   look	like the following to swish:

	       <person>
		   <person.age>
		       23
		   </person.age>
		   John	Doe
	       </person>

	   What	happens	to the text "23" will depend on	the setting of "Unde-
	   finedXMLAttributes":

	   disable
	     XML attributes are	not parsed and not indexed.  This is the de-
	     fault.

	   error
	     If	the concatenated meta name (e.g. person.age) is	not listed in
	     MetaNames then indexing will be halted and	an error reported.

	   ignore
	     The contents of the meta tag are ignored and not indexed unless a
	     metaname has been defined with the	"MetaNames" directive.

	   index
	     The contents of the meta tag are indexed, but placed in the main
	     index unless there's an enclosing metatag already in force.

	   auto
	     This method will create meta tags from the	combined element and
	     attributes	(and XML Class name) This options should be used with
	     caution as	it can generate	a lot of metaname entries.

	     See also the example below	"XMLClassAttribues".

       XMLClassAttributes *list	of XML attribute names*
	   Combines an XML class name with the element name to make up a
	   metaname.  For example:

	       XMLClassAttributes class

	       <person class="first">
		   John
	       </person>
	       <person class="last">
		   Doe
	       </person>

	   Will	appear to Swish-e as:

	       <person>
		   <person.first>
		   John
		   </person.first>
	       </person>
	       <person>
		   <person.last>
		   Doe
		   </person.last>
	       </person>

	   How the data	is indexed depends on "MetaNames" and "Undefined-
	   MetaTags".

	   Here's an example using the following configuration which combines
	   the two directives "XMLClassAttributes" and "UndefinedXMLAt-
	   tributes".

	       XMLClassAttributes class
	       UndefinedMetaTags auto
	       UndefinedXMLAttributes auto
	       IndexContents XML2 .xml

	   The source XML file looks like:

	       <xml> <person class="student" phone="555-1212" age="102"> John </person>
	       <person greeting="howdy">Bill</person> </xml>

	   Swish-e parses as:

	       ./swish-e -c 2 -i 1.xml -T parsed_tags  parsed_text  -v 0
	       Indexing	Data Source: "File-System"

	       <xml> (MetaName)

		   <person> (MetaName)
		       <person.student>	(MetaName)
			   <person.student.phone> (MetaName)
			       555-1212
			   </person.student.phone>
			   <person.student.age>	(MetaName)
			       102
			   </person.student.age>
			   John
		   </person>

		   <person> (MetaName)
		       <person.greeting> (MetaName)
			   howdy
		       </person.greeting>
		       Bill
		   </person>

	       </xml>
	       Indexing	done!

	   One thing to	note is	that the first <person>	block finds a class
	   name	"student" so all metanames that	are created from attributes
	   use the combined name "person.student".  The	second <person>	block
	   doesn't contain a "class" so, the attribute name is combined	di-
	   rectly with the element name	(e.g. "person.greeting").

       ExtractPath *metaname* [replace|remove|prepend|append|regex]
	   This	directive can be used to index extracted parts of a document's
	   path.  A common use would be	to limit searches to specific areas of
	   your	file tree.

	   The extracted string	will be	indexed	under the specified meta name.

	   See "ReplaceRules" for a description	of the various pattern re-
	   placement methods, but you will use the regex method.

	   For example,	say your file system (or web tree) was organized into
	   departments:

	       /web/sales/foo...
	       /web/parts/foo...
	       /web/accounting/foo...

	   And you wanted a way	to limit searches to just documents under
	   "sales".

	       ExtractPath department regex !^/web/([^/]+)/.*$!$1!

	   Which says, extract out the department name (as substring $1) and
	   index it as meta name "department".	Then to	limit a	search to the
	   sales department:

	       swish-e -w foo AND department=sales

	   Note	that the "regex" method	uses a substitution pattern, so	to in-
	   dex only a sub-string match the entire document path	in the regular
	   expression, as shown	above.	Otherwise any part that	is not matched
	   will	end up in the substitution pattern.

	   See the "ExtractPathDefault"	option for a way to set	a value	if not
	   patterns match.

	   Although unlikely, you may use more than one	"ExtractPath" direc-
	   tive.  More than one	directive of the same meta name	will operate
	   successively	(in order listed in the	configuration file) on the
	   path.  This allows you to use regular expressions on	the results of
	   the previous	pattern	substitution (as if piping the output from one
	   expression to the patter of the next).

	       ExtractPath foo regex !^(...).+$!$1!
	       ExtractPath foo regex !^.+(.)$!$1!

	   So, the third letter	is indexed as meta name	"foo" if both patterns
	   match.

	       ExtractPath foo regex !^X(...).+$!$1!
	       ExtractPath foo regex !^.+(.)$!$1!

	   Now (not the	"X"), if the first pattern doesn't match, the last
	   character of	the path name is indexed.  You must be clear on	this
	   behavior if you are using more than one "ExtractPath" directive
	   with	the same metaname.

	   The document	path operated on is the	real path swish	used to	access
	   the document.  That is, the "ReplaceRules" directive	has no effect
	   on the path used with "ExtractPath".

	   The full path is used for each meta name if more than one "Extract-
	   Path" directive is used.  That is, changes to the path used in "Ex-
	   tractPath foo" do not affect	the path used by "ExtractPath bar".

       ExtractPathDefault *metaname* default_value
	   This	can be used with "ExtractPath" to set a	default	string to in-
	   dex under the given metaname	if none	of the "ExtractPath" patterns
	   match.

	   For example,	say your want to index each document with a metaname
	   "department"	based on the following path examples:

	       /web/sales/foo...
	       /web/parts/foo...
	       /web/accounting/foo...

	   But you are also indexing documents that do not follow that pattern
	   and you want	to search those	separately, too.

	       ExtractPath department regex !^/web/([^/]+)/.*$!$1!
	       ExtractPathDefault department other

	   Now,	you may	search like this:

	       -w foo department=(sales)      -	limit searches to the sales documents
	       -w foo department=(parts)      -	limit searches to the parts documents
	       -w foo department=(accounting) -	limit searches to the accounting documents
	       -w foo department=(other)      -	everything but sales, parts, and accounting.

	   This	basically is a shortcut	for:

	       -w foo not department=(sales or parts or	accounting)

	   but you don't need to keep track of what was	extracted.

       PropertyNames *list of meta names*
       PropertyNamesCompareCase	*list of meta names*
       PropertyNamesIgnoreCase *list of	meta names*
	   Swish-e allows you to specify certain META tags that	can be used as
	   document properties.	 The contents of any META tag that has been
	   identified as a document property can be returned as	part of	the
	   search results along	with the rank, file name, title, and document
	   size	(see the "-p" and "-x" switches	in SWISH-RUN).

	   Properties are useful for returning additional data from documents
	   in search results --	this saves the effort of reading and parsing
	   the source files while reading Swish-e search results, and is espe-
	   cially useful when the source documents are no longer available or
	   slow	to access (e.g.	over http).

	   Another feature of properties is that Swish-e can use the Property-
	   Names for sorting the search	results	(see the "-s" switch).

	       PropertyNames author subjects

	   Two variations are available.  "PropertyNamesCompareCase" and
	   "PropertyNamesIgnoreCase".  These tell Swish-e to either ignore or
	   compare case	when sorting results.  The default for "PropertyNames"
	   is to ignore	the case.

	       PropertyNamesIgnoreCase subject
	       PropertyNamesCompareCase	keyword

	   The defaults	for "internal" properties are:

	       swishtitle	   --  ignore the case
	       swishdocpath	   --  compare case
	       swishdescription	   --  compare case

	   These can be	overridden with	"PropertyNamesCompareCase" and "Prop-
	   ertyNamesIgnoreCase".

	       PropertyNamesCompareCase	swishtitle

	   Use of PropertyNames	will increase the size of your index files,
	   sometimes significantly.  Properties	will be	compressed if Swish-e
	   is compiled with zlib as described in the INSTALL manual page.

	   If Swish-e finds more than one property of the same name in a docu-
	   ment	the property's contents	will be	concatinated for strings, and
	   a warning issues for	numeric	(or date) properties.

       PropertyNamesNoStripChars
	   PropertyNamesNoStripChars specifies that the	listed properties
	   should not have strings of low ASCII	characters replaced with a
	   space character.  Properties	will be	stored as found	in the docu-
	   ment.

	   When	printing properties with the swish-e binary newlines are re-
	   placed with a space character.  Use the swish-e library (or
	   SWISH::API perl module) to fetch properties without newlines	re-
	   placed.

       PropertyNamesNumeric
	   This	directive is similar to	"PropertyNames", but it	flags the
	   property as being a string of digits	(integer value)	that will be
	   stored as binary data instead of a string.  This allows sorting
	   with	"-s" and limiting with "-L" to sort and	limit the property
	   correctly.

	   Swish-e uses	strtoul(3) to convert the string into an unsigned long
	   integer.  Therefore,	only positive integers can be stored.

	   Future versions of Swish-e may be able to store different property
	   types (such as negative integers and	real numbers).	This directive
	   may change in future	releases of Swish.

       PropertyNamesDate
	   This	directive is exactly like "PropertyNamesNumeric", but it also
	   flags the number as a machine timestamp (seconds since Epoch), and
	   will	print a	formatted date when returning this property.  See "-x"
	   in SWISH-RUN.

	   Swish-e will	not parse dates	when indexing; you must	use a time-
	   stamp.

       PropertyNameAlias  *property name* *list	of aliases*
	   This	allows aliases for a property name.  For example, if you are
	   indexing HTML files,	plus XML files that are	written	in English,
	   German, and Spanish and thus	use the	tags "title", "titel", and
	   "titulo" you	can use:

	       PropertyNameAlias swishtitle title titel	titulo titulo

	   Note	that "swishtitle" is the built-in property used	to store the
	   title of a document,	and therefore you do not need to specify it as
	   a PropertyName before use.

       PropertyNamesMaxLength  integer *list of	meta names*
	   This	option will set	the max	length of the text stored in a prop-
	   erty.  You must specify a number between 0 and the max integer size
	   on your platform, and a list	of properties.	The properties speci-
	   fied	must not be aliases.

	   If any of the property names	do not exist they will be created
	   (e.g. you do	not need to define the property	with PropertyNames
	   first).

	   In general, this feature will only be useful	when parsing HTML or
	   XML with the	libxml2	parser.

	   For example:

	       PropertyNamesMaxLength 1000 swishdescription
	       PropertyNameAlias swishdescription body

	   Is somewhat like

	       StoreDescription	HTML <body> 1000
	       StoreDescription	XML <body> 1000
	       StoreDescription	HTML2 <body> 1000
	       StoreDescription	XML2 <body> 1000

	   but StoreDescription	allows setting the tag for each	parser type.

	       PropertyNamesMaxLength 1000 headings
	       PropertyNameAlias headings h1 h2	h3 h4

	   collects all	the heading text into a	single property	called "head-
	   ings", not to exceed	1000 characters.

       PropertyNamesSortKeyLength  integer *list of meta names*
	   Sets	the length of the string used when sorting.  The default is
	   100 characters.  The	-T metanames debugging option will list	the
	   current values for an index.

	   This	setting	is used	when sorting during indexing, and perhaps when
	   sorting while searching.  It	also effects the order when limiting
	   to a	range of values	with the -L option.

       PreSortedIndex *list of property	names*
	   By default Swish-e generates	presorted tables while indexing	for
	   each	property name.	This allows faster sorting when	generating re-
	   sults.  On large document collections this presorting may add to
	   the indexing	time, and also adds to the total size of the index.
	   This	directive can be used to customize exactly which properties
	   will	be presorted.

	   If "PreSortedIndex" it is not present in the	config file (default
	   action), all	the properties will be presorted at indexing time.  If
	   it is present without any parameter,	no properties will be pre-
	   sorted.  Otherwise, only the	property names specified will be pre-
	   sorted.

	   For example,	if you only wish to sort results by a property called
	   "title":

	       PropertyNames title age time
	       PreSortedIndex  title

       StoreDescription	[XML <tag> size|HTML <meta> size|TXT size]
	   StoreDescription allows you to store	a document description in the
	   index file.	This description can be	returned in your search	re-
	   sults when the "-x" switch is used to include the swishdescription
	   for extended	results, or by using "-p swishdescription".

	   The document	type (XML, HTML	and TXT) must match the	document type
	   currently being indexed as set by "IndexContents" or	"DefaultCon-
	   tents".  See	those directives for possible values.  A common	prob-
	   lem is using	"StoreDescription" yet not setting the document's type
	   with	"IndexContents"	or "DefaultContents".  Another problem is dif-
	   ferent types:

	       IndexContents HTML2 .html
	       StoreDescription	HTML <body>

	   Then	.html documents	are assigned a type of HTML2 (and parsed by
	   the libxml2 parser),	but the	description will not be	stored since
	   it is type HTML instead of HTML2.

	   For text documents you specify the type TXT (or TXT2	or TXT*) and
	   the number of characters to capture.

	       StoreDescription	TXT 20

	   The above stores only the first twenty characters from the text
	   file	in the Swish-e index file.

	   For HTML, and XML file types, specify the tag to use	for the	de-
	   scription, and optionally the number	of characters to capture.  If
	   not specified will capture the entire contents of the tag.

	       StoreDescription	HTML <body> 20000
	       StoreDescription	XML  <desc> 40

	   Again, note that documents must be assigned a document type with
	   "IndexContents" or "DefaultContents"	to use this feature.

	   Swish-e will	compress the descriptions (or any other	large prop-
	   erty) if compiled to	use zlib (see INSTALL).	 This is recommended
	   when	using StoreDescription and a large number of documents.	 Com-
	   pression of 30% to 50% is not uncommon with HTML files.

       PropCompressionLevel [0-9]
	   This	directive sets the compression level used when storing proper-
	   ties	to disk.  A setting of zero is no compression, and a setting
	   of nine is the most compression.

	   The default depends on the default setting compiled with zlib, but
	   is typically	six.

	   This	option is useful when using "StoreDescription" to store	a
	   large amount	text in	properties (or if using	"PropertyNames"	with
	   large property sizes).

	   Properties must be over a value defined in config.h (100 is the de-
	   fault) before compression will be attempted.	 Swish-e will never
	   store the results of	the compression	if the compressed data is
	   larger than the original data.

	   This	option is only available when Swish-e is compiled with zlib
	   support.

       TruncateDocSize *number of characters*
	   TruncateDocSize limits the size of a	document while indexing	docu-
	   ments and/or	using filters.	This config directive truncates	the
	   numbers of read bytes of a document to the specified	size.  This
	   means: if a document	is larger, read	only the specified numbers of
	   bytes of the	document.

	   Example:

	       TruncateDocSize	  10000000

	   The default is zero,	which means read all data.

	   Warning: If you use TruncateDocSize,	use it with care!  Truncate-
	   DocSize is a	safety belt only, to limit e.g.	 filteroutput, when
	   accessing databases,	or to limit "runnaway" filters.	 Truncating
	   doc input may destroy document structures for Swish-e (e.g.	swish
	   may miss closing tags for XML or HTML documents).

	   TruncateDocSize does	not currently work with	the "prog" input
	   source method.

       FuzzyIndexingMode NONE|Stemming|Soundex|Metaphone|DoubleMetaphone
	   Selects the type of index to	create.	 Only one type of index	may be
	   created.

	   It's	a good idea to create both a normal index and a	fuzzy index
	   and allow your search interface select which	index to use.  Many
	   people find the fuzzy searches to be	too fuzzy.

	   The available fuzzy indexing	options	can be displayed by running

	      swish-e -T LIST_FUZZY_MODES

	   Available options include:

	   None
	       Words are stored	in the index without any conversion.  This is
	       the default.

	   Stemming_*
	       This options uses one of	the installed Snowball stemmers
	       (http://snowball.tartarus.org/).

	       The installed stemmers can be viewed by running

		  swish-e -T LIST_FUZZY_MODES

	       For example, to use the Spanish stemming	module:

		  FuzzyIndexingMode Stemming_es

	   Stem	or Stemming_en
	       **This option is	no longer supported.**

	       Selects the legacy Swish-e English stemmer.

	       This is deprecated in favor of the Snowball English stemmer
	       Stemming_en1.

	       Words are converted using the Porter stemming algorithm.

	       From: http://www.tartarus.org/~martin/PorterStemmer/

		   The Porter stemming algorithm (or Porter stemmer) is	a
		   process for removing	the commoner morphological and inflexional
		   endings from	words in English. Its main use is as part of a
		   term	normalisation process that is usually done when	setting	up
		   Information Retrieval systems.

	       This will help a	search for "running" to	also find "run"	and
	       "runs", for example.

	       The stemming function does not convert words to their root,
	       rather programmatically removes endings on words	in an attempt
	       to make similar words with different endings stem to the	same
	       string of characters.  It's not a perfect system, and searches
	       on stemmed indexes often	return curious results.	 For example,
	       two entirely different words may	stem to	the same word.

	       Stemming	also can be confusing when used	with a wildcard	(trun-
	       cation).	 For example, you might	expect to find the word	"run-
	       ning" by	searching for "runn*".	But this fails when using a
	       stemmed index, as "running" stems to "run", yet searching for
	       "runn*" looks for words that start with "runn".

	   Soundex
	       Soundex was developed in	the 1880s so records for people	with
	       similar sounding	names could be found more readily.  Soundex is
	       a coded surname based on	the way	a surname sounds rather	than
	       spelling.  Surnames that	sound similar, like Smith and Smyth,
	       are filed together under	the same Soundex code.	This is	mostly
	       useful for US English.

	       Soundex should not be used to search for	sound-alike words.
	       Metaphone would be more appropriate for generic sound matching
	       of words.  Soundex should only be used where you	need to	search
	       multiple	documents for proper names which sound similar.	 This
	       is primarily used for indexing genealogical records.  This may
	       be useful for indexing other collections	of data	consisting
	       mostly of names.	 Many common name variations are matched by
	       Soundex.	 The only notable exception is the first letter	of the
	       name.  The first	letter is not matched for sound.

	   Metaphone and DoubleMetaphone
	       Words are transformed into a short series of letters represent-
	       ing the sound of	the word (in English).	Metaphone algorithms
	       are often used for looking up mis-spelled words in dictionary
	       programs.

	       From: http://aspell.sourceforge.net/metaphone/

		   Lawrence Philips' Metaphone Algorithm is an algorithm which returns
		   the rough approximation of how an English word sounds.

	       The "DoubleMetaphone" mode will sometimes generate two differ-
	       ent metaphones for the same word.  This is supposed to be use-
	       ful when	a word may be pronounced more than one way.

	       A metaphone index should	give results somewhere in between
	       Soundex and Stemming.

       UseStemming [yes|NO]
	   Put yes to apply word stemming algorithm during indexing, else no.

	       UseStemming no
	       UseStemming yes

	   When	UseStemming is set to "yes" every word is stemmed before plac-
	   ing it in to	the index.

	   This	option is deprecated.  It has been superceded by "FuzzyIndex-
	   ingMode".

       UseSoundex [yes|NO]
	   When	UseSoundex is set to "yes" every word is converted to a
	   Soundex code	before placing it in to	the index.

	   This	option is deprecated.  It has been superceded by "FuzzyIndex-
	   ingMode".

       IgnoreTotalWordCountWhenRanking [YES|no]
	   Put yes to ignore the total number of words in the file when	calcu-
	   lating ranking. Often better	with merges and	small files. Default
	   is yes.

	       IgnoreTotalWordCountWhenRanking no

	   The default was changed from	no to yes in version 2.2.

	   NOTE: must be set to	no if you intend to use	the -R 1 option	when
	   searching.

       MinWordLimit *integer*
	   Set the minimum length of an	word. Shorter words will not be	in-
	   dexed.  The default is 1 (as	defined	in src/config.h).

	       MinWordLimit 5

       MaxWordLimit *integer*
	   Set the maximum length of an	indexable word.	Every longer word will
	   not be indexed.  The	Default	is 40 (as defined in src/config.h).

       WordCharacters *string of characters*
       IgnoreFirstChar *string of characters*
       IgnoreLastChar *string of characters*
       BeginCharacters *string of characters*
       EndCharacters *string of	characters*
	   These settings define what a	word consists of to the	Swish-e	index-
	   ing engine.	Compiled in defaults are in src/config.h.

	   When	indexing Swish-e uses WordCharacters to	split up the document
	   into	words.	Words are defined by any string	of non-blank charac-
	   ters	that contain only the characters listed	in WordCharacters.  If
	   a string of characters includes a character that is not in Word-
	   Characters then the word will be spit into two or more separate
	   words.

	   For example:

	       WordCharacters abde

	   Would turn "abcde" into two words "ab" and "de".

	   Next, of these words, any characters	defined	in IgnoreFirstChar are
	   stripped off	the start of the word, and IgnoreLastChar characters
	   are stripped	off the	end of the word.  This allows, for example,
	   periods within a word (www.slashdot.com), but not at	the end	of a
	   word.  Characters in	IgnoreFirstChar	and IgnoreLastChar must	be in
	   WordCharacters.

	   Finally, the	resulting words	MUST begin with	one of the characters
	   listed in BeginCharacters and end with one of the characters	listed
	   in EndCharacters.  BeginCharacters and EndCharacters	must be	a sub-
	   set of the characters in WordCharacters.  Often, WordCharacters,
	   BeginCharacters and EndCharacters will all be the same.

	   Note	that the same process applies to the query while searching.

	   Getting these settings correct will take careful consideration and
	   practice.  It's helpful to create an	index of a single test file,
	   and then look at the	words that are placed in the index (see	the
	   "-v 4", "-D"	and "-k" searching switches).

	   Currently there is only support for eight-bit characters.

	   Example:

	       WordCharacters  .abcdefghijklmnopqrstuvwxyz
	       BeginCharacters abcdefghijklmnopqrstuvwxyz
	       EndCharacters   abcdefghijklmnopqrstuvwxyz
	       IgnoreFirstChar .
	       IgnoreLastChar  .

	   So the string

	       Please visit http://www.example.com/path/to/file.html.

	   will	be indexed as the following words:

	       please
	       visit
	       http
	       www.example.com
	       path
	       to
	       file.html

	   Which means that you	can search for "www.example.com" as a single
	   word, but searching for just	"example" will not find	the document.

	   Note: when indexing HTML documents HTML entities are	converted to
	   their character equivalents before being processed with these di-
	   rectives.  This is a	change from previous versions of Swish-e where
	   you were required to	include	the characters "0123456789&#;" to in-
	   dex entities.  See also "ConvertHTMLEntities"

       Buzzwords [*list	of buzzwords*|File: path]
	   The Buzzwords option	allows you to specify words that will be in-
	   dexed regardless of WordCharacters, BeginCharacters,	EndCharacters,
	   stemming, soundex and many of the other checks done on words	while
	   indexing.

	   Buzzwords are case insensitive.

	   Buzzwords should be separated by spaces and may span	multiple di-
	   rectives.  If the special format "File:filename" is used then the
	   Buzzwords will be read from an external file	during indexing.

	   Examples:

	       Buzzwords C++ TCP/IP

	       Buzzwords File: ./buzzwords.lst

	   If a	Buzzword contains search operator characters they must be
	   backslashed when searching.	For example:

	       Buzzwords C++ TCP/IP web=http

	       ./swish-e -w 'web\=http'

	   Buzzwords are found by splitting the	text on	whitespace, removing
	   "IgnoreFirstChar" and "IgnoreLastChar" characters from the word,
	   and then comparing with the list of "Buzzwords".  Therefore,	if
	   adding "Buzzwords" to an index you will probably want to define
	   "IgnoreFirstChar" and "IgnoreLastChar" settings.

	   Note: Buzzwords specific settings for "IgnoreFirstChar" and "Ig-
	   noreLastChar" may be	used in	the future.

       CompressPositions  [yes|NO]
	   This	option enables zlib compression	for individual word data in
	   the index file.  The	default	is NO, that is the index word data is
	   not compressed by default.

	   Enabling this option	can reduced the	size of	the index file,	but at
	   the expense of slower wildcard search times.

	   The default changed from YES	to NO starting with version 2.4.3.

       IgnoreWords [*list of stop words*|File: path]
	   The IgnoreWords option allows you to	specify	words to ignore,
	   called stopwords.  The default is to	not use	any stopwords.

	   Words should	be separated by	spaces and may span multiple direc-
	   tives.  If the special format "File:filename" is used then the stop
	   words will be read from an external file during indexing.

	   In previous versions	of Swish-e you could use the directive

	       IgnoreWords swishdefault	- obsolete!

	   to include a	default	list of	compiled in stopwords.	This keyword
	   is no longer	supported.

	   Examples:

	       IgnoreWords www http a an the of	and or

	       IgnoreWords File: ./stopwords.de

       UseWords	[*list of words*|File: path]
	   UseWords defines the	words that Swish-e will	index.	Only the words
	   listed will be indexed.

	   You can specify a list of words following the directive (you	may
	   specify more	than one "UseWords" directive in a config file),
	   and/or use the "File:" form to specify a path to a file containing
	   the words:

	       UseWords	perl python pascal fortran basic cobal php
	       UseWords	File: /path/to/my/wordlist

	   Please drop the Swish-e list	a note if you actually use this	fea-
	   ture.  It may be removed from future	versions.

       IgnoreLimit *integer integer*
	   This	automatically omits words that appear too often	in the files
	   (these words	are called stopwords). Specify a whole percentage and
	   a number, such as "80 256". This omits words	that occur in over 80%
	   of the files	and appear in over 256 files. Comment out to turn off
	   auto-stopwording.

	       IgnoreLimit 50 1000

	   Swish-e must	do extra processing to adjust the entire index when
	   this	feature	is used.  It is	recommended that instead of using this
	   feature that	you decided what words are stopwords and add them to
	   IngoreWords in your configuration file.  To do this,	use Ignore-
	   Limit one time and note the stop words that are found while index-
	   ing.	 Add this list to IgnoreWords, and then	remove IgnoreLimit
	   from	the configuration file.

       IgnoreMetaTags *list of names*
	   "IgnoreMetaTags" defines a list of metatags to ignore while index-
	   ing XML files (and HTML files if using libxml2 for parsing HTML).
	   All text within the tags will be ignored -- both for	indexing
	   ("MetaNames") and properties	("PropertyNames").  To still parse
	   properties, yet do not index	the text, see "UndefinedMetaTags".

	   This	option is useful to avoid indexing specific data from a	file.
	   For example:

	       <person>
		   <first_name>
		       William
		   </first_name> <last_name>
		       Shakespeare
		   </last_name>	<updated_date>
		       April 25, 1999
		   </updated_date>
	       </person>

	   In the above	example	you might not want to index the	updated	date,
	   and therefore prevent finding this record by	searching

	       -w 'person=(April)'

	   This	is solved by:

	       IgnoreMetaTags updated_date

	   See also "UndefinedMetaTags".

       IgnoreNumberChars *list of characters*
	   Experimental	Feature

	   This	experimental feature can be used to define a set of characters
	   that	describe a number.  If a word is found to contain only those
	   characters it will not be indexed.  The characters listed must be
	   part	of "WordCharacters" settings.  In other	words, the "word"
	   checked is a	word that Swish-e would	otherwise index.

	   For example,

	       IgnoreNumberChars 0123456789$.,

	   Then	Swish-e	would not index	the following:

	       123
	       123,456.78
	       $123.45

	   You might be	tempted	to avoid indexing hex numbers with:

	       IgnoreNumberChars 0123456789abcdef

	   which will not index	0D31, but will also not	index the word "bad".

	   This	is an experimental feature that	may change in future versions.
	   One possible	change is to use regular expressions instead.

       IndexComments [NO|yes]
	   This	option allows the user decide if to index the contents of HTML
	   comments.  Default is no. Set to yes	if comment indexing is re-
	   quired.

	       IndexComments yes

	   Note: This is a change in the default behavior prior	to version
	   2.2.

       TranslateCharacters [*string1 string2*|:ascii7:]
	   The TranslateCharacters directive maps the characters in string1 to
	   the characters listed in string2.

	   For example:

	       # This will index a_b as	a-b and	amo as amo
	       TranslateCharacters _a -a

	   "TranslateCharacters	:ascii7:" is a predefined set of characters
	   that	will translate eight bit characters to ascii7 characters.  Us-
	   ing the :ascii7: rule will translate	"Aac" to "aac".	This means:
	   searching "Celik", "celik" or "celik" will all match	the same word.

	   TranslateCharacters is done early in	the indexing process, after
	   converting HTML entities but	before splitting the input text	into
	   words based on WordCharacters.  So characters you are translating
	   from	do not need to be listed in word characters.

	   The same character translations take	place when searching.

       BumpPositionCounterCharacters *string*
	   When	indexing Swish-e assigns a word	position to each word.	This
	   enables phrase searching.  There may	be cases where you would like
	   to prevent phrase matching.	The BumpPositionCounterCharacters di-
	   rective allows you to specify a set of characters that when found
	   in the text will increment the word position	-- effectively pre-
	   venting phrase matches across that character.

	   For example,	if you have a tag:

	       <subjects>
		   computer programming	| apple	computers
	       </subjects>

	   You might want to prevent matching "programming apple" in that meta
	   name.

	       BumpPositionCounterCharacters |

	   There is no default,	and you	may list a string of characters.

       DontBumpPositionOnEndTags *list of names*
       DontBumpPositionOnStartTags *list of names*
	   Since metatags are typically	separate data fields, the word posi-
	   tion	counter	is automatically bumped	between	metatags (actually,
	   bumped when a start tag is found and	when an	end tag	is found).
	   This	prevents matching a phrase that	spans more than	one metaname.
	   "DontBumpPositionOnEndTags" and "DontBumpPositionOnStartTags" dis-
	   ables this feature for the listed metanames.

	   For example,

	       <person>
		   <first_name>
		       William
		   </first_name>
		   <last_name>
		       Shakespeare
		   </last_name>
		   <updated_date>
		       April 25, 1999
		   </updated_date>
	       </person>

	   In the configuration	file:

	       DontBumpPositionOnEndTags first_name
	       DontBumpPositionOnStartTags last_name

	   This	configuration allows this phrase search

	       -w 'person=("william shakespeare")'

	   but this phrase search will fail

	       -w 'person=("shakespeare	april")'

       Directives for the File Access method only

       Some directives have different uses depending on	the source of the doc-
       uments.	These directives are only valid	when using the File system
       method of indexing.

       IndexOnly *list of file suffixes*
	   This	directive specifies the	allowable file suffixes	(extensions)
	   while indexing.  The	default	is to index all	files specified	in In-
	   dexDir.

	       # Only index .html .htm and .q files
	       IndexOnly .html .htm .q

	   "IndexOnly" checks that the file end	in the characters listed.  It
	   does	not check "extensions".	 "IndexOnly" is	tested right before
	   "FileRules" is processed.

       FollowSymLinks [yes|NO]
	   Put "yes" to	follow symbolic	links in indexing, else	"no".  Default
	   is no.

	       FollowSymLinks no
	       FollowSymLinks yes

	   Note	that when set to "no" extra stat(2) system calls must be made
	   for each file.  For large number of files you may see a small re-
	   duction in indexing time by setting this to "yes".

	   See also the	"-l" switch in SWISH-RUN.

       FileRules [type]	[contains|is|regex] *regular expression*
       FileMatch [type]	[contains|is|regex] *regular expression*
	   FileRules and FileMatch are used to,	respectively, exclude and in-
	   clude files and directories to index.  Since, by default, Swish-e
	   indexes all files and recurses all directories (but see also	"Fol-
	   lowSymLinks") you will typically only use "FileRules" to exclude
	   files or directories.  "FileMatch" is useful	in a few cases,	for
	   example, to override	the behavior of	"IndexOnly".  Some examples
	   are included	below.

	   Except for "FileRules title ...", this feature is only available
	   for file access method (-S fs), which is the	default	indexing mode.
	   Also, any pathname modification with	"ReplaceRules" happens after
	   the check for "FileRules".  (It's unlikely that you would exclude
	   files with "FileRules" based	on text	you added with "Re-
	   placeRules"!)

	   The regular expression is a C regex.h extended regular expression.
	   You may supply more than one	regular	expression per line, or	use
	   separate directives.	 Preceding the regular expression with the
	   word	"not" negates the match.

	   The regular expression is compared against [type] as	described be-
	   low.

	   For historical reasons, you can specify "contains" or "is".	"is"
	   simply forces the regular expression	to match at the	start and end
	   of the string (by internally	prepending "^" and appending "$" to
	   the regular expression).

	   The "regex" option requires delimiter characters:

	       FileRules title regex /^private/i

	   The only advantage of "regex" is if you want	to do case insensitive
	   matches, or simply like your	regular	expressions to look like perl
	   regular expressions.	 You must use matching delimiters; (), {}, and
	   [], are not currently supported for no good reason other than lazi-
	   ness.

	   Use quotes (" or ') around a	pattern	if it contains any white
	   space.  Note	that the backslash character becomes the escape	char-
	   acter within	quotes.

	   For example,	these sets generate the	same regular expressions.

	       FileRules title is hello
	       FileRules title contains	^hello$
	       FileRules title regex /^hello$/

	   These all need quotes due to	the included space character

	       FileRules title is "hello there"
	       FileRules title contains	"^hello	there$"
	       FileRules title regex "!^hello there$!"

	   These show how the backslash	must be	doubled	inside of quotes.
	   Swish-e converts a double-backslash into a single backslash,	and
	   then	passes that single onto	the regular expression compiler.

	       FileRules filename regex	/\.pdf/
	       FileRules filename regex	"/\\.pdf/"

	       FileRules filename regex	!hello\\there!	   # need double for real backslash
	       FileRules filename regex	"!hello\\\\there!" # need double-double	inside of quotes

	   Matching Types

	   The following types of match	strings	my be supplied:

	       FileRules pathname
	       FileRules dirname
	       FileRules filename
	       FileRules directory
	       FileRules title

	       FileMatch pathname
	       FileMatch filename
	       FileMatch dirname
	       FileMatch directory

	   pathname matches the	regular	expression against the current path-
	   name.  The pathname may or may not be absolute depending on what
	   you supplied	to "IndexDir".

	   Example:

	       # Don't index paths that	contain	private	or hidden
	       FileRules pathname contains (private|hidden)

	       # Same thing
	       FileRules pathname regex	/(private|hidden)/

	       # Don't index exe files
	       FileRules pathname contains \.exe$

	   dirname and filename	split the path name by the last	delimiter
	   character into a directory name, and	a file name.  Then these are
	   compared against the	patterns supplied.  Directory names do not
	   have	a trailing slash.  All path names use the forward slash	as a
	   delimiter within Swish-e.

	   Example:

	       # Same as last example -	don't index *.exe files.
	       FileRules filename contains \.exe$

	       # Don't index any file called test.html files
	       FileRules filename contains ^test\.html$

	       # Same thing
	       FileRules filename is test\.html

	       # Don't index any directories that contain "old"	 (/usr/local/myold/docs)
	       FileRules dirname contains old

	       # Don't index any directories that contain the path segment "old" (/usr/local/old/foo)
	       FileRules dirname contains /old/

	       # Index only .htm, .html, plus any all-digit file names
	       IndexOnly .htm .html
	       FileMatch filename contains ^\d+$

	       # Same as previous, but maybe a little slower
	       FileRules filename regex	not !\.(htm|html)$!
	       FileMatch filename contains ^\d+$

	   Swish-e checks these	settings in the	order of "pathname",
	   "dirname", and "filename", and "FileMatch" patterns are checked be-
	   fore	"FileRules", in	general.  This allows you to exclude most
	   files with "FileRules", yet allow in	a few special cases with
	   "FileMatch".	For example:

	       # Exclude all files of .exe, .bin, and .bat
	       FileRules filename contains \.(exe|bin|bat)$
	       # But, let these	two in
	       FileMatch filename is baseball\.bat incoming_mail\.bin

	       # Same, but as a	single pattern
	       FileMatch filename is (baseball\.bat|incoming_mail\.bin)

	   The "directory" type	is somewhat unique. When Swish-e recurses into
	   a directory it will compare all the files in	the directory with the
	   pattern and then decide if that entire directory should or should
	   not be indexed (or recursed).  Note that you	are matching against
	   file	names in a directory --	and some of those names	may be direc-
	   tory	names.

	   A "FileRules	directory" match will cause Swish-e to ignore all
	   files and sub-directories in	the current directory.

	   Warning: A match with "FileMatch directory" says to index every-
	   thing in the	*current* directory and	ignore any FileRules for this
	   directory.

	   Example:

	       # Don't index any directories (and sub directories) that	contain
	       # a file	(or sub-directory) called "index.skip"
	       FileRules directory contains ^index\.skip$

	       # Don't index directories that contain a	.htaccess file.
	       FileRules directory contains ^\.htaccess

	   Note: While processing directories, Swish-e will ignore any files
	   or directories that begin with a dot	(".").	You may	index files or
	   directories that begin with a dot by	specifying their name with
	   "IndexDir" or "-i".

	   "title" checks for a	pattern	match in an HTML title.

	   Example:

	       FileRules title contains	construction example pointers

	       # This example says to ignore case
	       FileRules title regex "/^Internal document/i"

	   Note: "FileRules title" works for any input method (fs, prog, or
	   http) that is parsed	as HTML, and where a title was found in	the
	   document.

	   In case all this seems a bit	confusing, processing a	directory hap-
	   pens	in the following order.

	   First the directory name is checked:

	       FileRules dirname - reject entire directory if matches

	   Next	the directory is scanned and each file name (which might be
	   the name of a sub-directory)	is checked:

	       FileRules directory - reject entire dir if *any*	files match
	       FileMatch directory - accept entire dir if *any*	files match

	   Then, unless	"FileMatch directory" matched, each file is tested
	   with	FileMatch.  A match says to index the file without further
	   testing (i.e.  overrides FileRules and IndexOnly):

	       FileMatch pathname  \
	       FileMatch dirname   - file is accepted if any match
	       FileMatch filename  /

	   otherwise

	       IndexOnly - file	is checked for the correct file	extension

	       FileRules pathname  \
	       FileRules dirname   - file is rejected if any match
	       FileRules filename  /

	   finally, the	file is	indexed.

	   Files (not directories) listed with "IndexDir" or "-i" are pro-
	   cessed in a similar way:

	       FileMatch pathname  \
	       FileMatch dirname   - file is accepted if any match
	       FileMatch filename  /

	   otherwise, the file is rejected if it doesn't have the correct ex-
	   tension or a	FileRules matches.

	       IndexOnly - file	is checked for the correct file	extension

	       FileRules pathname  \
	       FileRules dirname   - file is rejected if any match
	       FileRules filename  /

	   Note:  If things are	not indexing as	you expect, create a directory
	   with	some test files	and use	the "-T	regex" trace option to see how
	   file	names are checked.  Start with very simple tests!

       Directives for the HTTP Access Method Only

       The HTTP	Access method is enabled by the	"-S http" switch when index-
       ing.  It	works by running a Perl	program	called SwishSpider which
       fetches documents from a	web server.

       Only text files (content-type of	"text/*") are indexed with the HTTP
       Access Method.  Other document types (e.g. PDF or MSWord) may be	in-
       dexed as	well.  The SwishSpider will attempt to make use	of the
       SWISH::Filter module (included with the Swish-e distribution) to	con-
       vert documents into a format that Swish-e can index.

       Note: The -S prog method	of spidering (using spider.pl) can be a	re-
       placement for the -S http method.  It offers more configuration options
       and better spidering speed.

       These directives	below are available when using the HTTP	Access Method
       of indexing.

       MaxDepth	*integer*
	   MaxDepth defines how	many links the spider should follow before
	   stopping.  A	value of 0 configures the spider to traverse all
	   links.  The default is MaxDepth 0.

	       MaxDepth	5

	   Note: The default was changed from 5	to 0 in	release	2.4.0

       Delay *seconds*
	   The number of seconds to wait between issuing requests to a server.
	   This	setting	allows for more	friendly spidering of remote sites.
	   The default is 5 seconds.

	       Delay 1

	   Note: The default was changed from 60 to 5 seconds in release 2.4.0

       TmpDir *path*
	   The location	of a writable temp directory on	your system.  The HTTP
	   access method tells the Perl	helper to place	its files in this lo-
	   cation, and the "-e"	switch causes Swish-e to use this directory
	   while indexing.  There is no	default.

	       TmpDir /tmp/swish

	   If this directory does not exist or is not writable Swish-e will
	   fail	with an	error during indexing.

	   Note, the environment variables of "TMPDIR",	"TMP", and "TEMP" (in
	   that	order) will override this setting.

       SpiderDirectory *path*
	   The location	of the Perl helper script called swishspider.  If you
	   use a relative directory, it	is relative to your directory when you
	   run Swish-e,	not to the directory that Swish-e is in.  The default
	   is the location swishspider was installed.  Normally	this does not
	   need	to be set.

	       SpiderDirectory /usr/local/swish

       EquivalentServer	*server	alias*
	   Often times the same	site may be referred to	by different names.  A
	   common example is that often	http://www.some-server.com and
	   http://some-server.com are the same.	 Each line should have a list
	   of all the method/names that	should be considered equivalent.  Mul-
	   tiple EquivalentServer directives may be used.  Each	directive de-
	   fines its own set of	equivalent servers.

	       EquivalentServer	http://library.berkeley.edu http://www.lib.berkeley.edu
	       EquivalentServer	http://sunsite.berkeley.edu:2000 http://sunsite.berkeley.edu

       Directives for the prog Access Method Only

       This section details the	directives that	are only available for the
       "prog" document source feature of Swish-e.  The "prog" access method
       runs an external	program	that "feeds" documents to Swish-e.  This al-
       lows indexing and filtering of documents	from any source.

       See prog	- general purpose access method	in the SWISH-RUN man page for
       more information.

       A number	of example programs for	use with the "prog" access method are
       provided	in the prog-bin	directory.  Please see those example if	you
       have questions about implementing a "prog" input	program.

       SwishProgParameters *list of parameters*
	   This	is a list of parameters	that will be sent to the external pro-
	   gram	when running with the "prog" document source method.

	       SwishProgParameters /path/to/config hello there
	       IndexDir	/path/to/program.pl

	   Then	running:

	       swish-e -c config -S prog

	   Swish-e will	execute	"/path/to/program.pl" and pass "/path/to/con-
	   fig hello there" as three command line arguments to the program.
	   This	directive makes	it easy	to pass	settings from the Swish-e con-
	   figuration file to the external program.

	   For example,	the "spider.pl"	program	(included in the "prog-bin"
	   directory) uses the "SwishProgParameters" to	specify	what file to
	   read	for configuration information.

	       SwishProgParameters spider.config
	       IndexDir	./spider.pl

	   The "spider.pl" program also	has a default action so	you can	avoid
	   using a configuration file:

	       SwishProgParameters default http://www.swishe.org/ http://some.other.site/
	       IndexDir	./spider.pl

	   And the spider program will use default settings for	spidering
	   those sites.

	   Swish-e can read documents from standard input, so another way to
	   run an external program with	parameters is:

	       ./spider.pl spider.conf | ./swish-e -S prog -i stdin

       Notes when using	MS Windows

       You should use unix style path separators to specify your external pro-
       gram.  Swish will convert forward slashes to backslashes	before calling
       the external program.  This is only true	for the	program	name specified
       with "IndexDir" or the "-i" command line	option.

       In addition, Swish-e will make sure the program specified actually ex-
       ists, which means you need to use the full name of the program.

       For example, to run the perl spider program spider.pl you would need a
       Swish-e configuration file such as:

	   IndexDir e:/perl/bin/perl.exe
	   SwishProgParameters prog-bin/spider.pl default http://swish-e.org

       and run indexing	with the command:

	   swish-e -c swish.cfg	-S prog	-v 9

       The "IndexDir" command tells Swish-e the	name of	the program to run.
       Under unix you can just specify the name	of the script, since unix will
       figure out the program from the first line of the script.

       The "SwishProgParameters" are the parameters passed to the program
       specified by "IndexDir" (perl.exe in this case).	 The first parameter
       is the perl script to run (prog-bin/spider.pl).	Perl passes the	rest
       of the parameters directly to the perl script.  The second parameter
       default tells the spider.pl program to use default settings for spider-
       ing (or you could specify a spider config file -- see "perldoc spi-
       der.pl" for details), and lastly, the URL is passed into	the spider
       program.

       Document	Filter Directives

       Internally, Swish-e knows how to	parse only text, HTML, and XML docu-
       ments.  With "filters" you can index other types	of documents.  For ex-
       ample, if all your web pages are	in gzip	format a filter	can uncompress
       these on	the fly	for indexing.

       You may wish to read the	Swish-e	FAQ question on	filtering before con-
       tinuing here.  How Do I filter documents?

       There are two suggested methods for filtering.

       Filtering with SWISH::Filter

       The Swish-e distribution	includes a Perl	module called SWISH::Filter
       and individual filters located in the filters directory.	 This system
       uses plug-in filters to extend the types	of documents that Swish-e can
       index.  The plug-in filters do not actually do the filtering, but
       rather provide a	standard interface for accessing programs that can
       filter or convert documents.  The programs that do the filtering	are
       not part	of the Swish-e distribution; they must be downloaded and in-
       stalled separately.

       The advantage of	this method is that new	filtering methods can be in-
       stalled easily.

       This system is designed to work with the	-S http	and -prog methods, but
       may also	be used	with the "FileFilter" feature and -S fs	indexing
       method.	See $prefix/share/doc/swish-e/examples/filter-bin/swish_fil-
       ter.pl for an example.

       See the filters/README file for more information.

       Filtering with the FileFilter feature

       A filter	is an external program that Swish-e executes while processing
       a document of a given type.  Swish-e will execute the filter program
       for each	file that matches the file suffix (extension) set in the File-
       Filter or FileFilterMatch directives.  FileFilterMatch matches using
       regular expressions and is described below.

       Filters may be used with	any type of input method (i.e. -S fs, -S http,
       or -S prog).  But because

       Swish-e calls the external program passing as default arguments:

       $0  the name of the filter program

       $1  the physical	path name of the file to read.	This may be a tempo-
	   rary	file location if indexing by the http method.

       $2  When	indexing under the file	system this will be the	same as	$1
	   (the	path to	the source file), but when indexing under the http
	   method this will be the URL of the source document.

       Swish-e can also	pass other parameters to the filter program.  These
       parameters can be defined using the FileFilter or FileFilterMatch di-
       rectives.  See Filter Options below.

       The filter program must open the	file, process its contents, and	return
       it to Swish-e by	printing to STDOUT.

       Note that this can add a	significant amount of time to the indexing
       process if your external	program	is a perl or shell script.  If you
       have many files to filter you should consider writing your filter in C
       instead of a shell or perl script, or using the "prog" Access Method
       along with SWISH::Filter.

       FilterDir  *path-to-directory*
	   Deprecated.

	   This	is the path to a directory where the filter programs are
	   stored.  Swish-e looks in this directory to find the	filter speci-
	   fied	in the FileFilter directive.

	   This	directive is not needed	if the filter program can be found in
	   your	system's path.	Even if	your filter is not in your system's
	   path	you can	specify	the full path to the filter in the FileFilter
	   or FileFilterMatch directives.

	   Example:

	       FilterDir /usr/local/swish/filters

       FileFilter   *suffix*   "filter-prog"   ["filter-options"]
	   This	maps file suffix (extension) to	a filter program.  If filter-
	   prog	starts with a directory	delimiter (absolute path), Swish-e
	   doesn't use the FilterDir settings, but uses	the given filter-prog
	   path	directly.

	   On systems that have	a working fork(2) system call the filter pro-
	   gram	is run by forking swish	then executing the filter.  This mean
	   the shell is	not used for running the filter	and no arguments are
	   passed through the shell.

	   On other systems (e.g. Windows) the arguments are double-quoted and
	   popen(3) is used to run the program.	 This does pass	argument
	   though the shell and	may be a security concern depending on the
	   abilities of	the shell.

	   Filter options:

	   Filter options are a	string passed as arguments to the filter-prog.
	   Filter options can contain variables, replaced by Swish-e.  If you
	   omit	filter-options Swish-e will use	default	parameters for the op-
	   tions listed	above.

	       Default:	     %p	%P
	       Which means:  pass   "workfile path" and	"documentfile path" to filter.

	   Variables in	filter options:

	       %%   =  %
	       %P   =  Full document pathname (e.g. URL, or path on filesystem)
	       %p   =  Full pathname to	work file (maybe a tmpfile or the real document	path on	filesystem)
	       %F   =  Filename	stripped from full document pathname
	       %f   =  Filename	stripped from "work" pathname
	       %D   =  Directoryname stripped from full	document pathname
	       %d   =  Directoryname stripped from full	"work" pathname

	   Examples of strings passed:

	       %P =  document pathname:	 http://myserver/path1/mydoc.txt
	       %p =  work pathname:	 /tmp/tmp.1234.mydoc.txt
	       %F =	mydoc.txt
	       %f =	tmp.1234.mydoc.txt
	       %D =	http://myserver/path1
	       %d =	/tmp

	   Notes when using MS Windows

	   Windows uses	double quotes to escape	shell metacharacters, so if
	   you need to use quotes then use single quotes around	the entire op-
	   tion	string.

	       FileFiler .mydoc	mydocfilter.exe	'--title "text with spaces"'

	   You can specify the filter program using forward slashes (unix
	   style).  Swish will convert the slashes to backslashes before run-
	   ning	your program.

	       FileFilter .mydoc     c:/some/path/mydocfilter.exe  '-d "%d" -example -url "%P" "%f"'

	   Examples of filters:

	       FileFilter .doc	     /usr/local/bin/catdoc "-s8859-1 -d8859-1 %p"
	       FileFilter .pdf	     pdftotext	 "%p -"
	       FileFilter .html.gz   gzip  "-c %p"
	       FileFilter .mydoc     "/some/path/mydocfilter"  "-d %d -example -url %P %f"

	   The above examples are running a binary filter program.  For	more
	   complicated filtering needs you may use a scripting language	such
	   as Perl or a	shell script.  Here's some examples of calling a shell
	   and perl script:

	       FileFilter .pdf	     pdf2html.sh
	       FileFilter .ps	     ghostscript-filter.pl

	   Using a scripting language (or any language that has	a large
	   startup cost) can greatly increase the indexing time.  For small
	   indexing jobs, this may not be an issue, but	for large collections
	   of files that require processing by a scripting language, you may
	   be better off using the "-S prog" access method where the script
	   will	only be	compiled once, instead of for each document.

	   Filters are probably	easier to write	than a "-S prog" program.
	   Which you decide to use depends on your requirements.  Examples of
	   filter scripts can be found in the filter-bin directory, and	exam-
	   ples	of "-S prog" programs can be found in the prog-bin directory.

       FileFilterMatch	 *filter-prog*	 *filter-options*  *regex* [*regex*
       ...]
	   This	is similar to "FileMatch" except uses regular expressions to
	   match against the file name.	 *filter-prog* is the path to the pro-
	   gram.  Unlike "FileFilter" this does	not use	the "FilterDir"	op-
	   tion.  Also unlike "FileFilter" you must specify the	*filter-op-
	   tions*.

	   Examples:

	       FileFilterMatch ./pdftotext "%p -" /\.pdf$/

	   Note	that will also match a file called ".pdf", so you may want to
	   use something that requires a filename that has more	than just an
	   extension.  For example:

	       FileFilterMatch ./pdftotext "%p -" /.\.pdf$/

	   To specify more than	one extension:

	       FileFilterMatch ./check_title.pl	"%p" /\.html$/	/\.htm$/

	   Or a	few ways to do the same	thing:

	       FileFilterMatch ./check_title.pl	%p /\.(html|html)$/
	       FileFilterMatch ./check_title.pl	%p /\.html?$/

	   And to ignore case:

	       FileFilterMatch ./check_title.pl	%p /\.html?$/i

	   You may also	precede	an expression with "not" to negate regular ex-
	   pression that follow.  For example, to match	files that do not have
	   an extension:

	       FileFilterMatch ./convert "%p %P" not /\..+$/

Document Info
       $Id: SWISH-CONFIG.pod 1846 2006-10-20 20:18:30Z whmoseley $

       .

2.4.7				  2009-04-04		       SWISH-CONFIG(1)

NAME | OVERVIEW | CONFIGURATION FILE | Document Info

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=SWISH-CONFIG&sektion=1&manpath=FreeBSD+12.1-RELEASE+and+Ports>

home | help