Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
NNMASTER(8)		    System Manager's Manual		   NNMASTER(8)

NAME
       nnmaster	- nn database manager

SYNOPSIS
       nnmaster	-I [lmit]
       nnmaster	-w
       nnmaster	-v
       nnmaster	-l [ "lock message" ]
       nnmaster	[ options ] [ groups ] nnmaster	-F [ options ] [ groups	]

DESCRIPTION
       nnmaster	 is the	daemon which is	responsible for	building and maintain-
       ing the database	used by	the nn(1) news reader.

       Normally, nnmaster is started when the system enters  multi-user	 mode,
       and  runs  until	system shutdown.  To facilitate	this, you should place
       the following call in /etc/rc (or similar) to invoke the	nnmaster  dae-
       mon:
	    $master/nnmaster -l	-r -C
       where  $master  is the MASTER_DIRECTORY defined during configuration of
       nn.

       When nnmaster is	started	as specified above, it will first  unlock  the
       database	 in  case  it  was locked (-l),	perform	a thorough consistency
       check on	the database (-C).

       Then, every 10 minutes (-r), it will look at the	time-stamp of the news
       active  file to see whether new articles	have arrived on	the system (or
       whether articles	have been expired). (See -U)

       If the active file has been modified, nnmaster will collect the	header
       information  from the new articles and enter them into the database (or
       remove the headers of the expired articles from the database).

       If it detects that some articles	have been expired, it  will  automati-
       cally  remove  the  header information of the expired articles from the
       database.

ARTICLE	COLLECTION OPTIONS
       Normally, nnmaster will collect all available news  groups  defined  in
       the  news  active  file.	 The set of collected groups can be controlled
       via the argument	line.  Groups can be either included or	excluded:
       A group name, e.g. comp,	will cause the group and all its subgroups  to
       be  collected.	Individual  groups, e.g. news.software.nn, can also be
       specified
       A group name preceded by	an exclamation mark, e.g. !talk.politics, will
       cause the group and all its subgroups to	be ignored.
       An  empty argument, i.e.	"", will cause all groups that are not ignored
       to be collected.	 For example, to collect everything but	rec and	 misc,
       use the following command:
	    nnmaster -r	!rec !misc ""
       If  the	empty  argument	 had been omitted, nothing would be collected,
       since the presence of any groups	arguments causes  nnmaster  to	ignore
       all groups which	are not	explicitly mentioned.
       Example	1:  The	 following commands can	be executed by cron to collect
       different sets of groups	at different intervals or under	different con-
       ditions:
	    nnmaster -B	-O14 rec misc sci -LBO -u
	    nnmaster !rec !misc	!sci ""	-u
       Example	2: The group arguments are used	in the given sequence, e.g. to
       leave out comp.sys, but still collect comp.sys.ibm.pc, use the command:
	    nnmaster -r	comp.sys.ibm.pc	!comp.sys ""
       The use of the -u option	in the first example is	essential, since  each
       of  the commands	will update the	active file time stamp which will pre-
       vent the	other command from detecting new articles that have arrived.
       Using this method to keep specific groups out of	the database  must  be
       used  with  great caution; just a single	invocation of nnmaster without
       any arguments will collect all the otherwise ignored groups!

COLLECTION OF ARTICLES
       The following options control how nnmaster performs the	collection  of
       new articles.

       -r [ min	]
	      Daemon  mode.   The  nnmaster  will put itself in	the background
	      (unless -f is also specified), and will checks  for  arrival  of
	      new  articles and	expired	articles every min minutes (and	update
	      the database accordingly).  If min is omitted, the default is to
	      check every 10 minutes.
	      Without  the  -r option, the nnmaster will just perform a	single
	      collection of new	articles (if any) and then exit.  This can  be
	      used  to	have the nnmaster started by cron(8) at	regular	inter-
	      vals instead of having it	 as  a	daemon	which  sleeps  between
	      checking	for  new articles.  Since the nnmaster is a bit	expen-
	      sive to start up (it has to read a few files), it	is up  to  you
	      to decide	which mode is best on your system.  (I have also heard
	      that it works to call nnmaster without -r	from inews(1).	I can-
	      not recommend this unless	you receive batched news; invoking nn-
	      master for every received	article	sounds too expensive to	me.)

       -h [ sec	]
	      Hold collection of new articles until next scan if new  articles
	      have  arrived withing the	last sec [60] seconds.	This is	useful
	      to prevent nnmaster from competing for disk i/o  with  an	 rnews
	      process  which  is unbatching incoming news, or a	running	expire
	      process.	It will	have the side effect of	limiting the number of
	      C: entries in the	log, since collection of incoming batches will
	      typically	be done	in larger thunks.

       -f     Run nnmaster in foreground in daemon mode	(see -r).   Useful  if
	      nnmaster	is  invoked  from  inittab.  (Notice that if you use a
	      respawn entry in inittab,	you will not be	able to	stop  nnmaster
	      using  the  -k option, since init	will immediately start another
	      master.)

       -C     Perform a	consistency check on the database on start-up, and re-
	      build  corrupted	database  files.   This	operation can be quite
	      time-consuming since it reads through all	the database files.

       -b     Normally,	articles without a proper news header (no  Newsgroups:
	      line)  are ignored.  Specifying the -b option causes these `bad'
	      articles to be included in the database (normally	with no	sender
	      or subject).

       -B     Remove  `bad'  articles.	 Sometimes,  articles without a	header
	      ends up in the news spool	directory.  These articles have	no ar-
	      ticle  id,  and  therefore,  they	 will  never be	expired	by ex-
	      pire(8).	This option will allow the nnmaster to silently	remove
	      these articles (a	`B' entry will occur in	the log	file).

       -O days
	      Ignore  articles	which are older	than the given number of days.
	      This may help keep old 'stray' articles out of the database.  If
	      the  -B  options is also specified, the old articles will	be re-
	      moved from the news spool	directories.  Old ignored  or  removed
	      articles	will  be  reported  with an `O'	entry in the log file.
	      This option can be disable for individual	groups by the  O  flag
	      in the GROUPS file (see below).

       -R N   Specifies	 how  the auto-recollect operation is performed	on the
	      groups having this option	set in the GROUPS  file	 (see  below).
	      Four methods are available (default is method 1):
	      1:  Run expire on	the group when new articles arrive.
	      2:  Run expire on	the group on every scan.
	      3:  Recollect all	articles when new articles arrive.
	      4:  Recollect all	articles on every scan.

       -M mode
	      Normally,	 nnmaster will send a message via mail to the news ad-
	      ministrator (OWNER) when an error	or potential problems (primar-
	      ily  nntp	 related) occur.  This can be restricted to only fatal
	      errors (nnmaster terminated) if mode is  1,  and	disabled  com-
	      pletely if mode is 0.

       -Q     Normally,	nnmaster will print a message on the system console or
	      in the syslog if a fatal error happens.  This option  will  pre-
	      vent this, so only a type	'E' entry is written to	the Log	file.

DATABASE EXPIRATION
       Since  articles	does not stay forever in the news system, the database
       must be cleaned up regularly to remove the information stored  for  ex-
       pired articles.	Expiration of the database is normally scheduled using
       the nnadmin(1M) command executed	by cron	at a suitable time when	expire
       on  the news articles has completed.  The following command will	send a
       message to the nnmaster and cause it to initiate	 expire	 on  all  news
       groups:
	    nnadmin =EYW

       Selective expiration of individual groups can be	done from nnadmin (in-
       teractive mode).	 It can	also be	done by	invoking nnmaster with the  -F
       option.	 For  example,	the  following	command	will run expire	on all
       groups except the `rec' groups:
	    nnmaster -F	-k !rec	""
       The -k option is	required to terminate  the  currently  running	master
       since  two daemons cannot be running at the same	time.  Thus to run ex-
       pire (on	all groups) in this way	from cron, the following commands must
       be used:	     nnmaster -Fk "" ; nnmaster	-r ...

       It  is  also  possible to have nnmaster detect expiration automatically
       (see -e).  This requires	that the min field in the active file  is  up-
       dated  by  the  news expire (this is not	the default case when Cnews is
       used).  However,	this is	not always a safe indication since  the	 first
       article	may  not have been expired, while a lot	of other articles have
       been deleted.

       There are several strategies available  in  the	nnmaster  to  do  this
       clean-up, each having their strengths and weaknesses.

       Method  1  (default):  Rebuilds the database from the existing database
       information by comparing	the current database contents  with  the  con-
       tents  of the news group	directories, eliminating entries whose file no
       longer exists.  This method is guaranteed to eliminate all expired  ar-
       ticles from the database, and it	is reasonably fast because it only has
       to read the directories,	not each article file.
	 If news is accessed remotely via nntp,	the list of existing  articles
       cannot efficiently be fetched by	reading	a local	directory.  Instead an
       XHDR request is sent to the nntp	server to get a	list of	articles.

       Method 2:  Eliminates only the expired articles before the first	 arti-
       cle in the group.  This is very fast since only the active file and the
       database	itself is accessed, but	it will	obviously leave	 some  expired
       articles	 in  the database.  This method	requires that the min field in
       the active file is updated by expire.

       Method 3:  Discard the current database information and recollects  all
       articles.   This	 is obviously very time	consuming, and it is therefore
       not recommended,	especially not with nntp.

       The options related to database expiration are:

       -E N   Select expire method N.  (If N is	omitted, the default method is
	      used).

       -e [N] Automatically run	expire in the database on groups where the min
	      number in	the active file	has increased by N (1 if omitted)  ar-
	      ticles.  This is disabled	by default (since the min field	is of-
	      ten unreliable).

       -F     Run expire once and exit.	 If a list of groups is	 specified  on
	      the  command line, the matched groups (see the rules above) will
	      be marked	for expiration.	 If no groups are specified, only  the
	      groups  already  scheduled  for  expire will be expired.	Conse-
	      quently, to expire all groups, a blank argument "" (matching all
	      groups) must be specified.

DATABASE LOCKING
       The  database can be locked centrally, which will normally disallow all
       access to the database, and even	block nnmaster	from  being  (acciden-
       tally) started.	When a lock is set on the database, all	currently run-
       ning clients will terminate the next time they try to access the	 data-
       base.   Setting	a  lock	on the database	can thus also be used to force
       running clients to terminate.

       The following options set and clear locks on the	database:

       -l message
	      Locks the	database with the given	message.  The message will  be
	      displayed	every time a program tries to access the database.

       -l     Unlock the database if it	was locked.

       -i     Ignore  a	 possible lock and continue.  This can be used to have
	      nnmaster operate on a database which is blocked from normal user
	      access.

       Since  only one nnmaster	can operate on the database at any one time, a
       running nnmaster	daemon must be stopped before a	lock can be set	on the
       database.   If  neither	-f  nor	-r is specified	with the -l option (in
       both forms), nnmaster will terminate  after  setting  or	 clearing  the
       lock.

DATABASE INITIALIZATION
       The  following  options	are  used to initialize	and update the central
       database	files:

       -I [limit]
	      Initialize database.  This option	will erase an  existing	 data-
	      base,  and  create  an empty database containing entries for the
	      currently	known groups.  nnmaster	will offer you to use  an  ex-
	      isting GROUPS file when initializing the database.
	      The  optional  limit can be used to put a	limit on the number of
	      articles that will be collected in each group during  the	 first
	      collection  of  articles	following the database initialization.
	      This is useful on	systems	where the 'min'	field  in  the	active
	      file  is	unreliable  or not maintained (Cnews doesn't) to limit
	      the work done to do the initial collection  of  news  after  the
	      initialization  of  the  database.  If news is accessed remotely
	      from an NNTP server, this	is even	more important!	 If  limit  is
	      omitted,	or is zero, nnmaster will trust	the min	field and col-
	      lect all articles	in the range min..last.

       -G     Reread the GROUPS	file.  This option is used to instruct	nnmas-
	      ter  to parse the	GROUPS file after it has been edited.  See the
	      section on the GROUPS file below.

MISCELLANEOUS OPTIONS
       The following options controls various details of the nnmaster's	behav-
       iour:

       -D [ N ]
	      Run  nnmaster  in	"debug mode".  If N is omitted,	or equals 1 or
	      3, this will produce a compact but still very informative	 trace
	      of  the  collection  or expire process directly on the terminal.
	      This is most useful when doing the first collection of  articles
	      after  initializing  the	database  with	-I.  If	N is 2 or 3, a
	      trace of the NNTP	traffic	is written to a	file  nnmaster.log  in
	      the TMP directory. This option disables -r.

       -H     Identifies  the  host  which nnmaster is running on as the nntp-
	      server for its clients, i.e. that	it can access the  news	 spool
	      directory	 locally  without using	NNTP.  Normally, nnmaster will
	      detect this by itself by comparing the host name to the contents
	      of  the nntp_server file,	so this	option should really be	super-
	      fluous.

       -y retries
	      In some networked	environment, opening an	article	 (shared  from
	      another  machine via NFS)	may fail for no	obvious	reason.	 Using
	      this option, it is possible to cause nnmaster to perform retries
	      attempts	to  open an article before marking the article as non-
	      existing in the database.

       -L types
	      Exclude the specified entry types	from the log  file.   This  is
	      normally	used  to exclude the 'C'ollecting and e'X'pire entries
	      (-LCXO).

       -t     Trace the	collection process.  This will place a lot of informa-
	      tion into	the log	file (T: entries).

       -u     Normally,	 nnmaster  will	just compare the time-stamp on the ac-
	      tive file	with a time-stamp saved	in the database	to see if  new
	      articles	have  arrived.	 The  -u option	forces the nnmaster to
	      read the active file on start-up to see if new articles have ar-
	      rived.

       -U     Some  SVR4  systems  (and	 maybe SunOS) have a useful "feature".
	      Writing files with mmap()	may not	update the last-changed	 time-
	      stamp on the file.  Since	INN uses mmap()	for writing the	active
	      file, this becomes a problem for nnmaster.  The -U option	causes
	      nnmaster	to  unconditionally read the active file each time the
	      repeat delay (-r)	time expires.

       -v     Print the	release	and version identification for	nnmaster,  and
	      exit.

       -w     Wakeup  the real nnmaster.  Send a signal	to the nnmaster	daemon
	      to have it check for new articles	immediately.

       -k     Kill the already running nnmaster	daemon before proceeding  with
	      the operation specified by the other options (or terminate if no
	      other operation is requested).

THE GROUPS FILE
       The primary purpose of the GROUPS file is to store  the	names  of  the
       news  groups represented	in the database.  Each line in the file	corre-
       sponds to an entry in the (binary) MASTER file, and the sequence	of the
       lines  in  the GROUPS file must never be	changed	unless the database is
       reinitialized afterwards.

       However,	the contents of	the lines in the GROUPS	file can be edited  to
       control how the nnmaster	should handle each individual group.

       The format of each line is:
	    news.group.name [ timestamp	] [ flags ]

       The  news  group	name is	the name of the	group, and must	not be changed
       in any way.  If the group is no longer in the  news  active  file,  and
       consequently the	group does no longer exist, group name can be replaced
       by a `@'	character which	will instruct nnmaster to  ignore  this	 entry
       without having to rebuild the entire database.

       The optional time stamp indicates when the line was added to the	GROUPS
       file and	is used	by nn to detect	new groups.  When the GROUPS  file  is
       built initially from the	active file, the time stamps are omitted which
       simply means that they are "old".

       One or more of the following flags can be added to the GROUPS  line  to
       control nnmaster's handling of the group:

       D      Causes  nnmaster	to treat all articles in the group as digests,
	      even when	they don't  initially  look  like  digests.   Articles
	      which  are  found	not to be digests after	all, are still not di-
	      gested.

       N      Instructs	nnmaster to never digest any articles in the group.

       O      Disables the -O option for this group, i.e. all  existing	 arti-
	      cles  will be included in	the database (and they will not	be re-
	      moved if the -B option is	specified).  This flag should  be  set
	      on  groups  which	 you  never expire, or have a very long	expire
	      time!

       R      Causes nnmaster to recollect all available articles in the group
	      whenever	a  new article is received.  This is said to be	useful
	      is some high-traffic clarinet groups with	many  cancelled	 arti-
	      cles.

       >file  Instructs	 nnmaster  to append all new articles to the specified
	      file.  This makes	it possible to keep specific groups out	of the
	      way of expire.  The archive file can be access directly from the
	      nn client	using the goto-group command.  The file	name must be a
	      full path	name to	a file in an existing, writeable directory.

       @      Instructs	 nnmaster  to  completely  ignore this group - this is
	      equivalent to setting the	group name to `@' as described above.

       ! or X Causes nnmaster to ignore	the group and not collect the  group's
	      articles in the database.

       Comments	 (starting  with  `#'  and  continuing to the end of line) and
       empty lines are allow in	the GROUPS file, but  it  is  strongly	recom-
       mended to keep the changes to the GROUPS	file as	small as possible, be-
       cause of	the implicit correspondence with the binary MASTER file.

       It is not recommended to	edit the GROUPS	file while nnmaster is running
       because	it  may	 add new groups	to the file.  After editing the	GROUPS
       file, the command
	    nnmaster -G
       must be run before restarting the nnmaster to parse and verify the  new
       GROUPS file.

NNTP SUPPORT
       The  nnmaster  can access the news articles from	a local	news spool di-
       rectory as well as from an NNTP server.	When compiled  with  NNTP  en-
       abled,  nnmaster	 will compare the name of the NNTP server and the name
       of the local host; if they are identical, nnmaster will bypass NNTP and
       access the articles directly.

       When  it	has to access the news articles	via NNTP, it cannot time-stamp
       the active file,	so instead it transfers	the entire  active  file  from
       the  NNTP  server  and compares it with a local copy of the last	active
       file fetched from the NNTP server.  This	is not very expensive in terms
       of  cpu-cycles,	disk-load,  or	net-bandwidth, but to stay on friendly
       terms with the NNTP server administrator, you should probably  not  use
       shorter update intervals	than the standard 10 minutes.

       Setting	a  much	higher update interval than the	standard 10 minutes is
       not really recommended either, since an update normally implies	fetch-
       ing  a  burst of	news articles from the NNTP server, so setting the in-
       terval too long may imply that the load on the NNTP server will be more
       un-even.

       In  expire method 1, the	use of XHDR just to get	a list of existing ar-
       ticles in a group is definitely a waste of resources on the nntp	server
       (but  still lower than using method 3).	Before using the XHDR request,
       nnmaster	will send a non-standard  "LISTGROUP"  request;	 if  the  nntp
       server  supports	 this request, it should return	an OK_HEAD status fol-
       lowed by	an (unordered) list of article numbers (one per	 line)	termi-
       nated  by a `.' line.  The nntp servers supporting this request will be
       much less loaded	during expire.

       The -O option does not work with	NNTP.  The -B option  will  only  work
       with NNTP if the	nnmaster is running on the NNTP	server.

FILES
       The  $db,  $master,  and	 $news	names  used below are synonyms for the
       DB_DIRECTORY, MASTER_DIRECTORY, and NEWS_LIB_DIRECTORY  defined	during
       configuration.

       $db/MASTER	   Database master index
       $db/GROUPS	   News	group names and	flags in MASTER	file order
       $db/DATA/nnn.[dx]   Database files for group number nnn
       .../.nn[dx]	   Database files if located in	the group directories
       $master/GATE	   Message channel from	nnadmin	to nnmaster
       $master/MPID	   The process id of the nnmaster daemon.
       $Log		   The	log file (the location is configuration	depen-
       dent)
       $news/active	   Existing articles and groups
       /usr/lib/nntp_serverContains the	name of	the NNTP server.

       The MASTER file contains	a record for each news group, occurring	in the
       same sequence as	the group names	in the GROUPS file.  The sequence also
       defines the group numbers used to identify the files in the  database's
       DATA directory.

       The  GATE  file	will be	created	by nnadmin when	needed,	and removed by
       nnmaster	when it	has read it.  Therefore, to send a message to the  nn-
       master requires that you	are allowed to write in	the $master directory.

       The contents of the Log file are	described in the nnadmin manual.

SEE ALSO
       nn(1), nncheck(1), nngrep(1), nntidy(1)
       nnadmin(1M), nnspew(8), nnusage(1M)

AUTHOR
       Kim F. Storm, Texas Instruments A/S, Denmark
       E-mail: storm@texas.dk

4th Berkeley Distribution	  Release 6.6			   NNMASTER(8)

NAME | SYNOPSIS | DESCRIPTION | ARTICLE COLLECTION OPTIONS | COLLECTION OF ARTICLES | DATABASE EXPIRATION | DATABASE LOCKING | DATABASE INITIALIZATION | MISCELLANEOUS OPTIONS | THE GROUPS FILE | NNTP SUPPORT | FILES | SEE ALSO | AUTHOR

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=nnmaster&sektion=8&manpath=FreeBSD+12.1-RELEASE+and+Ports>

home | help