Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
NQ(1)			FreeBSD	General	Commands Manual			 NQ(1)

     nq	-- job queue utility

     nq	[-c] [-q] command line ...
     nq	-t job id ...
     nq	-w job id ...

     The nq utility provides a very lightweight	queuing	system without requir-
     ing setup,	maintenance, supervision or any	long-running processes.

     Job order is enforced by a	timestamp nq gets immediately when started.
     Synchronization happens on	file-system level.  Timer resolution is	mil-
     liseconds.	 No sub-second file system time	stamps are required.  Polling
     is	not used.  Exclusive execution is maintained strictly.

     You enqueue(!) new	jobs into the queue by running

	   nq command line ...

     The job id	(a file	name relative to NQDIR,	which defaults to the current
     directory)	is output (unless suppressed using -q) and nq detaches from
     the terminal immediately, running the job in the background.  Standard
     output and	standard error are redirected into the job id file.  fq(1) can
     be	used to	conveniently watch the log files.

     The options are as	follows:

     -c	     Clean up job id file when process exited with status 0.

     -q	     Suppress output of	the job	id after spawning new job.

     -t	     Enter test	mode: exit with	status 0 when all of the listed	job
	     ids are already done, else	with status 1.

     -w	     Enter waiting mode: wait in the foreground	until all listed job
	     ids are done.

     NQDIR   Directory where lock files/job output resides.  Each NQDIR	can be
	     considered	a separate queue.  The current working directory is
	     used when NQDIR is	unset.	NQDIR is created if needed.

     NQJOBID The job id	of the currently running job, exposed to the job it-

     nq	owns all files in NQDIR	(respectively .) which start with "," or ".,".
     These files are created according to the following	scheme:


     The nq utility exits 0 on success,	and >0 if an error occurs; unless test
     mode is used, in which case exit status 1 means there is a	job running.

     On	fatal errors, exit codes 111 and 222 are used.

     Build make(1) targets clean, depends, all,	without	occupying the termi-

	   % nq	make clean
	   % nq	make depends
	   % nq	make all
	   % fq
	   ... look at output, can interrupt with C-c any time
	   without stopping the	build ...

     Simple download queue, accessible from multiple terminals:

	   % alias qget='NQDIR=/tmp/downloads nq wget'
	   % alias qwait='NQDIR=/tmp/downloads fq -q'
	   window1% qget http://mymirror/big1.iso
	   window2% qget http://mymirror/big2.iso
	   window3% qget http://mymirror/big3.iso
	   % qwait
	   ... wait for	all downloads to finish	...

     As	nohup(1) replacement (The benchmark will run in	background, every run
     gets a different output file, and the command line	you ran	is logged

	   % ssh remote
	   remote% nq ./run-benchmark
	   remote% ^D
	   % ssh remote
	   remote% fq
	   ... see output, fq exits when job finished ...

     The "file extension" of the log file is actually the PID of the job.  nq
     runs all jobs in a	separate process group,	so you can kill	an entire job
     process tree at once using	kill(1)	with a negative	PID.  Before the job
     is	started, it is the PID of nq, so you can cancel	a queued job by
     killing it	as well.

     Thanks to the initial exec	line in	the log	files, you can resubmit	a job
     by	executing it as	a shell	command	file, i.e. running

	   sh job id

     Creating nq wrappers setting NQDIR	to provide different queues for	dif-
     ferent purposes is	encouraged.

     Enforcing job order works like this:
     -	 every job has an output file locked using flock(2) and	named accord-
	 ing to	FILES.
     -	 every job starts only after all earlier flocked files are unlocked.
     -	 the lock is released by the kernel after the job terminates.

     nq	will only work correctly when:

     -	 NQDIR (respectively .)	is writable.

     -	 flock(2) works	correctly in NQDIR (respectively .).

     -	 gettimeofday(2) behaves monotonic (using CLOCK_MONOTONIC would	create
	 confusing file	names after reboot).

     -	 No other programs put files matching ,* into NQDIR (respectively .).

     fq(1), tq(1).

     Alternatives to the nq system include batch(1), qsub(1), schedule(1),
     srun(1), and ts(1).

     Leah Neukirchen <>

     All reliable queue	status information is in main memory only, which makes
     restarting	a job queue after a reboot difficult.

     nq	is in the public domain.

     To	the extent possible under law, the creator of this work	has waived all
     copyright and related or neighboring rights to this work.

FreeBSD	13.0			August 25, 2015			  FreeBSD 13.0


Want to link to this manual page? Use this URL:

home | help