Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
xjobs(1)			 User Commands			      xjobs(1)

       xjobs - construct command line and execute jobs in parallel

       xjobs [options] [utility	[argument ...]]

       xjobs  reads  job descriptions line by line and executes	them in	paral-
       lel. It limits the number of parallel executing	jobs  and  starts  new
       jobs when jobs finish.  Therefore, it combines the arguments from every
       input line with the utility and arguments given on the command line. If
       no utility is given as an argument to xjobs, then the first argument on
       every job line will be  used  as	 utility.  To  execute	utility	 xjobs
       searches	 the  directories  given  in the PATH environment variable and
       uses the	first file found in these directories.

       xjobs is	most useful on multiprocessor machines when one	needs to  exe-
       cute several time consuming commands that could possibly	be run in par-
       allel. With xjobs this can be achieved easily, and it  is  possible  to
       limit  the  load	of the machine to a useful value.  It works similar to
       xargs, but starts several processes simultaneously and gives  only  one
       line of arguments to each utility call.

       By  using  I/O redirectors the standard input, output, and error	stream
       of executed jobs	can be redirected. Use < to redirect standard input, >
       to  redirect  standard output, >! to redirect standard output and over-
       write an	existing file, >> to append standard  output  to  an  existing
       file,  >& to redirect both standard output and standard error output to
       the same	file, and >>& to append	both standard output and standard  er-
       ror output to the same file.

       If  passed on the command line, these operators specify the default I/O
       redirection that	can be overwritten by specifying another redirector to
       a specific job on its argument line.  After all these operators a file-
       name is expected. See EXAMPLES below for	an example.  If	you need  more
       advanced	 shell	features  than	the redirection	operators supported by
       xjobs, then use as utility a shell of your preference.

       Every job line can be preceeded by a "cd	directory;" command that tells
       xjobs in	which directory	the job	shall be executed. For every line this
       can only	be used	once. For more complex scripting, please pass the line
       to execute to a shell of	your choice.

       xjobs  constructs  the arguments	of the jobs to execute from each input
       line. Each input	line will create a seperate job, whereas newline char-
       acter are handled as regular whitespace by xargs. To be able to include
       whitespace charakters in	arguments, either preceed them	with  a	 back-
       slash  or quote them with single	or doublequote charakters. A backslash
       charakter preceeding a newline will make	xjobs ignore the newline char-
       acter,  thus  giving you	the ability to pass arguments for a single job
       across multiple lines. To include quotation marks in quoted  arguments,
       preceed	them with a backslash.	Lines passed to	xjobs beginning	with a
       # charakter are interpreted as comments.

       Finally,	xjobs also includes a mechanism	for serializing	the execution.
       Like  this  it is possible to parallelize independent jobs and sequence
       jobs that have a	dependency. This can be	achieved by inserting  a  line
       that  only consists of two percentage charakters	in sequence (%%).  All
       jobs before this	sequence point are executed at the requested number of
       jobs  in	 parallel. When	hitting	the sequence point xjobs waits for all
       processes to finish and then continues starting jobs  that  follow  the
       sequence	point.

       When  passing a named pipe (i.e.	a file name created by mkfifo) via op-
       tion -s as an input, xjobs will close and reopen	the fifo when reaching
       end-of-file.  Like  this	 it  is	 possible to setup an xjobs server and
       sending jobs to this server from	muliple	programs. See section EXAMPLES
       below for an example.

       -j <jobs>
	      Sets  the	 maximum  number of jobs that are started in parallel.
	      The default value	is to limit the	number executing jobs is equal
	      to  the number of	online processors in the system. If the	number
	      passed as	<jobs> is followed by an 'x'  charakter	 (e.g.	2.5x),
	      the value	is multiplied with the number of online	processors be-
	      fore setting the job limit. I.e.	having a machine with 4	online
	      processors  and  passing	2.5x  as an argument to	option -j will
	      yield a joblimit of 10 jobs.

       -s <script>
	      Use file script instead of the standard input to	read  the  job

       -n     Redirect	standard  output and standard error output of executed
	      jobs to /dev/null.

       -l <num>
	      Combine the arguments of <num> input lines for a single job.

       -p     Start jobs interactively,	prompting the user.

       -q <num>
	      Limits the number	of queued jobs to num elements.	Normally xjobs
	      reads  in	jobs from standard input or the	give script and	queues
	      them if they cannot be started at	once. With this	option,	 xjobs
	      will  stop  reading  as  soon as num jobs	are queued and restart
	      reading when a new job has been started. Like this  xjobs	 allo-
	      cates  less  memory. Use this option, if you pass	huge number of
	      jobs to xjobs, to	limit memory consumption. It can also increase
	      performance  of xjobs, but be sure that jobs get fed fast	enough
	      to xjobs.

       -1     Pass one argument	per job, which is expected to be terminated by
	      a	new-line character. No argument	parsing	is performed. That way
	      it is more easy to process  jobs	where  arguments  may  include
	      whitespace  character  or	 other	tokens that influence argument

       -0     Same as -1, but as a job and argument  termination  character  a
	      null-character (\0) is expected instead of a new-line character.
	      That way also arguments with new-line character can be processed
	      without escape sequences.

       -V     Print the	version	number of xjobs	and exit.

       -v <level>
	      Set  verbosity  of  xjobs	 to  level. Valid leves	are: 0=silent,
	      1=error, 2=warning, 3=info, 4=debug. The default level  of  ver-
	      bosity is	3.

       If  you	have  a	 lot  of .zip files that you want to extract, then use
       xjobs like this:

       $ ls -1 *.zip | xjobs unzip

       If you want to do the same without getting the  output  of  each	 unzip
       task on your terminal, then try this:

       $ ls -1 *.zip | xjobs -n	unzip

       To gzip all *.bak files in a given directory hierarchy, use it the fol-
       lowing way:

       $ find .	-name '*.bak' |	xjobs gzip

       To generate index files for a set of *.jar files, you can use the redi-
       rection feature of xjobs, and do	the following:

       $ ls -1 *.jar | sed 's/\(.*\)/\1	> \1.idx/' | xjobs jar tf

       If you also want	to capture the error output, than use >& instead of >.

       You  can	 also use it to	execute	several	different commands. Therefore,
       write a script file that	contains every job you	want  to  execute  and
       pass it to xjobs	with the option	-s:

       $ cat - > script
       tar xf my.tar
       lame --silent my.wav my.mp3
       crypt notsecret < mydata	> secretfile
       $ xjobs -s script

       To  be  able  to	 queue up jobs from multiple sources with xjobs, use a
       named pipe and pass it explicitly as input script. Then write the  jobs
       to the named pipe:
       $ mkfifo	/var/run/my_named_pipe
       $ xjobs -s /var/run/my_named_pipe &
       $ echo unzip >> /var/run/my_named_pipe
       $ echo tar cf /backup/myhome.tar	/home/me >> /var/run/my_named_pipe

       PATH Determines the location of command.

       Thomas Maier-Komor <>
       Donations via PayPal are	welcome!


       GNU General Public License Version 2


Thomas Maier-Komor		   20100915			      xjobs(1)


Want to link to this manual page? Use this URL:

home | help