xjobs(1)                         User Commands                        xjobs(1)



NAME
       xjobs - construct command line and execute jobs in parallel

SYNOPSIS
       xjobs [options] [utility [argument ...]]

DESCRIPTION
       xjobs reads job descriptions line by line and executes them in
       parallel. It limits the number of parallel executing jobs and starts
       new jobs when jobs finish.  Therefore, it combines the arguments from
       every input line with the utility and arguments given on the command
       line. If no utility is given as an argument to xjobs, then the first
       argument on every job line will be used as utility. To execute utility
       xjobs searches the directories given in the PATH environment variable
       and uses the first file found in these directories.

       xjobs is most useful on multiprocessor machines when one needs to
       execute several time consuming commands that could possibly be run in
       parallel. With xjobs this can be achieved easily, and it is possible to
       limit the load of the machine to a useful value.  It works similar to
       xargs, but starts several processes simultaneously and gives only one
       line of arguments to each utility call.

       By using I/O redirectors the standard input, output, and error stream
       of executed jobs can be redirected. Use < to redirect standard input, >
       to redirect standard output, >! to redirect standard output and
       overwrite an existing file, >> to append standard output to an existing
       file, >& to redirect both standard output and standard error output to
       the same file, and >>& to append both standard output and standard
       error output to the same file.

       If passed on the command line, these operators specify the default I/O
       redirection that can be overwritten by specifying another redirector to
       a specific job on its argument line.  After all these operators a
       filename is expected. See EXAMPLES below for an example.  If you need
       more advanced shell features than the redirection operators supported
       by xjobs, then use as utility a shell of your preference.

       Every job line can be preceeded by a "cd directory;" command that tells
       xjobs in which directory the job shall be executed. For every line this
       can only be used once. For more complex scripting, please pass the line
       to execute to a shell of your choice.

       xjobs constructs the arguments of the jobs to execute from each input
       line. Each input line will create a seperate job, whereas newline
       character are handled as regular whitespace by xargs. To be able to
       include whitespace charakters in arguments, either preceed them with a
       backslash or quote them with single or doublequote charakters. A
       backslash charakter preceeding a newline will make xjobs ignore the
       newline character, thus giving you the ability to pass arguments for a
       single job across multiple lines. To include quotation marks in quoted
       arguments, preceed them with a backslash.  Lines passed to xjobs
       beginning with a # charakter are interpreted as comments.

       Finally, xjobs also includes a mechanism for serializing the execution.
       Like this it is possible to parallelize independent jobs and sequence
       jobs that have a dependency. This can be achieved by inserting a line
       that only consists of two percentage charakters in sequence (%%).  All
       jobs before this sequence point are executed at the requested number of
       jobs in parallel. When hitting the sequence point xjobs waits for all
       processes to finish and then continues starting jobs that follow the
       sequence point.

       When passing a named pipe (i.e. a file name created by mkfifo) via
       option -s as an input, xjobs will close and reopen the fifo when
       reaching end-of-file. Like this it is possible to setup an xjobs server
       and sending jobs to this server from muliple programs. See section
       EXAMPLES below for an example.

OPTIONS
       -j <jobs>
              Sets the maximum number of jobs that are started in parallel.
              The default value is to limit the number executing jobs is equal
              to the number of online processors in the system. If the number
              passed as <jobs> is followed by an 'x' charakter (e.g. 2.5x),
              the value is multiplied with the number of online processors
              before setting the job limit. I.e.  having a machine with 4
              online processors and passing 2.5x as an argument to option -j
              will yield a joblimit of 10 jobs.

       -s <script>
              Use file script instead of the standard input to read the job
              descriptions.

       -n     Redirect standard output and standard error output of executed
              jobs to /dev/null.

       -l <num>
              Combine the arguments of <num> input lines for a single job.

       -p     Start jobs interactively, prompting the user.

       -q <num>
              Limits the number of queued jobs to num elements. Normally xjobs
              reads in jobs from standard input or the give script and queues
              them if they cannot be started at once. With this option, xjobs
              will stop reading as soon as num jobs are queued and restart
              reading when a new job has been started. Like this xjobs
              allocates less memory. Use this option, if you pass huge number
              of jobs to xjobs, to limit memory consumption. It can also
              increase performance of xjobs, but be sure that jobs get fed
              fast enough to xjobs.

       -1     Pass one argument per job, which is expected to be terminated by
              a new-line character. No argument parsing is performed. That way
              it is more easy to process jobs where arguments may include
              whitespace character or other tokens that influence argument
              parsing.

       -0     Same as -1, but as a job and argument termination character a
              null-character (\0) is expected instead of a new-line character.
              That way also arguments with new-line character can be processed
              without escape sequences.

       -V     Print the version number of xjobs and exit.

       -v <level>
              Set verbosity of xjobs to level. Valid leves are: 0=silent,
              1=error, 2=warning, 3=info, 4=debug. The default level of
              verbosity is 3.

EXAMPLES
       If you have a lot of .zip files that you want to extract, then use
       xjobs like this:

       $ ls -1 *.zip | xjobs unzip

       If you want to do the same without getting the output of each unzip
       task on your terminal, then try this:

       $ ls -1 *.zip | xjobs -n unzip

       To gzip all *.bak files in a given directory hierarchy, use it the
       following way:

       $ find . -name '*.bak' | xjobs gzip

       To generate index files for a set of *.jar files, you can use the
       redirection feature of xjobs, and do the following:

       $ ls -1 *.jar | sed 's/\(.*\)/\1 > \1.idx/' | xjobs jar tf

       If you also want to capture the error output, than use >& instead of >.

       You can also use it to execute several different commands. Therefore,
       write a script file that contains every job you want to execute and
       pass it to xjobs with the option -s:

       $ cat - > script
       unzip my.zip
       tar xf my.tar
       lame --silent my.wav my.mp3
       crypt notsecret < mydata > secretfile
       ^D
       $ xjobs -s script

       To be able to queue up jobs from multiple sources with xjobs, use a
       named pipe and pass it explicitly as input script. Then write the jobs
       to the named pipe:
       $ mkfifo /var/run/my_named_pipe
       $ xjobs -s /var/run/my_named_pipe &
       $ echo unzip 1.zip >> /var/run/my_named_pipe
       $ echo tar cf /backup/myhome.tar /home/me >> /var/run/my_named_pipe

ENVIRONMENT VARIABLES
       PATH Determines the location of command.

AUTHORS
       Thomas Maier-Komor <thomas@maier-komor.de>
       Donations via PayPal are welcome!

HOMEPAGE
       http://www.maier-komor.de/xjobs.html

LICENSE
       GNU General Public License Version 2

SEE ALSO
       xargs(1)



Thomas Maier-Komor                 20100915                           xjobs(1)
