submit.1
NAME
qsub, qsh, qlogin, qrsh, qalter, qresub - submit jobs to Grid Engine
SYNTAX
qsub [options] [command [command_args] | -- [command_args]]
qsh [options] [-- xterm_args]
qlogin [options]
qrsh [options] [command [command_args]]
qalter [options] wc_job_range_list [-- [command_args]]
qalter [options] -u user_list | -u * [-- [command_args]]
qresub [options] job_id_list
DESCRIPTION
Qsub submits batch jobs to the Grid Engine queuing system. Grid Engine
supports single- and multiple-node jobs. command can be a path to a
binary or a script (see -b below) which contains the commands to be run
by the job using a shell (for example, sh(1) or csh(1)). Arguments to
the command are given as command_args to qsub. If command is handled
as a script then it is possible to embed flags in the script. If the
first two characters of a script line either match '#$' or are equal to
the prefix string defined with the -C option described below, the line
is parsed for embedded command flags.
Qsh submits an interactive X-windows session to Grid Engine. An
xterm(1) is brought up from the executing machine with the display
directed either to the X-server indicated by the DISPLAY environment
variable or as specified with the -display option of qsh. Interactive
jobs are not spooled if no resource is available to execute them. They
are either dispatched to a suitable machine for execution immediately
or the user submitting the job is notified by qsh that appropriate
resources to execute the job are not available. xterm_args are passed
to the xterm(1) executable. Note, however, that the -e and -ls xterm
options do not work with qsh.
Qlogin is similar to qsh in that it submits an interactive job to the
queuing system. It does not open an xterm(1) window on the X display,
but uses the current terminal for user I/O. It establishes a
telnet(1)-like connection with the remote host, using the configured
mechanisms described in remote_startup(5). Qlogin is invoked exactly
like qsh and its jobs can only run on INTERACTIVE queues. Qlogin jobs
can only be used if the sge_execd(8) is running under the root account.
Qrsh is similar to qlogin in that it submits an interactive job to the
queuing system. It uses the current terminal for user I/O. With a
command, it establishes a rsh(1)-like connection with the remote host.
If no command is given to qrsh, an rlogin(1)-like session is
established. It uses the configured mechanisms described in
remote_startup(5). Qrsh jobs can only run in INTERACTIVE queues unless
the option -now no is used (see below). They can also only be run if
the sge_execd(8) is running under the root account.
Qrsh provides an additional useful feature for integrating with
interactive tools providing a specific command shell. If the
environment variable QRSH_WRAPPER is set when qrsh is invoked, the
command interpreter pointed to by QRSH_WRAPPER will be executed to run
qrsh commands instead of the user's login shell or any shell specified
in the qrsh command-line. The options -cwd, -v, -V, and -display
only apply to batch jobs.
Qalter can be used to change the attributes of pending jobs. For array
jobs with a mix of running and pending tasks (see the -t option below),
modification with qalter only affects the pending tasks. Qalter can
change most of the characteristics of a job (see the corresponding
statements in the OPTIONS section below), including those which were
defined as embedded flags in the script file (see above). An option
specified with qalter completely replaces any parameters previously
specified for the job by that option, e.g. if the job has a resource
requirement corresponding to -l h_rt=3600,h_vmem=2G, then after a
qalter -l h_rt=7200, the resource requirement is simply h_rt=7200; it
is not currently possible to override one or more values.
Some submit options, such as the job script, cannot be changed with
qalter. If parameters are set with qalter, the output from qstat -j
acquires an additional field, version, which is incremented each time.
If a server JSV is defined, modifications are forbidden unless they are
explicitly allowed by the sge_conf(5) parameter jsv_allowed_mod. (See
also jsv(1).)
Qresub allows the user to create jobs as copies of existing pending or
running jobs. The copied jobs will have exactly the same attributes as
the ones from which they were copied, except with a new job ID and with
a cleared hold state. The only modification to the copied jobs
supported by qresub is assignment of a new hold state with the -h
option. This option can be used to first copy a job and then change its
attributes via qalter.
Only a manager can use qresub on jobs submitted by another user.
Regular users can only use qresub on their own jobs.
See sge_shepherd(8) for the significance of exit codes returned by the
submitted job.
For qsub, qsh, qrsh, and qlogin the administrator and the user may
define default request files (see sge_request(5)) which can contain any
of the options described below. If an option in a default request file
is understood by qsub and qlogin but not by qsh the option is silently
ignored if qsh is invoked. Thus you can maintain shared default request
files for both qsub and qsh.
A cluster-wide default request file may be placed under
$SGE_ROOT/$SGE_CELL/common/sge_request. User private default request
files are processed under the locations $HOME/.sge_request and
$cwd/.sge_request. The working directory local default request file
has the highest precedence, then the home directory file and then the
cluster global file. The option arguments, the embedded script flags
and the options in the default request files are processed in the
following order:
left to right in the script line,
left to right in the default request files,
from top to bottom of the script file (qsub only),
from top to bottom of default request files,
from left to right of the command line.
In other words, the command line can be used to override the embedded
flags and the default request settings. The embedded flags, however,
will override the default settings.
Note, that the -clear option can be used to discard any previous
settings at any time in a default request file, in the embedded script
flags, or in a command-line option. It is, however, not available with
qalter.
The request options described below can be requested either hard or
soft. By default, all requests are considered hard until the -soft
option (see below) is encountered. The hard/soft status remains in
effect until its counterpart is encountered again. If all the hard
requests for a job cannot be met, the job will not be scheduled. Jobs
which cannot be run at the present time remain spooled.
OPTIONS
-@ optionfile
Forces qsub, qrsh, qsh, or qlogin to use the options contained
in optionfile. The indicated file may contain all valid options.
Comment lines must start with a "#" sign.
-a date_time
Available for qsub and qalter only.
Defines or redefines the time and date at which a job is
eligible for execution. date_time is of the form
[[CC]YY]MMDDhhmm[.SS]; for the details, please see date_time in
sge_types(5).
If this option is used with qsub or if a corresponding value is
specified in qmon then a parameter named a and the value in the
format CCYYMMDDhhmm.SS will be passed to the defined JSV
instances. (See -jsv option below or find more information
concerning JSV in jsv(1).)
-ac variable[=value],...
Available for qsub, qsh, qrsh, qlogin and qalter only.
Adds the given name/value pair(s) to the job's context. value
may be omitted. Grid Engine appends the given argument to the
list of context variables for the job. Multiple -ac, -dc, and
-sc options may be given. The order is important here.
The outcome of the evaluation of all -ac, -dc, and -sc options
or corresponding values in qmon is passed to defined JSV
instances as parameter with the name ac. (See -jsv option below
or find more information concerning JSV in jsv(1).)
Qalter allows changing this option even while the job executes.
-ar ar_id
Available for qsub, qalter, qrsh, qsh, or qlogin only.
Assigns the submitted job to be a part of an existing Advance
Reservation with id ar_id. The complete list of existing
Advance Reservations can be obtained using the qrstat(1)
command.
Note that the -ar option implicitly adds the -w e option if not
otherwise requested.
Qalter allows changing this option even while the job executes.
The modified parameter will only be in effect after a restart or
migration of the job however.
If this option or a corresponding value in qmon is specified
then this value will be passed to defined JSV instances as
parameter with the name ar. (See -jsv option below or find more
information concerning JSV in jsv(1).)
-A account_string
Available for qsub, qsh, qrsh, qlogin and qalter only.
Identifies the account to which the resource consumption of the
job should be charged. The account_string should conform to the
account_name definition in sge_types(5). In the absence of this
parameter Grid Engine will place the default account string
"sge" in the accounting record of the job.
Qalter allows changing this option even while the job executes.
If this option or a corresponding value in qmon is specified
then this value will be passed to defined JSV instances as
parameter with the name A. (See -jsv option below or find more
information concerning JSV in jsv(1).)
-b y[es]|n[o]
Available for qsub, qrsh only. Qalter does not allow changing
this option. This option cannot be embedded in the script file
itself.
Gives the user the possibility to indicate explicitly whether
command should be treated as binary or script. If the argument
of -b is 'y', then command may be a binary or script. The
command might not be accessible from the submission host.
Nothing except the path of the command will be transferred from
the submission host to the execution host. Path aliasing will be
applied to the path of command before it is executed.
If the argument of -b is 'n' then command needs to be a script,
and it will be handled as such. The script file has to be
accessible by the submission host. It will be transferred to the
execution host. qsub/qrsh will search directive prefixes within
scripts.
qsub will implicitly use -b n, whereas qrsh will apply the -b y
option if nothing else is specified.
The value specified with this option or the corresponding value
specified in qmon will only be passed to defined JSV instances
if the value is yes. The name of the parameter will be b. The
value will be y also when the long form yes was specified during
submission. (See -jsv option below or find more information
concerning JSV in jsv(1).)
Please note that submission of command as a script (-b n) can
have a significant performance impact, especially for short
running jobs and big job scripts. Script submission adds a
number of operations to the submission process: The job script
needs to be
- parsed at client side (for special comments)
- transferred from submit client to qmaster
- spooled in qmaster
- transferred to execd at job execution
- spooled in execd
- removed from spooling both in execd and qmaster once the job is done
If job scripts are available on the execution nodes, e.g. via
NFS, binary submission can be the better choice.
-binding [binding_instance] binding_strategy
A job can request a specific processor core binding (processor
affinity) with this parameter. This request is neither a hard
nor a soft request, it is a hint for the execution host to do
this if possible. Please note that the requested binding
strategy is not used for resource selection within Grid Engine.
As a result an execution host might be selected where Grid
Engine does not even know the hardware topology and therefore is
not able to apply the requested binding. (The binding facility
depends on the platform, and on Grid Engine being compiled with
appropriate support.)
To make Grid Engine select hardware on which the binding can be
applied, please use the -l switch in combination with the
complex attribute m_topology.
binding_instance is an optional parameter. It might either be
env, pe or set depending on which instance should accomplish the
job to core binding. If the value for binding_instance is not
specified then set will be used.
env means that only the environment variable SGE_BINDING will be
exported to the job environment of the job. This variable
contains the selected operating system internal processor
numbers. In presence of SMT or CMT it may contain more than the
number of cores selected, because each core could be represented
by multiple processor identifiers. The processor numbers are
space-separated.
pe means that the information about the selected cores appears
in the fourth column of the pe_hostfile. Here the logical core
and socket numbers are printed (they start at 0 and have no
holes) in colon-separated pairs (e.g. "0,0:1,0", which means
core 0 on socket 0 and core 0 on socket 1). For more
information about the $pe_hostfile check sge_pe(5). In
addition, SGE_BINDING is set as for env binding.
set (the default if nothing else is specified) means that the
binding strategy is applied by Grid Engine. Each shepherd (on
the master host, or those connected to with qrsh -inherit) is
bound to the relevant cores. The binding is inherited by the
task that is started, so that processes or threads it spawns are
bound to those cores, but it is up to the task to make any more
specific bindings of its sub-tasks within the set of bound cores
it inherits. How Grid Engine does this depends on the
underlying hardware architecture of the execution host where the
submitted job will be started. In addition, SGE_BINDING is set
as for env binding.
Binding will be done to restrict the job to run exclusively on
the selected cores, if possible on the platform; otherwise bound
processes may still be able to use other cores. The operating
system will probably allow other unbound processes to use these
cores. Note that if all cores on a node are specified to be
bound by one job, that is probably equivalent to no binding,
with the task free to use all the cores.
The loadcheck tool in the utilbin directory can be used to check
the host's capabilities. You can also use -sep in combination
with -cb of qconf(5) to find whether Grid Engine is able to
recognize the hardware topology. hwloc-ps(1) can be used to
check the bindings in force on a host.
Possible values for binding_strategy are as follows:
linear:[number[:socket,core]]
striding:number:n[:socket,core]
explicit:socket,core[:socket,core]...
For the binding strategies "linear" and "striding" there is an
optional socket and core pair attached. These denote the
mandatory starting point for the first core to bind on. Logical
socket and core numbers are used, per hwloc(7).
linear means that Grid Engine tries to bind the job on number
successive cores. If socket and core are omitted then Grid
Engine first allocates successive cores on the first empty
socket found. ("Empty" means that there are no jobs bound to
the socket by Grid Engine.) If this is not possible, or is not
sufficient, Grid Engine tries to find (further) cores on the
socket with the most unbound cores, and so on. If socket and
core are specified, then Grid Engine tries to find empty cores
with this starting point. If the binding request cannot be
satisfied, then binding is not done on the host concerned.
number may be omitted or specified as slots. In that case it
will be taken as the number of slots assigned to the job on a
per-host basis, on the assumption that one slot per core is
used. That allows binding to work when parallel jobs run across
nodes with variable slot counts per node. Multiple processing
units (hardware threads) per core are currently not taken into
account in this, but systems running such parallel jobs will
typically have only a single thread per core. -binding linear
is a reasonable overall default for sge_request(5) in many
cases.
striding means that Grid Engine tries to find cores with a
certain offset. It will select number of empty cores with an
offset of n-1 cores in between. Start point for the search
algorithm is socket 0 core 0. As soon as number cores are found
they will be used to do the job binding. If there are not
enough empty cores, or if the correct offset cannot be achieved,
then no binding will be done on the host concerned.
explicit binds the specified sockets and cores that are
mentioned in the provided socket/core list. Each socket/core
pair has to be specified only once. If a socket/core pair is
already in use by a different job, the whole binding request
will be ignored.
Note that a binding like "pe linear:number..." is only useful
if the job has exclusive access to its multiple compute nodes,
they all have the same topology, and the PE has a fixed
allocation rule (allocation_rule n). (The $pe_hostfile content
is created on the job's master host, with the same <socket,core>
specification for each.) This method can't be used for job
isolation. Similarly, set requires a fixed allocation per host
for distributed parallel jobs.
Qalter allows changing this option even while the job executes.
The modified parameter will only be in effect after a restart or
migration of the job, however.
If this option, or a corresponding value in qmon, is specified
then these values will be passed to defined JSV instances as
parameters with the names binding_strategy, binding_type,
binding_amount, binding_step, binding_socket, binding_core,
binding_exp_n, binding_exp_socketid, binding_exp_coreid.
Please note that the length of the socket/core value list of the
explicit binding is reported as binding_exp_n. id will be
replaced by the position of the socket/core pair within the
explicit list (0 <= id < binding_exp_n). The first socket/core
pair of the explicit binding will be reported with the parameter
names binding_exp_socket0 and binding_exp_core0.
Values that do not apply for the specified binding will not be
reported to JSV. E.g. binding_step will only be reported for
the striding binding and all binding_exp_* values will passed to
JSV if explicit binding was specified. (See -jsv option below
or find more information concerning JSV in jsv1(.))
-c occasion_specifier
Available for qsub and qalter only.
Defines or redefines whether the job should be checkpointed, and
if so, under what circumstances. The specification of the
checkpointing occasions with this option overwrites the
definitions of the when parameter in the checkpointing
environment (see checkpoint(5)) referenced by the qsub -ckpt
switch. Possible values for occasion_specifier are
n No checkpoint is performed;
s Checkpoint when batch server is shut down;
m Checkpoint at minimum CPU interval;
x Checkpoint when job gets suspended;
r Reschedule when execution host goes into the "unknown"
state;
interval
Checkpoint in the specified interval (a time_specifier
per sge_types(5)).
The minimum CPU interval is defined in the queue configuration
(see queue_conf(5) for details). If interval is specified, the
maximum of that and the queue's minimum CPU interval is used (to
ensure that a machine is not overloaded by checkpoints being
generated too frequently).
The value specified with this option or the corresponding value
specified in qmon will be passed to defined JSV instances. The
interval will be available as parameter with the name
c_interval. The character sequence specified will be available
as parameter with the name c_occasion. Please note that if you
change c_occasion via JSV, then the last setting of c_interval
will be overwritten, and vice versa. (See -jsv option below or
find more information concerning JSV in jsv(1).)
-ckpt ckpt_name
Available for qsub and qalter only.
Selects the checkpointing environment (see checkpoint(5)) to be
used for checkpointing the job. Also declares the job to be a
checkpointing job.
If this option or a corresponding value in qmon is specified
then this value will be passed to defined JSV instances as
parameter with the name ckpt. (See -jsv option below or find
more information concerning JSV in jsv(1).)
-clear Available for qsub, qsh, qrsh, and qlogin only.
Causes all elements of the job to be reset to the initial
default status prior to applying any modifications (if any)
appearing in this specific command.
-cwd Available for qsub, qsh, qrsh and qalter only.
Execute the job from the current working directory. This switch
will activate Grid Engine's path aliasing facility, if the
corresponding configuration files are present (see
sge_aliases(5)).
In the case of qalter, the previous definition of the current
working directory will be overwritten if qalter is executed from
a different directory than the preceding qsub or qalter.
Qalter allows changing this option even while the job executes.
The modified parameter will only be in effect after a restart or
migration of the job, however.
If this option or a corresponding value in qmon is specified
then this value will be passed to defined JSV instances as
parameter with the name cwd. The value of this parameter will be
the absolute path to the current working directory. JSV scripts
can remove the path from jobs during the verification process by
setting the value of this parameter to an empty string. As a
result the job behaves as if -cwd was not specified during job
submission. (See -jsv option below or find more information
concerning JSV in jsv(1).)
-C prefix_string
Available for qsub and qrsh with script submission (-b n).
Prefix_string defines the prefix that declares a directive in
the job's command. The prefix is not a job attribute, but
affects the behavior of qsub and qrsh. If prefix is a null
string, the command will not be scanned for embedded directives.
The directive prefix consists of two ASCII characters which,
when appearing in the first two bytes of a script line, indicate
that what follows is an Grid Engine command. The default is
"#$".
The user should be aware that changing the first delimiting
character can produce unforeseen side effects. If the script
file contains anything other than a "#" character in the first
byte position of the line, the shell processor for the job will
reject the line and may exit the job prematurely.
If the -C option is present in the script file, it is ignored.
-dc variable,...
Available for qsub, qsh, qrsh, qlogin and qalter only.
Removes the given variable(s) from the job's context. Multiple
-ac, -dc, and -sc options may be given. The order is important.
Qalter allows changing this option even while the job executes.
The outcome of the evaluation of all -ac, -dc, and -sc options
or corresponding values in qmon is passed to defined JSV
instances as parameter with the name ac. (See -jsv option below
or find more information concerning JSV in jsv(1).)
-display display_specifier
Available for qsh and qrsh with a command.
Directs xterm(1) to use display_specifier in order to contact
the X server. The display_specifier has to contain the hostname
part of the display name (e.g. myhost:1). Local display names
(e.g. :0) cannot be used in grid environments. Values set with
the -display option overwrite settings from the submission
environment and from -v command line options.
If this option or a corresponding value in qmon is specified
then this value will be passed to defined JSV instances as
parameter with the name display. This value will also be
available in the job environment which might optionally be
passed to JSV scripts. The variable name will be DISPLAY. (See
-jsv option below or find more information concerning JSV in
jsv(1).)
-dl date_time
Available for qsub, qsh, qrsh, qlogin and qalter only.
Specifies the deadline initiation time in [[CC]YY]MMDDhhmm[.SS]
format (see -a option above). The deadline initiation time is
the time at which a deadline job has to reach top priority to be
able to complete within a given deadline. Before the deadline
initiation time the priority of a deadline job will be raised
steadily until it reaches the maximum as configured by the Grid
Engine administrator.
This option is applicable only for users allowed to submit
deadline jobs.
If this option or a corresponding value in qmon is specified
then this value will be passed to defined JSV instances as
parameter with the name dl. The format for the date_time value
is CCYYMMDDhhmm.SS. (See -jsv option below or find more
information concerning JSV in jsv(1).)
-e [[hostname]:]path...
Available for qsub and qalter only, but currently silently
ignored by qsh, qrsh, and qlogin.
Defines or redefines the path used for the standard error stream
of the job. For qsh, qrsh and qlogin only the standard error
stream of prolog and epilog is redirected. If the path
constitutes an absolute path name, the error-path attribute of
the job is set to path, including the hostname. If the path name
is relative, Grid Engine expands path, either relative to the
current working directory (if the -cwd switch - see above - is
also specified) or to the home directory. If hostname is
present, the standard error stream will be placed in the
corresponding location only if the job runs on the specified
host. If the path contains a ":" without a hostname, a leading
":" has to be specified.
By default the file name for interactive jobs is /dev/null. For
batch jobs the default file name has the form job_name.ejob_id
and job_name.ejob_id.task_id for array job tasks (see -t option
below).
If path is a directory, the standard error stream of the job
will be put in this directory under the default file name. If
the pathname contains certain pseudo environment variables,
their value will be expanded at runtime of the job and will be
used to constitute the standard error stream path name. The
following pseudo environment variables are supported currently;
note the lack of an Grid Engine_ prefix in this context:
$HOME home directory on execution machine
$USER user ID of job owner
$JOB_ID current job ID
$JOB_NAME current job name (see -N option)
$HOSTNAME name of the execution host
$TASK_ID array job task index number
The tilde sign "~" can be used as an alternative to $HOME, as in
csh(1), bash(1), or ksh(1). Note that the "~" sign also works
in combination with user names, so that "~user" expands to the
home directory of user. Using another user ID than that of the
job owner requires corresponding permissions, of course.
Qalter allows changing this option even while the job executes.
The modified parameter will only be in effect after a restart or
migration of the job, however.
If this option or a corresponding value in qmon is specified
then this value will be passed to defined JSV instances as
parameter with the name e. (See -jsv option below or find more
information concerning JSV in jsv(1).)
-hard Available for qsub, qsh, qrsh, qlogin and qalter only.
Signifies that all -q and -l resource requirements following in
the command line will be hard requirements and must be satisfied
in full before a job can be scheduled.
As Grid Engine scans the command line and script file for Grid
Engine options and parameters it builds a list of resources
required by a job. All such resource requests are normally
considered as absolutely essential for the job to commence.
However, if the -soft option (see below) is encountered during
the scan then all following resources are designated as "soft
requirements" for execution, or "nice-to-have, but not
essential". If the -hard flag is encountered at a later stage of
the scan, all resource requests following it once again become
essential. The -hard and -soft options in effect act as toggles
during the scan.
If this option or a corresponding value in qmon is specified
then the corresponding -q and -l resource requirements will be
passed to defined JSV instances as parameter with the names
q_hard and l_hard. Find for information in the sections
describing -q and -l. (See -jsv option below or find more
information concerning JSV in jsv(1).)
-h | -h {u|s|o|n|U|O|S}...
Available for qsub (only -h), qrsh, qalter and qresub (hold
state is removed when not set explicitly).
List of holds to place on a job, a task or some tasks of a job.
`u' denotes a user hold.
`s' denotes a system hold.
`o' denotes a operator hold.
`n' denotes no hold (requires manager privileges).
As long as any hold other than `n' is assigned to the job the
job is not eligible for execution. Holds can be released via
qalter and qrls(1). In case of qalter this is supported by the
following additional option specifiers for the -h switch:
`U' removes a user hold.
`S' removes a system hold.
`O' removes a operator hold.
Grid Engine managers can assign and remove all hold types, Grid
Engine operators can assign and remove user and operator holds,
and users can only assign or remove user holds.
In the case of qsub only user holds can be placed on a job and
thus only the first form of the option with the -h switch alone
is allowed. As opposed to this, qalter requires the second form
described above.
An alternate means to assign a hold is provided by the qhold(1)
facility.
If the job is an array job (see the -t option below), all tasks
specified via -t are affected by the -h operation
simultaneously.
Qalter allows changing this option even while the job executes.
The modified parameter will only be in effect after a restart or
migration of the job, however.
If this option is specified with qsub or during the submission
of a job in qmon then the parameter h with the value u will be
passed to the defined JSV instances indicating that the job will
be in user hold after the submission finishes. (See -jsv option
below or find more information concerning JSV in jsv(1).)
-help Prints a listing of all options.
-hold_jid wc_job_list
Available for qsub, qrsh, and qalter only. See sge_types(5).
For the definition of wc_job_list.
Defines or redefines the job dependency list of the submitted
job. A reference by job name or pattern is only accepted if the
referenced job is owned by the same user as the referring job.
The submitted job is not eligible for execution unless all jobs
referenced in the comma-separated job id and/or job name list
have completed. If any of the referenced jobs exits with exit
code 100, the submitted job will remain ineligible for
execution.
With the help of job names or regular patterns, one can specify
a job dependency on multiple jobs satisfying the regular
pattern, or on all jobs with the requested name. The name
dependencies are resolved at submit time and can only be changed
via qalter. New jobs or name changes of other jobs will not be
taken into account. To remove a job dependency list with
qalter, use a null wc_job_list, i.e. use
qalter -hold_jid '' ...
Qalter allows changing this option even while the job executes.
The modified parameter will only be in effect after a restart or
migration of the job, however.
If this option or a corresponding value in qmon is specified
then this value will be passed to defined JSV instances as
parameter with the name hold_jid. (See -jsv option below or
find more information concerning JSV in jsv(1).)
-hold_jid_ad wc_job_list
Available for qsub, qrsh, and qalter only. See sge_types(5) for
the definition of wc_job_list.
Defines or redefines the job array dependency list of the
submitted job. A reference by job name or pattern is only
accepted if the referenced job is owned by the same user as the
referring job. Each sub-task of the submitted job is not
eligible for execution unless the corresponding sub-tasks of all
jobs referenced in the comma-separated job id and/or job name
list have completed. If dependent jobs have a different array
stride to referenced jobs, the dependency may be many-to-one or
one-to-many, e.g. if an array is submitted with -t 1:50:2,
depending on one submitted with -t 1:50, then task n of the
dependent depends on task n+1 of the other.
If any array task of the referenced jobs exits with exit code
100, the dependent tasks of the submitted job will remain
ineligible for execution.
With the help of job names or regular patterns, one can specify
a job dependency on multiple jobs satisfying the regular
pattern, or on all jobs with the requested name. The name
dependencies are resolved at submit time and can only be changed
via qalter. New jobs or name changes of other jobs will not be
taken into account.
If either the submitted job or any job in wc_job_list are not
array jobs with the same range of sub-tasks (see -t option
below), the request list will be rejected and the job create or
modify operation will return an error.
qalter allows changing this option even while the job executes.
The modified parameter will only be in effect after a restart or
migration of the job, however.
If this option or a corresponding value in qmon is specified,
then this value will be passed to defined JSV instances as a
parameter with the name hold_jid_ad. (See -jsv option below or
find more information concerning JSV in jsv(1).)
-i [[hostname]:]path,...
Available for qsub, and qalter only.
Defines or redefines the path used for the standard input stream
of the job. The path is handled as described in the -e option
for the standard error stream.
By default /dev/null is the input stream for the job.
It is possible to use certain pseudo variables, whose values
will be expanded at runtime of the job and will be used to
express the standard input stream as described in the -e option
for the standard error stream.
Qalter allows changing this option even while the job executes.
The modified parameter will only be in effect after a restart or
migration of the job, however.
If this option or a corresponding value in qmon is specified
then this value will be passed to defined JSV instances as
parameter with the name i. (See -jsv option below or find more
information concerning JSV in jsv(1).)
-inherit
Available only for qrsh and qmake(1).
qrsh allows the user to start a task in an already scheduled
parallel job. The option -inherit tells qrsh to read a job id
from the environment variable JOB_ID and start the specified
command as a task in this job. Please note that in this case,
the hostname of the host where the command will be executed must
precede the command to execute; the syntax changes to
qrsh -inherit [other options] hostname command [command_args]
Note also, that in combination with -inherit, most other command
line options will be ignored. Only the options -verbose, -v and
-V will be interpreted. As a replacement to option -cwd please
use -v PWD.
Usually a task should have the same environment (including the
current working directory) as the corresponding job, so
specifying the option -V should be suitable for most
applications.
Note: If in your system the qmaster tcp port is not configured
as a service, but rather via the environment variable
SGE_QMASTER_PORT, make sure that this variable is set in the
environment when calling qrsh or qmake with the -inherit option.
If you call qrsh or qmake with the -inherit option from within a
job script, export SGE_QMASTER_PORT with the option "-v
SGE_QMASTER_PORT" either as a command argument or an embedded
directive.
This parameter is not available in the JSV context. (See -jsv
option below or find more information concerning JSV in jsv(1).)
-j y[es]|n[o]
Available for qsub, qsh, qrsh, qlogin and qalter only.
Specifies whether or not the standard error stream of the job is
merged into the standard output stream.
If both the -j y and the -e options are present, Grid Engine
sets, but ignores, the error-path attribute.
Qalter allows changing this option even while the job executes.
The modified parameter will only be in effect after a restart or
migration of the job, however.
The value specified with this option or the corresponding value
specified in qmon will only be passed to defined JSV instances
if the value is yes. The name of the parameter will be j. The
value will be y also when the long form yes was specified during
submission. (See -jsv option below or find more information
concerning JSV in jsv(1).)
-js job_share
Available for qsub, qsh, qrsh, qlogin and qalter only.
Defines or redefines the job share of the job relative to other
jobs. Job share is an unsigned integer value. The default job
share value for jobs is 0.
The job share influences the Share Tree Policy and the
Functional Policy. It has no effect on the Urgency and Override
Policies (see share_tree(5), sched_conf(5) for further
information on the resource management policies supported by
Grid Engine).
In case of the Share Tree Policy, users can distribute the
tickets to which they are currently entitled among their jobs
using different shares assigned via -js. If all jobs have the
same job share value, the tickets are distributed evenly.
Otherwise, jobs receive tickets relative to the different job
shares. Job shares are treated like an additional level in the
share tree in the latter case.
In connection with the Functional Policy, the job share can be
used to weight jobs within the functional job category. Tickets
are distributed relative to any uneven job share distribution
treated as a virtual share distribution level underneath the
functional job category.
If both the Share Tree and the Functional Policy are active, the
job shares will have an effect in both policies, and the tickets
independently derived in each of them are added to the total
number of tickets for each job.
If this option or a corresponding value in qmon is specified
then this value will be passed to defined JSV instances as
parameter with the name js. (See -jsv option below or find more
information concerning JSV in jsv(1).)
-jsv jsv_url
Available for qsub, qsh, qrsh and qlogin only.
Defines a client JSV instance which will be executed to verify
the job specification before the job is sent to qmaster.
In contrast to other options this switch will not be overwritten
if it is also used in sge_request files. Instead all specified
JSV instances will be executed to verify the job to be
submitted.
The JSV instance which is directly passed with the command line
of a client is executed first to verify the job specification.
After that the JSV instance which might have been defined in
various sge_request files will be triggered to check the job.
Find more details in man page jsv(1) and sge_request(5).
The syntax of the jsv_url is specified in sge_types(5).
-l resource=expression,...
Available for qsub, qsh, qrsh, qlogin and qalter only.
Launch the job in a Grid Engine queue meeting the given resource
request list. In case of qalter the previous definition is
replaced by the specified one.
complex(5) describes how a list of available resources and their
associated valid value specifiers can be obtained, and section
"MATCHING TYPES" in sge_types(5) describes the possible forms of
the expression for the requested resources.
There may be multiple -l switches in a single command. You may
request multiple -l options to be soft or hard both in the same
command line. In case of a serial job multiple -l switches
refine the definition for the sought queue.
Qalter allows changing the value of this option even while the
job is running, but only if no change is made to requests for
resources marked as consumable. However the modification will
only be effective after a restart or migration of the job.
If this option or a corresponding value in qmon is specified the
these hard and soft resource requirements will be passed to
defined JSV instances as parameter with the names l_hard and
l_soft. If regular expressions will be used for resource
requests, then these expressions will be passed as they are.
Also shortcut names will not be expanded. (See -jsv option
above or find more information concerning JSV in jsv(1).)
-m b|e|a|s|n,...
Available for qsub, qsh, qrsh, qlogin and qalter only.
Defines or redefines under which circumstances mail is to be
sent to the job owner or to the users defined with the -M option
described below. The option arguments have the following
meaning:
`b' Mail is sent at the beginning of the job.
`e' Mail is sent at the end of the job.
`a' Mail is sent when the job is aborted or
rescheduled.
`s' Mail is sent when the job is suspended.
`n' No mail is sent.
Currently no mail is sent when a job is suspended.
Qalter allows changing the "b", "e", and "a" option arguments
even while the job executes. The modification of the option
arguments will only be in effect after a restart or migration of
the job, however.
If this option or a corresponding value in qmon is specified
then this value will be passed to defined JSV instances as
parameter with the name m. (See -jsv option above or find more
information concerning JSV in
-M user[@host],...
Available for qsub, qsh, qrsh, qlogin and qalter only.
Defines or redefines the list of users to which the server that
executes the job has to send mail, if the server sends mail
about the job. Default is the job owner at the originating
host.
Qalter allows changing this option even while the job executes.
The modification will only be in effect after a restart or
migration of the job, however.
If this option or a corresponding value in qmon is specified
then this value will be passed to defined JSV instances as
parameter with the name M. (See -jsv option above or find more
information concerning JSV in jsv(1).)
-masterq wc_queue_list
Available for qsub, qrsh, qsh, qlogin and qalter. Only
meaningful for parallel jobs, i.e. together with the -pe option.
Defines or redefines a list of cluster queues, queue domains and
queue instances which may be used to become the so called master
queue of this parallel job. A more detailed description of
wc_queue_list can be found in sge_types(5). The master queue is
defined as the queue where the parallel job is started. The
other queues to which the parallel job spawns tasks are called
slave queues. A parallel job only has one master queue.
This parameter has all the properties of a resource request and
will be merged with requirements derived from the -l option
described above.
Qalter allows changing this option even while the job executes.
The modified parameter will only be in effect after a restart or
migration of the job, however.
If this option, or a corresponding value in qmon, is specified,
this hard resource requirement will be passed to defined JSV
instances as a parameter with the name masterq. (See -jsv
option above or find more information concerning JSV in jsv(1).)
-notify
Available for qsub, qrsh (with command) and qalter only.
This flag, when set, causes Grid Engine to send "warning"
signals to a running job prior to sending the signals
themselves. If a SIGSTOP is pending, the job will receive a
SIGUSR1 several seconds before the SIGSTOP. If a SIGKILL is
pending, the job will receive a SIGUSR2 several seconds before
the SIGKILL. This option provides the running job a configured
time interval to do cleanup operations before receiving the
SIGSTOP or SIGKILL. The amount of time delay is controlled by
the notify parameter in each queue configuration (see
queue_conf(5)).
Qalter allows changing this option even while the job executes.
Only if this option is used the parameter named notify with the
value y will be passed to defined JSV instances. (See -jsv
option above or find more information concerning JSV in jsv(1).)
-now y[es]|n[o]
Available for qsub, qsh, qlogin and qrsh.
-now y tries to start the job immediately or not at all. The
command returns 0 on success, or 1 on failure (also if the job
could not be scheduled immediately). For array jobs submitted
with the -now option, if all tasks cannot be immediately
scheduled, no tasks are scheduled.
Jobs submitted with -now y option, can only run on INTERACTIVE
queues. -now y is default for qsh, qlogin and qrsh
With the -now n option, the job will be put into the pending
queue if it cannot be executed immediately. -now n is default
for qsub.
-N name
Available for qsub, qsh, qrsh, qlogin and qalter only.
The name of the job. The name should follow the "name"
definition in sge_types(5). Invalid job names will be denied at
submit time.
If the -N option is not present, Grid Engine assigns the name of
the job script to the job after any directory pathname has been
removed from the script name. If the script is read from
standard input, the job name defaults to STDIN.
In the case of qsh or qlogin with the -N option is absent, the
name "INTERACTIVE" or "QLOGIN" respectively is assigned to the
job.
In the case of qrsh if the -N option is absent, the resulting
job name is determined from the qrsh command line by using the
argument string up to the first occurrence of a semicolon or
whitespace and removing the directory pathname. With no
command, "QRLOGIN" is used.
Qalter allows changing this option even while the job executes.
The value specified with this option or the corresponding value
specified in qmon will be passed to defined JSV instances as
parameter with the name N. (See -jsv option above or find more
information concerning JSV in jsv(1).)
-noshell
Available only for qrsh with a command line.
Do not start the command line given to qrsh in the user's login
shell, i.e. execute it without the wrapping shell.
This option can be used to speed up execution as some overhead,
like the shell startup and sourcing the shell resource files, is
avoided.
This option can only be used if no shell-specific command line
parsing is required. If the command line contains shell syntax
like environment variable substitution or (back) quoting, a
shell must be started. In this case, either do not use the
-noshell option or include the shell call in the command line.
Example:
qrsh echo '$HOSTNAME'
Alternative call with the -noshell option
qrsh -noshell /bin/tcsh -f -c 'echo $HOSTNAME'
-nostdin
Available only for qrsh.
Suppress the input stream STDIN - qrsh will pass the option -n
to the rsh(1) command. This is especially useful, if multiple
tasks are executed in parallel using qrsh, e.g. in a make(1)
process it would be undefined which process would get the input.
-o [[hostname]:]path,...
Available for qsub and qalter only, but currently silently
ignored by qsh, qrsh, and qlogin.
Specify the path used for the standard output stream of the job.
path is handled as described in the -e option for the standard
error stream.
By default the file name for standard output has the form
job_name.ojob_id, or job_name.ojob_id.task_id for array job
tasks (see -t option below).
Qalter allows changing this option even while the job executes.
The modified parameter will only be in effect after a restart or
migration of the job, however.
If this option or a corresponding value in qmon is specified
then this value will be passed to defined JSV instances as
parameter with the name o. (See -jsv option above or find more
information concerning JSV in jsv(1).)
-ot override_tickets
Available for qalter only.
Changes the number of override tickets for the specified job.
Requires manager/operator privileges.
-P project_name
Available for qsub, qsh, qrsh, qlogin and qalter only.
Specifies the project to which this job is assigned. The
administrator needs to give permission to individual users to
submit jobs to a specific project (see -aprj option to
qconf(1)).
If this option or a corresponding value in qmon is specified
then this value will be passed to defined JSV instances as
parameter with the name P. (See -jsv option above or find more
information concerning JSV in jsv(1).)
-p priority
Available for qsub, qsh, qrsh, qlogin and qalter only.
Defines or redefines the priority of the job relative to other
jobs. priority is an integer in the range -1023 to 1024. The
default priority value for jobs is 0.
Users may only decrease the priority of their jobs. Grid Engine
managers and administrators may also increase the priority
associated with jobs. If a pending job has higher priority, it
is eligible for being dispatched earlier by the Grid Engine
scheduler.
If this option or a corresponding value in qmon is specified and
the priority is not 0 then this value will be passed to defined
JSV instances as parameter with the name p. (See -jsv option
above or find more information concerning JSV in jsv(1).)
-pe parallel_environment n[-[m]]|[-]m,...
Available for qsub, qsh, qrsh, qlogin and qalter only.
Parallel programming environment (PE) to instantiate. For more
detail about PEs, please see parallel_env in sge_types(5).
Qalter allows changing this option even while the job executes.
The modified parameter will only be in effect after a restart or
migration of the job, however.
If this option or a corresponding value in qmon is specified
then the parameters pe_name, pe_min and pe_max will be passed to
configured JSV instances where pe_name will be the name of the
parallel environment and the values pe_min and pe_max represent
the values n and m which have been provided with the -pe option.
A missing specification of m will be expanded as value 9999999
in JSV scripts and it represents the value infinity. (See -jsv
option above or find more information concerning JSV in jsv(1).)
-pty y[es]|n[o]
Available for qrsh, qsub and qlogin only, and normally only
effective for interactive jobs with the builtin remote startup
method (see remote_startup(5)).
-pty yes starts the job in a pseudo terminal (pty). If no pty is
available, the job start fails. -pty no starts the job without
a pty. By default, qrsh without a command and qlogin start the
job in a pty, and qrsh or qsub with a command starts the job
without a pty.
This parameter is not available in the JSV context. (See -jsv
option above or find more information concerning JSV in jsv(1).)
-q wc_queue_list
Available for qsub, qrsh, qsh, qlogin and qalter.
Defines or redefines a list of cluster queues, queue domains or
queue instances which may be used to execute this job. Please
find a description of wc_queue_list in sge_types(5). This
parameter has all the properties of a resource request and will
be merged with requirements derived from the -l option described
above.
Qalter allows changing this option even while the job executes.
The modified parameter will only be in effect after a restart or
migration of the job, however.
If this option or a corresponding value in qmon is specified
then these hard and soft resource requirements will be passed to
defined JSV instances as parameters with the names q_hard and
q_soft. If regular expressions will be used for resource
requests, then these expressions will be passed as they are.
Also shortcut names will not be expanded. (See -jsv option
above or find more information concerning JSV in jsv(1).)
-R y[es]|n[o]
Available for qsub, qrsh, qsh, qlogin and qalter.
Indicates whether a reservation for this job should be done.
Reservation is never done for immediate jobs, i.e. jobs
submitted using the -now yes option. Please note that
regardless of the reservation request, job reservation might be
disabled using max_reservation in sched_conf(5), and is limited
to that number of the highest priority jobs.
By default jobs are submitted with -R n.
The value specified with this option or the corresponding value
specified in qmon will only be passed to defined JSV instances
if the value is yes. The name of the parameter will be R. The
value will be y also when the long form yes was specified during
submission. (See -jsv option above or find more information
concerning JSV in jsv(1).)
-r y[es]|n[o]
Available for qsub and qalter only.
Specifies whether a job can be rerun or not. If the value of -r
is 'yes', the job will be rerun if it gets aborted without
leaving a consistent exit state. (This is typically the case if
the node on which the job is running crashes). If -r is 'no'
(the default), the job will not be rerun under such
circumstances. It will still be rerun if it finishes with exit
code 99 unless FORBID_RESCHEDULE is set in qmaster_params in
sge_conf(5).
Interactive jobs submitted with qsh, qrsh, or qlogin are not
rerunnable.
Qalter allows changing this option even while the job executes.
The value specified with this option or the corresponding value
specified in qmon will only be passed to defined JSV instances
if the value is yes. The name of the parameter will be r. The
value will be y also when the long form yes was specified during
submission. (See -jsv option above or find more information
concerning JSV in jsv(1).)
-sc variable[=value],...
Available for qsub, qsh, qrsh, qlogin and qalter only.
Sets the given name/value pairs as the job's context. value may
be omitted. Grid Engine replaces the job's previously defined
context with the one given as the argument. Multiple -ac, -dc,
and -sc options may be given. The order is important.
Contexts provide a way to dynamically attach and remove meta-
information to and from a job. The context variables are not
passed to the job's execution context in its environment, but
are available in its spool area.
Qalter allows changing this option even while the job executes.
The outcome of the evaluation of all -ac, -dc, and -sc options
or corresponding values in qmon is passed to defined JSV
instances as parameter with the name ac. (See -jsv option above
or find more information concerning JSV in jsv(1).)
-shell y[es]|n[o]
Available only for qsub.
-shell n causes qsub to execute the command line directly, as if
by exec(2). No command shell will be executed for the job.
This option only applies when -b y is also used. Without -b y,
-shell n has no effect.
This option can be used to speed up execution, as some overhead,
like the shell startup and sourcing the shell resource files, is
avoided.
This option can only be used if no shell-specific command line
parsing is required. If the command line contains shell syntax,
like environment variable substitution or (back) quoting, a
shell must be started. In this case either do not use the
-shell n option or execute the shell as the command line and
pass the path to the executable as a parameter.
If a job executed with the -shell n option fails due to a user
error, such as an invalid path to the executable, the job will
enter the error state.
-shell y cancels the effect of a previous -shell n. Otherwise,
it has no effect.
See -b and -noshell for more information.
The value specified with this option or the corresponding value
specified in qmon will only be passed to defined JSV instances
if the value is yes. The name of the parameter will be shell.
The value will be y also when the long form yes was specified
during submission. (See -jsv option above or find more
information concerning JSV in jsv(1).)
-soft Available for qsub, qsh, qrsh, qlogin and qalter only.
Signifies that all resource requirements following in the
command line will be soft requirements and are to be filled on
an "as available" basis.
As Grid Engine scans the command line and script file for Grid
Engine options and parameters, it builds a list of resources
required by the job. All such resource requests are normally
considered as absolutely essential for the job to commence.
However, if the -soft option is encountered during the scan then
all following resources are designated as "soft requirements"
for execution, or "nice-to-have, but not essential". If the
-hard flag (see above) is encountered at a later stage of the
scan, all resource requests following it once again become
essential. The -hard and -soft options in effect act as toggles
during the scan. However, requests for consumable resources
cannot be soft.
If this option or a corresponding value in qmon is specified
then the corresponding -q and -l resource requirements will be
passed to defined JSV instances as parameter with the names
q_soft and l_soft. Find for information in the sections
describing -q and -l. (See -jsv option above or find more
information concerning JSV in jsv(1).)
-sync y[es]|n[o]
Available for qsub.
-sync y causes qsub to wait for the job to complete before
exiting. If the job completes successfully, qsub's exit code
will be that of the completed job. If the job fails to complete
successfully, qsub will print an error message to the standard
error stream indicating why the job failed, and will have an
exit code of 1. If qsub is interrupted, e.g. with CTRL-C,
before the job completes, the job will be canceled.
With the -sync n option, qsub will exit with an exit code of 0
as soon as the job is submitted successfully. -sync n is
default for qsub.
If -sync y is used in conjunction with -now y, qsub will behave
as though only -now y were given until the job has been
successfully scheduled, after which time qsub will behave as
though only -sync y were given.
If -sync y is used in conjunction with -t, qsub will wait for
all the job's tasks to complete before exiting. If all the
job's tasks complete successfully, qsub's exit code will be that
of the first completed job task with a non-zero exit code, or 0
if all job tasks exited with an exit code of 0. If any of the
job's tasks fail to complete successfully, qsub will print an
error message to the standard error stream indicating why the
job task(s) failed, and will have an exit code of 1. If qsub is
interrupted, e.g. with CTRL-C, before the job completes, all of
the job's tasks will be canceled.
Information that this switch was specified during submission is
not available in the JSV context. (See -jsv option above or
find more information concerning JSV in jsv(1).)
-S [[hostname]:]pathname,...
Available for qsub, qsh and qalter.
Specifies the interpreting shell for the job. pathname must be
an executable file which interprets command-line options -c and
-s as /bin/sh does.
Only one pathname component without a host specifier is valid
and only one path name for a given host is allowed. Shell paths
with host assignments define the interpreting shell for the job
if the host is the execution host. The shell path without host
specification is used if the execution host matches none of the
hosts in the list.
Furthermore, the pathname can be constructed with pseudo
environment variables as described for the -e option above.
In the case of qsh the specified shell path is used to execute
the corresponding command interpreter in the xterm(1) (via its
-e option) started on behalf of the interactive job. Qalter
allows changing this option even while the job executes. The
modified parameter will only be in effect after a restart or
migration of the job, however.
If this option or a corresponding value in qmon is specified
then this value will be passed to defined JSV instances as
parameter with the name S. (See -jsv option above or find more
information concerning JSV in jsv(1).)
-t n[-m[:s]]
Available for qsub only.
Submits a so called Array Job, i.e. an array of identical tasks
being differentiated only by an index number and being treated
by Grid Engine almost like a series of jobs. The argument of -t
specifies the number of array job tasks and the index number
which will be associated with the tasks in terms of the range's
start (n), default 1, its end (m), and stride (s), default 1.
The index numbers will be exported to the job tasks via the
environment variable SGE_TASK_ID. The arguments n, m, and s will
be available through the environment variables SGE_TASK_FIRST,
SGE_TASK_LAST and SGE_TASK_STEPSIZE.
The following restrictions apply to the values n and m:
1 <= n <= MIN(2^31-1, max_aj_tasks)
1 <= m <= MIN(2^31-1, max_aj_tasks)
n <= m
max_aj_tasks is defined in the cluster configuration (see
sge_conf(5)).
The task id range specified in the option argument may be a
single number, a simple range of the form n-m or a range with a
step size. Hence, the task id range specified by 2-10:2 would
result in the task id indexes 2, 4, 6, 8, and 10, for a total of
5 identical tasks, each with the environment variable
SGE_TASK_ID containing one of the 5 index numbers.
All array job tasks inherit the same resource requests and
attribute definitions as specified in the qsub or qalter command
line, except for the -t option. The tasks are scheduled
independently and, provided enough resources exist,
concurrently, very much like separate jobs. However, an array
job or a sub-array there of can be accessed as a single unit by
commands like qmod(1) or qdel(1). See the corresponding manual
pages for further detail.
Array jobs are commonly used to execute the same type of
operation on varying input data sets correlated with the task
index number. The number of tasks in an array job is unlimited.
STDOUT and STDERR of array job tasks will be written into
different files with the default location
jobname.[e|o]job_id.task_id
In order to change this default, the -e and -o options (see
above) can be used together with the pseudo environment
variables $HOME, $USER, $JOB_ID, $JOB_NAME, $HOSTNAME, and
$TASK_ID.
Note, that you can use the output redirection to divert the
output of all tasks into the same file, but the result of this
is undefined.
If this option or a corresponding value in qmon is specified
then this value will be passed to defined JSV instances as
parameters with the names t_min, t_max and t_step (See -jsv
option above or find more information concerning JSV in jsv(1).)
-tc max_running_tasks
Available for qsub and qalter only.
Used in conjunction with array jobs (see -t option) to set a
self-imposed limit on the maximum number of concurrently running
tasks per job.
If this option, or a corresponding value in qmon is specified,
then this value will be passed to defined JSV instances as a
parameter with the name tc. (See the -jsv option above or find
more information concerning JSV in jsv(1).)
-terse Available for qsub only.
Causes qsub to display only the job-id of the job being
submitted rather than the regular "Your job ..." string. In
case of an error the error is reported on stderr as usual.
This can be helpful for scripts which need to parse qsub output
to get the job-id.
Information that this switch was specified during submission is
not available in the JSV context. (See -jsv option above or
find more information concerning JSV in jsv(1).)
-u username,...
Available for qalter only. Changes are only made on those jobs
which were submitted by users specified in the list of
usernames. For managers it is possible to use the qalter -u '*'
command to modify all jobs of all users.
If you use the -u switch it is not permitted to specify an
additional wc_job_range_list.
-v variable[=value],...
Available for qsub, qrsh (with command argument) and qalter.
Defines or redefines the environment variables to be exported to
the execution context of the job. If the -v option is present
Grid Engine will add the environment variables defined as
arguments to the switch and, optionally, values of specified
variables, to the execution context of the job.
Qalter allows changing this option even while the job executes.
The modified parameter will only be in effect after a restart or
migration of the job, however.
All environment variables specified with -v, -V or the DISPLAY
variable provided with -display will be exported to the defined
JSV instances only optionally when this is requested explicitly
during the job submission verification. (See -jsv option above
or find more information concerning JSV in jsv(1).)
-verbose
Available only for qrsh and qmake(1).
Unlike qsh and qlogin, qrsh does not output any informational
messages while establishing the session, compliant with the
standard rsh(1) and rlogin(1) system calls. If the option
-verbose is set, qrsh behaves like the qsh and qlogin commands,
printing information about the process of establishing the
rsh(1) or rlogin(1) session.
-verify
Available for qsub, qsh, qrsh, qlogin and qalter.
Instead of submitting a job, prints detailed information about
the would-be job as though qstat(1) -j were used, including the
effects of command-line parameters and the external environment.
-V Available for qsub, qsh, qrsh with command and qalter.
Specifies that all environment variables active within the qsub
utility be exported to the context of the job.
All environment variables specified with -v, -V or the DISPLAY
variable provided with -display will be exported to the defined
JSV instances only optionally when this is requested explicitly
during the job submission verification. (See -jsv option above
or find more information concerning JSV in jsv(1).)
-w e|w|n|p|v
Available for qsub, qsh, qrsh, qlogin and qalter.
Specifies a validation level applied to the job to be submitted
(qsub, qlogin, and qsh) or the specified queued job (qalter).
The specifiers e, w, n, p, and v define the following validation
modes:
e error - jobs with invalid requests will be rejected.
w warning - only a warning will be displayed for invalid
requests.
n none - switches off validation; the default for qsub,
qalter, qrsh, qsh and qlogin.
p poke - does not submit the job but prints a validation
report based on a cluster as is with all resource
utilizations in place.
v verify - does not submit the job but prints a validation
report based on a dry scheduling run on an empty cluster,
ignoring load values.
Resource requests exceeding the configured maximal thresholds or
requests for unavailable resource attributes are possible causes
for jobs to fail `v' validation.
Note that the necessary checks are performance consuming and
hence the checking is switched off by default. It should also
be noted that load values are not taken into account with the
verification since they are assumed to be too volatile. To cause
-w e verification to be passed at submission time, it is
possible to specify non-volatile values (non-consumables) or
maximum values (consumables) in complex_values.
If this option, or a corresponding value in qmon, is specified,
then this value will be passed to defined JSV instances as a
parameter with the name w. (See the -jsv option above or find
more information concerning JSV in jsv(1).)
-wd working_dir
Available for qsub, qsh, qrsh and qalter only.
Execute the job from the directory specified in working_dir.
This switch will activate Grid Engine's path aliasing facility,
if the corresponding configuration files are present (see
sge_aliases(5)).
Qalter allows changing this option even while the job executes.
The modified parameter will only be in effect after a restart or
migration of the job, however. The parameter value will be
available in defined JSV instances as parameter with the name
cwd. (See -cwd switch above or find more information concerning
JSV in jsv(1).)
command
Available for qsub and qrsh only.
The job's scriptfile or binary. If not present or if the
operand is the single-character string '-', qsub reads the
script from standard input.
The command will be available in defined JSV instances as
parameter with the name CMDNAME (See -jsv option above or find
more information concerning JSV in jsv(1).)
command_args
Available for qsub, qrsh and qalter only.
Arguments to the job. Not valid if the script is entered from
standard input.
Qalter allows changing this option even while the job executes.
The modified parameter will only be in effect after a restart or
migration of the job, however.
The number of command arguments is provided to configured JSV
instances as parameter with the name CMDARGS. Also the argument
values can by accessed. Argument names have the format
CMDARGnumber where number is a integer between 0 and
CMDARGS - 1. (See -jsv option above or find more information
concerning JSV in jsv(1).)
xterm_args
Available for qsh only.
Arguments to the xterm(1) executable, as defined in the
configuration. For details, refer to sge_conf(5)).
Information concerning xterm_args will be available in JSV
context as parameters with the name CMDARGS and CMDARGnumber.
Find more information above in section command_args. (See -jsv
option above or find more information concerning JSV in jsv(1).)
ENVIRONMENT VARIABLES
SGE_ROOT Specifies the location of the Grid Engine standard
configuration files.
SGE_CELL If set, specifies the default Grid Engine cell. To
address a Grid Engine cell qsub, qsh, qlogin or qalter
use the name of the cell specified in the environment
variable SGE_CELL if that is set, and otherwise the name
of the default cell, i.e. default.
SGE_DEBUG_LEVEL
If set, specifies that debug information should be
written to stderr. In addition the level of detail in
which debug information is generated is defined.
SGE_QMASTER_PORT
If set, specifies the tcp port on which sge_qmaster(8)
is expected to listen for communication requests. Most
installations will use a services map entry for the
service "sge_qmaster" instead to define that port.
DISPLAY For qsh jobs the DISPLAY has to be specified at job
submission. If the DISPLAY is not set by using the
-display or the -v switch, the contents of the DISPLAY
environment variable are used as default.
QRSH_WRAPPER Wrapper for qrsh commands (see above).
SGE_JSV_TIMEOUT
If the response time of the client JSV is greater than
this timeout value, then the JSV will attempt to be re-
started. The default value is 10 seconds, and this value
must be greater than 0. If the timeout has been reached,
the JSV will only try to re-start once, if the timeout
is reached again an error will occur.
In addition to those environment variables specified to be exported to
the job via the -v or the -V option (see above), qsub, qsh, and qlogin
add several variables of the form SGE_O_NAME whose value is taken from
NAME in the submission environment. For each, a variable with the same
value is added to the job context, but with a lower-case version of the
name, i.e. xqs_name_sxx_o_home in the job context corresponds to
xQS_NAME_Sxx_O_HOME in the job environment, and HOME in the submission
environment as follows. The context variables are displayed by
qstat(1) with the -j option.
SGE_O_HOME, SGE_O_HOST, SGE_O_MAIL, SGE_O_PATH, SGE_O_TZ, SGE_O_SHELL, SGE_O_TERM
As in the description of SGE_O_NAME above, but the SGE
clients use a library call to determine to determine the
host name if HOST is not set. Note that there is no
guarantee that these are in any way "correct", e.g. you
should not rely on SGE_O_HOST for any sort of access
control or auditing;
SGE_O_LOGNAME As above, but the corresponding context variable is
sge_log_name.
SGE_O_WORKDIR The absolute path of the current working directory of
the submitting client, reflected in the context as
sge_o_workdir and typically corresponding to the CWD
environment variable.
Grid Engine sets other variables in the job's environment, as follows.
ARC, SGE_ARCH The Grid Engine architecture name of the node on which
the job is running. The name is compiled-in into the
sge_execd(8) binary.
SGE_BINDING Contains the selected operating system internal
processor numbers. There might be more than the number
of selected cores in the presence of SMT or CMT because
each core could be represented by multiple processor
identifiers. The processor numbers are space-separated.
SGE_CKPT_ENV Specifies the checkpointing environment (as selected
with the -ckpt option) under which a checkpointing job
executes. Only set for checkpointing jobs.
SGE_CKPT_DIR Only set for checkpointing jobs. Contains path ckpt_dir
(see checkpoint(5)) of the checkpoint interface.
SGE_CWD_PATH Specifies the current working directory where the job
was started.
SGE_STDERR_PATH
The pathname of the file to which the standard error
stream of the job is diverted. Commonly used for
enhancing the output with error messages from prolog,
epilog, parallel environment start/stop or checkpointing
scripts.
SGE_STDOUT_PATH
The pathname of the file to which the standard output
stream of the job is diverted. Commonly used for
enhancing the output with messages from prolog, epilog,
parallel environment start/stop or checkpointing
scripts.
SGE_STDIN_PATH The pathname of the file from which the standard input
stream of the job is taken. This variable might be used
in combination with SGE_O_HOST in prolog/epilog scripts
to transfer the input file from the submit to the
execution host.
SGE_JOB_SPOOL_DIR
The directory used by sge_shepherd(8) to store job
related data during job execution. This directory is
owned by root or by a Grid Engine administrative
account.
SGE_TASK_ID The index number of the current array job task (see -t
option above). This is a unique number in each array job
and can be used to reference different input data
records, for example. This environment variable is set
to "undefined" for non-array jobs. It is possible to
change the predefined value of this variable with -v or
-V (see options above).
SGE_TASK_FIRST The index number of the first array job task (see -t
option above). It is possible to change the predefined
value of this variable with -v or -V (see options
above).
SGE_TASK_LAST The index number of the last array job task (see -t
option above). It is possible to change the predefined
value of this variable with -v or -V (see options
above).
SGE_TASK_STEPSIZE
The step size of the array job specification (see -t
option above). It is possible to change the predefined
value of this variable with -v or -V (see options
above).
ENVIRONMENT The ENVIRONMENT variable is set to BATCH to identify
that the job is being executed under Grid Engine
control.
HOME The user's home directory path from the passwd(5)
database.
HOSTNAME The hostname of the node on which the job is running.
JOB_ID A unique identifier assigned by the sge_qmaster(8) when
the job was submitted. The job ID is a decimal integer
in the range 1 to 99999.
JOB_NAME The job name. For batch jobs or jobs submitted by qrsh
with a command, the job name is the basename of the qsub
script/binary filename or the qrsh command. For
interactive jobs it is set to "INTERACTIVE" for qsh
jobs, "QLOGIN" for qlogin jobs and "QRLOGIN" for qrsh
jobs without a command.
This default may be overwritten by the -N option.
JOB_SCRIPT The path to the job script which is executed. The value
can not be overwritten by the -v or -V option.
LOGNAME The user's login name from the passwd database.
NHOSTS The number of hosts in use by a parallel job.
NQUEUES The number of queues allocated for the job (always 1 for
serial jobs).
NSLOTS The number of queue slots in use by a parallel job.
PATH A default shell search path of:
/usr/local/bin:/usr/ucb:/bin:/usr/bin
SGE_BINARY_PATH
The path where the Grid Engine binaries are installed.
The value is the concatenation of the cluster
configuration value binary_path and the architecture
name $SGE_ARCH environment variable.
PE The parallel environment under which the job executes
(for parallel jobs only).
PE_HOSTFILE The path of a file containing the definition of the
virtual parallel machine assigned to a parallel job by
Grid Engine. See the description of the $pe_hostfile
parameter in sge_pe(5) for details on the format of this
file. The environment variable is only available for
parallel jobs.
QUEUE The name of the cluster queue in which the job is
running.
REQUEST Available for batch jobs only.
The request name of a job as specified with the -N
switch (see above) or taken as the name of the job
script file.
RESTARTED This variable is set to 1 if a job was restarted after a
system crash, or after migration of a non-application-
level checkpointing job. It is set to 2 when a job with
application-level checkpointing is restarted with a
checkpoint available. Otherwise its value is 0.
SHELL The user's login shell from the passwd database. Note:
This is not necessarily the shell in use for the job.
TMPDIR The absolute path to the job's temporary working
directory. This is usually of the form
/tmp/$JOB_ID.$TASK_ID.$SGE_CELL).
TMP The same as TMPDIR; provided for compatibility with NQS.
TERM The terminal type imported from the submission session
(only for interactive jobs).
TZ The time zone variable imported from sge_execd(8) if
set.
USER The user's login name from the passwd database.
An additional variable, SGE_JOBEXIT_STAT, is set to the job exit status
in the epilog only (see also sge_conf(5)).
RESTRICTIONS
Without -pty y, there is no controlling terminal for batch jobs under
Grid Engine, and any tests or actions on a controlling terminal will
fail. If these operations are in your .login or .cshrc file, they may
cause your job to abort.
Insert the following test before any commands that are not pertinent to
batch jobs in your .login:
if ( $?JOB_NAME) then
echo "Grid Engine spooled job"
exit 0
endif
or in your .profile:
if [ ! -z "$JOB_NAME" ] then
echo "Grid Engine spooled job"
exit 0
fi
Don't forget to set your shell's search path in your shell start-up
before this code.
EXIT STATUS
The following exit values are returned:
0 Operation was executed successfully.
25 It was not possible to register a new job according to the
configured max_u_jobs or max_jobs limit. Additional information
may be found in sge_conf(5)
>0 Error occurred.
EXAMPLES
The following is the simplest form of a Grid Engine script file.
#!/bin/sh
./a.out
The next example is a more complex Grid Engine script.
#!/bin/sh
# Which account to be charged cpu time
#$ -A santa_claus
# date-time to run, format [[CC]yy]MMDDhhmm[.SS]
#$ -a 12241200
# to run I want 6 or more parallel processes
# under the PE pvm. the processes require
# 128M of memory
#$ -pe pvm 6- -l mem=128
# If I run on dec_x put stderr in /tmp/foo, if I
# run on sun_y, put stderr in /usr/me/foo
#$ -e dec_x:/tmp/foo,sun_y:/usr/me/foo
# Send mail to these users
#$ -M santa@nothpole,claus@northpole
# Mail at beginning/end/on suspension
#$ -m bes
# Export these environmental variables
#$ -v PVM_ROOT,FOOBAR=BAR
# The job is located in the current
# working directory.
#$ -cwd
FILES
$REQUEST.oJID[.TASKID] STDOUT of job #JID
$REQUEST.eJID[.TASKID] STDERR of job
$REQUEST.poJID[.TASKID] STDOUT of par. env. of job
$REQUEST.peJID[.TASKID] STDERR of par. env. of job
$cwd/.sge_aliases cwd path aliases
$cwd/.sge_request cwd default request
$HOME/.sge_aliases user path aliases
$HOME/.sge_request user default request
<sge_root>/<cell>/common/sge_aliases
cluster path aliases
<sge_root>/<cell>/common/sge_request
cluster default request
<sge_root>/<cell>/common/act_qmaster
Grid Engine master host file
SEE ALSO
sge_intro(1), qconf(1), qdel(1), qhold(1), qmod(1), qrls(1), qstat(1),
accounting(5), sge_aliases(5), sge_conf(5), sge_request(5), sge_pe(5),
sge_shepherd(8), complex(5), sge_types(5).
COPYRIGHT
If configured correspondingly, qrsh and qlogin contains portions of the
rsh, rshd, telnet and telnetd code copyrighted by The Regents of the
University of California. Therefore, the following note applies with
respect to qrsh and qlogin: This product includes software developed by
the University of California, Berkeley and its contributors.
See sge_intro(1) as well as the information provided in
<sge_root>/3rd_party/qrsh and <sge_root>/3rd_party/qlogin for a
statement of further rights and permissions.
SGE 8.1.3pre 2011-11-27 SUBMIT(1)
Man(1) output converted with
man2html