SCHEDULE
Automated Job Submission System
Guide and Reference Manual

A job that does not have any script commands associated with it is processed through the system in a slightly different fashion from one that does. Such a job is never entered into an execution batch queue. These jobs are marked as having successfully completed whenever execution is requested.

This type of job is very useful when a more complex scheduling mode is needed than what is provided by a single job. For example, two data files are required before a job can execute. A single job can only wait for a single file. In this case two jobs are defined each one waiting for one file.

The first job initiates the second job. The first job has no script commands associated with it. All the job commands are part of the second job. The first job will never be entered into a execution batch queue. It will however proceed through the various scheduling queues in a normal fashion. Once both files have been found the second job will be entered into a execution batch queue.

2.1.5 Submit attributes

Associated with each job is a list of submit attributes. These are all the options that can be used with the bg command. When a job is ready for execution it is submitted into a batch execution queue. All the following attributes can be specified.

Name	Description
NODE	The node or cluster that the job is to execute on.
SHELL	The UNIX shell from which the script commands will be issued. Examples are csh (C shell), ksh (Korn shell) and sh (Borne shell).
FLAGS	Any submit flags that are set for the job such as DELETE (Delete temporary shell script for job) and KEEP(Indicates that the job log file will be saved when the job is finished).
LOG FILE	The location to place the log file can be provided. The default location is the user's root and the default name is the name of the job.
NAME	Name specifies the name to use when entering the job into a background proces. The default value is to use the job name.
PARAMETER	Any or all of the P1 to P8 parameters can be specified for the job.
RESIDENT SET SIZE	The amount of physical memory that the job can use.
CPUTIME	The CPU time limit if any on the job.

For example to specify an execution queue priority of 150 use the following type of command.

Schedule> chjob ajob -submit=priority:150

2.1.6 Notices and Notifications

There are six types of events that trigger notification processing. The events are listed below:

Event	Description
SUBMITTED	job was submitted for execution
STARTED	job has started execution
DELAYED	job is currently behind schedule
COMPLETED	job has completed execution
FAILED	job execution has failed
OTHER	other problems have occurred

In addition, there are custom strings that can be added to the notification list that control the way notifications are processed. These are:

String	Description
EMAIL="address<,address>"	allows specification of additional e-mail addresses to send mail messages to. The list can be comma separated, or you can add multiple EMAIL= lines in the file.
BROADCAST="user<,user>"	allows specification of additional user opcoms/net sends/terminal messages. The list can be comma separated, or you can add multiple BROADCAST= lines in the file. MESSENGER= can be used as well.
ATTACHMENT="file"	allows the user to specify a file to attach and send with the email. Only one is allowed.
MAIL_SUBJECT="subject"	allows the user to customize the mail message subject. They can incorporate environment variables into the control string. The user would need to look at the notify script for a full list of the environment variables defined. The environment variables referenced in the MAIL_SUBJECT must be in the format required.
SEND_LOG	available for completed and failed events, and simply appends the most recent log file to the end of the mail message. It is ignored for other event types.
INCLUDE="text"	allows another job's notice text to be included in the current job's notice section at runtime. This is similar to the INCLUDE= functionality used in resource processing. Basically it allows the definitions that are used in multiple locations to be centralized.

Separate or common text can be provide for all six events. The general format of the notices is detailed below:

text common to all messages BEGIN_event text for this event END_event BEGIN_event text for this event END_event BEGIN_COMMENT local notes, not included in any message END_COMMENT

A typical notice list can be displayed by using the following command:

Schedule> more -notices /demo/a/update_a /demo/a/update_a.notices This job is the primary database update job. BEGIN_FAILED ***** Please call John Smith at extension 2345 immediately ***** END_FAILED BEGIN_DELAYED Please change the job priority so that this update can complete as quickly as possible. END_DELAYED

Transmission of the actual notification message is done via a batch job. When the event occurs all the conditions are captured and placed in a temporary command file. This file is then submitted to a batch execution queue for processing. The last line of this file is an "$@NOTIFY_COM_FILE". The name of this file and the attributes of how this job is submitted is controlled by five parameters which are defined in the parameter file. ³

The controlling parameters are listed below:

Parameter	Default	Description
NOTIFY_QUEUE	SYS$BATCH	batch queue to use
NOTIFY_USER	SYSTEM	user name to use
NOTIFY_ENABLE	1	a 1 if notification enabled
NOTIFY_PRIORITY	200	queue priority to use
NOTIFY_FILE	SCHEDULE_NOTIFY	command file to use

The standard notify procedure SCHEDULE_NOTIFY.COM that is provided with the system can send messages either via a REPLY statement or via a MAIL statement. If your system has other notification requirements you can provide your own customized notification procedure by changing the corresponding parameter.

For example, to specify that a particular user is to received mail for all FAILED and DELAYED events use the following type of command.

Schedule> chjob my_job -notify=(delayed=(smith,mail,broadcast),- failed=(smith,mail))

The target user name can be anything that is acceptable to MAIL. For example, to send the messages to two persons use the following command.

Schedule> chjob my_job -notify=(delayed=((johnson,smith),mail,broadcast),- failed=(smith,mail))

The total number of characters in the user name(s) is limited to 32 characters. The user name can also be a environment variable that in turn translates out to a list of user names. For example, if the job is to inform a group of users that are defined via an environment variable use the following commands.

UNIX> setenv operations johnson,smith,anderson,system Schedule> chjob my_job -notify=(delayed=(operations,mail,broadcast),- failed=(operations,mail))

2.1.7 Initiates

Figure 2-1 Initiates Job Example

For example in Figure 2-1 when Job A completes, Job B is readied and is immediately submitted for execution. When Job B completes Jobs C and D are ready and are submitted immediately for execution.

Each initiate branch can be conditioned only to occur under certain circumstances. The conditions are based on the completion code of the job. The severity level (success, warning, error and fatal) or the numeric value of the code can be specified.

The initiates are entered in a list that can be specified with the job entry in the database.

Each line in this list specifies a single job and under what conditions it should be initiated. The format of each line is detailed below:

General format: /directory/job_name [,condition,...] [,ONE_OF] the job name can be any of the following: job /directory/job /directory/subdirectory/job /subdirectory/job the conditions are one or more of the following: ANY - start this job always (default) SUCCESS - only start if job finished successfully WARNING - only start if job finished with a warning INFORMATIONAL - only start if job finished with informational message ERROR - only start if job finished with an error FATAL - only start if job finished with a fatal error CODE=n - start if status was "n" IF CONDITION - only start if condition is met

The default condition is ANY. If no conditions are specified then it is always started whether the job succeeded or failed. For a description of all conditional operators, see Section 3.1.4.

The ONE_OF control word indicates that only the first job of all the ones marked with ONE_OF that has a matching condition will be initiated. Any of the pieces of the name that are left off are picked up by default from the current job. A typical initiate list can be displayed by using the following command:

Schedule> more -initiate /demo/a/update_a /demo/a/update_a.initiates IF JOB_CHECK == FAILED THEN UPDATE_A_FAILED,ERROR,FATAL REPORT_1,SUCCESS,WARNING SUMMARY_A,SUCCESS,WARNING ERROR_4,CODE=18934028

The addition of a node name to a job name indicates that the system is to send a message to that remote node requesting that the job be initiated on that node. This is used in a "peer-to-peer" network configuration where another system (or cluster) has its own scheduling database and job synchronization is needed between these two independent systems.

As one job initiates another job three bits of information are sent along. They are the SET_ID, SET_PARAMETER, and SET_TAG. Of these the SET_ID is the most important. The SET_ID identifies a job as being a member of a set. Only jobs of the same set can be connected by prerequisites conditions.

2.1.8 Activated by

The activated-by list that is associated with each job in the SCHEDULE database specifies all jobs that reference this job on their initiate lists. The activated-by list is the inverse of the initiate list. This list can be manually updated if needed. Normally only the initiate list is modified and the system automatically updates the activated-by list.

The SCHEDULE REBUILD command can be used to recreate the activated-by and held-up-by lists for all jobs.

2.1.9 Prerequisites

A job can have any number of prerequisite jobs. A job cannot be submitted into a queue until all prerequisite jobs have completed successfully.

Figure 2-2 Prerequisite job example

For example Job D cannot be submitted until Jobs B and C have completed successfully. Job B cannot be submitted until Job A has completed successfully.

The prerequisites are entered in a list that can be associated with any job entry in the database. All the jobs on this list must complete successfully before the job is allowed to proceed.

Each line of this list specifies one job either on the local system or on a remote system. The format of each line is detailed below:

General format: /directory/job_name [IF condition] The job name can be any of the following: job /directory/job /directory/subdirectory/job /subdirectory/job

Any of the pieces of the name that are left off are picked up by default from the current job. A typical prerequisite list can be displayed by using the following command:

Schedule> more -prerequisites /demo/a/report_3 /demo/a/report_3.prerequisites IF COUNT .NE. 4 THEN UPDATE_C UPDATE_B UPDATE_A

The addition of a node name to a job name indicates that the system is to send a message to that remote node requesting a return message when the prerequisite job completes. This is used in a "peer-to-peer" network configuration where another system (or cluster) has its own scheduling database and job synchronization is needed between these two independent systems.

One very important point is that the SET_ID is used to clarify which job is a prerequisite job. In other words if a job C has jobs A and B as prerequisites they must have completed successfully and be members of the same set (i.e. have the same SET_ID) as job C. The mechanism that can be used to propagate the SET_ID is the initiate list. In this example all three jobs A, B, and C should be initiate by the same parent job.

2.1.10 Held up by

The held-up-by list that is associated with each job in the SCHEDULE database specifies all jobs that reference this job on their prerequisite list. The held-up-by list is the inverse of the prerequisite list. This list can be manually updated if needed. Normally only the prerequisite list is modified and the system automatically updates the held-up-by list.

The SCHEDULE REBUILD command can be used to recreate the activated-by and held-up-by lists for all jobs.

2.1.11 Distrib list

The distribution list is a list of PRINT commands to be issued. The idea is to provide a central mechanism for maintaining the `how many, where to and what printer' information in a common location separate from the main command stream.

Each statement appears on a separate line in the notice list. The format of each line is detailed below:

General format: command [-options] 'Pn' where: command - a single command qualifiers - can be any group of qualifiers Pn - any parameter P1...P8

The command can be any command. The more common ones would be lpr, mail, or cp. A typical distribution list can be displayed by using the command:

Schedule> more -distribution /demo/a/report_3 /demo/a/report_3.distribution unix> -queue=special/form=landscape 'p1' unix> lpr -copies=2/pages=(1,3) 'p1' unix> mail 'p1' system_manager

The distribution processing is invoked by using the following type of command.

unix> schedule_distribute 'schedule_full_name' report_1 [report_2...]

The above command is inserted into the main body of commands after the report files have been produced.

One useful variation on this is to set aside a special job that is then used by many other jobs to control how they distribute printouts. For example, create a job called ACCOUNTING_DEPARTMENT. In this job just set up a distribution list. In all jobs that generate reports for the accounting department just enter the following command.

unix> schedule_distribute accounting_department report_file

In this way if a change is made to the distribution list an edit operation only needs to be done in one place.