<chapter>
<title>Advanced Features of the Shell</title>
<para>
The basic shell functions were introduced in a prior chapter in order
to provide an understanding of how basic NitrOS-9 commands work. In
this section the more advanced capabilities of the shell are
discussed. In addition to basic command line processing, the shell
has functions that facilitate:
</para>

<itemizedlist mark="square">
<listitem><para>
I/O redirection (including filters)
</para></listitem>
<listitem><para>
Memory Allocation
</para></listitem>
<listitem><para>
Multitasking (concurrent execution)
</para></listitem>
<listitem><para>
Procedure File Execution (background processing)
</para></listitem>
<listitem><para>
Execution Control (built-in commands)
</para></listitem>
</itemizedlist>

<para>
There is a virtually unlimited combination of ways these
capabilities can be used, and it is impossible to give more than a
representative set of examples in this manual. You are therefore
encouraged to study the basic rules, use your imagination, and
explore the possibilities on your own.
</para>

<section>
<title>A More Detailed Description Command Line Processing</title>
<para>
The shell is a program that reads and processes command lines one
at a time from its input path (usually your keyboard). Each line is
first scanned (or &quot;parsed&quot;) in order to identify and process any of
the following parts which may be present:
</para>

<itemizedlist mark="square">
<listitem><para>
A program, procedure file, or built-in command name (&quot;verbs&quot;)
</para></listitem>
<listitem><para>
Parameters to be passed to the program
</para></listitem>
<listitem><para>
Execution modifiers to be processed by the shell
</para></listitem>
</itemizedlist>

<para>
Note that only the verb (the program or command name) need be
present, the other parts are optional. After the verb has been
identified, the shell processes modifiers (if any). Any other text
not yet processed is assumed to be parameters and passed to the program called.
</para>
<para>
Unless the verb is a &quot;built-in command&quot;, the shell will run the
program named as a new process (task). It then deactivates itself
until the program called eventually terminates, at which time it
gets another input line, then the process is repeated. This happens
over and over until an end-of-file condition is detected on the
shell's input path which causes the shell to terminate its own
execution.
</para>
<para>
Here is a sample shell line which calls the assembler:
</para>
<screen>
asm sourcefile l -o &gt;/p #12k
</screen>
<para>
In this example:
</para>

<informaltable frame="none">
<tgroup cols="2">
<colspec colwidth="2in"/>
<colspec colwidth="3in"/>
<tbody>
<row>
<entry>asm</entry>
<entry>is the verb</entry>
</row>
<row>
<entry>sourcefile l -o</entry>
<entry>are parameters passed to <command>asm</command></entry>
</row>
<row>
<entry>&gt;/p</entry>
<entry>is a modifier which redirects the output
(listing) to the system's printer</entry>
</row>
<row>
<entry>#12K</entry>
<entry>is a modifier which requests that the
process be assigned 12K bytes of memory
instead of its (smaller) default amount.</entry>
</row>
</tbody>
</tgroup>
</informaltable>

<para>
The verb must be the first name in the command line. After it
has been scanned, the shell first checks if it is a &quot;built-in&quot;
command. If it is, it is immediately executed. Otherwise, the shell
assumes it is a program name and attempts to locate and execute it.
</para>
</section>

<section>
<title>Execution Modifiers</title>
<para>
Execution modifiers are processed by the shell before the program
is run. If an error is detected in any of the modifiers, the run
will be aborted and the error reported. Characters which comprise
modifiers are stripped from the part(s) of the command line passed
to the program as parameters, therefore, the characters reserved for
use as modifiers ( # ; ! &lt; &gt; &amp; ) cannot be used inside parameters,
but can be used before or after the parameters.
</para>

<section>
<title>Alternate Memory Size Modifier</title>
<para>
When command programs are invoked by the shell, they are
allocated the minimum amount of working RAM memory specified in the
program's module header. A module header is part of all executable
programs and holds the program's name, size, memory requirements,
etc. Sometimes it is desirable to increase this default
memory size. Memory can be assigned in 256-byte pages using the
modifier &quot;#n&quot; where n is the decimal number of pages, or in 1024
byte increments using the modifier &quot;#nK&quot;. The two examples below
behave identically:
</para>
<screen>
OS9: copy #8 file1 file2     (gives 8*256 = 2048 bytes)
OS9: copy #2K file1 file2    (gives 2*1024 = 2048 bytes)
</screen>
</section>

<section>
<title>I/O Redirection Modifiers</title>
<para>
The second kind of modifier is used to redirect the program's
&quot;standard I/O paths&quot; to alternate files or devices. Well-written
NitrOS-9 programs use these paths for routine I/O. Because the programs
do not use specific file or device names, it is fairly simple to
&quot;redirect&quot; the I/O to any file or device without altering the
program itself. Programs which normally receive input from a
terminal or send output to a terminal use one or more of the
standard I/O paths as defined below:
</para>
<para>
STANDARD INPUT: This path normally passes data from the
terminal's keyboard to the program.
</para>
<para>
STANDARD OUTPUT PATH: This path is normally used to output
data from the program to the terminal's display.
</para>
<para>
STANDARD ERROR OUTPUT PATH: This path is used to output
routine status messages such as prompts and errors to the
terminal's display (defaults to the same device as the
standard output path). NOTE: The name &quot;error output&quot; is
sometimes misleading since many other kinds of messages besides
errors are sent on this path.
</para>
<para>
When new processes are created, they inherit their parent process'
standard I/O paths. Therefore, when the shell
creates new processes, they usually inherit its standard I/O paths.
When you log-on the shell's standard input is the terminal keyboard;
the standard output and error output is the terminal's display.
When a redirection modifier is used on a shell command line, the
shell will open the corresponding paths and pass them to the new
process as its standard I/O paths. There are three redirection
modifiers as given below:
</para>
<informaltable frame="none">
<tgroup cols="2">
<colspec colwidth="0.4in"/>
<colspec colwidth="3in"/>
<tbody>
<row>
<entry>&lt;</entry>
<entry>Redirect the standard input path</entry>
</row>
<row>
<entry>&gt;</entry>
<entry>Redirect the standard output path</entry>
</row>
<row>
<entry>&gt;&gt;</entry>
<entry>Redirect the standard error output path</entry>
</row>
</tbody>
</tgroup>
</informaltable>

<para>
When redirection modifiers are used on a command line, they must
be immediately followed by a pathlist describing the file or device
the I/O is to be redirected to or from. For example, the standard
output of <command>list</command> can be redirected to write to the system printer
instead of the terminal:
</para>
<screen>
OS9: list correspondence &gt;/p
</screen>
<para>
Files referenced by I/O redirection modifiers are automatically
opened or created, and closed (as appropriate) by the shell. Here is
another example, the output of the <command>dir</command> command is redirected to the
file <filename>/D1/savelisting</filename>:
</para>
<screen>
OS9: DIR &gt;/D1/savelisting
</screen>
<para>
If the <command>list</command> command is used on the file <filename>/D1/savelisting</filename>, output
from the <command>dir</command> command will be displayed as shown below:
</para>
<screen>
OS9: list /d1/savelisting

   Directory of .   10:15:00
myfile          savelisting       file1
</screen>
<para>
Redirection modifiers can be used before and/or after the program's
parameters, but each modifier can only be used once.
</para>
</section>
</section>

<section id="sec4.3">
<title>Command Separators</title>
<para>
A single shell input line can request execution of more than one
program. These programs may be executed sequentially or
concurrently. Sequential execution means that one program must
complete its function and terminate before the next program is
allowed to begin execution. Concurrent execution means that several
programs are allowed to begin execution and run simultaneously.

</para>

<section id="sec4.3.1">
<title>Sequential Execution</title>
<para>
Programs are executed sequentially when each is entered on a
separate line. More than one program can be specified on a single
shell command line by separating each
<replaceable>program name</replaceable> <replaceable>parameters</replaceable>
from the next one with a &quot;;&quot; character. For example:
</para>
<screen>
OS9: copy myfile /d1/newfile ; dir &gt;/p
</screen>
<para>
This command line will first execute the <command>copy</command> command and then the
<command>dir</command> command.
</para>
<para>
If an error is returned by any program, subsequent commands on
the same line are not executed (regardless of the state of the &quot;x&quot;
option), otherwise, &quot;;&quot; and &quot;return&quot; are identical
separators.
</para>
<para>
Here are some more examples:
</para>
<screen>
OS9: copy oldfile newfile; del oldfile; list newfile

OS9: dir &gt;/d1/myfile ; list temp &gt;/p; del temp
</screen>
<para>
All programs executed sequentially are in fact separate, child
processes of the shell. After initiating execution of a
program to be executed sequentially, the shell enters the &quot;wait&quot;
state until execution of the called program terminates.
</para>
</section>

<section id="sec4.3.2">
<title>Concurrent Execution</title>
<para>
The second kind of separator is the &quot;&amp;&quot; which implies concurrent
execution, meaning that the program is run (as a separate, child
process), but the shell does not wait for it to complete
before processing the next command.
</para>
<para>
The concurrent execution separator is therefore the means by
which multiprogramming (running two or more programs simultaneously)
is accomplished. The number of programs that can run at the same
time is not fixed: it depends upon the amount of free memory in the
system versus the memory requirements of the specific programs.
Here is an example:
</para>
<screen>
OS9: dir &gt;/p&amp;
&amp;007

OS9:
</screen>
<para>
This command line will cause shell to start the <command>dir</command> command
executing, print the process ID number (&amp;007), and then immediately
display the &quot;OS9:&quot; prompt and wait for another command to be
entered. Meanwhile the <command>dir</command> command will be busy sending a directory
listing to the printer. You can display a &quot;status summary&quot; of all
processes you have created by using the <command>procs</command> command. Below is
another example:
</para>
<screen>
OS9: dir &gt;/p&amp; list file1&amp; copy file1 file2 ; del temp
</screen>
<para>

</para>
<para>
Because they were followed by &quot;&amp;&quot; separators, the <command>dir</command>,
<command>list</command>, and
<command>copy</command> programs will run concurrently,
but the <command>del</command> program will not
run until the <command>copy</command> program has terminated because sequential
execution (&quot;;&quot;) was specified.
</para>
</section>

<section id="sec4.3.3">
<title>Pipes and Filters</title>
<para>
The third kind of separator is the &quot;!&quot; character which is used to
construct &quot;pipelines&quot;. Pipelines consist of two or more concurrent
programs whose standard input and/or output paths connect to each
other using &quot;pipes&quot;.
</para>
<para>
Pipes are the primary means-by which data is transferred from
process to process (interprocess communications). Pipes are first-in,
first-out buffers that behave like mass-storage files.
</para>
<para>
I/O transfers using pipes are automatically buffered and
synchronized. A single pipe may have several &quot;readers&quot; and several
&quot;writers&quot;. Multiple writers send, and multiple readers accept, data
to/from the pipe on a first-come, first-serve basis. An end-of-file
will occur if an attempt is made to read from a pipe but there are
no writers available to send data. Conversely, a write error will
occur if an attempt is made to write to a pipe having no readers.
</para>
<para>
Pipelines are created by the shell when an input line having one
or more &quot;!&quot; separators is processed. For each &quot;!&quot;, the standard
output of the program named to the left of the &quot;!&quot; is redirected via
a pipe to the standard input of the program named to the right of
the &quot;!&quot;. Individual pipes are created for each &quot;!&quot; present.
For example:
</para>
<screen>
OS9: update &lt;master_file ! sort ! write_report &gt;/p
</screen>
<para>
In the example above, the program <command>update</command> has its input redirected
from a path called <filename>master_file</filename>. Its standard output becomes the
standard input for the program <command>sort</command>. Its output, in turn, becomes
the standard input for the program <command>write_report</command>, which has its
standard output redirected to the printer.
</para>
<para>
All programs in a pipeline are executed concurrently. The pipes
automatically synchronize the programs so the output of one never
&quot;gets ahead&quot; of the input request of the next program in the
pipeline. This implies that data cannot flow through a pipeline any
faster than the slowest program can process it. Some of the most
useful applications of pipelines are jobs like character set
conversion, print file formatting, data compression/decompression,
etc. Programs which are designed to process data as components of a
pipeline are often called &quot;filters&quot;.
The <command>tee</command> command, which uses
pipes to allow data to be simultaneously &quot;broadcast&quot; from a single
input path to several output paths, is a useful filter.
</para>
</section>
</section>

<section>
<title>Command Grouping</title>
<para>
Sections of shell input lines can be enclosed in parentheses
which permits modifiers and separators to be applied to an entire
set of programs. The shell processes them by calling itself
recursively (as a new process) to execute the enclosed program list.
For example:
<screen>
OS9: (dir /d0; dir /d1) &gt;/p
</screen>
gives the same result as:
<screen>
OS9: dir /d0 &gt;/p; dir /d1 &gt;/p
</screen>
except for the subtle difference that the printer is &quot;kept&quot;
continuously in the first example; in the second case another user
could &quot;steal&quot; the printer in between the <command>dir</command> commands.
</para>
<para>
Command grouping can be used to cause a group of programs to be
executed sequentially, but also concurrently with respect to the
shell that initiated them, such as:
</para>
<screen>
OS9: (del file1; del file2; del file3)&amp;
</screen>
<para>
A useful extension of this form is to construct pipelines consisting
of sequential and/or concurrent programs. For example:
</para>
<screen>
OS9: (dir CMDS; dir SYS) ! makeuppercase ! transmit
</screen>
<para>
Here is a very practical example of the use of pipelines. Recall
that the <command>dsave</command> command generates a procedure file to copy all the
files in a directory. The example below shows how the output of
<command>dsave</command> can be pipelined to a shell which will execute the NitrOS-9
commands as they are generated by <command>dsave</command>. Assume that we want to
copy all files from a directory called <filename class="directory">WORKING</filename> to a directory called
<filename class="directory">ARCHIVE</filename>:
</para>
<screen>
OS9: chd /d0/WORKING; dsave /d0/ARCHIVE ! shell -p
</screen>
</section>

<section id="sec4.5">
<title>Built-in Shell Commands and Options</title>
<para>
When processing input lines, the shell looks for several special
names of commands or option switches that are built-in the shell.
These commands are executed without loading a program and creating a
new process, and generally affect how the shell operates. They can
be used at the beginning of a line, or following any program
separator (&quot;;&quot;, &quot;&amp;&quot;, or &quot;!&quot;).
Two or more adjacent built-in
commands can be separated by spaces or commas.
</para>
<para>
The built-in commands and their functions are:
</para>
<informaltable frame="none">
<tgroup cols="2">
<colspec colwidth="1.5in"/>
<colspec colwidth="3.5in"/>
<tbody>
<row>
<entry><command>chd</command> <replaceable>pathlist</replaceable></entry>
<entry>change the working data directory to the directory
specified by the pathlist.</entry>
</row>
<row>
<entry><command>chx</command> <replaceable>pathlist</replaceable></entry>
<entry>change the working execution directory to the
directory specified by the pathlist.</entry>
</row>
<row>
<entry><command>ex</command> <replaceable>name</replaceable></entry>
<entry>directly execute the module named. This
transforms the shell process so it ceases
to exist and a new module begins execution in
its place.</entry>
</row>
<row>
<entry><command>w</command></entry>
<entry>wait for any process to terminate.</entry>
</row>
<row>
<entry>* <replaceable>text</replaceable></entry>
<entry>comment: &quot;text&quot; is not processed.</entry>
</row>
<row>
<entry><command>kill</command> <replaceable>Proc ID</replaceable></entry>
<entry>abort the process specified.</entry>
</row>
<row>
<entry><command>setpr</command> <replaceable>Proc ID</replaceable> <replaceable>priority</replaceable></entry>
<entry>changes process' priority.</entry>
</row>
<row>
<entry>x</entry>
<entry>causes shell to abort on any error (default)</entry>
</row>
<row>
<entry>-x</entry>
<entry>causes shell not to abort on error</entry>
</row>
<row>
<entry>p</entry>
<entry>turns shell prompt and messages on (default)</entry>
</row>
<row>
<entry>-p</entry>
<entry>inhibits shell prompt and messages</entry>
</row>
<row>
<entry>t</entry>
<entry>makes shell copy all input lines to output</entry>
</row>
<row>
<entry>-t</entry>
<entry>does not copy input lines to output (default)</entry>
</row>
</tbody>
</tgroup>
</informaltable>


<para>
The change directory commands switch the shell's working directory
and, by inheritance, any subsequently created child process. The
<command>ex</command> command is used where the shell is needed to initiate execution
of a program without the overhead of a suspended <command>shell</command> process.
The name used is processed according to standard shell operation,
and modifiers can be used.
</para>
</section>

<section id="sec4.6">
<title>Shell Procedure Files</title>
<para>
The shell is a reentrant program that can be simultaneously
executed by more than one process at a time. As is the case with
most other NitrOS-9 programs, it uses standard I/O paths for routine
input and output. specifically, it requests command
lines from the standard input path and writes its prompts and other
data to the standard error path.
</para>
<para>
The shell can start up another process also running the shell by
means of the <command>shell</command> command. If the standard input path is
redirected to a mass storage file, the new &quot;incarnation&quot; of the
shell can accept and execute command lines from the file instead of
a terminal keyboard. The text file to be processed is
called a &quot;procedure file&quot;. It contains one or more command lines
that are identical to command lines that are manually entered from
the keyboard. This technique is sometimes called &quot;batch&quot; or
&quot;background&quot; processing.
</para>
<para>
If the <replaceable>program name</replaceable> specified on a shell command line can not be
found in memory or in the execution directory, shell will search the
data directory for a file with the desired name. If one is found,
shell will automatically execute it as a procedure file.
</para>
<para>
Execution of procedure files have a number of valuable
applications. It can eliminate repetitive manual entry of commonly-used
sequences of commands. It can allow the computer to execute a
lengthy series of programs &quot;in the background&quot; while the computer is
unattended or while the user is running other programs &quot;in the
foreground&quot;.
</para>
<para>
In addition to redirecting the shell's standard input to a
procedure file, the standard output and standard error output can be
redirected to another file which can record output for later review
or printing. This can also eliminate the sometimes-annoying output
of shell messages to your terminal at random times.
</para>
<para>
Here are two simple ways to use the shell to create another
shell:
</para>
<screen>
OS9: shell &lt;procfile

OS9: procfile
</screen>
<para>
Both do exactly the same thing: execute the commands of the file
<filename>procfile</filename>. To run the procedure file in a &quot;background&quot; mode you
simply add the ampersand operator:
</para>
<screen>
OS9: procfile&amp;
</screen>
<para>
NitrOS-9 does not have any constraints on the number of jobs that can be
simultaneously executed as long as there is memory available. Also, the procedure files can themselves cause sequential or
concurrent execution of additional procedure files. Here's a more
complex example of initiating two processing streams with
redirection of each shell's output to files:
</para>
<screen>
OS9: proc1 T &gt;&gt;stat1&amp; proc2 T &gt;&gt;stat2&amp;
</screen>
<para>
Note that the built-in command &quot;T&quot; (copy input lines to error
output) was used above. They make the output file contain a record
of all lines executed, but without useless &quot;OS9&quot; prompts intermixed.
The &quot;-x&quot; built-in command can be used if you do
<emphasis>not</emphasis> want processing
to stop if an error occurs. Note that the built-in commands only
affect the shell that executes them, and not any others that may
exist.
</para>
</section>

<section id="sec4.7">
<title>Error Reporting</title>
<para>
Many programs (including the shell) use NitrOS-9's standard error
reporting function, which displays an error number on the error
output path. The standard error codes are listed in the <xref linkend="errorcodes"/> of
this manual. If desired, the <command>printerr</command> command can be executed,
which replaces the smaller, built-in error display routine with a
larger (and slower) routine that looks up descriptive error messages
from a text file called <filename>/dd/sys/errmsg</filename>.
Once the <command>printerr</command>
command has been run it cannot be turned off. Also, its effect is
system-wide.
</para>
<para>
Programs called by the shell can return an error code in the CPU's
&quot;B&quot; register (otherwise B should be cleared) upon termination. This
type of error, as well as errors detected by the shell itself, will
cause an error message to be displayed and processing of the command
line or procedure file to be terminated unless the &quot;-x&quot; built-in
command has been previously executed.
</para>
</section>

<section id="sec4.8">
<title>Running Compiled Intermediate Code Programs</title>
<para>
Before the shell executes a program, it checks the program
module's language type. If its type is not &CPU; machine language,
shell will call the appropriate run-time system for that module.
Versions of the shell supplied for various systems are capable of
calling different run-time systems. Most versions of shell call
Basic09 when appropriate, and Level Two versions of shell can also
call the Pascal P-code interpreter (PascalN), or the CIS Cobol
runtime system (RunC).
</para>
<para>
For example, if you wanted to run a Basic09 I-code module called
<command>adventure</command>, you could type the command given below:
</para>
<screen>
OS9: basic09 adventure
</screen>
<para>
Or you could accomplish the same thing by typing the following:
</para>
<screen>
OS9: adventure
</screen>
</section>

<section id="sec4.9">
<title>Setting Up Timesharing System Procedure Files</title>

<para>
NitrOS-9 systems used for timesharing usually have a procedure file
that brings the system up by means of one simple command or by using
the system <filename>startup</filename> file. A procedure file which initiates the
timesharing monitor for each terminal is executed to start up the
system. The procedure file first starts the system clock, then
initiates concurrent execution of a number of processes that have
their I/O redirected to each timesharing terminal.
</para>
<para>
Usually one <command>tsmon</command> command program is started up concurrently for
each terminal in the system. This is a special program which
monitors a terminal for activity. When a carriage return character
is typed on any of these terminals, the <command>tsmon</command> command initiates the
<command>login</command> command program. If a user does not enter a correct password
or user number in three tries, the <command>login</command> command will be aborted.
Here's a sample procedure file for a 4-terminal timesharing system
having terminals names <quote>TERM</quote>, <quote>T1</quote>,
<quote>T2</quote>, and <quote>T3</quote>.
</para>
<programlisting>
* system startup procedure file
echo Please Enter the Date and Time
setime &lt;/term
printerr
tsmon /t1&amp;
tsmon /t2&amp;
tsmon /t3&amp;
</programlisting>
<para>
NOTE: This <command>login</command> procedure will not work until a password file
called <filename>/DD/SYS/PASSWORD</filename> has been created. For more information,
please see the <command>login</command> command description.
</para>
<para>
The example above deserves special attention. Note that the
<command>setime</command> command has its input redirected to the system console
&quot;term&quot;, which is necessary because it would otherwise attempt to
read the time information from its current standard input path,
which is the procedure file and not the keyboard.
</para>
</section>
</chapter>