Web Slinging Mainframe Part Two

The history of this OS dates back to through names like OS/390, MVS ESA, MVS XA,  and continuing back toward MVT etc: http://en.wikipedia.org/wiki/OS/360  The important features we will be looking at today is describing the batch processing side of this OS as we continue to process this web request to restore a user's dataset.

First the term dataset refers to a collection of data but is similar to files on the distributed side and can have more complex internal structure. A single sequential file is common on both platforms. However, z/OS supports partitioned files with a directory of files within the same file. Each file is called a member of the partitioned dataset. For our purpose, we will only be dealing with sequential written files since this is what the FTP created.

Catalogs are an important feature of z/OS and their purpose is locating datasets. In the distributed systems, many OSes use pathing to locate a file on disk. Typically, you need to know which disk and what path.  In z/OS, datasets generally reside on DASD (disk) volumes or tape. The catalog basically contains the dataset name, the device type,  the volume serial number, and some system management class indicators.

The device type or commonly called unit type can have the more esoteric names like disk or tape or more device model specific names like 3390-3. Volume serial name or commonly called volser is the device's name when mounted on the system. The nice thing about a catalog is that you can find the dataset by just knowing it's name. Therefore, these need to be unique names. Other datasets named the same will not be cataloged in this specific catalog. It is possible to catalog them in other catalogs and refer to these catalogs, but this practice is not as common. z/OS allows the dataset name to be a string of 44 characters. Typical naming convention is a maximum of 8 characters separated by periods that form levels. The first set of characters is a level typically called the high level qualifier (HLQ) and can be the name of a userid or a system name for example. The common structure of catalogs are such that the master catalog will contain entries for the system datasets and have alias entries for the HLQ that point to other user catalogs that contain the entries of the user datasets.

The FTP from the web application created on a DASD volume the sequential dataset which contains the character Y, a blank space and the dataset name to be restored for the user. This cataloged dataset name is userid.mvs.request, where userid is the login id the user supplied with a password. This dataset name created by FTP is comprised of the chars "mvs.request" as supplied by the variable in submitter.php and the default current remote directory in the FTP session. If we included the function call ftp_chdir($conn,"sys3") in the submitter.php we could have created a "sys3.mvs.request" dataset instead, and provided this user had the security rights to do so.  A centralized process might want the web app to include a timestamp in the name of the dataset and place them all under a specific level (remote directory) so that it may use this as a processing queue. In our educational example, we will poll for updates to our user's specific named dataset.   

Centralized processes can be an auto operator (program instead of a human submitting requests to do some command or start a batch job) or use a  homegrown routine that periodically runs batch processing. Batch processing is accomplished by submitting a job to be run in the background (asynchronously from the actions of a user in a foreground process). This is facilitated by an address space (task) and tasks (threads) called JES2 (Job Entry Subsystem). This process allows jobs ( a series of programs to run in a batch or command like process) to be queued and run by initiators (scheduling units). The jobs will process their datasets and if any printout (print) is required, it will be spooled (saved on disk) for later scheduling to available requested printers. Users may submit request to JES2 from their foreground session like TSO (Time sharing option with typically text menu panels ISPF) or it can come from a terminal session in an online environment for databases and transaction processing. These jobs need to be formatted in a language that the JES system understands and is called JCL (job control language).

Here is a look at the required JCL to restore this dataset:

//jobname job

//step1 exec pgm=restore

//input dd dsn=userid.mvs.restore,disp=shr

It has one job step and is to execute the program called restore. This program will read the input file named userid.mvs.restore which contains the user's requested dataset name, retrieve the appropriate backup information, and begin the restore.

The job is comprised of the job statement, the exec statement, and any optional dd (data definition) statements. In the dd statement the parameter disp=shr refers to the dataset disposition is share. It can have shared access and kept (not deleted) afterwards.

We could show a real world example of how to use IBM's DFSMShsm or dss products. However, I did not want to include any vendor specific programs and their methods of backup and restore with their JCL requirements just to keep this simple. For this exercise, lets say the system programmer wrote a program that will read the input file and look up its internal kept catalog of backups and retrieve the correct one for the restore. It will do this by dynamically allocating any units of tape or DASD volumes it needs to retrieve the backup. So therefore no dd statement is needed to pre-allocate these devices.

Just like in distributed systems that use AT command in Windows or CHRON is UNIX, JES has automatic command functions (timer functions) that can be used to issue a command that will poll for the user request.  You can use an auto-operator program to work similarly or you could have the web application notify the mainframe programs that an event (request) has occurred. In our case the dataset is just re-written by the web application and it is up to a polling program on z/OS that the system programmer wrote to see that a Y was specified as a new request and then submit the JES2 job above for processing the request.

As you can see there are many ways you can implement a restore task process along with the specific control information that would be needed to fully integrate the web application and the processing requirements on the mainframe. Of course as I indicated before, this exercise was an attempt to help bridge the gaps between these two platforms by showing an example of a two platform application. Perhaps you can think of a much better web application need.

 Once upon a time, system programmers like me actually did write features for their OS. Today many will find that these features are part of products ready to install and implement. What I like about open source is the same reason why we liked "home growing" features we needed. Yes, the two platforms are different. However, the needs are the same by IT facilitating their user community and both teams' love to get their hands on the mechanics of IT.

I hope I kept this short as an introduction as I tried not to bog you down with too many acronyms etc. Keep in touch....