History of IBM mainframe operating systems

Early time-sharing and virtual machine systems

MIT's Fernando Corbató produced the first experimental time-sharing systems, such as CTSS, from 1957 to the early 1960s, using slightly modified IBM 709, IBM 7090, and IBM 7094 mainframes; these systems were based on a proposal by John McCarthy. In the 1960s IBM's own laboratories created experimental time-sharing systems, using standard mainframes with hardware and microcode modifications to support virtual memory: IBM M44/44X in the early 1960s; CP-40 from 1964 to 1967; CP-67 from 1967 to 1972. The company even released CP-67 without warranty or technical support to several large customers from 1968 to 1972. CP-40 and CP-67 used modified System/360 CPUs, but the M44/44X was based on the IBM 7044, an earlier generation of CPU which was very different internally.

These experimental systems were too late to be incorporated into the System/360 series which IBM announced in 1964 but encouraged the company to add virtual memory and virtual machine capabilities to its System/370 mainframes and their operating systems in 1972:

• The M44/44X showed that a partial approach to virtual machines was not good enough and that thrashing could severely reduce the speed of virtual memory systems. Thrashing is a condition in which the system runs very slowly because it spends a lot of its time shuffling virtual memory pages between physical memory and disk files.
• IBM learned from CP-40 and CP-67: how to make the thrashing problem manageable; that its other virtual memory and virtual machine technologies were sufficiently fast and reliable to be used in the high-volume commercial systems which were its core business. In particular, IBM's David Sayre convinced the company that automated virtual memory management could consistently perform at least as well as the best programmer-designed overlay schemes.

In 1968 a consulting firm called Computer Software Systems used the released version of CP-67 to set up a commercial time-sharing service. The company's technical team included 2 recruits from MIT (see CTSS above), Dick Orenstein and Harold Feinleib. As it grew, the company renamed itself National CSS and modified the software to increase the number of paying users it could support until the system was sufficiently different that it warranted a new name, VP/CSS. VP/CSS was the delivery mechanism for National CSS' services until the early 1980s, when it switched to IBM's VM/370 (see below).

Universities produced three other S/360 time-sharing operating systems in the late 1960s:

• The Michigan Terminal System (MTS) was developed in 1967 by a consortium of universities led by the University of Michigan. All versions ran on IBM's mainframes which had virtual memory capability, starting with the S/360-67. MTS remained in use until 1999.
• McGill University in Montreal started developing MUSIC (McGill University System for Interactive Computing) in 1969. MUSIC was enhanced several times and eventually supported text searches, web publishing and email as well as software development. MUSIC was marketed by IBM mainly to educational institutions as a cost-effective operating system for its hardware, and eventually became an IBM Systems Product (MUSIC/SP or Multi-User System for Interactive Computing / System Product) in 1985. The last official version was released in 1999.
• ORVYL and WYLBUR were developed by Stanford University in 1967-68 for the IBM S/360-67. They provided some of the first time-sharing capabilities on IBM S/360 computers.

System/360 operating systems

Up to the early 1960s, IBM's low-end and high-end systems were incompatible, so programs could not easily be transferred from one to another, and the systems often used completely different peripherals such as disk drives. IBM concluded that these factors were increasing its design and production costs for both hardware and software to a level that was unsustainable, and were reducing sales by deterring customers from upgrading. So in 1964, the company announced System/360, a new range of computers which all used the same peripherals and most of which could run the same programs.

IBM originally intended that System/360 should have only one batch-oriented operating system, OS/360. There are at least two accounts of why IBM later decided it should also produce a simpler batch-oriented operating system, DOS/360:

• because it found that OS/360 would not fit into the limited memory available on the smaller System/360 models;
• or because it realized that the development of OS/360 would take much longer than expected, and introduced DOS/360 as one of a series of stop-gaps to prevent System/360 hardware sales from collapsing – the others were BOS/360 (Basic Operating System, for the smallest machines) and TOS/360 (Tape Operating System, for machines with only tape drives).

System/360's operating systems were more complex than previous IBM operating systems for several reasons, including:

• They had to support multiprogramming – switching to run another in-progress application when the current application was blocked waiting for I/O operations (such as disk reads) to complete. Without multiprogramming, the faster CPUs in the range would have spent most of their time idle, waiting for slow I/O operations. Hence, the operating systems had to be the real masters of the systems, to provide whatever services the applications validly requested, and to handle crashes or misbehavior in one application without stopping others that were running at the same time.
• They had to support a much wider range of machine sizes. Memory ranged from 16 KB to 1 MB and processor speeds from a few thousand instructions per second to 500,000.
• They had to support a wide range of application requirements. For example, some applications only needed to read through sequential files from start to finish; others needed fast, direct access to specific records in very large files; and a few applications spent nearly all their time doing calculations, with very little reading or writing of files.

This made the development of OS/360 and other System/360 software one of the largest software projects anyone had attempted, and IBM soon ran into trouble, with huge time and cost overruns and large numbers of bugs. These problems were only magnified because to develop and test System/360 operating systems on real hardware, IBM first had to develop Basic Programming Support/360 (BPS/360). BPS was used to develop the tools needed to develop DOS/360 and OS/360, as well as the first versions of tools it would supply with these operating systems – compilers for FORTRAN and COBOL, utilities including Sort, and above all the assembler it needed to build all the other software.

IBM's competitors took advantage of the delays in OS/360 and the System/360 to announce systems aimed at what they thought were the most vulnerable parts of IBM's market. To prevent sales of System/360 from collapsing, IBM released four stop-gap operating systems:

• Basic Operating System/360 (BOS/360), which loaded from a disk drive or tape drive and supported tape drives and a few disks. This system was supplied to beta test customers and may have been an early version of DOS/360.
• TOS/360, which was designed to provide an upgrade path for customers who had IBM 1401 computers with tape drives and no disks.
• DOS/360, which was built by the developers of BOS/360 and TOS/360 (IBM's small business computers division) and went on to become a mainstream operating system whose descendant z/VSE is still widely used.
• Operating System/360 (OS/360) with only the Primary Control Program (PCP) option, which didn't support multiprogramming.

When IBM announced the S/360-67 it also announced a timesharing operating system, TSS/360, that would use the new virtual memory capabilities of the 360/67. TSS/360 was late and early releases were slow and unreliable. By this time the alternative operating system CP-67, developed by IBM's Cambridge Scientific Center, was running well enough for IBM to offer it "without warranty" as a timesharing facility for a few large customers. CP-67 would go on to become VM/370 and eventually z/VM. IBM ultimately offered three releases of a TSS/370 PRPQ as a migration path for its TSS/360 customers, and then dropped it.

The traumas of producing the System/360 operating systems gave a boost to the emerging discipline of software engineering, the attempt to apply scientific principles to the development of software, and the management of software projects. Frederick P. Brooks, who was a senior project manager for the whole System/360 project and then was given specific responsibility for OS/360 (which was already long overdue), wrote an acclaimed book, The Mythical Man-Month, based on the problems encountered and lessons learned during the project, two of which were:

• Throwing additional resources (especially staff) at a struggling project quickly becomes unproductive or even counter-productive because of communication difficulties. This is the "Mythical Man-Month" syndrome which gave the book its title.
• The successor to a successful system often runs into difficulties because it gets overloaded with all the features people wished had been in the earlier system. Brooks called this the "second-system effect", and cited OS/360 as a very comprehensive example.

DOS/360

While OS/360 was the preferred operating system for the higher-end System/360 machines, DOS/360 was the usual operating system for the less powerful machines. It provided a set of utility programs, a macro assembler, and compilers for FORTRAN and COBOL. Support for RPG came later, and eventually a PL/I subset. And it supported a useful range of file organizations with access methods to help in using them:

• Sequential data sets were normally read one record at a time from beginning to end.
• In indexed (ISAM) files a specified section of each record was defined as a key that could be used to look up specific records.
• In direct access (BDAM) files, the application program had to specify the physical location on the disk of the data it wanted to access. BDAM programming was not easy and most customers never used it themselves, but it was the fastest way to access data on disks and many software companies used it in their products, especially database management systems such as ADABAS, IDMS and IBM's DL/I.

Sequential and ISAM files could store either fixed-length or variable-length records, and all types could occupy more than one disk volume.

DOS/360 also offered BTAM, a data communications facility that was primitive and hard to use by today's standards. But BTAM could communicate with almost any type of terminal, which was a big advantage at a time when there was hardly any standardization of communications protocols.

But DOS/360 had significant limitations compared with OS/360, which was used to control most larger System/360 machines:

• The first version could run only one program at a time. A later enhancement allowed 3 at the same time, in one of 3 "partitions" whose size was set by each customer when DOS/360 was installed.
• The JCL it used for submitting jobs was designed to be easy for the low-end machines to process, and as a result, programmers did not find it easy to read or write.
• There was no spooling sub-system to improve the efficiency of punched card and printer use. In the late 1960s, an independent software company started selling a spooler called GRASP.
• DOS/360 had no relocating loader, so users had to link edit a separate executable version of each program for each partition in which the program was likely to be run.
• Executable programs were stored in the Core Image Library, which did not reclaim space when programs were deleted or replaced by newer versions. When the Core Image Library became full, it had to be compressed by one of the utility programs, and this could halt development work for as much as half a day.
• Its application programming interface was different from that of OS/360. DOS/360 programs written in high level languages such as COBOL needed small modifications before they could be used with OS/360 and assembly language programs needed larger changes.

IBM expected that DOS/360 users would soon upgrade to OS/360, but despite its limitations, DOS/360 became the world's most widely used operating system because:

• System/360 hardware sold very well
• Over 90% of the 360 systems sold were Models 20, 30 & 40
• Most of these cheaper models had far less core memory than required by OS/360.

DOS/360 ran well on the System/360 processors which medium-sized organizations could afford, and it was better than the "operating systems" these customers had before. As a result, its descendant z/VSE is still widely used today, as of 2005.

OS/360

OS/360 included multiple levels of support, a single API, and much shared code. PCP was a stop-gap version that could run only one program at a time, but MFT ("Multiprogramming with a Fixed number of Tasks") and MVT ("Multiprogramming with a Variable number of Tasks") were used until at least the late 1970s, a good five years after their successors had been launched. It is unclear whether the divisions among PCP, MFT and MVT arose because MVT required too much memory to be usable on mid-range machines or because IBM needed to release a multiprogramming version of OS (MFT) as soon as possible.

PCP, MFT, and MVT had different approaches to managing memory (see below), but provided very similar facilities:

• The same application programming interface (API), so application programs could be transferred among PCP, MFT, and MVT without even needing re-compilation.
• The same JCL, which was more flexible and easier to use than that of DOS/360.
• The same facilities (access methods) as DOS/360 for reading and writing files (sequential, indexed, and direct) and for data communications (BTAM).
• An additional file structure, partitioned, and access method (BPAM), which was mainly used for managing program libraries. Although partitioned files needed to be compressed to reclaim free space, this seldom halted development work as it did with DOS/360's Core Image Library, because PCP, MFT, and MVT allowed an indefinite number of partitioned files, and each project generally had at least one.
• A file naming system allowing files to be managed as hierarchies, such as PROJECT.USER.FILENAME.
• A spooling facility (which DOS/360 lacked).
• Applications could create sub-tasks, which allowed multiprogramming within the one job.

Experience indicated that it was not advisable to install OS/360 on systems with less than 256 KB of memory, which was a common limitation in the 1960s.

MFT

When installing MFT, customers would specify up to four partitions of memory with fixed boundaries, in which application programs could be run simultaneously. MFT Version II (MFT-II) raised the limit to 52.

MVT

MVT was considerably larger and more complex than MFT and therefore was used on the most powerful System/360 CPUs. It treated all memory not used by the operating system as a single pool from which contiguous "regions" could be allocated as required by an indefinite number of simultaneous application programs. This scheme was more flexible than MFT's and in principle used memory more efficiently, but was liable to fragmentation – after a while one could find that, although there was enough spare memory in total to run a program, it was divided into separate chunks none of which was large enough.

In 1971 the Time Sharing Option (TSO) for use with MVT was added. TSO became widely used for program development because it provided: an editor, debuggers for some of the programming languages used on System/360, and the ability to submit batch jobs, be notified of their completion, and view the results without waiting for printed reports. TSO communicated with terminals by using TCAM (Telecommunications Access Method), which eventually replaced the earlier Queued Telecommunications Access Method (QTAM). TCAM's name suggests that IBM hoped it would become the standard access method for data communications, but in fact, TCAM was used almost entirely for TSO and was largely superseded by VTAM from the late 1970s onwards.

TP monitors

System/360's hardware and operating systems were designed for processing batch jobs which in extreme cases might run for hours. As a result, they were unsuitable for transaction processing, in which there are thousands of units of work per day and each takes between 30 seconds and a very few minutes. In 1968 IBM released IMS to handle transaction processing, and in 1969 it released CICS, a simpler transaction processing system which a group of IBM's staff had developed for a customer. IMS was only available for OS/360 and its successors, but CICS was also available for DOS/360 and its successors. For many years this type of product was known as a "TP (teleprocessing) monitor". Strictly speaking TP monitors were not operating system components but application programs which managed other application programs. In the 1970s and 1980s, several third-party TP monitors competed with CICS (notably COM-PLETE, DATACOM/DC, ENVIRON/1, INTERCOMM, SHADOW II, TASK/MASTER and WESTI), but IBM gradually improved CICS to the point where most customers abandoned the alternatives.

Special systems for airlines

In the 1950s airlines were expanding rapidly but this growth was held back by the difficulty of handling thousands of bookings manually (using card files). In 1957 IBM signed a development contract with American Airlines for the development of a computerized reservations system, which became known as SABRE. The first experimental system went live in 1960 and the system took over all booking functions in 1964 – in both cases using IBM 7090 mainframes. In the early 1960s IBM undertook similar projects for other airlines and soon decided to produce a single standard booking system, PARS, to run on System/360 computers.

In SABRE and early versions of PARS there was no separation between the application and operating system components of the software, but in 1968 IBM divided it into PARS (application) and ACP (operating system). Later versions of ACP were named ACP / TPF and then TPF (Transaction Processing Facility) as non-airline businesses adopted this operating system for handling large volumes of online transactions. The latest version is z/TPF.

IBM developed ACP and its successors because: in the mid-1960s IBM's standard operating systems (DOS/360 and OS/360) were batch-oriented and could not handle large numbers of short transactions quickly enough; even its transaction monitors IMS and CICS, which run under the control of standard general-purpose operating systems, are not fast enough for handling reservations on hundreds of flights from thousands of travel agents.

The last "public domain" version of ACP, hence its last "free" version, was ACP 9.2, which was distributed on a single mini-reel with an accompanying manual set (about two dozen manuals, which occupied perhaps 48 lineal inches of shelf space) and which could be restored to IBM 3340 disk drives and which would, thereby, provide a fully functional ACP system.

ACP 9.2 was intended, primarily, for bank cards like MasterCard and other financial applications, but it could also be utilized for airline reservation systems, too, as by this time ACP had become a more general-purpose OS.

ACP had by then incorporated a hypervisor module (CHYR) which supported a virtual OS (usually VS1, but possibly also VS2) as a guest, with which program development or file maintenance could be accomplished concurrently with the online functions.

In some instances, production work was run under VS2 under the hypervisor, including, possibly, IMS DB.

System/360 Model 20

The Model 20 was labeled as part of the System/360 range because it could be connected to some of the same peripherals, but it was a 16-bit machine and not entirely program-compatible with other members of the System/360 range. Three operating systems were developed by IBM's labs in Germany, for different 360/20 configurations; DPS—with disks (minimum memory required: 12 KB); TPS—no disk but with tapes (minimum memory required: 8 KB); and CPS—punched-card-based (minimum memory required: 4 KB). These had no direct successors since IBM introduced the System/3 range of small business computers in 1969 and System/3 had a different internal design from the 360/20 and different peripherals from IBM's mainframes.

System/360 Model 44

The 360/44 is another processor that uses the System/360 peripherals but has a modified instruction set. It was designed for scientific computation using floating point numbers, such as geological or meteorological analyses. Because of the internal differences and the specialized type of work for which it was designed, the 360/44 has its own operating system, PS/44. An optional feature allows a System/360 emulator to run in hidden storage and implement the missing instructions in order to run OS/360. The 360/44 and PS/44 have no direct successors.

System/370 and virtual memory operating systems

System/370 was announced in 1970 with essentially the same facilities as System/360 but with about 4 times the processor speeds of similarly-priced System/360 CPUs. Then in 1972 IBM announced "System/370 Advanced Functions", of which the main item was that future sales of System/370 would include virtual memory capability and this could also be retro-fitted to existing System/370 CPUs. Hence IBM also committed to delivering enhanced operating systems which could support the use of virtual memory.

Most of the new operating systems are distinguished from their predecessors by the presence of "/VS" in their names. "VS" stands for "Virtual Storage". IBM avoided the term "virtual memory", allegedly because the word "memory" might be interpreted to imply that IBM computers could forget things.

All modern IBM mainframe operating systems except z/TPF are descendants of those included in the "System/370 Advanced Functions" announcement – z/TPF is a descendant of ACP, the system which IBM initially developed to support high-volume airline reservations applications.

DOS/VS

DOS/VS is the successor to DOS/360, and offers similar facilities, with the addition of virtual memory. In addition to virtual memory DOS/VS provided other enhancements:

• Five memory partitions instead of three. Later releases increase this to seven.
• A relocating loader, so that it is no longer necessary to link-edit a separate copy of each program for each partition in which it is to run.
• An improved spooling component, POWER/VS.

DOS/VS was followed by significant upgrades: DOS/VSE and VSE/SP (1980s), VSE/ESA (1991), and z/VSE (2005).

OS/VS1

OS/VS1 succeeded MFT, with similar facilities, and adding virtual memory. IBM released fairly minor enhancements of OS/VS1 until 1983, and in 1984 announced that there would be no more. OS/VS1 and TSS/370 are the only IBM System/370 operating systems that do not have modern descendants.

The Special Real Time Operating System (SRTOS), Programming RPQ Z06751, is a variant of OS/VS1 extended to support real-time computing. It was targeted at such industries as electric utility energy management and oil refinery applications.

OS/VS2 and MVS

OS/VS2 Release 1 (SVS) is a replacement for MVT with virtual memory. There are many changes, but it retains the overall structure of MVT.

In 1974 IBM released what it described as OS/VS2 Release 2 but which is a major rewrite that was upwards-compatible with the earlier OS/VS2 SVS. The new system's most noticeable feature is support for multiple virtual address spaces. Different applications thought they were using the same range of virtual addresses, but the new system's virtual memory facilities mapped these to different ranges of real memory addresses. As a result, the new system rapidly became known as "MVS" (Multiple Virtual Storages), the original OS/VS2 became known as "SVS" (Single Virtual Storage). IBM itself accepted this terminology and labelled MVS's successors "MVS/...".

The other distinctive features of MVS are: its main catalog must be a VSAM catalog; it supports "tightly-coupled multiprocessing" (2 or more CPUs share the same memory and copy of the operating system); it includes a System Resources Manager (renamed Workload Manager in later versions) which allows users to load additional work on to the system without reducing the performance of high-priority jobs.

IBM has released several MVS upgrades: MVS/SE, MVS/SP Version 1, MVS/XA (1981), MVS/ESA (1985), OS/390 (1996) and currently z/OS (2001).

VM/370

VM/370 combines a virtual machine facility with a single-user system called Conversational Monitor System (CMS); this combination provides time-sharing by allowing each user to run a copy of CMS on a virtual machine. This combination was a direct descendant of CP/CMS. The virtual machine facility was often used for testing new software while normal production work continues on another virtual machine, and the CMS timesharing system was widely used for program development.

VM/370 was followed by a series of upgrades: VM/SEPP ("Systems Extensions Program Product"), VM/BSEPP ("Basic Systems Extensions Program Product"), VM/SP (System Product), VM/SP HPO ("High Performance Option"), VM/XA MA ("Extended Architecture Migration Aid"), VM/XA SF ("Extended Architecture System Facility"), VM/XA SP ("Extended Architecture System Product"), VM/ESA ("Enterprise Systems Architecture"), and z/VM. IBM also produced optional microcode assists for VM and successors, to speed up the hypervisor's emulation of privileged instructions (those which only operating systems can use) on behalf of "guest" operating systems. As part of 370/Extended Architecture, IBM added the Start Interpretive Execution (SIE) instruction to allow a further speedup of the CP hypervisor.