Copland (operating system)

Development officially began in March 1994. Over the next several years, previews of Copland garnered much press, introducing the Mac audience to operating system concepts such as object orientation, crash-proofing, and multitasking. In August 1995, David Nagel, a senior vice president, announced at Macworld Expo that Copland would be released in mid-1996. The following May, Gil Amelio stated that Copland was the primary focus of the company, aiming for a late-year release. Internally, however, the development effort was beset with problems due to dysfunctional corporate personnel and project management. Development milestones and developer release dates were missed repeatedly.

Ellen Hancock was hired to get the project back on track, but quickly concluded it could never ship. In August 1996, it was announced that Copland was canceled and Apple would look outside the company for a new operating system. Among many choices, they selected NeXTSTEP and purchased NeXT in 1997 to obtain it. In the interim period, while NeXTSTEP was ported to the Mac, Apple released the much more legacy-oriented Mac OS 8 in 1997 based upon adding components from Copland, and Mac OS 9 in 1999 to transition forward. Mac OS X became Apple's next-generation operating system in 2001.

The Copland development effort has been described as an example of feature creep. In 2008, PC World included Copland on a list of the biggest project failures in information technology history.

Design

Mac OS legacy

Launched in 1984, the Macintosh and its operating system were designed from the start as a single-user, single-tasking system, which allowed the hardware development to be greatly simplified.

The Macintosh lacks multitasking but tries to fake it, and it insists on a complicated user interface but leaves much of the work up to the application. These are serious drawbacks, and it is difficult to imagine elegant repairs for them.

— Adam Brooks Webber, Byte (September 1986)

These limits meant that supporting the multitasking of more than one program at a time would be difficult, without rewriting all of this operating system and application code. Yet doing so would mean the system would run unacceptably slow on existing hardware. Instead, Apple adopted a system known as MultiFinder in 1987, which keeps the running application in control of the computer, as before, but allows an application to be rapidly switched to another, normally simply by clicking on its window. Programs that are not in the foreground are periodically given short bits of time to run, but as before, the entire process is controlled by the applications, not the operating system.

Because the operating system and applications all share one memory space, it is possible for a bug in any one of them to corrupt the entire operating system, and crash the machine. Under MultiFinder, any crash anywhere will crash all running programs. Running multiple applications potentially increases the chances of a crash, making the system potentially more fragile.

Adding greatly to the severity of the problem is the patching mechanism used to add functions to the operating system, known as CDEVs and INITs or Control Panels and Extensions. Third party developers also make use of this mechanism to add features, including screensavers and a hierarchical Apple menu. Some of these third-party control panels became almost universal, like the popular After Dark screensaver package. Because there was no standard for use of these patches, it is not uncommon for several of these add-ons — including Apple's own additions to the OS — to use the same patches, and interfere with each other, leading to more crashing.

Copland design

Copland was designed to consist of the Mac OS on top of a microkernel named Nukernel, which would handle basic tasks such as application startup and memory management, leaving all other tasks to a series of semi-special programs known as servers. For instance, networking and file services would not be provided by the kernel itself, but by servers that would be sending requests through interapplication communications. The Copland system as a whole consists of the combination of Nukernel, various servers, and a suite of application support libraries to provide implementations of the well-known classic Macintosh programming interface.

Application services are offered through a single program known officially as the Cooperative Program Address Space. Mac programs run much as they do under System 7, as cooperative tasks that use the non-reentrant Toolbox calls. Copland provided much improved stability for applications running compared to System 7 – "the system won’t crash as it can in System 7. Instead, because the Copland microkernel tracks all resources (such as, tasks, timers, and areas), it can reclaim them when it terminates the process associated with your application.". A worst-case scenario is that an application in the CPAS environment crashes, taking down the entire environment with it. This does not result in the system as a whole going down, however, and the Cooperative Program Address Space environment is automatically restarted. In addition, Copland provides pre-emptive scheduling of tasks where "the Process Manager gives each primary task the opportunity to be preemptively scheduled by the microkernel."

New applications, written with Copland in mind, are able to directly communicate with the system servers and thereby gain many advantages in terms of performance and scalability. They can also communicate with the kernel to launch separate applications or threads, which run as separate processes in protected memory, as in most modern operating systems. These separate applications cannot use non-reentrant calls like QuickDraw, however, and thus could have no user interface. Apple suggested that larger programs could place their user interface in a normal Macintosh application, which then start worker threads externally.

Copland is fully PowerPC (PPC) native. System 7 had been ported to the PowerPC with great success; some parts of the system run as PPC code. There is enough 68k code left in the system to be run in emulation, and especially user applications, however that the operating system must map some data between the two environments. In particular, every call into the Mac OS requires a mapping between the interrupt systems of the 68k and PPC. Removing these mappings would greatly improve general system performance. At WWDC 1996, engineers claimed that system calls would execute as much as 50% faster.

Copland is also based on the then-recently defined Common Hardware Reference Platform, or CHRP, which standardized the Mac hardware to the point where it could be built by different companies and can run other operating systems (Solaris and AIX were two of many mentioned). This was a common theme at the time; many companies were forming groups to define standardized platforms to offer an alternative to the "Wintel" platform that was rapidly becoming dominant — examples include 88open, Advanced Computing Environment, and the AIM alliance.

The fundamental challenge of Copland's development and adoption was getting all of these functions to fit into an ordinary Mac. System 7.5 already uses up about 2.5 megabytes (MB) of RAM, which is a significant portion of the total RAM in most contemporaneous machines. Copland is a hybrid of two systems, as its native foundation also hosts the Cooperative Program Address Space application environment as well as Copland native Server tasks. Copland thus uses a Mach-inspired memory management system and relies extensively on shared libraries, with the goal of being about 50% larger than 7.5.

2 more chapters are free with a Lably account.

History

Pink and Blue

In March 1988, technical middle managers at Apple held an offsite meeting to plan the future course of Mac OS development. Ideas were written on index cards; features that seemed simple enough to implement in the short term (like adding color to the user interface) were written on blue cards; longer-term goals—such as preemptive multitasking—were on pink cards; and long-range ideas like an object-oriented file system were on red cards. Development of the ideas contained on the blue and pink cards was to proceed in parallel, and at first, the two projects were known simply as "blue" and "pink". Apple intended to have the Blue team release an updated version of the existing Macintosh operating system in the 1990–1991 timeframe, and the Pink team to release an all-new OS around 1993.

The Blue team, who came to call themselves the "Blue Meanies" after characters in the film Yellow Submarine, delivered what became known as System 7 on May 13, 1991, but the Pink team's efforts suffered from second-system effect and its release date continued to slip into the indefinite future. Some of the reason for this can be traced to problems that would become widespread at Apple as time went on; as Pink became delayed, its engineers moved to Blue instead. This left the Pink team constantly struggling for staffing, and suffering from the problems associated with high employee turnover. Management ignored these sorts of technical development issues, leading to continual problems delivering working products.

At this same time, the recently released NeXTSTEP was generating intense interest in the developer world. Features that were originally part of Red were folded into Pink, and the Red project (also known as "Raptor") was eventually canceled. This problem was also common at Apple during this period; to chase the "next big thing", middle managers added new features to their projects with little oversight, leading to enormous problems with feature creep. In the case of Pink, development eventually slowed to the point that the project appeared moribund.

Taligent

On April 12, 1991, Apple CEO John Sculley performed a secret demonstration of Pink running on a PS/2 Model 70 to a delegation from IBM. Though the system was not fully functional, it resembled System 7 running on a PC. IBM was extremely interested, and over the next few months, the two companies formed an alliance to further development of the system. These efforts became public in early 1992, under the new name "Taligent". At the time, Sculley summed up his concerns with Apple's own ability to ship Pink when he stated "We want to be a major player in the computer industry, not a niche player. The only way to do that is to work with another major player."

Infighting at the new joint company was legendary, and the problems with Pink within Apple soon appeared to be minor in comparison. Apple employees made T-shirts graphically displaying their prediction that the result would be an IBM-only project. On December 19, 1995, Apple officially pulled out of the project. IBM continued working alone with Taligent, and eventually released its application development portions under the new name "CommonPoint". This saw little interest and the project disappeared from IBM's catalogs within months.

Business as usual

While Taligent efforts continued, very little work addressing the structure of the original OS was carried out. Several new projects started during this time, such as the Star Trek project, a port of System 7 and its basic applications to Intel-compatible x86 machines, which reached internal demo status. But as Taligent was still a concern, it was difficult for new OS projects to gain any traction.

As the Taligent program slowed down, in 1990, Bill Bruffey who had built HFS for the Mac, got permission to develop a new kernel that was designed to run Mac OS in particular. This became the NuKernel project. It proposed what was essentially a simplified version of what would become Copland, using the microkernel to run the hardware and using a Mac OS emulator on top.

But during this time, Apple's Blue team continued adding new features to the original OS. During the early 1990s, Apple released a series of major new packages to the system; among them are QuickDraw GX, Open Transport, OpenDoc, PowerTalk, and many others. Most of these were larger than the original operating system. Problems with stability, which had existed even with small patches, grew along with the size and requirements of these packages, and by the mid-1990s the Mac had a reputation for instability and constant crashing.

As the stability of the operating system collapsed, the ready answer was that Taligent would fix this with all its modern foundation of full reentrance, preemptive multitasking, and protected memory. But as it became clear Taligent was not making progress, Apple was left with an aging OS and no clear solution. By 1994, the press buzz surrounding the upcoming release of Windows 95 started to crescendo, often questioning Apple's ability to respond to the challenge it presented. The press turned on the company, often introducing Apple's new projects as failures in the making.

Another try

Given this pressure, the collapse of Taligent, the growing problems with the existing operating system, after the release of System 7.5 in late 1994, Apple management decided that the decade-old operating system had run its course. A new system that did not have these problems was needed, and soon. Since so much of the existing system would be difficult to rewrite, Apple developed a two-stage approach to the problem.

In the first stage, the existing system would be moved on top of a new kernel-based OS with built-in support for multitasking and protected memory. The existing libraries, like QuickDraw, would take too long to be rewritten for the new system and would not be converted to be reentrant. Instead, the CPAS keeps applications and legacy code such as QuickDraw in a single memory block so they continue to run as they had in the past. CPAS runs in a distinct Copland memory space, so crashing legacy applications or extensions within CPAS cannot crash the entire machine.

In the next stage of the plan, once the new kernel was in place and this basic upgrade was released, development would move on to rewriting the older libraries into new forms that could run directly on the new kernel. At that point, applications would gain some added modern features.

In the musical code-naming pattern where System 7.5 is code-named "Mozart", this intended successor is named "Copland" after composer Aaron Copland. In turn, its proposed successor system, Gershwin, would complete the process of moving the entire system to the modern platform, but work on Gershwin would never officially begin.

Development

The Copland project was first announced by David Nagel in May 1994. Parts of Copland, such as an early version of the new file system, were demonstrated at Apple's Worldwide Developers Conference in May 1995. Apple promised that a beta release of Copland would be ready by the end of the year, for final commercial release in early 1996. Gershwin would follow the next year. Throughout the year, Apple released several mock-ups to various magazines showing what the new system would look like, and commented continually that the company was fully committed to this project. By the end of the year, however, no Developer Release had been produced.

As had happened during the development of Pink, developers within Apple soon started abandoning their own projects in order to work on the new system. Middle management and project leaders fought back by claiming that their project was vital to the success of the system, and moving it into the Copland development stream. Thus, it could not be canceled along with their employees being removed to work on some other part of Copland anyway. This process took on momentum across the next year.

"Anytime they saw something sexy it had to go into the OS." said Jeffrey Tarter, publisher of the software industry newsletter Softletter. "There were little groups all over Apple doing fun things that had no earthly application to Apple's product line." What resulted was a vicious cycle: As the addition of features pushed back deadlines, Apple was compelled to promise still more functions to justify the costly delays. Moreover, this Sisyphean pattern persisted at a time when the company could scarcely afford to miss a step.

Soon the project looked less like a new operating system and more like a huge collection of new technologies; QuickDraw GX, System Object Model (SOM), and OpenDoc became core components of the system, while completely unrelated technologies like a new file management dialog box (the open dialog) and themes support appeared also. The feature list grew much faster than the features could be completed, a classic case of creeping featuritis. An industry executive noted that "The game is to cut it down to the three or four most compelling features as opposed to having hundreds of nice-to-haves, I'm not sure that's happening."

As the "package" grew, testing it became increasingly difficult and engineers were commenting as early as 1995 that Apple's announced 1996 release date was hopelessly optimistic: "There's no way in hell Copland ships next year. I just hope it ships in 1997."

In mid-1996, information was leaked that Copland would have the ability to run applications written for other operating systems, including Windows NT. Simultaneously allegedly being confirmed by Copland engineers while being authoritatively denied by Copland project management, this feature had supposedly been in development for more than three years. One user claimed to have been told about these plans by members of the Copland development team. Some analysts projected that this ability would increase Apple's penetration into the enterprise market, others said it was "game over" and was only a sign of the Mac platform's irrelevancy.

Developer Release

At WWDC 1996, Apple's new CEO, Gil Amelio, used the keynote to talk almost exclusively about Copland, now known as System 8. He repeatedly stated that it was the only focus of Apple engineering and that it would ship to developers in a few months, with a full release planned for late 1996. Very few, if any, demos of the running system were shown at the conference. Instead, various pieces of the technology and user interface that would go into the package (such as a new file management dialog) were demonstrated. Little of the core system's technology was demonstrated and the new file system that had been shown a year earlier was absent.

There was one way to actually use the new operating system – by signing up for time in the developer labs. This did not go well:

There was a hands-on demo of the current state of OS 8. There were tantalizing glimpses of the goodies to come, but the overall experience was awful. It does not yet support text editing, so you couldn’t actually do anything except open and view documents (any dialog field that needed something typed into it was blank and dead). Also, it was incredibly fragile and crashed repeatedly, often corrupting system files on the disk in the process. The demo staff reformatted and rebuilt the hard disks at regular intervals. It was incredible that they even let us see the beast.

Several people at the show complained about the microkernel's lack of sophistication, notably the lack of symmetric multiprocessing, a feature that would be exceedingly difficult to add to a system due to ship in a few months. After that, Amelio came back on stage and announced that they would be adding that to the feature list.

In August 1996, "Developer Release 0" was sent to a small number of selected partners. Far from demonstrating improved stability, it often crashed after doing nothing at all, and was completely unusable for development. In October, Apple moved the target delivery date to "sometime", hinting that it might be 1997. One of the groups most surprised by the announcement was Apple's own hardware team, who had been waiting for Copland to allow the PowerPC to be natively represented, unburdened of software legacy. Members of Apple's software QA team joked that, given current resources and the number of bugs in the system, they could clear the program for shipping sometime around 2030.

Cancellation

Later in August 1996, the situation was no better. Amelio complained that Copland was "just a collection of separate pieces, each being worked on by a different team ... that were expected to magically come together somehow." Hoping to salvage the situation, Amelio hired Ellen Hancock away from National Semiconductor to take over engineering from Ike Nassi and get Copland development back on track.

After a few months on the job, Hancock came to the conclusion that the situation was hopeless; given current development and engineering, she believed Copland would never ship. Instead, she suggested that the various user-facing technologies in Copland be rolled out in a series of staged releases, instead of a single big release. Apple officially canceled Copland in August 1996, The CD envelopes for the developer's release had been printed, but the discs had not been mastered.

To address the aging infrastructure underneath these technologies, Amelio suggested looking outside the company for an unrelated new operating system. Candidates considered were Sun's Solaris and Windows NT. Hancock reportedly was in favor of going with Solaris, while Amelio preferred Windows. Amelio even reportedly called Bill Gates to discuss the idea, and Gates promised to put Microsoft engineers to work porting QuickDraw to NT.

After lengthy discussions with Be and rumors of a merger with Sun Microsystems, many were surprised at Apple's December 1996 announcement that they were purchasing NeXT and bringing Steve Jobs on in an advisory role. Amelio quipped that they "choose Plan A instead of Plan Be." The project to port NeXTSTEP to the Macintosh platform was named Rhapsody and was to be the core of Apple's cross-platform operating system strategy. This would inherit OpenStep's existing support for PowerPC, Intel x86, and DEC Alpha CPU architectures, and an implementation of the OpenStep libraries running on Windows NT. This would in effect open the Windows application market to Macintosh developers as they could license the library from Apple for distribution with their product, or depend on an existing installation.

Legacy

Following Hancock's plan, development continued with System 7.5 receiving integration of several elements of Copland. System 7 was renamed to Mac OS 7 with the release of 7.6, wherein stability and performance were improved. Many Copland features, including the new multithreaded Finder and support for themes (defaulting to Platinum) were integrated into the unreleased beta of Mac OS 7.7, which was instead rebranded and launched as Mac OS 8.

With the return of Jobs, this rebranding to version 8 also allowed Apple to exploit a legal loophole to terminate third-party manufacturers' licenses to System 7 and effectively shut down the Macintosh clone market. Later, Mac OS 8.1 finally added the new file system and Mac OS 8.6 updated the nanokernel to handle limited support for preemptive tasks. Its interface is Multiprocessing Services 2.x and later, but there is no process separation and the system still uses cooperative multitasking between processes. Even a process that is Multiprocessing Services-aware still has a part that also runs all single-threaded programs and the only task that can run 68k code.

The Rhapsody project was canceled after several Developer Preview releases, support for running on non-Macintosh platforms was dropped, and it was eventually released as Mac OS X Server 1.0. In 2001 this foundation was coupled to the Carbon library and Aqua user interface to form the modern Mac OS X product. Several features originally seen in Copland demos, including its advanced Find command, built-in web browser, piles of folders, and support for video-conferencing, have reappeared in subsequent releases of Mac OS X as Spotlight, Safari, Stacks, and iChat AV, respectively, although the implementation and user interface for each feature is very different.

Hardware requirements

According to the documentation included in the Developer Release, Copland supports the following hardware configurations:

NuBus-based Macintoshes: 6100/60, 6100/60AV (no AV functions), 6100/66, 6100/66 AV (no AV functions), 6100/66 DOS (no DOS functions), 7100/66, 7100/66 AV (no AV functions), 7100/80, 7100/80 AV (no AV functions), 8100/80/ 8100/100/ 8100/100 AV (no AV functions), 8100/110
NuBus-based Performas: 6110CD, 6112CD, 6115CD, 6117CD, 6118CD
PCI-based Macintoshes: 7200/75, 7200/90, 7500/100, 8500/120, 9500/120, 9500/132
Drives formatted with Drive Setup
For DR1 and earlier, the installer requires System 7.5 or later on a hard disk of 250MB or greater capacity.
Display set to 256 colors (8-bit) or Thousands (16-bit).