Solipsism Gradient

An amazing bit of rumor has surfaced recently: Apple’s “Yellow Box” is supposed to be coming back from the dead, under the codename “Dharma“.

The idea seems to be that writing to a “pure Cocoa” API would enable software to be compiled, unchanged, for both Mac OS X and Windows. Or it could even be published as a universal binary containing both versions.

Indeed, the first aspect was the original idea when the Yellow Box was first published in 1997, when Apple bought NeXT (or vice-versa). Based on OpenStep, both Cocoa and the Yellow Box would have the same API and support programs running on both platforms. The Yellow Box was even included with Rhapsody (later Mac OS X server), although I think actual deployment licenses were sold separately. WebObjects (based on Objective-C) and EnterpriseObjects (ditto) used the same multiplatform philosophy.

However, at the 2000 WWDC Apple backed away from the whole thing. EO was discontinued amidst much weeping and wailing from its devotees, WO was converted to Java, and the Yellow Box was quietly dropped without much explanation.

At the time the reasons were quite unclear, but in hindsight some justifications can be seen. In particular, doing a successful multiplatform technology has turned out to be much more difficult than most people believed at the time; especially as Windows, not being under Apple’s control, would have been a difficult moving target anyway.

Personally I think that, retrospectively, dropping EO and moving WO to Java were mistaken decisions; keeping them both alive using Objective-C and Mac-only would have been prudent. However, the adoption of Cocoa turned out to be much slower than anyone would have thought, and as major companies like Microsoft, Adobe and Macromedia stayed with Carbon, any actual Yellow Box usage would have been restricted to smaller software houses or ISVs – for which the $3000 for 10 seats licensing terms would have been too expensive anyway.

Apple deftly turned away from the idea of Carbon as a short-term transition technology and instead promoted Carbon and Cocoa as equal-strength APIs whose capabilities are converging over time. Now here is an aspect which the recent rumormongers – not being developers – have not thought of. Both NeXTSTEP and OpenStep were the lowest-level API for application programs, but this is not true of Cocoa. The multiple layering of technologies and APIs on Mac OS X means that Cocoa can’t do many things that are possible in other APIs; meaning, in practice, that any reasonably complex software must use a mixture of technologies. My own XRay, for instance, even though it’s nominally a Cocoa program, also calls BSD APIs to handle users, groups, and permissions, as well as Carbon APIs to access many aspects of the file system.

Therefore, were the Yellow Box magically resurrected, programs written for a “pure Cocoa” API would theoretically run on it – but few, if any, existing Cocoa applications can afford to be so pure. The current Cocoa framework would have to be extended quite a lot, and Yellow Box prices would have to be lowered drastically – perhaps even to zero – before any but a handful of software companies would be interested. And for what? To allow interesting applications for Mac OS X also to run on Windows, reducing incentives for users to switch? Sounds extremely unlikely to me…

A large shoe which Apple will drop is, of course, virtualization. Virtualization will tie in with a recent Apple patent which, linked to the TPM chip, will allow the new machines to run other OSes inside a sandboxed “virtual machine”. Those other OSes will believe they have a somewhat slower system all to themselves, while running inside a separate window (and, perhaps, disk partition or disk image).

Putting this together with the previously discussed flash memory hypothesis, the advantages to Apple become obvious. Why should they need to have a standard BIOS or EFI at all? They can boot directly into the virtualization kernel (or whatever it’s called at the moment), from encrypted flash memory, which in turn would run a subset of Mac OS X – perhaps only the kernel with drivers and the windows manager – to have a basic GUI to check out which virtual OSes the user wants to run. This could be a full version of Mac OS X, or any version of Windows, Unix or Linux that runs on the abstracted hardware presented by the virtualizer.

In fact, it seems that a virtual OS could be a stripped-down OS specialized for a single application – something like what’s used now for embedded systems. It might be a gaming OS optimized for full-screen interfaces, for instance, or a TiVo-like appliance, or a multimedia center.

So, Apple doesn’t have to tie itself to standards during the boot process; in fact, this means that even if the user wants to run some non-Apple OS most of the time, the Mac interface will be present all the time underneath. At the very least, you can imagine the current ghastly BIOS user interface neatly presented in Aqua…

Amazingly, I’ve almost caught up with my e-mail, almost a month after returning from the last trip… and the remaining offline work is clearing up, so I hope to be able to post more here again.

The catchy title is just to remind people that Apple has a lot of shoes to drop in the near future.

In particular, there’s been some news recently that I should comment on while I still have all the links. First of all, Apple will prepay $1.25 billion (yes, gigabucks) to a new Intel-Micron consortium to secure supplies of NAND flash memory – the flash memory used in some iPods. This of course is, obviously, also meant to dispel investor’s fears about iPod supply problems, as the present manufacturers aren’t quite able to keep up with Apple’s demands, let alone the rest of the industry.

Separately, the latest build of Tiger for X86 has been cracked; seems they’re not using the TPM encryption capabilties yet, but just checking for the chip’s presence. The latest build also extends Rosetta to emulate the G4 with AltiVec, so a wider range of PowerPC apps should be able to immediately run emulated on X86 – although there’s no word yet about speed ratios.

The Motley Fool and some other folks speculate that Apple will be introducing instant-on capabilities; this would use the suddenly plentiful flash chips to hold parts of the operating system while the power is off. Incidentally, the IBM/360 mainframes I worked with in the 1970’s had something similar; the core memories they used at the time held data when the power went off, so with some care – stopping the processor before turning it off – you often could just continue after turning it back on without a reboot.

Finally, it’s no surprise that Intel has a special “Apple Group” where engineers from both companies work together. As I believe it unlikely that Apple will use a standard Intel motherboard, the most likely focus of this group is to make special motherboards and custom chips for Apple.

Put these bits together and what do you get? I think the X86 hackers are in for a surprise when the new Macs come out. I think Apple will take OS-hardware integration to a new peak with the Intel Macs. They’ll have a gigabyte or so of flash memory where an encrypted version of the Mac OS X will reside – a return to the days of the first Mac 128, when most of the toolbox was in ROM and the “System” file just contained patches and late-minute additions.

This giga-firmware will be encrypted with each machine’s own unique ID – contained in the TPM chip – and will be decrypted on-the-fly as needed into a secure portion of system RAM. Future system updates will come encrypted and be re-coded by the update process, which will run in full secure mode, perhaps even inside the TPM chip itself. Since some Intel CPUs are rumored to have the TPM chip built-in, this hypothesis gains weight. The instant-on capability would be just a nice side-effect…

This should be my last post on the topic for quite a while. Pierre Igot over at Betalogue comments on various posts about the future of MacBU inside Microsoft. Worth reading if you use Office 2004 (I don’t)… and the Mac Business Unit is indeed inside the new Entertainment & Devices division. According to Rick Schaut, who works there, this makes sense; Mac Office users are very different from Windows Office users. I’d tend to agree.

Rainer Brockerhoff wrote:

No word yet on where their Mac software division will fit in…

This interesting article shows Microsoft executives discussing Apple as a competitor. At the outset it’s said that the Mac Business Unit is now in the games division… this must be either a joke, or a joke. Ehrrm. You know what I mean…

More on Microsoft:

Is Microsoft Vista the new QuarkXPress?, by Daniel Erans:

Quark has long owned the desktop publishing world. Yet, after a decade of dominance, the company stumbled, leaving the door open for serious competition just as Adobe was introducing a strong competing product. Is Microsoft about to do the same?

Worth reading, although I don’t agree with all his conclusions…

Also lots of stuff on the Mini-Microsoft blog.

In other news, Microsoft reorganizes into 3 main divisions. No word yet on where their Mac software division will fit in…

It appears that a new version of Mac OS X for the Intel Transition Kits is out. As I expected, it has new protection measures in place. (Note: I don’t have a transition kit and would be under NDA if I had one, so all this is speculation based on publicly available data…)

There’s still very little information out on what they actually changed regarding use of the TPM chip. Of course, the previously published patches don’t work on the new version; this also confirms my idea that Apple is using the pirates as a test bed for their protection algorithms. Also, it seems that applications compiled for the new version won’t run on the old version, as the ABI (Application Binary Interface) has changed. This was to be expected; it also happened several times before Mac OS X 10.0 came out; the fact that it also makes the patched systems out there useless for most practical purposes is just a positive side-effect for Apple.

Most comments I’ve seen from the PC user’s side show a lack of information about the TPM chip’s capabilities; they either have an unconditional faith that it will be hacked in a few days, or that it will kill their firstborn. As I’ve said before, my opinion is that it will be very hard to hack – maybe even impossible in practice – but that Apple won’t press its use beyond the one of protecting their intent of restricting Mac OS X to Apple-built machines.

There’s one further misconception to be addressed. Some people say that, once they buy a retail copy of Mac OS X, the part of the shrink-wrap license that says that they can run it only on Apple-built machines need not be obeyed. While I’m not sure if such restrictions have ever been tested in court, there are a few ways that Apple can strengthen its position.

For one, they could simply stop selling retail licenses of Mac OS X; a copy of the system would be included with every machine and they would sell only updates separately. Remember that the only reason that retail copies of Mac OS X for PowerPC were put on the market was to entice users to upgrade from Mac OS 9; it’s not like Microsoft selling retail copies of Windows for generic PCs. And of course I don’t believe that Apple will make Mac OS X available for generic PCs in the foreseeable future…

The first Mac, in 1984, had most of its operating system contained in ROM; the boot diskette contained the Finder and a System file which consisted essentially of patches to the ROM. Apple’s ROMs were jealously guarded – relatively easy as they were part of the hardware – but boot diskettes were freely distributed. With Flash memory capacity going up, Apple might even return to this model.

Michael Tsai links back to my previous post and to lots of other great comments. I particularly recommend Brent Simmon‘s analysis. And of course John Gruber‘s take is hors concours.