Solipsism Gradient

A few extra comments on the Dharma/Yellow Box rumor. By the way, Wil Shipley has also weighed in with similar reasons against a YB revival:

Seriously, people. Apple doesn’t WANT your current Intel machine to run Mac OS X software. If it could, they wouldn’t be able to sell you a new machine in June of 2006. Trace the dollars! How would Apple profit from this?

It’s been demonstrated (by the previous Yellow Box) that big developers won’t just write their software once for Yellow Box and then “call it good” on Windows. Hell, Adobe can hardly be convinced to come out of the CFM closet, much less dump their Windows codebase in favor of a Cocoa one.

I agree that for small developers YB might, just possibly, be an alternative way to write new cross-platform applications, even with all the juggling they’d have to do to accomodate platform-specific stuff. There are some cross-platform frameworks around, but I personally don’t know any compelling small apps that use them – then again, I don’t use Windows myself on a regular basis, and when I have to, I just use the included apps.

Still, I can’t see Apple reviving YB just to help small developers break into the Windows market. There’s only one substantial cross-platform framework they still support – QuickTime – and there their motives are very different.

On the other hand, Apple has been known to invest into cross-platform frameworks strictly for their own use; for instance, iTunes on Windows certainly uses some in-house Carbon compatibility layer, as well as parts of QuickTime. Should Apple decide to rewrite iTunes in Cocoa on both platforms, some in-house version of YB would certainly come in handy; the same would apply if Apple decided to port some other major app to Windows – Safari is often cited as a candidate for that.

In fact, since Apple admitted to keeping successive versions of Mac OS X for Intel alive in the labs all these years, it wouldn’t be a surprise if they still were keeping the Yellow Box around too. But going from using it in-house to releasing a supported version to the outside world is too unlikely a step, at least IMHO.

An amazing bit of rumor has surfaced recently: Apple’s “Yellow Box” is supposed to be coming back from the dead, under the codename “Dharma“.

The idea seems to be that writing to a “pure Cocoa” API would enable software to be compiled, unchanged, for both Mac OS X and Windows. Or it could even be published as a universal binary containing both versions.

Indeed, the first aspect was the original idea when the Yellow Box was first published in 1997, when Apple bought NeXT (or vice-versa). Based on OpenStep, both Cocoa and the Yellow Box would have the same API and support programs running on both platforms. The Yellow Box was even included with Rhapsody (later Mac OS X server), although I think actual deployment licenses were sold separately. WebObjects (based on Objective-C) and EnterpriseObjects (ditto) used the same multiplatform philosophy.

However, at the 2000 WWDC Apple backed away from the whole thing. EO was discontinued amidst much weeping and wailing from its devotees, WO was converted to Java, and the Yellow Box was quietly dropped without much explanation.

At the time the reasons were quite unclear, but in hindsight some justifications can be seen. In particular, doing a successful multiplatform technology has turned out to be much more difficult than most people believed at the time; especially as Windows, not being under Apple’s control, would have been a difficult moving target anyway.

Personally I think that, retrospectively, dropping EO and moving WO to Java were mistaken decisions; keeping them both alive using Objective-C and Mac-only would have been prudent. However, the adoption of Cocoa turned out to be much slower than anyone would have thought, and as major companies like Microsoft, Adobe and Macromedia stayed with Carbon, any actual Yellow Box usage would have been restricted to smaller software houses or ISVs – for which the $3000 for 10 seats licensing terms would have been too expensive anyway.

Apple deftly turned away from the idea of Carbon as a short-term transition technology and instead promoted Carbon and Cocoa as equal-strength APIs whose capabilities are converging over time. Now here is an aspect which the recent rumormongers – not being developers – have not thought of. Both NeXTSTEP and OpenStep were the lowest-level API for application programs, but this is not true of Cocoa. The multiple layering of technologies and APIs on Mac OS X means that Cocoa can’t do many things that are possible in other APIs; meaning, in practice, that any reasonably complex software must use a mixture of technologies. My own XRay, for instance, even though it’s nominally a Cocoa program, also calls BSD APIs to handle users, groups, and permissions, as well as Carbon APIs to access many aspects of the file system.

Therefore, were the Yellow Box magically resurrected, programs written for a “pure Cocoa” API would theoretically run on it – but few, if any, existing Cocoa applications can afford to be so pure. The current Cocoa framework would have to be extended quite a lot, and Yellow Box prices would have to be lowered drastically – perhaps even to zero – before any but a handful of software companies would be interested. And for what? To allow interesting applications for Mac OS X also to run on Windows, reducing incentives for users to switch? Sounds extremely unlikely to me…

A large shoe which Apple will drop is, of course, virtualization. Virtualization will tie in with a recent Apple patent which, linked to the TPM chip, will allow the new machines to run other OSes inside a sandboxed “virtual machine”. Those other OSes will believe they have a somewhat slower system all to themselves, while running inside a separate window (and, perhaps, disk partition or disk image).

Putting this together with the previously discussed flash memory hypothesis, the advantages to Apple become obvious. Why should they need to have a standard BIOS or EFI at all? They can boot directly into the virtualization kernel (or whatever it’s called at the moment), from encrypted flash memory, which in turn would run a subset of Mac OS X – perhaps only the kernel with drivers and the windows manager – to have a basic GUI to check out which virtual OSes the user wants to run. This could be a full version of Mac OS X, or any version of Windows, Unix or Linux that runs on the abstracted hardware presented by the virtualizer.

In fact, it seems that a virtual OS could be a stripped-down OS specialized for a single application – something like what’s used now for embedded systems. It might be a gaming OS optimized for full-screen interfaces, for instance, or a TiVo-like appliance, or a multimedia center.

So, Apple doesn’t have to tie itself to standards during the boot process; in fact, this means that even if the user wants to run some non-Apple OS most of the time, the Mac interface will be present all the time underneath. At the very least, you can imagine the current ghastly BIOS user interface neatly presented in Aqua…

Amazingly, I’ve almost caught up with my e-mail, almost a month after returning from the last trip… and the remaining offline work is clearing up, so I hope to be able to post more here again.

The catchy title is just to remind people that Apple has a lot of shoes to drop in the near future.

In particular, there’s been some news recently that I should comment on while I still have all the links. First of all, Apple will prepay $1.25 billion (yes, gigabucks) to a new Intel-Micron consortium to secure supplies of NAND flash memory – the flash memory used in some iPods. This of course is, obviously, also meant to dispel investor’s fears about iPod supply problems, as the present manufacturers aren’t quite able to keep up with Apple’s demands, let alone the rest of the industry.

Separately, the latest build of Tiger for X86 has been cracked; seems they’re not using the TPM encryption capabilties yet, but just checking for the chip’s presence. The latest build also extends Rosetta to emulate the G4 with AltiVec, so a wider range of PowerPC apps should be able to immediately run emulated on X86 – although there’s no word yet about speed ratios.

The Motley Fool and some other folks speculate that Apple will be introducing instant-on capabilities; this would use the suddenly plentiful flash chips to hold parts of the operating system while the power is off. Incidentally, the IBM/360 mainframes I worked with in the 1970’s had something similar; the core memories they used at the time held data when the power went off, so with some care – stopping the processor before turning it off – you often could just continue after turning it back on without a reboot.

Finally, it’s no surprise that Intel has a special “Apple Group” where engineers from both companies work together. As I believe it unlikely that Apple will use a standard Intel motherboard, the most likely focus of this group is to make special motherboards and custom chips for Apple.

Put these bits together and what do you get? I think the X86 hackers are in for a surprise when the new Macs come out. I think Apple will take OS-hardware integration to a new peak with the Intel Macs. They’ll have a gigabyte or so of flash memory where an encrypted version of the Mac OS X will reside – a return to the days of the first Mac 128, when most of the toolbox was in ROM and the “System” file just contained patches and late-minute additions.

This giga-firmware will be encrypted with each machine’s own unique ID – contained in the TPM chip – and will be decrypted on-the-fly as needed into a secure portion of system RAM. Future system updates will come encrypted and be re-coded by the update process, which will run in full secure mode, perhaps even inside the TPM chip itself. Since some Intel CPUs are rumored to have the TPM chip built-in, this hypothesis gains weight. The instant-on capability would be just a nice side-effect…

I usually pay relatively little attention to news about Microsoft, not counting the usual jokes about virii and blue screens. What one hears about XP and Vista is usually not too exciting, and even MSFT stock has been slowly but steadily going down since 2000.

But lately there’s been some philosophical discussion of the whole Microsoft story. For instance, Forbes has a very interesting discussion of Microsoft’s midlife crisis:

…In the dog years of Silicon Valley, Microsoft, at 30, is in advanced middle age. The company relies on Windows and a suite of desktop applications – products released a decade ago – for 80% of sales and 140% of profits. Newer products – the Xbox videogame machine, the MSN online service, the wireless and small-business software – collectively have racked up $7 billion in losses in four years…

The article goes on to draw parallels to IBM, which suffered a similar crisis in the Eighties. Then I saw a couple of Business Week articles, including one about employee dissatisfaction. It was interesting to read how far Microsoft has come from its beginnings – both in the positive and negative senses – and in what varied direction.

Then I saw this great post over at Matt Gemmell‘s blog:

I just don’t have any real anger towards Microsoft anymore…

…I’m not interested in Microsoft bashing or even much Windows criticism at this point, and that surprises the hell out of me, to be honest. I guess I just don’t really care anymore. I have an OS that I really like, without the sense of compromise I had way back in the days of classic Mac OS. I feel I’ve got the best end of the deal, and I still get the “elite club” thing we’ve all always cherished. I don’t want Apple to destroy Microsoft, nor are they going to…

…I wouldn’t want to use Windows on my own machine, but at long last I think I’m finally really comfortable with it being around. I guess that if I was Microsoft, I’d find that pretty terrifying.

Of course you should read the whole thing; I’ll be translating this for publication in Macmania magazine later on, since I thought Matt’s really spot on here.

Finally, while considering writing something up here, someone pointed me at the latest Rob Enderle Apple Death Knell (here are the other ones – currently 46!). I won’t dissect it too detailedly as The Mac Observer folks have already done so, but what I found most striking about that piece was the great disconnect between the writer’s view of Apple and “Linux” as minor/doomed competitors of Microsoft for the entire computing market, and my (and other’s) feelings of Apple and Microsoft inhabiting increasingly different spaces.

Yes, there’s still part of the rivalry over the desktop OS market – the Tiger/Vista face-off which both sides view as mostly decided, each in their favor. And the QuickTime/WindowsMedia thing, which has largely migrated out of the desktop space into the music player space. But, come on, it makes no sense anymore to speak broadly of “marketshare”. The computer market has split into multiple markets. Servers are going one way, cheapie home or office computers another, high-end game machines yet another, and so forth for at least a dozen of splinter markets. Apple is interested in some of these segments, but not in most others; and correctly so, as they shouldn’t fall into the trap of overextending their reach, as Microsoft seems to have done. Instead, with the iPod, they’ve defined and dominated one new market with great focus and competence.

I can’t resist quoting one paragraph of the Enderle piece, however:

For Office 12 the clear advancement is discoverability. The product has over 1,500 features and most of us use 30 or 40. The massive UI enhancement has to do with being able to find and understand these features, the vast majority of which have been included in earlier versions of the product. This is something where the words “long overdue” seem incredibly inadequate. Everything else, no matter how compelling, that is new in this product simply seems unimportant when compared to this one comprehensive overdue enhancement.

If I understood this alright… the user now needs a search facility to find out which of these 1,500 features should be used to turn off, say, automatic uppercasing? This is the microcosmic analogue of Microsoft’s error of trying to do all things for all markets. And failing, if Forbes’ numbers are correct.

If my hunches are correct (as I’ve explained in the previous post), Apple will probably seize the PowerPC->Intel transition to leave the retail OS market entirely. They’ll sell updates for holdout Mac OS 9 users for some time, and updates for MacIntel boxes, but in 2-3 years they may even stop breaking out OS update sales figures from other software sales figures. It will be great fun reading these columnist after that happens.

It appears that a new version of Mac OS X for the Intel Transition Kits is out. As I expected, it has new protection measures in place. (Note: I don’t have a transition kit and would be under NDA if I had one, so all this is speculation based on publicly available data…)

There’s still very little information out on what they actually changed regarding use of the TPM chip. Of course, the previously published patches don’t work on the new version; this also confirms my idea that Apple is using the pirates as a test bed for their protection algorithms. Also, it seems that applications compiled for the new version won’t run on the old version, as the ABI (Application Binary Interface) has changed. This was to be expected; it also happened several times before Mac OS X 10.0 came out; the fact that it also makes the patched systems out there useless for most practical purposes is just a positive side-effect for Apple.

Most comments I’ve seen from the PC user’s side show a lack of information about the TPM chip’s capabilities; they either have an unconditional faith that it will be hacked in a few days, or that it will kill their firstborn. As I’ve said before, my opinion is that it will be very hard to hack – maybe even impossible in practice – but that Apple won’t press its use beyond the one of protecting their intent of restricting Mac OS X to Apple-built machines.

There’s one further misconception to be addressed. Some people say that, once they buy a retail copy of Mac OS X, the part of the shrink-wrap license that says that they can run it only on Apple-built machines need not be obeyed. While I’m not sure if such restrictions have ever been tested in court, there are a few ways that Apple can strengthen its position.

For one, they could simply stop selling retail licenses of Mac OS X; a copy of the system would be included with every machine and they would sell only updates separately. Remember that the only reason that retail copies of Mac OS X for PowerPC were put on the market was to entice users to upgrade from Mac OS 9; it’s not like Microsoft selling retail copies of Windows for generic PCs. And of course I don’t believe that Apple will make Mac OS X available for generic PCs in the foreseeable future…

The first Mac, in 1984, had most of its operating system contained in ROM; the boot diskette contained the Finder and a System file which consisted essentially of patches to the ROM. Apple’s ROMs were jealously guarded – relatively easy as they were part of the hardware – but boot diskettes were freely distributed. With Flash memory capacity going up, Apple might even return to this model.

Michael Tsai links back to my previous post and to lots of other great comments. I particularly recommend Brent Simmon‘s analysis. And of course John Gruber‘s take is hors concours.

The phone thingy leaves me completely cold; I don’t own a cellphone, and don’t intend to buy one in the near future. Still, I suppose I should be glad that the market appeared to like it; AAPL even went over the magic $50 mark today for a few minutes.

The iPod nano looks great, even if it’s a little expensive per gigabyte. If I find a suckerbuyer for my 40GB (3G) iPod, I might even switch… given that I use it mostly as a glorified shuffle in the car and in the gym, and I have larger drives elsewhere to keep my main music database, it would make sense. And the “no moving parts” aspect is of course ideal.

(I was puzzled by repeated references to “nano” being a dumb name… until someone told me it refers to a 70s(?) TV show, which I’d never heard of. I wonder if all future “nano” products – and there will be zillions – will be affected by this?)

The third aspect is the new iTunes 5 interface. Dan Wood dislikes it, as do most others who’ve commented. It took me a moment to look beyond the Mail-style splitter control to notice that metal had gone, replaced by Yet Another Different Interface Style.

Personally, I thought the margin-less aspect is positive, and the new gradients aren’t too bad. I never use Mail, so I found the divider-less splitter very strange; although I like to experiment with new UI, this is one thing I won’t try out in my products. (However, users of RBSplitView should note that this is easy to do with it.)

Overall, this reinforces my opinion that Apple is now moving away from a intelligent design approach to UI, towards a more survival of the fittest sort of thing. Each application’s group tries out new UI elements and the ones that survive get incorporated into Interface Builder’s palettes one or two releases down the road. It’s certainly more exciting but also, at times, disconcerting…