Solipsism Gradient

In ye olden times, to boot a computer you had to toggle in a short program on the panel switches, or put a special punch card deck into the hopper, or whatever. Personal computers started out either with built-in BASIC in ROM or a very small boot ROM which had just enough code to load the first block from tape or disk, and jump to that. (The built-in ROM is also called “firmware”.)

Later on, the first Macs had most of the core of the OS built into ROM, along with the self-test and boot functions, while the first PCs had a simple configuration utility instead, which came to be known as BIOS. A little later, when many different models began to appear, the BIOS also began to set a number of internal configurations, check firmware for peripheral cards, and so forth. In contrast, Apple gradually gave up on trying to store the OS toolbox in ROM, and instead increased the sophistication of the self-test and boot code. The downside, in both cases, was that operating systems had to know about a large number of machine types, configurations, peripherals and options, while a specific BIOS had to be made for every new CPU board.

After the first PowerPCs came out, Apple recognized the need to handle booting and peripherals in a more machine-independent way and adopted Open Firmware (OF), initially developed by Sun. OF is a complex beast; it has a Forth-like bytecode interpreter to run test and boot software in a machine-independent way. Also, peripheral cards could contain their own driver software in ROM, in theory even for compatibility with various platforms. This feature was largely reponsible for the Mac’s plug-and-play facilities.

Meanwhile, the BIOS limitations in the PC world became apparent and various entities attempted to progress beyond that, with very little success; the pressure for legacy compatibility was too great to make any inroads into the commodity PC market. One of these initiatives, Intel’s Extensible Firmware Interface (EFI) has been shipped on their Itanium boxes, but hasn’t been adopted by the standard PC market. One roadblock is that no current OS is directly EFI-compatible; Microsoft’s Vista will be, though, when it comes out.

EFI has been adopted by Apple for the new Intel Macs and in that sense will be the first large-scale deployment. Like OF, EFI allows peripheral cards to contain their own drivers. Like OF, it has a machine-independent bytecode interpreter. Also, like OF, it allows for a great variety of functions to be run at power-on time. And most importantly, still like OF, it allows for basic device drivers to be loaded and cataloged before any OS is booted, and that OS can check or complement these drivers only if necessary. Thus, EFI can function as a basic hardware abstraction layer. Conversely, the OS can count on certain things being initialized and in place before it gets loaded, and call EFI functions (say, to access peripherals, set the clock, etc.) when it needs to.

The current EFI specification is quite complex, even more so than Open Firmware, often specifically to address legacy concerns. There are many modules and protocols which can be selected to be present by the motherboard’s manufacturer. From eyewitness reports at Macworld, the new Core Duo Macs boot directly into Mac OS X without any visible difference from old models. Nobody has yet reported on what becomes visible in “verbose” mode, though, although it’s possible to go to the EFI console by holding down certain keys during power-on. Command-option-E-F-I, perhaps?

One EFI module we can be pretty sure that Apple didn’t include: the legacy BIOS module, which would allow EFI to emulate a standard PC BIOS and allow booting of MS-DOS and its successors. Microsoft has said that EFI support will not be retrofitted to current Windows versions. So, the new Macs may eventually run Windows Vista, assuming they are otherwise Intel-standard; somehow it strikes me as unlikely that Apple will work with Microsoft to have its hardware certified as Vista-compatible.

A final point about EFI is that it expects bootable drives to use its partition standard, known as GPT. Apparently Apple has managed to make this coexist with its own partition schemes, so that, once a disk has been formatted and installed with the universal Mac OS X on one of the new Macs, it will become bootable both on Intel and PowerPC Macs; quite a feat.

In summary, a no-doubt-intended side-effect of EFI as implemented in the new Macs is that it makes it more difficult, if not perhaps entirely impossible, either to boot current Windows versions on them, or to run Mac OS X 10.4.4 and up on industry-standard non-EFI Intel PCs. Personally, I think dual-booting is a big hassle; Microsoft has already said they will eventually release a Virtual PC for the new Macs, and there’s the whole virtualization alternative, some which has been patented by Apple. More later about that.

Apple Macworld

Well, I finally watched the keynote. Streaming with the new codecs in QuickTime 7.0.4 has improved amazingly, I had almost no skipping or stalling at over double the size and quality of last year’s keynote.

Unfortunately the first hour of the keynote was extremely dull… all that US-centric stuff about “bowl games” and “SNL” (both of which I had to look up)… or long demos of messing around with iMovie and iDVD (which I’ve never used) or .mac (ditto)… or podcasts and photocasts. Apparently I’m no longer inside Apple’s main user demographic, and receding further away from it at high speed. I suppose I should be grateful cellphones were not mentioned.

That said, things picked up considerably when the new Macs were shown. No “Otel Inside”, oops, I mean, “Intel Inside”, quite predictably but still a relief. Otellini’s entrance was well-scripted and his mentioning that they had “over a thousand people working on this” was very interesting. (I skipped over the Microsoft part, though.)

Better still, from my standpoint as an investor, AAPL stock climbed steadily throughout the keynote and even gained in after-hours trading, probably a first. It used to fall on both bad news (“told you so”) and good news (“they can’t keep this up!”). While I write this, it’s up over $4 beyond the coincidental $80.86 it closed at on that day.

Regarding the new Macs themselves, I’ll post detailed comments soon. For now, it’s interesting to note that choosing the iMac and the PowerBook/MacBook Pro for refreshing now looks quite logical… the mini wouldn’t have weathered the “same design, same specs, same price” idea as well, for instance, and they couldn’t have done an iBook/MacBook non-Pro for the same price either. Developer discounts on the new Macs dropped to 10% for the iMac and 20% for the MacBook, which indicates that margins are slimmer for now.

One thing which nobody else has commented on yet is that all the new Macs have very customized motherboards. This was of course to be expected for the MacBook, as space in laptops is always at a premium, but the iMac is now fully laptop-like in its internal construction. The last iMac G5 looks completely different internally from the 2nd generation one, and I expect the iMac Core Duo (I suppose that’s the official name?) to be different again.

The effect of that will probably be that people will have little or no luck trying to find a standard-Intel motherboard to run pirated Mac OS X 10.4.4 on. We’ll soon know more details, but I’m reasonably sure that Apple is using their own controller chip, as they always do. Also, both models use EFI instead of BIOS or Open Firmware… as far as I know, there’s no standard 32-bit motherboard out with EFI either. More on that later.

Apple Macworld

Back in August, I wrote:
Rainer Brockerhoff wrote:

…Let’s suppose Steve Jobs goes psycho and deploys the Mac Intel machines with full TPM, TCPA, DRM and whatnot.

…Let’s suppose that we all go mad as well and continue to buy Macs at current quantities or better – say, 5 to 7 million a year.

Now Forbes says:

[J.P. Morgan research analyst Christopher] Danely said Intel should ship its processors into roughly 55% of Apple?s calendar 2006 PC shipments, which should translate into roughly 4.8 million processors.

That is, at the low end of my estimate. I supposed, for the sake of argument, that all new Macs would be Intel-based, but of course selling 10 million Macs a year is OK with me…

So my conclusion still stands:
Rainer Brockerhoff wrote:

What would happen?

Why, the current installed base is something over 30 million PowerPC Macs (or even more, depending on your sources). By the end of 2007, Intel Macs will be perhaps 15% of that. It will take at least 5 years, probably more, for Intel Macs to surpass the PowerPC Mac installed base. In other words, non-TPM systems will be in the majority for several years. Can you see Apple (or their stockholders, of which I’m one) restricting such important markets to 15% of their customers? Or even 50%? For years??

Interestingly, the article also says:

[Danely] estimates that every 1 million processors shipped to Apple Computer… would result in a little less than 1 cent of incremental earnings per share at Intel.

So Apple’s direct impact on Intel’s bottom line will be about 5%; relatively low, but in terms of marketing impact very significant. Intel’s recent announcements about branding and focus changes are very Apple-aligned… Intel’s complete silence about anything Apple-related at CES means they’re playing along with Steve Jobs to preserve the impact of next week’s keynote announcements. Let’s stay tuned.

A large shoe which Apple will drop is, of course, virtualization. Virtualization will tie in with a recent Apple patent which, linked to the TPM chip, will allow the new machines to run other OSes inside a sandboxed “virtual machine”. Those other OSes will believe they have a somewhat slower system all to themselves, while running inside a separate window (and, perhaps, disk partition or disk image).

Putting this together with the previously discussed flash memory hypothesis, the advantages to Apple become obvious. Why should they need to have a standard BIOS or EFI at all? They can boot directly into the virtualization kernel (or whatever it’s called at the moment), from encrypted flash memory, which in turn would run a subset of Mac OS X – perhaps only the kernel with drivers and the windows manager – to have a basic GUI to check out which virtual OSes the user wants to run. This could be a full version of Mac OS X, or any version of Windows, Unix or Linux that runs on the abstracted hardware presented by the virtualizer.

In fact, it seems that a virtual OS could be a stripped-down OS specialized for a single application – something like what’s used now for embedded systems. It might be a gaming OS optimized for full-screen interfaces, for instance, or a TiVo-like appliance, or a multimedia center.

So, Apple doesn’t have to tie itself to standards during the boot process; in fact, this means that even if the user wants to run some non-Apple OS most of the time, the Mac interface will be present all the time underneath. At the very least, you can imagine the current ghastly BIOS user interface neatly presented in Aqua…

Amazingly, I’ve almost caught up with my e-mail, almost a month after returning from the last trip… and the remaining offline work is clearing up, so I hope to be able to post more here again.

The catchy title is just to remind people that Apple has a lot of shoes to drop in the near future.

In particular, there’s been some news recently that I should comment on while I still have all the links. First of all, Apple will prepay $1.25 billion (yes, gigabucks) to a new Intel-Micron consortium to secure supplies of NAND flash memory – the flash memory used in some iPods. This of course is, obviously, also meant to dispel investor’s fears about iPod supply problems, as the present manufacturers aren’t quite able to keep up with Apple’s demands, let alone the rest of the industry.

Separately, the latest build of Tiger for X86 has been cracked; seems they’re not using the TPM encryption capabilties yet, but just checking for the chip’s presence. The latest build also extends Rosetta to emulate the G4 with AltiVec, so a wider range of PowerPC apps should be able to immediately run emulated on X86 – although there’s no word yet about speed ratios.

The Motley Fool and some other folks speculate that Apple will be introducing instant-on capabilities; this would use the suddenly plentiful flash chips to hold parts of the operating system while the power is off. Incidentally, the IBM/360 mainframes I worked with in the 1970’s had something similar; the core memories they used at the time held data when the power went off, so with some care – stopping the processor before turning it off – you often could just continue after turning it back on without a reboot.

Finally, it’s no surprise that Intel has a special “Apple Group” where engineers from both companies work together. As I believe it unlikely that Apple will use a standard Intel motherboard, the most likely focus of this group is to make special motherboards and custom chips for Apple.

Put these bits together and what do you get? I think the X86 hackers are in for a surprise when the new Macs come out. I think Apple will take OS-hardware integration to a new peak with the Intel Macs. They’ll have a gigabyte or so of flash memory where an encrypted version of the Mac OS X will reside – a return to the days of the first Mac 128, when most of the toolbox was in ROM and the “System” file just contained patches and late-minute additions.

This giga-firmware will be encrypted with each machine’s own unique ID – contained in the TPM chip – and will be decrypted on-the-fly as needed into a secure portion of system RAM. Future system updates will come encrypted and be re-coded by the update process, which will run in full secure mode, perhaps even inside the TPM chip itself. Since some Intel CPUs are rumored to have the TPM chip built-in, this hypothesis gains weight. The instant-on capability would be just a nice side-effect…

It appears that a new version of Mac OS X for the Intel Transition Kits is out. As I expected, it has new protection measures in place. (Note: I don’t have a transition kit and would be under NDA if I had one, so all this is speculation based on publicly available data…)

There’s still very little information out on what they actually changed regarding use of the TPM chip. Of course, the previously published patches don’t work on the new version; this also confirms my idea that Apple is using the pirates as a test bed for their protection algorithms. Also, it seems that applications compiled for the new version won’t run on the old version, as the ABI (Application Binary Interface) has changed. This was to be expected; it also happened several times before Mac OS X 10.0 came out; the fact that it also makes the patched systems out there useless for most practical purposes is just a positive side-effect for Apple.

Most comments I’ve seen from the PC user’s side show a lack of information about the TPM chip’s capabilities; they either have an unconditional faith that it will be hacked in a few days, or that it will kill their firstborn. As I’ve said before, my opinion is that it will be very hard to hack – maybe even impossible in practice – but that Apple won’t press its use beyond the one of protecting their intent of restricting Mac OS X to Apple-built machines.

There’s one further misconception to be addressed. Some people say that, once they buy a retail copy of Mac OS X, the part of the shrink-wrap license that says that they can run it only on Apple-built machines need not be obeyed. While I’m not sure if such restrictions have ever been tested in court, there are a few ways that Apple can strengthen its position.

For one, they could simply stop selling retail licenses of Mac OS X; a copy of the system would be included with every machine and they would sell only updates separately. Remember that the only reason that retail copies of Mac OS X for PowerPC were put on the market was to entice users to upgrade from Mac OS 9; it’s not like Microsoft selling retail copies of Windows for generic PCs. And of course I don’t believe that Apple will make Mac OS X available for generic PCs in the foreseeable future…

The first Mac, in 1984, had most of its operating system contained in ROM; the boot diskette contained the Finder and a System file which consisted essentially of patches to the ROM. Apple’s ROMs were jealously guarded – relatively easy as they were part of the hardware – but boot diskettes were freely distributed. With Flash memory capacity going up, Apple might even return to this model.

Well, sites are still abuzz: the Apple Intel Transition Kits have been opened and photographed, and at least some of them are based on a stock Intel motherboard. With an Infineon SLD9630-series TPM chip on it.

Some libertarians and open-source advocates came out swinging since, as usual, TPM, TCPA, DRM and even the moribund Palladium are usually used interchangeably. Some cooler heads popped up, but the Slashdot thread was particularly entertaining. Cory Doctorow even said:

Apple may never implement this in their own apps (though I’ll be shocked silly if it isn’t used in iTunes and the DVD player), but Trusted Computing in the kernel is like a rifle on the mantelpiece: if it’s present in act one, it’ll go off by act three.

…

It means that the price of being a Mac user will be eternal vigilance: you’ll need to know that your apps not only write to exportable formats, but that they also allow those exported files to be read by competing apps. That they eschew those measures that would lock you in and prevent you from giving your business to someone else. I’m pretty sure that apps like BBEdit and NetNewsWire won’t lock me out, as their authors are personally known to me to be wonderful, generous, honorable people. But personally familiarizing yourself with the authors of all the software you use doesn’t scale.

So that means that if Apple carries on down this path, I’m going to exercise my market power and switch away, and, for the first time since 1979, I won’t use an Apple product as my main computer. I may even have my tattoo removed.

Now, Cory is one of my favorite authors, and Boing Boing is the first site I check every day, but this unfortunately just plays into the hands of the FUD crowd.

The situation is somewhat complex. Granted that DRM is a bad thing and should be avoided. Granted that, as the EFF says, a trusted platform can be misused under certain circumstances. However, jumping from the mere presence of the TPM chip to the conclusion that DRM was one of the reasons for the entire Intel switch is very far-fetched. Even if the Mac OS X kernel checks for, and uses, the TPM chip to prevent users from running Mac OS X on a non-Apple PC – a use I consider entirely legitimate – suddenly switching in mid-paragraph to DRM protection of media, and encryption of formerly public data formats, is unwarranted. (This very common confusion between the aims and definitions of TPM, TCPA and DRM is quite well addressed in a rebuttal paper published by IBM a few years ago.)

And the examples given – BBEdit and NetNewsWire – are totally off anyway. BBEdit, by definition, saves stuff in plain text format! If I want to mess about with any other type of format I certainly wouldn’t use BBEdit anyway; and NetNewsWire exports data in OPML format because it’s public. Regarding closed formats, the StuffIt .sit has been very popular (although completely undocumented) since before Cory became a Mac user in 1979, yet few people have complained about this until very recently.

The obvious and ostensive purpose of the TPM chip is, of course, locking Mac OS X’s Intel version (which strictly speaking is a “universal binary” version) to running only on the transition kits. Apple has repeatedly and clearly stated that they won’t allow Mac OS X to run on non-Apple computers; and, since the clones were killed and Apple has started charging for the system, this has always been stated in the license terms:

2. Permitted License Uses and Restrictions.

A. This License allows you to install and use one copy of the Apple Software on a single Apple-labeled computer at a time.

This was sort of redundant in the past, as there were no other PowerPC-based general-purpose computers on the market at any time; and few people would have argued, as some seem to be doing now, that they have a “right” to install Mac OS X on computers explicitly disallowed by the license – not the least because doing so would have entailed an impossible amount of work. Strangely, the amount of work required has now been perceived to have shrunk enough to argue for this position…

So, what’s inside the TPM chip to make it so repulsive? Infineon has published some technical data. It’s a simple microcontroller with a standard LPC interface, like other on-board peripheral chips have; the interface’s peak bandwidth is about 4MB/s. It has several types of internal memory, including 16KB of EEPROM used to store secure keys, a random number generator, specialized hardware for fast generation and checking of SHA-1 digital hashes, similar hardware for RSA encryption and decryption, and several anti-tamper facilities. In other words, it’s a simple peripheral that can receive blocks of data and return other blocks of data, transforming them according to the algorithms and keys stored inside. These can be the same for all deployed chips or can be different for each equipment, presumably at some additional cost.

Naturally, all those functions could be equally well performed by software running on the main CPU. The difference is that with a relatively trivial effort, such software could be intercepted or decoded by a hostile party; this is several orders of magnitude harder to do with the TPM chip, as long as suitable precautions are taken by the software which accesses it. Note that usually the “hostile party” is not the computer’s owner, who always has the option of ripping the chip off the board, or more practically, spending his/her money elsewhere! It all depends on how the chip is used, as well see below.

I don’t have access to a transition kit myself, and would be under NDA if I had. However, from what I’ve read on the net, the Mac OS X boot process checks for the presence of the TPM chip and later on uses it to verify, or decrypt, some parts of its own object code; a perfectly legitimate use. Will this chip be in the Intel Macs when they come on the market? We can’t say, of course. Its presence in the transition kit units is of course explained by the fact that time constraints forced Apple to use a standard Intel motherboard. In their own systems Apple will be free to integrate TPM-like functions into the system controller chip, for instance, since they’re only using a small subset of the chip’s capabilities. They could also ask Intel to produce slightly modified versions of their CPUs, which might restrict Mac OS X to Macs as effectively as running on a PowerPC did in the past.

What if they opt for the easy way out and simply keep the full TPM chip for the standard Intel Macs? If the chip is used only for restricting Mac OS X to Macs, no problem. But what if they use it for more nefarious purposes, say, for strengthening the iTunes DRM encryption? I don’t see how this would be any more harmful than the existing situation; civil libertarians are concerned about the existence of DRM, not about its relative efficacy. By its construction, the chip’s functions are only accessible to “trusted” software modules, meaning that it wouldn’t be useable by any random application.

The TCPA specifies a complex system of interlocking security procedures to produce a “trusted” system – meaning a system where both the user and the software running on it can trust everything to be in a known state. That is, the BIOS (or EFI, or whatever) and the TPM chip are mutually certified, then every single piece of software or hardware that is loaded, or initialized, later must be similarly examined and certified by a trusted function. If a non-certified item is encountered, it must be either rejected altogether or the system must enter an “untrusted” state. So, to do this consistently, Apple would have to reject all third-party peripherals and software applications! Certainly the CIA, FBI and some corporations would be eager to buy a number of such systems, but who else would? Notice that you could also just protect certain data paths – as Microsoft’s Protected Media Path project is supposed to do; to be really effective, this would mean trusted DVD drives, loudspeakers, monitors and so on. Banks would need trusted keyboards and network cards. In other words, this is not something suitable for your general computing needs.

Even so, let’s suppose that Cory is right. Let’s suppose Steve Jobs goes psycho and deploys the Mac Intel machines with full TPM, TCPA, DRM and whatnot. Let’s suppose that all things at the iTunes Music Store will now be sold only with strong encryption, as well as all Pixar films, MTV music videos, and so on. Let’s suppose all other software companies are infected and stop support for plain text files, HTML, XML, etc. Let’s suppose that we all go mad as well and continue to buy Macs at current quantities or better – say, 5 to 7 million a year. What would happen?

Why, the current installed base is something over 30 million PowerPC Macs (or even more, depending on your sources). By the end of 2007, Intel Macs will be perhaps 15% of that. It will take at least 5 years, probably more, for Intel Macs to surpass the PowerPC Mac installed base. In other words, non-TPM systems will be in the majority for several years. Can you see Apple (or their stockholders, of which I’m one) restricting such important markets to 15% of their customers? Or even 50%? For years??

If you really believe this, I have a bridge here I’d like to sell…

…as I was posting this, I saw that John Gruber, as usual, has a highly cogent post about this very subject.

Posted by keoladonaghy:
“The soul of the Mac is the CPU”.

If I’m not mistaken the official name of the operating system used to be, and perhaps still is, simply “Macintosh.” The OS is the soul of the user experience, not the hardware, IMHO. As long as Apple continues to assure the quality of the user experience and the hardware, I don’t care what chip drives the thing.