Solipsism Gradient

Rainer Brockerhoff’s blog

Browsing Posts in Development

There’s been so many comments about Leopard over the weekend that I stopped reading – and there are too many of them that just repeat each other, too.

I’ve been running Leopard since the first seeds came out, and the last few have been really stable, especially the last one; I didn’t have to reboot it once in about a month.

The final release – 9A581 – was built on Oct. 12 but released to developers on Oct.26, the same day that it (theoretically) was released to users; some journalists got it earlier under embargo. In past major releases, the final build’s release date for developers was uneven – sometimes just a few days before, sometimes as much as a week after.

Buzz Andersen, a former Apple employee, wrote a very good post pointing out the difficulty of interpreting Apple’s actions from the outside. While I personally think Apple’s two-week delay in posting the final release for developers was unfortunate, I must point out that past releases leaked on the torrent sites in less than a day. If some developers won’t honor their NDAs, everybody will suffer for it.

Similarly there’s much controversy about stuff that got suddenly (or not-really-so-suddenly) taken out of the final release; 64-bit Carbon apps, ZFS support, Java 1.6, backing up over wireless are the ones that immediately come to mind. As usual, people are reading into that all sort of background motivations – Apple is following some Machiavellian scheme, or is completely stupid/clueless. I prefer to believe that they’re doing the best they can with their limited resources while trying to follow a multitude of small individual agendas. Ants carrying a large item into their nests come to mind… icon_smile.gif

For now, I’d just like to point out that, as in previous years, 10.5.1 will be out within 15 days, probably fixing at least one of those omissions. I think Apple made a good decision in, for the last month, concentrating on polishing existing features. Leopard is unusually smooth and “finished” for a .0 version.

On a personal note, and as I posted to the XRay Support Forum a few days ago, XRay 1.1 suffers from some problems in the final Leopard release. The most annoying is that the file browser doesn’t allow you XRay an item – it will crash.

I’m fully resolved to step up efforts to release XRay II, at least in public beta, as soon as other commitments allow. It will be Leopard-only and everybody who paid for XRay 1.x will get a free upgrade to the “standard” edition (there may be a “pro” edition, but I’m not sure yet).

One commitment which, unfortunately, is a great deal more pressing (literally!) is that I’ve contracted to write a book about “Programming Objective-C 2.0”. This also is specific for Leopard, and as you can imagine, deadlines are very short; ideally, of course, the book should be out today! But, the laws of physics and physiology permitting, it will be out as soon as possible. Watch this space for details.

Meanwhile

No comments

While I haven’t had time to really look at, or comment on, Leopard yet, this article by Matt Legend Gemmell is must reading for Cocoa developers. Nice job.

Overnight, Apple’s changed the Leopard Developer Tools page to confirm officially that the tool formerly known as Xray is now called Instruments.

Well, while both names are certainly better than the prototype name (which, supposedly, was “PowerTrace”), I’m both relieved and worried by the change. When the name first came to my attention over a year ago, some Apple folks told me privately that I shouldn’t worry about any conflict with my own XRay utility (now being reincarnated as XRay II). Still, I hoped that the similarity might drive some clients my way, and I even linked from my own page to Apple’s, to avoid any confusion.

Now, this last-minute change is a also little worrying, since it’s probably a symptom of a cease-and-desist letter. This came up so suddenly that even the icon on the Apple site still uses an “x-ray machine” theme. I can’t find any larger Mac software company using any variation of “X-ray”, but who understands how lawyer’s minds work? It might even be a non-software company. Hopefully my new second-generation name will be non-conflicting enough to avoid any trouble.

Steve Jobs just said (I guess I should say, Real Steve Jobs, hehe) on his blog:

…We want native third party applications on the iPhone, and we plan to have an SDK in developers’ hands in February.

…Nokia, for example, is not allowing any applications to be loaded onto some of their newest phones unless they have a digital signature that can be traced back to a known developer. …we believe it is a step in the right direction.

This seems to indicate that the application installer – which will in all probability be iTunes – will check if the application is properly signed. Whether they’ll allow developer-signed apps is anybody’s guess, but I wouldn’t rely on it. (Signed apps is one of the 300 Leopard features, by the way. I’ll comment on the Leopard day announcement in a few days.)

I wrote two weeks ago:
Rainer Brockerhoff wrote:

Conclusions:

– the current generation of iPhone/iPod touch will remain closed forever, just like the first generations of iPods; (I was wrong there, and a good thing too!)

– an SDK is likely to come out only after everything (especially the hardware) has stabilized;

So the February OS X version will be the first one with stable, public APIs… meaning current apps, written to reverse-engineered specs, will probably have to be seriously rewritten.

Rainer Brockerhoff wrote:

– Apple is unlikely to invest efforts into implementing TrustZone in the current generation, unless Moorestown (or whatever else they might adopt in the future) has a similar security feature – and maybe not even then

Now I wonder how they’ll handle such a hypothetical future hardware migration… probably fat binaries, with the “other” executables being stripped out by iTunes when installing an app; this would be the most flexible without upping memory footprint on the phone side.

Update: Seems that Intel and ARM are collaborating on new TrustZone implementations… might that foreshadow TrustZone on Moorestown…?

Now, some people say this proves that Apple is listening to complaints and that they’re changing their original plans; on the contrary, I think this had been the plan all the time, but the Leopard delay also delayed the SDK. Regarding the timing of this announcement, this might be a trial balloon to see if they can minimize the inevitable profit-taking after next week’s earnings announcement. Hopefully that will happen.

I said previously:
Rainer Brockerhoff wrote:

…So, keeping things closed for now means the software hasn’t stabilized, and very probably the hardware hasn’t stabilized either.

Here’s more evidence for that…

Erica Sadun over at TUAW announced preliminary results on the iPhone 1.1.1 software:

– Third Party apps run. Kind of. We probably have to recompile many of them for the new frameworks because many of them crash.

– Springboard no longer recognizes DisplayOrder.plist. And the list of “whitelisted” apps (that is, the official Applications including Safari, Photos, Calendar, etc) seems to be hard-coded into Springboard.app

– The 1.1.1 binaries barely work with 1.0.2 – at least not well enough to run the music store without major hacking.

Rumors say Apple may switch the iPhone main processor to Intel’s upcoming Moorestown.

It’s too early to speculate until details come out, but it wouldn’t be a surprise to hear that Apple is considering this. And it would explain the closedness of the iPhone/iPod touch architecture… after all, once Apple allows third-party apps in, and publishes a toolchain/SDK, they’re pretty much locked into the current architecture, and switching to a new one is a major/slow/costly undertaking.

Consider the previous iPods as a counterexample. Apple has switched architectures there – we can’t even say for sure how often – without any users noticing. With only the UI visible on OS X, and no toolchain/SDK or even documentation of the innards, Apple is free to change things radically between software updates. By all accounts, the 1.0.x software is pretty much a work in progress, and 1.1.1 has probably changed a lot.

So, keeping things closed for now means the software hasn’t stabilized, and very probably the hardware hasn’t stabilized either.

Conclusions:

– the current generation of iPhone/iPod touch will remain closed forever, just like the first generations of iPods;

– an SDK is likely to come out only after everything (especially the hardware) has stabilized;

– Apple is unlikely to invest efforts into implementing TrustZone in the current generation, unless Moorestown (or whatever else they might adopt in the future) has a similar security feature – and maybe not even then;

– the fabled OS X tablet will come out when the new hardware is ready; by that time screens will be ready in the proper sizes; Sony showed an 11″ OLED TV recently, remember…

After several months of tinkering and getting used to the new IB, I just published a first beta of the IB3 plugin for RBSplitView 1.1.4.

Some things haven’t been tested (copy&paste), others don’t work fully (undo), but it seems to work mostly. Please post bug reports in the source forum, or e-mail me. To install, close IB3, unzip the plugin in a convenient location (I don’t think there’s a standard one for IB3), and double-click it.

A full Leopard version will be out around the end of this month, along with source code for the plugin.

State of the iPhone

No comments

So, half a dozen softwares are now out there that unlock the iPhone in various ways. In the simpler case, they allow the installation of various third-party applications and/or twiddling details. In the more complex case, they mess around with the various phone/SIM settings to allow the iPhone to be used with other provider’s SIM cards.

As I wrote before, Apple has apparently allowed this to happen by not implementing strict security measures. Now that the various unlocking techniques have stabilized, Apple has announced that an upcoming software update might cause “modified” iPhones to become “permanently inoperable”. Just a few days later, the iPhone update to version 1.1.1 came out; it featured the same warning in bold on its installation screen; and it did, indeed, cause some modified iPhones to lock up – the new vernacular is “bricked”, which I think somewhat of an exaggeration. Furthermore, the new software seems as tamper-resistant as the iPod touch software, indicating that Apple has checked out current unlocking techniques and implemented harder locks.

So far, all that was to be expected. What was, to me, unexpected was the reaction of some sectors of the press and of some users – mostly the same people who opposed the iPhone price cut, it seems.

Legally, it seems Apple is in the clear. The warranty and license agreement clearly say that any such tampering is at the phone owner’s risk. Surprisingly, some people seem to feel “entitled” to get warranty support even if they completely disobeyed the license! (Just as they felt “entitled” to have the price kept constant for a long period after they bought it, I suppose.)

The core of the argument seems to be “I paid for the machine, therefore I have the right to do whatever I want with it…” (I completely agree so far) “…and Apple has the obligation to give me full support, warranty and updates no matter how I mess with it!” Now here is where we part company. Sure, I suppose current consumer protection legislation may sometimes be interpreted that way (note I’m not a lawyer and less familiar with US legislation than with the Brazilian one); but you surely can’t pretend that Apple is a public utility or a non-profit charity.

Even from the technical standpoint, these expectations are unreasonable; allow me to explain this in more detail. The problem is one of “state“, in this case defined as ” unique configuration of information in a program or machine”.

In the first computers, the state of the computer was completely predictable when it was turned on: if it had Core memory, it was in essentially the same state it had when it was last turned off, and if the computer had reasonable power supply sequencing, you could just press the start button and continue. For more complex machines this was too hard to do, and the manufacturers declared that the machine was in an undefined/unreliable state after power-on, and that therefore you had to reload the software. For newer machines with semiconductor memory, everything was of course lost during power-off, and software reloading became equally necessary. To do so, you had to enter a short program in machine language using the front-panel toggle switches; this program, in turn, would read the actual software you wanted to run from a peripheral. This was the direct consequence of the machine coming up in a “null state”.

It wasn’t long before people thought of several ingenious ways to make this process more convenient. On the IBM1130, for instance, the hardware was set to read a special card from the punched-card reader, interpret the 80 columns (12 holes each) as 80 16-bit instructions, and execute them. The most commonly used of these cards simply repeated the process with the built-in cartridge disk drive, reading the first sector on the cartridge and executing it. Later on, the falling cost of ROM led to the boot software simply being built into the machine – the Apple II had a complete BASIC interpreter built-in, for instance. The apex of this evolution was the original Mac 128, where most of the system software was in the boot ROM – the system disk simply contained additions and patches. (The QI900 microcomputer I helped design in the ’80s had all system software, with windowing, multitasking and debugging, built into its ROM.) Here we had a well-defined “state” when the machine came up – it would execute a well-known program, and do predictable things, until external software came into play.

In the ’90s the limitations of this became apparent. OSes grew to a size beyond what could be stored in ROM, and no single Boot ROM could do justice to all models and peripherals (*cough* BIOS). Flash memory came up, the built-in software was renamed to “firmware”, and updates to that became commonplace. It was easy to “brick” a system if power went out or if you otherwise interrupted a firmware update before it was complete. In that event, a motherboard swap was usually the only solution, because the interruption left the firmware in a partial, nonworking “state”.

Consider now the iPhone. Its entire system (OS X 1.x) is built into firmware, mostly in a compressed state. This is expanded and run by the main ARM processor, obeying a built-in boot ROM. Supposedly, there are at least two more processors, taking care of network communications and of the cellular radio; each of these has its own boot ROM, and the radio processor has separate flash memory to hold state information regarding the SIM card, cellular system activation and so forth. One of these processors no doubt controls the USB interface to allow the main processor’s flash memory to be reloaded externally. Furthermore, every SIM card also has flash memory on it, containing the IMSI number, network identity, encryption keys and so forth, bringing one more source of complexity to the process.

In other words, you have a complex system of at least 3 processors interacting, each one with a boot ROM, two with flash memory containing state information. Powering up such a beast is a complex dance of each one waking up, testing its peripherals, checking its own state, then trying to talk to each other, then communicating to bring the entire system into a working state. Furthermore, the necessities of the cellphone system and of testing out such a complex piece of hardware mean that the iPhone must decide, on each power-up, in which of several states it’s in: factory testing, just out of the box, activated, reloading the main firmware, working, “plane” mode, and so forth. This is usually done by writing special values to reserved sections of the various flash memories, and of making sure they are always consistent with each other by checksumming and other technical arcana. Should they be found inconsistent, the system will probably try to regress to a simpler state and start over there, in the extreme throwing up its metaphorical hands and plead to be returned to the factory. Ideally, firmware writers strive to make it impossible to “brick”, unless an actual hardware defect occurs, of course; in practice, it’s rarely possible to envision all possible combinations of what could happen, and too few designers do assume a malicious agency is trying to trip them up at all times.

So, what do these various hacks do to unlock the iPhone? They rely upon bugs in the communications software, firstly, to make the system fall back into a state where it pleads for an external agency to reload its main firmware; cleverly substituted instructions then make it do new things. After several, progressively more complex, phases of this, new applications can be installed. Up to this point, only the main flash memory has been affected and installing a new software update will just bring the system back to the standard state. Now, one of the new applications may try to mess with the radio firmware; it will clear or set regions of it to bring the radio processor’s state out of step with reality, or even write bogus activation data into it.

Now, of course, the system’s state has been moved completely out of the state space envisioned by its designers. When it powers up, the state is sufficiently consistent – the various checksums check out OK, for instance – for the various processors to confidently start working. However, a few actual values are different from the intended ones – enough to let a different SIM card work, say. Now, if the hackers had the actual source code and documentation available, all this could be done in a reliable way. But this not being the case, they had to work by testing changes in various places and observing what happened, clearly not an optimal process.

Consider, now, the software update process. It assumes that the iPhone’s various processors and firmware(s) are in one of the known states – indeed, this is required for the complex cooperation required for uploading new software. If this cooperation is disrupted, the update may not begin – leading to an error message – or, worse, it may begin but not conclude properly. At this point, one or more of the iPhones processors may try to enter a recovery routine, either wiping the flash memories or to reinitialize them to a known state. No doubt this will be successful in most cases, and the new update will then be installable on a second attempt. However, the recovery may fail – since the exact circumstances couldn’t be foreseen – or it may be assuming false preconditions (like, a valid AT&T SIM card being present). The system will probably try to recover at successively lower states until falling back to the “can’t think of anything more, take me back to the factory” mode; or it may even lock up and “brick”.

Should Apple’s firmware programmers have tried to prevent this from happening? Well, up to a point they certainly did, as many problems other than hackers can cause such errors – electrical noise, badly seated or marginally defective SIM card, low battery, for instance. The system has to fail gracefully. However, it’s certainly not reasonable to expect them to specifically recognize and work around (or even tolerate!) the various hacks; after all Apple’s contract with AT&T certainly requires them to evidence due diligence in preventing that.

Firmware for such a complex system evolves continuously. The new 1.1.1 iPhone software seems to do many things differently from the original version, even though much of the UI is the same; same goes for the iPod touch software. Neither has been hacked as I write this. Did they now put TrustZone into operation? No idea; time will tell. My hunch is that Apple will eventually come out with an SDK for third-party applications sometime; the question is when. Perhaps after Leopard, perhaps at the 2008 WWDC. Does Apple need AT&T, or any partner carrier, at all? Maybe for now they do, and the unlocking wrangle will continue. In the long run, Apple will be better off with a universal phone that will work anywhere; possibly we’ll have to wait for the current generation of carriers to die before this happens. Interesting times.

Photos licensed by Creative Commons license. Unless otherwise noted, content © 2002-2025 by Rainer Brockerhoff. Iravan child theme by Rainer Brockerhoff, based on Arjuna-X, a WordPress Theme by SRS Solutions. jQuery UI based on Aristo.