Solipsism Gradient

Posted by …from every angle:
…from every angle linked to this post

The Hubbub About iPhone SDK

I’ve been thinking about the possibility of writing an app for the iPhone ever since it appeared on the news. Two problems, though: I don’t really know how to program anything other than some basic php stuff, and some SQL…

Good extra argument against background processes from Craig Hockenberry:

The heart of the problem are the radios. Both the EDGE and Wi-Fi transceivers have significant power requirements. Whenever that hardware is on, your battery life is going to suck. My 5 minute refresh kept the hardware on and used up a lot of precious power.

Two interesting developments came up while I’m studying for my next post about code signing.

Firstly, every developer who has signed up for the $99/year program has gotten a letter which says, in part:

We have received your enrollment request. At this time, the iPhone Developer Program is available to a limited number of developers and we plan to expand during the beta period. We will contact you again regarding your enrollment status at the appropriate time.

The message for non-US developers also mentions that the program will be implemented in the US first. At this point, some developers went off in a huff, interpreting it as a rejection, and numerous complaints and conspiracy theories have been aired. So far (as I know) nobody who applied to the $99 program has been formally accepted – the ones who said they were are apparently newbies interpreting their ability to download the SDK as “acceptance”. On the other hand, a handful of companies have stated they’ve been accepted into the $299 enterprise program.

Well, my understanding of English may be faulty (it’s only my fourth language, after all) but let’s look at the 3 sentences I quoted above:

We have received your enrollment request.

Nice of them to reply.

At this time, the iPhone Developer Program is available to a limited number of developers and we plan to expand during the beta period.

This is just repeating what Steve Jobs said during the iPhone event… nobody would expect Apple to accept all applicants immediately and at the same time. There certainly are several thousands at the very least.

We will contact you again regarding your enrollment status at the appropriate time.

Meaning, as soon as we dig out from under, we’ll check your application. I see no problem there either.

Apparently, some people are indeed interpreting this as a poorly worded rejection, or at best as an ambiguous directive that you have to go to the end of the line or try again. Now, I’m the type of person who has difficulty in understanding the logic of “I’m afraid I’ll have to tell you that your credit card has been declined” – the first time I heard that in person I couldn’t follow at all (heh! he’s afraid of what? will he have to say what in some unspecified future? Why doesn’t he then say whatever he will have to say, then?) But I can’t see anything like that buried in that last line either.

Second item is that the SDK’s restriction on background process has been much-commented on, among others by John Gruber and Mike Ash. Along the way, there have been complaints about this and other restrictions imposed on third-party applications. Some have even pointed out (I can’t confirm or deny) that the APIs published in the SDK are just a small subset of the APIs ferreted out by people examining the previous iPhone firmware. Some commenters even seem to believe that Apple not publishing certain APIs is illegal in some way… witness the comments about unpublished APIs called by WebKit/Safari some weeks ago.

Well, Apple can certainly opt to tell people not to use certain APIs. In fact, on the Mac in the past, some large software publishers went ahead and used some of those “hidden” APIs anyway, with the result that Mac OS X still has to support lots of legacy calls that are broken, but can’t even be properly fixed without breaking those applications!

So, on the iPhone, they’re certainly making sure to avoid the legacy/unsupported/hidden API problem at all costs. Apparently the SDK license says that applications may not use any unpublished API, period. Note that this does NOT mean that Apple will have to examine your precious source code line-by-line; no, they’ll just run a tool that looks at your executable, sees what symbols in which frameworks are referenced, and produce a listing of whatever non-documented stuff you’re calling.

So, there are no documented APIs that let you run an application in the background, or start a background process. I’m sure we all agree that having any GUI application continue in the background after another GUI application starts will be too resource-hogging – after all, if you allow one, you have to allow any. From what I’ve read, no Apple application does that either; a few seem to have a background process or daemon running to take care of communications, but that’s of course much more lightweight.

I agree with Mike that obviously the iPhone OS does support multitasking in a generic way. I also agree that the relatively limited RAM (I heard that 64M are available to a third-party application) and CPU speed are no obstacles. To mention an even older example than he gives, I managed to write an embedded system running on a 4MHz Z80A CPU, with 32K of RAM, that handled a GUI, a foreground thread, and two background threads tracing several realtime signals on a screen.

But make no mistake, embedded systems are hard to keep responsive, even if you have no arbitrary code running at all. Apple apparently elected not to have full virtual memory swapping in the iPhone OS; here again, it’s not a limitation of the hardware (Flash memory can support swapping) or of the OS (it’s Unix-based after all) or of the CPU (it does have page tables etc.).

No, I think Apple simply is trying to keep “teh snappy” always happening, keep battery duration as high as possible, and avoid Flash RAM filling up by swapping. Also, swapping pages takes time and burns battery power. A wait of 2 or 3 seconds may be shrugged off if you sit at a desktop computer but may be unacceptable if you’re trying to answer a call on the iPhone.

Consider the MacBook Air with SSD. It has 64GB of flash and 2GB of RAM, with full swapping virtual memory of course. I’ve seen people leaving 20 or 30 applications running on a Mac and not noticing that their swap files grew to 12 or 20GB… but even that may already become a problem on the Air. I suppose Apple didn’t want to face telling people that they should leave 2 or 3GB free on their 8GB iPhone… and risk crashing or hanging when it suddenly fills up.

Still, I feel that if you do have a compelling application that absolutely depends on having a background process running, you’ll probably be able to ask Apple for an exemption… if you can prove that your process won’t hog the system. Also, Leopard (for one) has facilities to start processes on-demand – using launchd, you can have the system watch a certain port or socket and start up your process when data arrives there. No doubt we’ll see more such facilities being deployed.

Before getting to the actual discussion of code signing on the iPhone, here’s an interesting tidbit I discovered while researching details about Apple’s certificates: a PDF file detailing the iPhone certificate conditions – officially, it’s a CPS (“Certification Practice Statement”). This is publicly available, and linked to from Apple’s Root Certificate Authority page, by the way.

Some interesting parts:

This CPS is applicable to the following certificates issued by the WWDR Sub-CA:

? WWDR iPhone Software Development Certificates (“Development Certificates”)

? WWDR iPhone Software Submission Certificates (“Submission Certificates”)

? WWDR CRL Certificates (“CRL Certificates”)

That means, in order, certificates for testing an application on actual iPhone/iPod Touch hardware during development; for actuallly submitting an application for publishing on the App Store; and finally, for submitting a CRL (“Certificate Revocation List”). This shows that development and publishing keys are different, as I’d thought. It remains to be seen whether the publishing keys will be applied by developers themselves before submitting their apps, or (more likely) by Apple after verifying the apps for compliance. Also, CRLs are the technical explanation for Steve Jobs’ comment that “if [developers] write a malicious app we can track them down and tell their parents”… meaning, certificates can be revoked and those apps will cease to function.

No fees are charged for this service. Digital certificates are available at no additional cost to

members of the iPhone Developers? Program. Certificates are valid for the duration of the membership period unless otherwise revoked.

This clears up details about the $99 fee. This is an annual fee, then, for membership in the “iPhone Developer Program”, distinct from the “Mac Developer Program”. See the new http://developer.apple.com/ page, which is now split left/right between these two programs. The certificates themselves are free of charge. Elsewhere, expiration and renewal are discussed, but no actual expiration period is mentioned.

There doesn’t seem to be an equivalent document for signing Mac OS X applications, although a separate page says that you can apply to Apple for the purpose of getting your own root certificate recognized. The conditions (especially regarding auditing) appear to be much more stringent, so this looks to be directed at larger companies. Even so, there’s no fee for that either.

Mike Ash recently posted about code signing. We’d talked about the subject a few days before and that is a great summary of Mike’s thoughts about it.

Code signing on Mac OS X is still evolving, but with the release of the iPhone SDK, the subject is suddenly being thrust into the limelight. To quote the Wikipedia article:

Code signing is the process of digitally signing executables and scripts to confirm the software author and guarantee that the code has not been altered or corrupted since it was signed by use of a cryptographic hash.

In the past, code signing has been used on several platforms with differing intents and implementations; Microsoft, for instance, has a driver signing program that lets only Microsoft-certified device drivers run without warnings on Vista. In contrast, here’s a statement from an Apple engineer on the apple-cdsa mailing list:

Windows “code signing” is not only completely different in design and implementation, it also has a completely different *intent*. The Windows version tries to determine whether *Microsoft* thinks your program (or driver etc.) should be used. Mac OS X Code Signing is a general medium to let the manufacturer and user negotiate as to whether/how they want to trust each other, with the system providing the infrastructure to make this possible.

From what I’ve learned about the subject so far, the Mac OS X implementation seems indeed to be aimed in this direction – there’s very little documentation available. Even less is, of course, available for the iPhone OS implementation, and details are under NDA. But let me try to explain what I learned so far.

Mac OS X code signing is an implementation of the X.509 standard public key infrastructure. It depends on a cryptographical algorithm which gives reasonable assurance that signatures cannot be forged with currently wide-available technology. Of course that’s a rapidly moving target, and the twiddly mathematical details are beyond the scope of this post (and, in parts, of my poor brain too).

So, you can have signed executables (single binary files) and signed bundles (which can applications, frameworks, plug-ins and so forth). A signed bundle is easily distinguished by the presence of an extra “CodeResources” file inside the bundle’s Contents folder. This file is a XML file containing, among other data, a dictionary that contains a hash code for every file inside a bundle. It also has a list of “optional” files or folders; this allows you to remove things like unwanted localizations from an application without invalidating the signature. Of course, if you later re-add a localization its hash codes must match the one inside the file. Additionally, a signature is added to the bundle’s executable.

If you check Mac OS X’s system folder and applications, you’ll find that, beginning with Leopard, every framework, application and command-line tool is signed. This signature, by various convoluted means, is checkable (currently only by Apple’s codesign command-line tool). It also points back at Apple’s “root certificate“. This certificate is is a component in X.509’s chain of trust tree, which basically tells you which authorities certify that a particular signature is valid; in effect, when you reach the root of such a tree you have to decide whether the root certificate can be trusted. (X.509 also can be used for distributed, or peer-to-peer, trust networks.) Let’s see how you can check this in practice. Open Terminal and try this:

$ codesign -vvvv /Applications/Safari.app
/Applications/Safari.app: valid on disk
/Applications/Safari.app: satisfies its Designated Requirement

(as usual, type the line after the $ prompt and you should see the resulting lines.)

So this means that the Safari application is signed and its signature is intact. Otherwise, you’d get either a “code object is not signed” message, or if the signature has been tampered with, a “code or signature modified” message. Now let’s try a small variation on the command:

$ codesign -dvvvv /Applications/Safari.app
Executable=/Applications/Safari.app/Contents/MacOS/Safari
Identifier=com.apple.Safari
Format=bundle with Mach-O universal (i386 ppc7400)
CodeDirectory v=20001 size=7621 flags=0x0(none) hashes=375+3 location=embedded
Signature size=4064
Authority=Software Signing
Authority=Apple Code Signing Certification Authority
Authority=Apple Root CA
Info.plist entries=23
Sealed Resources rules=9 files=283
Internal requirements count=1 size=68

This gives more information. Specifically, we see that there are 3 “certification authorities” included into the signature, and that there is one “internal requirement”. To see this requirement, try:

$ codesign -dr- /Applications/Safari.app
Executable=/Applications/Safari.app/Contents/MacOS/Safari
host => anchor apple and identifier "com.apple.translate"
# designated => identifier "com.apple.Safari" and anchor apple

This is admittedly hard to wrap your neurons around. From what I understood, the “requirements” are actually a little language that establishes what conditions the signed bundle should satisfy. But satisfy to able to… what? Now there’s where everything is (on purpose) still a little vague. “anchor apple” apparently means that the basic requirement is that Apple’s root certificates be present and trusted. Open the Keychain Access utility, click on “certificates”, and you’ll see somewhere in the list a certificate called “Apple Root Certificate Authority”. You can double-click on it to see details.

But what is all this useful for? Your proverbial “Aunt Tillie” will certainly not be interested in typing commands into Terminal, following certificate chains, and reading up on trust audit procedures. Apple’s keychain software tries to make all that as transparent as possible. Simply logging into Mac OS X can open parts of the keychain, and when Aunt Tillie goes to her banking site, the bank’s certificate should be checked against the keychain’s certificates and with the issuing certification authority, and everything should “just work”. For that to be reliable, Safari’s code signature should also be checked – otherwise it could be hacked by some malicious entity to say that the bank’s site checks out OK even when it’s some phishing site masquerading as that bank!

Signatures and certificates are also checked by the parental control preference panel, by Mail when it receives signed e-mail, and so forth. You may be familiar with the dialog asking if a recently-updated application may access the keychain to store or retrieve passwords; for signed applications, this dialog will be issued only once. If a newly-installed update has the same signature as the previous version, which had already been authorized to access the keychain, it’s assumed to also be trusted to do so.

A similar scenario applies to critical code such as the codesign executable itself. Ideally, such code should refuse to run if it’s modified; Apple has some facilities already in place to do so, and even to detect if an executable ihas been modified while it’s already running by some malicious code injection. However, that sort of thing isn’t widely used yet; mostly because it would break certain facilities, like Input Managers, that people currently rely on.

One aspect I haven’t touched upon yet is the so-called “self-signed certificate”. This is a certificate which is its own root certificate, like the top certificates of the various ‘big” authorities. The difference is that these authorities are large companies that undergo serious auditions and verifications. As an example, Quay is currently self-signed; I’m still evaluating how to publish my certificate in a way that easily allows my users to check if new versions have indeed been signed by myself. Let’s run one of the same verifications on Quay:

$ codesign -dvvvv -r- /Applications/Quay.app
Executable=/Applications/Quay.app/Contents/MacOS/Quay
Identifier=net.brockerhoff.Quay
Format=bundle with Mach-O universal (i386 ppc)
CodeDirectory v=20001 size=245 flags=0x300(kill,hard) hashes=6+3 location=embedded
Signature size=1671
Authority=Rainer Brockerhoff
Signed Time=Mar 8, 2008 08/03 22:28:09
Info.plist entries=14
Sealed Resources rules=4 files=18
host => anchor apple and identifier "com.apple.translate"
# designated => identifier "net.brockerhoff.Quay" and anchor leaf = H"4cbb97c74336f7ee6aa566122a5e7688e1c725dc"

See the difference? It shows only my name as authority, and a SHA1 hashcode at the end. I could publish this hashcode on my site and it would be not too difficult for a technically knowledgeable user to verify that his copy of Quay matches that.

What does this really mean? Actually, not too much. It means that two versions of Quay that give identical names and hashcodes, which additionally match the ones published on this site, giving reasonable assurance that the author of both versions and of the site are the same; but nothing at all beyond that. However, there are ways around that – SHA1 hashcodes can be duplicated with some effort, though perhaps not yet for certificates, and this site doesn’t have a secure (https) variant. A malicious person could easily strip away my code signatures, resign everything with a new self-signed certificate using my name, make a phishing site that looks like mine, and so forth, and to a normal user everything would appear to be OK. Would someone do that for a €7 utility program? Very probably not worth the trouble, but you can’t be sure.

Then again, I could invest in a secure site, an official certificate signed by a trusted authority – perhaps even by Apple itself – and all that would just make the code signature itself more reliable; it wouldn’t necessarily correspond to what extent the code itself can be trusted to always “do the right thing”.

As Mike points out, there’s a huge gap between “signed” and “trusted”. Ultimately you have to trust somebody, or you would be too scared to buy a box at a computer store and open it, much less connect that computer to the Internet or let your children touch it. If I buy a certificate from Apple, and use that to sign my code, you have to trust someone in the certificate chain even if my code seems to “just work”… it wouldn’t necessarily mean that Apple has audited and quadruple-checked my source code. Indeed, it doesn’t even touch the question of whether Apple as a company can be trusted!

All that is still largely theoretical; Mac OS X still runs unsigned code with no problem, even though with some small restrictions regarding use of the keychain, the firewall, and internet access as I mentioned above. But Apple engineers say that in the future code signing will become mandatory. In the next post, I’ll try to discuss the implications of that – especially regarding the upcoming iPhone OS 2.0, where it will be mandatory from the outset.

Forgot to link to Macworld’s iPhone FAQ, which also tries to answer many questions about the business aspect.

To sum up, anyone can register as a developer and run applications on the included “Aspen” simulator. However, to run your app on an actual iPhone or iPod Touch, you need to invest $99. This gives you a code signing certificate that will be attached to your applications. I’ll post more about code signing later, but in essence this certificate is a code that uniquely identifies you (the developer) and is tied to Apple’s master certificate. It’s not counterfeitable by current technology. Whenever some software is executed by the iPhone, the system first checks its signature. If it’s not signed, it won’t run at all. If it is signed, the certificate will be either Apple’s own – in which case the software presumably will have full access to all system functions – or it will be a “normal” developer certificate. In that case, the application will be allowed to run with restrictions inside a sandbox – a separate folder contain the application and its data.

Once your application is tested, you must send it to Apple to be considered for inclusion into the App Store. This is a central marketplace that will be accessible directly from the iPhone or from iTunes. Apple may not approve your application; some broad guidelines were mentioned, and hopefully will be clarified before the store goes online by the end of June (after WWDC). In any event, approved or not, you’ll have no alternative way of distributing; both iTunes and the iPhone OS will only load new applications from the App Store. Presumably approved applications will have a stronger certificate which is checked at installation time, and which can be remotely revoked by Apple if your application is latter deemed to be “untrusted”.

One particular “forbidden” application type is one that downloads and runs plugins, interpretable or executable code from elsewhere. Of course that can’t be taken too literally, or it would exclude (for instance) JavaScript on a web page. Still, it would at first glance forbid Java; even so, Sun has announced intentions to produce a free Java virtual machine for the iPhone. No word yet from Apple about that, but you can presume that all those rules will have exceptions whenever Apple considers it convenient to make them.

For “indie” developers like myself, the App Store seems to be a good idea. You get 70% of the sales price. Free apps are allowed, too. In comparison, I get 85 to 90% of the sales price in my current arrangements for Mac software – but I spend some of that difference on bandwidth, site maintenance, and customer support. I personally spend almost nothing on marketing and advertising, but some developers do spend more. The App Store idea is that Apple takes care of all the business details for you, and it’s not an unreasonable deal.

Many people are discussing the implications of consenting to Apple’s rules in order to being allowed to publish iPhone applications. Personally, I’m not too bothered. Like any finite game, you can consent to play or not, but once you do, you have to play within the rules.

Speaking of rules, some of them are none too clear yet. It seems that publishing a free version that is unlocked by a serial number, as we’re used to, is not allowed as such; I suppose you’ll have to do a free version and a paid update. How would I send my app to another user for beta-testing? Unless that user also has the SDK and a certificate, and if I’m willing to send over my source, that seems to be impossible. For now, I suppose you can compile a simulator version and send the object code to some other SDK user; good for checking out the user interface maybe, but not a real-world test.

Support is also tricky. Will Apple do support for all the apps? I doubt it. If someone emails you with a support request, how can you, since there are no serial numbers, check that the request comes from a legitimate user? And you can’t send out a special version for testing, either. Hopefully Apple will explain this soon.

Next: code signing.

Update: forgot to explain the $99. This appears to be an annual fee, much like the $500 I pay now every year for the “ADC Select” program. Does this mean that the developer certificates expire after a year? Well possible, but will that also affect applications already on the App Store? More questions…

Well, there’s little actual information to add to Gruber‘s and TidBITS usual comprehensive write-up, but maybe I can explain some parts in more detail, or provide an opinion on how this will impact developers in the future.

Regarding SDK details as such, most of it is under NDA, which I signed yesterday; but some details have been published already. It’s common knowledge that it includes Xcode 3.1, which will very soon be available as an independent (and free) download, and of course included on installation DVDs with any Mac as soon as it gets out of beta. It also includes Interface Builder 3.1 but not the special objects to directly build iPhone windows; this will no doubt follow in the next release.

I was looking forward to getting my hands on the iPhone UI – to date, I haven’t seen one in the flesh (same for the iPod Touch). But fortunately the simulator includes Safari, so at least I can check how this site looks; some pages come in too wide, and I’ll tweak them as soon as I find time. Still, the UI for the few included apps looks great and it’s useful to get a feel for all the animations. Speaking of the simulator, it’s not a hardware-level simulator; it seems to run x86 code, no doubt an easy way to make it even harder for hackers to glean information about the actual iPhone OS implementation.

I suppose this would also explain why the current SDK is Intel-only; the PowerPC version might not have made it in time, or may be too slow, or whatever. Still, it’s a bad precedent. Intel Macs seem to be about 50% of the installed base by now, a year before I thought that would happen, but it’s still too soon for people to stop doing Universal applications. For what it’s worth, I had to install everything on the only Intel Mac I have – a Core Solo mini with 512MB RAM – but it seems to run quite well despite the cramped hardware.

This “iPhone OS” name now seems to be the new “OS X” and it’s… clumsy. Not a good way to indicate a new platform that already includes the Touch and before the end of this year (or so I think) will get new members. Indeed, there are hints that both “mobile” and “touch” were at one time or another the platform name du jour, and hopefully this will prove to be just a beta name.

Certainly nobody in the Mac developer community was surprised that the SDK is based on Xcode, Cocoa and Objective-C; doing otherwise would mean a complete rewrite of the current 1.1.x firmware, and Apple hasn’t enough manpower to pull such a thing off. Besides, it’s certainly a good way to get Macs into the hands of new developers. Still, no doubt some people are miffed or surprised that the SDK doesn’t run in Visual Studio, or that there’s no Java or .NET support; I can imagine them complaining to Microsoft about this (hey, Microsoft owns a good chunk of Apple, or so they probably believe).

Next: the business part. That’s where it gets interesting.

Steve Jobs just said (I guess I should say, Real Steve Jobs, hehe) on his blog:

…We want native third party applications on the iPhone, and we plan to have an SDK in developers’ hands in February.

…Nokia, for example, is not allowing any applications to be loaded onto some of their newest phones unless they have a digital signature that can be traced back to a known developer. …we believe it is a step in the right direction.

This seems to indicate that the application installer – which will in all probability be iTunes – will check if the application is properly signed. Whether they’ll allow developer-signed apps is anybody’s guess, but I wouldn’t rely on it. (Signed apps is one of the 300 Leopard features, by the way. I’ll comment on the Leopard day announcement in a few days.)

I wrote two weeks ago:
Rainer Brockerhoff wrote:

Conclusions:

– the current generation of iPhone/iPod touch will remain closed forever, just like the first generations of iPods; (I was wrong there, and a good thing too!)

– an SDK is likely to come out only after everything (especially the hardware) has stabilized;

So the February OS X version will be the first one with stable, public APIs… meaning current apps, written to reverse-engineered specs, will probably have to be seriously rewritten.

Rainer Brockerhoff wrote:

– Apple is unlikely to invest efforts into implementing TrustZone in the current generation, unless Moorestown (or whatever else they might adopt in the future) has a similar security feature – and maybe not even then

Now I wonder how they’ll handle such a hypothetical future hardware migration… probably fat binaries, with the “other” executables being stripped out by iTunes when installing an app; this would be the most flexible without upping memory footprint on the phone side.

Update: Seems that Intel and ARM are collaborating on new TrustZone implementations… might that foreshadow TrustZone on Moorestown…?

Now, some people say this proves that Apple is listening to complaints and that they’re changing their original plans; on the contrary, I think this had been the plan all the time, but the Leopard delay also delayed the SDK. Regarding the timing of this announcement, this might be a trial balloon to see if they can minimize the inevitable profit-taking after next week’s earnings announcement. Hopefully that will happen.