Solipsism Gradient

Rainer Brockerhoff’s blog

Browsing Posts published in April, 2010

Re: iPad time

No comments

richardl wrote:

Rainer, it is interesting to note that Adobe built their ActionScript/Flash compiler for iPhone on top of LLVM.

http://cs.illinois.edu/news/2009/Oct8-2

http://www.adobe.com/devnet/logged_in/abansod_iphone.html

Thanks, I wasn’t aware of that. However, I don’t think that this invalidates my original argument, which is more based on Clang than on LLVM. Christopher Lloyd also remarked on this:

…Yet Adobe uses llvm for the compiled Flash, so they don’t like Adobe’s parser? silly. http://bit.ly/MQtRZ

Also thanks to Daveed Vandevoorde, who remarked that the LLVM bytecode isn’t as platform-independent as I thought – things like sizes of variables and ABIs (Application Binary Interfaces) stand in the way. Still, it’s not impossible to see Apple defining a generic target platform, in terms of sizes of pointers, integers and so forth, and a specific endianness and runtime ABI, and then staying within the bytecode for that.

Re: iPad time

No comments

Posted by richardl:

Rainer Brockerhoff wrote:

Via John Gruber, I just saw an interesting post by Steve Cheney:

It’s pretty evident that Apple isn’t wed to individual suppliers. Not only are they back to creating their own chips, but they are also one of the only ‘compute’ companies to have used each of the top 3 processor architectures over time – ARM, x86, and Power PC.

…This week Apple confined developers to a specific set of tools (XCode [sic]).

…By telling developers to move to XCode tools, Apple is setting the stage to potentially switch architectures.

…In 2003, Apple advised developers to switch to XCode tools. …2 years later Apple moved to Intel across its entire Mac line.

…perhaps the A4 is NOT an ARM architecture. In fact, it’s highly possible that the A4 is a dual core Power Architecture…

The last sentence is of course false, as Gruber says; the A4 does run ARM code.

While I don’t think that forcing developers to switch to Xcode (which is the correct spelling_ means that Apple will soon be switching architectures on their iPad/iPhone line, or on their desktop/laptop line for that matter, Xcode does offer developers a future-proof environment that hasn’t been commented on by other observers: the Clang/LLV project. Briefly, Clang is a compiler frontend for C-based languages, backed by LLVM – which stands for “Low Level Virtual Machine”. Both projects are heavily backed and staffed by Apple.

Clang has been increasingly supported by Xcode as a substitute for the gcc compiler toolchain. The details are quite esoteric, but one interesting capability is that C-based languages – C, Objective-C and C++ – are compiled to the LLVM bytecode, which is then translated into native machine language by a back-end. The last phase could even happen in a just-in-time fashion, allowing apps to be distributed in LLVM code (therefore running on all current and future Apple machines). Some groups are even working on chips that execute LLVM bytecode directly.

In other words, it’s no coincidence that Apple is now instructing developers to switch to Clang-supported languages and their Clang-wrapping IDE (Xcode). There may not be an architecture switch coming soon, but Apple will have much more freedom in doing their own CPUs for iPad/iPhone, and more ammunition in negotations with Intel and other top-end chip companies.

Rainer, it is interesting to note that Adobe built their ActionScript/Flash compiler for iPhone on top of LLVM.

http://cs.illinois.edu/news/2009/Oct8-2

http://www.adobe.com/devnet/logged_in/abansod_iphone.html

Re: iPad time

1 comment

Via John Gruber, I just saw an interesting post by Steve Cheney:

It’s pretty evident that Apple isn’t wed to individual suppliers. Not only are they back to creating their own chips, but they are also one of the only ‘compute’ companies to have used each of the top 3 processor architectures over time – ARM, x86, and Power PC.

…This week Apple confined developers to a specific set of tools (XCode [sic]).

…By telling developers to move to XCode tools, Apple is setting the stage to potentially switch architectures.

…In 2003, Apple advised developers to switch to XCode tools. …2 years later Apple moved to Intel across its entire Mac line.

…perhaps the A4 is NOT an ARM architecture. In fact, it’s highly possible that the A4 is a dual core Power Architecture…

The last sentence is of course false, as Gruber says; the A4 does run ARM code.

While I don’t think that forcing developers to switch to Xcode (which is the correct spelling) means that Apple will soon be switching architectures on their iPad/iPhone line, or on their desktop/laptop line for that matter, Xcode does offer developers a future-proof environment that hasn’t been commented on by other observers: the Clang/LLVM project. Briefly, Clang is a compiler frontend for C-based languages, backed by LLVM – which stands for “Low Level Virtual Machine”. Both projects are heavily backed and staffed by Apple.

Clang has been increasingly supported by Xcode as a substitute for the gcc compiler toolchain. The details are quite esoteric, but one interesting capability is that C-based languages – C, Objective-C and C++ – are compiled to the LLVM bytecode, which is then translated into native machine language by a back-end. The last phase could even happen in a just-in-time fashion, allowing apps to be distributed in LLVM code (therefore running on all current and future Apple machines). Some groups are even working on chips that execute LLVM bytecode directly.

In other words, it’s no coincidence that Apple is now instructing developers to switch to Clang-supported languages and their Clang-wrapping IDE (Xcode). There may not be an architecture switch coming soon, but Apple will have much more freedom in doing their own CPUs for iPad/iPhone, and more ammunition in negotations with Intel and other top-end chip companies.

Just a thought to complement my previous post on the subject. A few days ago, Apple announced iPhone OS 4.0 with “over 1500 new APIs” (not a permanent link, sorry).

It’s a safe bet that most of these APIs have been there in the previous iPhone OS release(s), but in their private form. Some of them were simply published – that is, included in the public headers. Beyond that, some had bugs fixed, or their functionality changed, or even their names and parameters changed; and, no doubt, many other APIs stayed private or were, even, dropped. And the common developer can never be sure which is which.

Only Apple’s in-house developers have all this information and for them it’s a mad scramble to update apps with every OS release. This is also why it’s, generally speaking, not advisable to use any apps that come with Mac OS (Preview, Mail, Address Book and so forth) with any previous or subsequent releases; they’ll crash in interesting ways. And this is why Apple lists new features and bug fixes inside those apps when a new Mac OS release comes out.

Digital Photographer Brasil is coming out in a few days: a Brazilian edition of the UK’s Digital Photographer magazine. Congratulations to the editor, Mario Amaya and all our friends at Editora Digerati.

Special thanks to Mario for featuring one of my pictures on page 14 of the magazine; you can see it on his post, it’s the first top left picture after the cover.

The subject of private APIs has been raised again with the recent availability of the iPad. (Unless noted, the following discussion applies to both the Mac and the iPad/iPhone/iPod touch platforms.)

Brief recap: API means “application programming interface”; in other words, it’s the way that an application programmer calls some system routine. APIs are grouped in libraries or frameworks – the distinction between these is largely unimportant here, and we’ll use the term “framework” from now on. Frameworks can be (and usually are) layered, either by explicitly being contained within each other, or by calling APIs from other frameworks.

If you browse into the /System/Library folder on a Mac, you’ll find both “Framework” and “Private Framework” folders. The “Framework” folder contains nearly a hundred subfolders with the .framework extension. Each one will have either a binary file containing the actual executables or a folder of contained frameworks, and a “Headers” folder with one or more .h files. These header files, in turn, list what is known as the “public APIs” for that framework. Browsing will reveal such well-known frameworks as Carbon and Cocoa, as well as more obscure ones like GLUT, LDAP and so forth.

Now look into “Private Frameworks”; there, too, is a bunch of .framework folders, but none of those have a “Headers” folder. This means that there is no public API for these frameworks at all; normal application programs aren’t supposed to use these frameworks. The names are often intriguing: AppleFSCompression, BezelServices, CoreMedia, DiskManagement, SystemMigration, iTunesAccess and so forth. Many of these are actually shared between several applications; for instance, the ProKit framework is shared by Apple’s “pro” applications (Final Cut Pro, and so forth) and this makes these applications smaller and easier to fix.

But not only the private frameworks have private API. The public frameworks too have APIs that aren’t included in the header files; or, rather, the header files for those APIs are available only internally at Apple. In other words, private APIs (some of which are also known as SPIs: “system programming interfaces”) are everywhere, but common mortals should not use them. Indeed, on Apple’s mailing lists, discussing private APIs and SPIs is a definite no-no. Like APIs, the formats and contents of some system files are undocumented, and fall under the same restrictions.

At this point, it’s useful to recall the basic motivations for having an API (be it private or public) at all. First, an API prevents the programmer from having to reinvent the wheel by doing several, often complex, lower-level steps to get the same results; second, isolating these steps into a single place means fixing a bug there will fix the bug for all programs or frameworks that call the API. In fact, having such APIs calling each other is the magic that enables Apple to keep older software working while changing parts of the infrastructure, as happens with every major OS release.

Still, it’s fun to find out about private APIs and test them, but inadvisable to use them for actual shipping applications, as they may vanish as suddenly as they appear, or change radically from one system version to the next.

The official process of API “promotion” is explained by Bertrand Serlet in this snippet from a recent WWDC presentation: an API begins as private, is reviewed internally, then is exposed to developers in some beta version of the system, and after some time is “committed” as public API. And (Serlet doesn’t say), several versions later an API may be “deprecated” – meaning, it will still work but isn’t recommended for new applications – and, some more versions later, it may be removed entirely from the system. The latter process happens to private APIs as well, and no doubt even more frequently, since Apple can safely assume that no shipping application other than their own uses these APIs directly. And this of course is why Apple’s own applications are always updated along with every system revision.

Now, some people see an insidious conspiracy here: Apple’s programmers have access to all these powerful resources that are denied to third-party programmers. In the past, Microsoft was ordered by a court to disclose APIs used by their Internet Explorer browser; these APIs were supposed to give them an unfair advantage over browsers from other companies. See, for instance, Alex Payne:

Apple should remove the concept of private APIs from its developer offerings. Give developers the same tools that Apple’s own programmers get to use. If an API is still too unsafe or experimental for developers to make use of, don’t ship it, or gate it to development versions. Don’t restrict third parties from taking full advantage of the device and its software.

I find this comment puzzling. There’s no way to “remove the concept” of private APIs from the system, nor are they “shipped” – in fact, they’re private precisely because they’re not shipped (though I’d say documented, instead of shipped). But they have to exist, otherwise there’d be no way to make a layered system.

By the way, even the first Mac 128 in 1984 had private APIs! There were no real layers to that first System 1.0, but some functions in the ROM were not supposed to be called by the application programmer. Needless to say, some enterprising programmer soon found out their machine addresses and thus could call them directly; also, needless to say, these hacks broke immediately when a new model came out.

Marco Arment says:

iBooks’ use of tons of private APIs is frustrating on a few levels, the biggest that it makes all third-party reading-related apps second-class citizens.

I won’t be able to offer some features that iBooks has (such as a true brightness control), but my customers will expect them, making my app inferior to Apple’s in key areas.

This app’s undocumented API use wouldn’t pass the App Store submission process, yet developers need to compete with it for App Store attention. One of the great potential failures of an app-review system is inconsistent or unfair enforcement of the rules.

While it is of course possible for Apple to abuse the system in this way, it’s not reasonable to expect Apple’s applications not to use any new APIs while they are being tested; after all, this is the best (and perhaps only) way for them to be tested!

Marco also points at proof that iBooks uses private frameworks (and the Kindle app, of course, doesn’t). Some of those, no doubt, are justified code sharing between Apple apps – WebKit, iTunes stuff, etc. Others, like WebKit, Accessibility and JavaScriptCore may be just still in that “testing” stage; by keeping them private, Apple is free to change all those frameworks (along with all of their apps that use them) at every minor system revision. I don’t doubt that WebKit, for instance, will be moved to the public framework stage in a not-so-distant revision.

In conclusion, I don’t think that worrying overmuch over private APIs will be productive. Better to observe Apple’s applications closely, and then file (early, and often) detailed requests for specific APIs that it would be productive to have available – Apple does listen to such requests, although they may end up doing things a little differently.

iPad time

No comments

No, I haven’t ordered one myself; they’ll take some months to appear hereabouts, and will be expensive. Maybe at WWDC I’ll buy one, though I may not be able to attend this year.

Of course most of the reviewers already have theirs, and the divide between the geek and non-geek users is being endlessly rehashed. David Pogue at the NYT went so far as to do a double review. One predictable vote for the geek side came from Cory Doctorow:

Why I won’t buy an iPad (and think you shouldn’t, either)

…Relying on incumbents to produce your revolutions is not a good strategy. They’re apt to take all the stuff that makes their products great and try to use technology to charge you extra for it, or prohibit it altogether.

…But with the iPad, it seems like Apple’s model customer is that same stupid stereotype of a technophobic, timid, scatterbrained mother as appears in a billion renditions of “that’s too complicated for my mom”…

…As an adult, I want to be able to choose whose stuff I buy and whom I trust to evaluate that stuff. I don’t want my universe of apps constrained to the stuff that the Cupertino Politburo decides to allow for its platform.

Now, Cory is one of my favorite writers, but I think he went a little overboard there. Once you accept that Apple’s model customer indeed is the “technophobic, timid, scatterbrained mother”, all his other arguments become much weaker, since they apply only to old-fashioned tinkerers who are concerned about absolute freedom for their “universe of apps”, or even want to just run Linux on the thing.

Both Joel Johnson and John Gruber agree with me that the iPad is essentially a consumer device, not a device for tinkerers and makers.

Now, I consider myself a serious tinkerer. I learned programming in the days when that meant plugboards and punch cards, and programming books started out with chapters of flip-flop and half-adder schematics. I’ve owned a 1961 VW of which I knew every single wire and screw, learned how to do radio and TV maintenance, designed hardware and OS for special-purpose computers, and so forth.

However, even as a tinkerer, I don’t really want to open up my computer and tinker with it just for the fun of it – or install Linux on it, or whatever – as long as it works like I want it to. Now, I bought my first Mac in 1984, and have owned at least two dozen different models since then; I’ve opened up (and in many cases, expanded or modified) nearly all of them. The exceptions are my current iMac and MacBook Air; they just work fine, and I won’t risk marring the finish. icon_wink.gif If I want to tinker, I’ll get an old junk PC somewhere. But, nowadays, I find writing applications is more satisfying.

But I digress. What I do disagree with is some of this “freedom to tinker” entitlement philosophy. Quoting Cory again:

Having gotten into business with the two industries that most believe that you shouldn’t be able to modify your hardware, load your own software on it, write software for it, override instructions given to it by the mothership (the entertainment industry and the phone companies), Apple has defined its business around these principles.

Exactly; that’s Apple’s privilege, and as an Apple stockholder I agree with their position. Cory (or anyone) should be perfectly free to modify an iPad, write their own software for it and so forth – with one caveat: they can’t force Apple to help them with that, and once they do it, it’s not an Apple device anymore for all practical purposes: no warranty, at the very least, no support, and so forth. I suppose that they want all of that, too; quite unrealistic.

Update: yes, there’s the AppStore. I dislike the current terms, at least from the developer’s side, and they’re one of the factors that keep me from developing for the iPhone/iPad right now. I do think (make that hope) that the terms will – very gradually – become less restrictive. All this “politburo” stuff is, regrettably, fueled by today’s insanely litigious society and Apple’s reaction to avoid liability at all costs.

Photos licensed by Creative Commons license. Unless otherwise noted, content © 2002-2017 by Rainer Brockerhoff. Iravan child theme by Rainer Brockerhoff, based on Arjuna-X, a WordPress Theme by SRS Solutions. jQuery UI based on Aristo.