The subject of private APIs has been raised again with the recent availability of the iPad. (Unless noted, the following discussion applies to both the Mac and the iPad/iPhone/iPod touch platforms.)
Brief recap: API means “application programming interface”; in other words, it’s the way that an application programmer calls some system routine. APIs are grouped in libraries or frameworks – the distinction between these is largely unimportant here, and we’ll use the term “framework” from now on. Frameworks can be (and usually are) layered, either by explicitly being contained within each other, or by calling APIs from other frameworks.
If you browse into the /System/Library folder on a Mac, you’ll find both “Framework” and “Private Framework” folders. The “Framework” folder contains nearly a hundred subfolders with the .framework extension. Each one will have either a binary file containing the actual executables or a folder of contained frameworks, and a “Headers” folder with one or more .h files. These header files, in turn, list what is known as the “public APIs” for that framework. Browsing will reveal such well-known frameworks as Carbon and Cocoa, as well as more obscure ones like GLUT, LDAP and so forth.
Now look into “Private Frameworks”; there, too, is a bunch of .framework folders, but none of those have a “Headers” folder. This means that there is no public API for these frameworks at all; normal application programs aren’t supposed to use these frameworks. The names are often intriguing: AppleFSCompression, BezelServices, CoreMedia, DiskManagement, SystemMigration, iTunesAccess and so forth. Many of these are actually shared between several applications; for instance, the ProKit framework is shared by Apple’s “pro” applications (Final Cut Pro, and so forth) and this makes these applications smaller and easier to fix.
But not only the private frameworks have private API. The public frameworks too have APIs that aren’t included in the header files; or, rather, the header files for those APIs are available only internally at Apple. In other words, private APIs (some of which are also known as SPIs: “system programming interfaces”) are everywhere, but common mortals should not use them. Indeed, on Apple’s mailing lists, discussing private APIs and SPIs is a definite no-no. Like APIs, the formats and contents of some system files are undocumented, and fall under the same restrictions.
At this point, it’s useful to recall the basic motivations for having an API (be it private or public) at all. First, an API prevents the programmer from having to reinvent the wheel by doing several, often complex, lower-level steps to get the same results; second, isolating these steps into a single place means fixing a bug there will fix the bug for all programs or frameworks that call the API. In fact, having such APIs calling each other is the magic that enables Apple to keep older software working while changing parts of the infrastructure, as happens with every major OS release.
Still, it’s fun to find out about private APIs and test them, but inadvisable to use them for actual shipping applications, as they may vanish as suddenly as they appear, or change radically from one system version to the next.
The official process of API “promotion” is explained by Bertrand Serlet in this snippet from a recent WWDC presentation: an API begins as private, is reviewed internally, then is exposed to developers in some beta version of the system, and after some time is “committed” as public API. And (Serlet doesn’t say), several versions later an API may be “deprecated” – meaning, it will still work but isn’t recommended for new applications – and, some more versions later, it may be removed entirely from the system. The latter process happens to private APIs as well, and no doubt even more frequently, since Apple can safely assume that no shipping application other than their own uses these APIs directly. And this of course is why Apple’s own applications are always updated along with every system revision.
Now, some people see an insidious conspiracy here: Apple’s programmers have access to all these powerful resources that are denied to third-party programmers. In the past, Microsoft was ordered by a court to disclose APIs used by their Internet Explorer browser; these APIs were supposed to give them an unfair advantage over browsers from other companies. See, for instance, Alex Payne:
Apple should remove the concept of private APIs from its developer offerings. Give developers the same tools that Apple?s own programmers get to use. If an API is still too unsafe or experimental for developers to make use of, don?t ship it, or gate it to development versions. Don?t restrict third parties from taking full advantage of the device and its software.
I find this comment puzzling. There’s no way to “remove the concept” of private APIs from the system, nor are they “shipped” – in fact, they’re private precisely because they’re not shipped (though I’d say documented, instead of shipped). But they have to exist, otherwise there’d be no way to make a layered system.
By the way, even the first Mac 128 in 1984 had private APIs! There were no real layers to that first System 1.0, but some functions in the ROM were not supposed to be called by the application programmer. Needless to say, some enterprising programmer soon found out their machine addresses and thus could call them directly; also, needless to say, these hacks broke immediately when a new model came out.
Marco Arment says:
iBooks? use of tons of private APIs is frustrating on a few levels, the biggest that it makes all third-party reading-related apps second-class citizens.
I won?t be able to offer some features that iBooks has (such as a true brightness control), but my customers will expect them, making my app inferior to Apple?s in key areas.
This app?s undocumented API use wouldn?t pass the App Store submission process, yet developers need to compete with it for App Store attention. One of the great potential failures of an app-review system is inconsistent or unfair enforcement of the rules.
While it is of course possible for Apple to abuse the system in this way, it’s not reasonable to expect Apple’s applications not to use any new APIs while they are being tested; after all, this is the best (and perhaps only) way for them to be tested!
Marco also points at proof that iBooks uses private frameworks (and the Kindle app, of course, doesn’t). Some of those, no doubt, are justified code sharing between Apple apps – WebKit, iTunes stuff, etc. Others, like WebKit, Accessibility and JavaScriptCore may be just still in that “testing” stage; by keeping them private, Apple is free to change all those frameworks (along with all of their apps that use them) at every minor system revision. I don’t doubt that WebKit, for instance, will be moved to the public framework stage in a not-so-distant revision.
In conclusion, I don’t think that worrying overmuch over private APIs will be productive. Better to observe Apple’s applications closely, and then file (early, and often) detailed requests for specific APIs that it would be productive to have available – Apple does listen to such requests, although they may end up doing things a little differently.
If I want to tinker, I’ll get an old junk PC somewhere. But, nowadays, I find writing applications is more satisfying.
