Solipsism Gradient

WWDC 2020 opens next June 22nd and all indications are that the highest-impact announcement will be the Mac’s migration from Intel to the ARM architecture.

While CPU architecture migrations are infrequent — they happen every decade or so — Apple has a good track record of pulling them off successfully.

The first major migration was the move from Motorola 68K to PowerPC chips around 1994, followed by moving from the Classic Mac System 9 to Mac OS X around 2000. Relevant here was that for some time Mac OS X ran older applications in the “Classic Environment”: a compatibility sandbox that emulated the APIs of System 9 and the instruction set of the 68K.

This worked reasonably well as PowerPC CPUs were several times faster than the old 68K ones. It also introduced the concept of “fat binaries“; the same application file contained code for both old and new environments.

A better historical precedent is the move from PowerPC to Intel processors in 2006. This was more traumatic for developers, as PowerPCs were “big-endian” and Intel CPUs were “little-endian”. This meant that, except for strings, values stored in memory, files or transmitted over networks had a different byte sequence ordering. To have the same program source code work on both systems you could no longer assume it would just work, but had to bracket your instructions with macros or function calls that would do nothing on one platform but swap bytes around on the other.

If you’re not an oldtimer like myself you probably never had to think about this — every Mac, iPhone, iPad, Apple Watch or Apple TV use little-endian values, and I even had to dig into documentation to make sure of it. ARM CPUs can be run in big-endian mode by setting a special bit at boot time but this is not the default, and no Apple device uses that mode.

Now, this meant that in 2006 developers could not just migrate their apps to Intel by recompiling; we had to look through every line to either check that it was endian-neutral, and if it wasn’t, those special macros had to be used. For people who had very CPU-specifically optimized code — perhaps even in (shudder) assembly language — separate code sections were necessary.

Having done all this, you recompiled your app twice; once for PowerPC, once for Intel; and the magic of fat binaries allowed you to ship it all in one app. Later on, some apps even needed 3 or 4 different code sections, depending not only on endianness but also on whether they would run on a 32- or 64-bit CPU!

Another — today mostly forgotten — aspect was that Apple prepared for the Intel migration by gradually modernizing and building their developer toolchain in-house. LLVM, Clang, LLDB etc. allowed Apple to ensure that, for whatever CPU they wanted to support, compilers were ready beforehand and could be optimized continuously later on, without depending on outsiders.

Still, in 2006 Apple had to ship special hardware, “Developer Transition Kits”, to select developers for testing. For software that couldn’t be converted to the new architecture, Apple introduced a limited compatibility box: Rosetta. If I recall correctly, it did on-the-fly translation of PowerPC code into Intel code, which was then cached. Because of its limitations it didn’t work for many larger applications and was soon phased out.

Moving in parallel to the PowerPC to Intel migration was a slower-motion shift in operating system APIs. Most notably, this involves Carbon and Cocoa.

Carbon was a C-based API introduced in 2000 to ease migration from Classic System 9 to Mac OS X. Cocoa, introduced around the same time, was an Objective-C based API for modern object-oriented programming, itself an evolution of NeXT’s OpenStep system. Underneath both APIs, in the now well-known layer model, was Core Foundation, which could be used from both types of apps; and some apps (like my own) could mix calls to both APIs with some care.

Not too long after the Intel migration, Apple announced that 64-bit was the future, and that Carbon would not be migrated to that environment. This process was stretched over several years and involved redefining what APIs were really considered “Carbon”; some, like the File Manager, were “de-carbonized” and lived on until macOS 10.5 (Catalina) came out.

Cocoa, on the other hand, continues to be used everywhere in macOS. The Finder, the Dock, Xcode, and Safari are all Cocoa apps. Even when Swift came out a few years ago most of it was built on top of Cocoa and Objective-C objects; the notable exception is the Swift toolchain itself.

So, after all this, here we’re looking at Yet Another Hardware Migration for Macs. Let’s look at the implications.

Economically, it makes sense for Apple, as many others have already commented. They’ll no longer be bound to a foreign evolution roadmap on which they have little influence. They have extensive experience in producing high-performance, low-power CPUs for their mobile devices, and the latest versions already outperform Intel in some situations.

Technically, it’s a huge win. Switching to ARM64 — and not just the standard ARMv8.x architecture licensed from ARM, but with their own, extensive modifications — will allow them to have unified GPUs, Neural Engines, memory controllers and so forth on all their line, with more uniform device drivers and low-level programming.

For 99% of developers, I think nothing will change. The new chips are little-endian also, so a simple recompile will have Xcode produce a fat binary for the new Macs which should run outright. Of course, if you have assembly language sections in your program and/or write kernel extensions/device drivers, time to learn a new architecture…

Snags will come for people who dislike, or can’t use, Xcode. Some have to use Intel’s compilers, for instance; I know too little about such cases to have an informed opinion, sorry.

Some pundits seem to expect a sudden concurrent change in macOS; something like Objective-C and/or Cocoa being obsoleted in favor of Swift and SwiftUI. Or even the Mac going away entirely, some sort of huge desktop iPad taking its place. In my view this won’t happen. For one, what would developers or even most Apple engineers use for development?

A big question is: will Apple be able to provide an Intel compatibility box on the ARM Macs? Certainly Boot Camp will not be available. Running a virtualizer like VMware Fusion or Parallels seems almost as difficult, unless the new CPUs have some sort of hardware assist to decode x86-64 instructions. This may not be as outlandish as it sounds; current Intel/AMD processors already break x86 CISC instructions into RISC micro-operations which are then cached and executed by the “inner” CPU. This is a gross oversimplification but in theory nothing — except silicon space — bars Apple from breaking x86 instructions into ARM instructions.

A Rosetta-like box seems more feasible for running individual Intel applications, but who needs that? Game users? Performance will be limited. Most virtualizer app users want the complete OS running and with native speed. Linux/BSD might be available soon; perhaps Windows for ARM.

But what about Catalyst, some of you may ask? Here I can only shrug. In its present form it certainly is not an important future technology for macOS. While simple apps can be done with it — perhaps purely for the benefit of developers unfamiliar with AppKit — can you envision a Catalyst Finder? SwiftUI is still very new and primitive, and will continue to be layered on top of AppKit/UIKit for some time. They may merge in the future, or be renamed gradually like Carbon was, but that’s a long time out.

Finally, hardware. I don’t think the existing A13 SoCs would be applicable to any Mac, though. Some version of the Mac mini would be the obvious candidate to be the first to get the all-new CPU. It would then percolate up through the laptop line and the iMac. In these cases, reduced power usage would be a bonus — even for the iMac, it would mean a smaller power supply, less heat and a thinner enclosure.

The Mac Pro should be the last Intel redoubt. Multiple CPUs, OEM graphic cards, generic PCIe cards — Apple will have to address a huge range of problems there and this will take years.

Enough handwaving for now; the usual disclaimers apply and I’m really looking forward to the keynotes next week.

Update:
— corrected date for the 68K->PowerPC migration. Thanks to Chris Adamson for catching the error.
— fixed some awkward language about virtualization. Thanks to Maurício Sadicoff.

A little over a month has passed since Swift came out, and I’ve just pushed a substantial update of my SwiftChecker app to GitHub. (Update: while writing this, I discovered a bug and pushed an updated update!)

While I still have to go back now and then to delete semicolons and to correct “String s” to “let s: String” (!@#$% muscle memory!) I finally can write out dozens of lines of Swift code without going back to the documentation or, worse, spend hours on the Internet checking why the compiler is balking. I suppose that means I’m assimilating the new syntax. It’s been over a decade since I had to learn a new language, so this isn’t too bad. 🙂

Looking at my new code, I’m particularly impressed by the conciseness of the language — the interaction with Cocoa APIs is still quite verbose, of course, but once I got used to type inference, the various map/filter/reduce/join functions, and of course generics, type extensions and operator declarations, I saw new ways of refactoring my code to be both more concise and more understandable.

In my almost 14 years of writing ObjC code I experimented with adding categories to existing classes but for the most part it always felt fragile and, after the initial experimental period, I went back writing subclasses or container classes. Using C macros to simplify stuff was tricky and error-prone. In Swift, by using generic types, it’s possible to write extensions and operators such that they don’t conflict with the existing implementation and this feels much more safe and natural. Although, of course, you still can unintentionally conflict with future additions to the Swift Library, namespaces should mostly take care of that.

Generics and type safety are a great help once I got used to them, which admittedly took some futzing around — the compiler is still very sensitive and the error messages are often cryptic or downright wrong. The type constraints and matching used for more complicated generics, admittedly, can become very complex — especially to someone not current on this new-fangled “type theory”. Still I was able to understand and build some simple generic extensions and functions that could be safely factored out into a few lines and greatly simplify other parts of my code.

Regarding optionals, I quickly got used to them; the old “returning nil” dance is now much less error-prone and at every step it’s clear which type is being handled and when it can be nil or not. The code is more readable in that regard, even if some nested “if let” statements have to remain when handling Cocoa return values. And ARC is practically transparent in Swift.

Speaking of values and ARC, the most troublesome parts are still those APIs — like the Security framework APIs I use in SwiftChecker — which haven’t yet been properly annotated. Certainly this will take many months, but no doubt pending updates to Clang will also take advantage of that, so that we’ll finally get rid of those pesky bridging casts.

One other positive side-effect is that the new whitespace rules, while mostly aligned with my own preferences, are finally making me insert spaces after commas, colons and around infix operators, and take other measures for increased readability.

All in all I’m very optimistic about Swift’s future. Can’t wait for the next Xcode beta to come out; the rumor mill says tomorrow, so let’s hope!

This appeared on xkcd a a few weeks ago: (click to embiggen)

many other developers will sympathize.

So, I’ve been developing a system to pass you, gentle app user, arbitrary applications. Since, as I said before, a group of Mac utilities is in the works – with the first four even having icons and all – of course I thought to “save time in the long run”.

It’s been more than 20 minutes though, for which I apologize. Things have been unusually complex for me this year, not to mention a couple of recent health scares (all solved, I hasten to mention).

Returning to the condiments apps. My intention is, of course, to write an ever-expanding suite of small utilities, though Apple still hasn’t published details on how to pass info from between apps in such a suite if they are on the Mac App Store. (And there’s the upcoming app sandboxing deadline to consider – an added complication.)

Anyway, all apps will work in a similar manner: file(s) are dropped onto the app’s icon, or selected from the standard Open Panel. Then something will be done to those files – information summarized, files counted, permissions checked and optionally changed, whatnot; all expected functions should be reasonably obvious from the UI.

So we have a host of common functions, namely, implementing the App Store receipt checking, sandboxing considerations, receiving dropped and opened files, scanning over them (and perhaps over their contents, if they’re folders), showing the About Box and some help, and doing all that in a consistent manner.

I’m happy to report that everything along those lines is now working perfectly, and with the new workspace facility in Xcode 4, expanding from one to several apps will be a piece of cake. Let’s leave the culinary metaphors aside for the moment and ponder how I’ll can deliver – considering that my record regarding past deadlines has been not so good. (OK, abysmal.)

The answer is obvious: take a cruise. In recent years, everything significant I’ve released had been mostly written and polished on a cruise ship. No distractions, no phones, almost no Internet, no relatives (haha)… and I can impress my fellow passengers by saying “well, I’m making money for the next cruise here on board!”.

So, for over a year we’ve been planning a major cruise – it might be our last long cruise for the foreseeable future, even. And I’m very happy that we leave early tomorrow to return in the last days of January 2012. (Should give us time to prepare for the Mayacalypse, anyway.)

I’ll be posting from underway without saying too much about destinations, to add to the suspense. We should have occasional – though expensive – Internet onship, so email etc. should work. Our next stop should be Santiago del Chile. Stay tuned!

A couple of friendly publications have asked me to write about the very recent passing of Apple’s former CEO, Steve Jobs. I refused. While some of the stories published in the past 24 hours are moving, interesting, informative and even funny, some are also inappropriate, self-serving, offensive, vapid, or overly sentimental. (And few people agree as to which are which!)

I also have, in the past, refrained from writing about personal stuff here. There are people (both living and deceased) who I admire for certain personal qualities, but it would be unseemly for me to publish a “fanboi” list of these people, much less directly address the departed ones or their families. What I can mention about Steve Jobs is:

• I’ve been a few feet away from him twice at trade shows, but that’s it. No conversations.
• I’ve been an Apple customer, almost exclusively, since 1977, a Mac developer since 1984, and a (very modest!) Apple stockholder since the beginning of the millennium.

As often happens with such public figures, a good part of the public’s perception is shaped through anecdotes and legends which may not closely correspond to what really happened. While I’m as happy to repeat such tidbits in personal conversation as anybody else, there’s only one that I feel completely comfortable to express here: all agree that Steve Jobs really cared about building better things – hardware, software, but mostly better systems.

Which happens to agree with my personal philosophy here – if you’ve looked at my products page, the header says “finely crafted software for the Macintosh”. So, the best way to do something that Steve Jobs would agree with is to go on building better stuff.

I have been remiss in mentioning my current work here, and I decided it’s time to give at least a hint. So, here are the icons for four forthcoming Mac utilities:

The top two should come out first. They’ll all be on the Mac App Store, if all goes well. Details, prices and so forth are still in flux but should be available soon. Some of their functions are intended to replace parts of my defunct XRay application, but Quay will also be updated afterwards to work in concert with the new apps. Current customers of both applications will get free updates within the (unfortunately) narrow conditions imposed by the Mac App Store.

More details will be published as things develop (hehe). Stay tuned. I’m working hard on better stuff.

By the way, the icons are by Sergio Bergocce.

Well, Xcode 4.1 is out of beta. Please note that the “GM” build previously posted on the developer site was NOT the released version, which is now available for free on the Mac App Store. (Open the “Welcome to Xcode” window, it should say “Version 4.1 (4B110)”.)

I’d promised to several people at WWDC to investigate how Xcode 4.1’s lack of support for IB plugins would work out for RBSplitView, and now I can finally post the results here. Unfortunately the news is not good.

Compiling RBSplitView’s “Sample App” target works well under Xcode 4.1 with just a few changes to modernize build settings sand avoid new warnings. The problem comes when trying to open the .nib file. First, the release notes say you should do this in Terminal:

	defaults write com.apple.InterfaceBuilder3 "IBKnownPluginPaths.3.2.7"
		-dict-add "net.brockerhoff.RBSplitView.IBPlugin"
		"/Users/<username>/<path-to-RBSplitView.ibplugin>"

which I did.

Opening the nib file then offers to remove the dependency on the plug-in:

and going ahead lists several warnings and issues.

The way this works is interesting. All RBSplitViews and RBSplitSubviews are converted to NSCustomViews – meaning that they’re stored as plain NSViews in the nib file, -initWithFrame: is called on unarchiving (instead of -initWithCoder:); the view’s class is reset to RBSplitView or RBSplitSubview, as the case may be, and all custom attributes are then set through key-value coding.

I suppose this would work quite well for less complex views, but it didn’t work out of the box for RBSplitView. First of all, this KVC stuff was quite new-fangled when I wrote it, and I didn’t see any need to use it until I had to update the RBSplitView.ibplugin for Xcode 3.1. At the time, I simply wrote some KVC methods for the ibplugin additions, mainly to simplify setting all those attributes from inside Interface Builder… it all worked fine.

Converting the nib file generates a lot of exceptions as the KVC methods just aren’t there in the framework code. I tried a quick fix, copying and pasting them from the plugin code, but that didn’t work out too well: RBSplitView doesn’t like being reincarnated from the nib file piecewise like that, and it seems that the attributes get set too late or in the wrong order.

I suppose some fiddling with the copied methods will fix that, but it’ll be at best a stop-gap measure. The converted nib file no longer adjusts the RBSplitSubviews properly and it’d be too easy to make a big mess of it, should you try to change anything in there. Even so, I’ll try to make some time available to get this working.

Apple says Xcode 3.2.6 is “unsupported” under Lion. If you already have it installed in a separate folder when you upgrade it will mostly continue to work, but I found that running Interface Builder 3 crashes when you have the RBSplitView plugin open. Probably the best bet, for now, is running Snow Leopard Server in a virtual machine and installing Xcode 3.2.6 in there.

Brent Simmons, author of NetNewsWire, writes:

On Macs we have a long-standing culture of apps working together.

…

On the iPad (and iPhone) we can sort-of do the same thing. We don’t have AppleScript or Apple events, but we do have the URL scheme thing for inter-application communication. It’s technically possible to do some of these same things.

But we don’t have an easy way to get back to the calling app.

…

What if the calling app added, as a parameter to the URL, a URL to call when the task is completed?

This way the helper app (NetNewsWire in this case) would know, once the task is complete, how to get the user back to their place in the calling app (Twitterrific in this case).

I was thinking about the same issue, coincidentally, and one idea which occurred to me is to use the equivalent of the http referrer URL (often misspelled as “referer” for historical reasons).

For  a standard http request the referrer is the URL of the document containing the clicked link, so you can get back to that document by clicking on your browser’s “back” button. Now, this isn’t really contained in the URL itself – rather, the browser appends this as one of the fields in the http request headers – but the analogy is interesting.

For this to work locally between applications, the referrer URL would need to be set into the URL targeted at the called application. And indeed, there is a -[NSURL setResourceValue:forKey:error:] method that, in theory could add any key/value pair to the URL. (Currently only the “scheme” property seems to be available, though.)

Update: Mike Abdullah points out in the comments that, in practice, this method applies properties to the file a file URL points at, not to the URL itself. Drat.

On the receiving side, the “referrer” value would have to be pulled out from the NSAppleEventDescriptor that the application gets with the ‘GURL’ event.

All this needs filing enhancement requests and Apple would have to do the required twiddling inside their code base – or rather, bases, as this would be useful to have both on the iPhone/iPad and the Mac side. It would take a long time, if it’s done at all.

The alternative would be to developers to, informally, establish a convention for incorporating this into the URL itself. So we’d have something like:
calledscheme:///the/url/parameters&?REF=callingscheme://some/url/to/return

Probably the second part would have to be suitable encoded/escaped, too. Maybe this would be a good topic for discussion at WWDC?

Just published a droplet application that fixes the spelling error in nibs produced by my previous (beta) version of the RBSplitView Interface Builder plugin. Details on the RBSplitView page.

So, to start things going again… I was looking at my RSS feeds for the first time in almost 5 months, and read Dan Wood of Karelia fame explaining about Converting Rich Text to TEXT/styl resources for an SLA on a Disk Image.

My own workflow for building a release disk image uses a small tool I’ve written for that. You can download it here. Here’s the help text it prints out if you run it without arguments:

   Add one license at a time to a (unflattened) disk image.
   Usage: AddLicense /path/to/TheUnflattened.dmg Language /path/to/TheLicense.rtf
   Languages supported: da nl ko ja fr it fi pt sv en es de nb
   You can also use long equivalents like English, French etc.
   The first language added will be the default language (usually English).

Here's an actual usage example from a build script:

   hdiutil unflatten "$SOURCE_ROOT/My.dmg"
   "$BUILT_PRODUCTS_DIR/AddLicense" "$SOURCE_ROOT/My.dmg" English "$SOURCE_ROOT/EnglishLicense.rtf"
   "$BUILT_PRODUCTS_DIR/AddLicense" "$SOURCE_ROOT/My.dmg" French "$SOURCE_ROOT/FrenchLicense.rtf"
   hdiutil flatten "$SOURCE_ROOT/My.dmg"

and I use the “flatten” and “unflatten” arguments to hdiutil to massage the disk image.

The trick (as Dan points out in his post update) is that you can use ‘RTF ‘ resources instead of TEXT/styl. I received this interesting tidbit through oral tradition; I’m not a 100% certain, but I think it was through some code that Marko Karppinen showed me a few years ago. Pass it on.

If there’s interest, I may clean up and publish the source sometime, though it uses all sort of gronky old APIs (Resource Manager etc.).

Update: the link above now downloads the complete Xcode project. I also included it on my source code page.