Solipsism Gradient

An amazing bit of rumor has surfaced recently: Apple’s “Yellow Box” is supposed to be coming back from the dead, under the codename “Dharma“.

The idea seems to be that writing to a “pure Cocoa” API would enable software to be compiled, unchanged, for both Mac OS X and Windows. Or it could even be published as a universal binary containing both versions.

Indeed, the first aspect was the original idea when the Yellow Box was first published in 1997, when Apple bought NeXT (or vice-versa). Based on OpenStep, both Cocoa and the Yellow Box would have the same API and support programs running on both platforms. The Yellow Box was even included with Rhapsody (later Mac OS X server), although I think actual deployment licenses were sold separately. WebObjects (based on Objective-C) and EnterpriseObjects (ditto) used the same multiplatform philosophy.

However, at the 2000 WWDC Apple backed away from the whole thing. EO was discontinued amidst much weeping and wailing from its devotees, WO was converted to Java, and the Yellow Box was quietly dropped without much explanation.

At the time the reasons were quite unclear, but in hindsight some justifications can be seen. In particular, doing a successful multiplatform technology has turned out to be much more difficult than most people believed at the time; especially as Windows, not being under Apple’s control, would have been a difficult moving target anyway.

Personally I think that, retrospectively, dropping EO and moving WO to Java were mistaken decisions; keeping them both alive using Objective-C and Mac-only would have been prudent. However, the adoption of Cocoa turned out to be much slower than anyone would have thought, and as major companies like Microsoft, Adobe and Macromedia stayed with Carbon, any actual Yellow Box usage would have been restricted to smaller software houses or ISVs – for which the $3000 for 10 seats licensing terms would have been too expensive anyway.

Apple deftly turned away from the idea of Carbon as a short-term transition technology and instead promoted Carbon and Cocoa as equal-strength APIs whose capabilities are converging over time. Now here is an aspect which the recent rumormongers – not being developers – have not thought of. Both NeXTSTEP and OpenStep were the lowest-level API for application programs, but this is not true of Cocoa. The multiple layering of technologies and APIs on Mac OS X means that Cocoa can’t do many things that are possible in other APIs; meaning, in practice, that any reasonably complex software must use a mixture of technologies. My own XRay, for instance, even though it’s nominally a Cocoa program, also calls BSD APIs to handle users, groups, and permissions, as well as Carbon APIs to access many aspects of the file system.

Therefore, were the Yellow Box magically resurrected, programs written for a “pure Cocoa” API would theoretically run on it – but few, if any, existing Cocoa applications can afford to be so pure. The current Cocoa framework would have to be extended quite a lot, and Yellow Box prices would have to be lowered drastically – perhaps even to zero – before any but a handful of software companies would be interested. And for what? To allow interesting applications for Mac OS X also to run on Windows, reducing incentives for users to switch? Sounds extremely unlikely to me…

Yesterday I spent some time at one of the prototype Macs with a Pentium inside. There was one of them with the cover removed. The most remarkable thing was – and, now that photos have leaked, I don’t think I violate any NDA by saying this – that there was nothing remarkable about the box (a standard cheese-grater PowerMac) or about the motherboard (a standard Intel motherboard). There was nothing remarkable about it usage-wise either; unless you looked at the “About This Mac” window, or at the System Profiler Report, or at the Processor preference panel, there was no way of telling what CPU was inside. It ran some unreleased build of Tiger, and there was this huge conspicuous security cable on it for some reason .

But it walked like a Mac, it quacked like a Mac, it was a Mac to all intents and purposes. I downloaded a dozen of random software packages off the Internet, they all just worked – under the Rosetta translator, which I had to see working to really believe in. The perceptual speed was, perhaps, a little faster than my 1GHz PowerBook; quite usable. I suppose this will get faster after a year of tweaking; another word in everybody’s mouth these past days.

Ah, and a bit of news which also leaked out today: Steve Job’s machine was not a souped-up quad processor monster, just the same box I had been using, but with some extra RAM. So, Rosetta is one cool app. I talked to one of the guys on the Rosetta team and he confirmed what I saw happening at the keynote: when you launch an app the second time, it uses the cached translated binary, so it launches much faster.

I looked at the installed libraries, drivers and applications: of course they were all “fat” binaries. The entire system is fat. Oops, sorry, “universal binary” is the politically correct version now. It’s not quite double the size of the standard Tiger installation, but who cares in these days of 100+GB disk drives?

I checked out some of my own projects on the new Xcode 2.1. Nearly all standard Cocoa stuff just compiled and ran with no modifications, no matter what combination of architectures I compiled it for. You can even step-debug PowerPC binaries on this thing, they somehow made gdb Rosetta-aware, so that the translated executable is back-linked to your PowerPC source code; very cool. This is probably a bonus of the Mach-O executable format, like the universal binary format itself. The old Carbon CFM format will run under Rosetta, but not natively; CFM is the new Classic, it seems. The old Classic appears to be dead at last; I suppose getting the Classic compatibility layer running under Rosetta would be a huge pain.

I don’t have any straight Carbon projects to test. I do have one new project that twiddles bytes that flow to and from the disk at a lot of places, because it uses a legacy format (one dating from 1984, by the way). I got most of it converted in less than an hour by scanning for certain source code patterns and putting byte swap function calls around the pertinent expressions. Now, the publicly available Guidelines list dozens of exceptions, where porting takes some extra work: if you use custom resource formats or Apple Events, if you use bitfields, if you want to divide by zero, and so forth. I think the biggest headache will be for whoever has invested time in writing great gobs of code in PowerPC assembly or Altivec; fortunately I never did this myself.

One place where I later lost another hour of work was in a somewhat obscure open source module which made unwarranted assumptions about the order local variables were allocated on the stack. Now, this is something which certainly works for one-off applications, but to actually publish such a thing without calling attention to it, is somewhat foolhardy. This is where many of the conversion failures will come from, I believe; sloppy coding and unwarranted assumptions.

Same place as usual. I really hope this will be stable for some time, at least until after WWDC…

An interesting thing happened while doing this version. (The following discussion will make no sense unless you’re familiar with RBSplitView.)

I have an adjustSubviews method call which does all the heavy lifting, computing the new positions and sizes for all subviews. This worked fine until I saw the need to (sometimes) have a specific subview stay the same size whenever the window was resized. In other words, I needed an adjustSubviewsExcepting: method.

At first I saw no easy way to do this, but then I hit upon a simple trick; I would temporarily set that subview’s minimum and maximum dimensions to the current dimension, call adjustSubviews, then put the original limits back. This was way back in version 1.0.2.

All seemed to work fine until I received a bug report that this could cause the subview to be collapsed in certain circumstances. So this time I tried several ways to prevent this from happening; the code inside adjustSubviewsExcepting: grew ever more large and cumbersome, and nothing appeared to work in all cases.

Then the solution appeared in my sleep (as often happens). The thing was to make adjustSubviewsExcepting: the general case, and adjustSubviews the exception! I already looped twice over the subviews; first, to try to accomodate them just by resizing and limiting some to their constraints, then again if that didn’t work, to collapse or expand some until all constraints were satisfied.

So I added a few lines to double this double loop; once while holding the excepted subview constant (if there was any), then once again while allowing it to resize if a solution was not reached. This proved to work quite well, and I was already near to publishing until I again found a circumstance where it didn’t work.

Turned out my initial implementation was faulty in that 3 passes (not 2!) were needed to find a viable solution. So the algorithm now loops 3 times if there is no excepted subview and 6 if there isn’t. By now, after refactoring the inner loop several times, no serious speed penalty is incurred and everything works much more smoothly.

Moral: refactoring should also consider what is the rule and what the exception…

Cocoa Programming

Well, it took over a month, but version 1.1 of RBSplitView is now out.

Originally I was calling it 1.0.5, but several people made so many good feature requests it became clear that 1.1 would be more appropriate. Special thanks to Dan Wood, Steve Gehrman and Brad Miller for their input and help with debugging.

When I began coding on XRay 2 some months ago, I ran into severe limitations with Cocoa’s NSSplitView. After a couple of frustrating weeks to make it behave like it should, I began looking for alternatives, to no avail; so I started coding my own version.

Then, as I realized that many other people were having similar problems, and that numerous Apple applications also seemed to have their own handrolled extensions to NSSplitView, I decided to publish my source. It has been a great learning experience.

With my recent decision to attend WWDC, I think the time has come to stop fiddling around with RBSplitView and return to XRay 2, in order to have a working alpha to bug the Apple engineers with. This will be fun!

Cocoa Programming

Yes, I know, it’s been over two weeks. I’ve been holding back some posts I’ve wanted to make, since they demanded preliminary work I couldn’t do at the time… scanning stuff and processing pictures, and so on. Hopefully next week…

Meanwhile, my proposed paper for the 2005 Advanced Developers Hands On Conference has been accepted. ADHOC (formerly famous as MacHack) will happen July 27-31, 2005, in Dearborn MI (near Detroit). A great conference for Mac developers.

Regarding the paper, the working title is: “Out of the Bottle: Beyond the Genie Effect”.

One Cocoa FAQ is how to do the Genie Effect. Unfortunately, the effect itself is done behind the curtain by the Window Manager. We’ll show how to do it in a few easy steps, which will teach you how to:

1) Overlay a transparent window over the screen and draw into it

2) Use OpenGL in that window to move images around

3) Make it appear that your windows are actually doing cool stuff.

Most important, this is the first paper I’ll be doing with a coauthor: Jeff Biggus, the mild-mannered secret identity of HyperJeff (cue applause!). Jeff will be doing the OpenGL part – something about which I know very little right now – and I’ll be doing the graphic interface part. He’ll also attend the conference to present the paper, as I won’t be able to make it this year.

In other news, RBSplitView 1.0.5 is nearly ready for publication. There’s still one feature request and a couple of bugs to take care of, but I hope to have it ready over the weekend. So watch this space…

ADHOC MacHack Cocoa

Time for another update on XRay 2.0.

With a little help from some friends, I have a preliminary idea of the GUI roughed out. It’s still too soon to present a screenshot, but I decided to go with the single-window idea I described previously. The main tool for the user to adapt the interface to personal needs will be the split view paradigm.

Vertically, the window will be divided into several splits, most of which will have subsplits. From the top down, we’ll have a simple file browser like XRay 1.1 has, with the Panther Finder sidebar at the left in a subsplit that can be collapsed. Below that, there’ll be a collapsible “history” list on the left, and the “Information” panel on the right. The latter will contain the actual information for any item selected in the file browser. But I plan to change from XRay 1.1’s switchable pane paradigm to an outline view, somewhat like the Finder’s “Get Info” windows.

In all probability, the information window will also be split – vertically, now – into “Metadata” and “File Content” sections, and the respective contents will be furnished entirely by plugins. Below that, there’ll be a third split for the batch processing interface, again subdivided into two subsplits, items to be processed on the left and actions to be taken on the right. Details are still hazy here, as I’ve never done batch processing before (at least not on the Mac).

After taking some time to get the splits working exactly right, I ran into serious limitations with Cocoa’s built-in NSSplitView class. For instance, programmatic collapsing/expanding of splits isn’t supported, and assigning minimum sizes to the splits works only under certain circumstances; in others, the subviews of the split can be pushed out pf view without any notice. I had both problems almost solved – the first by generating fake mouse drag events, the second by intercepting subclassing most of the default resizing methods – when I ran into some conditions where my solutions would fail to work correctly, with no alternative in sight.

So much for trying to fix NSSplitView. After examining and discarding some other fixes floating around on the net, I decided to reimplement NSSplitView from the ground up as a NSView subclass. This, in turn, launched me into writing an Interface Builder palette so I could set up my window as usual.

Unfortunately, after over a week of fiddling around, I have to report that the current documentation for writing complex palettes is almost non-existent and that there’s a serious bug in Interface Builder (at least in the current 2.4.2 version that comes with Xcode 1.5). Dragging other views into my container view doesn’t work at all and I was unable to find any workaround.

Still, I haven’t given up and I’m now deciding whether I should build my window entirely programmatically or spend a “Support Incident” at Apple ADC to get an official workaround… stay tuned.

It’s been a busy month with many interruptions. So we interrupt our hiatus to present you a <cue trumpets!> philosophical update on XRay 2.0.

As I mentioned before, 2.0 is being rewritten from the ground up. No source code, graphics or window designs from 1.x will be reused. XRay was my very first Cocoa application and while I got it working, often in ignorance of the right way of doing things, it’s beginning to show its age and maintaining it is becoming a difficult and, dare I say, embarrassing proposition. Starting again from the ground up has so far been as much fun as I had hoped.

The first important decision was of course to code for Panther (Mac OS X 10.3) and up. The statistics show that around 93% of the users that downloaded 1.1 are already on Panther, and until 2.0 comes out earlyish next year I hope that most of the holdouts will also have upgraded; so this hopefully won’t be a big issue. Cocoa on Panther has many added features that I’ve always been eager to take advantage of, so the user’s benefits will be smaller code with less hacking around limitations.

One new-for-Panther API that I won’t use yet is Cocoa bindings. From the many questions and head-scratching I see on the developer lists, this technology still isn’t as fully documented as I’d like, and some of it is still evolving. Also, it acts in an indirect way – bindings seem to be, essentially, useful side effects – that appears to be difficult to debug for someone who has only now fully absorbed things like Interface Builder’s outlets and actions. So I’ll pass on this for now and will look again at it when Tiger (Mac OS X 10.4) comes out.

Q: How many Mac developers do you need to change a lightbulb?

A: Six. One to come up with a snappy project codename, one to design the team T-shirt, one to draw the icon, one to design a completely new GUI, one to hack out a public beta of the lightbulb over the weekend, and one to draft the memo to management to ask for more powerful development hardware. 😆

As it’s just me toiling away here in the code mine, I skipped steps #1 and #2; the last step will hopefully be possible when a new 17″ PowerBook comes out. So I decided to make a new GUI and base the new icon on it. At the same time, I needed to have a strong reference to the original XRay icon:

I drew this icon overnight at MacHack 2001 and it’s become one of my most successful icons. Still, it’s based on the original Mac OS X aqua interface and I wanted to emphasize that the new version will be Panther-only, so here’s a preliminary rendition of what it will look like:

By the way, I successfully resisted changing the blue X to Jaguar fur (shudder) when 10.2 came out; it didn’t scale well. And it looks like Tiger will keep the metallic look of 10.3, so the new design should look up-to-date for some time.

While trying to draw the new icon I became obsessed with getting the metal texture exactly right, while not copying outright the metallic X best seen in Apple’s Installer window background. Not only because of trademark considerations, but also because it proved devilishly hard to reproduce. For one, the X isn’t drawn in any publicly available font; for another, the subtle shading effects would be mostly covered by the XRay “film”. So I settled for using the standard “metal window” background for the X, which is also instantly recognizable.

As I tried to draw the new icon in several sizes, including the 256×256 extra-large size required for Tiger compatibility, I noticed that the metal texture and edge shading shouldn’t scale with different sizes; also, the curvature of the film corners looked better when kept the same. In order to impress other programmers with my l33t C0c04 sk1llz, I decided to make the “About Box” window’s background into a scaleable version of the icon itself. This in turn led me to learn about partially transparent windows, the way window backgrounds, views and window controls are drawn, and how to produce various interesting animation effects. Eye candy, to be sure, but useful and impressive eye candy.

Again, what I learned in doing the “About Box” immediately suggested interesting options for the main GUI. Originally, XRay was a complement to the Finder’s single “Get Info” window, which in the days of 10.0 and 10.1 was seriously underpowered. Also, users migrating from Mac OS 9 were clamoring for separate info windows for different items, so XRay was patterned after the Mac OS 9 “Get Info”. XRay information windows were supposed to be relatively large and well-readable while still being small enough to allow having several of them open.

But times changed. With 10.2 and 10.3 the Finder’s “Get Info” changed to a different format and it also became able to do formerly missing things like changing permissions and deleting stubborn files with Administrator authorization. I noticed that I very seldom had several windows open anymore, and that my usage of XRay’s permissions panel was also rare. Many users were asking for batch-mode operations which the one-window-per-file paradigm didn’t accomodate well. At the same time, my idea of writing plug-ins to allow XRay to examine the internals of different file formats was running into snags, both because of the limitations of my first plug-in API design and because of the smallness of XRay’s windows.

So the new XRay will have one large window instead of several smaller ones. In other words, it will follow the quite successful browser paradigm of applications like iPhoto and iTunes: lists of favorite items, a structured large (but collapsible) browser pane, and another structured+collapsible information pane. Exact details are still being worked out, so I can’t show a window image yet. However, one problem presented itself immediately: Apple’s guidelines suggest using a metal window for this sort of application, as indeed iTunes, iPhoto, the Address book and several others do. It turns out that I like the metal interface well myself, but most of my beta testers hate it with a passion. Making the interface a user preference was immediately obvious, but forcing the user to restart the application to change it didn’t seem right; unfortunately Cocoa windows can’t be changed after they are open.

Well, my experiences with the non-standard “About Box” immediately suggested a possible way to do user-selectable window themes on-the-fly, and after some surprising detours I’m happy to report that it seems to be working well. Of course, all user interface widgets have to change theme as well when the window’s theme changes, and as I had to do some non-standard widgets for the “About Box” anyway, it proved relatively easy to extend the new concepts to the whole UI. This in turn led to the possibility of developing an entirely new theme, which in turn caused me to investigate other parts of Cocoa I hadn’t had occasion to explore before…

The challenge in these cases is to do new widgets in a style that is immediately obvious and coherent with existing widgets, or even in a way that most people won’t even notice that they’re not standard. Looking through current Apple applications that use the metal interface I was surprised to see a wide variety of non-standard widgets; evidently Apple is using their own applications as a development laboratory, in the sense that the most useful new widgets tend to become standard interface items in subsequent system versions. Hopefully my own UI items will be unobtrusive in this way.

More later…

Matt Gemmell has posted source code for a trick to stroke inside a NSBezierPath. The default stroke lies on the path’s boundary, which generates some awkward situations with sharp corners.

We came up with this during a long and fruitful discussion of options (via iChat) for Matt’s upcoming Pie Chart control for Cocoa. Watch that space…

Update: forgot to say that there’s more good Cocoa source on this page. Enjoy.