Solipsism Gradient

Rainer Brockerhoff’s blog

Browsing Posts tagged Cocoa

Cocoa musings pt.1

No comments

One of the reasons for my taking a week or two off to work mostly on the just-released Klicko was that I like to rework, and group together, code snippets that worked well for me in earlier applications, and see if I can update them to conform to my slowly growing experience. I’m also prone to digress; one such digression (several months!) resulted in the release of the unexpectedly popular RBSplitView.

Both Klicko and Quay use code that, like RBSplitView, were destined for XRay II, the supposed successor of XRay, which sadly is not reliable under Leopard. Alas, at this writing, it sounds like XRay II will remain in the freezer, its mummy mined for code snippets and general philosophical experience… but the basic idea persists, and something quite equivalent (but also quite different in detail) is already being conceived.

Both Quay and Klicko do part of their seeming magic with a technology called “Quarz Event Taps” (PDF file). This was introduced in Tiger, and perfected in Leopard. Briefly, an event tap is a C callback routine that is called to filter low-level user input events at some points in the system’s event processing, which is actually quite complex. Events can be generated, examined, modified or even suppressed before they’re delivered to an application. Since user input events are usually routed to the foreground window (that is, to the foreground application, even if it has no window), this makes event taps quite powerful.

You can make a global event tap, or a per-process tap. Quay sets up a tap on the Dock process to intercept clicks on Dock icons. Klicko uses a global tap to check for clicks on background windows.

Tapping one application is, in principle, easy: you locate the application to be tapped by its PSN (Process Serial Number), set up the tap, tie it to your application’s main run loop, and that’s it. Here’s what a bare-bones implementation would look like:

// This is the callback routine, called for every tapped event.

CGEventRef ProcessEvent(CGEventTapProxy tapProxy, CGEventType type, CGEventRef event, void *refcon) {
   switch (type) {
      case kCGEventLeftMouseDown:
         // process a mouse down
      case kCGEventLeftMouseUp:
         // process a mouse down
   return event;   // return the tapped event (might have been modified, or set to NULL)
               // returning NULL means the event isn't passed forward

// Here's how you set up the tap: we're catching mouse-down and mouse-up

   ProcessSerialNumber psn;
   // get the PSN for the app to be tapped; usually with the routines in <Processes.h>
   CFMachPortRef tapg = CGEventTapCreateForPSN(&psn, kCGTailAppendEventTap, kCGEventTapOptionDefault, 
   if (!tapg) {   // bail out if the tap couldn't be created
      NSLog(@"application tap failed");
      [NSApp terminate:nil];
   CFRunLoopSourceRef source = CFMachPortCreateRunLoopSource(kCFAllocatorDefault, tapg, 0);
   if (!source) {   // bail out if the run loop source couldn't be created
      NSLog(@"runloop source failed");
      [NSApp terminate:nil];
   CFRelease(tapg);   // can release the tap here as the source will retain it; see below, however
   CFRunLoopAddSource(CFRunLoopGetCurrent(), source, kCFRunLoopCommonModes);
   CFRelease(source);  // can release the source here as the run loop will retain it

After that, all should work – in principle. The devil is in the details. Here’s how you locate a running application by its application ID and return its PSN:

BOOL GetPSNForApplicationID(NSString* appid, ProcessSerialNumber* outPSN) {
   outPSN.highLongOfPSN = outPSN.lowLongOfPSN = kNoProcess;
   while (GetNextProcess(outPSN)==noErr) {
      NSDictionary* pdict = [(NSDictionary*)ProcessInformationCopyDictionary(&psn,
         kProcessDictionaryIncludeAllInformationMask) autorelease];
      if ([[pdict stringForKey:(id)kCFBundleIdentifierKey] isEqualToString:appid]) {
         return YES;
   return NO;

To make a global tap, you don’t need a PSN. Just use the following tap creation call instead:

   CFMachPortRef tapg = CGEventTapCreate(kCGAnnotatedSessionEventTap, kCGEventTapOptionDefault, 

More details. If you’re tapping an application, it may not be running; CGEventTapCreateForPSN will return NULL in that case. Or it may quit while you have the tap set up. You probably want to monitor that process and either quit, or rerun the application, or wait for it to come back up. In the latter cases, you’ll have to back out of the now-dead tap carefully:

   CFRelease(tapg);   // this CFRelease has been moved from before the CFRunLoopAddSource

supposing, of course, that you have held on to those two variables. Note how the CFRelease(tapg) should, in such a case, happen only after the source has been removed from the run loop; otherwise invalidating the tap will cause the run loop to crash. You can use the same technique to close a global event tap, though usually there’s no need; if your app crashes or quits, the tap will be closed automatically.

However, there’s a serious problem while debugging an event tap. If you’re tapping a single application, and set a breakpoint inside yours (or break into the debugger anywhere because of a crash or exception), both applications will stop. If the same happens while a global tap is active, the entire system stops accepting user input! The only way to recover is to ssh/telnet in from another machine, and kill Xcode. So even if you prefer NSLog/printf calls to breakpoints, this will be very inconvenient for all but the simplest callback code.

The solution I found was to always use an application tap while debugging. An easy way is to define, as I always do, a special macro inside the main project build configuration panel (but for the debug configuration only): inside the “Preprocessor Macros Not Used In Precompiled Headers” (aka GCC_PREPROCESSOR_DEFINITIONS_NOT_USED_IN_PRECOMPS) write “DEBUG”, and then, instead of the global tap, compile in an application tap on some always-present application (like the Finder) by using #ifdef DEBUG/#else/#endif statements.

Even that isn’t always sufficient, as Xcode 3 notoriously may invoke the debugger (even on your release build!) if your app crashes. You must either get used to never clicking on “Build & Go” for your release build, or you must make a runtime check for the debugger. The latter will prevent inadvertent freezes, but if you forget to take it out before deployment, your application will behave oddly if a curious user runs it under a debugger.

This post is already too long, so I’ll talk only briefly about what you can do inside the event tap callback itself. Every possible execution path should be short and contain no long loops or wait points. If you’re just watching events, always return the same event passed in as parameter. Return NULL if you want to suppress an event; however, be careful to suppress entire event groups. For instance, if you decide to suppress a mouse-down, store the event number and also suppress subsequent mouse-dragged and mouse-up events with the same number; otherwise the destination application may behave oddly. Some apps may behave oddly when tapped, by the way.

Update: I previously said here that to intercept or generate keyboard events, your application must run with setgid 0 (the “wheel” group). I was mistaken; my apologies. Your application must run setuid root to make an event tap at the third tap point (which I didn’t mention here), which is where the events enter the window server (kCGHIDEventTap).

Blind Cook Attack

No comments

Release of Quay 1.1 has been stuck at the “one more bug” stage, as I’ve mentioned. Today I’ve finally unterstood what is happening, and a fix should be easy – I only have to rebuild most of my inter-thread communication. Herewith the cautionary tale.

Quay 1.01 was (as its help file boasted) “just a simple Cocoa app”. Indeed, clicking on a Quay item in the Dock opened the “QuayMenu” background application (which uses the NSUIElement flag). QuayMenu was optimized for quick startup – it read that item to get the folder it was pointing to, read the folder contents, showed a popup menu with them, and quit again when the menu was closed. Nothing easier; a very linear process.

Then in 1.1, all this changed. Now QuayMenu runs all the time in the background. There are two threads (plus a few more that don’t matter for this discussion). One thread listens (via Quartz Event Taps) for clicks on the Dock’s icons, finds out which icon it was, and if Quay should handle it, does a performSelectorOnMainThread: passing data about the icon to the main thread.

The main thread, in its turn, waits until it gets such an event and then pretty much does what the original 1.01 did: puts up a menu with the folder contents. It also shows an arrow over the clicked icon, both to show which icon it was and to allow the user to distinguish Quay menus from Dock menus.

Of course the arrow has to be in a window to be shown onscreen. 1.01 had an invisible window, since a popup menu has to be associated with a window – but now the window has to be shown only when the menu is onscreen, and hidden again when the menu is taken down.

Then around 1.1b2 I got complaints that, after a menu was canceled, the application that was active at the time of the initial wasn’t properly restored to the background – easy to fix by hiding the QuayMenu app after the menu went down. Soon after that, I thought that it would be prudent to imitate the Dock behavior when successive icons were clicked; that is, clicking on one icon, then on another, would produce two successive menus without any need to cancel the first one. And it also would be nice to be able to cancel a long menu while it was being built.

OK, so I introduced some inter-thread communication – mostly some flags and a cancel message – to take care of that. And after some fiddling it seemed to work, some conveniently ignored exceptions excepted. Then my MacBook Air arrived (my first multi-core Mac), and I noticed that the exceptions were more frequent; in fact, about 1 in 10 times, when I clicked on a second icon the first memu was taken down, and the second one either didn’t appear at all or it only flickered onscreen very briefly.

Well, this of course indicated a threading problem – as my experience with interthread communication was a little limited, that was to be expected – and I tried to find out where I was doing something dumb like not properly locking a shared resource or whatever. Since the bug never happened when I tried to break somewhere and single-step from there, it was indeed related to timing. But where?

Tons of NSLog()s were sprinkled about and everything seemed to be working properly. The messages from the first thread left at the proper time, the main thread caught them, the first menu was closed when necessary, the second menu was popped up as expected, but it mysteriously closed down immediately (1 in 10 attempts) or not (9 in 10 attempts). Everything else was as expected. Moreover, no canceling message was sent, or left over, that could close the second menu! It… just curled up and closed. Argh.

Finally – and it was really dumb of me not to think of that immediately – it occurred to me to look at the value returned to my Carbon Menu event handler for the kEventMenuEndTracking event. (Yes, I have a Carbon event handler because for complex UI reasons I need more control – and a shorter build time – than a Cocoa menu could give me.) Turns out that 9 out of 10 times the menu gets closed by kHIMenuDismissedBySelection (when the user picks a menu item); by kHIMenuDismissedByUserCancel (when the user closes the menu without selecting any item); or by kHIMenuDismissedByCancelMenuTracking (when my first thread canceled the menu because the user clicked on another Dock icon).

However – and here was the smoking gun – 1 out of 10 times the menu turned out to close by kHIMenuDismissedByAppSwitch (meaning QuayMenu went into hiding to put another application into the foreground). Now, that should indeed be happening briefly after the first menu was closed, but before the second menu was opened QuayMenu should jump into the foreground again. Now that I considered it, that’s a somewhat costly process that should be optimized out. But why was the application hiding after the second menu popped up instead of before? The sequence of events shouldn’t allow that at all!

Well, at least now I had a clue of where to put my NSLog()s and after some hours of sweating I had the answer: putting all that window hiding and showing in with application hiding and showing, and trying to optimize that by balancing the load over two threads – in brief, putting all urgent stuff into the first thread and all the potentially slow stuff and UI into the main thread – was fundamentally broken. It shouldn’t have worked at all, ever! So why was it failing only 1 in 10 times?

At this point I was visiting my mother and I was attempting to explain to her what was going on. Now, my mom is 95 – she’s still very sharp but knows absolutely zilch about programming. On the other hand, she’s an excellent work optimizer. I suppose 80+ years of kitchen duty will do that to you. At any rate, I was trying to explain what I was doing by resorting to a metaphor.

Assume two blind cooks working in a restaurant. The specialty is pancakes. An order for a pancake comes in to the first cook. He checks out (by Braille I suppose) what goes into the pancake, throws the ingredient into his pan, waves them briefly over his stove, then with a practiced flip of his hand he flicks the pancake to the second blind cook who’s supposed to finish the pancake.

Since they’ve practiced this a lot, and worked out some sort of signals, the second cook knows when to put up his frying pan just in time and in the right position to catch the pancake flying into his direction, and after that it’s easy. He tosses the pancake to a waiter who in turn serves it to the client.

Now, what was happening is that when a second order comes in before the first pancake is finished the second cook has to clean his pan. OK, we may have ridden the metaphor here until it breaks down (hehe) but let’s assume that he usually manages to do so, and proceeds to finish the second pancake in peace. Maybe the client has changed his mind about what sort of pancake he wanted, or something.

Only 1 times in 10 something weird happens and the second pancake is also thrown out unfinished. The first cook notices nothing amiss; he’s getting orders, setting up, and throwing the pancakes. The second cook notices nothing amiss; he’s catching the pancakes like he’s supposed to, cleaning his pan and all when necessary. It’s just 1 in 10 clients that complain because they never get their pancake!

Well, said my mom, so who was stealing the pancake then? The only explanation I found was that the kitchen shouldn’t have worked at all – it was working just by coincidence! In other words, the second cook wasn’t tossing the pancake to the waiter; he was tossing it out of the window! It just happened that at that crucial time a providential windmill wing swung by in front of the window, the pancake bounced off that and, who’da thought, right on the waiter’s plate!

Of course such a thing can’t work reliably and now and then one of the cook’s timing was off just enough to make the pancake miss the windmill wing, and the pancake would be gone with none of them noticing.

So my mom and I had a good laugh (“I can’t believe you actually think of your work that way!”, she said) and I went home. Tomorrow I must redo the whole kitchen routine. Stand by for news.

Some incremental improvements and a bugfix to the FolderSweep source code. Check it out if you’re using it. Also available from Matt Gemmell‘s svn server if you prefer that.

In other news, I had a couple more requests for handling “Smart Folders” in Quay. I’d looked briefly at the option before – I don’t use them myself – and had pushed it off to the next version. This time, however, curiosity made me look up how I might implement them if I had time to do so – and less than 2 hours later I had them working in the upcoming 1.1! Wouldn’t have been possible without Cocoa, of course.

Regarding 1.1, I’ve been stuck at the “just one more bug” stage for over a week. Meanwhile several smaller bugs were solved, the French localization (courtesy of Ronald Leroux) is finally finished and revised, and I really truly hope to release 1.1 over this weekend. Stay tuned.

Based on a question from Matt Legend Gemmell, we seized the occasion to write some source code to quickly sweep over a given folder and subfolders, getting file attributes and even contents where necessary. See details on my source code page.

This is based on some code I had handy in Quay, generalized for easy adoption. As usually happens with these side tracks, it also helped me find a couple of bugs in my original code, and I learned some new tricks.

Work on Quay itself is still going on, and I’m very near the 1.1b4 release, or perhaps I should call it 1.1fc1. I had some delay because of problems with my main development machine – the trusty iMac G5, now getting long in the tooth – since its occasional habit of turning off spontaneously suddenly got way too frequent.

Fortunately Jim Correia, a fellow developer, pointed me at an Apple note about this very problem, and it seems my machine is covered under the repair program. If all goes well I’ll have a new power supply next week, and that should solve it. Thanks Jim!

Mike Ash recently posted about code signing. We’d talked about the subject a few days before and that is a great summary of Mike’s thoughts about it.

Code signing on Mac OS X is still evolving, but with the release of the iPhone SDK, the subject is suddenly being thrust into the limelight. To quote the Wikipedia article:

Code signing is the process of digitally signing executables and scripts to confirm the software author and guarantee that the code has not been altered or corrupted since it was signed by use of a cryptographic hash.

In the past, code signing has been used on several platforms with differing intents and implementations; Microsoft, for instance, has a driver signing program that lets only Microsoft-certified device drivers run without warnings on Vista. In contrast, here’s a statement from an Apple engineer on the apple-cdsa mailing list:

Windows “code signing” is not only completely different in design and implementation, it also has a completely different *intent*. The Windows version tries to determine whether *Microsoft* thinks your program (or driver etc.) should be used. Mac OS X Code Signing is a general medium to let the manufacturer and user negotiate as to whether/how they want to trust each other, with the system providing the infrastructure to make this possible.

From what I’ve learned about the subject so far, the Mac OS X implementation seems indeed to be aimed in this direction – there’s very little documentation available. Even less is, of course, available for the iPhone OS implementation, and details are under NDA. But let me try to explain what I learned so far.

Mac OS X code signing is an implementation of the X.509 standard public key infrastructure. It depends on a cryptographical algorithm which gives reasonable assurance that signatures cannot be forged with currently wide-available technology. Of course that’s a rapidly moving target, and the twiddly mathematical details are beyond the scope of this post (and, in parts, of my poor brain too).

So, you can have signed executables (single binary files) and signed bundles (which can applications, frameworks, plug-ins and so forth). A signed bundle is easily distinguished by the presence of an extra “CodeResources” file inside the bundle’s Contents folder. This file is a XML file containing, among other data, a dictionary that contains a hash code for every file inside a bundle. It also has a list of “optional” files or folders; this allows you to remove things like unwanted localizations from an application without invalidating the signature. Of course, if you later re-add a localization its hash codes must match the one inside the file. Additionally, a signature is added to the bundle’s executable.

If you check Mac OS X’s system folder and applications, you’ll find that, beginning with Leopard, every framework, application and command-line tool is signed. This signature, by various convoluted means, is checkable (currently only by Apple’s codesign command-line tool). It also points back at Apple’s “root certificate“. This certificate is is a component in X.509’s chain of trust tree, which basically tells you which authorities certify that a particular signature is valid; in effect, when you reach the root of such a tree you have to decide whether the root certificate can be trusted. (X.509 also can be used for distributed, or peer-to-peer, trust networks.) Let’s see how you can check this in practice. Open Terminal and try this:

$ codesign -vvvv /Applications/
/Applications/ valid on disk
/Applications/ satisfies its Designated Requirement

(as usual, type the line after the $ prompt and you should see the resulting lines.)

So this means that the Safari application is signed and its signature is intact. Otherwise, you’d get either a “code object is not signed” message, or if the signature has been tampered with, a “code or signature modified” message. Now let’s try a small variation on the command:

$ codesign -dvvvv /Applications/
Format=bundle with Mach-O universal (i386 ppc7400)
CodeDirectory v=20001 size=7621 flags=0x0(none) hashes=375+3 location=embedded
Signature size=4064
Authority=Software Signing
Authority=Apple Code Signing Certification Authority
Authority=Apple Root CA
Info.plist entries=23
Sealed Resources rules=9 files=283
Internal requirements count=1 size=68

This gives more information. Specifically, we see that there are 3 “certification authorities” included into the signature, and that there is one “internal requirement”. To see this requirement, try:

$ codesign -dr- /Applications/
host => anchor apple and identifier ""
# designated => identifier "" and anchor apple

This is admittedly hard to wrap your neurons around. From what I understood, the “requirements” are actually a little language that establishes what conditions the signed bundle should satisfy. But satisfy to able to… what? Now there’s where everything is (on purpose) still a little vague. “anchor apple” apparently means that the basic requirement is that Apple’s root certificates be present and trusted. Open the Keychain Access utility, click on “certificates”, and you’ll see somewhere in the list a certificate called “Apple Root Certificate Authority”. You can double-click on it to see details.

But what is all this useful for? Your proverbial “Aunt Tillie” will certainly not be interested in typing commands into Terminal, following certificate chains, and reading up on trust audit procedures. Apple’s keychain software tries to make all that as transparent as possible. Simply logging into Mac OS X can open parts of the keychain, and when Aunt Tillie goes to her banking site, the bank’s certificate should be checked against the keychain’s certificates and with the issuing certification authority, and everything should “just work”. For that to be reliable, Safari’s code signature should also be checked – otherwise it could be hacked by some malicious entity to say that the bank’s site checks out OK even when it’s some phishing site masquerading as that bank!

Signatures and certificates are also checked by the parental control preference panel, by Mail when it receives signed e-mail, and so forth. You may be familiar with the dialog asking if a recently-updated application may access the keychain to store or retrieve passwords; for signed applications, this dialog will be issued only once. If a newly-installed update has the same signature as the previous version, which had already been authorized to access the keychain, it’s assumed to also be trusted to do so.

A similar scenario applies to critical code such as the codesign executable itself. Ideally, such code should refuse to run if it’s modified; Apple has some facilities already in place to do so, and even to detect if an executable ihas been modified while it’s already running by some malicious code injection. However, that sort of thing isn’t widely used yet; mostly because it would break certain facilities, like Input Managers, that people currently rely on.

One aspect I haven’t touched upon yet is the so-called “self-signed certificate”. This is a certificate which is its own root certificate, like the top certificates of the various ‘big” authorities. The difference is that these authorities are large companies that undergo serious auditions and verifications. As an example, Quay is currently self-signed; I’m still evaluating how to publish my certificate in a way that easily allows my users to check if new versions have indeed been signed by myself. Let’s run one of the same verifications on Quay:

$ codesign -dvvvv -r- /Applications/
Format=bundle with Mach-O universal (i386 ppc)
CodeDirectory v=20001 size=245 flags=0x300(kill,hard) hashes=6+3 location=embedded
Signature size=1671
Authority=Rainer Brockerhoff
Signed Time=Mar 8, 2008 08/03 22:28:09
Info.plist entries=14
Sealed Resources rules=4 files=18
host => anchor apple and identifier ""
# designated => identifier "net.brockerhoff.Quay" and anchor leaf = H"4cbb97c74336f7ee6aa566122a5e7688e1c725dc"

See the difference? It shows only my name as authority, and a SHA1 hashcode at the end. I could publish this hashcode on my site and it would be not too difficult for a technically knowledgeable user to verify that his copy of Quay matches that.

What does this really mean? Actually, not too much. It means that two versions of Quay that give identical names and hashcodes, which additionally match the ones published on this site, giving reasonable assurance that the author of both versions and of the site are the same; but nothing at all beyond that. However, there are ways around that – SHA1 hashcodes can be duplicated with some effort, though perhaps not yet for certificates, and this site doesn’t have a secure (https) variant. A malicious person could easily strip away my code signatures, resign everything with a new self-signed certificate using my name, make a phishing site that looks like mine, and so forth, and to a normal user everything would appear to be OK. Would someone do that for a €7 utility program? Very probably not worth the trouble, but you can’t be sure.

Then again, I could invest in a secure site, an official certificate signed by a trusted authority – perhaps even by Apple itself – and all that would just make the code signature itself more reliable; it wouldn’t necessarily correspond to what extent the code itself can be trusted to always “do the right thing”.

As Mike points out, there’s a huge gap between “signed” and “trusted”. Ultimately you have to trust somebody, or you would be too scared to buy a box at a computer store and open it, much less connect that computer to the Internet or let your children touch it. If I buy a certificate from Apple, and use that to sign my code, you have to trust someone in the certificate chain even if my code seems to “just work”… it wouldn’t necessarily mean that Apple has audited and quadruple-checked my source code. Indeed, it doesn’t even touch the question of whether Apple as a company can be trusted!

All that is still largely theoretical; Mac OS X still runs unsigned code with no problem, even though with some small restrictions regarding use of the keychain, the firewall, and internet access as I mentioned above. But Apple engineers say that in the future code signing will become mandatory. In the next post, I’ll try to discuss the implications of that – especially regarding the upcoming iPhone OS 2.0, where it will be mandatory from the outset.

I pushed Quay 1.1b3 out a few days ago and immediately had to quick-fix and slip in a new build (same version number) because of an endian issue on Intel Macs.

Yes, I should have tested this; for some reason, all people I asked to run preliminary builds also were on PowerPC Macs – including two I’d have sworn were on Intel. Murphy never sleeps. My own solitary Intel machine – a rock-bottom mini (Core Solo, 512MB) – had been down for almost a month, having had some indigestion with one of the last Leopard betas, and I finally spent an afternoon wiping it and reinstalling 10.5.1 on it (also seizing the opportunity to confirm that Qay still runs well, and with almost all features, on that version).

Curiously enough, this made me notice that two steps in my build chain (utilities that generate intermediary files) are still PowerPC-only and break when I build the project on an Intel machine. Must fix that because pretty soon I’ll have an Intel laptop available, and it must be able to do full builds.

I hope to have the next version out in the middle of next week. I’ve already fixed one issue – an oversight in the last rewrite made it stop working for folder aliases dragged directly into the Dock. There are a smaller bugs outstanding, but nothing too serious. Unless something turns up that requires serious testing in the field, the next release will (I hope) be 1.1 final.

Quay is the first software I wrote from the outset to be localized. Once you take some precautions this is relatively easy to do in a Cocoa application, and I’m talking to Ronald Leroux, a well-known localizer for French, to see whether we can get a pilot French version in time for 1.1. There are a few obstacles; I’m used to fiddling around with text right until the last minute; current localization software isn’t quite working yet for the new-fangles nib files used by Leopard/Xcode3; and code signing proves to be an unexpected puzzle.

Well, of course I can put all the current localization into the application bundle and sign it, and everything will “just work”. However, 99.9% of users will use only one of the included languages and some of those would be happy if no others were present. Code signing (at least the standard way which I’m using) doesn’t like files or folders removed from the application, nor does it like added items. It would indeed be nice if the various languages were present on the installation disk image but the user should be able to opt only for the currently enabled language, for instance. And it would also be nice be able to download additional or new languages as they become available.

I do have some ideas for workarounds to that issue, but I’m not sure they’ll work in time for next week’s release – be it 1.1b4 or 1.1 final. In any event I’m very anxious to finish off 1.1, as I’ve got serious plans for 1.2; a new GUI is only one aspect of that. Stay tuned.

Update: regarding localizations and signing, Mike Ash just reminded me that you can remove localizations without messing up the signature, but you can’t add new ones. I found out, however, that you can put a new localization somewhere outside the application bundle and then add a symlink from the resources folder from there. Interesting.

Design snowball

No comments

Interesting how some design choices you make early on cause a “snowball effect” – many product details end up depending on these choices. Sometimes you have to go back and change the initial direction of the snowball, so to speak, because some of those end effects turns out to be unacceptable.

When I started rewriting Quay for the first 1.1 public beta, I decided to change the way the background application (QuayMenu) was run. In 1.0, a Quay icon in the Dock was a file hidden away inside the Quay database package. When the user clicked on the icon, QuayMenu started up (if it wasn’t running already), and presented the menu. This simple scheme interfered not at all with the Dock itself but depended on several tricks to find out, approximately, the on-screen location of the clicked icon – something that a normal application doesn’t need to know. (It also didn’t work at all if you accessed the Dock over keyboard navigation.)

For 1.1, I decided to convert QuayMenu into a LaunchAgent: a per-user background application that runs constantly and is restarted by the system if it fails. I also worked out a way to have QuayMenu monitor click and keyboard events for the Dock, and use the Accessibility APIs to work out exact locations and details of a clicked Dock icon. So when the user clicks on a Dock icon, QuayMenu checks out if it ought to handle that click and pass it on if not.

This design decision had immediate consequences. It’s much easier to control (and update) a LaunchAgent if it’s in a fixed, known place. I decided to store the main Quay application inside the Quay database itself (which is in ~/Library/Application Support), both to prevent the user from moving it and to have a single known location for updating. The idea of having everything inside ~/Library stems from complaints I had about XRay storing stuff inside /Library, years before. So there was an a priori concern to make Quay a simple, no-hassle, per-user application; nothing stored outside the user’s home folder, no administrator password needed, no security concerns.

However, this immediately conflicted with the need of using the Accessibility API. Basically, if you use this API to ask other applications about their user interface, you have two options. You can wimp out and ask the user to turn on “Enable access for assistive devices” in the Universal Access preference panel, or you can run a “trusted” accessibility client. The first option means that you depend on the user having this turned on all the time; if it gets turned off, things stop working and you have to show a dialog asking for it to be turned on again; clumsy. Also, some people (including myself) don’t like turning this on because it changes the way tabbing between text fields behaves – it then tabs over buttons too.

The second option – running as a trusted client – was the better one, then. However, it too comes with a trade-off. A trusted accessibility client runs “setgid 90”, meaning the executable is forced to run from the special “accessibility” group. This is a tamer version of the tricky, all-powerful and potentially unsafe “setuid root” executable; its only advantage over a plain vanilla executable is that it can see (and affect) other applications’ user interface elements. However, there’s one common aspect; to turn the setgid bit on, you need to ask for an administrator password. As a side-effect, copying the executable to another place turns the bit off again, and it’s ignored altogether if run from an external volume or a disk image.

All this meant that I would have to write an installer to move the applications into their place, ask for an administrator password and turn on the setgid bit. Apple’s guidelines say that in such a case you should do a standard Apple Installer package and have all this accomplished by scripts inside the package. I decided against that for several reasons.

One, it’s harder to check an installer package against unauthorized modifications, while an installer application can use code signing to prevent those. Personally, after some bad experiences with badly-written install scripts, I distrust installer packages a bit more than separate installers – although I know people who distrust installers even more. The third option, of writing a self-checking, self-repairing application that could be drop-installed anywhere – as Quay 1.0x was – I reluctantly discarded. Some people prefer to have multiple copies of applications available, some use weird separate application folders (as was usual in the Classic days), and if you back up your applications with Time Machine, you’d have copies right there which shouldn’t be accessed except when restoring from the backup volume.

So the decision of writing a suitably self-explanatory and “just works”-type installer appears to be the right one. It’s actually working pretty well in the current (1.1b2) beta… except that I’d forgotten one important thing. FileVault (on the list of system features I don’t use myself) is based on a special “sparse” disk image that is mounted in place of your home folder. Meaning that, if you use FileVault, you can’t have setuid or setgid executables inside your home folder – and certainly not inside ~/Library.

OK, so I’m dutifully rewriting my installer to move everything into /Library which has no such limitations. An added advantage is that now there’s a single copy of Quay installed for all users; but should the serial number, then, also be valid for all users? That means the added hassle of separating the preferences into yet another file, installed into /Library/Preferences; one more thing to uninstall. An added disadvantage is that, now, the uninstaller – which I’m building into the Quay application itself – will need to ask for an administrator password, too. Or maybe I should have a setuid root uninstaller tool inside the application; I don’t like that either, but having such a tool would also mean an easy way of adding extra functionality to the Quay popups.

Well, all these are trade-offs, and I hope that when 1.1 (final) comes out, that the net effect will be positive. If all goes well, 1.1b3 should be out sometime next week, and you’ll have the chance to comment. Stay tuned.

So, somewhere around the beginning of this month I was resting a bit from writing my book (which I’ve mentioned in passing here), and it struck me that the complaints about Leopard’s Stacks weren’t dying down – if anything, they were growing more numerous. I hadn’t really used Tiger’s Dock popup menus much, but I really thought someone would find out a way around it soon.

But when I searched – not too intensely at first – for such a solution, I found that the existing ones were really quite (searching for a charitable word here)… feeble. They all depended on something outside the Dock itself. And this struck me as what Apple likes to call a “third-party opportunity”.

So while I was testing out some sample code for the book, a few days later, I thought I could play around with it and find out what could be done with the Dock. I immediately noticed that putting a proxy icon in the Dock might be doable, and popping up a menu from there might be too. In a few days I had a primitive version of the menu itself working – I decided it had to be a background/foreground application pair.

Up to then I hadn’t put much time in but when I tried to do the foreground application I began hitting some snags. On November 10 – 20 days ago – I decided to throw everything I’d done so far away and start over from zero. Two days later I had a working alpha to show to some friends, and on Nov.15 I decided it was good enough for a public beta. It took a day to make the necessary changes to my site, prepare listings on the various software sites, write a press release and so forth.

Response to the first beta was so good I decided to keep at it full-time; I also figured that, now that people could see how it might be done, competitors would soon appear. Anyway, on Nov.27 – version 1.0 came out, and today, exact 20 days after restarting the project, 99% of known bugs are fixed in 1.0.1. (In case you’re curious, double-clicking still doesn’t work satisfactorily.) And I still haven’t heard of any full-featured competitor… yes, the principle is easy, but the details aren’t.

So far I’m quite satisfied with the responses. One thing I hadn’t foreseen was the number of people not content with popups – many also want to drag & drop files on the Dock folders. And this didn’t crop up in the complaints I read, because you can drop files on Stacks! But I rarely or never did that, so it was a surprise to me.

The problem is, to the current Dock, every type of item is handled differently. An application goes into the left side. Files and folders go into the right side; folders become Stacks, and can receive drops; files can’t. On the other hand, files can be clicked on – that is, the application that “owns” them is notified – but for folders, nothing happens. I tried various combinations of packages and bundles to get a folder that could be both clicked and dropped on, but no luck.

Compounding the difficulties was my resolution to “use no magic”. I knew this would be a powerful positive factor; users are tired of side effects caused by applications hacking the system. However, this meant that the Dock would never know I was doing something around its functionality, and, since there’s no public API to get information into and out of the Dock, I was quite restricted in what I could do.

I was also surprised that, despite my explaining all that on the product page, and repeatedly in response to questions in the support forum, people kept writing in with suggestions that showed that, for them, Quay did hack the Dock somehow – or even was a plugin for it. That’s the downside of when your technology “just works”: people’s mental models of what you’re doing can often diverge severely from reality.

Speaking of which, here’s my favorite review:

Just been playing with Quay. Rainer is a mad man. It takes true insanity to hack the dock without actually hacking the dock.

None of this would have been possible without Cocoa; I can’t imagine writing such a complex (internally, that is) pair of applications in anything else in 20 days. Many of the pieces – regarding registration and serial number handling, especially – came from my work on XRay II. Some of the icon and file handling routines were also adapted from there. The flipping “About” window is straight from my Flipr code; as soon as I find time, I’ll rewrite that for CoreAnimation.

Looking over the code today I was struck by the diversity. The main Quay app uses mostly NSWorkspace and NSFileManager, but then suddenly dips into Carbon to handle icons and some menu aspects. The background app (QuayMenu) uses mostly FSRefs and some BSD stuff for file handling, but it’s of course written in Cocoa, too.

A great side-effect of Cocoa is that, once your infrastructure is in place and coded properly, adding most new features is just a matter of a few lines; when it requires more, it shows that the original design wasn’t powerful enough. A good learning experience.

Photos licensed by Creative Commons license. Unless otherwise noted, content © 2002-2022 by Rainer Brockerhoff. Iravan child theme by Rainer Brockerhoff, based on Arjuna-X, a WordPress Theme by SRS Solutions. jQuery UI based on Aristo.