Solipsism Gradient

Some incremental improvements and a bugfix to the FolderSweep source code. Check it out if you’re using it. Also available from Matt Gemmell‘s svn server if you prefer that.

In other news, I had a couple more requests for handling “Smart Folders” in Quay. I’d looked briefly at the option before – I don’t use them myself – and had pushed it off to the next version. This time, however, curiosity made me look up how I might implement them if I had time to do so – and less than 2 hours later I had them working in the upcoming 1.1! Wouldn’t have been possible without Cocoa, of course.

Regarding 1.1, I’ve been stuck at the “just one more bug” stage for over a week. Meanwhile several smaller bugs were solved, the French localization (courtesy of Ronald Leroux) is finally finished and revised, and I really truly hope to release 1.1 over this weekend. Stay tuned.

Based on a question from Matt Legend Gemmell, we seized the occasion to write some source code to quickly sweep over a given folder and subfolders, getting file attributes and even contents where necessary. See details on my source code page.

This is based on some code I had handy in Quay, generalized for easy adoption. As usually happens with these side tracks, it also helped me find a couple of bugs in my original code, and I learned some new tricks.

Work on Quay itself is still going on, and I’m very near the 1.1b4 release, or perhaps I should call it 1.1fc1. I had some delay because of problems with my main development machine – the trusty iMac G5, now getting long in the tooth – since its occasional habit of turning off spontaneously suddenly got way too frequent.

Fortunately Jim Correia, a fellow developer, pointed me at an Apple note about this very problem, and it seems my machine is covered under the repair program. If all goes well I’ll have a new power supply next week, and that should solve it. Thanks Jim!

Via John Siracusa, I just read this great rant from Jens Alfke about Adobe’s Photoshop Elements 6 Installer. Money quotes:

First things first: After inserting the disc, I had no idea how to start the installation. I can’t remember the last time that’s happened.

…

It’s hard to get too worked up about an installer, one way or the other, but it’s annoying when it insists on installing over 2 gigabytes of stuff on my disk (most of which seems to be clip-art) without any choice to skip the inessentials. Nor would it even tell me ahead of time what it was installing, besides the ominously-named “system components”.

…

Is there an Adobe Updater Updater that puts up that alert and updates the updater? And what if the Updater Updater needs an update? (I can start to see where that 2 gigabytes went, now.)

Well, Adobe (and Microsoft, and…) are notorious for weird, complex and/or unusable installers and updaters. I myself dislike installers and Apple Installer packages; whenever I see an application with an installer package, I first use Charles Srstka’s excellent Pacifist to see what it install where, and what the installer scripts do.

So why does Quay, in its 1.1 betas, use a proprietary built-in installer? (Quay 1.0x was drag & drop.)

It was a difficult choice. Quay 1.0x was a pair of simple applications that could run from anywhere, and the background application ran by demand. Quay 1.1 still has a foreground and a background application, but the background application runs as a per-user launch agent, meaning it’s started by the system and runs all the time. It needs special permissions to use the accessibility API, and there’s an on-demand tool inside which also needs special permissions.

Of course, I could have done that with an Apple Installer package. However, writing shell scripts is not my cup of tea, and I feel more comfortable doing that myself; especially as my installer seeks out and uninstalls all previous versions, generates a cache file for the foreground app’s popup menu (the one with all the icons), and does other things hard or impossible to do in a shell script.

So, I reluctantly decided on a proprietary installer that runs only from the disk image. Indeed, the installer app is the same app as Quay itself; it runs as the installer if you run it from anywhere but the “installed” location (inside /Library/Application Support). What remains is a documentation and usability challenge, as Jens’ post shows. Here’s what the user sees when the Quay disk image mounts:

The upper icon explains what it does and asks the user to run it, the lower icon is the in-app help file. Notice that both are aliases (or, rather, symbolic links) and the originals are hidden inside an invisible folder. This hopefully prevents less-technical users from doing a drag-install that will behave oddly from their point of view, and also prevents them keeping a copy around in a unpredictable location – this makes the uninstaller’s job, later, much simpler. The installer also unmounts the disk image afterwards. All this is also explained in the installation screens.

I’m still tweaking the help file to be more, erhm, helpful – but few people seem to read help files nowadays. Hopefully this approach represents the lesser evil. So far, no user has complained!

I pushed Quay 1.1b3 out a few days ago and immediately had to quick-fix and slip in a new build (same version number) because of an endian issue on Intel Macs.

Yes, I should have tested this; for some reason, all people I asked to run preliminary builds also were on PowerPC Macs – including two I’d have sworn were on Intel. Murphy never sleeps. My own solitary Intel machine – a rock-bottom mini (Core Solo, 512MB) – had been down for almost a month, having had some indigestion with one of the last Leopard betas, and I finally spent an afternoon wiping it and reinstalling 10.5.1 on it (also seizing the opportunity to confirm that Qay still runs well, and with almost all features, on that version).

Curiously enough, this made me notice that two steps in my build chain (utilities that generate intermediary files) are still PowerPC-only and break when I build the project on an Intel machine. Must fix that because pretty soon I’ll have an Intel laptop available, and it must be able to do full builds.

I hope to have the next version out in the middle of next week. I’ve already fixed one issue – an oversight in the last rewrite made it stop working for folder aliases dragged directly into the Dock. There are a smaller bugs outstanding, but nothing too serious. Unless something turns up that requires serious testing in the field, the next release will (I hope) be 1.1 final.

Quay is the first software I wrote from the outset to be localized. Once you take some precautions this is relatively easy to do in a Cocoa application, and I’m talking to Ronald Leroux, a well-known localizer for French, to see whether we can get a pilot French version in time for 1.1. There are a few obstacles; I’m used to fiddling around with text right until the last minute; current localization software isn’t quite working yet for the new-fangles nib files used by Leopard/Xcode3; and code signing proves to be an unexpected puzzle.

Well, of course I can put all the current localization into the application bundle and sign it, and everything will “just work”. However, 99.9% of users will use only one of the included languages and some of those would be happy if no others were present. Code signing (at least the standard way which I’m using) doesn’t like files or folders removed from the application, nor does it like added items. It would indeed be nice if the various languages were present on the installation disk image but the user should be able to opt only for the currently enabled language, for instance. And it would also be nice be able to download additional or new languages as they become available.

I do have some ideas for workarounds to that issue, but I’m not sure they’ll work in time for next week’s release – be it 1.1b4 or 1.1 final. In any event I’m very anxious to finish off 1.1, as I’ve got serious plans for 1.2; a new GUI is only one aspect of that. Stay tuned.

Update: regarding localizations and signing, Mike Ash just reminded me that you can remove localizations without messing up the signature, but you can’t add new ones. I found out, however, that you can put a new localization somewhere outside the application bundle and then add a symlink from the resources folder from there. Interesting.

Interesting how some design choices you make early on cause a “snowball effect” – many product details end up depending on these choices. Sometimes you have to go back and change the initial direction of the snowball, so to speak, because some of those end effects turns out to be unacceptable.

When I started rewriting Quay for the first 1.1 public beta, I decided to change the way the background application (QuayMenu) was run. In 1.0, a Quay icon in the Dock was a file hidden away inside the Quay database package. When the user clicked on the icon, QuayMenu started up (if it wasn’t running already), and presented the menu. This simple scheme interfered not at all with the Dock itself but depended on several tricks to find out, approximately, the on-screen location of the clicked icon – something that a normal application doesn’t need to know. (It also didn’t work at all if you accessed the Dock over keyboard navigation.)

For 1.1, I decided to convert QuayMenu into a LaunchAgent: a per-user background application that runs constantly and is restarted by the system if it fails. I also worked out a way to have QuayMenu monitor click and keyboard events for the Dock, and use the Accessibility APIs to work out exact locations and details of a clicked Dock icon. So when the user clicks on a Dock icon, QuayMenu checks out if it ought to handle that click and pass it on if not.

This design decision had immediate consequences. It’s much easier to control (and update) a LaunchAgent if it’s in a fixed, known place. I decided to store the main Quay application inside the Quay database itself (which is in ~/Library/Application Support), both to prevent the user from moving it and to have a single known location for updating. The idea of having everything inside ~/Library stems from complaints I had about XRay storing stuff inside /Library, years before. So there was an a priori concern to make Quay a simple, no-hassle, per-user application; nothing stored outside the user’s home folder, no administrator password needed, no security concerns.

However, this immediately conflicted with the need of using the Accessibility API. Basically, if you use this API to ask other applications about their user interface, you have two options. You can wimp out and ask the user to turn on “Enable access for assistive devices” in the Universal Access preference panel, or you can run a “trusted” accessibility client. The first option means that you depend on the user having this turned on all the time; if it gets turned off, things stop working and you have to show a dialog asking for it to be turned on again; clumsy. Also, some people (including myself) don’t like turning this on because it changes the way tabbing between text fields behaves – it then tabs over buttons too.

The second option – running as a trusted client – was the better one, then. However, it too comes with a trade-off. A trusted accessibility client runs “setgid 90”, meaning the executable is forced to run from the special “accessibility” group. This is a tamer version of the tricky, all-powerful and potentially unsafe “setuid root” executable; its only advantage over a plain vanilla executable is that it can see (and affect) other applications’ user interface elements. However, there’s one common aspect; to turn the setgid bit on, you need to ask for an administrator password. As a side-effect, copying the executable to another place turns the bit off again, and it’s ignored altogether if run from an external volume or a disk image.

All this meant that I would have to write an installer to move the applications into their place, ask for an administrator password and turn on the setgid bit. Apple’s guidelines say that in such a case you should do a standard Apple Installer package and have all this accomplished by scripts inside the package. I decided against that for several reasons.

One, it’s harder to check an installer package against unauthorized modifications, while an installer application can use code signing to prevent those. Personally, after some bad experiences with badly-written install scripts, I distrust installer packages a bit more than separate installers – although I know people who distrust installers even more. The third option, of writing a self-checking, self-repairing application that could be drop-installed anywhere – as Quay 1.0x was – I reluctantly discarded. Some people prefer to have multiple copies of applications available, some use weird separate application folders (as was usual in the Classic days), and if you back up your applications with Time Machine, you’d have copies right there which shouldn’t be accessed except when restoring from the backup volume.

So the decision of writing a suitably self-explanatory and “just works”-type installer appears to be the right one. It’s actually working pretty well in the current (1.1b2) beta… except that I’d forgotten one important thing. FileVault (on the list of system features I don’t use myself) is based on a special “sparse” disk image that is mounted in place of your home folder. Meaning that, if you use FileVault, you can’t have setuid or setgid executables inside your home folder – and certainly not inside ~/Library.

OK, so I’m dutifully rewriting my installer to move everything into /Library which has no such limitations. An added advantage is that now there’s a single copy of Quay installed for all users; but should the serial number, then, also be valid for all users? That means the added hassle of separating the preferences into yet another file, installed into /Library/Preferences; one more thing to uninstall. An added disadvantage is that, now, the uninstaller – which I’m building into the Quay application itself – will need to ask for an administrator password, too. Or maybe I should have a setuid root uninstaller tool inside the application; I don’t like that either, but having such a tool would also mean an easy way of adding extra functionality to the Quay popups.

Well, all these are trade-offs, and I hope that when 1.1 (final) comes out, that the net effect will be positive. If all goes well, 1.1b3 should be out sometime next week, and you’ll have the chance to comment. Stay tuned.

As predictable, interest in Quay has flagged a little with the release of 10.5.2. To quote one comment:

…pointless… since 10.5.2 brings back the functionality.

Well, in a certain way, true; to the extent that, once Apple shipped TextEdit, it’s pointless to use other text editors; it’s free and edits text, right?

Well, not really. I’ve changed the Quay product page to explain:

So why should you use Quay at all? Extra information, more flexibility. For one, the Dock’s popups are limited to about 500 items; Quay’s limit is in the tens of thousands. You can have a Quay popup on both sides of the Dock; Apple has them only on the document size.

And some comparative screenshots (this one is for “Sort by Kind”, 10.5.2’s popup is on the right):

Let’s hope this makes my point.

Mac OS X 10.5.2 (9C31) came out yesterday and I’m running it now. Seems they declared it stable enough to release…

So I’m declaring 1.1b2 stable enough to release too. I’m just checking if everything’s still working and updating the documentation; expect a release later today.

If you’ve installed 10.5.2, you’ll see an extra menu item in Quay’s general preference popup (option-command-click on the Dock divider stripe), to make Quay act only on Stacks set to “List” view. Some other 10.5.2-only goodies will show up in the next (and hopefully, final) 1.1.

I’ve gotten a serious amount of feedback on this beta. Thanks to all of you who sent in bug reports and suggestions. It’s the first shareware I’ve written for a larger audience and I couldn’t have foreseen all those various operating conditions without this sort of feedback.

OK, Quay 1.1b1 is out. Get it here.

Thanks to all who’ve sent in bug reports and suggestions. Quay 1.1b2 (or 1.1 final, let’s hope) will be out in a week or so.