Solipsism Gradient

Some updates in random order. No time to re-find URLs for them, sorry.

It seems pretty much certain that the iPhone (at least the prototype) uses an ARM processor. That said, much of the hardware – except that affected by the FCC certification process, not sure if that would include the “computer” parts – may still change in the next few months.

It’s now being said that Cingular isn’t helping Apple with iPhone pricing at all, meaning that those prices are the actual prices. Apple is supposed to handle support, too. I don’t know enough about cellphones to say much about that, but it strikes me as less restrictive than the usual abusive tie-ins.

Many people seem convinced the narrow black strip on the iPhone’s left side is a phone chip or SD card slot. It’s not. Now that I’ve finally watched the keynote, it’s clear (and Steve Jobs says so) that it’s the volume control. This means that the phone chip, and the battery, are built in. However, they should be replaceable with the same amount of care (read: a whole lot) as in the iPods.

Negotiations between Cisco/LinkSys and Apple about the iPhone seem to have broken down. Cisco says they’re suing, while Apple haven’t commented; but from all indications they seem to have decided that the Apple iPhone is sufficiently different from the LinkSys iPhone (which is a cordless/VoIP phone) that there should be no confusion. Apple seems to have entered an iPhone trademark request through a dummy company.

“OS X” seems to be a new generic term, with “Mac OS X” now being understood as “OS X for the Mac”. Makes sense. Birdies tell me the iPhone OS X (or whatever it’ll be called) has a different kernel, which would make sense if the ARM processor is used – embedded kernels should be very finely tuned to the hardware.

Phil Schiller is supposed to have confirmed that the iPhone will remain closed to third-party software. Some sources add the words “for now”. Apple ADC says people interested should contact them, which sort of confirms my theory that, for now, it’ll be invitation-only, like it is for iPod games.

Several reasons are being discussed for the closure. One, of course, is that Apple wants to ensure the UI quality of the system. Another one is that third-party software running on a cellphone must be isolated into a sandbox, to disallow them tampering with the phone hardware – disrupting US 911 systems or whatever. This is of course easier for phones that use Java apps, and it might be a reason for allowing only widgets on the iPhone. A third reason would be that releasing any developer kit might give away too much about the generic Leopard/OS X, which certainly isn’t beta-ready for now.

The new Airport Extreme now accepts any number of printers or hard drives over its USB port. meaning it’s a NAS server. I posted some months ago about inexpensive NAS being a necessity, and several such are being launched at CES, too. This ties in nicely to Time machine’s need for an external backup drive. Let’s hope it (or some future firmware for it) work with ZFS, too.

Nokia in an interview welcomed the iPhone competition. Curiously enough, a few days before they introduced a new cellphone while saying “this is a computer, not just a cellphone”. Maybe they’ll change their name to “Nokia Computer, Inc.”?

People are relearning in a hurry that the symbol can be typed (on my keyboard, at least) as shift-option-K; there’s no easy HTML equivalent though, so I had to make a new small image for showing it here. So, with the tv out, why didn’t they name it the phone…? Probably just to associate it with the iPod, a name they’d be ill-advised to change.

Hm. Did someone leak the iPhone design to LG? Or is it a case of “great minds think alike”? Not that you can say from just one photo…

Scott Stevenson just posted “The Year in Mac Development“, an excellent summary. Two comments on specific points…

Re: interface guidelines:

If you need one rule to follow, make it this: don’t introduce new behaviors for existing controls. Don’t make a checkbox act like a push button or a slider act like a scroll bar. That will confuse users. If you need new behavior, make a new control. And when in doubt about UI standards, just do as Apple does.

I’d just sent a build of XRay II for a developer friend to look at – a very early pre-alpha, I hasten to add – and he called my attention to a detail on the UI. While replying to this, it occurred to me that almost all of my UI was custom controls; the only exception were scrollers and text fields. However, my friend hadn’t caught on to this at all, as the custom controls looked and behaved for the most part like Apple’s controls – or at least, like users think Apple’s controls should behave. A good example for this is of course RBSplitView; you can twiddle it to look just like any variation of Apple’s split views look, while underneath it’s all new code.

So, I’d amend Scott’s comment here to “don’t introduce new behaviors to existing controls unless they’re expected. Make a checkbox act as a better checkbox, not like a pushbutton.” Flashy visuals may well be nice or even appropriate in specific situations, but they’re liable to get in the way of the “just works” experience.

Re: marketing shareware controversies:

Perhaps most importantly, these events took the idea of buying independent Mac software out of the shadows and brought it into the community’s consciouness. A lot of the stigma of supporting smaller developers has been lifted, hopefully improving things for everyone involved.

For various reasons, I’ve stayed away from (or wasn’t invited, which was just as well) these new marketing events, tempting though they looked at first sight. They seem directed at a more general public than what is intended for my main application. Still, it’s interesting to note that my downloads and sales have picked up noticeably in the last 6 months. No idea if this is due to the effect Scott comments on, though…

Coming back to MacWorld, here are some more expectations I saw on various sites:

– iApps. Well, of course they’ll be improved as usual, and the Leopard versions will use stuff from the new frameworks. I still can’t understand why so many people turn this concept completely around and mention small app changes when they talk about the OS releases, though.

– iPods. Another area where incremental improvements go without saying. Apple might have a reply to the Zune ready, perhaps with a larger screen for video, perhaps with the patented touchscreen/body. Still, I’m not excited about this.

– Some people are – maybe only half in jest – suggesting that Steve Jobs will announce a career change. Perhaps a Bill Gates-like lateral promotion to “Chief Technologist”? There might something to that. The recent options flap showed that Wall Street’s expectations of a CEO’s functions are increasingly dissociated from what Jobs really does (or should be doing, anyway) at Apple. Of course you and me know that Tim Cook, the COO, already does the dull back-end operations stuff, but do the analysts understand that? I don’t think so.

Searching for some cool eye candy to add to my applications, I thought of having the main window flip around to present the preferences window – an effect pioneered by Dashboard widgets. Not finding a ready-made solution I set out to do it myself, and learn how to use CoreImage from Cocoa.

So, here’s some new source code: a category on NSWindow to flip from some window to another window. Please read the “ReadMe” file for details and caveats. The code should work on any PowerPC Macs with AltiVec, and any Intel Macs.

That said, it’s not as fast as I thought it would be, and I don’t want to spend more time on it right now. I’ll be looking into Quartz Composer to do something similar, although the upcoming CoreAnimation (for Leopard) will probably make this completely obsolete very shortly.

Rainer Brockerhoff wrote:

…Imagine my surprise when I learned that Peter had just published the complete Hex Fiend source code, and also started up a Wiki to explain details…

That said, I haven’t had time to look at details of his data backend yet, but that too looks like it will save me at least a month of tinkering.

Well, I’ve now had a little time to look at the Hex Fiend code.

As I expected, it builds a tree of referenced/changed byte ranges for an edited file. There’s a generic “ByteSlice” class with concrete subclasses that represent either a range of bytes inside a file or a range of bytes in memory – the latter would be the result of an editing operation like typing or pasting in stuff. That much I’m already doing myself, albeit with different names.

The interesting part comes when an edited file is saved. Hex Fiend goes to great lengths in optimizing writing time, allocated RAM, and disk space; it uses threaded AVL trees and lots of neuron grease, and while I understood very generally what’s supposed to be going on, the details are extremely daunting. My hat’s off to the wizard. And he spent similar care on optimized searching, too.

Now, just lifting all that code and plopping it into XRay II just wouldn’t be cost-effective. Yes, the result is that you can open a 240GB file on a 250GB disk, swap huge chunks of it around, and insert random bytes in the middle, and save it with no problem. Do I see this situation arising frequently for XRay II users? Frankly, no. Remember, the idea is to do structured editing of file contents, not necessarily pure hex editing… and Hex Fiend is already terrific at this (not to mention, free). I see the hex editing panels in XRay II more suited to editing small amounts of data and as a convenience to view raw file contents without necessarily changing them.

So, falling back on the old method of saving an edited file to a temporary file, then swapping it with the original if creation succeeded, means that complexity will go way down at the expense of speed (noticeable only for really huge files) and of the necessity of extra free space (same).

A second problem is that, for Hex Fiend, a file is just a sequence of bytes – no structure. For me, on the contrary, changing a byte in one place – meaning editing a representation of (say) an ID3 tag in a music file, or a QuickTime atom in a movie file – will usually mean that elsewhere in the file one (or even several) count or length fields will also have to change, preserving the file’s integrity. So my data representation tree also needs to reflect a particular file’s format as decoded by a plugin – and there may be several plugins seeing the file in different ways – and the nodes need to be more intelligent, notifying each other when necessary.

Still, seeing the Hex Fiend code has given me assurance that I can do it myself, so that’s good…

So, there I was in my trusty BBEdit (thanks Rich!) twiddling HTML code for a friend’s website, trying to get stuff aligned “just so”, checking out the CSS, cleaning up some redundant markup – and of course reloading the browser after every change to be absolutely sure it worked. My friend, who knows how to operate her Mac but is otherwise non-technical, watched this with astonishment… after we were finished (took most of an evening) she exclaimed, in German, “Das ist ja zum Mäusemelken!”

I burst out laughing at this colloquial expression I hadn’t encountered before, and which literally means “that’s for milking mice!”, but actually is used in the sense of “that’s extremely exasperating!”. Or so Google tells me. However, in the sense of twiddling with HTML code – or Objective-C code for that matter – it struck me that it can be taken literally. To quote Tom Digby’s “Little Tiny Eyes” (found via Tim Bray):

“OH YOU NEED LITTLE TEENY EYES FOR READING LITTLE TEENY PRINT

LIKE YOU NEED LITTLE TEENY HANDS FOR MILKING MICE.”

Yep, you have to position those little teeny hands just so and be extremely patient because it’ll take a long time to do it right and get enough… and it takes a very peculiar and determined kind of person to sit there N hours per day, day in and day out, just twiddling away at the little bits to get everything running “just so”.

And that is why my main page has the subtitle “finely crafted software for the Macintosh”. Hm. Should I change this to “Digital Mice Milking” or something? But I suppose that, just because I put this title on this post, that Google will already put me somewhere on its mousemilking results, forever after confusing hapless NIH researchers. But then, that’s the price of fame…

As expected, I’ve worked a lot on XRay II during the trip. I’d hoped to get the raw data updating backend working, but unfortunately debugging the plugin interface took much more time than I had anticipated. In particular, I ran into edge cases on my automatic view resizing algorithms and had to refactor them completely; it turned out that I had 3 different cases – for NSTextFields, for my outline views, and for plugin views – which were fighting each other and thus had to be folded into a single model. Anyway, the new scheme seems conceptually sound and the only thing missing are some optimizations.

To test all this out I started work on a QuickTime plugin. Currently this has a movie preview pane and a QuickTime atoms pane. The preview pane was actually working well when we started out on the trip but it turned out to be surprisingly hard to adapt to the view resizing scheme, so I’ll have to redo it. I’m using Tiger’s new QTMovie view which does a lot of work for you, but at some point it’s unavoidable to dip into the old Carbon QuickTime API which is, to put it mildly, a confusing patchwork.

No doubt there are sound historical reasons for this, but it’s a huge pain to get it all working in a modern Cocoa environment. For instance, the movie inside a QTMovie view doesn’t necessarily obey Cocoa’s clipping rules while it is playing – especially if it’s a QTVR panorama. QTVRs also seem to override Cocoa’s cursors even if the view itself is hidden or clipped out.

The second pane does a hierarchical display of QuickTime atoms. There’s a huge and confusingly documented roster of possible QuickTime atoms and they may be nested in often unintuitive ways. So far, I’ve been just using them to debug my nested container views and the autoresizing scheme, but it’s clear that this will probably be an extreme test case for both; other plugins won’t stress these aspects so much.

With all this, the data updating scheme took a back seat. Briefly, XRay II plugins will interact with the file system over an abstraction class called XRayItem. An item can represent an actual file system item, the entire contents of one such an item’s data or resource forks, or specific subparts of those. The idea is to have plugins chop XRayItems up into smaller pieces and pass them to other plugins to format and display them. Ultimately, once a data portion is changed by some editing action by the user the changed data are cached and the changes are passed upward the chain so other plugins can update their own representations accordingly. Then, when the changes are saved, they have to be collected and written out into a reasonably efficient manner.

Turns out this is not as easy as it sounds. Starting with just the default plugins that show the hex view of a file’s data fork, the user might want to open a 20GB file, select half of it, cut it to the clipboard, past it back at the end of the file (or even another file), go back to the middle and change a single byte, then do a search/replace loop over the result. Since, on Tiger, a typical Cocoa program can allocate just a little over 2GB of RAM (and not necessarily in a single chunk), this becomes a non-trivial memory management problem.

Of course, for the vast majority of files up to a certain limit – let’s say up to 128MB or so – keeping all that in RAM and changing it there would be the simplest and not-to-slow solution, and I’ll certainly have a fallback implementation for that. And if this were a Leopard-only app with a 64-bit version, this limit could be pushed a lot higher – but it has to run on Tiger and on 32-bit systems too.

So I was putting this off while tinkering with the other parts of the app, and seriously considering asking Peter Ammon for more hints, as I knew he’d solved the problem in his excellent HexFiend hex editor. Imagine my surprise when I learned that Peter had just published the complete HexFiend source code, and also started up a Wiki to explain details. Thanks a lot, Peter. Every Cocoa developer should download this gem, there’s lots of cool stuff inside from a Cocoa team insider; I’ve already learned how to filter out all fixed-width fonts, for instance.

That said, I haven’t had time to look at details of his data backend yet, but that too looks like it will save me at least a month of tinkering.

We’ve spent a couple of excellent days at our friend’s bed & breakfast near Verona. I’ve spent a few hours redoing their site in Sandvox, which I hadn’t really used much before; it’s a very cool app, although it takes a few minutes to get the hang of its use of “pagelets”.

They’re still in the age of dialup for now, I’ve spent some time downloading a couple of thousand e-mails, and even lost a few to an Eudora crash… so if you’ve emailed me and I haven’t answered, sorry. I’ll try to catch up when we get back.

We’re off again tomorrow to Savona to catch another ship, and will be back home before the end of the month. Stay tuned.

Yet Another interim progress report on XRay II.

Turns out some of my problems with the hex editor were due to overuse of the idea to have my scrollview’s delegate do everything (including standing in for the First Responder). So I changed back to a simpler model. I now have a single view inside my scroll view as the delegate, and it stays in place – it just checks the scrollbar’s status to decide what to draw.

As this subview is declared as NSView<NSTextInput> – meaning it’s a plain subclass of NSView that obeys the NSTextInput protocol – it’s also a descendant of NSResponder, and now stuff started “just working”. Seems obvious in retrospect, of course. There’s lots of details about implementing that protocol, but it’s mostly working now; I’m still not doing any actual editing, but that’s the top item on my list of things to do next.

Several other issues are being worked on in the meantime. Many things are more complex than I anticipated, but I believe the final result will be worthwhile. The actual plugin interface is still mutating from day to day, mostly because the actual usage patterns are hard to anticipate. I have two plugins partially working: File Metadata and File Forks (which use the basic hex editor to do its work).

I’m now making a separate version of the main XRay II application to be used as the plugin developer base. This will be a stripped-down version of the application: it’ll run only the two basic plugins and a third one, the one being developed. It will also have some debugging infrastructure in place.

Next, I’ll use this to start up projects for at least two more, perhaps more, plugins of different types, both reusing basic UI elements from the main application. Hopefully this will give me enough variations to publish a rich but robust plugin interface. I envision stopping development on the main application as soon as possible – perhaps even this year, if all goes well – and doing all the rest inside plugins.