Solipsism Gradient

As expected, I’ve worked a lot on XRay II during the trip. I’d hoped to get the raw data updating backend working, but unfortunately debugging the plugin interface took much more time than I had anticipated. In particular, I ran into edge cases on my automatic view resizing algorithms and had to refactor them completely; it turned out that I had 3 different cases – for NSTextFields, for my outline views, and for plugin views – which were fighting each other and thus had to be folded into a single model. Anyway, the new scheme seems conceptually sound and the only thing missing are some optimizations.

To test all this out I started work on a QuickTime plugin. Currently this has a movie preview pane and a QuickTime atoms pane. The preview pane was actually working well when we started out on the trip but it turned out to be surprisingly hard to adapt to the view resizing scheme, so I’ll have to redo it. I’m using Tiger’s new QTMovie view which does a lot of work for you, but at some point it’s unavoidable to dip into the old Carbon QuickTime API which is, to put it mildly, a confusing patchwork.

No doubt there are sound historical reasons for this, but it’s a huge pain to get it all working in a modern Cocoa environment. For instance, the movie inside a QTMovie view doesn’t necessarily obey Cocoa’s clipping rules while it is playing – especially if it’s a QTVR panorama. QTVRs also seem to override Cocoa’s cursors even if the view itself is hidden or clipped out.

The second pane does a hierarchical display of QuickTime atoms. There’s a huge and confusingly documented roster of possible QuickTime atoms and they may be nested in often unintuitive ways. So far, I’ve been just using them to debug my nested container views and the autoresizing scheme, but it’s clear that this will probably be an extreme test case for both; other plugins won’t stress these aspects so much.

With all this, the data updating scheme took a back seat. Briefly, XRay II plugins will interact with the file system over an abstraction class called XRayItem. An item can represent an actual file system item, the entire contents of one such an item’s data or resource forks, or specific subparts of those. The idea is to have plugins chop XRayItems up into smaller pieces and pass them to other plugins to format and display them. Ultimately, once a data portion is changed by some editing action by the user the changed data are cached and the changes are passed upward the chain so other plugins can update their own representations accordingly. Then, when the changes are saved, they have to be collected and written out into a reasonably efficient manner.

Turns out this is not as easy as it sounds. Starting with just the default plugins that show the hex view of a file’s data fork, the user might want to open a 20GB file, select half of it, cut it to the clipboard, past it back at the end of the file (or even another file), go back to the middle and change a single byte, then do a search/replace loop over the result. Since, on Tiger, a typical Cocoa program can allocate just a little over 2GB of RAM (and not necessarily in a single chunk), this becomes a non-trivial memory management problem.

Of course, for the vast majority of files up to a certain limit – let’s say up to 128MB or so – keeping all that in RAM and changing it there would be the simplest and not-to-slow solution, and I’ll certainly have a fallback implementation for that. And if this were a Leopard-only app with a 64-bit version, this limit could be pushed a lot higher – but it has to run on Tiger and on 32-bit systems too.

So I was putting this off while tinkering with the other parts of the app, and seriously considering asking Peter Ammon for more hints, as I knew he’d solved the problem in his excellent HexFiend hex editor. Imagine my surprise when I learned that Peter had just published the complete HexFiend source code, and also started up a Wiki to explain details. Thanks a lot, Peter. Every Cocoa developer should download this gem, there’s lots of cool stuff inside from a Cocoa team insider; I’ve already learned how to filter out all fixed-width fonts, for instance.

That said, I haven’t had time to look at details of his data backend yet, but that too looks like it will save me at least a month of tinkering.

This time, I’ve come back with 99% of my photos already organized. Which is good, since I took about 1500 on this trip; those new 2GB SD cards come in very handy. As you can see on the left, I’ve already uploaded several of the new pictures (and half a dozen from the August trip to San Francisco). And a few dozen more are already in the queue; I plan to upload 8 pictures every day over this week and the next.

Sorry, but while I’m away I’ve locked all the forums and topics against posting by non-registered users; spam posts have increased a lot this week, although all links posted are “rel=nofollow”…

I’ll be working on some possible solutions but will be able to implement them only next month. I apologize for the inconvenience.

And, back. Life is good. Whew. 32 hours door-to-door, so we’re very tired and there are a few thousand emails to catch up on. More tomorrow.

We’ve spent a couple of excellent days at our friend’s bed & breakfast near Verona. I’ve spent a few hours redoing their site in Sandvox, which I hadn’t really used much before; it’s a very cool app, although it takes a few minutes to get the hang of its use of “pagelets”.

They’re still in the age of dialup for now, I’ve spent some time downloading a couple of thousand e-mails, and even lost a few to an Eudora crash… so if you’ve emailed me and I haven’t answered, sorry. I’ll try to catch up when we get back.

We’re off again tomorrow to Savona to catch another ship, and will be back home before the end of the month. Stay tuned.

On board the Costa Mediterranea, using their ruinously expensive (30 Euros per hour) satellite connection. All is well, I’ll have more news on the weekend.

I’ve written about Apple’s use of the TPM chip before. My basic conclusion was, there’s no evidence Apple is using the chip for anything sinister, or at all in current versions (Tiger). However, I also said Apple should use the chip as a basis for secure vitualization in Leopard:

…Apple should write a fully trusted hypervisor into the EFI (using the TPM) and run everything inside virtual machines, including Mac OS X for Intel itself. Booting some version of Windows into a second VM would be easy, then, and there wouldn’t be a full version of Mac OS X for Intel for people to run on standard PCs either. I don’t think dual-booting is a good solution, I believe Apple was just testing the waters with BootCamp.

I still think virtualization is a good idea… however, there’s new evidence that Apple doesn’t think so, or at least not in conjunction with the TPM chip.

First, ifixit posted a disassembly of the new Core 2 Duo MacBook Pros, with zoomed-in photos of the logic board. They’re not detailed enough to show all IC part numbers, but I can say with some confidence that there’s no TPM chip at all. However, to the right of the RAM socket in the second picture, there’s an empty space for a 28-pin flat-pack IC – just the size of the Infineon SLB9635TT chip found on all previous Intel Macs. I’ve been searching for a similarly detailed picture of the Mac Pro’s motherboard, with no luck so far.

Second, Amit Singh of Mac OS X Internals fame – which I bought and read recently, BTW – has posted, in his usual precise style, details on how to use those Macs’ TPM chip. Here are some salient points:

The media has been discussing “Apple’s use of TPM” for a long time now. There have been numerous reports of system attackers bypassing “Apple’s TPM protection” and finding “Apple’s TPM keys.” Nevertheless, it is important to note that Apple does not use the TPM. If you have a TPM-equipped Macintosh computer, you can use the TPM for its intended purpose, with no side effect on the normal working of Mac OS X.

…

At the time of this writing (October 2006), the newest Apple computer models, such as the MacPro and the revised MacBook Pro do not contain an onboard TPM. Theoretically, Apple could bring the TPM back, perhaps, if there were enough interest (after all, it is increasingly common to find TPMs in current notebook computers), but that’s another story.

He then goes on with very detailed instructions on how to write, install and use a device driver for the TPM chip.

All this is very interesting, but as the TPM isn’t anymore standard equipment you could rely on finding on any Intel Mac, this is more an academic exercise. I doubt that Apple will implement anything important in Leopard that won’t run on the new Pro machines, so no trusted hypervisor for me. Ah well…

We’re also packing for yet another trip. If all goes well, we’ll be in Milano (Italy) early on Sunday, catch a train to Venezia, and from there start the first of two Mediterranean cruises. In between we’ll visit friends in Verona. Back around the end of November…

Of course, I’ll take my trusty laptop and plan to make excellent progress on the XRay II project. So stay tuned for developments.