Solipsism Gradient

hmafra wrote:

Well, I meant PBS Cringely (Robert E. Cringely). I didn’t remember the other Cringely, who has an X. instead of E.

Sorry, they both have an X…

hmafra wrote:

I can’t see much use for the kernel changing as well, but the part where he describes a possible use for virtualization sounds good (Easy DOS It)

It’s well-known that Windows running under Virtual PC is more stable too… that’s not news. I don’t think Apple will do any explicit marketing of the fact, when there are much more interesting things to do with virtualization, as I said before.

Rainer Brockerhoff wrote:

…Indeed, the usual hardware developer notes which used to come out a few weeks after a new PowerPC Mac was released are still absent for all Intel Macs – not that these notes ever went into chip-level detail of any Apple hardware.

The developer note for the Intel iMacs just came out (along with a few others). There are some interesting tidbits – for instance, EFI Flash memory size is 2 megabytes – but no mention of the TPM chip, as expected. Also, all the recent notes are very terse and go into less detail, even for PowerPC machines…

Rainer Brockerhoff wrote:

…But the main advantage is that the OSes for the virtual machines can be simplified. All the tricky little kexts and drivers you see on current PowerPC Macs will be substituted by one or two “generic” versions which will interface to the virtual peripherals simulated by the hypervisor, and the actual machine’s peripheral drivers will be in EFI or on the cards themselves.

One variation of this idea would be for Apple to sit down with game developers and define a basic Intel VM; just enough to see a drive partition, input devices and the video card. This would make porting very easy, as most games try to take over as much of the machine as possible anyway, and you’d have optimum performance.

Now and then I read complaints about Xcode on blogs and mailing lists. It’s come a long way but some parts are still slow and cumbersome, granted. One of the complaints – which usually comes from Java or Windows C++ migrants – is that Xcode has no refactoring aids. Some people even publish workarounds.

So what is this refactoring thing anyway? According to Wikipedia:

Refactoring is the process of rewriting a computer program or other material to improve its structure or readability, while explicitly keeping its meaning or behavior…

Refactoring does not fix bugs or add new functionality. Rather it is designed to improve the understandability of the code or change its structure and design, and remove dead code, to make it easier for human maintenance in the future. In particular, adding new behavior to a program might be difficult with the program’s given structure, so a developer might refactor it first to make it easy, and then add the new behavior.

I’d tend to agree with that, up to a point. I usually refactor when I reach a dead end in the software’s structure, that is, when the current structure won’t allow me to proceed implementing what I want to implement. Or – probably the same thing, essentially – when I find myself implementing things I don’t want to implement anymore.

But my tendency (see fractal programming) is to do it in the reverse order; I write some code that does new stuff in a new way. Then I migrate lots of old code into the new scheme, often rewriting it radically if necessary, or throwing entire blocks away. (Well, not literally at first; I prefer to comment such blocks out or move them into a “dead code” file for later reference.)

Now, the aforementioned migrants usually don’t see it that way. Rather, they want some automation to make the process easier:

An automated tool such as a SCIDs to help you do might work like this:

– I have a method which has some code that I would like to pull out into its own method.

– I highlight the offending code.

– I select Extract Method from a popup menu

– The RefactoringBrowser asks me to name the method and automatically creates it and inserts the highlighted code.

– In the current method, the highlighted code is replace by an invocation to the newly created method.

All very nice, but it presumes several things which I don’t see coming to Xcode (at least not to the Objective-C parts):

– You have a very regular, structured style of coding that conforms to standards the “RefactoringBrowser” understands.

– You always use the standard refactoring methods, such as expanding, collapsing, pulling out, pushing in, whatever.

– All source code in your project has been previously parsed and stored in the SCID (source code in database), so the browser and refactoring software have a perfect understanding of your code.

This is perfectly possible (or at least I’m told it is) in Java and perhaps in C++ – though I’m skeptical about the latter. I was astounded when a friend, who was qualifying to some Java certificate or other, asked me to have a look at his source code. A quite trivial program was expanded to several dozen source files, consisting of literally hundreds of small methods that differed from each other only by name and a few characters or line. No doubt everything was set up very logically and hierarchically and according to whatever standards a certified Java programmer must obey, but… it was completely illegible by my (admittedly eccentric) standards. It was code only a SCID could love.

So, suppose my friend decided to refactor his code. Just renaming a few classes must necessarily entail profound changes in all source and project files. Not only must the filenames themselves be changed, but all mentions of this class must also be changed. Wait, don’t we have global search & replace for that…?

But, of course, renaming classes or methods is trivial. Maybe it’s suddenly obvious that you need to push some methods off into a subclass, or pull them up into their superclass. Wouldn’t it be nice to have this done automatically?

Well, not really. First off, inheritance is much less used in Objective-C – or at least, I use it less than I used to do in my C++ days (I refuse to learn Java). Runtime binding, introspection and categories mean you usually don’t have to subclass more than one or two deep from the standard Cocoa classes. In fact, I believe I just today went to the third subclass level for the first time. So, automating such a superfluous process makes little sense.

Second, remember that Objective-C is just a small superset of C (unlike Java and C++ which are C-like languages). And that Mac OS X is Unix-based, with many headers pulled in from heterogenous sources. This means, of course, that all the old crufty things of the old C days are still there… pointers, pointer casting, weird #defines, other tricks – you name it. And all of this is liable to be #included into your poor unsuspecting code if you do any work at all with Carbon or BSD APIs, as most non-trivial applications need to.

In other words, I don’t believe it’s possible to feed all of this into a SCID and expect it to behave rationally; of course the gcc compiler has to make sense of all this eventually, but I seriously doubt it could be easily refactored to go back and forth from its internal representation to the source code while it’s being edited. It’s still, essentially, a batch processor.

Suppose someone pulled this transformation off. Suppose all the old C headers were tamed to make them compatible. Suppose we had everything in a hierarchical, intelligent, refactoring browser/editor. Now what?

It may be some congenital deficiency in my own neural wiring, but I can’t recall ever refactoring my code twice the same way (except for that trivial class/method renaming). So again, not much for an automated “RefactoringBrowser” to do.

Well. All this to, finally, say that I’ve been stuck refactoring some of my code – specifically, the XRay II file system browser back-end… and of course, no automation would have helped me. Nor would I have trusted it to do anything like what I want.

This is perhaps the third or fourth version of it, and it’s easily the most complex refactoring I’ve ever done. Unfortunately there are no intermediate steps. 20 days ago everything was compiling and running nicely (except, of course, for the problems that led me to this refactoring attempt). Then suddenly it’s like open-heart surgery. Nothing compiles and the number of error messages is so great that gcc throws its metaphorical hands up and goes off to sulk in a corner. I can’t close the patient up again until everything has been put back into place – even if it’s not the same place. And it’s a lot of information to hold in one’s head at the same time. I suppose I must get a second monitor, but that’s not practical at this moment.

And the availability of powerful time-sinks like the Internet means that it’s almost impossible to summon the necessary concentration to do the surgery in a single run. I’ve made serious progress over this weekend by the simple expedient of staying overnight with some friends who don’t have an Internet connection (or, even, a phone line). Still, sometimes it’s necessary to read and write e-mails, chat, even write long posts about refactoring…

Even so, I hope to get past this obstacle during the next few days and write a little about actual results. Turns out I learned a lot about in the process. More as soon as possible, then.

Cocoa

I was about to write a long, carefully reasoned post, explaining why Apple should not sell Mac OS X as a separate item for generic Intel boxes…

…when the CodePoet pre-empted me with even better arguments. So please go read “Should Apple sell Mac OS X for beige Boxes?“…

Well, who’d have thought it. Custom icons are now a security risk, and in fact always have been, since the first Macs came out!

Just look here:

It is now still possible for hackers to construct a file that appears to be a safe file type, such as an image or movie, but is actually an application, they said…

Apple acknowledged that, despite its patch, it is still possible to make a malicious file look innocent.

and here:

However, the new “download validation” – which warns users that the file may be malicious – does not completely solve the widely touted, ‘extremely critical’ Mac OS X zero-day exploit that allows hackers to disguise malicous [sic] files as routine files, thus allowing Safari browser or other internet application to automatically unpack and execute the file.

and here:

The unresolved vulnerability is due to a problem with the Mac OS Finder, the component of the operating system used to view and organize files, […] said. The operating system assigns an identifying image, or icon, for a file based on the file extension. However, it decides which application will handle the file based on information that is stored separately from the file, called metadata…

…the problem has nagged Apple for years, yet it has not been fixed. “This vulnerability derives from the exact same flaw deep inside the OS that should have been addressed by Apple several times in the past two years.”

Yes, all this refers to custom icons, a facility introduced by the Finder since its very early releases, and which was much used (many would say over-used) in the Classic days to “customize” one’s system.

Now, it seems, custom icons have gone from being an innovation to an “extremely critical zero-day exploit”.

Also, apparently, this unsettling misfeature has appeared only “in the last two years” instead of being in place since, at least, 1990, and Apple should have done something about it as soon as it appeared – a hacker’s conspiracy, no doubt.

The last quote above (for the quoted expert’s sake, I hope he was misquoted by a clueless journalist) also repeats the widely held myth that Mac OS X only considers the file extension for obvious functions and that other, more obscure, functions are affected by the mysterious “metadata” (or sometimes it’s the other way around?), which nevertheless were wrongly implemented by the clueless newbie programmers that apparently are entrusted by Apple to write the OS’s most critical layers. Hah. By implication, it also asserts another myth, namely that custom icons – which are a type of “resources” – are stored in those eldritch metadata. Double hah.

In fact, it goes like this. Files on Mac OS X, as they were on Mac OS Classic, can contain unstructured data (in what is called the “data fork”), as well as unstructured data (in what is called the “resource fork”) as well as metadata, which is to say, data describing the file or its contents (in what is called the “directory”). In contrast, most other operating systems have only unstructured data, so there’s no concept of separate “forks” for a file, and they also usually have less varieties of metadata. So this confusion often affects people not familiar with the details of the Mac’s file system.

So, when the Finder displays the icon for a file it doesn’t check only the file extension (if any). It also checks various metadata – some of which can even cause the extension to be ignored – to index into a database of existing applications, and from that loads the appropriate icon to display. However, this icon can be overridden for a particular file by assigning a custom icon to it, which is stored in the resource fork and whose presence is announced by a flag in the metadata.

Now, application and document icons are usually assigned to minimize confusion as to what that particular application or document does or contains. As soon as custom icons were invented, we could occasionally see custom icons designed to cause confusion instead – usually as a prank, sometimes to implement special functions, even more rarely maliciously.

Still, assigning a malicious icon only affects the file’s icon itself as displayed by the Finder’s icon view (does anybody still use that?). The Finder’s information panes still show the correct file type, and everybody should check that as a matter of routine before double-clicking on any icon in the Finder. (Does anybody still do that? Apparently yes. I offer Zingg! as a more secure alternative.)

Still, the sky is not falling. Falling prey to a custom icon disguising a malicious application as a warm, fuzzy well-known document would apparently affect people who install and run everything as “root”, or as an administrator, turn most warnings and safeguards off, then ignore the message that this application is being run for the first time, then blithely furnish their administrator password when solicited. No doubt there’s a surfeit of such people, but should they run a computer unsupervised..?

So, the suggested solutions range all the way from the not-very-useful but popular “Apple should get a clue” to the radical “disable all custom icons forever”. I think a more of middle-of-the road approach would be best. There should be a Finder preference to show custom icons or not, and perhaps applications with custom icons should have a little “app” badge in one corner. I suppose this would be easy to implement with some hackery. Pity I’m too busy right now…

Well, here I’m plugging away at XRay II and making reasonably good, if at times uneven, progress.

People ask now and then how I work. Just now I was reflecting on how, sometimes, reams of code get turned out on one day and then almost nothing on another day. Or how, sometimes, many parts are changed or refactored, while at other times updates are confined to a single routine or source file…

It may just be true for my style of programming, but I now see the process resembles very much the way a Mandelbrot Set is plotted. I start out with an empty project and iterate over everything repeatedly, adding a handful of code at a time, zooming in on as much details as is needed, then zooming out again to recheck the broad outlines, then zooming in again elsewhere.

I never manage to plan ahead how it will work, except for having a broad idea of what I want to do, and often I need to change direction radically at some point, either in fine or broad detail. Sometimes I need to take off for hours or days (or even months, as was the case for RBSplitView) to investigate a possible solution. I spent a couple of months learning about WebKit to format my information, only to run into trouble on some details and abandoning it again.

Then, of course, there are phases where it seems necessary to refactor some stuff that’s already done, either to make it smaller, more elegant, or just nice to look at – even if nobody else ever sees this code. It makes for slow progress sometimes, but the results are usually very satisfactory.

Posted by Rainer Brockerhoff (away):
We’re back in Buenos Aires, on the way home. All’s fine and I’ve lots of pictures to post later.

I’ve also made excellent progress working on XRay 2. Most of the infrastructure is now in place and part of the plug-in architecture too. More on that next weekend when I get back.

More comments on this by DrunkenBatman and John Siracusa; worth a read.

Apple Finder