Solipsism Gradient

I should have known better than returning to Theodore Gray‘s site when I had other things to do. This is a huge time sink; be careful!

Gray is co-founder of Wolfram Research, best known as the makers of Mathematica. He is the author of several books and is well-known for his amazing Periodic Table Table, for which he won the 2002 IgNobel Prize in Chemistry. As an old-time fan of the periodic table of elements, I was enchanted with the photos and stories about each element. I carried a small periodic table in my pocket all through secondary school and had it pretty well memorized at one time; before discovering computers I even considered working in chemistry.

There are movies and even sound samples, as well as interesting stories such as this visit from Oliver Sacks, one of my favorite authors. Highly recommended; make sure you’re prepared to stay glued to your computer for several hours.

I’ve just posted a new installment of my Interesting Times column, after quite a hiatus. This time, I’m commenting on the recent MP3Concept Trojan flap. Here are some highlights:

…Let’s see what we have so far:

– You can name an application anything and have it show any icon you wish. Always could.

– You can code an application to do anything, even harmful or deceitful things.

– The Finder will always show if it’s an application – but it can’t protect you from misleading icons.

– If you download such an application over a browser, file types, resource forks, or folder structures won’t be properly preserved unless it’s in an encoded form such as .bin, .zip, .sit or .dmg.

– If you get such an application as an e-mail attachment, most attachment formats will preserve resource fork and file type, but most e-mail clients will present an alert when you double-click on an application that came in as an attachment.

– Nevertheless, if there’s a deceitful document icon and an enticing name the user may double-click on the application even if there’s no extension at all.

…

What can the user do, then? Standard cautions apply. Don’t download applications from unknown sources. Check data files in the Finder to see if the “kind” field says “Application”. Don’t trust file icons on downloaded files. Don’t double-click on files you’ve downloaded; either drag the file onto the application you want to use it with or use the “Open With” contextual menu (or my Zingg! contextual menu to open it in a specific application. Don’t trust antivirus programs either, as they will either give you too many false positives or a false feeling of security. Disable automatic unstuffing, unzipping or running of downloaded items.

…What could Apple do? Options are limited here. Recall that this is not a Finder or Launch Services bug, so there’s no obvious fix.

…Summing up, a “social engineering” trojan application is not a new concept, but now that attention has been called to it, we may begin to see malicious implementations of it, more’s the pity.

Read the whole article if you want more details…

As most readers will have noticed, the Mac press for the last few days was full of scary and conflicting news about a new “MP3Concept” virus or trojan.

It started on April 8 when Intego, a company that sells several security and antivirus programs, announced “Protection Against the First Mac OS X Trojan Horse”:

This Trojan horse, MP3Concept (MP3Virus.Gen), exploits a weakness in Mac OS X where applications can appear to be other types of files.

The Trojan horse’s code is encapsulated in the ID3 tag of an MP3 (digital music) file. This code is in reality a hidden application that can run on any Macintosh computer running Mac OS X.

Mac OS X displays the icon of the MP3 file, with an .mp3 extension, rather than showing the file as an application, leading users to believe that they can double-click the file to listen to it. But double clicking the file launches the hidden code, which can damage or delete files on computers running Mac OS X, then iTunes to play the music contained in the file, to make users think that it is really an MP3 file.

Apparently they’re referring to this March 20 post by Bo Lindbergh, where he presents his proof of concept virus.mp3 application:

Download and unstuff to get a Carbon/CFM application that runs on both Mac OS 9 and Mac OS X. The PowerPC code fragment is stored as a general encapsulated object inside the ID3 information. It tries to locate iTunes and tell it to open the file as audio, displays an alert, and then quits. (If you’re paranoid, create a throwaway account to test it with.)

…this was in response to a thread where people were speculating on the feasibility of writing a “social engineering” virus as recently seen on other platforms.

Intego’s press release misrepresented some aspects of the issue; in particular there’s no specific “weakness in Mac OS X where applications can appear to be other types of files”: it has always been possible on Mac OS (since System 1.0 in 1984!) to write an application, put a document icon on it, and name it in a way that it appears to be a document. (More about that later on.) However, many people understood that to mean that this exploits a bug in Mac OS X. Even such a usually well-informed site as MacFixit fell into this trap, reporting that:

…the OS X Finder can be fooled to represent an application as a file. This occurs because the Finder depends on two different sources of information for how to display an object: it uses both file type/creator codes, for compatibility with OS 9; and also can use .extensions, introduced in OS X.

In this instance, the file type is set to “application”, and the .extension is set to “file”.

This “exploit” is only one step further from simply renaming an application file with a different extension, “.mp3” for instance. The only difference is that Mac OS X can be fooled into actually launching an application based on the false file type/creator codes and .extensions – iTunes for a file with a .mp3 extension for instance.

This is quite wrong. The Finder is not fooled at all and correctly identifies the file as an application; this can be seen in both column and list view, and in the “Get Info” window. Only in icon view (does anyone use this anymore?) can the user be fooled.

In reality, the ‘APPL’ file type takes precedence over the extension, so double-clicking it launches it as an application. The MP3 icon is not generated by the Finder in response to the .mp3 extension; rather, it’s contained inside the file itself as a normal application icon – which happens to be identical to the iTunes document icon. After the application launches, it in turn launches iTunes and passes itself to iTunes. This makes use of the fact that an MP3 file consists of a sequence of data blocks, each one identified by a tag. For this particular type of application, the executable code can start anywhere inside the file’s data fork which can contain unused space both before and after the code; so from the standpoint of iTunes reading the file’s data fork, the code is hidden inside an unused part of the identification (ID3) tag’s data block.

It’s easy to check this out. Save some file (from inside TextEdit, for instance). Open it in my XRay utility, change its name to something.mp3, set its file type to ‘APPL’ and clear the creator code. After saving those changes, it will appear in the Finder as a generic application, not as an MP3 file. Double-clicking it will give the usual icon zoom-out effect but nothing will happen – and iTunes won’t be notified.

Yes, XRay erroneously will say that this file, despite being an application, will be opened by iTunes; it won’t. That would be true only if it were a Classic application, in which case it wouldn’t be run at all. I’ll have this fixed in the next version…

The day after, Intego had a new press release out, where they attempted to explain some details:

First of all, Mac OS X runs two types of applications: Cocoa and Carbon. Cocoa applications are native OS X applications, and have an .app extension. Cocoa applications are, in fact, folders containing all the bits and pieces of a program code, resources, graphics, etc. The .app extension tells Mac OS X that an application is going to run natively.

Carbon applications are different. Most Carbon programs can run under either Mac OS 9 or Mac OS X, and, for this reason, have no .app extension. The Mac OS knows they are executable programs because of two resources, carb and cfrg. The carb resource indicates that it is a Carbon application and the cfrg resource indicates the location of executable code in a file’s data fork.

This Trojan horse is, in reality, an MP3 file, to which the two resources mentioned above (carb and cfrg) have been added. In addition, the ID3 tag of the MP3 file contains the actual code of the Trojan horse and the cfrg resource contains a pointer to that location in the file’s data fork.

Not at all. Firstly, Carbon applications, too, can be bundled and have the .app extension; just look at any recent Adobe or Macromedia program for an example. Single-file applications are recognized first by the ‘APPL’ file type and then further defined by several resources in the resource fork.

Single-file applications can be in several formats. Legacy 68K Classic applications (like ResEdit) have ‘CODE’ resources where the executable code is; the data fork can be empty or can contain anything at all. In the old days, it would have been easy to write an application that had a deceptive name and icon, and valid data for some document format in its data fork. As soon as Apple Events came out, convincing another application to open the data fork would have been easy too – except that most applications in those days would also check the file type and reject any ‘APPL’.

When the PowerPC came out, ‘CODE’ resources were deprecated and code was moved into the data fork; it used a new format called PEF (Preferred Executable Format). PowerPC code could come in several “code fragments”, so they were indexed from a ‘cfrg’ resource – that’s why these applications are also called CFM (Code Fragment Manager) apps. This resource is a list of code fragments and for each one it contains (among other data) the length of each fragment and its displacement within the data fork. Of course, normally there’s only one fragment which starts at zero. So, the programmer is perfectly free to put other stuff into the data fork as long as it doesn’t interfere with the parts that contain executable code. With a tagged data format such as MP3, TIFF, GIF, QuickTime (and so on…) that’s easy to do; just include the code into a tag data block that’s unused, or invalid, or reserved for comments. I used this technique in a dictionary application I wrote some years ago; the database is embedded into an unused part of the data fork.

When Mac OS X came out, the old Classic programming interfaces were redone into what became known as Carbon. To distinguish Carbon applications that could run natively on Mac OS X from the Classic apps, the ‘carb’ resource was defined. When it is present, the application is known to use the Carbon interface.

The native Mac OS X executable format, for a variety of reasons, is not PEF but Mach-O. Although most Mach-O applications are of the bundled variety, you can also write single-file Mach-O apps (AppleScript Studio can generate such applications). These are very similar to the CFM apps described above, except that there is no ‘carb’ resource and the code is indexed by a ‘CFRg’ resource instead. By the way, several people are proposing fiddling with the “executable” permission bits on downloaded files. That won’t work; Mach-O applications need them to be set, and they’re just ignored on CFM applications.

Coming back to the Intego press release, I think they have it the wrong way around. The sample application is a normal CFM app with several resources (not just ‘cfrg’ and ‘carb’). The writer added MP3 data blocks to its data fork and shifted the displacement in the ‘cfrg’ resource accordingly.

Now let’s look at the bundled application format. It’s actually a folder with the .app extension (which is always hidden by the Finder) and a certain folder structure. The actual executable can be PEF or Mach-O – it can even be a dual-fork file – but it’s hidden a few levels down in the folder, and other subfolders can contain anything whatsoever. So, it would be easy to write a Cocoa application called whatever.mp3.app (the .app would be hidden), define the application’s icon to be an MP3 document icon, and the application could then pass an actual MP3 file to iTunes when it executes.

All those formats – except for the bundled format – depend on the resource forks and file types to work. So usually, unless you download a deceitful application in .bin, .sit or .dmg format, the resource fork goes away and you’re left with a simple data fork.

So. Let’s see what we have so far:

• You can name an application anything and have it show any icon you wish. Always could.
• You can code an application to do anything, even harmful or deceitful things.
• The Finder will always show if it’s an application – but it can’t protect you from misleading icons.
• If you download such an application over a browser, file types, resource forks, or folder structures won’t be properly preserved unless it’s in an encoded form such as .bin, .zip, .sit or .dmg.
• If you get such an application as an e-mail attachment, most attachment formats will preserve resource fork and file type, but most e-mail clients will present an alert when you double-click on an application that came in as an attachment.
• Nevertheless, if there’s a deceitful document icon and an enticing name the user may double-click on the application even if there’s no extension at all.

In sum, any application can be a Trojan, but the user has to be careless to be affected. Now let’s look at what can be done. Apple has published a cautious statement saying that they’re aware of the issue and “investigating”. Intego, Symantec and other antivirus makers have announced that their latest upgrades detect that particular application. I suppose they detect non-zero offsets in the ‘cfrg’ resource and emit a warning; however, this will of course catch many legitimate applications (such as the dictionary I mentioned earlier). But as we’ve seen, even a Cocoa application can be deceitful.

What can the user do, then? Standard cautions apply. Don’t download applications from unknown sources. Check data files in the Finder to see if the “kind” field says “Application”. Don’t trust file icons on downloaded files. Don’t double-click on files you’ve downloaded; either drag the file onto the application you want to use it with or use the “Open With” contextual menu (or my Zingg! contextual menu) to open it in a specific application. Don’t trust antivirus programs either, as they will either give you too many false positives or a false feeling of security. Disable automatic unstuffing, unzipping or running of downloaded items.

What could Apple do? Options are limited here. Recall that this is not a Finder or Launch Services bug, so there’s no obvious fix. I see the following possibilities:

1) Dave Schroeder says on MacNN:

The only way for Apple to “fix” this would be to universally visually identify executable applications in some fashion.

I’ve seen this suggestion somewhere else too, but couldn’t recall the URL. Mac OS X could overlay a badge over any application icon, like it already does with aliases and locked files. The downside is that this would be visible only in the Finder and not be easily visible in all but the icon view… and there adding an “(application)” text tag might be more appropriate. And overlaying a huge red “caution” badge would probably be met with howls of protest from icon designers .

2) Apple (or someone) could patch LaunchServices to throw up a warning when a previously unregistered application is executed. This could be general, suppressed for applications in the standard /Applications folders, or even be shown only for certain folders… perhaps the user could mark certain folders (such as the default download folder) as not allowing applications to be run. This would probably catch most sneaky applications, at the cost of some extra work for the user. A ready-made alternative would be to extend the “Limitations” tab in the System Preferences to administrators.

3) Apple (or someone) could patch LaunchServices to check the application’s extension and see if another app is already claiming to handle this extension. This is already done for Classic applications, mainly for the special case of Classic .smi applications, which were self-mounting disk images but now are handled as files by the disk mounting utility. They would have to ignore a hidden .app extension if present and do some other extra checking. This might slow application launching down a little and would not, of course, catch Trojans that have a document icon and no extension.

On the other hand, most proposals I’ve so far read on the net are impractical, such as:

• Turn off executable permission flags by default (with umask or some other way): wouldn’t affect CFM Trojans.
• Deprecate CFM applications completely: will kill Adobe and Macromedia (and hundreds of others) and won’t affect Mach-O Trojans.
• Disallow extensions in application names: will kill compatibility with many applications who put version numbers in the name, and won’t affect extensionless Trojans.
• Remove the resource fork of suspicious files: will work against single-file Trojans but not against bundled ones.
• Wait for antivirus software to catch up: unrealistic, as they usually scan for certain binary patterns, and there’s no general pattern to Trojans. They can catch specific programs after the fact, but not pre-emptively.

Summing up, a “social engineering” trojan application is not a new concept, but now that attention has been called to it, we may begin to see malicious implementations of it, more’s the pity.

Still more comments on the Ronco Spray-On Usability debate, itself (just to remind people) sparked by this ESR article.

Matt Gemmell, the Irate Scotsman, weighs in with a long and reasoned article:

UI must be designed from the start. Interface considerations infest your design choices, like it or not. Functionality can rarely be completely divorced from the controls which will trigger and modify it. Output must often lead to feedback for the user. The core modes of the application entirely determine the user’s type of experience. Like it or not, we have to realise that as much as it might offend our software engineering sensibilities, for the user the UI is the software.

…Hopefully it’s clear that interface and interaction design is a fine art, and is incredibly difficult to do well. It requires a huge amount of knowledge of your software’s platform, purpose, target market, and indeed of human psychology both in general and in the specific field of human-computer interaction. Certainly, it’s not for the faint of heart, and nor is it to be rushed or attempted without due experience and consideration.

Likewise, Rui Carmo at The Tao of Mac also writes at length:

…usability is a lot of hard work spent on ironing out even the smallest wrinkle. It is not something you “spray on” after the fact. It is either part of the design from the outset, or it isn’t. No amount of paint will disguise hard, unforgiving edges.

Usability has no tolerance for hard edges – the only kind of edge users notice is the bleeding edge, and guess what, they don’t like it.

Both are extremely recommended reading for interface designers.

Stan Kelly-Bootle (who posted a comment here some days ago) has a new Son of Devil’s Advocate column out – it links back to this humble weblog and threatens to do so again next month!

Stan also reprints, as usual, his “Devil’s Advocate” column 16 years ago in Unix Review magazine, where he attacks one of my pet peeves – the harmful punctuation-within-quotes convention:

And so it comes about that the author’s

If X$ is set to “x”, the string Y$ is set to “end”.

is usually mispublished as:

If X$ is set to “x,” the string Y$ is set to “end.”

This, of course, is “no big deal,” until the billion-dollar space probe reaches the wrong galaxy. (Wasn’t that a misplaced Fortran colon?…Ed.)

I can’t understand why this nonsensical and potentially harmful convention still survives…

I seize the occasion to plug, again, Stan’s magnum opus (or at least my favorite of his several opuses? opi? opae?), The Computer Contradictionary, 2nd Edition, which as usual is on my desk as I write this. Now… where’s the much-overdue 3rd Edition…?

…If your mission in life is not already to
preserve the English tongue, it should be…

Fellow grammar god Bill Poser, over at the Language Log, takes issue with this paragraph:

…The author of this quiz probably has good intentions, but I’m afraid that my mission in life is not “to preserve the English tongue”. For one thing, it doesn’t need saving. English is spoken natively by hundreds of millions of people and as a second language my hundreds of millions more. I’ll devote my efforts to languages that are actually endangered, such as the native languages of British Columbia.

Of course, the author may be concerned with the preservation of the English language in another sense. He or she probably has the idea that to the extent that prescriptive rules are not followed, the language is somehow deteriorating…

But this is something that happens to dying languages, not languages like English that are growing. By any real measure of linguistic vitality or expressiveness, it makes not the slightest difference whether you use will or shall according to prescriptive rules or how you spell the possessive form of words ending in <s>. It is silly to think that English is in need of preservation because people don’t follow arbitrary rules.

He also comments on several other aspects. If you took the test, you should read the full article!

You are a GRAMMAR GOD!

If your mission in life is not already to
preserve the English tongue, it should be.
Congratulations and thank you!

How grammatically sound are you?
brought to you by Quizilla

Thanks to Burning Bird for the link. This certainly wasn’t an easy test for me as I’ve had nearly no formal English grammar training; I did it mostly by the “looks right” method. I only wish they would post the right answers…

John “Daring Fireball” Gruber has posted an update on his previous Ronco Spray-On Usability article, responding to e-mails and comments:

…Perhaps the biggest misconception is that I’m somehow “rooting against” desktop Linux. I really don’t see how anything I’ve written implies that, unless you subscribe to the “if you’re not with us, you’re against us” school of thought.

Regardless, it’s not the case. I’m not rooting against desktop Linux, nor have I ever claimed it can’t succeed. What I am saying is:

1) It hasn’t succeeded yet.

2) It’s unlikely to succeed without direction and substantial commercial support.

Where by “succeed” I mean “provide a terrific user experience”.

…

The key is that there’s never going to be a good desktop user interface for Linux that pleases the Linux nerds who don’t care about usability. If the reason you use Linux is that you value tweakability over usability, or if you get off on the fact that a normal person couldn’t sit down in front of your computer and figure out how to use it, you’re probably not going to like a system that doesn’t even have a replaceable “window manager”. Trying to create a cohesive GUI system that appeals to these guys is like trying to write music that appeals to the tone deaf.