Solipsism Gradient

When I began programming many years ago (now 55 and counting!), computing was in its infancy. We wrote programs on blue coding sheets, had them converted into decks of punchcards, and queued them on a shelf for “batch processing”. Usually the reward was a program listing with some obscure error messages like IEC107D, and I would mentally step through the program, repeating the process until being rewarded with a working “run”. I soon found employment at the local university, where more often than not I could run my program deck through myself and figure out things faster — and later on, in the wee hours, even use the huge mainframe computer as a primitive but enthralling personal device.

After some years the first video terminals came out, where you could view all lines of the program in glowing green rows, edit them directly, and enter the program into the batch queue. Was this the future? Well, there still was much to come. Very soon I realized I could buy one of those new Apple IIe computers along with a small TV and program/debug in the comfort of my home. And, miraculously, people actually paid me to sit at a computer keyboard and program.

And then the future arrived. When I saw the reports of the first Macintosh — a full graphics screen and a mouse — I knew that was it! The point-and-click user interface was the best. No more command lines. No more moving a clunky cursor around with the keyboard. It was heaven! And it only got better: color, larger screens, huge amounts of RAM and storage, networking! And then the internet came in, wonder of wonders.

It took years to evolve the expected standard behaviors of mouse gestures and UI conventions; application programmers had a library of items to use, so that pretty soon everything worked as expected and programs that flouted those conventions were downrated.

I was in the audience, in 2007, when Steve Jobs introed the first iPhone as a combination cellphone, iPod, and internet device. I wasn’t very impressed; I already had an iPod which I used little, had no plans of getting a cellphone (and indeed, held out until 2016 to buy one!), and the internet part was cool but the screen was small, the browser was limited and my laptop Mac had a much better feature set. OK, it wasn’t as portable but I always had it with me…

The real revolution here was the multitouch interface, an evolution of the point-and-click interface but with your fingers standing in for the mouse. It took years to evolve, and as with the Mac, Apple offered standard UI items to developers, which could then concentrate on functionality instead of reinventing the wheel.

But then in 2010 the first iPad came out and I bought one right away. Now here was a portable computer good enough to use as a personal device, and latter versions became more and more impressive. My iPad today is my main device for reading, listening to music and casual browsing, although I still fall back on the Mac for developing and writing longer texts. With my recent eye troubles I tend to fall back on my 77″ TV for watching movies, though, and programming has been curtailed.

Now the future has arrived again. I have no doubts that Apple’s Vision Pro — and, of course, “spatial computing” — is the Next Big New Thing.

Oddly enough, one of my arguments here is the sheer volume of vitriol about the device that one can find on the social networks: it’s unwieldy, it’s expensive, the external battery sucks, it’s been done already by others, it’s isolating and dystopian, it’s one more dragon in Apple’s evil ecosystem… Apple is doooomed, I tell you! Sound familiar? (And I’m not even on most of those networks!)

Hey, all those negatives were also rolled out in the past — and before we had Apple’s evil ecosystem, we had IBM’s evil ecosystem, remember? Every time such a futuristic device is sighted, it is clunky, expensive, power-hungry, and so forth. But also, if conditions are just right… the next version appears just in time, and it’s lighter, easier to use, and we can’t live without it anymore.

The real futuristic paradigm is now look-and-click, evolved from the old point-and-click way, and many other gestures are possible; standardisation is no doubt in progress. Why hold a mouse, or a control, or lift your hands unnecessarily, when you can convey all with small, subtle gestures? And our brains are evolved for a 3D environment — all our language and thinking is constructed around 3D metaphors.

Now, finally, we have a minimum viable system to explore 3D user interfaces. Discussing whether the Vision Pro is AR, VR or XR is besides the point; those are just implementation details and will evolve along with the UI.

Details on the hardware are scarce as I write this, and some may even be uninteresting — this thing is a self-contained computer on your face and the only relevant spec will probably be how much SSD space is left for user data. Everything else will be just good enough to be effectively transparent. And that’s the major point about the Vision Pro hardware: it’s a 3D input device for your brain, the first with sufficient quality to be transparent. Who needs holograms?

But, people will ask, what is the use case for this thing for the normal user/gamer/TV watcher/whatever? My answer: we don’t really know! Apple has, of course, selected some classic cases for the previous tech: FaceTime, games, 3D photos/videos, widescreen movies, Excel spreadsheets hanging in space (ugh), etc.; they had to do it, to get people’s attention. But between now and whenever this comes on the market in early 2024, developers will burn the midnight oil to build compelling use cases, most of which nobody (not even Apple!) had thought of before.

And this is where the Vision turns Pro. I believe that, rather than just designating a higher-capacity device compared to a non-Pro version, the Vision’s Pro name indicates that, at least until the 2nd or even 3rd generation comes out, this is a device for professionals. This is not (yet!) for casual gamers, zoomers or moviewatchers. Here’s a partial list I and a few friends came up with in a few minutes:

• Architects, engineers, designers (as usual)
• Doctors, dentists, psychologists, therapists and researchers in general
• Educators and grad/postgrad students
• Technicians in hitech fields like power generation, aircraft maintenance
• Industrial applications are boundless
• Astronomers, archaeologists, artists

And most of these can afford (or their companies can) the current $3499 price.

Com to think of it, I’ll probably buy two; it’ll still come in cheaper than paying for a huge 3D TV, multiple big screens/projectors and a couple of M2 Macs.

More as details emerge.

Just was rereading Cathy Shive’s blog (prompted by a tweet from Buzz Andersen), and found her post showing some classic logos by Paul Rand.

If you’re interested in that sort of thing, here are Mario Amaya‘s mash-ups of some of those logos (and others). Worth a visit.

Nando wrote:

Well this wouldn’t work on our actual system I guess, but if you prepared the system to have an ID or other metadata for each icon, I really think it could be done. But all I want for now is to keep my day job

Well, I sympathize with your last sentence .

Still, there’s no way to make an ID for each icon that reliably tells if some custom isual is visually similar to a document icon already on the system. At least with the current state-of-the-art of image recognition software.

Nando wrote:

The problem really lies within the file’s data fork, once you change that you can easily have the application open a file that is not attached to it, which might not be a succesful task.

I think you’re misinformed about what’s in the data fork; the contents of that have very little to do with file-application binding. It’s just unstructured data as far as the system is concerned.

Nando wrote:

One thing that could work is have the software check if the icon it’s file holds is not the icon for another application’s file. The system has a list of the icons for each file type, so a background check on that wouldn’t take long at all, probably not over 0.3 seconds…

Unfortunately, that’s impossible. The system would have to locate all the icons from the list (usually thousands), read them (they’re usually from 32K to 64K in size) and compare all pixels with that custom icon; even so, changing one single pixel would cause the comparison to fail. It would take several minutes and be completely unreliable…

Posted by Nando:
I agree with what you’ve said from a security and development point of view, the current methods the system uses to identify the application to which a file is registered seems quite secure to me though. I’ve decided to play with some of my files, changing the extension of a .mov file to .doc, and the system would recognize it as a Quicktime movie just fine. I did that with non-apple formats, and even a document kind I created for fun, and it would open on the right application even if saved with anotehr extension.

The problem really lies within the file’s data fork, once you change that you can easily have the application open a file that is not attached to it, which might not be a succesful task. But in the case of malicious files, the icon is what really makes the confusion, as you have said on a recent post.

Custom icons are easy to apply to files, you can do it easily from Finder itself, with special applications such as IconFactory’s Pixadex or Unsanity’s Shapeshifter. The second one, as far as I know, really does play with the data forks, changing the icons from inside out, while Pixadex just applies them as a cover that is easily removable.

On either case, killing custom icons is really not a solution. First of all because it would mean a dead end to those who, like me, make a living out of that. Ask the guys from IconFactory and MacThemes Forums, they’ll probably like their Macs pretty with custom themes and icons than a “secure” one – and we know this doesn’t actually threatens anyone’s security, yet.

One thing that could work is have the software check if the icon it’s file holds is not the icon for another application’s file. The system has a list of the icons for each file type, so a background check on that wouldn’t take long at all, probably not over 0.3 seconds. If the icon is unknown, the software would ask the system “Is this a themed OS?” (something that would be set on System Preferences against Apple’s will or through Third-Party software), if it is themed, carry on with the loading, if it is not, warn the user of the possible security threat.

As complicated as this solution might sound, I don’t see any other way to keep everyone happy – and secure. So let’s just hope in 5 years the Mac is still as secure as now, it’s probably the best we can do.

Forgot to say this, but the Unsanity folks had previously posted details on a different fake icon method. This uses a “strong binding” resource to change both the icon and the application used to open a document; when the document is a malicious Terminal script, the effect is much the same as hiding an application under a document icon.

In any event, the principle is the same: forcing the Finder to display an icon different from the one ordinarily defined by the normal characteristics of the file. Both methods subvert a useful feature which was implemented with some cost.

Well, who’d have thought it. Custom icons are now a security risk, and in fact always have been, since the first Macs came out!

Just look here:

It is now still possible for hackers to construct a file that appears to be a safe file type, such as an image or movie, but is actually an application, they said…

Apple acknowledged that, despite its patch, it is still possible to make a malicious file look innocent.

and here:

However, the new “download validation” – which warns users that the file may be malicious – does not completely solve the widely touted, ‘extremely critical’ Mac OS X zero-day exploit that allows hackers to disguise malicous [sic] files as routine files, thus allowing Safari browser or other internet application to automatically unpack and execute the file.

and here:

The unresolved vulnerability is due to a problem with the Mac OS Finder, the component of the operating system used to view and organize files, […] said. The operating system assigns an identifying image, or icon, for a file based on the file extension. However, it decides which application will handle the file based on information that is stored separately from the file, called metadata…

…the problem has nagged Apple for years, yet it has not been fixed. “This vulnerability derives from the exact same flaw deep inside the OS that should have been addressed by Apple several times in the past two years.”

Yes, all this refers to custom icons, a facility introduced by the Finder since its very early releases, and which was much used (many would say over-used) in the Classic days to “customize” one’s system.

Now, it seems, custom icons have gone from being an innovation to an “extremely critical zero-day exploit”.

Also, apparently, this unsettling misfeature has appeared only “in the last two years” instead of being in place since, at least, 1990, and Apple should have done something about it as soon as it appeared – a hacker’s conspiracy, no doubt.

The last quote above (for the quoted expert’s sake, I hope he was misquoted by a clueless journalist) also repeats the widely held myth that Mac OS X only considers the file extension for obvious functions and that other, more obscure, functions are affected by the mysterious “metadata” (or sometimes it’s the other way around?), which nevertheless were wrongly implemented by the clueless newbie programmers that apparently are entrusted by Apple to write the OS’s most critical layers. Hah. By implication, it also asserts another myth, namely that custom icons – which are a type of “resources” – are stored in those eldritch metadata. Double hah.

In fact, it goes like this. Files on Mac OS X, as they were on Mac OS Classic, can contain unstructured data (in what is called the “data fork”), as well as unstructured data (in what is called the “resource fork”) as well as metadata, which is to say, data describing the file or its contents (in what is called the “directory”). In contrast, most other operating systems have only unstructured data, so there’s no concept of separate “forks” for a file, and they also usually have less varieties of metadata. So this confusion often affects people not familiar with the details of the Mac’s file system.

So, when the Finder displays the icon for a file it doesn’t check only the file extension (if any). It also checks various metadata – some of which can even cause the extension to be ignored – to index into a database of existing applications, and from that loads the appropriate icon to display. However, this icon can be overridden for a particular file by assigning a custom icon to it, which is stored in the resource fork and whose presence is announced by a flag in the metadata.

Now, application and document icons are usually assigned to minimize confusion as to what that particular application or document does or contains. As soon as custom icons were invented, we could occasionally see custom icons designed to cause confusion instead – usually as a prank, sometimes to implement special functions, even more rarely maliciously.

Still, assigning a malicious icon only affects the file’s icon itself as displayed by the Finder’s icon view (does anybody still use that?). The Finder’s information panes still show the correct file type, and everybody should check that as a matter of routine before double-clicking on any icon in the Finder. (Does anybody still do that? Apparently yes. I offer Zingg! as a more secure alternative.)

Still, the sky is not falling. Falling prey to a custom icon disguising a malicious application as a warm, fuzzy well-known document would apparently affect people who install and run everything as “root”, or as an administrator, turn most warnings and safeguards off, then ignore the message that this application is being run for the first time, then blithely furnish their administrator password when solicited. No doubt there’s a surfeit of such people, but should they run a computer unsupervised..?

So, the suggested solutions range all the way from the not-very-useful but popular “Apple should get a clue” to the radical “disable all custom icons forever”. I think a more of middle-of-the road approach would be best. There should be a Finder preference to show custom icons or not, and perhaps applications with custom icons should have a little “app” badge in one corner. I suppose this would be easy to implement with some hackery. Pity I’m too busy right now…

Busy busy busy, learning the intricacies of WebKit… but even so I spent some time browsing through Kenneth Snelson‘s work, which I hadn’t heard of before. The atom section is especially entrancing. Warning: time sink!

Update: More info here.