Solipsism Gradient

OK, so here are the details on remote debugging; I’ve finished this phase of XRay 2 development and will in the next few weeks be fully available for pressing on with it.

The basic idea is that I have only PowerPC Macs, and since nobody I know in Brazil has received an Intel Mac (except for a couple of DTKs, which I didn’t want to use), the solution was to use Xcode‘s remote debugging capabilites, running my executable at a machine in the ADC Compatibility Labs. These are open at no extra charge to paying developers, but most of what I’ll detail would apply to any other machine.

Most of it is explained in the Xcode User Guide. Note that I used Xcode 2.2.1, but I think this facility has been available at least since 2.0. Click on the “Remote Debugging in Xcode” section in the left frame. First, however, send e-mail to adc.labs(at)mail.apple.com and ask for machine time, explaining for how long you need the machine; I asked for 3 days (thanks, Joe Holmes!). Basically, they’ll set up a newly formatted Mac with everything standard installed, including the latest developer tools. You should check that you have the same version, I suppose. They’ll have ssh and Apple Remote Desktop activated, and will send you the IP address, usercode and password. For illustration, let’s say the IP number is 10.1.1.1 (NOT an actual IP!), the usercode is “adclabs” and the password is “stevejobs”; substitute the actual values as appropriate.

In other words, all you’ll do there will be inside the default home folder “adclabs”. This user is also an administrator, so you’ll be able to use the password whenever needed for that. If you have a second Mac handy, you could rehearse first with that, of course; it’s what I did, as I’m normally not that handy with the Terminal. (Thanks, by the way, to John C. Randolph, Mike Ash and several others for helping me with details.)

First step is to generate a public and private key pair; you can skip this if you already have done so in the past. Open Terminal and type:

ls ~/.ssh/

If it lists a few files, among them one called “id_rsa.pub”, you already have a key pair. If not, type:

ssh-keygen -b 2048 -t dsa

This will take about a minute and then prompt you for a file path; type <return> to use the default path. It will then prompt you for a passphrase, twice. Don’t leave this empty and don’t use too short a phrase. You now should have the id_rsa.pub file in the ~/.ssh directory.

Second step is to open Terminal and type:

ssh adclabs@10.1.1.1

wait for the Password: prompt and type in “stevejobs”, or whatever password they sent you. You should see the normal Terminal prompt now, with a name like “CE-Lab-ADC-Compatibility-Labs-Intel-iMac” at the start of the line.

Now you’d better change the password to the same passphrase you used for the RSA key – yes, usually it’s recommended to use different passwords here, but that way you won’t have to remember which one to use where; it’s just for a couple of days, anyway. Type

passwd

and you’ll be prompted for the original password, then twice for the new password. Create a .ssh directory with

mkdir ~/.ssh

and log out by typing

exit

Next step is to transfer the public key to the remote Mac. To do this, at your local prompt, type

scp ~/.ssh/id_rsa.pub adclabs@10.1.1.1:~/.ssh/authorized_keys

it will ask for your password again, and transfer the file over. Now log in again with:

ssh adclabs@10.1.1.1

and if all is well, it won’t ask for your passphrase or password again, but just log in. Now restrict permissions on your key by typing

chmod go-rwx ~/.ssh/authorized_keys

Now you need to set up a local build folder. The trick here is that both machines should see your build folder at the same absolute path. There are several ways to achieve that; on a local network, you could have one of the machines serve the entire folder to the other, then use symbolic links to map the same path. However, I found that over a long distance it’s most efficient to have mirrored folders at both machines, and copy the contents over with an extra build phase. Here’s what I did.

At the remote machine, type

mkdir ~/build

which will create an empty build folder in the Home folder. Log out and close Terminal.

Now, on your local machine, you need to prep Xcode for what you’ll do. Double-click on your main project group and go to the “General” tab. Click “Place Build Products In: Custom location” and type in “/Users/adclabs/build” as the location. (Supposing, of course, that you don’t have a user called “adclabs”…)

Also check “Place Intermediate Build Ïiles In: Default intermediates location”, which probably will already be checked. Now click on your target and, from the Project menu, select “New Run Script Build Phase”. Make sure the new build phase is the last one, and enter this line as the script:

rsync -rz /Users/adclabs/build adclabs@10.1.1.1:/Users/adclabs

Finally, double-click on your executable in Xcode, and in the “Debugging” tab, select “Use Pipe for standard input/output”, check “Debug executable remotely via SSH”, and in the “Connect to:” field, type

adclabs@10.1.1.1

Now you’re ready. You’ll notice a delay of a few minutes while the last build phase transfers the files over, and on the start of a debugging run there’ll be several errors logged to the debug console, but eventually you’ll be debugging and single-stepping as usual, albeit more slowly. For GUI debugging, of course, you’ll have to use Apple Remote Desktop; I wish Apple would include a 1-user license for this in the Select package, as it’s rather expensive…

Have fun! I’ve tried to double-check most of this as I typed it in, please tell me if something didn’t work.

Update: fixed a typing error.

Mac Xcode Debugging

Now and then I read complaints about Xcode on blogs and mailing lists. It’s come a long way but some parts are still slow and cumbersome, granted. One of the complaints – which usually comes from Java or Windows C++ migrants – is that Xcode has no refactoring aids. Some people even publish workarounds.

So what is this refactoring thing anyway? According to Wikipedia:

Refactoring is the process of rewriting a computer program or other material to improve its structure or readability, while explicitly keeping its meaning or behavior…

Refactoring does not fix bugs or add new functionality. Rather it is designed to improve the understandability of the code or change its structure and design, and remove dead code, to make it easier for human maintenance in the future. In particular, adding new behavior to a program might be difficult with the program’s given structure, so a developer might refactor it first to make it easy, and then add the new behavior.

I’d tend to agree with that, up to a point. I usually refactor when I reach a dead end in the software’s structure, that is, when the current structure won’t allow me to proceed implementing what I want to implement. Or – probably the same thing, essentially – when I find myself implementing things I don’t want to implement anymore.

But my tendency (see fractal programming) is to do it in the reverse order; I write some code that does new stuff in a new way. Then I migrate lots of old code into the new scheme, often rewriting it radically if necessary, or throwing entire blocks away. (Well, not literally at first; I prefer to comment such blocks out or move them into a “dead code” file for later reference.)

Now, the aforementioned migrants usually don’t see it that way. Rather, they want some automation to make the process easier:

An automated tool such as a SCIDs to help you do might work like this:

– I have a method which has some code that I would like to pull out into its own method.

– I highlight the offending code.

– I select Extract Method from a popup menu

– The RefactoringBrowser asks me to name the method and automatically creates it and inserts the highlighted code.

– In the current method, the highlighted code is replace by an invocation to the newly created method.

All very nice, but it presumes several things which I don’t see coming to Xcode (at least not to the Objective-C parts):

– You have a very regular, structured style of coding that conforms to standards the “RefactoringBrowser” understands.

– You always use the standard refactoring methods, such as expanding, collapsing, pulling out, pushing in, whatever.

– All source code in your project has been previously parsed and stored in the SCID (source code in database), so the browser and refactoring software have a perfect understanding of your code.

This is perfectly possible (or at least I’m told it is) in Java and perhaps in C++ – though I’m skeptical about the latter. I was astounded when a friend, who was qualifying to some Java certificate or other, asked me to have a look at his source code. A quite trivial program was expanded to several dozen source files, consisting of literally hundreds of small methods that differed from each other only by name and a few characters or line. No doubt everything was set up very logically and hierarchically and according to whatever standards a certified Java programmer must obey, but… it was completely illegible by my (admittedly eccentric) standards. It was code only a SCID could love.

So, suppose my friend decided to refactor his code. Just renaming a few classes must necessarily entail profound changes in all source and project files. Not only must the filenames themselves be changed, but all mentions of this class must also be changed. Wait, don’t we have global search & replace for that…?

But, of course, renaming classes or methods is trivial. Maybe it’s suddenly obvious that you need to push some methods off into a subclass, or pull them up into their superclass. Wouldn’t it be nice to have this done automatically?

Well, not really. First off, inheritance is much less used in Objective-C – or at least, I use it less than I used to do in my C++ days (I refuse to learn Java). Runtime binding, introspection and categories mean you usually don’t have to subclass more than one or two deep from the standard Cocoa classes. In fact, I believe I just today went to the third subclass level for the first time. So, automating such a superfluous process makes little sense.

Second, remember that Objective-C is just a small superset of C (unlike Java and C++ which are C-like languages). And that Mac OS X is Unix-based, with many headers pulled in from heterogenous sources. This means, of course, that all the old crufty things of the old C days are still there… pointers, pointer casting, weird #defines, other tricks – you name it. And all of this is liable to be #included into your poor unsuspecting code if you do any work at all with Carbon or BSD APIs, as most non-trivial applications need to.

In other words, I don’t believe it’s possible to feed all of this into a SCID and expect it to behave rationally; of course the gcc compiler has to make sense of all this eventually, but I seriously doubt it could be easily refactored to go back and forth from its internal representation to the source code while it’s being edited. It’s still, essentially, a batch processor.

Suppose someone pulled this transformation off. Suppose all the old C headers were tamed to make them compatible. Suppose we had everything in a hierarchical, intelligent, refactoring browser/editor. Now what?

It may be some congenital deficiency in my own neural wiring, but I can’t recall ever refactoring my code twice the same way (except for that trivial class/method renaming). So again, not much for an automated “RefactoringBrowser” to do.

Well. All this to, finally, say that I’ve been stuck refactoring some of my code – specifically, the XRay II file system browser back-end… and of course, no automation would have helped me. Nor would I have trusted it to do anything like what I want.

This is perhaps the third or fourth version of it, and it’s easily the most complex refactoring I’ve ever done. Unfortunately there are no intermediate steps. 20 days ago everything was compiling and running nicely (except, of course, for the problems that led me to this refactoring attempt). Then suddenly it’s like open-heart surgery. Nothing compiles and the number of error messages is so great that gcc throws its metaphorical hands up and goes off to sulk in a corner. I can’t close the patient up again until everything has been put back into place – even if it’s not the same place. And it’s a lot of information to hold in one’s head at the same time. I suppose I must get a second monitor, but that’s not practical at this moment.

And the availability of powerful time-sinks like the Internet means that it’s almost impossible to summon the necessary concentration to do the surgery in a single run. I’ve made serious progress over this weekend by the simple expedient of staying overnight with some friends who don’t have an Internet connection (or, even, a phone line). Still, sometimes it’s necessary to read and write e-mails, chat, even write long posts about refactoring…

Even so, I hope to get past this obstacle during the next few days and write a little about actual results. Turns out I learned a lot about in the process. More as soon as possible, then.

Cocoa

Well, here I’m plugging away at XRay II and making reasonably good, if at times uneven, progress.

People ask now and then how I work. Just now I was reflecting on how, sometimes, reams of code get turned out on one day and then almost nothing on another day. Or how, sometimes, many parts are changed or refactored, while at other times updates are confined to a single routine or source file…

It may just be true for my style of programming, but I now see the process resembles very much the way a Mandelbrot Set is plotted. I start out with an empty project and iterate over everything repeatedly, adding a handful of code at a time, zooming in on as much details as is needed, then zooming out again to recheck the broad outlines, then zooming in again elsewhere.

I never manage to plan ahead how it will work, except for having a broad idea of what I want to do, and often I need to change direction radically at some point, either in fine or broad detail. Sometimes I need to take off for hours or days (or even months, as was the case for RBSplitView) to investigate a possible solution. I spent a couple of months learning about WebKit to format my information, only to run into trouble on some details and abandoning it again.

Then, of course, there are phases where it seems necessary to refactor some stuff that’s already done, either to make it smaller, more elegant, or just nice to look at – even if nobody else ever sees this code. It makes for slow progress sometimes, but the results are usually very satisfactory.

Rafael Fischmann wrote:

Rainer, here’s your opportunity:

Google wants you (if you’re a Mac developer)

A new posting on Yahoo! hotjobs finds Google seeking the talents of a “Senior Macintosh Developer.”

I suppose you mean this job listing? Sounds interesting, but living in California’s not my idea of fun these days. I’ll look into it when I get back from the trip, perhaps.

A couple of days ago I managed to finish off most of my remaining offline problems and, since I’d seen news that the ADC Compatibility Labs now had the new iMacs for testing, applied for a couple of days at one of them.

No, I’m not going to Cupertino, but rather wanted to try out Xcode’s Remote Debugging.

As a result, I now can confidently say that XRay 2 will be fully compatible with the new Intel Macs. I’ll also be licensing the new Universal logo.

It’s rather late in the day, but tomorrow I’ll post technical details…

Mac Debugging

Nando wrote:

Rainer Brockerhoff wrote:

This of course disqualifies me…

Maybe it’s time you learn something ‘old’ and get one of those positions at Apple. I don’t know of anyone more prepared than you.

Thanks for your vote of confidence; 10 or perhaps even 5 years ago I’d have been interested. Now, I’m more interested in learning more about Cocoa. I suppose Steve Gehrman of Path Finder fame would give you the same answer, and he’s way more “prepared” for writing a new Finder than I am… now, if they’d specified Cocoa and telecommuting, that’d be another story.

Apple Finder

Hm, Apple’s searching for a Software Engineer to work on its next-generation Finder:

The Finder team is seeking an energetic, motivated software engineer to help develop next generation versions of the Finder, the notorious file browser for Mac OS X.

You will be responsible for developing new features of an application that is often perceived by our users as the “face of the system”.

But it seems the new Finder will still be a Carbon app, probably to the dismay of many:

Preferred experience:

5+ years of experience developing software.

Experienced in using STL, Boost.

Shipped shrink-wrapped products.

Knowlege of Core Graphics, HIView and Carbon, Core Foundation.

This of course disqualifies me; I’ve carefully avoided most of those so far. Hmmm…

Rainer Brockerhoff wrote:

…So we regret to inform you that we can’t make it this year, but will try again in 2006.

Well, it seems that the conference organizers have decided that – at least for now – there will be no ADHOC next year. Seems that attendance has fallen too much; a pity.

Update: more info and comments at DrunkenBlog.

Update 2: just saw this long and detailed account. A must read!

Hopefully some heavyweight sponsor will pick it up. Metrowerks, who used to be such a sponsor, have recently announced that they’re end-of-lifing CodeWarrior for Mac OS X. Also, they’ve sold their x86 system to Nokia – no one seems to really understand this – and so are out of the Mac-Intel migration anyway.