Solipsism Gradient

Rainer Brockerhoff’s blog

Browsing Posts published by Rainer Brockerhoff

Hiatus: progress!

1 comment

I know you’ve all been waiting for an update on my eye surgery
…not really? Well, here’s one anyway.

Fast recap: here’s how things looked from my right eye —  it’s just slightly exaggerated. It’s called a macular pucker, and reading Spalding Gray‘s monologue about it wasn’t very comforting. The darkened, twisty part was unfortunately almost right in the middle, so reading with that eye was impossible, and the whole 3D experience was unreliable.

So, 3 weeks ago I had that somewhat scary vitrectomy operation and I’m happy to report that the surgeon said that, technically, everything went well — no infection, no visible scarring on the retina, no harm to other parts of the eye. Whew. In the meantime, I still have two weeks of antibiotic and steroid eyedrops every few hours; a small price to pay, of course!

Subjectively, the darkened blob is almost gone and the twisty parts are very gradually spreading apart and getting smaller; there remain two gray curved marks around the center, only noticeable when looking at a uniform background. Everything should stabilise in about a year and, with some luck, my eyeglasses should need no changing.

In another week I should be ready to resume my (relatively) new favorite activity: serious table tennis. I’ve started training regularly again two years ago, after about 40 years off the tables, and will write regularly about the topic here.

Warning: long, technical post; no TL;DR.

As I’ve said a few days ago, and then explained why:

My software development activities are (now, officially and indefinitely) on hiatus.

Most of my software projects already were marked as “legacy”, and RB App Quarantine had insignificant download counts, but RB App Checker Lite continued to be reasonably popular despite the lack of updates and an increasing number of crashes after macOS 10.12 came out.

In the past years, as I was beginning to consider the possibility of making this official, I replied to a few dozen bug/crash report emails mentioning that I might have to do so. (Of course, it never crashes here on my own machines… 🤔) Almost everybody replied with polite wishes but also mentioned that I should publish it as open source. Unfortunately, that is impossible; see below.

My idea with the RB Utilities (including several that never were published, one of which was the first one!) was to have a common foundation for all sorts of apps that examined, or did things to, files, folders and the file system in general. Some of you old-timers may remember my very well-reviewed XRay utility, which was not only my first Cocoa app, but also brought reasonable financial returns.

I wrote the core of XRay for MacOS 10.1 during an all-nighter session at MacHack 2001, so unsurprisingly XRay proved to contain serious design flaws and did not update well as file system APIs evolved. For some time I worked on a rewrite to be called XRay II, which made heavy use of plugins for most of its functions. But I was distracted by other work and in the end decided to shelve it.

Still, the idea of plugins proved too tempting, especially as Objective-C offered so many neat facilities for it, and in early 2011 I began writing a generic RB Utility app that would be specialised into a specific utility app by incorporating a single plugin, built right into the executable (rather than being a separate code/resources bundle elsewhere). The generic app would have both a Developer ID version and a Mac App Store version. It would do the heavy lifting, such as startup, integrity/signature/receipt checking, copy protection (for the devID version), show the About Box and generally do all the common work of showing windows, scanning folders and whatnot.

My usual work style is to write the basic stuff, run/debug it, then do increasing detail in sort of a fractal way. While writing the app framework I also wrote a very simple plugin that would be the basis of my first utility, RB File Counter. It would just scan a given folder at great speed and report the number of files and subfolders inside that.

As things progressed I was not very pleased with the complex (but interesting!) details of certificates, codesigning, requirements for the Mac App Store, and so forth; and I realised also that many developer buddies were having even more problems than I had. The logical thing was to halt work on replicating the old XRay functions like counting files, changing permissions etc., and first do an app that would help me (and others) to make sure our work was in shape. This became RB App Checker, with the “Lite” added on because I wanted to either do a “Pro” version or a general consumer version; both were to be paid-for, and there had to be some infrastructure built for that.

RB App Checker Lite came out in early 2012 after over a year of work and was well received; many developers sent in suggestions and bug reports, and I was also busy keeping things updated and making sure the app was both small and fast. When I went back to testing the other utilities, some infrastructure changes proved to be not generic enough and I had to review my initial design, of course.

In retrospect my main (and usual) mistake was to heavily over-engineer parts of the code in a matter that wasn’t future-proof. The “interesting” parts were heavily optimised, obfuscated, compressed and otherwise squeezed into hyperspace, while also trying to use all available CPU cores; and some extremely paranoid parts of the code were checking other parts of the code. In fact, I showed some of the source to a very senior Apple engineer at my last WWDC; he blinked and gabbled something like “ahem, well, this is most unexpected and I’ll be interested to know if it passes the App Store code review” while backing away slowly.

So, to come back to my original point, this means that the code as such isn’t fit to be published as open source. I may disinfect and publish a few of the saner parts later in a very controlled manner. If all goes well for my eyes this year,and now that Swift is nearing ABI stability, I’ll restart the app entirely in Swift — a good opportunity to relearn how to use GitHub and (ahem) unit testing.

Stay tuned for more news…

Hiatus: why?

1 comment

As I’ve said yesterday:

My software development activities are (now, officially and indefinitely) on hiatus.

The dictionary says:
hiatus | hīˈādəs | noun (plural hiatuses) [usually in singular] a pause or gap in a sequence, series, or process…

Let me tell you about the “why” first. Thanks to all of you who immediately asked if I’m OK, by the way.

Short answer: yes, I’m OK, though not as OK as in the past — but then, that’s to be expected, with all these anniversaries streaking by. Not helped at all by miscellaneous stresses originating in Real Life™. But: see below. 😳

Longer answer: in the past several years, my eyes have been getting progressively worse, to the point that reading on paper now works only with excellent illumination. OK, paper is not something I have to do on a daily basis nowadays, but reading on a screen also no longer works at the — admittedly, very small — font sizes I was used to. Jacking up the font size worked up to a point but my coding habits call for a huge amount of text visible at the same time. Stopping work altogether to wait for an improvement has, retrospectively, killed all my momentum. (I’m still waiting for that improvement, unfortunately.) Also: see below…

Medical answer: during a trip to Bhutan (highly recommended!) in 2014 I suffered a posterior vitreous detachment in my right eye; no doubt due to day-long van trips over their highways-in-progress. A couple of months later, a beginning retinal detachment was fixed by laser surgery, and everything seemed OK for a time. However, visual contrast worsened until, in 2016, I had to have cataract surgery on both eyes; this was when I stopped working altogether, waiting for my eyeglass prescriptions to stabilise. While that took longer than expected, I also found that there seemed to be no combination of monitor brightness/contrast that made working longer periods tolerable. To make things worse, the vitreous detachment seems to have left a scar on the right retina, causing a macular pucker; surgery on that (a vitrectomy, yikes!) is now scheduled to happen before the end of this year. My eye surgeon assures me that should take care of everything… digits crossed!

Needless to say, all this has been somewhat disheartening, and — as far as coding and blogging are concerned — I seem to have joined the Procrastination Monks for a few more years.

In the meantime, other things have happened, some quite positive! Stand by for more news…

LEGACY NOTICE

No comments

Today I have removed my remaining apps from the Mac App Store and posted official “Legacy” status notices on the product pages. My software development activities are (now, officially and indefinitely) on hiatus.

I know this notice has been long overdue, as I’ve been unable to update anything for more than 2 years. My apologies; I’ll try to post more about this here in the following weeks.

[Note for English-language readers: I describe a transient DNS poisoning attack against Banco do Brasil.]

Ontem, dia 4 de maio, em torno de 13:00 locais (17:00 GMT) entrei no site do Banco do Brasil. Na tela normal de login, tem-se que informar agência, conta e a senha de 8 dígitos. Feito isto, apareceu uma tela — inusitada! — solicitando, também, a senha de 6 dígitos!

Apesar da tela normal, com todos os logos etc. no lugar, desconfiei e olhei alguns dos links que saíam desta tela. Vários deles se dirigiam a um servidor, também, inusitado: ndninternetbbseguro.bb.com.br.  Fui verificar via whois:

whois ndninternetbbseguro.bb.com.br
...
domain:      bb.com.br
owner:       BANCO DO BRASIL S.A.
ownerid:     000.000.000/0001-91

ou seja, aparentemente normal. Porém, mirando este servidor eu vi:

PING ndninternetbbseguro.bb.com.br (46.166.173.180): 56 data bytes
64 bytes from 46.166.173.180: icmp_seq=0 ttl=48 time=1203.666 ms

um endereço que não me lembrava de ter visto para um site brasileiro. De fato:

whois 46.166.173.180
...
inetnum:        46.166.173.0 - 46.166.173.255
netname:        BALTICSERVERS-LT-DEDICATED
descr:          Dedicated servers
country:        LT
...
person:         Martynas Simkevicius
address:        Tilzes 74
address:        LT-76140 Siauliai

ou seja, um servidor localizado na Lituânia! Obviamente se tratava de um ataque “DNS Cache Poisoning” e isto, provavelmente, nos DNS da Vivo, meu provedor atual. Claro, não prossegui nas páginas nem tentei seguir os links; mas, com toda certeza, minha senha de 8 dígitos já tinha sido enviada ao impostor.

Vi outro relato possivelmente relacionado ao incidente nesta mensagem:
https://twitter.com/andreas_schutz/status/727143088582434817
“Your connection is not secure…” e, depois, um erro na configuração SSL/certificados do servidor www2.bancobrasil.com.br, que eu sabia ser um dos normalmente usados no internet banking do BB. Tentei acessar este servidor aqui, com o mesmo resultado.

relatos esporádicos de tais ataques contra o BB no passado, e muitos contra outros alvos.

Por precaução, fui imediatamente ao banco e alterei as minhas senhas — e, de fato, do terminal de lá conseguia acessar a conta normalmente. De volta, mudei para outro DNS e apaguei os caches; também, agora, tudo de volta ao normal aqui em casa.

You may have noticed the small lock icon next to the URL on this site? Well, this means that it’s now (almost) fully buzzword-compliant, completely served over the secure https protocol via a brand-new digital certificate! TL;DR: it’s more secure for both you and me; at least, it’s much better than it was before. But nobody’s perfect.

I know; some pages (perhaps, even, this blog post you’re reading) still haven’t adapted to our new secure overlords and may not show the lock icon, or even be completely broken — I’m working on that. In a week or two all should be fine. I’m using the invaluable https://www.whynopadlock.com/ checker for this, and already managed to get an A rating from the equally invaluable https://www.ssllabs.com/ssltest/ page. Credit to my hosting provider, DreamHost, for making this migration both possible and almost painless.

In other news, we’re just back from Yet Another Trip — this time to Patagonia and Antarctica. I’ve had plenty of time to work offline, meaning that things which are sometimes fascinating but often boring got done without excuses; I had to push off procrastination for weeks! In another week or two — or a month or two, if Zeno’s Paradox kicks in — a new RB Utility should be available. The app itself is 95% done, but as I plan to release it on both the Mac App Store and from this site, and it’s a paid app, all the tedious back-end stuff had to be learned and developed. Watch this space for news.

Update: I did some more tinkering with the security headers and I now also get a B rating from Yet Another Invaluable site: https://securityheaders.io/. (The A rating seems quite tricky, but I’m looking at the requirements.)

Update#2: more tinkering, and https://www.ssllabs.com/ssltest/ now says “A+”! Yay.

Boom: the Return

No comments

A few years ago I wrote a series of posts about Apple’s then-new Lightning connector for iOS devices:

No doubt you’re noticing a trend there… 🙂

Anyway, the recently-released iPad Pro seems to have the much-awaited USB3 capability on its Lightning connector. It does ship with a Lightning-to-USB2 cable, though, and USB3 capability isn’t mentioned in the tech specs.

The main objection to this actually happening is that Lightning, with its 8 pins, doesn’t have enough pins to support the standard USB 3 specification. This is, again, the old assumption that Lightning cables are “just… wires leading from one end to the other”.

To restate what I posted previously, if you actually look at the USB3 pinout, there are the two differential pairs which Lightning already has, and one additional pair for USB2 compatibility. So a legacy wire-to-wire USB3 cable would need 9 pins — but, remember, Lightning connectors don’t work that way!

In other words, if you plug in an old Lightning-to-USB2 cable into an iOS device, the cable itself already has to convert the two differential pairs to USB2’s single pair. So, no need to have the extra legacy pair on the Lightning connector itself — a future Lightning-to-USB3 cable will generate that as well, and use the two high-speed pairs when plugged into a USB3 peripheral. The current pinout is, therefore, quite sufficient.

A Tale of Two Certs

6 comments

I’m keeping this post updated as details develop…

About ten days ago, something strange happened on my Mac: I was debugging the next version of my RB App Checker Lite app and suddenly I saw the dreaded dialog box:Damaged

Completely abnormal, especially as I was debugging using the Developer ID version (not the Mac App Store version!) from inside Xcode. When I opened Terminal, the same dialog; when I opened Safari, same thing! No new process was allowed to run. Of course I had to reboot to be able to do anything, everything worked fine afterwards, and I couldn’t reproduce the problem, so…

OK, a couple of days ago I concluded all was ready and I uploaded my app for review. A few hours after I announced so on Twitter, the reports began appear: the sky is falling! Major Mac App Store meltdown, everybody was getting the “damaged” dialog, Apple’s certificates were the culprit. I started testing my local apps from the MAS and, sure enough, the MAS leaf cert had expired; no problems, some of them asked anew for the AppleID password, some didn’t. RB App Checker Lite showed the expiration but no other problems, but I pulled it from review just in case.

Two days of confusion and frantic coding later, I had submitted (and pulled!) 4 more builds until I was reasonably sure that everything was working correctly. Thanks to several fellow developers on Twitter, the upcoming version seems to show everything correctly; it turned out that my receipt checks were somewhat obsolete. I usually publish the direct download version only after the MAS version has passed review, but decided to release version 1.1.4, build 351 immediately: you can get it here. It has a long list of improvements and fixes.

Meanwhile, the consensus is that rebooting and re-entering the AppleID and passwords (or even deleting and reinstalling) the affected apps solves 99% of the problems.

There are actually several different unfortunate problems here. First, the “damaged” dialog seems to be caused by some sort of cache or memory corruption in the system processes that coordinate to implement GateKeeper and the app store updates; some reports say killing the “storeagentd” process solves this problem without rebooting. (My system doesn’t seem to run this, FWIW.) What not everyone knows is that this dialog appears before the app it allowed to run; that is, it’s not affected by any checking done inside the app itself!

Second, asking for a new AppleID password. This is caused by the app itself checking the store receipt; something strongly recommended by Apple, since otherwise, it’s easy to copy a downloaded app to another computer and having it run there; I remember some early games not doing this and being widely pirated.

When an app is downloaded from the MAS, a proper receipt for that AppleID and that computer is already inside. A missing or corrupted receipt is the only normal circumstance in which the “damaged” dialog should appear. But if you copy the app to another computer, this will be noticed by the app itself.

Once a MAS app starts up, the first thing it should do is to check the receipt. It’s a complex process and not everybody implements it the same way. At first, checking the receipt’s cert chain would cause the receipt to be rejected in the case of expiration; the app exits with a special numeric code (exit 173) and this code signals the system to put up the dialog asking to confirm the purchaser’s AppleID and password. This, in turn, will cause a new receipt to be downloaded, and the app can now run with no problems. Update: reports indicate that, in at least some cases, the system doesn’t respond properly to exit 173.

A few years ago receipts began to include a new field containing the receipt’s creation date, and developers now had to check the certs against that date (and not against the current date), therefore obviating the need to reenter the password. Unfortunately this was not widely divulged, and Apple’s own sample code hasn’t yet been updated accordingly; I confess to not seeing this myself!

As is usual in disasters, several things have to go wrong at the same time: some bug corrupts a critical system cache, certificates expire normally, some apps incorrectly test for expiration, receipts are corrupted or the AppleID validation servers become slow or unreachable (because of the huge number of simultaneous requests), and… boom.

Many articles, unfortunately, published factual errors or wrong assumptions.Let’s try to counter a few:

  • Apple “allowed” their Mac App Store certificate to expire. Wrong on several levels. First, there’s not one but 5 (!) certificates involved in any app from the store: Apple’s root certificate: and 4 others: two intermediate and two leaf certificates.
    The way these certs work is by so-called certificate chains; every cert vouches for the lower-level ones. At the top is Apple’s Root certificate, which is one of a hundred or so in the System Keychain. There are two different certificate chains in every MAS app; the first is used in the code signature:and the second is used to sign the store receipt:Note the expired certificate there? This is a leaf certificate. These, usually, have a short life — one or two years — and the intermediate certificates usually last a little longer.
    So, when a cert expires, is that a serious problem? No – unless it is the root cert, which is why they all expire somewhen in the 2030s — hopefully, by that time, they’ll have figured out something better, Apple will have updated the cert via Software Update, or the horse will have learned to sing.
    The root cert can be updated via Software Update because it’s stored in System Keychain — but it’s impractical to push cert updates to each and every signed app, bundle or library; there are many thousands of them! So an expired cert in the code signature doesn’t affect the app at all. What’s important is that the certs were valid when the app was signed. When and if you get a new version of the app, all certs will probably be new ones. So there’s no “allowing” a leaf cert to expire — they do so naturally.
  • Apple “pushed” a new certificate that expires in 2035. This is probably just looking in the wrong place — not knowing which certificate had expired, someone glanced at the root certificate and noticed the “new” 2035 date. Nothing new to see, of course; that cert was created in 2006! Even more confusingly, someone else deduced from that that Apple let their original root cert expire; also wrong.
  • The system hasn’t been updated to check SHA2 (256) certificates. Wrong; it’s true that older systems used a version of the OpenSSL library that understood only SHA1 (128) certs, but that actually means 10.5 or so. Newer systems understand SHA2, and in any event, since the MAS went up, Apple has always recommended developers to not use the system’s OpenSSL library (I think it’s not even included anymore), so only very old apps would be affected by that.
    Update: Glenn Fleishman has informed me about the SSL situation: there’s the new 1.0.x library branch and the older 0.9.x branch. Both apparently got SHA2 support in 2010, when 1.0.0 and 0.9.8o came out, but some developers seem to have kept older versions, no doubt for valid reasons; space precludes, etc.
  • Apple is blaming developers. Apparently this can be traced to a single report of misinformation from an anonymous Apple Support person. As I write this, Apple hasn’t yet said anything; I doubt they’ll say anything over the weekend.
  • This is a serious security/cryptography failure.  Nope. This confusion arises from the fact that digital certificates (and libraries like OpenSSL)  are used for both secure, encrypted communications and for app/receipt signing. In the latter case, an expired cert doesn’t expose any information or makes the system or apps easier to hack.
  • Developers are better off not doing any, or little, receipt checking. Not really. True, apps which don’t do full receipt checking might have not been affected in this single instance, but under usual circumstances they’re more vulnerable to hacking or piracy.
  • Apple’s store/system infrastructure is brittle and can’t be trusted. True, it’s a very complex system that depends on many twisted little interlocking parts to work properly. And, as we’ve seen, this particular instance of failure is as self-amplifying as electrical grid failures _ once it starts, the demands on the working parts grow so huge that those fail, too. In Apple’s defense, it’s very hard to test for or simulate. Let’s hope that all involved have learned something from this incident; I certainly have learned a lot.

Update: forgot to comment on this particular post:

But when I tried to convince my Mac to run this app as an unsigned app, I encountered what is extremely likely to be the store DRM: I initially got the “your app was bought on another machine” message, so I tried deleting the receipt, but then I got the dreaded “app damaged” message, at which point I removed the signature.

…the only way I can see is to create a new root CA which I install on the machine as a trusted root, and redo the signing chain, and even that might not work if the DRM is somehow tied to the signature chain.

While I can understand the frustration implicit in not being able to run purchased apps “forever”, I think this is a fundamentally wrong approach. Let’s educate developers to check receipts properly, as I mentioned above. Figuring out a way to run store apps (or even developer-ID purchased apps) without “DRM” means that anyone else can use the same method to install pirated copies; we wouldn’t be able to trust users anymore.

Much worse, re-signing someone else’s app and expecting it to run is an even greater violation of trust. The days when you could hack someone’s app with ResEdit and having fun making it look different, or do unexpected things, are long gone. I implement very strict checks that my complete app bundle has not been altered in any way and that it’s running with my original signature, otherwise any user could freely alter files, hack the code, change graphic resources or even — and such cases have happened! — repost the app somewhere else as being their own. No, flawed as the current approach may be in implementation, I see no better alternative.

Update: reports are in from some helpful fellow developers, confirming my suspicions of cache corruption — RB App Checker Lite says the app bundle and receipt contents are OK, yet the apps will not run. I use the same APIs (hopefully) that the system processes use — but those APIs can take a long time to run, so the results are cached somewhere.

Update: Apple sent email to all developers:

In anticipation of the expiration of the old Mac App Store certificate, we issued a new certificate in September.

As I said — no “let[ting] certificates expire”. They all do.

We are addressing this caching issue in an upcoming OS X update.

Confirms this is a caching issue, as I suspected.

…some apps are running receipt validation code using very old versions of OpenSSL that don’t support SHA-2. We addressed this by replacing the new SHA-2 certificate with a new SHA-1 certificate last Thursday night.

I’m a little surprised that the number of apps using“very old versions” justifies going back to SHA1; but, OK.

Please ensure your code adheres to the Receipt Validation Programming Guide and check that all receipt validation issues are resolved.

Good, but:

  • the link goes to the page detailing the online receipt validation. Very few apps use that IMHO — you have to be online every time the app runs, you have to have a reasonably fast connection, and app launch will be significantly slower. Linking to this page (Validating Receipts Locally) would’ve been better;
  • it would have been more helpful to call out specifically the certificate expiration check and update the sample code to properly use the receipt creation date.

Update: I’ve now submitted rdar:///23611335 — a bug report to call attention to this documentation problem.

Update: Fixed the “Validating Receipts Locally” link, which was also pointing to the wrong page. Sorry. Also, here’s one way to do the correct date checking (copied from Matt Steven’s code):

X509_STORE *store;
// set up the store
X509_VERIFY_PARAM *param = X509_VERIFY_PARAM_new();
X509_VERIFY_PARAM_set_time(param, time_from_receipt); // option 1: verify using a specific time
X509_STORE_set1_param(store, param);
X509_VERIFY_PARAM_free(param);
// call PKCS7_verify() using configured store

Photos licensed by Creative Commons license. Unless otherwise noted, content © 2002-2019 by Rainer Brockerhoff. Iravan child theme by Rainer Brockerhoff, based on Arjuna-X, a WordPress Theme by SRS Solutions. jQuery UI based on Aristo.