As backing up is even less exciting than doing your tax return and certainly easier to forget, an application named SuperDuper! takes care of the chore daily and automatically. It’s the first back-up application for the Mac that speaks to you in English rather than Geek and works perfectly. Ever the Doubting Thomas, I check the ‘bootability’ of the external drive monthly and compare the files sizes on the two 250 gB Lacies weekly to see that they remain identical.

via Thomas Pindelski’s Photographs, Photographers and Photography. Thanks, Thomas!

One of the interesting things that happens when you do a “live” backup—that is, back up the volume that you’re booted from—is copy files that are currently open.

Normally, this isn’t a huge problem. But, on occasion, things go horribly w0rng. This is a story of one of those things that happens every so often with SuperDuper, and what we’ve done about it. It’s a mite technical, but hopefully interesting.

As you probably know, under OS X files have an owner, a group, and a “mode”. These three things work together to determine who can read, write, execute and do other things to the files on your system.

Less well known are the “flags” that are associated with the files as well. For example, the “immutable” flag—uchg—can be set to prevent a file from being modified, regardless of its “mode”. There are a number of flags that can be associated with files and SuperDuper needs to properly replicate them when copying.

So far so good.

Every so often, though, we get reports of users who have files on their destination volumes that can’t be removed during Smart Update. The error always looks like this:

| 06:02:18 PM | Info | Warning: error clearing immutable flags: 77600100 for: /Volumes/G5/private/tmp/sort9qlI6c
| 06:02:18 PM | Info | Error removing item: /Volumes/G5/private/tmp/sort9qlI6c of type: 100000, Operation not permitted
| 06:02:18 PM | Error | SDCopy: Error deleting /Volumes/G5/private/tmp/sort9qlI6c
| 06:02:18 PM | Error | : Operation not permitted

We found that we needed to remove a file on the destination. But when we tried to clear its immutable flag, we couldn’t. The only solution was to use Erase, then copy which—while it worked—was inconvenient.

We were recently finally able to reproduce the issue in house, and what we found was quite curious.

When a file is being written at the same time we’re backing it up, it’s possible for that file to look “weird” in the file system. When the above happens, it seems that the directory entry isn’t completely updated, and—in this unusual state—some flags are set that result in the above behavior. Here’s the above file’s directory entry, listed with “ls -lo” (which lists flags too):

---s-wS--- 1 nobody 3221220928 sappnd,arch,schg 524237 Dec 31 1969 sort9qlI6c

So, weird stuff. The group is complete garbage, and the sappnd and schg flags are set. And—worse—since the system returned that junk to us, we faithfully replicated it on the copy.

Why worse, you might ask? Because while you can set the sappnd and schg flags, you cannot reset them while booted normally. And, schg is the “system immutable” flag: it prevents the file from being deleted. Once you have this flag set, that file’s going nowhere.

The reason for this is that FreeBSD—the Unix that OS X is built on top of—has a concept of “security levels”. By default, our Macs boot up and run in “Secure level 1”, which prevents anyone from writing in certain folders like /dev/mem and /dev/kmem, even with escalated privileges (such as “root” access). And, in secure level 1, schg and sappnd flags cannot be removed.

And how can you get to securelevel 0 or -1, where they can? You have to boot into single user mode.

(You can learn some more about the various BSD security levels at this page.)

Yikes! What that means is once you’ve set these flags, you’ve basically got to go super-geek to unset them.

It’s clear we can’t ask our users to reboot into single user mode to fix this kind of thing, so we decided that the best thing is to detect the “garbage” case as best we could, and—in that situation—mask out the schg and sappnd flags when copying. The data we’re given by the OS is still wrong, but at least the copy won’t suffer for it, and the user won’t have to erase their backup next time around.

This fix, as well as some other stuff I’ll be talking about over the next few days (including native Intel support), will be coming in v2.1 of SuperDuper—keep an eye out for it!

Fraser Speirs on Nice Software (he was kind enough to include SuperDuper! in the list):

...I also think it’s much harder to impart that serious ‘weightiness’ in something that doesn’t really have a physical aspect beyond the CD it comes on. Actually, the product packaging can play a part in my attitude to software. If the box looks serious and has the right texture and weight, I’m already better disposed to it.

He’s absolutely right: it is harder to give a product “presence” these days, especially when you’re “just” selling licenses on the Web. With Shirt Pocket, I’ve tried to concentrate on usability and support—key differentiators in the software market.  I also try to make our products do one “pocket-sized” thing, and do it exceptionally well… and that means a lot of time spent polishing the software and the documentation.

These are all things I did in my previous companies, too, but we would also spend an enormous amount of effort—not to mention money—on the packaging of that content.

BRIEF, for example, had two enclosed Wire-O bound manuals, on paper of significant weight, along with a tri-fold Quick Reference Card, all packaged in a nice slipcase. The graphics were printed using a high quality process, and sealed with a matte plastic film: we knew the documentation was going to be used, and we wanted to make sure it held up well, and looked good in an office. As I recall, this cost us a huge amount per-product. But we were able to charge $195 a seat, and wanted to deliver $195 of value to the user. And —whenever they picked it up—our users got a “nice” feeling from the quality materials and presentation.

The same went for Track Record, although the size of the documentation made a slipcase impractical. Instead, we used the same kind of enclosed binding in a “candy box”, with a separate “How Do I?” guide that was designed with individually cut-back pages that acted as an “index”. Less expensive to produce, but still very “nice”. (Interestingly, when we were acquired, one of the first things Compuware/NuMega did—other than remove the space between “Track” and “Record”—was cut back on the quality of the materials… and raise the price.)

It’s something I miss, actually: the physical embodiment of the work we all put into our software. A giant box full of air and a CD just doesn’t do it for me…

"norrebo” blogs about his experiences backing up his PowerBook to an HFS+ volume that he also accesses from his Windows machines using Mediafour’s MacDrive:

The total cost for this adventure was $265.94 ($188 for the disk, $27.95, for SuperDuper!, and $49.99 for MacDrive). But knowing that I’ve got a bootable image of my Powerbook disk? Priceless.

Khoi Vinh, late of Behavior (now the Design Director for the New York Times) and the genius of grid-based designs (I wish my own site looked one-one-millionth as good as stuff he tosses into the trash), recently had a catastrophic hard drive failure (all too common, believe me):

I feel very fortunate, though; immediately after the first signs of trouble, I quit my procrastination and scheduled full backups of the hard drive on alternating nights to two different external FireWire drives — aided in no small part by the sheer awesomeness of Shirt Pocket Software’s invaluable SuperDuper! product. As a result, I lossed less than a day’s worth of work; not perfect, but it could have been loads worse.

SuperDuper! made it exceedingly easy for me to create a complete, bootable mirror of my hard drive, which actually allows me to continue to use the same system — with all of my files, preferences and software tweaks intact — with another Macintosh. I pulled my old Aluminum 15-inch PowerBook G4 out of retirement and booted it from one of the backup FireWire drives.

via Subtraction, Khoi’s blog. Exactly the idea—glad SuperDuper! could be of service!

Our “production” iMac hit the Shirt Pocket loading dock (aka my front step) yesterday, and I’ve been busily working on updates to netTunes and launchTunes while Bruce works on completing the next update to SuperDuper!, which looks like it will be Universal as well.†

It’s been nice to get back to Xcode after some time off (my design/marketing/documentation/support/etc duties keep me away from it more than I’d like), and it didn’t take more than a few hours to do the netTunes conversion—it’s up and running as I type, and works great.

The big challenge with netTunes was the screen “scraping”—the code has some byte-order dependencies because of pixels, and I wasn’t looking forward to re-writing the “inner loop” to handle all the various cases with Thousands and Millions of colors using both MMX and Altivec. Blegh.

Fortunately, there was an elegant solution. Since the Intel Mac is Tiger-only, I was able to make use of some the new Accelerate framework APIs to eliminate byte order dependencies and—at least under Tiger—about two pages of nasty code. Since the Intel Macs can’t run Panther, I didn’t have to change the already-working parts. The result? Both the Intel and PPC versions benefit, at least under 10.4.

It’s always a good feeling when you can delete code and leave the hardware-specific optimization up to the guys who live and breathe vector processing units, let me tell you.

Anyway, we’re working hard to get the Universal binaries out the door. Obviously, it’s important to do extensive testing using real, production hardware—and we’re doing just that.

† - Updated for the exclamation point haters. You know who you are.

You can boot off USB drives on Intel-based Macs. This opens the world of USB 2.0-only drives to Macs users as bootable backup devices. That sound? It’s David Nanian drooling.

from Tales from the Red Shed & Jonathan Rentzsch

Egad, was I that loud? >wipes chin<

Although I haven’t yet seen the full review, some UK Macworld readers have written in to tell me that there’s a full page review of SuperDuper in the latest issue. 4.5 stars, and this great quote:

There are just three groups of people who should be using this excellent utility. Lunatic frontiersmen with a penchant for early beta software, paranoid data hoarders who think that the next crash is just a restart away, and everybody else.

from Richard Dyce’s review in Macworld UK‘s February 2006 issue. Thanks, Richard & Macworld UK!

Hey, cool! It looks like SuperDuper! received a Gold MaxFixIt Toolbox Award this year!

A good backup strategy should be the base of any troubleshooting plan. This versatile disk copying program can make a straight copy, or “clone”—useful when you want to move all your data from one machine to another, or do a simple backup. The real power, however, lies in its ability “checkpoint” your system, preserving your computer’s critical applications and files while you run on a working, bootable copy.

It’s hard to believe another year is gone, but belief has nothing to do with it. A few hours and *poof*—goodbye, 2005.

It’s been a big year for Shirt Pocket, with a number of product releases, none bigger than v2.0 of SuperDuper!—something Bruce and I had been working on since the year before. And our customers seem to love it, which is gratifying indeed.

We were honored to receive another Eddy, this year for SuperDuper! 1.5—our second in a row, after 2004’s Eddy for netTunes. Incredibly cool to get that kind of recognition.

On a more personal level, I started blogging this year, about 5 years late to the party, but at least I’m here and enjoying it.

Zabeth is deep into her third year of Veterinary School, and transitioning to clinical studies early in 2006, which might mean I need to call her “Dr.” soon.

And Ketzl is still with us: happy, healthy (apart from the progressing DM), and the source of much joy and sadness.

No doubt next year will bring more changes, more releases, more joy, more sorrow.

Thanks to all of you for your support, encouragement, criticism and for spreading the word about Shirt Pocket and our products. Without you, the year would have been a lesser one: I’m sure I’ll be talking to more of you on the forums, through the support lines, and on the blog.

I’m looking forward to it. A toast: to the end of 2005, and to you!