As backing up is even less exciting than doing your tax return and certainly easier to forget, an application named SuperDuper! takes care of the chore daily and automatically. It’s the first back-up application for the Mac that speaks to you in English rather than Geek and works perfectly. Ever the Doubting Thomas, I check the ‘bootability’ of the external drive monthly and compare the files sizes on the two 250 gB Lacies weekly to see that they remain identical.
One of the interesting things that happens when you do a “live” backup—that is, back up the volume that you’re booted from—is copy files that are currently open.
Normally, this isn’t a huge problem. But, on occasion, things go horribly w0rng. This is a story of one of those things that happens every so often with SuperDuper, and what we’ve done about it. It’s a mite technical, but hopefully interesting.
As you probably know, under OS X files have an owner, a group, and a “mode”. These three things work together to determine who can read, write, execute and do other things to the files on your system.
Less well known are the “flags” that are associated with the files as well. For example, the “immutable” flag—uchg—can be set to prevent a file from being modified, regardless of its “mode”. There are a number of flags that can be associated with files and SuperDuper needs to properly replicate them when copying.
So far so good.
Every so often, though, we get reports of users who have files on their destination volumes that can’t be removed during Smart Update. The error always looks like this:
| 06:02:18 PM | Info | Warning: error clearing immutable flags: 77600100 for: /Volumes/G5/private/tmp/sort9qlI6c
| 06:02:18 PM | Info | Error removing item: /Volumes/G5/private/tmp/sort9qlI6c of type: 100000, Operation not permitted
| 06:02:18 PM | Error | SDCopy: Error deleting /Volumes/G5/private/tmp/sort9qlI6c
| 06:02:18 PM | Error | : Operation not permitted
We found that we needed to remove a file on the destination. But when we tried to clear its immutable flag, we couldn’t. The only solution was to use Erase, then copy which—while it worked—was inconvenient.
We were recently finally able to reproduce the issue in house, and what we found was quite curious.
When a file is being written at the same time we’re backing it up, it’s possible for that file to look “weird” in the file system. When the above happens, it seems that the directory entry isn’t completely updated, and—in this unusual state—some flags are set that result in the above behavior. Here’s the above file’s directory entry, listed with “ls -lo” (which lists flags too):
---s-wS--- 1 nobody 3221220928 sappnd,arch,schg 524237 Dec 31 1969 sort9qlI6c
So, weird stuff. The group is complete garbage, and the sappnd and schg flags are set. And—worse—since the system returned that junk to us, we faithfully replicated it on the copy.
Why worse, you might ask? Because while you can set the sappnd and schg flags, you cannot reset them while booted normally. And, schg is the “system immutable” flag: it prevents the file from being deleted. Once you have this flag set, that file’s going nowhere.
The reason for this is that FreeBSD—the Unix that OS X is built on top of—has a concept of “security levels”. By default, our Macs boot up and run in “Secure level 1”, which prevents anyone from writing in certain folders like /dev/mem and /dev/kmem, even with escalated privileges (such as “root” access). And, in secure level 1, schg and sappnd flags cannot be removed.
And how can you get to securelevel 0 or -1, where they can? You have to boot into single user mode.
(You can learn some more about the various BSD security levels at this page.)
Yikes! What that means is once you’ve set these flags, you’ve basically got to go super-geek to unset them.
It’s clear we can’t ask our users to reboot into single user mode to fix this kind of thing, so we decided that the best thing is to detect the “garbage” case as best we could, and—in that situation—mask out the schg and sappnd flags when copying. The data we’re given by the OS is still wrong, but at least the copy won’t suffer for it, and the user won’t have to erase their backup next time around.
This fix, as well as some other stuff I’ll be talking about over the next few days (including native Intel support), will be coming in v2.1 of SuperDuper—keep an eye out for it!
More months pass, and I’m happy to say Ketzl’s still with us.
She’s still in good spirits, but slowing down as the disease continues to take its inevitable toll. Her front is weak enough that our walks require me to hold the wheelchair’s “yoke” to relieve pressure on her front; fortunately she’s not having placement issues. But, walks don’t hold the joy for her they once did: they’re hard, but necessary, work.
Her weekly physical therapy swimming, while still fun, is also pretty tiring for her. When once she did 30-40 laps, now she wants to stop after 10-15. And since exhausting a DM sufferer is a bad idea, we push her a little more, but not a lot. She needs the energy.
Despite the decline, Ketzl’s still playful, bright and alert. Games are different than they were before—we play “get the blanket off your head”, “tear up some newspaper”, “catch the popcorn” and the like—but they’re still games, and she’s clearly still enjoying them.
So we continue to watch Degenerative Myleopathy take our dog, bit by bit: an intractable enemy who will win in the end. I’m thankful she doesn’t know, that dogs are so much in the present, that she can’t see the tears well up in my eyes when I watch her struggle to roll sternal from one side or the other, her trademark Berner grin intact as she faces the challenge and (with occasional help from her adopted pack) succeeds.
Fraser Speirs on Nice Software (he was kind enough to include SuperDuper! in the list):
...I also think it’s much harder to impart that serious ‘weightiness’ in something that doesn’t really have a physical aspect beyond the CD it comes on. Actually, the product packaging can play a part in my attitude to software. If the box looks serious and has the right texture and weight, I’m already better disposed to it.
He’s absolutely right: it is harder to give a product “presence” these days, especially when you’re “just” selling licenses on the Web. With Shirt Pocket, I’ve tried to concentrate on usability and support—key differentiators in the software market. I also try to make our products do one “pocket-sized” thing, and do it exceptionally well… and that means a lot of time spent polishing the software and the documentation.
These are all things I did in my previous companies, too, but we would also spend an enormous amount of effort—not to mention money—on the packaging of that content.
BRIEF, for example, had two enclosed Wire-O bound manuals, on paper of significant weight, along with a tri-fold Quick Reference Card, all packaged in a nice slipcase. The graphics were printed using a high quality process, and sealed with a matte plastic film: we knew the documentation was going to be used, and we wanted to make sure it held up well, and looked good in an office. As I recall, this cost us a huge amount per-product. But we were able to charge $195 a seat, and wanted to deliver $195 of value to the user. And —whenever they picked it up—our users got a “nice” feeling from the quality materials and presentation.
The same went for Track Record, although the size of the documentation made a slipcase impractical. Instead, we used the same kind of enclosed binding in a “candy box”, with a separate “How Do I?” guide that was designed with individually cut-back pages that acted as an “index”. Less expensive to produce, but still very “nice”. (Interestingly, when we were acquired, one of the first things Compuware/NuMega did—other than remove the space between “Track” and “Record”—was cut back on the quality of the materials… and raise the price.)
It’s something I miss, actually: the physical embodiment of the work we all put into our software. A giant box full of air and a CD just doesn’t do it for me…
"norrebo” blogs about his experiences backing up his PowerBook to an HFS+ volume that he also accesses from his Windows machines using Mediafour’s MacDrive:
The total cost for this adventure was $265.94 ($188 for the disk, $27.95, for SuperDuper!, and $49.99 for MacDrive). But knowing that I’ve got a bootable image of my Powerbook disk? Priceless.
viaThe Simple Life. Glad you like it, “norrebo”, and thanks for sharing the technique you use!
Those who know me know that I’m relatively obsessed with coffee… a harmless vice. I always grind my own beans, but over many years I hadn’t found a really great grinder that wasn’t dedicated purely to espresso. Good ones, sure, but nothing great.
I have to admit, when I first saw the Kitchen Aid Pro Line grinder, I didn’t take it seriously. The overly stylized design, combined with the total lack of any historical “corporate” experience with coffee from Kitchen Aid, basically screamed “keep far, far away”! I mean, come on—these guys make stand mixers, and they’re echoing that design in the grinder!
Then I kept seeing it mentioned positively in a lot of different places, including the utterly obsessed Coffee Geek site. I mean, they liked it? Hmm…
I tend to brew “regular coffee” using various “direct infusion” methods: French Press ("Push Pot"), an Eva Solo Cafe Solo (great pot, actually), and a number of different Vacuum pots. All of these have one common element: they need a coarse grind, or the cup ends up muddy.
It’s harder than you might think to get a consistent coarse grind. Many grinders, including my previous Solis Maestro, can be set coarsely, but the coffee they produce isn’t uniform: fine particles are mixed with coarse ones. So, when the Solis decided that it wasn’t going to feed beans without constant prodding from a chopstick even after extensive cleaning (resulting in a delightful “woody” mix of bean and stick), I decided to give the Kitchen Aid a shot.
And I’m glad I did. I still think the design is over the top, but the grinder itself works great. The coffee particles are remarkably consistent, and there’s very little residue left in the cup. The unit is quiet, and the bean hopper/auger setup delivers a consistent flow of beans to the vertical burrs. And the whole thing is relatively static free, too.
Overall, a big thumbs up. If you grind your own coffee, you’ll love this thing. (And it’s $25 off at Amazon right now if you use the coupon code VALENTIN at checkout.)
Khoi Vinh, late of Behavior (now the Design Director for the New York Times) and the genius of grid-based designs (I wish my own site looked one-one-millionth as good as stuff he tosses into the trash), recently had a catastrophic hard drive failure (all too common, believe me):
I feel very fortunate, though; immediately after the first signs of trouble, I quit my procrastination and scheduled full backups of the hard drive on alternating nights to two different external FireWire drives — aided in no small part by the sheer awesomeness of Shirt Pocket Software’s invaluable SuperDuper! product. As a result, I lossed less than a day’s worth of work; not perfect, but it could have been loads worse.
SuperDuper! made it exceedingly easy for me to create a complete, bootable mirror of my hard drive, which actually allows me to continue to use the same system — with all of my files, preferences and software tweaks intact — with another Macintosh. I pulled my old Aluminum 15-inch PowerBook G4 out of retirement and booted it from one of the backup FireWire drives.
via Subtraction, Khoi’s blog. Exactly the idea—glad SuperDuper! could be of service!
Our “production” iMac hit the Shirt Pocket loading dock (aka my front step) yesterday, and I’ve been busily working on updates to netTunes and launchTunes while Bruce works on completing the next update to SuperDuper!, which looks like it will be Universal as well.†
It’s been nice to get back to Xcode after some time off (my design/marketing/documentation/support/etc duties keep me away from it more than I’d like), and it didn’t take more than a few hours to do the netTunes conversion—it’s up and running as I type, and works great.
The big challenge with netTunes was the screen “scraping”—the code has some byte-order dependencies because of pixels, and I wasn’t looking forward to re-writing the “inner loop” to handle all the various cases with Thousands and Millions of colors using both MMX and Altivec. Blegh.
Fortunately, there was an elegant solution. Since the Intel Mac is Tiger-only, I was able to make use of some the new Accelerate framework APIs to eliminate byte order dependencies and—at least under Tiger—about two pages of nasty code. Since the Intel Macs can’t run Panther, I didn’t have to change the already-working parts. The result? Both the Intel and PPC versions benefit, at least under 10.4.
It’s always a good feeling when you can delete code and leave the hardware-specific optimization up to the guys who live and breathe vector processing units, let me tell you.
Anyway, we’re working hard to get the Universal binaries out the door. Obviously, it’s important to do extensive testing using real, production hardware—and we’re doing just that.
† - Updated for the exclamation point haters. You know who you are.
