Looks like I’ll be joining Craig Crossman on his Computer America radio program Monday, November 14th, from 11pm to Midnight, EDT.
Don’t know what we’ll be talking about, but you can listen online or at various stations across the country—hope you can tune in!
Every day, I get asked one question over and over—namely:
My backup is failing, and it’s saying that it couldn’t disable ignore permissions! What’s going on?
This started the day Tiger shipped, and has continued on a hundreds-of-times-a-week basis since. And the worst part? It’s a bug in Tiger. So, put on your waders—we’re going in, hip-deep.
Basically, here’s the deal. OSX has a file called “volinfo.database”, which is stored in a normally-hidden-from-you folder named “/var/db”. It’s a simple text file that looks something like this:
290AC3DC4B28C8FC: 00000001
9A081A3451A451C7: 00000001
435F42E92DC994B5: 00000001
C3301F4FEBB1BC41: 00000001
F36BADBD7ACFBD6F: 00000001
AB47F9CD780BF0A8: 00000001
76E8B50DFBBBD9BB: 00000001
893D1C0A293603C7: 00000001
That looks like gobbledegook to most, but it’s really a database of the volumes you’ve got connected to your Mac, listed by UUID (a low-level internal identifier that’s constant, even if you rename the volume), followed by the state of their “Ownership” flag: 0 if ownership is off, 1 if it’s on.
When you attach a new drive to the system, the UUID of the drive’s volumes are added to the database, along with their “ownership” setting. And, when you check/uncheck it in Finder’s Get Info panel, the value changes here, too.
Apple ships a command-line tool, vsdbinfo, that allows programs to check and adjust the ownership state of a given volume.
It’s very important for SuperDuper! to ensure that ownership is turned on when it makes a copy. If it’s off, the system does strange things with the file’s owner and group (see Floating ownership nearly sinks us for the story of that particular horror). So, we check it, with vsdbutil, during the “preparation” phase before starting a copy.
Which brings us to Tiger. When Tiger installs, it goofs up when it manipulates this file. So, instead of what you see above, you get:
290AC3DC4B28C8FC: 00000001
9A081A3451A451C7: 00000001
435F42E92DC994B5: 00000001
C3301F4FEBB1BC41: 00000001F36BADBD7ACFBD6F: 00000001
F36BADBD7ACFBD6F: 00000001
F36BADBD7ACFBD6F: 00000001
F36BADBD7ACFBD6F: 00000001
It’s a really simple bug. And while I’ve reported it, it hasn’t been fixed. We’ve waited, provided our users with a workaround, and waited, and waited… and it’s just not getting fixed.
This is really frustrating for our users, because things don’t work in a mysterious (and ungrammatical) way. And it’s frustrating for us, because it makes us look bad, incompetent and/or lazy. Honestly, we’re not.
So, since it’s not getting fixed by Apple, we decided on an alternate approach. I did the research necessary to ensure the problem was consistent, and to figure out how we could fix it for them. After looking at hundreds of these, and confirming that the problem is as described above, we’ve integrated a fix into v2.0.
Specifically, we examine the structure of this text file. If it’s missing the carriage return, we correct it, leaving the semantics intact (all volumes with ownership on/off stay the way they are).
In our testing, this solves the problem 100% of the time… and I, for one, won’t miss directing people to the FAQ entry that, as of this writing, has been viewed 5325 times since April.
I’m sure our users will be much happier, too!
As two of you know, SuperDuper! has provided an AppleScript interface for some time. We’ve used it to provide the rudimentary scheduling in the v1.x series, and we’ve enhanced as necessary to improve the experience in v2.0.
But, one rather tricky thing fell out during the process: users can (and will) schedule any number of backups in ways that might overlap, or even start them at exactly the same time. And, given the nature of a multi-processor machine, and the fact that each “scheduled job” is an AppleScript external to SuperDuper! that has no knowledge of the other scripts around it, there had to be a way to serialize execution or the scripts would all interfere with each other.
In the v1.x series, we’d put a rudimentary system in place: the script can ask about the current status about SuperDuper! and—if SuperDuper! is busy running another copy, it’ll say just that, and the script just waits until it’s done. This was mostly done to prevent the script from “grabbing” the interface away from a user who’s actively using SuperDuper!.
But, with v2.0, it’s trivial to schedule two copy jobs for the same time, and with the old technique both could be told that SuperDuper! was, indeed, idle. And then, both would try to run at the same time. Clearly, it’s unreasonable to ask users to schedule their own backups in a way that would take this into account: we had to handle it internally.
Of course, another way to handle this is to break SuperDuper! into a true background engine, and have the UI and scripts all “drive” this process (or processes). But, beyond breaking existing scripts, doing so required enough reengineering to push it well beyond the scope of this particular release, much as it would have been cool. Gotta ship sometime, you know?
So. Bruce and I talked about this for a while, and Bruce came up with the idea of just changing the status into a queue. The first process who asked for status is at the front, and only the guy at the front gets the real status at any given time: every other script always gets “busy” until they move to the front of the queue.
Nice, simple solution and—apart from various tricky bits that always plague any solution—it seemed to do the trick.
Which led to another problem. To try to match user expectations, the scheduled jobs try to leave SuperDuper! in the same state it was in when they started. So, if SuperDuper! is running, we leave it running; if not, we quit when we’re done.
The scripts are responsible for this state preservation. And, once again, when launching at the same time, on fast systems both would see SuperDuper! as not running, and would take responsibility for quitting. To avoid quitting the application out from under the others, each would be waiting for the other script to finish… and then, due to the deadlock, SuperDuper! wouldn’t ever quit.
Great.
More discussion, and another simple solution: the quit command basically waits until the queue is empty, and all external processes are done, before it does its thing. And this simplifies the scripts, too, because they just don’t have to worry about it.
And, all without changing the script dictionary… not too bad!
So, we’re sweating the details here. Or trying to.
Yeah, OK, so the technical definition of a Beta test is that it’s feature complete. But, that’s not how I’ve ever really done it. I consider a Beta a build sent to external testers that’s stable and usable. And, SuperDuper! v2.0 has been in this type of testing since November 22, 2004.
Yeah, that’s right. We’ve had 70 external testers pounding on v2.0 since last November.
The testing has been going great, with very few problems reported, especially considering how much has been changed over the months. And I’m happy to say that we reached an important milestone yesterday: the last new feature of v2.0 was finally released to our external testers. We’re feature complete. Things are working really well.
I feel like I’ve been at sea for a year and have just sighted my first gull. Land can’t be far away!
As a long time Mac user (since 1984) a have one really big complaint about MacOSX, and that is the incapability of the Finder to easily copy a bootable System folder to an external disk and boot from there. Enter SuperDuper!, a 20$ software for MacOS X, that allows you to copy entire disks, even in the unpaid version. The paid version has tons of additional features, but all with a really easy and understandable user interface. Highly recommended and compatible with MacOS X 10.4, too.
via teddythebear: Thanks, Mr. Bear!
Right after Tiger arrived on my doorstep, I went shopping for a firewire harddrive. I waded through a ton of reviews and finally settled on an OtherWorld Computing Mercury harddrive. So far it’s working very well. Although I could have used the included Retrospect program, I opted to go with a shareware program named SuperDuper!. And that was probably one of the better choices in software I’ve made. As soon as I used it, I knew it was worth every penny of the shareware price (which I immediately paid). Sometimes, I look at a piece of software and have a debate about whether the fee is worth it, whether there might not be something out there that’s freeware or open source, whether I’d rather just use something that’s produced by one of those big software companies…but there was no debate about this one. I fired up this app and just knew.
via Ms. Cranky Pants Techie Geeky Mac Stuff (Thanks, Alice!)
Just noticed this:
Painless disk cloning - Just a quick plug for SuperDuper which may end up joining DiskWarrior as my must-have Mac utility. It does exactly two things very well: a) it creates bootable DMG clones of Mac disks and b) it lets you very easily update the disk images using boilerplate or customized scripts. Part 1 has historically been the domain of Carbon Copy Cloner — a great little app. One thing SuperDuper adds is a simple interface for scripting the items/folders you do and don’t want to clone as well as how you want them to behave (new vs. changed, etc.). This basically means you now have a perfect copy of your drive that can be restored very easily in case of calamity. Ditto for doing a “safety clone” before you install a bunch of dicey system upgrades. Highly recommended utility.
via 43 Folders: Thanks, Merlin! (and don’t miss Merlin’s classic 5ives, either!)
Hey, so it’s been a while since I’ve written specifically about SuperDuper! v2.0. And it’s not because I’ve got nothing to say.
So, let’s do a bit of a status update, and another sneak peek.
Where We Are Now
As you might expect, we get asked when SuperDuper! v2.0 is going to ship all the time.
I know you want the new version: I want to give it to you! But, it’s not ready. We’re close, but we’re not done: there’s one significant thing left to do, and it’s a bit more difficult than we anticipated. We want to get it right—as we always do—so, it’s taking a little time.
But, once that’s complete, we’ve got a bit of polishing left, and we’re done. I don’t know how long it’s going to take. We’re going as fast as we can.
So, How’s it Look?
Actually, it’s looking great. Every part of SuperDuper! v2.0 has been improved, from the main window to Options, from the log to status… the copying is better… really, it’s all good.
You Mean, You’ll Never Have to Update it Ever Again?
Yeah. Right.
Well, C’mon—How About Some Screen Shots?
Coming right up!
In the new version we’ve broken out the v1.x options into two sets: General and Advanced. This emphasizes the common options—such as the During copy method—while putting more advanced ones—like custom shell scripts, package install and post-copy imaging—into its own tab.
You can also see that we’ve taken our What’s going to happen? approach a bit further, and explain the During copy option you’ve selected right under it (as well as in the main text).
You might also notice the pop-up in On successful completion. In there, we’ve got many more options for what happens after the copy completes, including the ability to shut down the computer, put it to sleep, simply set the destination as the startup volume, etc.
And then, in the Advanced tab, much geekier things that most users don’t need.
We’ve added an additional Before copy option to run a script—this allows people to do cool things like dump MySQL databases before backing up and do other “before you touch things” operations, and renamed things a bit to make their purpose clearer.
So, we ended up with:
So—there you go… and yes, everything we’ve posted about so far works, and works well.
Back to the grind!
The Pavlov’s Beep post got me thinking a bit about how user expectations can set a product up for failure out of the gate… and that made me think of the late, lamented (by some), lambasted (by many) Newton.
Ignoring the obvious hype surrounding the rollout—something the product could never live up to—one of the things that was coolest about the Newton was also the thing that caused the most frustration: handwriting recognition.
Handwriting and speech recognition are two areas that have seen a lot of advancement in the past years—for example, the recognizer that comes with Windows XP Tablet PC Edition 2005 Elite Pro Extreme (or whatever that product is called) is exceptional, and from all reports is improving in its next iteration. And, according to those who use these things routinely, like David Pogue, the latest Naturally Speaking (ex-Dragon, now ScanSoft) is similarly surprisingly accurate.
All that said, though, they’re doomed to teeny market segments, because no matter how accurate, when they make mistakes they frustrate their users.
Why? Because we’ve asked the computer to play by our rules, rather than asking us to play by its.
When a device imposes an unnatural input method on its user, any mistakes you make are your own. Consider Graffiti, as used on Palm OS: it specifically requires you to make certain “letter shapes” if you want them to be recognized. It doesn’t try to recognize your handwriting. If a letter comes out wrong, you might be frustrated, but you know it’s “your fault”—you didn’t make the shape right.
But, with the Newton, it was supposed to recognize your handwriting. Not an alphabet of its own design. And, when it doesn’t, the first thing that runs through your head is: this thing sucks! Even if it got 90% right, which is pretty remarkable considering many people can’t read their own handwriting!
Speech is hard, too—especially if used in an “unrestricted vocabulary” scenario. Not only do you have to “speak” all punctuation (try it sometime), but correction is an incredible pain. And speech is such a natural part of most people’s daily interaction that it’s near impossible to rise to an acceptable level of performance.
Newton, near the end of its life, tried to get around this problem by taking a bit of a Graffiti approach: it encouraged people to print, and there’s a lot less letter variation in printed characters. Suddenly, with the new Rosetta printed character recognizer (still in use in Mac OSX today), people started saying the Newton did a good job recognizing handwriting when—in actuality—it just shifted the burden onto the user some more by restricting input.
Similarly, speech is becoming more common in situation where the vocabulary is highly restricted and expectations are low: package tracking #s; pick 1-2-3 menus; interacting with “speed trade” stock systems. Correction is pretty easy and, given the relatively robotic nature of the interaction, expectations are low. (You’ll note that as the systems got better, they started using more animated, natural voices at the other end—raising expectations they knew, from testing, they could meet.)
So—if you’re designing a UI, remember that the way you “frame” the interaction often sets user expectations.
Leaving aside speech and handwriting, free form input can be really, really cool—see Simson Garfinkle’s SBook and Microsoft Outlook’s date/address fields for examples.
But if you put up a free-form field , you better make damn sure that it accepts all sorts of wacky variations. If date-based, expect things like “tomorrow”, “next wednesday”, etc—every one you miss, given the free-form nature of the field presented, is the fault of the program, from the user’s perspective. Similarly, what about international addresses? And how do you tell the user you’ve made the inevitable mistake?
A lot of words to say something obvious, I know. But it’s a mistake I make all the time… e.g. “Safety Clone”…
We’re asked, every so often, why we don’t offer a time-limited full version of SuperDuper!.
It’s pretty simple, really.
If you’re going to time-limit a program, you have three choices:
The second is nasty for obvious reasons.
The third, while doable, forces users to announce themselves to me when they just want to evaluate the software. We’re very conscious of privacy here, and a lot of people are understandably touchy about this kind of thing. The last thing we wanted to do was to alienate them right out of the gate, and force them to register just to take a look.
So, given those not-so-good choices, we decided to:
