Apple can sometimes make small changes that have unexpected consequences.

Take Partition Schemes, for example.

This is the kind of thing most Mac users never think about. You buy a drive, plug it in, drive comes up, things work.

Done.

What you probably don't know is that most drives came partitioned and formatted for Windows, even when the drive is listed as "Mac Compatible", typically using Master Boot Record as the partition scheme and FAT32 as the format.

Before OSX 10.4.6 or so, this wasn't a big deal, at least for SuperDuper! Since FAT32 isn't appropriate for storing Mac files with full fidelity (and has various other issues, like file size limits, etc), we would instruct the user to erase the drive, using HFS+ (Mac OS Extended) as the format. Disk Utility wouldn't allow HFS+ volumes to be hosted on Master Boot Record partitioned drives, and so the user would need to select the drive, rather than the volume, and erase that way. No big deal.

Then, the Intel Macs came out, and with them, 10.4.6.

To help with something-or-other (probably to allow HFS+ and FAT32 volumes to be hosted on the same drive in separate partitions for Boot Camp users), Apple decided to change Disk Utility so that it allows HFS+ volumes to be hosted on Master Boot Record partitioned disks. Which I'm sure didn't seem like a big deal: in fact, it's quite convenient, and had probably been requested as a feature over the years.

All that's great, except for one thing: Macs don't support starting up from Master Boot Record volumes.

This is clearly not a big deal for most users. But, for SuperDuper! users (and, I'm sure, for those who use similar applications) it's been a big problem: drives that look like they should act as startup volumes won't, and the reason is really obscure.

Suddenly, users who used to use Disk Utility to erase a drive now have to head to the Partition tab (which doesn't even appear if you select the volume instead of the hosting drive), click Options, understand the 3 partition types, and partition their drive appropriately.

So, this one small change -- allow HFS+ volumes on Master Boot Record partitioned drives -- ends up having a pretty big impact on users with external drives. This is mostly due to its implementation (and lack of documentation).

Worse, users aren't steered to the right choice by Disk Utility. In fact, the default is to do the absolutely wrong thing for most: retain a Windows partition scheme.

This probably hasn't had much of an impact on Apple's support, but it's sure hurt here!

The other day, I was pushing at the limits of my existing Infrant ReadyNAS NV setup, and needed to increase its size. Normally, this would be a huge project, but with the ReadyNAS it was incredibly easy to do.

You see, the ReadyNAS uses Infrant's proprietary X-RAID. X-RAID basically RAID 6RAID 5 (see comments, below) with the ability to dynamically increase the total size of the RAID as well.

So, not only will the ReadyNAS run with a single drive faiure (and hot-rebuild the drive), it can dynamically increase the size of the RAID set as well. So, all I had to do was:

1. Buy four drives of the appropriate size. I went from four 250GB drives with a total size of about 700GB, to four 500GB drives with a total size of about 1.6TB.

   The reason you don't get "all" the space on the drives is because redundant information is spread across each drive that allows any drive that goes "down" to be replaced and rebuilt with no data loss.

2. With the ReadyNAS on, and in use, pull out the first of the four drives.

   Yeah. Scary. But that's what to do!

3. Unscrew the four screws that attach the SATA drive to the tray from and attach it to the new.

4. Slide the new drive into place.

   At this point, the ReadyNAS will automatically rebuild the data that was on the original drive on this new drive. All of this has been done with the unit on and operating.

5. Wait for the rebuild to complete (it'll send you email when it's done).

6. Repeat with the next drive.

Yeah. That's it. When you're done, you do need to restart the ReadyNAS to get the volume to expand, but that can be postponed until you're ready to do it... and that's the only time the unit is "down".

Pretty cool, eh?

(Yeah, I know I sound like a pitchman for Infrant, but I'm honestly not affiliated with them in any way at all. I just think it's a great product.)

Many of you probably know that, for some reason, iSight cameras can get into a weird state where they start causing serious errors with FireWire drives.

The symptoms usually include a bunch of I/O errors while copying to a FireWire drive: failed copies, flaky behavior, crashes. All of this goes away if you power off, disconnect the iSight, wait a while and power back up.

Sometimes, the iSight stops working with iChat (it says the camera's in use when it's not, or the light comes on, but you don't get any picture), and in it was in that second state today when I had a kernel panic.

No data was lost, I'm happy to say, but I decided that I've had enough of this. Since 10.4.9 supports USB cameras, I've switched to a Logitech QuickCam Ultra Vision. (All they need to do is add Super Deluxe Extreme Edition to the end to make it really cool!)

It's not as elegant as the original iSight (it's fixed focus, horizontal format, a bit gaudy), but it's got a nice wide angle lens, built in microphone, works with any monitor, and seems reliable. Plug the thing in, and it works.

Given that you can't buy "real" iSights any more, and based on direct experience, I give it a thumbs-up.

Sorry, iSight. I'll miss you!

When the new Airport Extreme Base Station came out, I was happy to see that it had some support for attached USB drives. It's a reasonable (though not perfect) way to share data in a home/SOHO situation, and I figured people would be using it to back up with SuperDuper! So -- to help out...

General Comments

A few things to point out right off the bat:

• As you've likely seen elsewhere, the Airport Disks are not fast. Don't expect blazing speeds: wired, I've seen a maximum of about 1.5MB/s.

• The AEBS gets very, very cranky if you get to a disk full situation. I've seen it crash more than once. Don't do that.

• Remember that HFS+ drives are made available through AFP, and FAT32 through SMB.

  Don't format any drives you're going to use with SuperDuper! as FAT32: use HFS+ (and partition properly for the Mac processor type you're using -- GUID for Intel, Apple Partition Map for Power PC).

• Make sure to connect a power supply to the drive.

• As general advice, please don't cheap out when you get an external drive. Really. Get one with a real Oxford chipset, USB/FireWire if possible in case you want to attach it directly to your Mac.

• Don't expect miracles. This is an inexpensive solution, and it behaves like one. If you want a real NAS, I suggest an Infrant ReadyNAS NV+: it's faster, redundant, recoverable. I'll have another post about the ReadyNAS soon.

• Remember, this is a first generation ("Rev A") product. It's likely to go through teething pains. Don't rely on them as your only backup!

Using Airport Disks with SuperDuper!

You'll note that your Airport Disks don't show up in the SuperDuper! pop-ups. This is by design: we can't currently copy directly to or from a network volume due to authentication/permission/metadata issues.

Instead, you'll follow the steps in Backing up over a network in the SuperDuper! User's Guide (Help > User's Guide), and back up to a read/write sparse image stored on the Airport Disk.

We often get asked why SuperDuper! can't back up directly to a network volume. What most people don't realize is that, for security reasons, you can't directly authenticate as "root" over a network, and that means it's not possible to store files with system ownership on a network drive.

An image, on the other hand, acts as a "local drive", and can be authenticated against, even though it's stored remotely. This ensures that your files are backed up with full fidelity, including proper ownership and permissions. And since it's formatted as HFS+, it avoids various situations that can ensue trying to emulate HFS+ semantics and metadata storage on a non-HFS+ drive, while still storing in a native, non-proprietary, Mac-native format.

I'd suggest doing your first full backup directly to the USB drive, rather than over the network. This'll be a lot faster. You can then connect the drive to the base station, and re-select the image using the "Disk Image..." choice in SuperDuper!'s destination drive pop-up. (Note that although the image will be grey, you can still pick it, and ignore the "overwrite" warning. Yes, I know that UI sucks.)

That should do it: enjoy the base station!

Hopefully no one out there noticed, but—for the first time in three years—Zabeth and I managed to get away for a two week vacation at Red Mountain (in Rossland, BC).

It’s not too bad a trip, about three hours north of Spokane, WA, a straight shot after a (normally) easy flight or two. A few screw-ups this time meant various additional hops, but in the end we made it.

It’s my very favorite place to ski. It’s kind of an “old fashioned” ski hill, with four “low speed” chairs, a T-Bar and a (new) “magic carpet” lift. The base lodge is quite basic, with lockers in the bottom level, a cafeteria on the 2nd and a bar on the 3rd. Up in the “Paradise” area, there’s another small lodge/warming hut/eatery… and that’s pretty much it.

And, honestly, all that works great. The food’s good, the people are great. But what Red’s about is the skiing.

The skiing is awesome.

Red had a ton of early snow—about nine feet—and though we didn’t get much fresh snow in the two weeks the mountain was in fine form. The first few days were almost spring-like conditions on the front face, soft even early in the day. Red’s a deceptive mountain, though, where you can ski 360 degrees around the various peaks, with a ton of off-piste skiing (some of which can get quite extreme). And the backsides, even during the sunny days, stayed shaded, the snow light.

Which was a good thing, because once the temperatures dropped, the front firmed up, which made the less groomed, exposed trails much less enjoyable early in the day. So, after a few groomers, we typically headed to the various back glades, bowls and other pitches, finishing up in the sun at the end of the day.

We stayed at Greene’s Family Guest House, a great little place in Rossland run by Rick and Sue Greene (thanks for the hospitality, Rick & Sue). It’s a few minutes’ drive to Red itself, and—fortunate for me—they’ve recently installed a wireless network. So, via the Miracle of the Tubes, work from 7-9am, skiing from 9-3, work from 3-7, dinner, and work until bed (usually while we caught up with episodes of Lost we’d missed, thanks to an iPod full of episodes plugged into the TV).

Seamless, I hope, for the SuperDuper! users who needed help.

It was a great time, and quite relaxing despite having full work days every day, and I feel fortunate to have a job that lets me do what I need to do even when far from home.

My parents were also able to take Taiko for the time, and I hope he was well behaved while there. They seem to have done really well, and Z and I are really grateful that they could watch him. (Sorry that he’s still jumping!)

Got back late Friday (during a snowstorm, of course), and picked up Taiko on Saturday, who was happy to see us and is warming my feet as I type. The mail revealed that Zabeth has passed the Veterinary Boards, which is terrific news too (not that I had any doubt), and she heads back into rotations early tomorrow morning.

And so, it’s back to the grind—refreshed.

Wow.

It looks like SuperDuper! 2.1 has won another Macworld Eddy! Back-to-back awards for SuperDuper! in 2005 and 2006, and one for netTunes in 2004—w00t!

Thanks, Macworld—and thanks to all the users, too!

The software business is a strange one, because the products you make eventually just… vanish. The OS goes away, or the market moves on, and soon there’s nothing left to show when someone asks what you do, or what you did.

Getting these three Eddy awards—one for netTunes and two for SuperDuper!—has been really gratifying, both because it’s an acknowledgment of good work, and because it’s more of a permanent thing—a physical record of what I was doing for all those years.

Plus, the trophy doubles as a weapon in an emergency. Man, these things are heavy!

Well, one year ago today we released SuperDuper! 2.0, and the past 12 months have gone quickly indeed.

In that time we’ve released a number of great updates, and I’ve communicated personally with thousands of you through support mail, IM, the forums and this blog. It’s been fun, rewarding and—on occasion—exhausting. But, mostly, fun and rewarding.

Looking over my blog post from this day last year, I spent my entire birthday, save for about 40 minutes, responding to hundreds and hundreds of support messages as people asked questions about the new release, and Bruce and I tried to fix a rollout glitch or two. Quite a day.

This year, though, it’s much calmer, and I’m going to take most of the rest of the day off—pretty much my first in a few years—to celebrate Thanksgiving (& my encroaching decrepitude) with my family, aunts, uncles, cousins, Zabeth and Taiko.

While I’m doing that, I also want to raise a glass of Thanksgiving wine in a toast to thank all of you, out in the tubes that make up teh Internets, for your support, encouragement, criticism, and compliments. Bruce and I couldn’t do this without you!

Happy Thanksgiving!

No doubt you’ve noticed that I’ve been neglecting the blog a bit recently.

It’s been rather busy here, with various projects taking time as we hurtle through fall toward winter. Taiko’s quickly growing into his paws, and while it’s kinda silly to compare his personality to Ketzl’s, he seems to be a bit more mischevious, and he’s certainly a lot more willing to get up on his hind legs. It’s taking a lot of time to supervise him, correct him when he takes things off counters (or jumps on people) and get him the exercise and socialization he needs. But things are going nicely: he’s about 50lbs and shaping up to be a good boy. More photos soon.

Zabeth’s fourth year of veterinary school is going by quickly as she prepares to take the boards while, at the same time, doing her clinical rotations. She’s running on coffee and adrenaline at this point, and Taiko and I are trying to stay out of the way.

On the Shirt Pocket side, things have been busy. The release of 10.4.8 brought with it what looks to be a bug in Core Graphics: many applications—including SuperDuper!—are crashing on some Intel Macs when two threads are trying to draw at the same time. This happens in a lot of cases, but in ours we have some NSProgressIndicators that use the standard option that runs them on their own thread. If we’re updating the status view (in our main thread) at the same time the progress indicator tries to update, CoreGraphics uses a lock to handle the contention… but crashes.

Of course, it’s intermittent due to the timing issues, which makes it frustrating, but we’ve reported it both through the standard methods (rdar://4789778) and through other channels. We’re looking at workarounds here, since it’s unlikely 10.4.9 would come out based on this one problem.

For testing purposes, I brought a Mac Pro into Shirt Pocket Headquarters, it’s proven to be an excellent Mac. It’s very fast (although its I/O to a striped RAID set is much slower than I’d expect), very quiet and—so far—reliable. My few Boat Anchor applications are running beautifully in Parallels Desktop now that their MacPro compatible version is out—in fact, it continually surprises me how well Parallels works. If you need to run Windows, and don’t need high performance graphics, it’s a highly recommended solution. (Just make sure your VM is shut down before you back it up, of course!)

More as I get time!

OK! netTunes and launchTunes release (and netTunes re-release—sorry about the Purple Rain) done, so it’s time to get back to what I keep getting asked about: SuperDuper! and Time Machine.

To get the “Frequently Asked Questions” out of the way right at the top of this post: no, we’re not dead, we’re not angry, and Apple has no obligation to leave market opportunities for independent developers, notify us that things are coming, or pretty much anything else.

This is business. It’s difficult for Apple to come up with 150 features to add into the next version of the OS, and harder still to make those features compelling enough that we’ll all pony up our hard-earned dollars to upgrade.

(As an aside, does anyone else out there think there was a definite hint, in the “feature” presentation of the keynote that bragged that OS X is now a “bigger, all-inclusive bundle”, that the price will be higher when Leopard is released?)

Some sort of backup functionality belongs in the OS. It’s been a long time coming. The fact that it wasn’t there left opportunities for 3rd parties, but that doesn’t mean Apple shouldn’t address the missing functionality.

And so, they have, with Time Machine. Really, I think that’s a great thing. People need to back up more often, and I hope Time Machine encourages them to do so.

Now, I can’t really get into a lot of details, because our NDA prevents disclosure of anything that wasn’t in the keynote. But let’s talk about what we’ve seen there, and why SuperDuper! remains both relevant and necessary—a true complement to the functionality in Time Machine.

First, as is likely obvious, Time Machine is designed to provide automatic “temporal” backup (discussed in broad terms in the post The Ninety-Nine-Per-Cent Solution many months ago).  Its primary usage scenario—and the one that the keynote focused on—is to allow quick recovery of files and data that have gone missing, etc. It does this in a way that’s highly integrated with the OS, with a unique UI that’s both cool and kinda cheesy… and, as was the case with Spotlight, with a certain amount of application-level impact (something 3rd parties like Shirt Pocket could never mandate).

What’s important to note is that this isn’t, and never was, what SuperDuper! was designed to do.

Our tagline, Heroic System Recovery for Mere Mortals, tries to sum up the whole idea: SuperDuper! is designed to provide excellent failover support for the all-too-common case where things fail in a pretty catastrophic way, such as when a drive fails, or your system becomes unbootable. We do this by quickly and efficiently creating a fully bootable copy of your source drive. Perhaps more importantly, recovery is near immediate, even if the original drive is completely unusable, because you can start up from your backup and continue working.

You can even take your backup to a totally different Macintosh, start up from it, and work while your failed Macintosh is in the shop… then, when it comes back all fresh and shiny, restore things and keep working.

All of this is done with a minimum of fuss and bother, and with respect for your time. And while Time Machine can restore a full system (the details of which were not shown, so I can’t comment on them), as can other similar products, that’s not its strength. Doing so requires you to actually take the time to restore the backup in full, which interrupts your workflow, requires a destination device, and takes a lot of your time—at the exact moment when you can least afford it.

So, when Leopard comes out, and Time Machine is released, be assured that we’ll continue to be relevant and necessary. We’ll work alongside its rapid recovery of individual files, and will seamlessly augment that with our rapid system recovery.

And, of course, we’ll continue to improve every part of SuperDuper! to make backups faster and easier for all.

(Digg this post.)

A few days ago, Shigeru Harada of MacFreak contacted me with a strange problem: he was unable to schedule backups with SuperDuper. Every time he tried, he’d get an error that the script wouldn’t compile.

In the past, we’ve seen this when drives were named with slash characters, but his were quite normal.

After some back-and-forth, I had him take the script template we use, and try to compile it himself, in Script Editor. Shockingly—it failed to compile, and the problem didn’t make any sense.

If you’re familiar with the Japanese keyboard, the backslash key () is replaced by the symbol for the Yen (¥). Way back when, we did a Japanese version of BRIEF, so I was familiar with this phenomenon—paths would be separated by Yen symbols, but everything worked as expected.

But, in AppleScript, it seemed the backslash/yen “swap” completely prevented backslash from doing its normal thing, so this:

set the URL_A_chars to “$+!’,?;&@=#%><{}[]"~`^\|*()”

completely failed to compile, because it looked like this:

set the URL_A_chars to “$+!’,?;&@=#%><{}[]¥"~`^¥¥|*()”

and ¥ didn’t escape as you’d expect.

A huge surprise to me. I did find it discussed in one place (thank you Google and Takaaki Naganoya), and also a reference to bug fixes in Tiger (Backslash characters and Yen sign characters will now compile correctly when the primary language is set to Japanese. [3765766]), but it didn’t seem to be fixed for him—probably because he’s using Panther (which we need to support).

Anyway, workaround in hand, I modified the script to:

set quoteChar to ASCII character 34
set backslashChar to ASCII character 92
set the URL_A_chars to “$+!’,?;&@=#%><{}[]” & quoteChar & “~`^” & backslashChar & “|*()”

Here’s hoping it works… I hope it’s not a problem with chevrons («») too, because I have no idea how I’d work around that one…