As promised, at the bottom of this post is a link to our Catalina beta.

This post is mostly a just-the-facts-ma'am discussion of changes in Catalina that have affected us and may affect you. Details of our journey may come later, when I'm in a more narrative mood.

This version has been a long time coming, and while we hoped to have a beta during Catalina's development, we really didn't like the stability of Catalina during the process. Many things involving the drive setup were changing and being reworked as the summer wore on.

Rather than put out something that was at the mercy of the underlying system, whose release process we couldn't control or predict, we decided to hold off until the real release. Once that was out, we finalized our testing, and now we're putting the public beta into your waiting arms/hands/drives/computers...whatever metaphor here is appropriate.

Drives are Different

Apple's intention is to hide the details, so it's possible you didn't notice, but in Catalina APFS drives are now quite a bit different than they were before. Let's just do a quick refresher of how we got to where we are now.

Stage 1

macOS started out with pretty "standard" Unix-style protections. Permissions and ownership were used to establish what you could or couldn't do with a file. The "super user" could override those permissions, so while you were normally denied access to "system" files, a little "permission escalation" allowed you to modify anything you wanted.

And, since "you" could modify anything with a little work, so could something that had unkind intent.

Stage 2

In 10.4, Apple added ACLs, which are more fine-grained "Access Control Lists": super detailed permissions for files and folders that can be inherited. Most people aren't aware these exist, but they're super handy.

Stage 3

The next step was adding a special attribute ("com.apple.rootless") that caused the file system driver to "protect" certain files on the drive.

That caused some initial consternation that I documented on this blog, because Apple "overprotected" the system, even when it wasn't the startup drive. Fortunately, before shipping, they fixed that problem, and copying worked as expected.

Stage 4

Next came System Integrity Protection (SIP), which took that concept and extended it to some user files as well. Some areas of the drive couldn't even be viewed, let alone copied, but with certain types of authorization (like Full Disk Access), copying could be done.

Stage 5

And now we're at Stage 5, which is either denial or acceptance. I forget, but basically this is where the entire system has been made read-only, and placed on its own, separate volume.

It doesn't look that way in Finder: everything seems like it was. But behind the scenes, two separate volumes have been created. Your files, and dynamic system files, are on a new "Data" volume. The system is on its own "System" volume. And they're stitched together with "firmlinks", a new, system-only feature that makes two volumes look like one at the directory level.

Make this invisible

Again, Apple has tried to make this invisible, and for the most part, that's what they've done. And we've tried to do the same.

In SuperDuper, you won't notice any changes for the most part. But behind the scenes, we've made a lot of modifications:

  • In order to replicate this new volume setup, system backups of APFS volumes must be to APFS formatted volumes. SuperDuper automatically converts any HFS+ destinations to APFS volumes for you (after prompting), so you won't have to do anything manually in most cases.
  • Any existing APFS volumes are properly split into these new Data and System volumes. If your backup was previously called "Backup", your files are now on a volume called "Backup - Data". The read-only system volume, which contains Apple's files, is called "Backup". These volumes are now set up in a "volume group". Again, all this is done for you, and we hide the details...but if you see some new volumes, you now know why.
  • And because of this, a single copy is really two copies: the system volume (when unmodified) and the data volume (where your stuff resides)
  • Those two volumes are further linked together with "firmlinks", which tunnel folders from one volume to the other in a way that should be transparent to the user. But they can't be transparent to us, so we had to figure out how to recreate them on the copy, even though there's no documented API.
  • Plus, we've needed to make sure we do the right thing when you copy from a regular volume to something that was previously a volume group, intuiting what you mean to do from your copy method.
  • I really could go on and on and on here. It certainly felt endless during the process!

Yeah, no matter how much one might long for turtles, it's hacks all the way down.

Seriously, though, this is the kind of thing that requires an enormous amount of testing—not just tests of the program, but testing during development as well. One "little" change that totaled about 10 lines of code required over 1000 lines of unit tests to verify proper operation. And that's just one internal change.

Except when you can't.

Despite our efforts, the new setup means some adjustments must be made.

  • You can't turn an already encrypted APFS volume into a volume group. As such, you'll have to decrypt any existing bootable volumes. Once you've backed up, you can boot to that backup, turn on FileVault, and boot back. Subsequent Smart Updates will maintain the encryption.

That doesn't mean you have to back up critical files to an unencrypted volume. If this is important to you (HIPAA or similar requirements are often quite strict about encryption), create a separate account called, say, Intermediate. Use a SuperDuper copy script to exclude your normal User folder (where the critical data is - for example, /Users/dnanian - see Help > User's Guide for a detailed discussion of Copy Scripts). Then, use that new script in "using" to back up to the unencrypted volume.

Start up from the unencrypted volume and log into Intermediate. Turn on FileVault.

Now, start back up from the regular volume and use "Backup - all files" with "Smart Update" to update the backup. This will add your regular account data, which will remain encrypted, as will subsequent Smart Updates.

  • Unmounted images are a bit of a problem. We don't know what's "in" the image when it's a file, and so we don't have enough information to warn you before the copy starts.

    Because of the reasons listed in Make this Invisible, we may need to convert the image to APFS. Some cases might not work, and some we don't want to do without prompting you.

    So, we'll give you an error, after the image has been opened. You can then select your image's "open" volume (not the image file itself), and do the backup, once, like that. We'll warn you normally, do the conversions needed, and make the copy. Subsequent updates that don't require conversions will work normally.

  • Because we've had to work all through the summer while everyone else was having fun, support may be crankier than normal. Sorry/not sorry.

Wrapping it up

So that's about it. Things should generally feel "normal" to you, although there will be some prompts for those who have encrypted or HFS+ destinations and a Catalina APFS source. Overall, though, the goal is for it to be relatively painless...and I'm sure you'll get in contact if you feel otherwise.

To get started, download the beta below. Note that you'll automatically get subsequent beta releases, and the final Catalina version when available. Once that final version is released, you'll only get release versions...until you manually install another beta.

Thanks for your patience, and we look forward to your (hopefully positive) feedback.

Download SuperDuper! 3.3 Beta 1 (v119.2)