To follow-up on yesterday’s post, the inevitable question is: how do files get like this in the first place?
As far as I can figure out, this floating owner “feature” is part of OSX’s OS9 compatibility, and also has to do with the “Ignore ownership on this volume” checkbox you’ll find in the Get Info window for non-boot volumes.
It’s important to remember that OS9 has no concept of ownership whatsoever. So, when a file is created by an OS9 application, running in OS9 itself, it has to choose a user that’s going to own the file.
But what user should it choose? As I said, there’s no real concept of ownership in OS9. And if you randomly just pick a user—say, the first one—there’s no guarantee that you only have one user on your machine… and no way to ensure that, once you’re in OSX, the correct user owns the file.
There’s a bit more complexity to this whole “user” thing. While a user has a name (the “short user name” you select when you first create the account), the system doesn’t really assign the “name” as the owner of the file. Instead, it uses a User ID, which is a number.
On OSX, the first user account created is given the User ID 501. The next account created is 502, etc. These numbers are what are associated with the files on your disk, not the short user name, which is more for your convenience than anything else.
This can create some confusing situations, though, when you bring a disk to another computer, and turn “Ownership” on. If it’s another user’s computer, and their account was the first created (most are), that means their User ID is the same as yours. So, your files will look like they’re owned by them!
Similarly, if you have more than one computer in your office (or family), and the accounts weren’t created in the same order, your account may be “501” on one machine, and “505” on another! (Here, to avoid this problem, I make a habit of creating all my accounts in the same order on all machines.)
No doubt this is why Apple allows you to turn ownership off when you attach a FireWire drive to your machine… this causes all the file owners to float!
So, Apple compromised. Since they couldn’t randomly pick a user, and they didn’t want to force permissions on OS9—which would cause serious compatibility problems—they did the next best thing. They made these OS9 files owned by “everyone”, all at once. In other words, they made the owner float.
When the owner’s floating, it looks like it’s owned by the user looking at it. So, if the file is saved in OS9 (or stored on a FireWire disk with “Ignore ownership on this volume” checked), it’ll be owned by whatever user looks at it.
Even if that user is a Backup program running as root, like SuperDuper!. Which brings us full circle.
25 Apr 2005 at 10:53 am | #
Here’s one for you that will _really_ bake your noodle though… take all this stuff you have been talking about, and now throw in some Open Directory based users into the mix ... they still use UID to identify , but .... I’ve seen several cases now where my own account (being the sys admin of the network) has gotten identity crisis and the UID has been reset on ALL of my files. Of course you can see where this is going (SuperDuper can’t deal with this either .. and it shouldn’t have to). . . but I digress.
The issue with Open Directory based user accounts is also a generic UID one. These UID’s and groups become somewhat invalid when seperated with the OD server. So just wanted to give you something else to ponder here for this issue. This could also be a messy one, let me give you an example…
You have a group of users that all have r/w access to shared files. When they are copied off of the machine onto a laptop to be taken offsite, things work as expected (the user has full r/w access).... but ... when they modify the files when not attached to the OD network, the GID on all the files comes back as incorrect.. they copy them back and all the GID’s are wrong (they are setup to local GID’s only) as the mobile account on the machines know their own information, but they no longer apply a proper umask to the files according to GID or sticky bits as they don’t know about the remote groups. I think in this case that ‘floating ownership’ deal applies here as well, but in a different context.
Not directly related, but again something worth taking into consideration in the global scope of how apple is handling UID/GID stuff. Traditionally with standard Unix/Linux machines this wasn’t so much of a problem (at least in my experience) as most stuff was fairly locked down, ever when accessing from a server and the UID/GID stuff is much more strict, but with the advent of portable unix based stuff things can get messy, fast (as you guys have stumbled accross here).
25 Apr 2005 at 11:09 am | #
Oh yeah. There’s no question that account mobility isn’t handled as well as it could be. I’m hoping Tiger Server cleans some of this stuff up! And, hopefully, whatever happens will work better than roaming profiles (etc) do in the Windows world.
Thanks for the comments!
25 Apr 2005 at 07:55 pm | #
After reading your posts on the development process of SuperDuper! (and its related problems), you’ve convinced me I can trust your product. I’m picking up an iBook next week, and SuperDuper! is going to be one of my first purchases. Backup, backup!
By the way, the text boxes for the comment form here are extremely hard to see (i.e. very low contrast).
25 Apr 2005 at 08:05 pm | #
Hey, glad to hear it!
Regarding the text boxes, what browser are you using, “R”? Do you mean the displayed comment or the place where you enter text?
25 Apr 2005 at 08:33 pm | #
Fred Sanchez wrote a nice article about this “floating owner” phenomenon a few years back: http://www.wsanchez.net/papers/USENIX_2000/
To quote:
We should also be aware that while the primary goal here is to get Mac OS X to operate properly with these differing software stacks, we still have millions of Classic Mac OS users and these users need to share files and media with our new Mac OS X users. One example of trouble is Unix permissions on HFS+ volumes. Mac OS 9 will not read or write Unix metadata; though HFS+ supports it, Mac OS 9 will ignore those bits completely. That’s acceptable until you start moving a disk between Mac OS 9 and Mac OS X systems (or if you dual boot your system and also, to a lesser degree, if you run the Classic environment in Mac OS X).
This may be a problem if you expect permissions to be enforced across systems, but if you are handing someone your disk or booting into another OS, you are bound to get disappointed eventually. A more interesting situation, however, occurs when files get created in Mac OS 9 and are seen later by Mac OS X. The files have no Unix metadata set, and therefore no permissions or owner. User 0 (root) and group 0 (wheel) are reasonable defaults, but a file mode of 0000 is almost certainly not what the user wants. Fortunately, we are able to distinguish between “no metadata set” and zero values for mode and owner info. What we do in this case is present the owner, group, and mode as some reasonable default, which is determined by examining the permissions on the directory node on which the file system is mounted. Note that the actual permissions on disk remain unset unless someone sets them explicitly; an unmount and remount with a different default will effect the mode of such files. This allows for a flexible administration of the default mode such that some disk default to more open permissions than others, as desired.
[end quote]
The “solution”—not that there is one that’s right for everyone—is to manually assign permissions to affected files. This can also be done by simply copying the files to a volume that does not ignore ownership. Alternatively, you can just re-check that box after running your backup software (as backup software typically unchecks the box so it functions properly).
Mike
25 Apr 2005 at 08:58 pm | #
Thanks for the comments and pointer, Mike!
The interesting thing we found is that files on a user’s “normal” system—not on mounted external drives, but their real, internal volume—were floating. That’s what was surprising: these files were on a volume that does not ignore ownership.
To be clear, SuperDuper! does, indeed, uncheck the box on volumes it’s copying to, and will not copy to anything that’s not “permission enabled”. What was unexpected was that files that the user normally interacted with—like their own home folder—were floating.
Since SuperDuper!—much like CCC—runs as root, the files floated to root, even though they were copied from two ownership-enabled volumes.
Wacky stuff, no?
25 Apr 2005 at 10:05 pm | #
(Ack! My own blog won’t let me edit my own comments!
)
What I’m getting at, above, is that checking/unchecking the box doesn’t work. The box is always unchecked on the boot volume, and yet there were floating files on it. So there’s no easy workaround, and it’s very surprising to the user when they find their backup is whacked in this weird way!
Fortunately, as I mentioned in the previous post, we figured this out and found a way around the issue.
25 Apr 2005 at 10:06 pm | #
Indeed, you can simulate this behavior by setting the owner of a file to “99” ("unknown").
% touch test
% ls -l test
-rw-r--r-- 1 admin staff 0 25 Apr 21:40 test
% ls -ln test
-rw-r--r-- 1 501 20 0 25 Apr 21:40 test
% sudo chown 99 test
% ls -l test
-rw-r--r-- 1 admin staff 0 25 Apr 21:40 test
% ls -ln test
-rw-r--r-- 1 501 20 0 25 Apr 21:40 test <-- Wha?! (this is preferred behavior though!)
% sudo ditto test test2
% ls -l test*
-rw-r--r-- 1 admin staff 0 25 Apr 21:40 test
-rw-r--r-- 1 root staff 0 25 Apr 21:40 test2
% ls -ln test*
-rw-r--r-- 1 501 20 0 25 Apr 21:40 test
-rw-r--r-- 1 0 20 0 25 Apr 21:40 test2 <-- Eek! This is where things break…
To understand what is breaking, obtain a root shell:
% ls -l test*
-rw-r--r-- 1 admin staff 0 25 Apr 21:40 test
-rw-r--r-- 1 root staff 0 25 Apr 21:40 test2
% ls -ln test*
-rw-r--r-- 1 501 20 0 25 Apr 21:40 test <-- this file is still actually owned by “99”
-rw-r--r-- 1 0 20 0 25 Apr 21:40 test2
% sudo -s
# ls -l test*
-rw-r--r-- 1 root staff 0 25 Apr 21:40 test
-rw-r--r-- 1 root staff 0 25 Apr 21:40 test2
# ls -ln test*
-rw-r--r-- 1 0 20 0 25 Apr 21:40 test <-- yikes! root has no idea of the actual ownership!
-rw-r--r-- 1 0 20 0 25 Apr 21:40 test2
When root looks at the files, they appear to belong to him. Worse yet, root cannot change the ownership of a file to 99.
This (filesystem? OS?) behavior, which is errant, has been resolved in Tiger. Now files owned by 99 appear to be owned by the console user, but appear to be owned by 99 to other users, including root. Therefore, ditto, and any other command run with uid/euid = 0 *can* preserve the ownership of 99.
Without knowing what files are affected, I’d guess that there is some application setting the owner to “unknown” or these items may be in a shared folder.
Wacky? OS 9 was wacky!
25 Apr 2005 at 11:52 pm | #
Yep, that’s exactly what we found, Mike. The “uid 99” thing was only in one place I could find: the HFS+ implementors guide. I think there’s one line in there that mentions it.
Anyway, SuperDuper! does handle this case under Jaguar, Panther and Tiger, so it’s all good!
(And you’re right—OS 9 was wacky. But this was pretty confusing to those of us who don’t have direct access to the guys who know the score!
Glad you found it interesting enough to stop by, though—nice to ‘meet’ you!)
26 Apr 2005 at 04:20 pm | #
I’m almost certain that I saw something about this in an interview with an engineer at Apple, about the problems about implementing OS 9 compability. According to him, any file created by OS 9 has user/group 0/0 (which is root/wheel) and mode 0000. root/wheel are reasonable defaults, I suppose, but mode 0000 is not very useful. Any drive that comes from OS 9 thus needs some special pleading.
The import procedure also has changed - I think the problem you describe comes from drives that were imported in an earlier version of OS X. Panther may do the importing differently.
26 Apr 2005 at 04:38 pm | #
Patrik: some of the previous comments talk about a paper that was published that deals with this issue a bit, but leaves out some of the critical details. But while it does talk about some things—like permissions—it doesn’t really go into ownership very much, and never indicates that you’d have files on a drive with ownership on that might be treated as if they have no real owner.
You can also see, though, that Mike Bombich from Apple indicated that the problem is fixed in Tiger—so, in the future, it won’t be as much of a problem.
Now that we know what going on, it’s more obvious, but at the time it sure was surprising!
19 Mar 2007 at 06:18 am | #
Well, one of the best ways to find out things you don’t know is by asking, even if the question seems rather stupid, so here goes:
I see “mode 0000” mentioned a few times, and I’m wondering what exactly it is… Could someone give me a short explanation?
19 Mar 2007 at 08:59 am | #
I’m guessing that Patrik means “permissions set to 000”, which means no ability to do anything with the file. That is:
touch moose
chmod 0 moose
ls -l moose
---------- 1 dnanian staff 0 Mar 19 09:57 moose