[lug] help needed with LVM situation
nate at natetech.com
Wed Jun 13 03:25:50 MDT 2007
A friend of mine recently passed away, and another friend has been
painstakingly working on recovering some important data from one of
A short re-cap:
He had two systems online at his home. We'll call them system A and
The folks trying to recover data from them had numerous problems
1. No one had any of the passwords.
2. One machine NFS mounted the other.
3. Calling them, machine A and machine B... Machine B had a
motherboard failure within a day or two of our mutual friend's passing.
4. Both machines were (thankfully) set up with non-LVM root filesystems.
So they've been working on it for a while now, they're not Linux
admins, but they work on Unix at work as Engineers for a living.
They got Machine A online and figured out how to get into it by
mucking around with the root password since they had physical
access. No problem. Just a learning curve.
Then they found that Machine A had a bunch of hard-coded NFS mounts
to Machine B which was off-line, and contained the CVS repository and
other things they were struggling to recover.
They placed the hard disk from Machine B into Machine A and attempted
to figure out how to mount it, thinking they could just grab a quick
backup... and they found... LVM. Actually, as I talked to one of my
friends about this on the phone tonight, they found both machines
were set up with LVM. None of them had ever used LVM before, or had
to deal with it. Again, a learning curve, but not insurmountable (no
We also attempted this evening, simply mounting up /etc off Machine B
and wiping out the root password, and then booting from that disk.
What we found was that the LVM tools are complaining that they're
"not the right version" for the LVM currently on the disk. I
*think*, but haven't had a chance to prove (hard to do this on the
phone) is that Machine A's LVM version is higher than Machine B's,
and it "did something" to the LVM VG it found suddenly on /dev/hdb3
when the disk was added.
We figured out that both machines had single hard disks, and
apparently both had some version of Debian LVM on them. They may not
have been on the same release of Debian, nor using the same version
of LVM, but initial digging didn't turn up much. We can get the LVM
version (lvm10 package is installed on Machine A) from Machine A.
We also figured out, and here's the annoying part... both VG's are
named the same. If you use the LVM tools to look at things (trying
not to trash the only copy of this data), you see there's a PV in /
dev/hdb3 (when Machine B's disk is physically in Machine A as /dev/
hdb), and the VG is named the same as the currently active VG on
Machine A... vg00.
(There's a case to be made here for never accepting default names for
VG's, I think... if one machine had machineA_vg00 as it's VG name,
and the other machineB_vg00, this wouldn't be a problem.)
To add insult to injury, it would appear that both machines are using
ReiserFS for their filesystems in their VG's.
With the recent Linux RAID discussions, I have this gut feeling that
tells me there are some real LVM gurus lurking on this list. What
say you? How would you get this silly thing mounted?
One thought we've had, but it was getting late and messing with disks
and LVM late at night when everyone's tired and it's the only known
copy of the data, is a bad idea... was to rename the VG that's
currently active on the working Machine A. Then there would be no
namespace clash between the two.
The other thought that seemed saner, and slightly safer, would be to
load a completely clean copy of Debian Sarge up on another machine,
install the disk from Machine B into it, and then just activate the
VG there. Personally I like that idea better, but it is more
difficult (since no one has any "extra" machines lying around at the
moment -- or if we do, we're all separated by at least 50 miles at
our homes or more).
So I'm appealing to the LVM gurus here... any thoughts?
Of course, for all of us, this is a good example of why to keep known-
good onsite and offsite backups that are easily readable by
anything... CD/DVD comes to mind here.
What it makes more clear even than that, is if there's someone elses
important data on any of your machines, a good document kept with
your will or whatever... somewhere someone would find it... that
explains your backup scheme, perhaps has your root passwords or
instructions about how to break into your machines, etc... is
probably not a bad idea for anyone who has to struggle through losing
a friend, and at the same time has to try to retrieve any of your
A Digital Systems Will, if you like. Who gets the drives, how do
they recover the data and who's responsible to distribute the data if
multiple people need copies for themselves, if your machines ever go
silent... because you did. The named executor of your "Digital Estate"?
nate at natetech.com
More information about the LUG