monline

Let's assume you've an LVM2 physical volume on the /dev/sdb1 partition. The PE size is the usual 4MB (4194304 bytes) and the PV size displayed by pvdisplay is: 465.76 GiB / not usable 544.50 KiB. You take a look at how this space is distributed by running pvdisplay -m --units b /dev/sdb1 and get ...


Let's try to get rid of the free space at the end of the PV. We see that the extents in the range 0 - 89975 are allocated, thus 89976 extents are in use. With a 4 MB extent size that's 89976 * 4194304 = 377386696704 bytes. Let's try to set the PV size to this:

What??? What the hell is going on here? The answer is pretty simple, but both pvdisplay and pvresize "forget" to mention it: metadata. The physical volume has some metadata on it which obviously requires some space. Now the "PV Size" (as reported by pvdisplay) must provide enough space for both the PV extents and the PV metadata. The latter can be checked on with pvck:

So there's a 188 KB metadata section in our PV that we've to consider when we set/determine the PV size!

But the truth is not that simple.
I tried to decrease my PV's size gradually to find the smallest size it'd allow me to set.
When I was closing in on the limit, I hit something strange.
The following command still worked as expected:
If I substract the not usable part from the total PV Size, I get the Total PE size: 377387026432 - 329728 = 377386696704 = 89976 * 4194304. So far so good. (Although if you look carefully enough you'll see that I specified 377387223040 bytes for the new PV size and pvdisplay reported 377387026432 bytes. I guess pvresize rounds up or down the size specified on the commandline to a value that is close to a multiple of the PE Size + the metadata.)
But the next try resulted in something weird:

Now substracting 194560 from 377386893312 results in 377386698752, which would stand for 89976.000488281 extents. Or going the other way around: substract 377386696704 from 377386893312 and you get 196608 ... which is exactly 2 KB more than the pvdisplay reported unused area. Strange.

And if that was not enough ...
This time I tried to grow the PV:

The reported PV Size remained the same and the not usable area shrunk. I guess there's some magic here at works ...
If we go back to the PV Size vs. Total PE size difference (i.e. 377386893312 - 377386696704 = 196608) and compare this with the pvck output, we can probably assume that this is due to the 192512 byte metadata section which starts at offset=4096, so 192512 + 4096 = 196608.
It'd be interesting to see how the not usable value is calculated by pvdisplay. From "outside" it certainly seems to be buggy.

To summarize what I've learned today about shrinking a PV:

• first run pvdisplay -m to get the PE Size and the last allocated/mapped PE number
• then run pvck to get the size of the metadata (and include the offset of the metadata too!)
• then calculate the new PV Size by: Min. PV Size = (<last allocated PE number> + 1) * <PE Size> + <metadata size>
• then set the PV size by executing pvresize -v --setphysicalvolumesize ...

P.S.: running pvs gives confirmation about the above assumptions.