CODEWITHSUNDEEP
filesystemszfs
Pavel Bogdanov
Pavel Bogdanov

Reputation: 47

Why do these 2 ZFS pools have different allocation and capacity, even thought they have the same files?

I have two zpools on ZFD:

The data is EXACTLY the same. I even ran diff -qr /top/zpool/ /bottom/zpool/ to confirm.

Why is it that ALLOC and CAP fields differ, if data is exactly duplicated?

enter image description here

Upvotes: 0

Views: 647

Answers (2)

Pavel Bogdanov
Pavel Bogdanov

Reputation: 47

So I figured out why the amount of space is different on two different ZPOOLs:

Under normal circumstances 8 disks 2TB each is 16TB of storage in the first group of disks. And 4 disks 4TB each is also 16 TB of storage in the second group of disks. Because this is raidZ3 the parity across two pools is NOT the same. raidZ3 is designed to survive a loss of 3 disks from the Zpool. If the first ZPOOL loses 3 disks then it can remain mostly functional and have 5 disks of 2TB data on each. That is 10TB of readable data from the first Zpool. If the second Zpool suffers a loss of 3 disks at 4TB each then only one 4TB disk remains readable from the second Zpool. Because 4TB != 10TB there is a difference in capacity in ZFSRaidZ3 between the pools.

Upvotes: 0

freezed
freezed

Reputation: 1339

Dealing with ZFS space could be hard.

To be sure that The data is EXACTLY the same, give a try to zfs list -o space.

About the difference between ALLOC and CAP, docs.oracle.com says:

ALLOC: The amount of physical space allocated to all datasets and internal metadata. Note that this amount differs from the amount of disk space as reported at the file system level.

CAP (CAPACITY): The amount of disk space used, expressed as a percentage of the total disk space.

And for a more detailed answer you need to consider the block size and the average size of the data stored. A whole explanation can by found in Matt Ahrens explanation on Delphix blog.

Upvotes: 2

Related Questions