CODEWITHSUNDEEP
c++performancenfsdd
Jmoney38
Jmoney38

Reputation: 3304

dd - Understanding block size

I've used "dd" for creating test files and performing backups across HDDs. No problem.

Currently, I'm trying to use it to test NFS transfer rates. At first, I was varying the block size ("bs" argument)... But this got me thinking, why would I need to vary this argument?

A typical use-case that I want to simulate is:

In this case, the typical C/C++ code for a 2D array would be:

FILE *ptr = fopen("path_to_nfs_area", "w");
for (int i = 0; i < data.size(); ++i)
   fwrite(data[i], sizeof(float), width, ptr);
...

So in this case, we're writing to a buffer in 32bit increments (sizeof(float)) - and since this is a FILE object, it's probably being buffered as well (maybe that's not a good thing, but might be irrelevant for this discussion).

I'm having a hard time making the jump from "dd" writing from if->of in "bs" chunks versus an application writing out variables from memory (and simulating this with dd).

Does it make sense to say that it is pointless to vary the value of "bs" less than the system PAGE_SIZE?

Here's my current understanding, so I don't see why changing the "dd" block size would matter:

Interaction between app and NFS/TCP

Upvotes: 1

Views: 1050

Answers (1)

Guntram Blohm
Guntram Blohm

Reputation: 9819

You might get better answers on superuser.com, as this question is a bit off-topic here.

But consider the possibility that the nfs share is not mounted with an async flag - in that case, each single write would need to be confirmed by the nfs server before the next write can evens start. So bs=1 would need about double time compared to bs=2, and each of them would be MUCH slower that a sensible block size.

If the async flag is set on the nfs mount, your kernel might merge several small writes into one big one anyway, so the effect of setting bs should be neglectable.

Anyway, if you're testing to set up an environment for a specific application, use that application for testing, nothing else. Performance might depend on so much application-specific behaviour that any generic tool won't be able to reproduce.

Upvotes: 1

Related Questions