CODEWITHSUNDEEP
linuxdiskext3
Wei Dengos
Wei Dengos

Reputation: 1

linux 2.6.43, ext3, 10K RPM SAS disk, 2 sequential write(direct io) on different file acting like random write

I recently stall on this one problem:

"2 sequential write(direct io 4KB alignemnt block) on different file acting like random write, which yield poor write performance in 10K RPM SAS disk".

The thing confuse me most: I got batch of server, all equip with same kind of disk (raid 1 with 2 300GB 10K RPM disk), but response different.

  • several servers seams ok with this kind of write pattern, disk happy accepted up to 50+MB/s; (same kernel version, same filesystem, with different lib (libc 2.4))
  • others not so much, 100 op/s seams reach the limit of underlying disk, which confirm the random write performance of disk; ((same kernel version, same filesystem, with different lib (libc 2.12)))

[NOTE: I check the "pwrite" code of different libc, which tell nothing but simple "syscall"]

I have managed to rule out the possibly: 1. software bug in my own program; by a simple deamon(compile with no dynamic link), do sequcetial direct io write; 2. disk problem; switch 2 different version of linux system on one test machine, which perform well on my direct io write pattern, and a couple of day after switch to old lib version, the bad random write;

I try to compare:

  1. /sys/block/sda/queue/*, which may different in both way;
  2. filefrag show nothing but two different file interleaved sequenctial grow physical block id;

there must be some kind of write strategy lead to this problem, but i don't know where to start:

  1. different kernel setting ?, may be related to how ext3 allocate disk block ?
  2. raid cache(write back) or disk cache write strategy?
  3. or underlying disk strategy to mapping logical block into real physical block ?

really appreciate

Upvotes: 0

Views: 68

Answers (1)

Wei Dengos
Wei Dengos

Reputation: 1

THE ANS IS: it's because of /sys/block/sda/queue/schedule setting:

  1. MACHINE A: display schedule: cfq, but undlying, it's deadline;
  2. MACHINE B: the schedule is consistent with cfq; //=> SINCE my server is db svr, deadline is my best option;

Upvotes: 0

Related Questions