All the mail mirrored from lore.kernel.org
 help / color / mirror / Atom feed

From: Michael Zugelder <michael@zugelder.org>
To: dm-crypt@saout.de
Subject: [dm-crypt] Read starvation during writeback
Date: Mon, 08 Oct 2012 00:33:20 +0200	[thread overview]
Message-ID: <1349649200.2196.6.camel@vpcz1> (raw)

Hi everyone,

I'm having an issue reading data via dm-crypt when there is heavy
writeback going on (i.e. copying large files or using dd). A single
read takes anywhere from a few seconds to multiple minutes.

Testing setup:
 * Fedora 17, stock 3.5.4-2.fc17 kernel and a self-compiled 3.6.1 kernel
 * 320 GiB USB hard drive (sdb)
 * dmsetup library version 1.02.74 (2012-03-06), driver version 4.23.0
 * dm-crypt was set up using cipher_null:
   echo "0 $(blockdev --getsize /dev/sdb) crypt cipher_null - 0 /dev/sdb 0"|dmsetup create test
 * seeker [1] to benchmark random read performance
 * writeback induced by running 'dd if=/dev/zero of=$target bs=1M'

Raw device performance (idle):
  55 seeks/second, 18.15 ms random access time

dm-crypt performance (idle):
  54 seeks/second, 18.33 ms random access time

Raw device performance (writeback):
  49 seeks/second, 20.35 ms random access time

dm-crypt performance (writeback):
  0 seeks/second, 3750.00 ms random access time
  0 seeks/second, 30000.00 ms random access time
  0 seeks/second, 30000.00 ms random access time
  3 seeks/second, 297.03 ms random access time
  47 seeks/second, 21.14 ms random access time
  15 seeks/second, 64.24 ms random access time
  0 seeks/second, 10000.00 ms random access time

Sometimes it almost works, but most of the time, the device is
completely unusable.

I experimented a bit with the other device mapper targets, namely linear
and stripe, but both worked completely fine. I also tried putting a
linear mapping above dm-crypt, with no impact on performance. Comparing
the content of the /sys/block/$DEV files of the linear mapping and
dm-crypt, there are no differences beside the name, dev no, stats,
uevent and inflight files.

But the inflight counters seem to provide some interesting data. When
writing to the raw device, there are 127--131 writes in flight and a
single read (seeker seems to use a queue depth of 1). But in the
dm-crypt case, there are about 160000--250000 writes in flight for the
mapping device, with 127--300 on the raw device. There is also a single
read but only at the dm-crypt device, the raw device stays at
0 reads the whole time. The linear mapping, in contrast, has "only" 
~3500--9000 writes in flight, and the reads actually reach the raw
device.

Any pointers would be appreciated, I haven't found much on the web about
this issue.

Keep up the good work.


Michael

[1] http://www.linuxinsight.com/files/seeker.c

                 reply	other threads:[~2012-10-07 22:33 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1349649200.2196.6.camel@vpcz1 \
    --to=michael@zugelder.org \
    --cc=dm-crypt@saout.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.