From: Sergey Senozhatsky <senozhatsky@chromium.org>
To: Andrew Morton <akpm@linux-foundation.org>,
Minchan Kim <minchan@kernel.org>
Cc: linux-kernel@vger.kernel.org, linux-block@vger.kernel.org,
Sergey Senozhatsky <senozhatsky@chromium.org>
Subject: [PATCHv2 00/17] zram: convert to custom comp API and allow algorithms configuration
Date: Mon, 6 May 2024 16:58:13 +0900 [thread overview]
Message-ID: <20240506075834.302472-1-senozhatsky@chromium.org> (raw)
This patch set moves zram from crypto API to a custom compression
API which allows us to tune and configure compression algorithms,
something that crypto API, unfortunately, doesn't support. Basically,
this seroes brings back the bits of comp "backend" code that we had
many years ago. This means that if we want zram to support new
compression algorithms we need to implement corresponding backends.
Currently, zram supports a pretty decent number of comp backends:
lzo, lzorle, lz4, lz4hc, 842, deflate, zstd
At this point we handle 2 parameters: a compression level and
a pre-trained compression dictionary. Which seems like a good enough
start. The list will be extended in the future.
Examples:
- changes default compression level
echo "algo=zstd level=11" > /sys/block/zram0/comp_algorithm
- passes path to a pre-trained dictionary
echo "algo=zstd dict=/etc/dictionary" > /sys/block/zram0/comp_algorithm
TEST
====
using default zstd
/sys/block/zram0/mm_stat
1750315008 504602831 514256896 0 514256896 1 0 34204 34204
using zstd level=7 dict=/etc/dictionary
/sys/block/zram0/mm_stat
1750310912 432540606 441712640 0 441712640 1 0 34187 34187
v2:
-- forcibly select LZO when nono backeneds were selected (Masahiro Yamada)
-- add 842 compression backend
-- added comp config preprocessing and caching
-- reworked zstd dict handling
Sergey Senozhatsky (17):
zram: move from crypto API to custom comp backends API
zram: add lzo and lzorle compression backends support
zram: add lz4 compression backend support
zram: add lz4hc compression backend support
zram: add zstd compression backend support
zram: pass estimated src size hint to zstd
zram: add zlib compression backend support
zram: add 842 compression backend support
zram: check that backends array has at least one backend
zram: introduce zcomp_config structure
zram: extend comp_algorithm attr write handling
zram: support compression level comp config
zram: add support for dict comp config
zram: add dictionary support to zstd backend
zram: add config init/release backend callbacks
zram: share dictionaries between per-CPU contexts
Documentation/zram: add documentation for algorithm parameters
Documentation/admin-guide/blockdev/zram.rst | 38 +++-
drivers/block/zram/Kconfig | 76 +++++--
drivers/block/zram/Makefile | 8 +
drivers/block/zram/backend_842.c | 79 +++++++
drivers/block/zram/backend_842.h | 10 +
drivers/block/zram/backend_deflate.c | 143 +++++++++++++
drivers/block/zram/backend_deflate.h | 10 +
drivers/block/zram/backend_lz4.c | 58 +++++
drivers/block/zram/backend_lz4.h | 10 +
drivers/block/zram/backend_lz4hc.c | 84 ++++++++
drivers/block/zram/backend_lz4hc.h | 10 +
drivers/block/zram/backend_lzo.c | 55 +++++
drivers/block/zram/backend_lzo.h | 10 +
drivers/block/zram/backend_lzorle.c | 55 +++++
drivers/block/zram/backend_lzorle.h | 10 +
drivers/block/zram/backend_zstd.c | 221 ++++++++++++++++++++
drivers/block/zram/backend_zstd.h | 10 +
drivers/block/zram/zcomp.c | 155 +++++++-------
drivers/block/zram/zcomp.h | 42 +++-
drivers/block/zram/zram_drv.c | 120 ++++++++++-
drivers/block/zram/zram_drv.h | 1 +
21 files changed, 1094 insertions(+), 111 deletions(-)
create mode 100644 drivers/block/zram/backend_842.c
create mode 100644 drivers/block/zram/backend_842.h
create mode 100644 drivers/block/zram/backend_deflate.c
create mode 100644 drivers/block/zram/backend_deflate.h
create mode 100644 drivers/block/zram/backend_lz4.c
create mode 100644 drivers/block/zram/backend_lz4.h
create mode 100644 drivers/block/zram/backend_lz4hc.c
create mode 100644 drivers/block/zram/backend_lz4hc.h
create mode 100644 drivers/block/zram/backend_lzo.c
create mode 100644 drivers/block/zram/backend_lzo.h
create mode 100644 drivers/block/zram/backend_lzorle.c
create mode 100644 drivers/block/zram/backend_lzorle.h
create mode 100644 drivers/block/zram/backend_zstd.c
create mode 100644 drivers/block/zram/backend_zstd.h
--
2.45.0.rc1.225.g2a3ae87e7f-goog
next reply other threads:[~2024-05-06 7:58 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-06 7:58 Sergey Senozhatsky [this message]
2024-05-06 7:58 ` [PATCHv2 01/17] zram: move from crypto API to custom comp backends API Sergey Senozhatsky
2024-05-06 7:58 ` [PATCHv2 02/17] zram: add lzo and lzorle compression backends support Sergey Senozhatsky
2024-05-06 7:58 ` [PATCHv2 03/17] zram: add lz4 compression backend support Sergey Senozhatsky
2024-05-06 7:58 ` [PATCHv2 04/17] zram: add lz4hc " Sergey Senozhatsky
2024-05-06 7:58 ` [PATCHv2 05/17] zram: add zstd " Sergey Senozhatsky
2024-05-06 7:58 ` [PATCHv2 06/17] zram: pass estimated src size hint to zstd Sergey Senozhatsky
2024-05-06 7:58 ` [PATCHv2 07/17] zram: add zlib compression backend support Sergey Senozhatsky
2024-05-06 7:58 ` [PATCHv2 08/17] zram: add 842 " Sergey Senozhatsky
2024-05-06 7:58 ` [PATCHv2 09/17] zram: check that backends array has at least one backend Sergey Senozhatsky
2024-05-06 7:58 ` [PATCHv2 10/17] zram: introduce zcomp_config structure Sergey Senozhatsky
2024-05-06 7:58 ` [PATCHv2 11/17] zram: extend comp_algorithm attr write handling Sergey Senozhatsky
2024-05-06 7:58 ` [PATCHv2 12/17] zram: support compression level comp config Sergey Senozhatsky
2024-05-06 7:58 ` [PATCHv2 13/17] zram: add support for dict " Sergey Senozhatsky
2024-05-06 7:58 ` [PATCHv2 14/17] zram: add dictionary support to zstd backend Sergey Senozhatsky
2024-05-06 7:58 ` [PATCHv2 15/17] zram: add config init/release backend callbacks Sergey Senozhatsky
2024-05-06 7:58 ` [PATCHv2 16/17] zram: share dictionaries between per-CPU contexts Sergey Senozhatsky
2024-05-06 7:58 ` [PATCHv2 17/17] Documentation/zram: add documentation for algorithm parameters Sergey Senozhatsky
2024-05-08 7:41 ` [PATCHv2 00/17] zram: convert to custom comp API and allow algorithms configuration Sergey Senozhatsky
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240506075834.302472-1-senozhatsky@chromium.org \
--to=senozhatsky@chromium.org \
--cc=akpm@linux-foundation.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=minchan@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).