From: Ian Rogers <irogers@google.com>
To: Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Namhyung Kim <namhyung@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Jiri Olsa <jolsa@kernel.org>, Ian Rogers <irogers@google.com>,
Adrian Hunter <adrian.hunter@intel.com>,
Kan Liang <kan.liang@linux.intel.com>,
James Clark <james.clark@arm.com>,
Athira Rajeev <atrajeev@linux.vnet.ibm.com>,
Colin Ian King <colin.i.king@gmail.com>,
nabijaczleweli@nabijaczleweli.xyz, Leo Yan <leo.yan@linux.dev>,
Song Liu <song@kernel.org>,
Ilkka Koskinen <ilkka@os.amperecomputing.com>,
Ben Gainey <ben.gainey@arm.com>,
K Prateek Nayak <kprateek.nayak@amd.com>,
Yanteng Si <siyanteng@loongson.cn>,
Sun Haiyong <sunhaiyong@loongson.cn>,
Changbin Du <changbin.du@huawei.com>,
Andi Kleen <ak@linux.intel.com>,
Thomas Richter <tmricht@linux.ibm.com>,
Masami Hiramatsu <mhiramat@kernel.org>,
Dima Kogan <dima@secretsauce.net>,
zhaimingbing <zhaimingbing@cmss.chinamobile.com>,
Paran Lee <p4ranlee@gmail.com>, Li Dong <lidong@vivo.com>,
Tiezhu Yang <yangtiezhu@loongson.cn>,
Chengen Du <chengen.du@canonical.com>,
linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH v6 0/7] dso/dsos memory savings and clean up
Date: Sat, 4 May 2024 14:37:56 -0700 [thread overview]
Message-ID: <20240504213803.218974-1-irogers@google.com> (raw)
7 more patches from:
https://lore.kernel.org/lkml/20240202061532.1939474-1-irogers@google.com/
a near half year old adventure in trying to lower perf's dynamic
memory use. Bits like the memory overhead of opendir are on the
sidelines for now, too much fighting over how
distributions/C-libraries present getdents. These changes are more
good old fashioned replace an rb-tree with a sorted array and add
reference count tracking.
The changes migrate dsos code, the collection of dso structs, more
into the dsos.c/dsos.h files. As with maps and threads, this is done
so the internals can be changed - replacing a linked list (for fast
iteration) and an rb-tree (for fast finds) with a lazily sorted
array. The complexity of operations remain roughly the same, although
iterating an array is likely faster than iterating a linked list, the
memory usage is at least reduced by half.
As fixing the memory usage necessitates changing operations like find,
modify these operations so that they increment the reference count to
avoid races like a find in dsos and a remove. Similarly tighten up
lock usage so that operations working on dsos state hold the
appropriate lock. Note, since this series is partially applied in the
perf-tools-next tree currently some memory leaks have been introduced.
v5. Rebase, adding use of accessors to dso as necessary. Previous
versions were all rebases or dropping merged patches.
v6. Rebase, move dsos__purge NULL assignment to dso->dsos to above
dso__put to avoid a warning reported by Arnaldo. This was part of
patch 5 in the v5 series.
Ian Rogers (7):
perf dsos: Switch backing storage to array from rbtree/list
perf dsos: Remove __dsos__addnew
perf dsos: Remove __dsos__findnew_link_by_longname_id
perf dsos: Switch hand code to bsearch
perf dso: Add reference count checking and accessor functions
perf dso: Reference counting related fixes
perf dso: Use container_of to avoid a pointer in dso_data
tools/perf/builtin-annotate.c | 6 +-
tools/perf/builtin-buildid-cache.c | 2 +-
tools/perf/builtin-buildid-list.c | 18 +-
tools/perf/builtin-inject.c | 71 ++-
tools/perf/builtin-kallsyms.c | 2 +-
tools/perf/builtin-mem.c | 4 +-
tools/perf/builtin-report.c | 6 +-
tools/perf/builtin-script.c | 8 +-
tools/perf/builtin-top.c | 4 +-
tools/perf/builtin-trace.c | 2 +-
tools/perf/tests/code-reading.c | 8 +-
tools/perf/tests/dso-data.c | 67 ++-
tools/perf/tests/hists_common.c | 6 +-
tools/perf/tests/hists_cumulate.c | 4 +-
tools/perf/tests/hists_output.c | 2 +-
tools/perf/tests/maps.c | 4 +-
tools/perf/tests/symbols.c | 8 +-
tools/perf/tests/vmlinux-kallsyms.c | 6 +-
tools/perf/ui/browsers/annotate.c | 6 +-
tools/perf/ui/browsers/hists.c | 8 +-
tools/perf/ui/browsers/map.c | 4 +-
tools/perf/util/annotate-data.c | 18 +-
tools/perf/util/annotate.c | 17 +-
tools/perf/util/auxtrace.c | 2 +-
tools/perf/util/block-info.c | 2 +-
tools/perf/util/bpf-event.c | 8 +-
tools/perf/util/build-id.c | 38 +-
tools/perf/util/callchain.c | 2 +-
tools/perf/util/data-convert-json.c | 2 +-
tools/perf/util/db-export.c | 6 +-
tools/perf/util/disasm.c | 40 +-
tools/perf/util/dlfilter.c | 12 +-
tools/perf/util/dso.c | 429 ++++++++-------
tools/perf/util/dso.h | 500 ++++++++++++++++--
tools/perf/util/dsos.c | 286 +++++-----
tools/perf/util/dsos.h | 18 +-
tools/perf/util/event.c | 8 +-
tools/perf/util/header.c | 8 +-
tools/perf/util/hist.c | 4 +-
tools/perf/util/intel-pt.c | 22 +-
tools/perf/util/machine.c | 50 +-
tools/perf/util/map.c | 78 +--
tools/perf/util/maps.c | 14 +-
tools/perf/util/print_insn.c | 2 +-
tools/perf/util/probe-event.c | 25 +-
.../util/scripting-engines/trace-event-perl.c | 6 +-
.../scripting-engines/trace-event-python.c | 21 +-
tools/perf/util/sort.c | 19 +-
tools/perf/util/srcline.c | 65 +--
tools/perf/util/symbol-elf.c | 145 +++--
tools/perf/util/symbol-minimal.c | 4 +-
tools/perf/util/symbol.c | 186 +++----
tools/perf/util/symbol_fprintf.c | 4 +-
tools/perf/util/synthetic-events.c | 24 +-
tools/perf/util/thread.c | 4 +-
tools/perf/util/unwind-libunwind-local.c | 18 +-
tools/perf/util/unwind-libunwind.c | 2 +-
tools/perf/util/vdso.c | 8 +-
58 files changed, 1410 insertions(+), 933 deletions(-)
--
2.45.0.rc1.225.g2a3ae87e7f-goog
next reply other threads:[~2024-05-04 21:38 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-04 21:37 Ian Rogers [this message]
2024-05-04 21:37 ` [PATCH v6 1/7] perf dsos: Switch backing storage to array from rbtree/list Ian Rogers
2024-05-04 21:37 ` [PATCH v6 2/7] perf dsos: Remove __dsos__addnew Ian Rogers
2024-05-04 21:37 ` [PATCH v6 3/7] perf dsos: Remove __dsos__findnew_link_by_longname_id Ian Rogers
2024-05-04 21:38 ` [PATCH v6 4/7] perf dsos: Switch hand code to bsearch Ian Rogers
2024-05-04 21:38 ` [PATCH v6 5/7] perf dso: Add reference count checking and accessor functions Ian Rogers
2024-05-04 21:38 ` [PATCH v6 6/7] perf dso: Reference counting related fixes Ian Rogers
2024-05-06 14:44 ` Arnaldo Carvalho de Melo
2024-05-06 15:26 ` Ian Rogers
2024-05-06 18:35 ` Arnaldo Carvalho de Melo
2024-05-04 21:38 ` [PATCH v6 7/7] perf dso: Use container_of to avoid a pointer in dso_data Ian Rogers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240504213803.218974-1-irogers@google.com \
--to=irogers@google.com \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=ak@linux.intel.com \
--cc=alexander.shishkin@linux.intel.com \
--cc=atrajeev@linux.vnet.ibm.com \
--cc=ben.gainey@arm.com \
--cc=changbin.du@huawei.com \
--cc=chengen.du@canonical.com \
--cc=colin.i.king@gmail.com \
--cc=dima@secretsauce.net \
--cc=ilkka@os.amperecomputing.com \
--cc=james.clark@arm.com \
--cc=jolsa@kernel.org \
--cc=kan.liang@linux.intel.com \
--cc=kprateek.nayak@amd.com \
--cc=leo.yan@linux.dev \
--cc=lidong@vivo.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mhiramat@kernel.org \
--cc=mingo@redhat.com \
--cc=nabijaczleweli@nabijaczleweli.xyz \
--cc=namhyung@kernel.org \
--cc=p4ranlee@gmail.com \
--cc=peterz@infradead.org \
--cc=siyanteng@loongson.cn \
--cc=song@kernel.org \
--cc=sunhaiyong@loongson.cn \
--cc=tmricht@linux.ibm.com \
--cc=yangtiezhu@loongson.cn \
--cc=zhaimingbing@cmss.chinamobile.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).