From: Tao Klerks <tao@klerks.biz>
To: git <git@vger.kernel.org>
Subject: Windows: core.useBuiltinFSMonitor without core.untrackedcache - performance hazard?
Date: Thu, 10 Jun 2021 12:24:04 +0200 [thread overview]
Message-ID: <CAPMMpog7bNNPm3suZKu6OppHA+KDYgCfmaxW4HqTAr7_tTVAPQ@mail.gmail.com> (raw)
Hi folks,
With the new "core.useBuiltinFSMonitor" support in the Windows
installer, I think this subject is worth calling out explicitly (and
my apologies if I missed prior discussion):
TL;DR:
- I believe "core.untrackedcache" should be enabled by default in
Windows (and it does not appear to be)
- If a user enables "core.useBuiltinFSMonitor" (eg in the installer)
in the hopes of getting snappy "git status" on a repo with a large
deep working tree, they will be *unnecessarily* disappointed if
"core.untrackedcache" is not enabled
- There is also a lingering "problem" with "git status -uall", with
both "core.useBuiltinFSMonitor" and "core.fsmonitor", but that seems
far less trivial to address
Detail:
I just started testing the new "core.useBuiltinFSMonitor" option in
the new installer, and it's amazing, thanks Ben, Alex, Johannes and
Kevin!
However, when I first enabled it, I was getting slightly *worse* git
status times than without it... and those worse git status times were
accompanied by a message along the lines of:
---
It took 5.88 seconds to enumerate untracked files. 'status -uno' may
speed it up, but you have to be careful not to forget to add new files
yourself (see 'git help status').
---
For context, this is in a repo with 200,000 or so files, within 40,000
folders (avg path depth 4 I think?), with a reasonably-intricate set
of .gitignore patterns. Obviously that's not "your average user", but
I would imagine it matches "the target audience for
'core.useBuiltinFSMonitor'" pretty well.
After a little head-scratching, I recalled an exchange with Johannes
from last year:
https://lore.kernel.org/git/CAPMMpohJicVeCaKsPvommYbGEH-D1V02TTMaiVTV8ux+9z9vkQ@mail.gmail.com/
I never did understand the relevant code paths in much detail, but the
practical conclusions were:
- Without "core.untrackedcache" enabled, git ends up iterating
through the entire path structure of the working tree *even if
"core.fsmonitor" (and now "core.useBuiltinFSMonitor") is enabled*,
looking for untracked files to report
- Even with "core.untrackedcache" enabled, if "core.fsmonitor" (and
now "core.useBuiltinFSMonitor") is enabled, git iterates through the
entire path structure of the working tree *single-threaded* when the
"--untracked-files" mode is set to "all" (by config or command-line)
Now, I imagine that addressing/improving these behaviors is very
non-trivial, but the impact could be reasonably limited if:
- core.untrackedcache were defaulted to "true", at least under
Windows, at least when the installer is asked to set
core.useBuiltinFSMonitor
- The "It took N.NN seconds to enumerate untracked files" message
were to include a hint about core.untrackedcache, at least when the
"--untracked-files" mode is set to "normal".
Final note: I personally would love to see "core.useBuiltinFSMonitor
actually makes things slower, when --untracked-files=all is specified"
behavior be addressed, because common windows git integrations or
front-ends like Git Extensions or IntelliJ IDEA commonly use those
options, and therefore "suffer" a performance degradation on at least
some operations when core.useBuiltinFSMonitor is enabled.
I don't know whether this is the right place to report Windows-centric
concerns, if not, my apologies.
Thanks,
Tao
next reply other threads:[~2021-06-10 10:25 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-10 10:24 Tao Klerks [this message]
2021-06-11 9:49 ` Windows: core.useBuiltinFSMonitor without core.untrackedcache - performance hazard? Johannes Schindelin
2021-06-21 12:50 ` Tao Klerks
2021-06-21 18:41 ` Jeff Hostetler
2021-06-21 20:52 ` Tao Klerks
2021-06-24 18:51 ` Tao Klerks
2021-06-24 5:25 ` Tao Klerks
2021-06-24 13:10 ` Jeff Hostetler
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAPMMpog7bNNPm3suZKu6OppHA+KDYgCfmaxW4HqTAr7_tTVAPQ@mail.gmail.com \
--to=tao@klerks.biz \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.