From: Jeff King <peff@peff.net>
To: Patrick Steinhardt <ps@pks.im>
Cc: Taylor Blau <me@ttaylorr.com>, git@vger.kernel.org
Subject: Re: [PATCH 2/2] receive-pack: use advertised reference tips to inform connectivity check
Date: Tue, 1 Nov 2022 04:28:25 -0400 [thread overview]
Message-ID: <Y2DYqZRlO7emYu8/@coredump.intra.peff.net> (raw)
In-Reply-To: <Y1/fm1prlAs3U1NE@ncase>
On Mon, Oct 31, 2022 at 03:45:47PM +0100, Patrick Steinhardt wrote:
> Indeed, I'm not trying to say that this can lead to repository
> corruption. If at all you can argue that this is more security-related.
> Suppose an object is not reachable from any public reference and that
> `allowAnySHA1InWant=false`. Then you could make these hidden objects
> reachable by sending a packfile with an object that references the
> hidden object. It naturally requires you to somehow know about the
> object ID, so I don't think this is a critical issue.
I'd have to double check, but isn't this all moot with the v2 protocol
anyway? I didn't think it even respected allowAnySHA1InWant.
Even that aside, there are other tricks you can do. E.g., pushing up an
object which claims to be a delta of X (though you have the trick then
of figuring out how to reference it), or pushing up a test object then
fetching it back while claiming to have X, treating the server as an
oracle which may give you a delta against X.
In short, I don't think it's worth considering unreachable objects in a
repository to be considered secret.
> > Why do we see a slowdown when there there aren't any hidden references?
> > Or am I misunderstanding your patch message which instead means "we see
> > a slow-down when there are no hidden references [since we still must
> > store and enumerate all advertised references]"?
>
> I have tried to dig down into the code of `revision.c` but ultimately
> returned empty-handed. I _think_ that this is because of the different
> paths we use when reading revisions from stdin as we have to resolve the
> revision to an OID first, which is more involved than taking the OIDs as
> returned by the reference backend. I have tried to short-circuit this
> logic in case the revision read from stdin is exactly `hash_algo->hexsz`
> long so that we try to parse it as an OID directly instead of trying to
> do any of the magic that is required to resolve a revision. But this
> only speed things up by a small margin.
>
> Another assumption was that this is overhead caused by using stdin
> instead of reading data from a file, but flame graphs didn't support
> this theory, either.
Certainly read_revisions_from_stdin() will allocate a bunch of extra
data per item it reads (via add_rev_cmdline() and add_pending), which is
going to be way more than parsing or stdin overhead.
Much worse is that it will call get_reference(), which is very keen to
actually open and parse the object (as you know, since you added
commit-graph handling there). That might have gotten a bit better in
v2.38.0 if you have any references to blobs (as we'd now skip the extra
hash check).
Of course the original "rev-list --not --all" would suffer from the same
thing, so some of that may not explain any difference between the two.
-Peff
next prev parent reply other threads:[~2022-11-01 8:29 UTC|newest]
Thread overview: 88+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-10-28 14:42 [PATCH 0/2] receive-pack: use advertised reference tips to inform connectivity check Patrick Steinhardt
2022-10-28 14:42 ` [PATCH 1/2] connected: allow supplying different view of reachable objects Patrick Steinhardt
2022-10-28 14:54 ` Ævar Arnfjörð Bjarmason
2022-10-28 18:12 ` Junio C Hamano
2022-10-30 18:49 ` Taylor Blau
2022-10-31 13:10 ` Patrick Steinhardt
2022-11-01 1:16 ` Taylor Blau
2022-10-28 14:42 ` [PATCH 2/2] receive-pack: use advertised reference tips to inform connectivity check Patrick Steinhardt
2022-10-28 15:01 ` Ævar Arnfjörð Bjarmason
2022-10-31 14:21 ` Patrick Steinhardt
2022-10-31 15:36 ` Ævar Arnfjörð Bjarmason
2022-10-30 19:09 ` Taylor Blau
2022-10-31 14:45 ` Patrick Steinhardt
2022-11-01 1:28 ` Taylor Blau
2022-11-01 7:20 ` Patrick Steinhardt
2022-11-01 11:53 ` Patrick Steinhardt
2022-11-02 1:05 ` Taylor Blau
2022-11-01 8:28 ` Jeff King [this message]
2022-10-28 16:40 ` [PATCH 0/2] " Junio C Hamano
2022-11-01 1:30 ` Taylor Blau
2022-11-01 9:00 ` Jeff King
2022-11-01 11:49 ` Patrick Steinhardt
2022-11-03 14:37 ` [PATCH v2 0/3] receive-pack: only use visible refs for " Patrick Steinhardt
2022-11-03 14:37 ` [PATCH v2 1/3] refs: get rid of global list of hidden refs Patrick Steinhardt
2022-11-03 14:37 ` [PATCH v2 2/3] revision: add new parameter to specify all visible refs Patrick Steinhardt
2022-11-05 12:46 ` Jeff King
2022-11-07 8:20 ` Patrick Steinhardt
2022-11-08 14:32 ` Jeff King
2022-11-05 12:55 ` Jeff King
2022-11-03 14:37 ` [PATCH v2 3/3] receive-pack: only use visible refs for connectivity check Patrick Steinhardt
2022-11-05 0:40 ` [PATCH v2 0/3] " Taylor Blau
2022-11-05 12:55 ` Jeff King
2022-11-05 12:52 ` Jeff King
2022-11-07 12:16 ` [PATCH v3 0/6] " Patrick Steinhardt
2022-11-07 12:16 ` [PATCH v3 1/6] refs: get rid of global list of hidden refs Patrick Steinhardt
2022-11-07 12:16 ` [PATCH v3 2/6] revision: move together exclusion-related functions Patrick Steinhardt
2022-11-07 12:16 ` [PATCH v3 3/6] revision: introduce struct to handle exclusions Patrick Steinhardt
2022-11-07 12:51 ` Ævar Arnfjörð Bjarmason
2022-11-08 9:11 ` Patrick Steinhardt
2022-11-07 12:16 ` [PATCH v3 4/6] revision: add new parameter to exclude hidden refs Patrick Steinhardt
2022-11-07 13:34 ` Ævar Arnfjörð Bjarmason
2022-11-07 17:07 ` Ævar Arnfjörð Bjarmason
2022-11-08 9:48 ` Patrick Steinhardt
2022-11-08 9:22 ` Patrick Steinhardt
2022-11-08 0:57 ` Taylor Blau
2022-11-08 8:16 ` Patrick Steinhardt
2022-11-08 14:42 ` Jeff King
2022-11-07 12:16 ` [PATCH v3 5/6] revparse: add `--exclude-hidden=` option Patrick Steinhardt
2022-11-08 14:44 ` Jeff King
2022-11-07 12:16 ` [PATCH v3 6/6] receive-pack: only use visible refs for connectivity check Patrick Steinhardt
2022-11-08 0:59 ` [PATCH v3 0/6] " Taylor Blau
2022-11-08 10:03 ` [PATCH v4 " Patrick Steinhardt
2022-11-08 10:03 ` [PATCH v4 1/6] refs: get rid of global list of hidden refs Patrick Steinhardt
2022-11-08 13:36 ` Ævar Arnfjörð Bjarmason
2022-11-08 14:49 ` Patrick Steinhardt
2022-11-08 14:51 ` Jeff King
2022-11-08 10:03 ` [PATCH v4 2/6] revision: move together exclusion-related functions Patrick Steinhardt
2022-11-08 10:03 ` [PATCH v4 3/6] revision: introduce struct to handle exclusions Patrick Steinhardt
2022-11-08 10:03 ` [PATCH v4 4/6] revision: add new parameter to exclude hidden refs Patrick Steinhardt
2022-11-08 15:07 ` Jeff King
2022-11-08 21:13 ` Taylor Blau
2022-11-11 5:48 ` Patrick Steinhardt
2022-11-08 10:03 ` [PATCH v4 5/6] rev-parse: add `--exclude-hidden=` option Patrick Steinhardt
2022-11-08 10:04 ` [PATCH v4 6/6] receive-pack: only use visible refs for connectivity check Patrick Steinhardt
2022-11-11 6:49 ` [PATCH v5 0/7] " Patrick Steinhardt
2022-11-11 6:49 ` [PATCH v5 1/7] refs: fix memory leak when parsing hideRefs config Patrick Steinhardt
2022-11-11 6:49 ` [PATCH v5 2/7] refs: get rid of global list of hidden refs Patrick Steinhardt
2022-11-11 6:50 ` [PATCH v5 3/7] revision: move together exclusion-related functions Patrick Steinhardt
2022-11-11 6:50 ` [PATCH v5 4/7] revision: introduce struct to handle exclusions Patrick Steinhardt
2022-11-11 6:50 ` [PATCH v5 5/7] revision: add new parameter to exclude hidden refs Patrick Steinhardt
2022-11-11 6:50 ` [PATCH v5 6/7] rev-parse: add `--exclude-hidden=` option Patrick Steinhardt
2022-11-11 6:50 ` [PATCH v5 7/7] receive-pack: only use visible refs for connectivity check Patrick Steinhardt
2022-11-11 22:18 ` [PATCH v5 0/7] " Taylor Blau
2022-11-15 17:26 ` Jeff King
2022-11-16 21:22 ` Taylor Blau
2022-11-16 22:04 ` Jeff King
2022-11-16 22:33 ` Taylor Blau
2022-11-17 5:45 ` Patrick Steinhardt
2022-11-17 5:46 ` [PATCH v6 " Patrick Steinhardt
2022-11-17 5:46 ` [PATCH v6 1/7] refs: fix memory leak when parsing hideRefs config Patrick Steinhardt
2022-11-17 5:46 ` [PATCH v6 2/7] refs: get rid of global list of hidden refs Patrick Steinhardt
2022-11-17 5:46 ` [PATCH v6 3/7] revision: move together exclusion-related functions Patrick Steinhardt
2022-11-17 5:46 ` [PATCH v6 4/7] revision: introduce struct to handle exclusions Patrick Steinhardt
2022-11-17 5:46 ` [PATCH v6 5/7] revision: add new parameter to exclude hidden refs Patrick Steinhardt
2022-11-17 5:47 ` [PATCH v6 6/7] rev-parse: add `--exclude-hidden=` option Patrick Steinhardt
2022-11-17 5:47 ` [PATCH v6 7/7] receive-pack: only use visible refs for connectivity check Patrick Steinhardt
2022-11-17 15:03 ` [PATCH v6 0/7] " Jeff King
2022-11-17 21:24 ` Taylor Blau
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Y2DYqZRlO7emYu8/@coredump.intra.peff.net \
--to=peff@peff.net \
--cc=git@vger.kernel.org \
--cc=me@ttaylorr.com \
--cc=ps@pks.im \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).