From: Taylor Blau <me@ttaylorr.com>
To: Jeff King <peff@peff.net>
Cc: git@vger.kernel.org, Chris Torek <chris.torek@gmail.com>,
Derrick Stolee <derrickstolee@github.com>,
Junio C Hamano <gitster@pobox.com>,
Patrick Steinhardt <ps@pks.im>
Subject: Re: [PATCH v4 12/16] refs/packed-backend.c: ignore complicated hidden refs rules
Date: Tue, 4 Jul 2023 14:22:10 -0400 [thread overview]
Message-ID: <ZKRjUkN/ggLBeIUu@nand.local> (raw)
In-Reply-To: <20230703061839.GH3502534@coredump.intra.peff.net>
On Mon, Jul 03, 2023 at 02:18:39AM -0400, Jeff King wrote:
> On Tue, Jun 20, 2023 at 10:22:02AM -0400, Taylor Blau wrote:
>
> > In subsequent commits, we'll teach `receive-pack` and `upload-pack` to
> > use the new jump list feature in the packed-refs iterator by ignoring
> > references which are mentioned via its respective hideRefs lists.
> >
> > However, the packed-ref jump lists cannot handle un-hiding rules (that
> > begin with '!'), or namespace comparisons (that begin with '^'). Detect
> > and avoid these cases by falling back to the normal enumeration without
> > a jump list when such patterns exist.
>
> I'm a fan of punting on such cases to keep things simple and
> incremental. But the location here seems weird to me:
>
> > diff --git a/refs/packed-backend.c b/refs/packed-backend.c
> > index 80b877e00c..2aeec5c601 100644
> > --- a/refs/packed-backend.c
> > +++ b/refs/packed-backend.c
> > @@ -1008,6 +1008,25 @@ static void populate_excluded_jump_list(struct packed_ref_iterator *iter,
> > if (!excluded_patterns)
> > return;
> >
> > + for (pattern = excluded_patterns; *pattern; pattern++) {
> > + /*
> > + * We also can't feed any excludes from hidden refs
> > + * config sections, since later rules may override
> > + * previous ones. For example, with rules "refs/foo" and
> > + * "!refs/foo/bar", we should show "refs/foo/bar" (and
> > + * everything underneath it), but the earlier exclusion
> > + * would cause us to skip all of "refs/foo". We likewise
> > + * don't implement the namespace stripping required for
> > + * '^' rules.
> > + *
> > + * Both are possible to do, but complicated, so avoid
> > + * populating the jump list at all if we see either of
> > + * these patterns.
> > + */
> > + if (**pattern == '!' || **pattern == '^')
> > + return;
> > + }
> > +
>
> This is deep in the packed-refs code, but the magic of "!" and "^" are
> specific to ref_is_hidden().
>
> So if I did:
>
> git for-each-ref --exclude='!refs/heads/foo'
>
> my understanding is that "!" would _not_ have an affect normally, but
> now it is turning off this optimization.
Yeah, that makes sense -- I agree that it is silly to have a reference
with "!" at the beginning, but since it's allowed we should absolutely
support it.
> Something along the lines of (you'd want a similar tweak for
> upload-pack):
Yep. Here's the extra tweak for upload-pack:
--- 8< ---
commit 5a8902731b91a8fc6900586968a79ebc6272e502
Author: Taylor Blau <me@ttaylorr.com>
Date: Tue Jul 4 14:21:22 2023 -0400
fixup! upload-pack.c: avoid enumerating hidden refs where possible
diff --git a/upload-pack.c b/upload-pack.c
index 3a176a7209..ef2ca36feb 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -610,6 +610,7 @@ static int allow_hidden_refs(enum allow_uor allow_uor)
static void for_each_namespaced_ref_1(each_ref_fn fn,
struct upload_pack_data *data)
{
+ const char **excludes = NULL;
/*
* If `data->allow_uor` allows fetching hidden refs, we need to
* mark all references (including hidden ones), to check in
@@ -619,12 +620,13 @@ static void for_each_namespaced_ref_1(each_ref_fn fn,
* has the OUR_REF bit set or not, so do not need to visit
* hidden references.
*/
- if (allow_hidden_refs(data->allow_uor))
- for_each_namespaced_ref(NULL, fn, data);
- else
- for_each_namespaced_ref(data->hidden_refs.v, fn, data);
+ if (!allow_hidden_refs(data->allow_uor))
+ excludes = hidden_refs_to_excludes(&data->hidden_refs);
+
+ for_each_namespaced_ref(excludes, fn, data);
}
+
static int is_our_ref(struct object *o, enum allow_uor allow_uor)
{
return o->flags & ((allow_hidden_refs(allow_uor) ? HIDDEN_REF : 0) | OUR_REF);
--- >8 ---
Thanks,
Taylor
next prev parent reply other threads:[~2023-07-04 18:22 UTC|newest]
Thread overview: 149+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-08 21:59 [PATCH 00/15] refs: implement skip lists for packed backend Taylor Blau
2023-05-08 21:59 ` [PATCH 01/15] refs.c: rename `ref_filter` Taylor Blau
2023-05-08 21:59 ` [PATCH 02/15] ref-filter.h: provide `REF_FILTER_INIT` Taylor Blau
2023-05-08 21:59 ` [PATCH 03/15] ref-filter: clear reachable list pointers after freeing Taylor Blau
2023-05-08 21:59 ` [PATCH 04/15] ref-filter: add ref_filter_clear() Taylor Blau
2023-05-08 22:29 ` Junio C Hamano
2023-05-08 22:33 ` Taylor Blau
2023-05-09 15:14 ` Patrick Steinhardt
2023-05-09 19:11 ` Taylor Blau
2023-05-08 21:59 ` [PATCH 05/15] ref-filter.c: parameterize match functions over patterns Taylor Blau
2023-05-08 22:36 ` Junio C Hamano
2023-05-09 20:13 ` Taylor Blau
2023-05-08 21:59 ` [PATCH 06/15] builtin/for-each-ref.c: add `--exclude` option Taylor Blau
2023-05-08 23:22 ` Junio C Hamano
2023-05-09 20:22 ` Taylor Blau
2023-05-08 22:00 ` [PATCH 07/15] refs: plumb `exclude_patterns` argument throughout Taylor Blau
2023-05-09 15:14 ` Patrick Steinhardt
2023-05-09 20:23 ` Taylor Blau
2023-05-08 22:00 ` [PATCH 08/15] refs/packed-backend.c: refactor `find_reference_location()` Taylor Blau
2023-05-08 23:56 ` Junio C Hamano
2023-05-09 20:29 ` Taylor Blau
2023-05-08 22:00 ` [PATCH 09/15] refs/packed-backend.c: implement skip lists to avoid excluded pattern(s) Taylor Blau
2023-05-09 0:10 ` Chris Torek
2023-05-09 20:39 ` Taylor Blau
2023-05-09 15:15 ` Patrick Steinhardt
2023-05-09 20:55 ` Taylor Blau
2023-05-09 21:15 ` Taylor Blau
2023-05-10 7:25 ` Patrick Steinhardt
2023-05-09 23:40 ` Junio C Hamano
2023-05-10 2:30 ` Taylor Blau
2023-05-08 22:00 ` [PATCH 10/15] refs/packed-backend.c: add trace2 counters for skip list Taylor Blau
2023-05-08 22:00 ` [PATCH 11/15] revision.h: store hidden refs in a `strvec` Taylor Blau
2023-05-08 22:00 ` [PATCH 12/15] refs/packed-backend.c: ignore complicated hidden refs rules Taylor Blau
2023-05-08 22:00 ` [PATCH 13/15] refs.h: let `for_each_namespaced_ref()` take excluded patterns Taylor Blau
2023-05-08 22:00 ` [PATCH 14/15] upload-pack.c: avoid enumerating hidden refs where possible Taylor Blau
2023-05-09 15:15 ` Patrick Steinhardt
2023-05-09 21:34 ` Taylor Blau
2023-05-08 22:00 ` [PATCH 15/15] builtin/receive-pack.c: avoid enumerating hidden references Taylor Blau
2023-05-15 19:23 ` [PATCH v2 00/16] refs: implement jump lists for packed backend Taylor Blau
2023-05-15 19:23 ` [PATCH v2 01/16] refs.c: rename `ref_filter` Taylor Blau
2023-05-15 19:23 ` [PATCH v2 02/16] ref-filter.h: provide `REF_FILTER_INIT` Taylor Blau
2023-05-15 19:23 ` [PATCH v2 03/16] ref-filter: clear reachable list pointers after freeing Taylor Blau
2023-05-15 19:23 ` [PATCH v2 04/16] ref-filter: add `ref_filter_clear()` Taylor Blau
2023-05-15 19:23 ` [PATCH v2 05/16] ref-filter.c: parameterize match functions over patterns Taylor Blau
2023-05-15 19:23 ` [PATCH v2 06/16] builtin/for-each-ref.c: add `--exclude` option Taylor Blau
2023-05-15 19:23 ` [PATCH v2 07/16] refs: plumb `exclude_patterns` argument throughout Taylor Blau
2023-05-15 19:23 ` [PATCH v2 08/16] refs/packed-backend.c: refactor `find_reference_location()` Taylor Blau
2023-05-15 19:23 ` [PATCH v2 09/16] refs/packed-backend.c: implement jump lists to avoid excluded pattern(s) Taylor Blau
2023-06-06 7:00 ` Patrick Steinhardt
2023-06-20 12:15 ` Taylor Blau
2023-05-15 19:23 ` [PATCH v2 10/16] refs/packed-backend.c: add trace2 counters for jump list Taylor Blau
2023-05-15 19:23 ` [PATCH v2 11/16] revision.h: store hidden refs in a `strvec` Taylor Blau
2023-06-06 7:00 ` Patrick Steinhardt
2023-06-20 12:16 ` Taylor Blau
2023-05-15 19:23 ` [PATCH v2 12/16] refs/packed-backend.c: ignore complicated hidden refs rules Taylor Blau
2023-05-15 19:23 ` [PATCH v2 13/16] refs.h: let `for_each_namespaced_ref()` take excluded patterns Taylor Blau
2023-06-06 7:01 ` Patrick Steinhardt
2023-06-20 12:18 ` Taylor Blau
2023-05-15 19:23 ` [PATCH v2 14/16] builtin/receive-pack.c: avoid enumerating hidden references Taylor Blau
2023-05-15 19:23 ` [PATCH v2 15/16] upload-pack.c: avoid enumerating hidden refs where possible Taylor Blau
2023-05-15 19:23 ` [PATCH v2 16/16] ls-refs.c: " Taylor Blau
2023-06-06 7:01 ` [PATCH v2 00/16] refs: implement jump lists for packed backend Patrick Steinhardt
2023-06-20 12:22 ` Taylor Blau
2023-06-07 10:40 ` [PATCH v3 " Taylor Blau
2023-06-07 10:40 ` [PATCH v3 01/16] refs.c: rename `ref_filter` Taylor Blau
2023-06-13 22:19 ` Junio C Hamano
2023-06-07 10:40 ` [PATCH v3 02/16] ref-filter.h: provide `REF_FILTER_INIT` Taylor Blau
2023-06-07 10:41 ` [PATCH v3 03/16] ref-filter: clear reachable list pointers after freeing Taylor Blau
2023-06-07 10:41 ` [PATCH v3 04/16] ref-filter: add `ref_filter_clear()` Taylor Blau
2023-06-07 10:41 ` [PATCH v3 05/16] ref-filter.c: parameterize match functions over patterns Taylor Blau
2023-06-13 22:37 ` Junio C Hamano
2023-06-07 10:41 ` [PATCH v3 06/16] builtin/for-each-ref.c: add `--exclude` option Taylor Blau
2023-06-07 10:41 ` [PATCH v3 07/16] refs: plumb `exclude_patterns` argument throughout Taylor Blau
2023-06-13 23:42 ` Junio C Hamano
2023-06-20 11:52 ` Taylor Blau
2023-06-07 10:41 ` [PATCH v3 08/16] refs/packed-backend.c: refactor `find_reference_location()` Taylor Blau
2023-06-07 10:41 ` [PATCH v3 09/16] refs/packed-backend.c: implement jump lists to avoid excluded pattern(s) Taylor Blau
2023-06-14 0:27 ` Junio C Hamano
2023-06-20 12:05 ` Taylor Blau
2023-06-20 18:49 ` Junio C Hamano
2023-06-07 10:41 ` [PATCH v3 10/16] refs/packed-backend.c: add trace2 counters for jump list Taylor Blau
2023-06-14 0:32 ` Junio C Hamano
2023-06-20 12:08 ` Taylor Blau
2023-06-07 10:41 ` [PATCH v3 11/16] revision.h: store hidden refs in a `strvec` Taylor Blau
2023-06-07 10:41 ` [PATCH v3 12/16] refs/packed-backend.c: ignore complicated hidden refs rules Taylor Blau
2023-06-14 0:40 ` Junio C Hamano
2023-06-07 10:41 ` [PATCH v3 13/16] refs.h: let `for_each_namespaced_ref()` take excluded patterns Taylor Blau
2023-06-07 10:42 ` [PATCH v3 14/16] builtin/receive-pack.c: avoid enumerating hidden references Taylor Blau
2023-06-07 10:42 ` [PATCH v3 15/16] upload-pack.c: avoid enumerating hidden refs where possible Taylor Blau
2023-06-07 10:42 ` [PATCH v3 16/16] ls-refs.c: " Taylor Blau
2023-06-12 21:05 ` [PATCH v3 00/16] refs: implement jump lists for packed backend Junio C Hamano
2023-06-20 14:20 ` [PATCH v4 " Taylor Blau
2023-06-20 14:21 ` [PATCH v4 01/16] refs.c: rename `ref_filter` Taylor Blau
2023-07-03 5:13 ` Jeff King
2023-06-20 14:21 ` [PATCH v4 02/16] ref-filter.h: provide `REF_FILTER_INIT` Taylor Blau
2023-07-03 5:15 ` Jeff King
2023-07-03 17:07 ` Taylor Blau
2023-06-20 14:21 ` [PATCH v4 03/16] ref-filter: clear reachable list pointers after freeing Taylor Blau
2023-07-03 5:16 ` Jeff King
2023-06-20 14:21 ` [PATCH v4 04/16] ref-filter: add `ref_filter_clear()` Taylor Blau
2023-07-03 5:19 ` Jeff King
2023-07-03 17:13 ` Taylor Blau
2023-07-03 17:32 ` Jeff King
2023-06-20 14:21 ` [PATCH v4 05/16] ref-filter.c: parameterize match functions over patterns Taylor Blau
2023-07-03 5:27 ` Jeff King
2023-07-03 17:18 ` Taylor Blau
2023-07-03 17:22 ` Taylor Blau
2023-07-03 17:33 ` Jeff King
2023-06-20 14:21 ` [PATCH v4 06/16] builtin/for-each-ref.c: add `--exclude` option Taylor Blau
2023-06-20 14:21 ` [PATCH v4 07/16] refs: plumb `exclude_patterns` argument throughout Taylor Blau
2023-06-20 14:21 ` [PATCH v4 08/16] refs/packed-backend.c: refactor `find_reference_location()` Taylor Blau
2023-06-20 14:21 ` [PATCH v4 09/16] refs/packed-backend.c: implement jump lists to avoid excluded pattern(s) Taylor Blau
2023-07-03 5:56 ` Jeff King
2023-07-03 17:38 ` Taylor Blau
2023-06-20 14:21 ` [PATCH v4 10/16] refs/packed-backend.c: add trace2 counters for jump list Taylor Blau
2023-06-20 14:21 ` [PATCH v4 11/16] revision.h: store hidden refs in a `strvec` Taylor Blau
2023-07-03 5:59 ` Jeff King
2023-06-20 14:22 ` [PATCH v4 12/16] refs/packed-backend.c: ignore complicated hidden refs rules Taylor Blau
2023-07-03 6:18 ` Jeff King
2023-07-04 18:22 ` Taylor Blau [this message]
2023-06-20 14:22 ` [PATCH v4 13/16] refs.h: let `for_each_namespaced_ref()` take excluded patterns Taylor Blau
2023-06-20 14:22 ` [PATCH v4 14/16] builtin/receive-pack.c: avoid enumerating hidden references Taylor Blau
2023-06-20 14:22 ` [PATCH v4 15/16] upload-pack.c: avoid enumerating hidden refs where possible Taylor Blau
2023-07-03 6:26 ` Jeff King
2023-07-04 18:43 ` Taylor Blau
2023-06-20 14:22 ` [PATCH v4 16/16] ls-refs.c: " Taylor Blau
2023-07-03 6:27 ` Jeff King
2023-07-03 6:29 ` [PATCH v4 00/16] refs: implement jump lists for packed backend Jeff King
2023-07-10 21:12 ` [PATCH v5 " Taylor Blau
2023-07-10 21:12 ` [PATCH v5 01/16] refs.c: rename `ref_filter` Taylor Blau
2023-07-10 21:12 ` [PATCH v5 02/16] ref-filter.h: provide `REF_FILTER_INIT` Taylor Blau
2023-07-10 21:12 ` [PATCH v5 03/16] ref-filter: clear reachable list pointers after freeing Taylor Blau
2023-07-10 21:12 ` [PATCH v5 04/16] ref-filter: add `ref_filter_clear()` Taylor Blau
2023-07-10 21:12 ` [PATCH v5 05/16] ref-filter.c: parameterize match functions over patterns Taylor Blau
2023-07-10 21:12 ` [PATCH v5 06/16] builtin/for-each-ref.c: add `--exclude` option Taylor Blau
2023-07-10 21:12 ` [PATCH v5 07/16] refs: plumb `exclude_patterns` argument throughout Taylor Blau
2023-07-10 21:12 ` [PATCH v5 08/16] refs/packed-backend.c: refactor `find_reference_location()` Taylor Blau
2023-07-10 21:12 ` [PATCH v5 09/16] refs/packed-backend.c: implement jump lists to avoid excluded pattern(s) Taylor Blau
2023-07-10 21:12 ` [PATCH v5 10/16] refs/packed-backend.c: add trace2 counters for jump list Taylor Blau
2023-07-10 21:12 ` [PATCH v5 11/16] revision.h: store hidden refs in a `strvec` Taylor Blau
2023-07-10 21:12 ` [PATCH v5 12/16] refs.h: let `for_each_namespaced_ref()` take excluded patterns Taylor Blau
2023-07-10 21:12 ` [PATCH v5 13/16] refs.h: implement `hidden_refs_to_excludes()` Taylor Blau
2023-07-10 21:12 ` [PATCH v5 14/16] builtin/receive-pack.c: avoid enumerating hidden references Taylor Blau
2023-07-10 21:12 ` [PATCH v5 15/16] upload-pack.c: avoid enumerating hidden refs where possible Taylor Blau
2023-07-10 21:12 ` [PATCH v5 16/16] ls-refs.c: " Taylor Blau
2023-07-10 22:35 ` [PATCH v5 00/16] refs: implement jump lists for packed backend Junio C Hamano
2023-07-11 9:37 ` Patrick Steinhardt
2023-07-11 15:56 ` Junio C Hamano
2023-07-11 17:19 ` Taylor Blau
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZKRjUkN/ggLBeIUu@nand.local \
--to=me@ttaylorr.com \
--cc=chris.torek@gmail.com \
--cc=derrickstolee@github.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=peff@peff.net \
--cc=ps@pks.im \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).