From: Jeff King <peff@peff.net>
To: Taylor Blau <me@ttaylorr.com>
Cc: git@vger.kernel.org, Chris Torek <chris.torek@gmail.com>,
Derrick Stolee <derrickstolee@github.com>,
Junio C Hamano <gitster@pobox.com>,
Patrick Steinhardt <ps@pks.im>
Subject: Re: [PATCH v4 12/16] refs/packed-backend.c: ignore complicated hidden refs rules
Date: Mon, 3 Jul 2023 02:18:39 -0400 [thread overview]
Message-ID: <20230703061839.GH3502534@coredump.intra.peff.net> (raw)
In-Reply-To: <845904853eeae1741d681a35fdd7816ea16b939a.1687270849.git.me@ttaylorr.com>
On Tue, Jun 20, 2023 at 10:22:02AM -0400, Taylor Blau wrote:
> In subsequent commits, we'll teach `receive-pack` and `upload-pack` to
> use the new jump list feature in the packed-refs iterator by ignoring
> references which are mentioned via its respective hideRefs lists.
>
> However, the packed-ref jump lists cannot handle un-hiding rules (that
> begin with '!'), or namespace comparisons (that begin with '^'). Detect
> and avoid these cases by falling back to the normal enumeration without
> a jump list when such patterns exist.
I'm a fan of punting on such cases to keep things simple and
incremental. But the location here seems weird to me:
> diff --git a/refs/packed-backend.c b/refs/packed-backend.c
> index 80b877e00c..2aeec5c601 100644
> --- a/refs/packed-backend.c
> +++ b/refs/packed-backend.c
> @@ -1008,6 +1008,25 @@ static void populate_excluded_jump_list(struct packed_ref_iterator *iter,
> if (!excluded_patterns)
> return;
>
> + for (pattern = excluded_patterns; *pattern; pattern++) {
> + /*
> + * We also can't feed any excludes from hidden refs
> + * config sections, since later rules may override
> + * previous ones. For example, with rules "refs/foo" and
> + * "!refs/foo/bar", we should show "refs/foo/bar" (and
> + * everything underneath it), but the earlier exclusion
> + * would cause us to skip all of "refs/foo". We likewise
> + * don't implement the namespace stripping required for
> + * '^' rules.
> + *
> + * Both are possible to do, but complicated, so avoid
> + * populating the jump list at all if we see either of
> + * these patterns.
> + */
> + if (**pattern == '!' || **pattern == '^')
> + return;
> + }
> +
This is deep in the packed-refs code, but the magic of "!" and "^" are
specific to ref_is_hidden().
So if I did:
git for-each-ref --exclude='!refs/heads/foo'
my understanding is that "!" would _not_ have an affect normally, but
now it is turning off this optimization.
The point may be somewhat academic for "^", as it is not allowed in a
refname anyway. But I don't think "!" is forbidden (as stupid as it
would be to include it in this way), is it?
It feels like the hiderefs code should be the one checking for these,
and then feeding only non-adorned refnames to the "exclude" list (though
there is no need to un-adorn them; once we see any with either form of
magic, we know we cannot use this "exclude" feature at all).
Something along the lines of (you'd want a similar tweak for
upload-pack):
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 2b2faa5d18..80a6b11c90 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -339,7 +339,8 @@ static void write_head_info(void)
static struct oidset seen = OIDSET_INIT;
refs_for_each_fullref_in(get_main_ref_store(the_repository), "",
- hidden_refs.v, show_ref_cb, &seen);
+ hidden_refs_to_excludes(&hidden_refs),
+ show_ref_cb, &seen);
for_each_alternate_ref(show_one_alternate_ref, &seen);
oidset_clear(&seen);
if (!sent_capabilities)
diff --git a/refs.c b/refs.c
index 3065e514fd..213412efd4 100644
--- a/refs.c
+++ b/refs.c
@@ -1482,6 +1482,31 @@ int ref_is_hidden(const char *refname, const char *refname_full,
return 0;
}
+const char **hidden_refs_to_excludes(const struct strvec *hide_refs)
+{
+ const char **pattern;
+
+ for (pattern = hide_refs->v; *pattern; pattern++) {
+ /*
+ * We also can't feed any excludes from hidden refs
+ * config sections, since later rules may override
+ * previous ones. For example, with rules "refs/foo" and
+ * "!refs/foo/bar", we should show "refs/foo/bar" (and
+ * everything underneath it), but the earlier exclusion
+ * would cause us to skip all of "refs/foo". We likewise
+ * don't implement the namespace stripping required for
+ * '^' rules.
+ *
+ * Both are possible to do, but complicated, so avoid
+ * populating the jump list at all if we see either of
+ * these patterns.
+ */
+ if (**pattern == '!' || **pattern == '^')
+ return NULL;
+ }
+ return hide_refs->v;
+}
+
const char *find_descendant_ref(const char *dirname,
const struct string_list *extras,
const struct string_list *skip)
diff --git a/refs.h b/refs.h
index 27d341d282..50c92d1f55 100644
--- a/refs.h
+++ b/refs.h
@@ -829,6 +829,8 @@ int parse_hide_refs_config(const char *var, const char *value, const char *,
*/
int ref_is_hidden(const char *, const char *, const struct strvec *);
+const char **hidden_refs_to_excludes(const struct strvec *hide_refs);
+
/* Is this a per-worktree ref living in the refs/ namespace? */
int is_per_worktree_ref(const char *refname);
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 9dd1795bf2..59c3fe9d91 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1009,25 +1009,6 @@ static void populate_excluded_jump_list(struct packed_ref_iterator *iter,
if (!excluded_patterns)
return;
- for (pattern = excluded_patterns; *pattern; pattern++) {
- /*
- * We also can't feed any excludes from hidden refs
- * config sections, since later rules may override
- * previous ones. For example, with rules "refs/foo" and
- * "!refs/foo/bar", we should show "refs/foo/bar" (and
- * everything underneath it), but the earlier exclusion
- * would cause us to skip all of "refs/foo". We likewise
- * don't implement the namespace stripping required for
- * '^' rules.
- *
- * Both are possible to do, but complicated, so avoid
- * populating the jump list at all if we see either of
- * these patterns.
- */
- if (**pattern == '!' || **pattern == '^')
- return;
- }
-
for (pattern = excluded_patterns; *pattern; pattern++) {
struct jump_list_entry *e;
next prev parent reply other threads:[~2023-07-03 6:18 UTC|newest]
Thread overview: 149+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-08 21:59 [PATCH 00/15] refs: implement skip lists for packed backend Taylor Blau
2023-05-08 21:59 ` [PATCH 01/15] refs.c: rename `ref_filter` Taylor Blau
2023-05-08 21:59 ` [PATCH 02/15] ref-filter.h: provide `REF_FILTER_INIT` Taylor Blau
2023-05-08 21:59 ` [PATCH 03/15] ref-filter: clear reachable list pointers after freeing Taylor Blau
2023-05-08 21:59 ` [PATCH 04/15] ref-filter: add ref_filter_clear() Taylor Blau
2023-05-08 22:29 ` Junio C Hamano
2023-05-08 22:33 ` Taylor Blau
2023-05-09 15:14 ` Patrick Steinhardt
2023-05-09 19:11 ` Taylor Blau
2023-05-08 21:59 ` [PATCH 05/15] ref-filter.c: parameterize match functions over patterns Taylor Blau
2023-05-08 22:36 ` Junio C Hamano
2023-05-09 20:13 ` Taylor Blau
2023-05-08 21:59 ` [PATCH 06/15] builtin/for-each-ref.c: add `--exclude` option Taylor Blau
2023-05-08 23:22 ` Junio C Hamano
2023-05-09 20:22 ` Taylor Blau
2023-05-08 22:00 ` [PATCH 07/15] refs: plumb `exclude_patterns` argument throughout Taylor Blau
2023-05-09 15:14 ` Patrick Steinhardt
2023-05-09 20:23 ` Taylor Blau
2023-05-08 22:00 ` [PATCH 08/15] refs/packed-backend.c: refactor `find_reference_location()` Taylor Blau
2023-05-08 23:56 ` Junio C Hamano
2023-05-09 20:29 ` Taylor Blau
2023-05-08 22:00 ` [PATCH 09/15] refs/packed-backend.c: implement skip lists to avoid excluded pattern(s) Taylor Blau
2023-05-09 0:10 ` Chris Torek
2023-05-09 20:39 ` Taylor Blau
2023-05-09 15:15 ` Patrick Steinhardt
2023-05-09 20:55 ` Taylor Blau
2023-05-09 21:15 ` Taylor Blau
2023-05-10 7:25 ` Patrick Steinhardt
2023-05-09 23:40 ` Junio C Hamano
2023-05-10 2:30 ` Taylor Blau
2023-05-08 22:00 ` [PATCH 10/15] refs/packed-backend.c: add trace2 counters for skip list Taylor Blau
2023-05-08 22:00 ` [PATCH 11/15] revision.h: store hidden refs in a `strvec` Taylor Blau
2023-05-08 22:00 ` [PATCH 12/15] refs/packed-backend.c: ignore complicated hidden refs rules Taylor Blau
2023-05-08 22:00 ` [PATCH 13/15] refs.h: let `for_each_namespaced_ref()` take excluded patterns Taylor Blau
2023-05-08 22:00 ` [PATCH 14/15] upload-pack.c: avoid enumerating hidden refs where possible Taylor Blau
2023-05-09 15:15 ` Patrick Steinhardt
2023-05-09 21:34 ` Taylor Blau
2023-05-08 22:00 ` [PATCH 15/15] builtin/receive-pack.c: avoid enumerating hidden references Taylor Blau
2023-05-15 19:23 ` [PATCH v2 00/16] refs: implement jump lists for packed backend Taylor Blau
2023-05-15 19:23 ` [PATCH v2 01/16] refs.c: rename `ref_filter` Taylor Blau
2023-05-15 19:23 ` [PATCH v2 02/16] ref-filter.h: provide `REF_FILTER_INIT` Taylor Blau
2023-05-15 19:23 ` [PATCH v2 03/16] ref-filter: clear reachable list pointers after freeing Taylor Blau
2023-05-15 19:23 ` [PATCH v2 04/16] ref-filter: add `ref_filter_clear()` Taylor Blau
2023-05-15 19:23 ` [PATCH v2 05/16] ref-filter.c: parameterize match functions over patterns Taylor Blau
2023-05-15 19:23 ` [PATCH v2 06/16] builtin/for-each-ref.c: add `--exclude` option Taylor Blau
2023-05-15 19:23 ` [PATCH v2 07/16] refs: plumb `exclude_patterns` argument throughout Taylor Blau
2023-05-15 19:23 ` [PATCH v2 08/16] refs/packed-backend.c: refactor `find_reference_location()` Taylor Blau
2023-05-15 19:23 ` [PATCH v2 09/16] refs/packed-backend.c: implement jump lists to avoid excluded pattern(s) Taylor Blau
2023-06-06 7:00 ` Patrick Steinhardt
2023-06-20 12:15 ` Taylor Blau
2023-05-15 19:23 ` [PATCH v2 10/16] refs/packed-backend.c: add trace2 counters for jump list Taylor Blau
2023-05-15 19:23 ` [PATCH v2 11/16] revision.h: store hidden refs in a `strvec` Taylor Blau
2023-06-06 7:00 ` Patrick Steinhardt
2023-06-20 12:16 ` Taylor Blau
2023-05-15 19:23 ` [PATCH v2 12/16] refs/packed-backend.c: ignore complicated hidden refs rules Taylor Blau
2023-05-15 19:23 ` [PATCH v2 13/16] refs.h: let `for_each_namespaced_ref()` take excluded patterns Taylor Blau
2023-06-06 7:01 ` Patrick Steinhardt
2023-06-20 12:18 ` Taylor Blau
2023-05-15 19:23 ` [PATCH v2 14/16] builtin/receive-pack.c: avoid enumerating hidden references Taylor Blau
2023-05-15 19:23 ` [PATCH v2 15/16] upload-pack.c: avoid enumerating hidden refs where possible Taylor Blau
2023-05-15 19:23 ` [PATCH v2 16/16] ls-refs.c: " Taylor Blau
2023-06-06 7:01 ` [PATCH v2 00/16] refs: implement jump lists for packed backend Patrick Steinhardt
2023-06-20 12:22 ` Taylor Blau
2023-06-07 10:40 ` [PATCH v3 " Taylor Blau
2023-06-07 10:40 ` [PATCH v3 01/16] refs.c: rename `ref_filter` Taylor Blau
2023-06-13 22:19 ` Junio C Hamano
2023-06-07 10:40 ` [PATCH v3 02/16] ref-filter.h: provide `REF_FILTER_INIT` Taylor Blau
2023-06-07 10:41 ` [PATCH v3 03/16] ref-filter: clear reachable list pointers after freeing Taylor Blau
2023-06-07 10:41 ` [PATCH v3 04/16] ref-filter: add `ref_filter_clear()` Taylor Blau
2023-06-07 10:41 ` [PATCH v3 05/16] ref-filter.c: parameterize match functions over patterns Taylor Blau
2023-06-13 22:37 ` Junio C Hamano
2023-06-07 10:41 ` [PATCH v3 06/16] builtin/for-each-ref.c: add `--exclude` option Taylor Blau
2023-06-07 10:41 ` [PATCH v3 07/16] refs: plumb `exclude_patterns` argument throughout Taylor Blau
2023-06-13 23:42 ` Junio C Hamano
2023-06-20 11:52 ` Taylor Blau
2023-06-07 10:41 ` [PATCH v3 08/16] refs/packed-backend.c: refactor `find_reference_location()` Taylor Blau
2023-06-07 10:41 ` [PATCH v3 09/16] refs/packed-backend.c: implement jump lists to avoid excluded pattern(s) Taylor Blau
2023-06-14 0:27 ` Junio C Hamano
2023-06-20 12:05 ` Taylor Blau
2023-06-20 18:49 ` Junio C Hamano
2023-06-07 10:41 ` [PATCH v3 10/16] refs/packed-backend.c: add trace2 counters for jump list Taylor Blau
2023-06-14 0:32 ` Junio C Hamano
2023-06-20 12:08 ` Taylor Blau
2023-06-07 10:41 ` [PATCH v3 11/16] revision.h: store hidden refs in a `strvec` Taylor Blau
2023-06-07 10:41 ` [PATCH v3 12/16] refs/packed-backend.c: ignore complicated hidden refs rules Taylor Blau
2023-06-14 0:40 ` Junio C Hamano
2023-06-07 10:41 ` [PATCH v3 13/16] refs.h: let `for_each_namespaced_ref()` take excluded patterns Taylor Blau
2023-06-07 10:42 ` [PATCH v3 14/16] builtin/receive-pack.c: avoid enumerating hidden references Taylor Blau
2023-06-07 10:42 ` [PATCH v3 15/16] upload-pack.c: avoid enumerating hidden refs where possible Taylor Blau
2023-06-07 10:42 ` [PATCH v3 16/16] ls-refs.c: " Taylor Blau
2023-06-12 21:05 ` [PATCH v3 00/16] refs: implement jump lists for packed backend Junio C Hamano
2023-06-20 14:20 ` [PATCH v4 " Taylor Blau
2023-06-20 14:21 ` [PATCH v4 01/16] refs.c: rename `ref_filter` Taylor Blau
2023-07-03 5:13 ` Jeff King
2023-06-20 14:21 ` [PATCH v4 02/16] ref-filter.h: provide `REF_FILTER_INIT` Taylor Blau
2023-07-03 5:15 ` Jeff King
2023-07-03 17:07 ` Taylor Blau
2023-06-20 14:21 ` [PATCH v4 03/16] ref-filter: clear reachable list pointers after freeing Taylor Blau
2023-07-03 5:16 ` Jeff King
2023-06-20 14:21 ` [PATCH v4 04/16] ref-filter: add `ref_filter_clear()` Taylor Blau
2023-07-03 5:19 ` Jeff King
2023-07-03 17:13 ` Taylor Blau
2023-07-03 17:32 ` Jeff King
2023-06-20 14:21 ` [PATCH v4 05/16] ref-filter.c: parameterize match functions over patterns Taylor Blau
2023-07-03 5:27 ` Jeff King
2023-07-03 17:18 ` Taylor Blau
2023-07-03 17:22 ` Taylor Blau
2023-07-03 17:33 ` Jeff King
2023-06-20 14:21 ` [PATCH v4 06/16] builtin/for-each-ref.c: add `--exclude` option Taylor Blau
2023-06-20 14:21 ` [PATCH v4 07/16] refs: plumb `exclude_patterns` argument throughout Taylor Blau
2023-06-20 14:21 ` [PATCH v4 08/16] refs/packed-backend.c: refactor `find_reference_location()` Taylor Blau
2023-06-20 14:21 ` [PATCH v4 09/16] refs/packed-backend.c: implement jump lists to avoid excluded pattern(s) Taylor Blau
2023-07-03 5:56 ` Jeff King
2023-07-03 17:38 ` Taylor Blau
2023-06-20 14:21 ` [PATCH v4 10/16] refs/packed-backend.c: add trace2 counters for jump list Taylor Blau
2023-06-20 14:21 ` [PATCH v4 11/16] revision.h: store hidden refs in a `strvec` Taylor Blau
2023-07-03 5:59 ` Jeff King
2023-06-20 14:22 ` [PATCH v4 12/16] refs/packed-backend.c: ignore complicated hidden refs rules Taylor Blau
2023-07-03 6:18 ` Jeff King [this message]
2023-07-04 18:22 ` Taylor Blau
2023-06-20 14:22 ` [PATCH v4 13/16] refs.h: let `for_each_namespaced_ref()` take excluded patterns Taylor Blau
2023-06-20 14:22 ` [PATCH v4 14/16] builtin/receive-pack.c: avoid enumerating hidden references Taylor Blau
2023-06-20 14:22 ` [PATCH v4 15/16] upload-pack.c: avoid enumerating hidden refs where possible Taylor Blau
2023-07-03 6:26 ` Jeff King
2023-07-04 18:43 ` Taylor Blau
2023-06-20 14:22 ` [PATCH v4 16/16] ls-refs.c: " Taylor Blau
2023-07-03 6:27 ` Jeff King
2023-07-03 6:29 ` [PATCH v4 00/16] refs: implement jump lists for packed backend Jeff King
2023-07-10 21:12 ` [PATCH v5 " Taylor Blau
2023-07-10 21:12 ` [PATCH v5 01/16] refs.c: rename `ref_filter` Taylor Blau
2023-07-10 21:12 ` [PATCH v5 02/16] ref-filter.h: provide `REF_FILTER_INIT` Taylor Blau
2023-07-10 21:12 ` [PATCH v5 03/16] ref-filter: clear reachable list pointers after freeing Taylor Blau
2023-07-10 21:12 ` [PATCH v5 04/16] ref-filter: add `ref_filter_clear()` Taylor Blau
2023-07-10 21:12 ` [PATCH v5 05/16] ref-filter.c: parameterize match functions over patterns Taylor Blau
2023-07-10 21:12 ` [PATCH v5 06/16] builtin/for-each-ref.c: add `--exclude` option Taylor Blau
2023-07-10 21:12 ` [PATCH v5 07/16] refs: plumb `exclude_patterns` argument throughout Taylor Blau
2023-07-10 21:12 ` [PATCH v5 08/16] refs/packed-backend.c: refactor `find_reference_location()` Taylor Blau
2023-07-10 21:12 ` [PATCH v5 09/16] refs/packed-backend.c: implement jump lists to avoid excluded pattern(s) Taylor Blau
2023-07-10 21:12 ` [PATCH v5 10/16] refs/packed-backend.c: add trace2 counters for jump list Taylor Blau
2023-07-10 21:12 ` [PATCH v5 11/16] revision.h: store hidden refs in a `strvec` Taylor Blau
2023-07-10 21:12 ` [PATCH v5 12/16] refs.h: let `for_each_namespaced_ref()` take excluded patterns Taylor Blau
2023-07-10 21:12 ` [PATCH v5 13/16] refs.h: implement `hidden_refs_to_excludes()` Taylor Blau
2023-07-10 21:12 ` [PATCH v5 14/16] builtin/receive-pack.c: avoid enumerating hidden references Taylor Blau
2023-07-10 21:12 ` [PATCH v5 15/16] upload-pack.c: avoid enumerating hidden refs where possible Taylor Blau
2023-07-10 21:12 ` [PATCH v5 16/16] ls-refs.c: " Taylor Blau
2023-07-10 22:35 ` [PATCH v5 00/16] refs: implement jump lists for packed backend Junio C Hamano
2023-07-11 9:37 ` Patrick Steinhardt
2023-07-11 15:56 ` Junio C Hamano
2023-07-11 17:19 ` Taylor Blau
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230703061839.GH3502534@coredump.intra.peff.net \
--to=peff@peff.net \
--cc=chris.torek@gmail.com \
--cc=derrickstolee@github.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=me@ttaylorr.com \
--cc=ps@pks.im \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).