* [PATCH 0/8] fetch: introduce machine-parseable output
@ 2023-04-19 12:31 Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 1/8] fetch: split out tests for output format Patrick Steinhardt
` (14 more replies)
0 siblings, 15 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-19 12:31 UTC (permalink / raw)
To: git; +Cc: Jonathan Tan
[-- Attachment #1: Type: text/plain, Size: 2841 bytes --]
Hi,
this is the second part of my quest to introduce a machine-parseable
output for git-fetch(1) after the initial refactorings that have been
merged via e9dffbc7f1 (Merge branch 'ps/fetch-ref-update-reporting',
2023-04-06).
Parsing the output of fetches is mostly impossible. It prettifies
reference names that are about to be updated, doesn't print the old and
new object IDs the refs are being updated from and to, and prints all of
that information in nice columns. In short, it is designed to be read by
humans rather than machines.
This makes it hard to use in a script way though, e.g. to learn about
which references actually have been updated or which have not been
updated. This patch series intends to fix that by introducing a new
machine-parseable interface:
```
$ git fetch --output-format=porcelain --no-progress
fff5a5e7f528b2ed2c335991399a766c2cf01103 af67688dca57999fd848f051eeea1d375ba546b2 refs/remotes/origin/master
* 0000000000000000000000000000000000000000 e046fe5a36a970bc14fbfbcb2074a48776f6b671 refs/remotes/origin/x86-rep-insns
* 0000000000000000000000000000000000000000 bb81ed6862b864c9eb99447f04d49a84ecb647e5 refs/tags/v6.3-rc4
* 0000000000000000000000000000000000000000 83af7b1468c0dca86b4dc9e43e73bfa4f38d9637 refs/tags/v6.3-rc5
* 0000000000000000000000000000000000000000 ab3affb8ed84f68638162fe7e6fd4055e15bff5b refs/tags/v6.3-rc6
* 0000000000000000000000000000000000000000 1c8c28415e8743368a2b800520a6dd0b22ee6ec2 refs/tags/v6.3-rc7
```
The series is structured as following:
- Patches 1 and 2 improve test coverage for output formats.
- Patch 3 fixes a bug with the current output format.
- Patch 4 to 6 perform some preliminary refactorings.
- Patch 7 introduces a new `--output-format=` option for
git-fetch(1) that allows the user to configure the output more
directly.
- Patch 8 introduces the new "porcelain" output format.
Patrick
Patrick Steinhardt (8):
fetch: split out tests for output format
fetch: add a test to exercise invalid output formats
fetch: fix missing from-reference when fetching HEAD:foo
fetch: introduce `display_format` enum
fetch: move display format parsing into main function
fetch: move option related variables into main function
fetch: introduce new `--output-format` option
fetch: introduce machine-parseable "porcelain" output format
Documentation/config/fetch.txt | 4 +-
Documentation/fetch-options.txt | 5 +
Documentation/git-fetch.txt | 17 +-
builtin/fetch.c | 406 +++++++++++++++++++-------------
t/t5510-fetch.sh | 53 -----
t/t5574-fetch-output.sh | 209 ++++++++++++++++
6 files changed, 475 insertions(+), 219 deletions(-)
create mode 100755 t/t5574-fetch-output.sh
--
2.40.0
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH 1/8] fetch: split out tests for output format
2023-04-19 12:31 [PATCH 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
@ 2023-04-19 12:31 ` Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 2/8] fetch: add a test to exercise invalid output formats Patrick Steinhardt
` (13 subsequent siblings)
14 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-19 12:31 UTC (permalink / raw)
To: git; +Cc: Jonathan Tan
[-- Attachment #1: Type: text/plain, Size: 4269 bytes --]
We're about to introduce a new porcelain mode for the output of
git-fetch(1). As part of that we'll be introducing a set of new tests
that only relate to the output of this command.
Split out tests that exercise the output format of git-fetch(1) so that
it becomes easier to verify this functionality as a standalone unit. As
the tests assume that the default branch is called "main" we set up the
corresponding GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME environment variable
accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
t/t5510-fetch.sh | 53 ----------------------------------
t/t5574-fetch-output.sh | 63 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 63 insertions(+), 53 deletions(-)
create mode 100755 t/t5574-fetch-output.sh
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index dc44da9c79..4f289063ce 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -1118,59 +1118,6 @@ test_expect_success 'fetching with auto-gc does not lock up' '
)
'
-test_expect_success 'fetch aligned output' '
- git clone . full-output &&
- test_commit looooooooooooong-tag &&
- (
- cd full-output &&
- git -c fetch.output=full fetch origin >actual 2>&1 &&
- grep -e "->" actual | cut -c 22- >../actual
- ) &&
- cat >expect <<-\EOF &&
- main -> origin/main
- looooooooooooong-tag -> looooooooooooong-tag
- EOF
- test_cmp expect actual
-'
-
-test_expect_success 'fetch compact output' '
- git clone . compact &&
- test_commit extraaa &&
- (
- cd compact &&
- git -c fetch.output=compact fetch origin >actual 2>&1 &&
- grep -e "->" actual | cut -c 22- >../actual
- ) &&
- cat >expect <<-\EOF &&
- main -> origin/*
- extraaa -> *
- EOF
- test_cmp expect actual
-'
-
-test_expect_success '--no-show-forced-updates' '
- mkdir forced-updates &&
- (
- cd forced-updates &&
- git init &&
- test_commit 1 &&
- test_commit 2
- ) &&
- git clone forced-updates forced-update-clone &&
- git clone forced-updates no-forced-update-clone &&
- git -C forced-updates reset --hard HEAD~1 &&
- (
- cd forced-update-clone &&
- git fetch --show-forced-updates origin 2>output &&
- test_i18ngrep "(forced update)" output
- ) &&
- (
- cd no-forced-update-clone &&
- git fetch --no-show-forced-updates origin 2>output &&
- test_i18ngrep ! "(forced update)" output
- )
-'
-
for section in fetch transfer
do
test_expect_success "$section.hideRefs affects connectivity check" '
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
new file mode 100755
index 0000000000..f91b654d38
--- /dev/null
+++ b/t/t5574-fetch-output.sh
@@ -0,0 +1,63 @@
+#!/bin/sh
+
+test_description='git fetch output format'
+
+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+
+. ./test-lib.sh
+
+test_expect_success 'fetch aligned output' '
+ git clone . full-output &&
+ test_commit looooooooooooong-tag &&
+ (
+ cd full-output &&
+ git -c fetch.output=full fetch origin >actual 2>&1 &&
+ grep -e "->" actual | cut -c 22- >../actual
+ ) &&
+ cat >expect <<-\EOF &&
+ main -> origin/main
+ looooooooooooong-tag -> looooooooooooong-tag
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success 'fetch compact output' '
+ git clone . compact &&
+ test_commit extraaa &&
+ (
+ cd compact &&
+ git -c fetch.output=compact fetch origin >actual 2>&1 &&
+ grep -e "->" actual | cut -c 22- >../actual
+ ) &&
+ cat >expect <<-\EOF &&
+ main -> origin/*
+ extraaa -> *
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success '--no-show-forced-updates' '
+ mkdir forced-updates &&
+ (
+ cd forced-updates &&
+ git init &&
+ test_commit 1 &&
+ test_commit 2
+ ) &&
+ git clone forced-updates forced-update-clone &&
+ git clone forced-updates no-forced-update-clone &&
+ git -C forced-updates reset --hard HEAD~1 &&
+ (
+ cd forced-update-clone &&
+ git fetch --show-forced-updates origin 2>output &&
+ test_i18ngrep "(forced update)" output
+ ) &&
+ (
+ cd no-forced-update-clone &&
+ git fetch --no-show-forced-updates origin 2>output &&
+ test_i18ngrep ! "(forced update)" output
+ )
+'
+
+test_done
--
2.40.0
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH 2/8] fetch: add a test to exercise invalid output formats
2023-04-19 12:31 [PATCH 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 1/8] fetch: split out tests for output format Patrick Steinhardt
@ 2023-04-19 12:31 ` Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
` (12 subsequent siblings)
14 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-19 12:31 UTC (permalink / raw)
To: git; +Cc: Jonathan Tan
[-- Attachment #1: Type: text/plain, Size: 1203 bytes --]
Add a testcase that exercises the logic when an invalid output format is
passed via the `fetch.output` configuration.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
t/t5574-fetch-output.sh | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index f91b654d38..0e45c27007 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -7,6 +7,23 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
. ./test-lib.sh
+test_expect_success 'fetch with invalid output format configuration' '
+ test_when_finished "rm -rf clone" &&
+ git clone . clone &&
+
+ test_must_fail git -C clone -c fetch.output= fetch origin >actual 2>&1 &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}${SQ}
+ EOF
+ test_cmp expect actual &&
+
+ test_must_fail git -C clone -c fetch.output=garbage fetch origin >actual 2>&1 &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}garbage${SQ}
+ EOF
+ test_cmp expect actual
+'
+
test_expect_success 'fetch aligned output' '
git clone . full-output &&
test_commit looooooooooooong-tag &&
--
2.40.0
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo
2023-04-19 12:31 [PATCH 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 1/8] fetch: split out tests for output format Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 2/8] fetch: add a test to exercise invalid output formats Patrick Steinhardt
@ 2023-04-19 12:31 ` Patrick Steinhardt
2023-04-26 19:20 ` Jacob Keller
` (2 more replies)
2023-04-19 12:31 ` [PATCH 4/8] fetch: introduce `display_format` enum Patrick Steinhardt
` (11 subsequent siblings)
14 siblings, 3 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-19 12:31 UTC (permalink / raw)
To: git; +Cc: Jonathan Tan
[-- Attachment #1: Type: text/plain, Size: 9537 bytes --]
When displaying reference updates, we print a line that looks similar to
the following:
```
* branch master -> master
```
The "branch" bit changes depending on what kind of reference we're
updating, while both of the right-hand references are computed by
stripping well-known prefixes like "refs/heads/" or "refs/tags".
The logic is kind of intertwined though and not easy to follow: we
precompute both the kind (e.g. "branch") and the what, which is the
abbreviated remote reference name, in `store_updated_refs()` and then
pass it down the call chain to `display_ref_update()`.
There is a set of different cases here:
- When the remote reference name is "HEAD" we assume no kind and
will thus instead print "[new ref]". We keep what at the empty
string.
- When the remote reference name has a well-known prefix then the
kind would be "branch", "tag" or "remote-tracking branch". The
what is the reference with the well-known prefix stripped and in
fact matches the output that `prettify_refname()` would return.
- Otherwise, we'll again assume no kind and keep the what set to the
fully qualified reference name.
Now there is a bug with the first case here, where the remote reference
name is "HEAD". As noted, "what" will be set to the empty string. And
that seems to be intentional because we also use this information to
update the FETCH_HEAD, and in case we're updating HEAD we seemingly
don't want to append that to our FETCH_HEAD value.
But as mentioned, we also use this value to display reference updates.
And while the call to `display_ref_update()` correctly figures out that
we meant "HEAD" when `what` is empty, the call to `update_local_ref()`
doesn't. `update_local_ref()` will then call `display_ref_update()` with
the empty string and cause the following broken output:
```
$ git fetch --dry-run origin HEAD:foo
From https://github.com/git/git
* [new ref] -> foo
```
The HEAD string is clearly missing from the left-hand side of the arrow,
which is further stressed by the point that the following commands work
as expected:
```
$ git fetch --dry-run origin HEAD
From https://github.com/git/git
* branch HEAD -> FETCH_HEAD
$ git fetch --dry-run origin master
From https://github.com/git/git
* branch master -> FETCH_HEAD
* branch master -> origin/master
```
Fix this bug by instead unconditionally passing the full reference name
to `display_ref_update()` which learns to call `prettify_refname()` on
it. This does fix the above bug and is otherwise functionally the same
as `prettify_refname()` would only ever strip the well-known prefixes
just as intended. So at the same time, this also simplifies the code a
bit.
Note that this patch also changes formatting of the block that computes
the "kind" and "what" variables. This is done on purpose so that it is
part of the diff, hopefully making the change easier to comprehend.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 37 +++++++++++++++++++------------------
t/t5574-fetch-output.sh | 19 +++++++++++++++++++
2 files changed, 38 insertions(+), 18 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index c310d89878..7c64f0c562 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -918,12 +918,14 @@ static void display_ref_update(struct display_state *display_state, char code,
}
width = (summary_width + strlen(summary) - gettext_width(summary));
+ remote = prettify_refname(remote);
+ local = prettify_refname(local);
strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
if (!display_state->compact_format)
- print_remote_to_local(display_state, remote, prettify_refname(local));
+ print_remote_to_local(display_state, remote, local);
else
- print_compact(display_state, remote, prettify_refname(local));
+ print_compact(display_state, remote, local);
if (error)
strbuf_addf(&display_state->buf, " (%s)", error);
strbuf_addch(&display_state->buf, '\n');
@@ -934,7 +936,7 @@ static void display_ref_update(struct display_state *display_state, char code,
static int update_local_ref(struct ref *ref,
struct ref_transaction *transaction,
struct display_state *display_state,
- const char *remote, const struct ref *remote_ref,
+ const struct ref *remote_ref,
int summary_width)
{
struct commit *current = NULL, *updated;
@@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,
if (oideq(&ref->old_oid, &ref->new_oid)) {
if (verbosity > 0)
display_ref_update(display_state, '=', _("[up to date]"), NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 0;
}
@@ -959,7 +961,7 @@ static int update_local_ref(struct ref *ref,
*/
display_ref_update(display_state, '!', _("[rejected]"),
_("can't fetch into checked-out branch"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 1;
}
@@ -970,12 +972,12 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("updating tag", ref, transaction, 0);
display_ref_update(display_state, r ? '!' : 't', _("[tag update]"),
r ? _("unable to update local ref") : NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"),
_("would clobber existing tag"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 1;
}
}
@@ -1008,7 +1010,7 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref(msg, ref, transaction, 0);
display_ref_update(display_state, r ? '!' : '*', what,
r ? _("unable to update local ref") : NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return r;
}
@@ -1030,7 +1032,7 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("fast-forward", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : ' ', quickref.buf,
r ? _("unable to update local ref") : NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
strbuf_release(&quickref);
return r;
} else if (force || ref->force) {
@@ -1042,12 +1044,12 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("forced-update", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : '+', quickref.buf,
r ? _("unable to update local ref") : _("forced update"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
strbuf_release(&quickref);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"), _("non-fast-forward"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 1;
}
}
@@ -1252,14 +1254,13 @@ static int store_updated_refs(struct display_state *display_state,
if (!strcmp(rm->name, "HEAD")) {
kind = "";
what = "";
- }
- else if (skip_prefix(rm->name, "refs/heads/", &what))
+ } else if (skip_prefix(rm->name, "refs/heads/", &what)) {
kind = "branch";
- else if (skip_prefix(rm->name, "refs/tags/", &what))
+ } else if (skip_prefix(rm->name, "refs/tags/", &what)) {
kind = "tag";
- else if (skip_prefix(rm->name, "refs/remotes/", &what))
+ } else if (skip_prefix(rm->name, "refs/remotes/", &what)) {
kind = "remote-tracking branch";
- else {
+ } else {
kind = "";
what = rm->name;
}
@@ -1277,7 +1278,7 @@ static int store_updated_refs(struct display_state *display_state,
display_state->url_len);
if (ref) {
- rc |= update_local_ref(ref, transaction, display_state, what,
+ rc |= update_local_ref(ref, transaction, display_state,
rm, summary_width);
free(ref);
} else if (write_fetch_head || dry_run) {
@@ -1288,7 +1289,7 @@ static int store_updated_refs(struct display_state *display_state,
*/
display_ref_update(display_state, '*',
*kind ? kind : "branch", NULL,
- *what ? what : "HEAD",
+ rm->name,
"FETCH_HEAD", summary_width);
}
}
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index 0e45c27007..55f0f05b6a 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -54,6 +54,25 @@ test_expect_success 'fetch compact output' '
test_cmp expect actual
'
+test_expect_success 'fetch output with HEAD and --dry-run' '
+ test_when_finished "rm -rf head" &&
+ git clone . head &&
+
+ git -C head fetch --dry-run origin HEAD >actual 2>&1 &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * branch HEAD -> FETCH_HEAD
+ EOF
+ test_cmp expect actual &&
+
+ git -C head fetch --dry-run origin HEAD:foo >actual 2>&1 &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * [new ref] HEAD -> foo
+ EOF
+ test_cmp expect actual
+'
+
test_expect_success '--no-show-forced-updates' '
mkdir forced-updates &&
(
--
2.40.0
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH 4/8] fetch: introduce `display_format` enum
2023-04-19 12:31 [PATCH 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (2 preceding siblings ...)
2023-04-19 12:31 ` [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
@ 2023-04-19 12:31 ` Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 5/8] fetch: move display format parsing into main function Patrick Steinhardt
` (10 subsequent siblings)
14 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-19 12:31 UTC (permalink / raw)
To: git; +Cc: Jonathan Tan
[-- Attachment #1: Type: text/plain, Size: 5113 bytes --]
We currently have two different display formats in git-fetch(1) with the
"full" and "compact" formats. This is tracked with a boolean value that
simply denotes whether the display format is supposed to be compacted
or not. This works reasonably well while there are only two formats, but
we're about to introduce another format that will make this a bit more
awkward to use.
Introduce a `enum display_format` that is more readily extensible.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 101 ++++++++++++++++++++++++++++++------------------
1 file changed, 64 insertions(+), 37 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 7c64f0c562..e03fcd1b2f 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -48,11 +48,17 @@ enum {
TAGS_SET = 2
};
+enum display_format {
+ DISPLAY_FORMAT_UNKNOWN = 0,
+ DISPLAY_FORMAT_FULL,
+ DISPLAY_FORMAT_COMPACT,
+};
+
struct display_state {
struct strbuf buf;
int refcol_width;
- int compact_format;
+ enum display_format format;
char *url;
int url_len, shown_url;
@@ -784,7 +790,6 @@ static int refcol_width(const struct ref *ref, int compact_format)
static void display_state_init(struct display_state *display_state, struct ref *ref_map,
const char *raw_url)
{
- struct ref *rm;
const char *format = "full";
int i;
@@ -809,31 +814,42 @@ static void display_state_init(struct display_state *display_state, struct ref *
git_config_get_string_tmp("fetch.output", &format);
if (!strcasecmp(format, "full"))
- display_state->compact_format = 0;
+ display_state->format = DISPLAY_FORMAT_FULL;
else if (!strcasecmp(format, "compact"))
- display_state->compact_format = 1;
+ display_state->format = DISPLAY_FORMAT_COMPACT;
else
die(_("invalid value for '%s': '%s'"),
"fetch.output", format);
- display_state->refcol_width = 10;
- for (rm = ref_map; rm; rm = rm->next) {
- int width;
+ switch (display_state->format) {
+ case DISPLAY_FORMAT_FULL:
+ case DISPLAY_FORMAT_COMPACT: {
+ struct ref *rm;
- if (rm->status == REF_STATUS_REJECT_SHALLOW ||
- !rm->peer_ref ||
- !strcmp(rm->name, "HEAD"))
- continue;
+ display_state->refcol_width = 10;
+ for (rm = ref_map; rm; rm = rm->next) {
+ int width;
- width = refcol_width(rm, display_state->compact_format);
+ if (rm->status == REF_STATUS_REJECT_SHALLOW ||
+ !rm->peer_ref ||
+ !strcmp(rm->name, "HEAD"))
+ continue;
- /*
- * Not precise calculation for compact mode because '*' can
- * appear on the left hand side of '->' and shrink the column
- * back.
- */
- if (display_state->refcol_width < width)
- display_state->refcol_width = width;
+ width = refcol_width(rm, display_state->format == DISPLAY_FORMAT_COMPACT);
+
+ /*
+ * Not precise calculation for compact mode because '*' can
+ * appear on the left hand side of '->' and shrink the column
+ * back.
+ */
+ if (display_state->refcol_width < width)
+ display_state->refcol_width = width;
+ }
+
+ break;
+ }
+ default:
+ BUG("unexpected display foramt %d", display_state->format);
}
}
@@ -904,30 +920,41 @@ static void display_ref_update(struct display_state *display_state, char code,
const char *remote, const char *local,
int summary_width)
{
- int width;
-
if (verbosity < 0)
return;
strbuf_reset(&display_state->buf);
- if (!display_state->shown_url) {
- strbuf_addf(&display_state->buf, _("From %.*s\n"),
- display_state->url_len, display_state->url);
- display_state->shown_url = 1;
+ switch (display_state->format) {
+ case DISPLAY_FORMAT_FULL:
+ case DISPLAY_FORMAT_COMPACT: {
+ int width;
+
+ if (!display_state->shown_url) {
+ strbuf_addf(&display_state->buf, _("From %.*s\n"),
+ display_state->url_len, display_state->url);
+ display_state->shown_url = 1;
+ }
+
+ width = (summary_width + strlen(summary) - gettext_width(summary));
+ remote = prettify_refname(remote);
+ local = prettify_refname(local);
+
+ strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
+
+ if (display_state->format != DISPLAY_FORMAT_COMPACT)
+ print_remote_to_local(display_state, remote, local);
+ else
+ print_compact(display_state, remote, local);
+
+ if (error)
+ strbuf_addf(&display_state->buf, " (%s)", error);
+
+ break;
}
-
- width = (summary_width + strlen(summary) - gettext_width(summary));
- remote = prettify_refname(remote);
- local = prettify_refname(local);
-
- strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
- if (!display_state->compact_format)
- print_remote_to_local(display_state, remote, local);
- else
- print_compact(display_state, remote, local);
- if (error)
- strbuf_addf(&display_state->buf, " (%s)", error);
+ default:
+ BUG("unexpected display format %d", display_state->format);
+ };
strbuf_addch(&display_state->buf, '\n');
fputs(display_state->buf.buf, stderr);
--
2.40.0
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH 5/8] fetch: move display format parsing into main function
2023-04-19 12:31 [PATCH 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (3 preceding siblings ...)
2023-04-19 12:31 ` [PATCH 4/8] fetch: introduce `display_format` enum Patrick Steinhardt
@ 2023-04-19 12:31 ` Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 6/8] fetch: move option related variables " Patrick Steinhardt
` (9 subsequent siblings)
14 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-19 12:31 UTC (permalink / raw)
To: git; +Cc: Jonathan Tan
[-- Attachment #1: Type: text/plain, Size: 5425 bytes --]
Parsing the display format happens inside of `display_state_init()`. As
we only need to check for a simple config entry, this is a natural
location to put this code as it means that display-state logic is neatly
contained in a single location.
We're about to introduce a output format though that is intended to be
parseable by machines, for example inside of a script. In that case it
becomes a bit awkward of an interface if you have to call git-fetch(1)
with the `fetch.output` config key set. We're thus going to introduce a
new `--output-format` switch for git-fetch(1) so that the output format
can be configured more directly.
This means we'll have to hook parsing of the display format into the
command line options parser. Having the code to determine the actual
output format scattered across two different sites is hard to reason
about though.
Refactor the code such that callers are expected to pass the display
format that is to be used into `display_state_init()`. This allows us to
lift up the code into the main function, where we can then hook it into
command line options parser in a follow-up commit.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 41 ++++++++++++++++++++++++-----------------
1 file changed, 24 insertions(+), 17 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index e03fcd1b2f..bcc156a9ce 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -788,14 +788,13 @@ static int refcol_width(const struct ref *ref, int compact_format)
}
static void display_state_init(struct display_state *display_state, struct ref *ref_map,
- const char *raw_url)
+ const char *raw_url, enum display_format format)
{
- const char *format = "full";
int i;
memset(display_state, 0, sizeof(*display_state));
-
strbuf_init(&display_state->buf, 0);
+ display_state->format = format;
if (raw_url)
display_state->url = transport_anonymize_url(raw_url);
@@ -812,15 +811,6 @@ static void display_state_init(struct display_state *display_state, struct ref *
if (verbosity < 0)
return;
- git_config_get_string_tmp("fetch.output", &format);
- if (!strcasecmp(format, "full"))
- display_state->format = DISPLAY_FORMAT_FULL;
- else if (!strcasecmp(format, "compact"))
- display_state->format = DISPLAY_FORMAT_COMPACT;
- else
- die(_("invalid value for '%s': '%s'"),
- "fetch.output", format);
-
switch (display_state->format) {
case DISPLAY_FORMAT_FULL:
case DISPLAY_FORMAT_COMPACT: {
@@ -1614,7 +1604,8 @@ static int backfill_tags(struct display_state *display_state,
}
static int do_fetch(struct transport *transport,
- struct refspec *rs)
+ struct refspec *rs,
+ enum display_format display_format)
{
struct ref_transaction *transaction = NULL;
struct ref *ref_map = NULL;
@@ -1700,7 +1691,7 @@ static int do_fetch(struct transport *transport,
if (retcode)
goto cleanup;
- display_state_init(&display_state, ref_map, transport->url);
+ display_state_init(&display_state, ref_map, transport->url, display_format);
if (atomic_fetch) {
transaction = ref_transaction_begin(&err);
@@ -2076,7 +2067,8 @@ static inline void fetch_one_setup_partial(struct remote *remote)
}
static int fetch_one(struct remote *remote, int argc, const char **argv,
- int prune_tags_ok, int use_stdin_refspecs)
+ int prune_tags_ok, int use_stdin_refspecs,
+ enum display_format display_format)
{
struct refspec rs = REFSPEC_INIT_FETCH;
int i;
@@ -2143,7 +2135,7 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,
sigchain_push_common(unlock_pack_on_signal);
atexit(unlock_pack_atexit);
sigchain_push(SIGPIPE, SIG_IGN);
- exit_code = do_fetch(gtransport, &rs);
+ exit_code = do_fetch(gtransport, &rs, display_format);
sigchain_pop(SIGPIPE);
refspec_clear(&rs);
transport_disconnect(gtransport);
@@ -2155,6 +2147,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
{
int i;
const char *bundle_uri;
+ enum display_format display_format = DISPLAY_FORMAT_UNKNOWN;
struct string_list list = STRING_LIST_INIT_DUP;
struct remote *remote = NULL;
int result = 0;
@@ -2181,6 +2174,19 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
argc = parse_options(argc, argv, prefix,
builtin_fetch_options, builtin_fetch_usage, 0);
+ if (display_format == DISPLAY_FORMAT_UNKNOWN) {
+ const char *format = "full";
+
+ git_config_get_string_tmp("fetch.output", &format);
+ if (!strcasecmp(format, "full"))
+ display_format = DISPLAY_FORMAT_FULL;
+ else if (!strcasecmp(format, "compact"))
+ display_format = DISPLAY_FORMAT_COMPACT;
+ else
+ die(_("invalid value for '%s': '%s'"),
+ "fetch.output", format);
+ }
+
if (recurse_submodules_cli != RECURSE_SUBMODULES_DEFAULT)
recurse_submodules = recurse_submodules_cli;
@@ -2309,7 +2315,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
} else if (remote) {
if (filter_options.choice || has_promisor_remote())
fetch_one_setup_partial(remote);
- result = fetch_one(remote, argc, argv, prune_tags_ok, stdin_refspecs);
+ result = fetch_one(remote, argc, argv, prune_tags_ok, stdin_refspecs,
+ display_format);
} else {
int max_children = max_jobs;
--
2.40.0
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH 6/8] fetch: move option related variables into main function
2023-04-19 12:31 [PATCH 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (4 preceding siblings ...)
2023-04-19 12:31 ` [PATCH 5/8] fetch: move display format parsing into main function Patrick Steinhardt
@ 2023-04-19 12:31 ` Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 7/8] fetch: introduce new `--output-format` option Patrick Steinhardt
` (8 subsequent siblings)
14 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-19 12:31 UTC (permalink / raw)
To: git; +Cc: Jonathan Tan
[-- Attachment #1: Type: text/plain, Size: 12225 bytes --]
The options of git-fetch(1) which we pass to `parse_options()` are
declared globally in `builtin/fetch.c`. This means we're forced to use
global variables for all the options, which is more likely to cause
confusion than explicitly passing state around.
Refactor the code to move the options into `cmd_fetch()`. Move variables
that were previously forced to be declared globally and which are only
used by `cmd_fetch()` into function-local scope.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 194 ++++++++++++++++++++++++------------------------
1 file changed, 98 insertions(+), 96 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index bcc156a9ce..81581b0033 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -79,9 +79,8 @@ static int all, append, dry_run, force, keep, multiple, update_head_ok;
static int write_fetch_head = 1;
static int verbosity, deepen_relative, set_upstream, refetch;
static int progress = -1;
-static int enable_auto_gc = 1;
-static int tags = TAGS_DEFAULT, unshallow, update_shallow, deepen;
-static int max_jobs = -1, submodule_fetch_jobs_config = -1;
+static int tags = TAGS_DEFAULT, update_shallow, deepen;
+static int submodule_fetch_jobs_config = -1;
static int fetch_parallel_config = 1;
static int atomic_fetch;
static enum transport_family family;
@@ -92,17 +91,11 @@ static struct string_list deepen_not = STRING_LIST_INIT_NODUP;
static struct strbuf default_rla = STRBUF_INIT;
static struct transport *gtransport;
static struct transport *gsecondary;
-static const char *submodule_prefix = "";
static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
-static int recurse_submodules_cli = RECURSE_SUBMODULES_DEFAULT;
-static int recurse_submodules_default = RECURSE_SUBMODULES_ON_DEMAND;
static struct refspec refmap = REFSPEC_INIT_FETCH;
static struct list_objects_filter_options filter_options = LIST_OBJECTS_FILTER_INIT;
static struct string_list server_options = STRING_LIST_INIT_DUP;
static struct string_list negotiation_tip = STRING_LIST_INIT_NODUP;
-static int fetch_write_commit_graph = -1;
-static int stdin_refspecs = 0;
-static int negotiate_only;
static int git_fetch_config(const char *k, const char *v, void *cb)
{
@@ -160,92 +153,6 @@ static int parse_refmap_arg(const struct option *opt, const char *arg, int unset
return 0;
}
-static struct option builtin_fetch_options[] = {
- OPT__VERBOSITY(&verbosity),
- OPT_BOOL(0, "all", &all,
- N_("fetch from all remotes")),
- OPT_BOOL(0, "set-upstream", &set_upstream,
- N_("set upstream for git pull/fetch")),
- OPT_BOOL('a', "append", &append,
- N_("append to .git/FETCH_HEAD instead of overwriting")),
- OPT_BOOL(0, "atomic", &atomic_fetch,
- N_("use atomic transaction to update references")),
- OPT_STRING(0, "upload-pack", &upload_pack, N_("path"),
- N_("path to upload pack on remote end")),
- OPT__FORCE(&force, N_("force overwrite of local reference"), 0),
- OPT_BOOL('m', "multiple", &multiple,
- N_("fetch from multiple remotes")),
- OPT_SET_INT('t', "tags", &tags,
- N_("fetch all tags and associated objects"), TAGS_SET),
- OPT_SET_INT('n', NULL, &tags,
- N_("do not fetch all tags (--no-tags)"), TAGS_UNSET),
- OPT_INTEGER('j', "jobs", &max_jobs,
- N_("number of submodules fetched in parallel")),
- OPT_BOOL(0, "prefetch", &prefetch,
- N_("modify the refspec to place all refs within refs/prefetch/")),
- OPT_BOOL('p', "prune", &prune,
- N_("prune remote-tracking branches no longer on remote")),
- OPT_BOOL('P', "prune-tags", &prune_tags,
- N_("prune local tags no longer on remote and clobber changed tags")),
- OPT_CALLBACK_F(0, "recurse-submodules", &recurse_submodules_cli, N_("on-demand"),
- N_("control recursive fetching of submodules"),
- PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
- OPT_BOOL(0, "dry-run", &dry_run,
- N_("dry run")),
- OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
- N_("write fetched references to the FETCH_HEAD file")),
- OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
- OPT_BOOL('u', "update-head-ok", &update_head_ok,
- N_("allow updating of HEAD ref")),
- OPT_BOOL(0, "progress", &progress, N_("force progress reporting")),
- OPT_STRING(0, "depth", &depth, N_("depth"),
- N_("deepen history of shallow clone")),
- OPT_STRING(0, "shallow-since", &deepen_since, N_("time"),
- N_("deepen history of shallow repository based on time")),
- OPT_STRING_LIST(0, "shallow-exclude", &deepen_not, N_("revision"),
- N_("deepen history of shallow clone, excluding rev")),
- OPT_INTEGER(0, "deepen", &deepen_relative,
- N_("deepen history of shallow clone")),
- OPT_SET_INT_F(0, "unshallow", &unshallow,
- N_("convert to a complete repository"),
- 1, PARSE_OPT_NONEG),
- OPT_SET_INT_F(0, "refetch", &refetch,
- N_("re-fetch without negotiating common commits"),
- 1, PARSE_OPT_NONEG),
- { OPTION_STRING, 0, "submodule-prefix", &submodule_prefix, N_("dir"),
- N_("prepend this to submodule path output"), PARSE_OPT_HIDDEN },
- OPT_CALLBACK_F(0, "recurse-submodules-default",
- &recurse_submodules_default, N_("on-demand"),
- N_("default for recursive fetching of submodules "
- "(lower priority than config files)"),
- PARSE_OPT_HIDDEN, option_fetch_parse_recurse_submodules),
- OPT_BOOL(0, "update-shallow", &update_shallow,
- N_("accept refs that update .git/shallow")),
- OPT_CALLBACK_F(0, "refmap", NULL, N_("refmap"),
- N_("specify fetch refmap"), PARSE_OPT_NONEG, parse_refmap_arg),
- OPT_STRING_LIST('o', "server-option", &server_options, N_("server-specific"), N_("option to transmit")),
- OPT_SET_INT('4', "ipv4", &family, N_("use IPv4 addresses only"),
- TRANSPORT_FAMILY_IPV4),
- OPT_SET_INT('6', "ipv6", &family, N_("use IPv6 addresses only"),
- TRANSPORT_FAMILY_IPV6),
- OPT_STRING_LIST(0, "negotiation-tip", &negotiation_tip, N_("revision"),
- N_("report that we have only objects reachable from this object")),
- OPT_BOOL(0, "negotiate-only", &negotiate_only,
- N_("do not fetch a packfile; instead, print ancestors of negotiation tips")),
- OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
- OPT_BOOL(0, "auto-maintenance", &enable_auto_gc,
- N_("run 'maintenance --auto' after fetching")),
- OPT_BOOL(0, "auto-gc", &enable_auto_gc,
- N_("run 'maintenance --auto' after fetching")),
- OPT_BOOL(0, "show-forced-updates", &fetch_show_forced_updates,
- N_("check for forced-updates on all updated branches")),
- OPT_BOOL(0, "write-commit-graph", &fetch_write_commit_graph,
- N_("write the commit-graph after fetching")),
- OPT_BOOL(0, "stdin", &stdin_refspecs,
- N_("accept refspecs from stdin")),
- OPT_END()
-};
-
static void unlock_pack(unsigned int flags)
{
if (gtransport)
@@ -2145,13 +2052,108 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,
int cmd_fetch(int argc, const char **argv, const char *prefix)
{
- int i;
const char *bundle_uri;
+ const char *submodule_prefix = "";
enum display_format display_format = DISPLAY_FORMAT_UNKNOWN;
struct string_list list = STRING_LIST_INIT_DUP;
struct remote *remote = NULL;
int result = 0;
int prune_tags_ok = 1;
+ int enable_auto_gc = 1;
+ int unshallow = 0;
+ int max_jobs = -1;
+ int recurse_submodules_cli = RECURSE_SUBMODULES_DEFAULT;
+ int recurse_submodules_default = RECURSE_SUBMODULES_ON_DEMAND;
+ int fetch_write_commit_graph = -1;
+ int stdin_refspecs = 0;
+ int negotiate_only = 0;
+ int i;
+
+ struct option builtin_fetch_options[] = {
+ OPT__VERBOSITY(&verbosity),
+ OPT_BOOL(0, "all", &all,
+ N_("fetch from all remotes")),
+ OPT_BOOL(0, "set-upstream", &set_upstream,
+ N_("set upstream for git pull/fetch")),
+ OPT_BOOL('a', "append", &append,
+ N_("append to .git/FETCH_HEAD instead of overwriting")),
+ OPT_BOOL(0, "atomic", &atomic_fetch,
+ N_("use atomic transaction to update references")),
+ OPT_STRING(0, "upload-pack", &upload_pack, N_("path"),
+ N_("path to upload pack on remote end")),
+ OPT__FORCE(&force, N_("force overwrite of local reference"), 0),
+ OPT_BOOL('m', "multiple", &multiple,
+ N_("fetch from multiple remotes")),
+ OPT_SET_INT('t', "tags", &tags,
+ N_("fetch all tags and associated objects"), TAGS_SET),
+ OPT_SET_INT('n', NULL, &tags,
+ N_("do not fetch all tags (--no-tags)"), TAGS_UNSET),
+ OPT_INTEGER('j', "jobs", &max_jobs,
+ N_("number of submodules fetched in parallel")),
+ OPT_BOOL(0, "prefetch", &prefetch,
+ N_("modify the refspec to place all refs within refs/prefetch/")),
+ OPT_BOOL('p', "prune", &prune,
+ N_("prune remote-tracking branches no longer on remote")),
+ OPT_BOOL('P', "prune-tags", &prune_tags,
+ N_("prune local tags no longer on remote and clobber changed tags")),
+ OPT_CALLBACK_F(0, "recurse-submodules", &recurse_submodules_cli, N_("on-demand"),
+ N_("control recursive fetching of submodules"),
+ PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
+ OPT_BOOL(0, "dry-run", &dry_run,
+ N_("dry run")),
+ OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
+ N_("write fetched references to the FETCH_HEAD file")),
+ OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
+ OPT_BOOL('u', "update-head-ok", &update_head_ok,
+ N_("allow updating of HEAD ref")),
+ OPT_BOOL(0, "progress", &progress, N_("force progress reporting")),
+ OPT_STRING(0, "depth", &depth, N_("depth"),
+ N_("deepen history of shallow clone")),
+ OPT_STRING(0, "shallow-since", &deepen_since, N_("time"),
+ N_("deepen history of shallow repository based on time")),
+ OPT_STRING_LIST(0, "shallow-exclude", &deepen_not, N_("revision"),
+ N_("deepen history of shallow clone, excluding rev")),
+ OPT_INTEGER(0, "deepen", &deepen_relative,
+ N_("deepen history of shallow clone")),
+ OPT_SET_INT_F(0, "unshallow", &unshallow,
+ N_("convert to a complete repository"),
+ 1, PARSE_OPT_NONEG),
+ OPT_SET_INT_F(0, "refetch", &refetch,
+ N_("re-fetch without negotiating common commits"),
+ 1, PARSE_OPT_NONEG),
+ { OPTION_STRING, 0, "submodule-prefix", &submodule_prefix, N_("dir"),
+ N_("prepend this to submodule path output"), PARSE_OPT_HIDDEN },
+ OPT_CALLBACK_F(0, "recurse-submodules-default",
+ &recurse_submodules_default, N_("on-demand"),
+ N_("default for recursive fetching of submodules "
+ "(lower priority than config files)"),
+ PARSE_OPT_HIDDEN, option_fetch_parse_recurse_submodules),
+ OPT_BOOL(0, "update-shallow", &update_shallow,
+ N_("accept refs that update .git/shallow")),
+ OPT_CALLBACK_F(0, "refmap", NULL, N_("refmap"),
+ N_("specify fetch refmap"), PARSE_OPT_NONEG, parse_refmap_arg),
+ OPT_STRING_LIST('o', "server-option", &server_options, N_("server-specific"), N_("option to transmit")),
+ OPT_SET_INT('4', "ipv4", &family, N_("use IPv4 addresses only"),
+ TRANSPORT_FAMILY_IPV4),
+ OPT_SET_INT('6', "ipv6", &family, N_("use IPv6 addresses only"),
+ TRANSPORT_FAMILY_IPV6),
+ OPT_STRING_LIST(0, "negotiation-tip", &negotiation_tip, N_("revision"),
+ N_("report that we have only objects reachable from this object")),
+ OPT_BOOL(0, "negotiate-only", &negotiate_only,
+ N_("do not fetch a packfile; instead, print ancestors of negotiation tips")),
+ OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
+ OPT_BOOL(0, "auto-maintenance", &enable_auto_gc,
+ N_("run 'maintenance --auto' after fetching")),
+ OPT_BOOL(0, "auto-gc", &enable_auto_gc,
+ N_("run 'maintenance --auto' after fetching")),
+ OPT_BOOL(0, "show-forced-updates", &fetch_show_forced_updates,
+ N_("check for forced-updates on all updated branches")),
+ OPT_BOOL(0, "write-commit-graph", &fetch_write_commit_graph,
+ N_("write the commit-graph after fetching")),
+ OPT_BOOL(0, "stdin", &stdin_refspecs,
+ N_("accept refspecs from stdin")),
+ OPT_END()
+ };
packet_trace_identity("fetch");
--
2.40.0
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH 7/8] fetch: introduce new `--output-format` option
2023-04-19 12:31 [PATCH 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (5 preceding siblings ...)
2023-04-19 12:31 ` [PATCH 6/8] fetch: move option related variables " Patrick Steinhardt
@ 2023-04-19 12:31 ` Patrick Steinhardt
2023-04-26 19:40 ` Glen Choo
2023-04-19 12:31 ` [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
` (7 subsequent siblings)
14 siblings, 1 reply; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-19 12:31 UTC (permalink / raw)
To: git; +Cc: Jonathan Tan
[-- Attachment #1: Type: text/plain, Size: 5106 bytes --]
It is only possible to configure the output format that git-fetch(1)
uses by setting it via a config key. While this interface may be fine as
long as we only have the current "full" and "compact" output formats,
where it is unlikely that the user will have to change them regularly.
But we're about to introduce a new machine-parseable interface where the
current mechanism feels a little bit indirect and rigid.
Introduce a new `--output-format` option that allows the user to change
the desired output format more directly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
Documentation/fetch-options.txt | 5 ++++
builtin/fetch.c | 17 +++++++++++
t/t5574-fetch-output.sh | 50 +++++++++++++++++++++++++--------
3 files changed, 60 insertions(+), 12 deletions(-)
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 622bd84768..654f96f79d 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -78,6 +78,11 @@ linkgit:git-config[1].
--dry-run::
Show what would be done, without making any changes.
+--output-format::
+ Control how ref update status is printed. Valid values are
+ `full` and `compact`. Default value is `full`. See section
+ OUTPUT in linkgit:git-fetch[1] for detail.
+
ifndef::git-pull[]
--[no-]write-fetch-head::
Write the list of remote refs fetched in the `FETCH_HEAD`
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 81581b0033..22ba75a958 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -2050,6 +2050,21 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,
return exit_code;
}
+static int opt_parse_output_format(const struct option *opt, const char *arg, int unset)
+{
+ enum display_format *format = opt->value;
+ if (unset || !arg)
+ return 1;
+ else if (!strcmp(arg, "full"))
+ *format = DISPLAY_FORMAT_FULL;
+ else if (!strcmp(arg, "compact"))
+ *format = DISPLAY_FORMAT_COMPACT;
+ else
+ return error(_("unsupported output format '%s'"), arg);
+
+ return 0;
+}
+
int cmd_fetch(int argc, const char **argv, const char *prefix)
{
const char *bundle_uri;
@@ -2101,6 +2116,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
OPT_BOOL(0, "dry-run", &dry_run,
N_("dry run")),
+ OPT_CALLBACK(0, "output-format", &display_format, N_("format"), N_("output format"),
+ opt_parse_output_format),
OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
N_("write fetched references to the FETCH_HEAD file")),
OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index 55f0f05b6a..5144d5ed21 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -24,14 +24,37 @@ test_expect_success 'fetch with invalid output format configuration' '
test_cmp expect actual
'
+test_expect_success 'fetch with invalid output format via command line' '
+ test_must_fail git fetch --output-format >actual 2>&1 &&
+ cat >expect <<-EOF &&
+ error: option \`output-format${SQ} requires a value
+ EOF
+ test_cmp expect actual &&
+
+ test_must_fail git fetch --output-format= origin >actual 2>&1 &&
+ cat >expect <<-EOF &&
+ error: unsupported output format ${SQ}${SQ}
+ EOF
+ test_cmp expect actual &&
+
+ test_must_fail git fetch --output-format=garbage origin >actual 2>&1 &&
+ cat >expect <<-EOF &&
+ error: unsupported output format ${SQ}garbage${SQ}
+ EOF
+ test_cmp expect actual
+'
+
test_expect_success 'fetch aligned output' '
- git clone . full-output &&
+ test_when_finished "rm -rf full-cfg full-cli" &&
+ git clone . full-cfg &&
+ git clone . full-cli &&
test_commit looooooooooooong-tag &&
- (
- cd full-output &&
- git -c fetch.output=full fetch origin >actual 2>&1 &&
- grep -e "->" actual | cut -c 22- >../actual
- ) &&
+
+ git -C full-cfg -c fetch.output=full fetch origin >actual-cfg 2>&1 &&
+ git -C full-cli fetch --output-format=full origin >actual-cli 2>&1 &&
+ test_cmp actual-cfg actual-cli &&
+
+ grep -e "->" actual-cfg | cut -c 22- >actual &&
cat >expect <<-\EOF &&
main -> origin/main
looooooooooooong-tag -> looooooooooooong-tag
@@ -40,13 +63,16 @@ test_expect_success 'fetch aligned output' '
'
test_expect_success 'fetch compact output' '
- git clone . compact &&
+ test_when_finished "rm -rf compact-cfg compact-cli" &&
+ git clone . compact-cli &&
+ git clone . compact-cfg &&
test_commit extraaa &&
- (
- cd compact &&
- git -c fetch.output=compact fetch origin >actual 2>&1 &&
- grep -e "->" actual | cut -c 22- >../actual
- ) &&
+
+ git -C compact-cfg -c fetch.output=compact fetch origin >actual-cfg 2>&1 &&
+ git -C compact-cli fetch --output-format=compact origin >actual-cli 2>&1 &&
+ test_cmp actual-cfg actual-cli &&
+
+ grep -e "->" actual-cfg | cut -c 22- >actual &&
cat >expect <<-\EOF &&
main -> origin/*
extraaa -> *
--
2.40.0
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-04-19 12:31 [PATCH 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (6 preceding siblings ...)
2023-04-19 12:31 ` [PATCH 7/8] fetch: introduce new `--output-format` option Patrick Steinhardt
@ 2023-04-19 12:31 ` Patrick Steinhardt
2023-04-26 19:52 ` Glen Choo
2023-04-24 20:17 ` [PATCH 0/8] fetch: introduce machine-parseable output Felipe Contreras
` (6 subsequent siblings)
14 siblings, 1 reply; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-19 12:31 UTC (permalink / raw)
To: git; +Cc: Jonathan Tan
[-- Attachment #1: Type: text/plain, Size: 15135 bytes --]
The output of git-fetch(1) is obviously designed for consumption by
users, only: we neatly columnize data, we abbreviate reference names, we
print neat arrows and we don't provide information about actual object
IDs that have changed. This makes the output format basically unusable
in the context of scripted invocations of git-fetch(1) that want to
learn about the exact changes that the command performs.
Introduce a new machine-parseable "porcelain" output format that is
supposed to fix this shortcoming. This output format is intended to
provide information about every reference that is about to be updated,
the old object ID that the reference has been pointing to and the new
object ID it will be updated to. Furthermore, the output format provides
the same flags as the human-readable format to indicate basic conditions
for each reference update like whether it was a fast-forward update, a
branch deletion, a rejected update or others.
The output format is quite simple:
```
<flag> <old-object-id> <new-object-id> <local-reference>\n
```
We assume two conditions which are generally true:
- The old and new object IDs have fixed known widths and cannot
contain spaces.
- References cannot contain newlines.
With these assumptions, the output format becomes unambiguously
parseable. Furthermore, given that this output is designed to be
consumed by scripts, the machine-readable data is printed to stdout
instead of stderr like the human-readable output is. This is mostly done
so that other data printed to stderr, like error messages or progress
meters, don't interfere with the parseable data.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
Documentation/config/fetch.txt | 4 +-
Documentation/fetch-options.txt | 4 +-
Documentation/git-fetch.txt | 17 ++++++-
builtin/fetch.c | 50 +++++++++++++++-----
t/t5574-fetch-output.sh | 84 +++++++++++++++++++++++++++++++++
5 files changed, 142 insertions(+), 17 deletions(-)
diff --git a/Documentation/config/fetch.txt b/Documentation/config/fetch.txt
index 568f0f75b3..70734226c0 100644
--- a/Documentation/config/fetch.txt
+++ b/Documentation/config/fetch.txt
@@ -52,8 +52,8 @@ fetch.pruneTags::
fetch.output::
Control how ref update status is printed. Valid values are
- `full` and `compact`. Default value is `full`. See section
- OUTPUT in linkgit:git-fetch[1] for detail.
+ `full`, `compact` and `porcelain`. Default value is `full`.
+ See section OUTPUT in linkgit:git-fetch[1] for detail.
fetch.negotiationAlgorithm::
Control how information about the commits in the local repository
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 654f96f79d..5ca8a67fe8 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -80,8 +80,8 @@ linkgit:git-config[1].
--output-format::
Control how ref update status is printed. Valid values are
- `full` and `compact`. Default value is `full`. See section
- OUTPUT in linkgit:git-fetch[1] for detail.
+ `full`, `compact` and `porcelain`. Default value is `full`.
+ See section OUTPUT in linkgit:git-fetch[1] for detail.
ifndef::git-pull[]
--[no-]write-fetch-head::
diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index fba66f1460..efd22cd372 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -197,13 +197,26 @@ The output of "git fetch" depends on the transport method used; this
section describes the output when fetching over the Git protocol
(either locally or via ssh) and Smart HTTP protocol.
-The status of the fetch is output in tabular form, with each line
-representing the status of a single ref. Each line is of the form:
+The output format can be chosen either via the `fetch.output` config
+(see linkgit:git-config[1]), or via the `--output-format` switch.
+Supported values include:
+
+For the `full` and `compact` output formats, the status of the fetch is
+output in tabular, with each line representing the status of a single
+ref. Each line is of the form:
-------------------------------
<flag> <summary> <from> -> <to> [<reason>]
-------------------------------
+The `porcelain` output format is intended to be machine-parseable. In
+contrast to the human-readable output formats it thus prints to standard
+output instead of standard error. Each line is of the form:
+
+-------------------------------
+<flag> <old-object-id> <new-object-id> <local-reference>
+-------------------------------
+
The status of up-to-date refs is shown only if the --verbose option is
used.
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 22ba75a958..a99e24448b 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -52,6 +52,7 @@ enum display_format {
DISPLAY_FORMAT_UNKNOWN = 0,
DISPLAY_FORMAT_FULL,
DISPLAY_FORMAT_COMPACT,
+ DISPLAY_FORMAT_PORCELAIN,
};
struct display_state {
@@ -745,8 +746,11 @@ static void display_state_init(struct display_state *display_state, struct ref *
break;
}
+ case DISPLAY_FORMAT_PORCELAIN:
+ /* We don't need to precompute anything here. */
+ break;
default:
- BUG("unexpected display foramt %d", display_state->format);
+ BUG("unexpected display format %d", display_state->format);
}
}
@@ -815,8 +819,12 @@ static void print_compact(struct display_state *display_state,
static void display_ref_update(struct display_state *display_state, char code,
const char *summary, const char *error,
const char *remote, const char *local,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
int summary_width)
{
+ FILE *f = stderr;
+
if (verbosity < 0)
return;
@@ -849,12 +857,17 @@ static void display_ref_update(struct display_state *display_state, char code,
break;
}
+ case DISPLAY_FORMAT_PORCELAIN:
+ strbuf_addf(&display_state->buf, "%c %s %s %s", code,
+ oid_to_hex(old_oid), oid_to_hex(new_oid), local);
+ f = stdout;
+ break;
default:
BUG("unexpected display format %d", display_state->format);
};
strbuf_addch(&display_state->buf, '\n');
- fputs(display_state->buf.buf, stderr);
+ fputs(display_state->buf.buf, f);
}
static int update_local_ref(struct ref *ref,
@@ -872,7 +885,8 @@ static int update_local_ref(struct ref *ref,
if (oideq(&ref->old_oid, &ref->new_oid)) {
if (verbosity > 0)
display_ref_update(display_state, '=', _("[up to date]"), NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 0;
}
@@ -885,7 +899,8 @@ static int update_local_ref(struct ref *ref,
*/
display_ref_update(display_state, '!', _("[rejected]"),
_("can't fetch into checked-out branch"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 1;
}
@@ -896,12 +911,14 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("updating tag", ref, transaction, 0);
display_ref_update(display_state, r ? '!' : 't', _("[tag update]"),
r ? _("unable to update local ref") : NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"),
_("would clobber existing tag"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 1;
}
}
@@ -934,7 +951,8 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref(msg, ref, transaction, 0);
display_ref_update(display_state, r ? '!' : '*', what,
r ? _("unable to update local ref") : NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return r;
}
@@ -956,7 +974,8 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("fast-forward", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : ' ', quickref.buf,
r ? _("unable to update local ref") : NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
strbuf_release(&quickref);
return r;
} else if (force || ref->force) {
@@ -968,12 +987,14 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("forced-update", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : '+', quickref.buf,
r ? _("unable to update local ref") : _("forced update"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
strbuf_release(&quickref);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"), _("non-fast-forward"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 1;
}
}
@@ -1214,7 +1235,9 @@ static int store_updated_refs(struct display_state *display_state,
display_ref_update(display_state, '*',
*kind ? kind : "branch", NULL,
rm->name,
- "FETCH_HEAD", summary_width);
+ "FETCH_HEAD",
+ &rm->new_oid, &rm->old_oid,
+ summary_width);
}
}
}
@@ -1354,6 +1377,7 @@ static int prune_refs(struct display_state *display_state,
for (ref = stale_refs; ref; ref = ref->next) {
display_ref_update(display_state, '-', _("[deleted]"), NULL,
_("(none)"), ref->name,
+ &ref->new_oid, &ref->old_oid,
summary_width);
warn_dangling_symref(stderr, dangling_msg, ref->name);
}
@@ -2059,6 +2083,8 @@ static int opt_parse_output_format(const struct option *opt, const char *arg, in
*format = DISPLAY_FORMAT_FULL;
else if (!strcmp(arg, "compact"))
*format = DISPLAY_FORMAT_COMPACT;
+ else if (!strcmp(arg, "porcelain"))
+ *format = DISPLAY_FORMAT_PORCELAIN;
else
return error(_("unsupported output format '%s'"), arg);
@@ -2201,6 +2227,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
display_format = DISPLAY_FORMAT_FULL;
else if (!strcasecmp(format, "compact"))
display_format = DISPLAY_FORMAT_COMPACT;
+ else if (!strcasecmp(format, "porcelain"))
+ display_format = DISPLAY_FORMAT_PORCELAIN;
else
die(_("invalid value for '%s': '%s'"),
"fetch.output", format);
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index 5144d5ed21..28d02a93e4 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -80,6 +80,72 @@ test_expect_success 'fetch compact output' '
test_cmp expect actual
'
+test_expect_success 'fetch porcelain output' '
+ test_when_finished "rm -rf porcelain-cfg porcelain-cli" &&
+
+ # Set up a bunch of references that we can use to demonstrate different
+ # kinds of flag symbols in the output format.
+ MAIN_OLD=$(git rev-parse HEAD) &&
+ git branch "fast-forward" &&
+ git branch "deleted-branch" &&
+ git checkout -b force-updated &&
+ test_commit --no-tag force-update-old &&
+ FORCE_UPDATED_OLD=$(git rev-parse HEAD) &&
+ git checkout main &&
+
+ # Clone and pre-seed the repositories. We fetch references into two
+ # namespaces so that we can test that rejected and force-updated
+ # references are reported properly.
+ refspecs="refs/heads/*:refs/unforced/* +refs/heads/*:refs/forced/*" &&
+ git clone . porcelain-cli &&
+ git clone . porcelain-cfg &&
+ git -C porcelain-cfg fetch origin $refspecs &&
+ git -C porcelain-cli fetch origin $refspecs &&
+
+ # Now that we have set up the client repositories we can change our
+ # local references.
+ git branch new-branch &&
+ git branch -d deleted-branch &&
+ git checkout fast-forward &&
+ test_commit --no-tag fast-forward-new &&
+ FAST_FORWARD_NEW=$(git rev-parse HEAD) &&
+ git checkout force-updated &&
+ git reset --hard HEAD~ &&
+ test_commit --no-tag force-update-new &&
+ FORCE_UPDATED_NEW=$(git rev-parse HEAD) &&
+
+ cat >expect <<-EOF &&
+ - $MAIN_OLD $ZERO_OID refs/forced/deleted-branch
+ - $MAIN_OLD $ZERO_OID refs/unforced/deleted-branch
+ $MAIN_OLD $FAST_FORWARD_NEW refs/unforced/fast-forward
+ ! $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/unforced/force-updated
+ * $ZERO_OID $MAIN_OLD refs/unforced/new-branch
+ $MAIN_OLD $FAST_FORWARD_NEW refs/forced/fast-forward
+ + $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/forced/force-updated
+ * $ZERO_OID $MAIN_OLD refs/forced/new-branch
+ $MAIN_OLD $FAST_FORWARD_NEW refs/remotes/origin/fast-forward
+ + $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/remotes/origin/force-updated
+ * $ZERO_OID $MAIN_OLD refs/remotes/origin/new-branch
+ EOF
+
+ # Execute a dry-run fetch first. We do this to assert that the dry-run
+ # and non-dry-run fetches produces the same output. Execution of the
+ # fetch is expected to fail as we have a rejected reference update.
+ test_must_fail git -C porcelain-cfg -c fetch.output=porcelain fetch --dry-run --prune origin $refspecs >actual-dry-run-cfg &&
+ test_must_fail git -C porcelain-cli fetch --output-format=porcelain --dry-run --prune origin $refspecs >actual-dry-run-cli &&
+ test_cmp actual-dry-run-cfg actual-dry-run-cli &&
+ test_cmp expect actual-dry-run-cfg &&
+
+ # And now we perform a non-dry-run fetch.
+ test_must_fail git -C porcelain-cfg -c fetch.output=porcelain fetch --prune origin $refspecs >actual-cfg &&
+ test_must_fail git -C porcelain-cli fetch --output-format=porcelain --prune origin $refspecs >actual-cli &&
+ test_cmp actual-cfg actual-cli &&
+ test_cmp expect actual-cfg &&
+
+ # Ensure that the dry-run and non-dry-run output matches.
+ test_cmp actual-dry-run-cfg actual-cfg
+'
+
test_expect_success 'fetch output with HEAD and --dry-run' '
test_when_finished "rm -rf head" &&
git clone . head &&
@@ -99,6 +165,24 @@ test_expect_success 'fetch output with HEAD and --dry-run' '
test_cmp expect actual
'
+test_expect_success 'fetch porcelain output with HEAD and --dry-run' '
+ test_when_finished "rm -rf head" &&
+ git clone . head &&
+ COMMIT_ID=$(git rev-parse HEAD) &&
+
+ git -C head fetch --output-format=porcelain --dry-run origin HEAD >actual &&
+ cat >expect <<-EOF &&
+ * $ZERO_OID $COMMIT_ID FETCH_HEAD
+ EOF
+ test_cmp expect actual &&
+
+ git -C head fetch --output-format=porcelain --dry-run origin HEAD:foo >actual &&
+ cat >expect <<-EOF &&
+ * $ZERO_OID $COMMIT_ID refs/heads/foo
+ EOF
+ test_cmp expect actual
+'
+
test_expect_success '--no-show-forced-updates' '
mkdir forced-updates &&
(
--
2.40.0
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* Re: [PATCH 0/8] fetch: introduce machine-parseable output
2023-04-19 12:31 [PATCH 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (7 preceding siblings ...)
2023-04-19 12:31 ` [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
@ 2023-04-24 20:17 ` Felipe Contreras
2023-04-25 9:58 ` Patrick Steinhardt
2023-04-26 18:54 ` Glen Choo
` (5 subsequent siblings)
14 siblings, 1 reply; 120+ messages in thread
From: Felipe Contreras @ 2023-04-24 20:17 UTC (permalink / raw)
To: Patrick Steinhardt, git; +Cc: Jonathan Tan
Patrick Steinhardt wrote:
> Parsing the output of fetches is mostly impossible. It prettifies
> reference names that are about to be updated, doesn't print the old and
> new object IDs the refs are being updated from and to, and prints all of
> that information in nice columns. In short, it is designed to be read by
> humans rather than machines.
>
> This makes it hard to use in a script way though, e.g. to learn about
> which references actually have been updated or which have not been
> updated. This patch series intends to fix that by introducing a new
> machine-parseable interface:
>
> ```
> $ git fetch --output-format=porcelain --no-progress
> fff5a5e7f528b2ed2c335991399a766c2cf01103 af67688dca57999fd848f051eeea1d375ba546b2 refs/remotes/origin/master
> * 0000000000000000000000000000000000000000 e046fe5a36a970bc14fbfbcb2074a48776f6b671 refs/remotes/origin/x86-rep-insns
> * 0000000000000000000000000000000000000000 bb81ed6862b864c9eb99447f04d49a84ecb647e5 refs/tags/v6.3-rc4
> * 0000000000000000000000000000000000000000 83af7b1468c0dca86b4dc9e43e73bfa4f38d9637 refs/tags/v6.3-rc5
> * 0000000000000000000000000000000000000000 ab3affb8ed84f68638162fe7e6fd4055e15bff5b refs/tags/v6.3-rc6
> * 0000000000000000000000000000000000000000 1c8c28415e8743368a2b800520a6dd0b22ee6ec2 refs/tags/v6.3-rc7
> ```
Makes sense, my only question is what other format could `git fetch` have? I
think `--format=porcelain` is clear enough.
--
Felipe Contreras
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 0/8] fetch: introduce machine-parseable output
2023-04-24 20:17 ` [PATCH 0/8] fetch: introduce machine-parseable output Felipe Contreras
@ 2023-04-25 9:58 ` Patrick Steinhardt
2023-04-26 19:14 ` Jacob Keller
0 siblings, 1 reply; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-25 9:58 UTC (permalink / raw)
To: Felipe Contreras; +Cc: git, Jonathan Tan
[-- Attachment #1: Type: text/plain, Size: 1808 bytes --]
On Mon, Apr 24, 2023 at 02:17:31PM -0600, Felipe Contreras wrote:
> Patrick Steinhardt wrote:
> > Parsing the output of fetches is mostly impossible. It prettifies
> > reference names that are about to be updated, doesn't print the old and
> > new object IDs the refs are being updated from and to, and prints all of
> > that information in nice columns. In short, it is designed to be read by
> > humans rather than machines.
> >
> > This makes it hard to use in a script way though, e.g. to learn about
> > which references actually have been updated or which have not been
> > updated. This patch series intends to fix that by introducing a new
> > machine-parseable interface:
> >
> > ```
> > $ git fetch --output-format=porcelain --no-progress
> > fff5a5e7f528b2ed2c335991399a766c2cf01103 af67688dca57999fd848f051eeea1d375ba546b2 refs/remotes/origin/master
> > * 0000000000000000000000000000000000000000 e046fe5a36a970bc14fbfbcb2074a48776f6b671 refs/remotes/origin/x86-rep-insns
> > * 0000000000000000000000000000000000000000 bb81ed6862b864c9eb99447f04d49a84ecb647e5 refs/tags/v6.3-rc4
> > * 0000000000000000000000000000000000000000 83af7b1468c0dca86b4dc9e43e73bfa4f38d9637 refs/tags/v6.3-rc5
> > * 0000000000000000000000000000000000000000 ab3affb8ed84f68638162fe7e6fd4055e15bff5b refs/tags/v6.3-rc6
> > * 0000000000000000000000000000000000000000 1c8c28415e8743368a2b800520a6dd0b22ee6ec2 refs/tags/v6.3-rc7
> > ```
>
> Makes sense, my only question is what other format could `git fetch` have? I
> think `--format=porcelain` is clear enough.
Yeah, I'd be perfectly happy to rename this to `--format=porcelain`.
I'll wait for the Review Club that discusses this patch set tomorrow and
will send a new version with that change afterwards if nobody disagrees.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 0/8] fetch: introduce machine-parseable output
2023-04-19 12:31 [PATCH 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (8 preceding siblings ...)
2023-04-24 20:17 ` [PATCH 0/8] fetch: introduce machine-parseable output Felipe Contreras
@ 2023-04-26 18:54 ` Glen Choo
2023-04-26 21:14 ` Glen Choo
2023-04-26 19:17 ` Jacob Keller
` (4 subsequent siblings)
14 siblings, 1 reply; 120+ messages in thread
From: Glen Choo @ 2023-04-26 18:54 UTC (permalink / raw)
To: Patrick Steinhardt, git; +Cc: Jonathan Tan
Hi Patrick!
Thanks for the pleasant read! I thought this was a great topic for
Review Club. It's too bad that we missed you, but we post all relevant
feedback here anyway.
Nevertheless, if you'd like to see the meeting notes, you can find them
at:
https://docs.google.com/document/d/14L8BAumGTpsXpjDY8VzZ4rRtpAjuGrFSRqn3stCuS_w/edit
Patrick Steinhardt <ps@pks.im> writes:
> Parsing the output of fetches is mostly impossible. It prettifies
> reference names that are about to be updated, doesn't print the old and
> new object IDs the refs are being updated from and to, and prints all of
> that information in nice columns. In short, it is designed to be read by
> humans rather than machines.
>
> This makes it hard to use in a script way though, e.g. to learn about
> which references actually have been updated or which have not been
> updated. This patch series intends to fix that by introducing a new
> machine-parseable interface:
>
> ```
> $ git fetch --output-format=porcelain --no-progress
> fff5a5e7f528b2ed2c335991399a766c2cf01103 af67688dca57999fd848f051eeea1d375ba546b2 refs/remotes/origin/master
> * 0000000000000000000000000000000000000000 e046fe5a36a970bc14fbfbcb2074a48776f6b671 refs/remotes/origin/x86-rep-insns
> * 0000000000000000000000000000000000000000 bb81ed6862b864c9eb99447f04d49a84ecb647e5 refs/tags/v6.3-rc4
> * 0000000000000000000000000000000000000000 83af7b1468c0dca86b4dc9e43e73bfa4f38d9637 refs/tags/v6.3-rc5
> * 0000000000000000000000000000000000000000 ab3affb8ed84f68638162fe7e6fd4055e15bff5b refs/tags/v6.3-rc6
> * 0000000000000000000000000000000000000000 1c8c28415e8743368a2b800520a6dd0b22ee6ec2 refs/tags/v6.3-rc7
> ```
Having machine-parseable output seems like an obviously good goal to me.
The finer points of the interface and output format are worth
discussing. I'll do so in the patches themselves.
Overall, I found the series very well-structured and easy to follow
along. Thanks.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 0/8] fetch: introduce machine-parseable output
2023-04-25 9:58 ` Patrick Steinhardt
@ 2023-04-26 19:14 ` Jacob Keller
2023-04-26 20:23 ` Junio C Hamano
2023-04-26 20:24 ` Junio C Hamano
0 siblings, 2 replies; 120+ messages in thread
From: Jacob Keller @ 2023-04-26 19:14 UTC (permalink / raw)
To: Patrick Steinhardt, Felipe Contreras; +Cc: git, Jonathan Tan
On 4/25/2023 2:58 AM, Patrick Steinhardt wrote:
> On Mon, Apr 24, 2023 at 02:17:31PM -0600, Felipe Contreras wrote:
>> Patrick Steinhardt wrote:
>>> Parsing the output of fetches is mostly impossible. It prettifies
>>> reference names that are about to be updated, doesn't print the old and
>>> new object IDs the refs are being updated from and to, and prints all of
>>> that information in nice columns. In short, it is designed to be read by
>>> humans rather than machines.
>>>
>>> This makes it hard to use in a script way though, e.g. to learn about
>>> which references actually have been updated or which have not been
>>> updated. This patch series intends to fix that by introducing a new
>>> machine-parseable interface:
>>>
>>> ```
>>> $ git fetch --output-format=porcelain --no-progress
>>> fff5a5e7f528b2ed2c335991399a766c2cf01103 af67688dca57999fd848f051eeea1d375ba546b2 refs/remotes/origin/master
>>> * 0000000000000000000000000000000000000000 e046fe5a36a970bc14fbfbcb2074a48776f6b671 refs/remotes/origin/x86-rep-insns
>>> * 0000000000000000000000000000000000000000 bb81ed6862b864c9eb99447f04d49a84ecb647e5 refs/tags/v6.3-rc4
>>> * 0000000000000000000000000000000000000000 83af7b1468c0dca86b4dc9e43e73bfa4f38d9637 refs/tags/v6.3-rc5
>>> * 0000000000000000000000000000000000000000 ab3affb8ed84f68638162fe7e6fd4055e15bff5b refs/tags/v6.3-rc6
>>> * 0000000000000000000000000000000000000000 1c8c28415e8743368a2b800520a6dd0b22ee6ec2 refs/tags/v6.3-rc7
>>> ```
>>
>> Makes sense, my only question is what other format could `git fetch` have? I
>> think `--format=porcelain` is clear enough.
>
> Yeah, I'd be perfectly happy to rename this to `--format=porcelain`.
> I'll wait for the Review Club that discusses this patch set tomorrow and
> will send a new version with that change afterwards if nobody disagrees.
>
> Patrick
We had some discussion during review club about this, where the idea of
using "--porcelain" came up because many commands use that when
switching into a machine readable format.
In addition, this format not only changes the output but also moves it
from being on stderr to stdout, which is a hint that the intended usage
of the command is now a little different.
I don't have strong opinion here but want to note that "--format" is
often used by commands like log which changes how we structure the
output of git objects.
Obviously using "--porcelain" is a bit weird when dealing with the
pre-existing compact and full outputs, and perhaps --format wouldn't be
confusing.
I didn't find any command which used --output-format today, and all the
uses of --format I saw were for object formatting like git log.
I'm ok with --format, but wanted to note the potential confusion.
git status also uses --porcelain and has a -z option for NUL terminating
instead of newline terminating. We thought -z might be useful to handle
the potential of weird refnames that include characters. No one on the
review could remember what rules were enforced on refnames to confirm if
it was legal to have '\n' in a refname or not.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 0/8] fetch: introduce machine-parseable output
2023-04-19 12:31 [PATCH 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (9 preceding siblings ...)
2023-04-26 18:54 ` Glen Choo
@ 2023-04-26 19:17 ` Jacob Keller
2023-04-27 11:13 ` [PATCH v2 " Patrick Steinhardt
` (3 subsequent siblings)
14 siblings, 0 replies; 120+ messages in thread
From: Jacob Keller @ 2023-04-26 19:17 UTC (permalink / raw)
To: Patrick Steinhardt, git; +Cc: Jonathan Tan
On 4/19/2023 5:31 AM, Patrick Steinhardt wrote:
>
> ```
> $ git fetch --output-format=porcelain --no-progress
> fff5a5e7f528b2ed2c335991399a766c2cf01103 af67688dca57999fd848f051eeea1d375ba546b2 refs/remotes/origin/master
> * 0000000000000000000000000000000000000000 e046fe5a36a970bc14fbfbcb2074a48776f6b671 refs/remotes/origin/x86-rep-insns
> * 0000000000000000000000000000000000000000 bb81ed6862b864c9eb99447f04d49a84ecb647e5 refs/tags/v6.3-rc4
> * 0000000000000000000000000000000000000000 83af7b1468c0dca86b4dc9e43e73bfa4f38d9637 refs/tags/v6.3-rc5
> * 0000000000000000000000000000000000000000 ab3affb8ed84f68638162fe7e6fd4055e15bff5b refs/tags/v6.3-rc6
> * 0000000000000000000000000000000000000000 1c8c28415e8743368a2b800520a6dd0b22ee6ec2 refs/tags/v6.3-rc7
> ```
>
One thing that the standard output (maybe with --progress?) shows to
stderr is which remote is being fetched and what URL was fetched.
This seems like useful data to include in a machine readable format. It
wasn't clear if such output would still be made and whether it would go
to stderr or stdout, nor whether that existing output was machine readable.
Obviously you can somewhat infer remotes based on the refs/remotes, but
that doesn't include other refs like tags.
When fetching from multiple reports (--all or explicitly asking), I
wonder if it makes sense to include some lines to differentiate where
each block of updates for a given remote starts and ends?
Thanks,
Jake
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo
2023-04-19 12:31 ` [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
@ 2023-04-26 19:20 ` Jacob Keller
2023-04-27 10:58 ` Patrick Steinhardt
2023-04-26 19:21 ` Jacob Keller
2023-04-26 19:25 ` Glen Choo
2 siblings, 1 reply; 120+ messages in thread
From: Jacob Keller @ 2023-04-26 19:20 UTC (permalink / raw)
To: Patrick Steinhardt, git; +Cc: Jonathan Tan
On 4/19/2023 5:31 AM, Patrick Steinhardt wrote:
> When displaying reference updates, we print a line that looks similar to
> the following:
>
> ```
> * branch master -> master
> ```
>
> The "branch" bit changes depending on what kind of reference we're
> updating, while both of the right-hand references are computed by
> stripping well-known prefixes like "refs/heads/" or "refs/tags".
>
> The logic is kind of intertwined though and not easy to follow: we
> precompute both the kind (e.g. "branch") and the what, which is the
> abbreviated remote reference name, in `store_updated_refs()` and then
> pass it down the call chain to `display_ref_update()`.
>
> There is a set of different cases here:
>
> - When the remote reference name is "HEAD" we assume no kind and
> will thus instead print "[new ref]". We keep what at the empty
> string.
>
> - When the remote reference name has a well-known prefix then the
> kind would be "branch", "tag" or "remote-tracking branch". The
> what is the reference with the well-known prefix stripped and in
> fact matches the output that `prettify_refname()` would return.
>
> - Otherwise, we'll again assume no kind and keep the what set to the
> fully qualified reference name.
>
> Now there is a bug with the first case here, where the remote reference
> name is "HEAD". As noted, "what" will be set to the empty string. And
> that seems to be intentional because we also use this information to
> update the FETCH_HEAD, and in case we're updating HEAD we seemingly
> don't want to append that to our FETCH_HEAD value.
>
> But as mentioned, we also use this value to display reference updates.
> And while the call to `display_ref_update()` correctly figures out that
> we meant "HEAD" when `what` is empty, the call to `update_local_ref()`
> doesn't. `update_local_ref()` will then call `display_ref_update()` with
> the empty string and cause the following broken output:
>
> ```
> $ git fetch --dry-run origin HEAD:foo
> From https://github.com/git/git
> * [new ref] -> foo
> ```
>
> The HEAD string is clearly missing from the left-hand side of the arrow,
> which is further stressed by the point that the following commands work
> as expected:
>
> ```
> $ git fetch --dry-run origin HEAD
> From https://github.com/git/git
> * branch HEAD -> FETCH_HEAD
>
> $ git fetch --dry-run origin master
> From https://github.com/git/git
> * branch master -> FETCH_HEAD
> * branch master -> origin/master
> ```
>
> Fix this bug by instead unconditionally passing the full reference name
> to `display_ref_update()` which learns to call `prettify_refname()` on
> it. This does fix the above bug and is otherwise functionally the same
> as `prettify_refname()` would only ever strip the well-known prefixes
> just as intended. So at the same time, this also simplifies the code a
> bit.
>
> Note that this patch also changes formatting of the block that computes
> the "kind" and "what" variables. This is done on purpose so that it is
> part of the diff, hopefully making the change easier to comprehend.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
The commit message here has a lot of context, but I found it a bit hard
to parse through, especially relative to the actual fix in code.
One suggestion was to load the paragraphs a bit more with the actual
problem being solved first, before beginning a lot of the context.
We also discussed the block of format changes and felt a bit mixed on
whether to include it or not. It does match the coding style guidelines,
but there is no actual functional change made to those lines in this series.
I think its a good improvement, and it does force some extra context
into the diff which makes reading the resulting change easier.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo
2023-04-19 12:31 ` [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
2023-04-26 19:20 ` Jacob Keller
@ 2023-04-26 19:21 ` Jacob Keller
2023-04-27 10:58 ` Patrick Steinhardt
2023-04-26 19:25 ` Glen Choo
2 siblings, 1 reply; 120+ messages in thread
From: Jacob Keller @ 2023-04-26 19:21 UTC (permalink / raw)
To: Patrick Steinhardt, git; +Cc: Jonathan Tan
On 4/19/2023 5:31 AM, Patrick Steinhardt wrote:
>
> +test_expect_success 'fetch output with HEAD and --dry-run' '
> + test_when_finished "rm -rf head" &&
> + git clone . head &&
> +
> + git -C head fetch --dry-run origin HEAD >actual 2>&1 &&
> + cat >expect <<-EOF &&
> + From $(test-tool path-utils real_path .)/.
> + * branch HEAD -> FETCH_HEAD
> + EOF
> + test_cmp expect actual &&
> +
> + git -C head fetch --dry-run origin HEAD:foo >actual 2>&1 &&
> + cat >expect <<-EOF &&
> + From $(test-tool path-utils real_path .)/.
> + * [new ref] HEAD -> foo
> + EOF
> + test_cmp expect actual
> +'
> +
The test mentions HEAD and --dry-run, but the bug seems to exist
regardless of whether --dry-run is used. I understand the use of
--dry-run for testing fetch output so that you can repeatably run git
fetch and get the same results.
The tests here should probably also have a test that covers fetch
without --dry-run though.
Thanks,
Jake
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo
2023-04-19 12:31 ` [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
2023-04-26 19:20 ` Jacob Keller
2023-04-26 19:21 ` Jacob Keller
@ 2023-04-26 19:25 ` Glen Choo
2023-04-27 10:58 ` Patrick Steinhardt
2 siblings, 1 reply; 120+ messages in thread
From: Glen Choo @ 2023-04-26 19:25 UTC (permalink / raw)
To: Patrick Steinhardt, git; +Cc: Jonathan Tan
Rearranging the lines slightly,
Patrick Steinhardt <ps@pks.im> writes:
> When displaying reference updates, we print a line that looks similar to
> the following:
>
> ```
> * branch master -> master
> ```
>
> The "branch" bit changes depending on what kind of reference we're
> updating, while both of the right-hand references are computed by
> stripping well-known prefixes like "refs/heads/" or "refs/tags".
>
> [...]
> we also use this value to display reference updates.
> And while the call to `display_ref_update()` correctly figures out that
> we meant "HEAD" when `what` is empty, the call to `update_local_ref()`
> doesn't. `update_local_ref()` will then call `display_ref_update()` with
> the empty string and cause the following broken output:
>
> ```
> $ git fetch --dry-run origin HEAD:foo
> From https://github.com/git/git
> * [new ref] -> foo
> ```
>
> [...]
>
> Fix this bug by instead unconditionally passing the full reference name
> to `display_ref_update()` which learns to call `prettify_refname()` on
> it. This does fix the above bug and is otherwise functionally the same
> as `prettify_refname()` would only ever strip the well-known prefixes
> just as intended. So at the same time, this also simplifies the code a
> bit.
The bug fix is obviously good. I'm surprised we hadn't caught this
sooner.
As a nitpicky comment, the commit message goes into a lot of detail,
which makes it tricky to read on its own (though the level of detail
makes it easy to match to the diff, making the diff quite easy to
follow). I would have found this easier to read by summarizing the
high-level mental model before diving into the background, e.g.
store_updated_refs() parses the remote ref name to create a 'note' to
write to FETCH_HEAD. This note is usually the prettified ref name, so
it is used to diplay ref updates (display_ref_update()). But if the
remote ref is HEAD, the note is the empty string [insert bug
description]. Instead, use the note only as a note and have
display_ref_update() prettify the ref name itself...
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> index c310d89878..7c64f0c562 100644
> --- a/builtin/fetch.c
> +++ b/builtin/fetch.c
> @@ -918,12 +918,14 @@ static void display_ref_update(struct display_state *display_state, char code,
> }
>
> width = (summary_width + strlen(summary) - gettext_width(summary));
> + remote = prettify_refname(remote);
> + local = prettify_refname(local);
>
> strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
> if (!display_state->compact_format)
> - print_remote_to_local(display_state, remote, prettify_refname(local));
> + print_remote_to_local(display_state, remote, local);
> else
> - print_compact(display_state, remote, prettify_refname(local));
> + print_compact(display_state, remote, local);
> if (error)
> strbuf_addf(&display_state->buf, " (%s)", error);
> strbuf_addch(&display_state->buf, '\n');
As expected, we now prettify the name isntead of trusting the 'note'
that came in the parameter...
> @@ -934,7 +936,7 @@ static void display_ref_update(struct display_state *display_state, char code,
> static int update_local_ref(struct ref *ref,
> struct ref_transaction *transaction,
> struct display_state *display_state,
> - const char *remote, const struct ref *remote_ref,
> + const struct ref *remote_ref,
> int summary_width)
> {
> struct commit *current = NULL, *updated;
> @@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,
> if (oideq(&ref->old_oid, &ref->new_oid)) {
> if (verbosity > 0)
> display_ref_update(display_state, '=', _("[up to date]"), NULL,
> - remote, ref->name, summary_width);
> + remote_ref->name, ref->name, summary_width);
> return 0;
> }
>
> @@ -959,7 +961,7 @@ static int update_local_ref(struct ref *ref,
> */
> display_ref_update(display_state, '!', _("[rejected]"),
> _("can't fetch into checked-out branch"),
> - remote, ref->name, summary_width);
> + remote_ref->name, ref->name, summary_width);
> return 1;
> }
>
> @@ -970,12 +972,12 @@ static int update_local_ref(struct ref *ref,
> r = s_update_ref("updating tag", ref, transaction, 0);
> display_ref_update(display_state, r ? '!' : 't', _("[tag update]"),
> r ? _("unable to update local ref") : NULL,
> - remote, ref->name, summary_width);
> + remote_ref->name, ref->name, summary_width);
> return r;
> } else {
> display_ref_update(display_state, '!', _("[rejected]"),
> _("would clobber existing tag"),
> - remote, ref->name, summary_width);
> + remote_ref->name, ref->name, summary_width);
> return 1;
> }
> }
> @@ -1008,7 +1010,7 @@ static int update_local_ref(struct ref *ref,
> r = s_update_ref(msg, ref, transaction, 0);
> display_ref_update(display_state, r ? '!' : '*', what,
> r ? _("unable to update local ref") : NULL,
> - remote, ref->name, summary_width);
> + remote_ref->name, ref->name, summary_width);
> return r;
> }
>
> @@ -1030,7 +1032,7 @@ static int update_local_ref(struct ref *ref,
> r = s_update_ref("fast-forward", ref, transaction, 1);
> display_ref_update(display_state, r ? '!' : ' ', quickref.buf,
> r ? _("unable to update local ref") : NULL,
> - remote, ref->name, summary_width);
> + remote_ref->name, ref->name, summary_width);
> strbuf_release(&quickref);
> return r;
> } else if (force || ref->force) {
> @@ -1042,12 +1044,12 @@ static int update_local_ref(struct ref *ref,
> r = s_update_ref("forced-update", ref, transaction, 1);
> display_ref_update(display_state, r ? '!' : '+', quickref.buf,
> r ? _("unable to update local ref") : _("forced update"),
> - remote, ref->name, summary_width);
> + remote_ref->name, ref->name, summary_width);
> strbuf_release(&quickref);
> return r;
> } else {
> display_ref_update(display_state, '!', _("[rejected]"), _("non-fast-forward"),
> - remote, ref->name, summary_width);
> + remote_ref->name, ref->name, summary_width);
> return 1;
> }
> }
...
> @@ -1277,7 +1278,7 @@ static int store_updated_refs(struct display_state *display_state,
> display_state->url_len);
>
> if (ref) {
> - rc |= update_local_ref(ref, transaction, display_state, what,
> + rc |= update_local_ref(ref, transaction, display_state,
> rm, summary_width);
> free(ref);
> } else if (write_fetch_head || dry_run) {
> @@ -1288,7 +1289,7 @@ static int store_updated_refs(struct display_state *display_state,
> */
> display_ref_update(display_state, '*',
> *kind ? kind : "branch", NULL,
> - *what ? what : "HEAD",
> + rm->name,
> "FETCH_HEAD", summary_width);
> }
> }
and we stop passing the 'note' as a parameter. Looks good.
> @@ -1252,14 +1254,13 @@ static int store_updated_refs(struct display_state *display_state,
> if (!strcmp(rm->name, "HEAD")) {
> kind = "";
> what = "";
> - }
> - else if (skip_prefix(rm->name, "refs/heads/", &what))
> + } else if (skip_prefix(rm->name, "refs/heads/", &what)) {
> kind = "branch";
> - else if (skip_prefix(rm->name, "refs/tags/", &what))
> + } else if (skip_prefix(rm->name, "refs/tags/", &what)) {
> kind = "tag";
> - else if (skip_prefix(rm->name, "refs/remotes/", &what))
> + } else if (skip_prefix(rm->name, "refs/remotes/", &what)) {
> kind = "remote-tracking branch";
> - else {
> + } else {
> kind = "";
> what = rm->name;
> }
I really appreciate that this makes the patch easier to read. I don't
really appreciate this sort of churn, but it _is_ following
CodingGuidelines:
- When there are multiple arms to a conditional and some of them
require braces, enclose even a single line block in braces for
consistency. E.g.:
if (foo) {
doit();
} else {
one();
two();
three();
}
(I initialy thought it wasn't. Thanks to other Review Club participants
for pointing this out).
> diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
> index 0e45c27007..55f0f05b6a 100755
> --- a/t/t5574-fetch-output.sh
> +++ b/t/t5574-fetch-output.sh
> @@ -54,6 +54,25 @@ test_expect_success 'fetch compact output' '
> test_cmp expect actual
> '
>
> +test_expect_success 'fetch output with HEAD and --dry-run' '
The commit message and diff didn't imply that this is a --dry-run only
bug. I tested locally, and it seems to reproduce without --dry-run too,
so I think we should drop "--dry-run" from this test name. In a later
patch, you also add a test for porcelain output with --dry-run, but
since this test seems designed for just this bug, I think we can drop
the later test.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 7/8] fetch: introduce new `--output-format` option
2023-04-19 12:31 ` [PATCH 7/8] fetch: introduce new `--output-format` option Patrick Steinhardt
@ 2023-04-26 19:40 ` Glen Choo
2023-04-27 10:58 ` Patrick Steinhardt
0 siblings, 1 reply; 120+ messages in thread
From: Glen Choo @ 2023-04-26 19:40 UTC (permalink / raw)
To: Patrick Steinhardt, git; +Cc: Jonathan Tan
Patrick Steinhardt <ps@pks.im> writes:
> @@ -2101,6 +2116,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
> PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
> OPT_BOOL(0, "dry-run", &dry_run,
> N_("dry run")),
> + OPT_CALLBACK(0, "output-format", &display_format, N_("format"), N_("output format"),
> + opt_parse_output_format),
> OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
> N_("write fetched references to the FETCH_HEAD file")),
> OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
This change is good enough for fetching from a single remote, but if we
want to support "--all", we'd also need to propagate the CLI flag to the
child "fetch" processes. (The config option wouldn't have this bug
because the child processes would parse config and get the correct
value.)
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-04-19 12:31 ` [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
@ 2023-04-26 19:52 ` Glen Choo
2023-04-27 10:58 ` Patrick Steinhardt
0 siblings, 1 reply; 120+ messages in thread
From: Glen Choo @ 2023-04-26 19:52 UTC (permalink / raw)
To: Patrick Steinhardt, git; +Cc: Jonathan Tan
Patrick Steinhardt <ps@pks.im> writes:
> The output format is quite simple:
>
> ```
> <flag> <old-object-id> <new-object-id> <local-reference>\n
> ```
This format doesn't show the remote name or url that was fetched. That
seems okay when fetching with a single remote, but it seems necessary
with "--all". Perhaps you were planning to add that in a later series?
If so, I think it's okay to call the "porcelain" format experimental,
and forbid porcelain + --all until then.
>
> We assume two conditions which are generally true:
>
> - The old and new object IDs have fixed known widths and cannot
> contain spaces.
>
> - References cannot contain newlines.
This seems like a non-issue if we add a -z CLI option to indicate that
entries should be NUL terminated instead of newline terminated, but that
can be done as a followup.
> With these assumptions, the output format becomes unambiguously
> parseable. Furthermore, given that this output is designed to be
> consumed by scripts, the machine-readable data is printed to stdout
> instead of stderr like the human-readable output is. This is mostly done
> so that other data printed to stderr, like error messages or progress
> meters, don't interfere with the parseable data.
Sending the 'main output' to stdout makes sense to me, but this (and
possibly respecting -z) sounds like a different mode of operation, not
just a matter of formats. It seems different enough that I'd prefer not
to piggyback on "fetch.output" for this (even though this adds more
surface to the interface...).
We could add --porcelain and say that "fetch.output" is ignored if
--porcelain is also given. That also eliminates the need for
--output-format, I think.
The .c changes look good to me.
> +test_expect_success 'fetch porcelain output with HEAD and --dry-run' '
> + test_when_finished "rm -rf head" &&
> + git clone . head &&
> + COMMIT_ID=$(git rev-parse HEAD) &&
> +
> + git -C head fetch --output-format=porcelain --dry-run origin HEAD >actual &&
> + cat >expect <<-EOF &&
> + * $ZERO_OID $COMMIT_ID FETCH_HEAD
> + EOF
> + test_cmp expect actual &&
> +
> + git -C head fetch --output-format=porcelain --dry-run origin HEAD:foo >actual &&
> + cat >expect <<-EOF &&
> + * $ZERO_OID $COMMIT_ID refs/heads/foo
> + EOF
> + test_cmp expect actual
> +'
As mentioned upthread, I think this test isn't needed because
"porcelain" wouldn't run into the bug we are checking for anyway.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 0/8] fetch: introduce machine-parseable output
2023-04-26 19:14 ` Jacob Keller
@ 2023-04-26 20:23 ` Junio C Hamano
2023-04-26 20:30 ` Jacob Keller
` (2 more replies)
2023-04-26 20:24 ` Junio C Hamano
1 sibling, 3 replies; 120+ messages in thread
From: Junio C Hamano @ 2023-04-26 20:23 UTC (permalink / raw)
To: Jacob Keller; +Cc: Patrick Steinhardt, git
Jacob Keller <jacob.e.keller@intel.com> writes:
>> Yeah, I'd be perfectly happy to rename this to `--format=porcelain`.
>> I'll wait for the Review Club that discusses this patch set tomorrow and
>> will send a new version with that change afterwards if nobody disagrees.
>>
>> Patrick
>
> We had some discussion during review club about this, where the idea of
> using "--porcelain" came up because many commands use that when
> switching into a machine readable format.
>
> In addition, this format not only changes the output but also moves it
> from being on stderr to stdout, which is a hint that the intended usage
> of the command is now a little different.
A little different from what? I do not think the answer would be
"other program's --porcelain mode", as sending them to stdout would
be one of the things that make the output easier for programs to
parse, so it does sound like very much in the same spirit as "git
status --porcelain" where its output format gets tweaked to be more
machine friendly.
The output with "--porcelain" option enabled tend to be less human
friendly and the distinction between Porcelain (for humans) and
plumbing (for scripts) is reversed in the use of the word there---it
started as "this is the option for those who write Porcelain
commands to use", but still it is not a very good name for the
option.
I am perfectly OK if the plan is to uniformly use --output-format
(or something equally more descriptive) and migrate and deprecate
the "--porcelain" option away from existing commands.
Thanks.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 0/8] fetch: introduce machine-parseable output
2023-04-26 19:14 ` Jacob Keller
2023-04-26 20:23 ` Junio C Hamano
@ 2023-04-26 20:24 ` Junio C Hamano
1 sibling, 0 replies; 120+ messages in thread
From: Junio C Hamano @ 2023-04-26 20:24 UTC (permalink / raw)
To: Jacob Keller; +Cc: Patrick Steinhardt, git
Jacob Keller <jacob.e.keller@intel.com> writes:
>> Yeah, I'd be perfectly happy to rename this to `--format=porcelain`.
>> I'll wait for the Review Club that discusses this patch set tomorrow and
>> will send a new version with that change afterwards if nobody disagrees.
>>
>> Patrick
>
> We had some discussion during review club about this, where the idea of
> using "--porcelain" came up because many commands use that when
> switching into a machine readable format.
>
> In addition, this format not only changes the output but also moves it
> from being on stderr to stdout, which is a hint that the intended usage
> of the command is now a little different.
A little different from what? I do not think the answer would be
"other program's --porcelain mode", as sending them to stdout would
be one of the things that make the output easier for programs to
parse, so it does sound like very much in the same spirit as "git
status --porcelain" where its output format gets tweaked to be more
machine friendly.
The output with "--porcelain" option enabled tend to be less human
friendly and the distinction between Porcelain (for humans) and
plumbing (for scripts) is reversed in the use of the word there---it
started as "this is the option for those who write Porcelain
commands to use", but still it is not a very good name for the
option.
I am perfectly OK if the plan is to uniformly use --output-format
(or something equally more descriptive) and migrate and deprecate
the "--porcelain" option away from existing commands.
Thanks.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 0/8] fetch: introduce machine-parseable output
2023-04-26 20:23 ` Junio C Hamano
@ 2023-04-26 20:30 ` Jacob Keller
2023-04-27 10:58 ` Patrick Steinhardt
2023-04-27 22:49 ` Glen Choo
2 siblings, 0 replies; 120+ messages in thread
From: Jacob Keller @ 2023-04-26 20:30 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Patrick Steinhardt, git
On 4/26/2023 1:23 PM, Junio C Hamano wrote:
> Jacob Keller <jacob.e.keller@intel.com> writes:
>
>>> Yeah, I'd be perfectly happy to rename this to `--format=porcelain`.
>>> I'll wait for the Review Club that discusses this patch set tomorrow and
>>> will send a new version with that change afterwards if nobody disagrees.
>>>
>>> Patrick
>>
>> We had some discussion during review club about this, where the idea of
>> using "--porcelain" came up because many commands use that when
>> switching into a machine readable format.
>>
>> In addition, this format not only changes the output but also moves it
>> from being on stderr to stdout, which is a hint that the intended usage
>> of the command is now a little different.
>
> A little different from what? I do not think the answer would be
> "other program's --porcelain mode", as sending them to stdout would
> be one of the things that make the output easier for programs to
> parse, so it does sound like very much in the same spirit as "git
> status --porcelain" where its output format gets tweaked to be more
> machine friendly.
A little different from using git fetch normally where all output is
stderr and is generally "this is what I did" but which was obviously not
intended to be parsed by scripts but instead by the human who ran the
command.
>
> The output with "--porcelain" option enabled tend to be less human
> friendly and the distinction between Porcelain (for humans) and
> plumbing (for scripts) is reversed in the use of the word there---it
> started as "this is the option for those who write Porcelain
> commands to use", but still it is not a very good name for the
> option.
Yea the option sort of means "to be used by those implementing
porcelain" and its definitely a bit confusing.
>
> I am perfectly OK if the plan is to uniformly use --output-format
> (or something equally more descriptive) and migrate and deprecate
> the "--porcelain" option away from existing commands.
>
> Thanks.
That makes sense to me as well.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 0/8] fetch: introduce machine-parseable output
2023-04-26 18:54 ` Glen Choo
@ 2023-04-26 21:14 ` Glen Choo
0 siblings, 0 replies; 120+ messages in thread
From: Glen Choo @ 2023-04-26 21:14 UTC (permalink / raw)
To: Patrick Steinhardt, git; +Cc: Jonathan Tan, Jacob Keller
Glen Choo <chooglen@google.com> writes:
> Hi Patrick!
>
> Thanks for the pleasant read! I thought this was a great topic for
> Review Club. It's too bad that we missed you, but we post all relevant
> feedback here anyway.
>
> Nevertheless, if you'd like to see the meeting notes, you can find them
> at:
>
> https://docs.google.com/document/d/14L8BAumGTpsXpjDY8VzZ4rRtpAjuGrFSRqn3stCuS_w/edit
It seems like Jacob Keller and I have mostly the same comments (since we
did both go to the same Review Club, after all :)), so feel free to
respond to only Jacob's messages and ignore mine (where the messages
overlap) - I'll catch up on those conversations.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-04-26 19:52 ` Glen Choo
@ 2023-04-27 10:58 ` Patrick Steinhardt
2023-04-27 23:20 ` Glen Choo
2023-05-02 20:55 ` Felipe Contreras
0 siblings, 2 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-27 10:58 UTC (permalink / raw)
To: Glen Choo; +Cc: git, Jonathan Tan
[-- Attachment #1: Type: text/plain, Size: 4957 bytes --]
On Wed, Apr 26, 2023 at 12:52:46PM -0700, Glen Choo wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
> > The output format is quite simple:
> >
> > ```
> > <flag> <old-object-id> <new-object-id> <local-reference>\n
> > ```
>
> This format doesn't show the remote name or url that was fetched. That
> seems okay when fetching with a single remote, but it seems necessary
> with "--all". Perhaps you were planning to add that in a later series?
> If so, I think it's okay to call the "porcelain" format experimental,
> and forbid porcelain + --all until then.
The reason is mostly that I didn't find an output format that I really
liked here. We'd basically have to repeat the remote URL for every
single reference: just repeating it once per remote doesn't fly because
with `--parallel` the output could be intermingled. But doing that feels
wasteful to me, so I bailed. I guess I'm also biased here because it
just wouldn't be useful to myself.
So with that in mind, I'd like to continue ignoring this issue for now
and just not report the remote that the ref came from. But I'd also
argue that we don't have to restrict porcelain mode to single-remote
fetches: it can still be useful to do multi-remote fetches even without
the information where a certain reference update comes from. So any kind
of restriction would feel artificial to me here.
Furthermore, I'd argue that it is not necessary to label the format as
experimental only because of this limitation. With the refactorings done
in this and the preceding patch series it is easy to add a new format in
case there indeed is somebody that would have a usecase for this. The
"porcelain" format should stay stable, and if we decide that we want to
also report the remote for each reference in a follow-up we can easily
add a "porcelain-v2" or "porcelain-with-remote" format.
As I said though: I'm clearly biased, so if you feel like my train of
though is simply me being lazy then I'd carve in and adapt.
> > We assume two conditions which are generally true:
> >
> > - The old and new object IDs have fixed known widths and cannot
> > contain spaces.
> >
> > - References cannot contain newlines.
>
> This seems like a non-issue if we add a -z CLI option to indicate that
> entries should be NUL terminated instead of newline terminated, but that
> can be done as a followup.
Yeah, either via `-z` or a new porcelain output format. But both of
these conditions should generally be true anyway, so I don't see that
those should become a problem.
> > With these assumptions, the output format becomes unambiguously
> > parseable. Furthermore, given that this output is designed to be
> > consumed by scripts, the machine-readable data is printed to stdout
> > instead of stderr like the human-readable output is. This is mostly done
> > so that other data printed to stderr, like error messages or progress
> > meters, don't interfere with the parseable data.
>
> Sending the 'main output' to stdout makes sense to me, but this (and
> possibly respecting -z) sounds like a different mode of operation, not
> just a matter of formats. It seems different enough that I'd prefer not
> to piggyback on "fetch.output" for this (even though this adds more
> surface to the interface...).
>
> We could add --porcelain and say that "fetch.output" is ignored if
> --porcelain is also given. That also eliminates the need for
> --output-format, I think.
I was thinking about this initially, as well. But ultimately I decided
against this especially because of your second paragraph: we'd now need
to think about precedence of options and mutual exclusion, and that to
me feels like an interface that is less obvious than a single knob that
works as you'd expect.
> The .c changes look good to me.
>
> > +test_expect_success 'fetch porcelain output with HEAD and --dry-run' '
> > + test_when_finished "rm -rf head" &&
> > + git clone . head &&
> > + COMMIT_ID=$(git rev-parse HEAD) &&
> > +
> > + git -C head fetch --output-format=porcelain --dry-run origin HEAD >actual &&
> > + cat >expect <<-EOF &&
> > + * $ZERO_OID $COMMIT_ID FETCH_HEAD
> > + EOF
> > + test_cmp expect actual &&
> > +
> > + git -C head fetch --output-format=porcelain --dry-run origin HEAD:foo >actual &&
> > + cat >expect <<-EOF &&
> > + * $ZERO_OID $COMMIT_ID refs/heads/foo
> > + EOF
> > + test_cmp expect actual
> > +'
>
> As mentioned upthread, I think this test isn't needed because
> "porcelain" wouldn't run into the bug we are checking for anyway.
The only reason that the other bug was able to survive for so long was
that we didn't have test coverage there. So I think it makes sense to
explicitly test this, too, also because it causes us to walk a different
code path.
Last but not least: this test uncovered a segfault I had in a previous
version. So I'd rather keep it :)
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 7/8] fetch: introduce new `--output-format` option
2023-04-26 19:40 ` Glen Choo
@ 2023-04-27 10:58 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-27 10:58 UTC (permalink / raw)
To: Glen Choo; +Cc: git, Jonathan Tan
[-- Attachment #1: Type: text/plain, Size: 1019 bytes --]
On Wed, Apr 26, 2023 at 12:40:13PM -0700, Glen Choo wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
> > @@ -2101,6 +2116,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
> > PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
> > OPT_BOOL(0, "dry-run", &dry_run,
> > N_("dry run")),
> > + OPT_CALLBACK(0, "output-format", &display_format, N_("format"), N_("output format"),
> > + opt_parse_output_format),
> > OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
> > N_("write fetched references to the FETCH_HEAD file")),
> > OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
>
> This change is good enough for fetching from a single remote, but if we
> want to support "--all", we'd also need to propagate the CLI flag to the
> child "fetch" processes. (The config option wouldn't have this bug
> because the child processes would parse config and get the correct
> value.)
Oh, right, good catch. Will fix, thanks!
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo
2023-04-26 19:20 ` Jacob Keller
@ 2023-04-27 10:58 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-27 10:58 UTC (permalink / raw)
To: Jacob Keller; +Cc: git, Jonathan Tan
[-- Attachment #1: Type: text/plain, Size: 1032 bytes --]
On Wed, Apr 26, 2023 at 12:20:15PM -0700, Jacob Keller wrote:
> On 4/19/2023 5:31 AM, Patrick Steinhardt wrote:
[snip]
> The commit message here has a lot of context, but I found it a bit hard
> to parse through, especially relative to the actual fix in code.
>
> One suggestion was to load the paragraphs a bit more with the actual
> problem being solved first, before beginning a lot of the context.
>
> We also discussed the block of format changes and felt a bit mixed on
> whether to include it or not. It does match the coding style guidelines,
> but there is no actual functional change made to those lines in this series.
>
> I think its a good improvement, and it does force some extra context
> into the diff which makes reading the resulting change easier.
I initially really struggled to explain to myself why my change actually
works, and that shows in the commit message. I've reworded it a bit and
took some of the suggested changes that Glen posted to hopefully make
this clearer.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo
2023-04-26 19:21 ` Jacob Keller
@ 2023-04-27 10:58 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-27 10:58 UTC (permalink / raw)
To: Jacob Keller; +Cc: git, Jonathan Tan
[-- Attachment #1: Type: text/plain, Size: 1156 bytes --]
On Wed, Apr 26, 2023 at 12:21:40PM -0700, Jacob Keller wrote:
>
>
> On 4/19/2023 5:31 AM, Patrick Steinhardt wrote:
> >
> > +test_expect_success 'fetch output with HEAD and --dry-run' '
> > + test_when_finished "rm -rf head" &&
> > + git clone . head &&
> > +
> > + git -C head fetch --dry-run origin HEAD >actual 2>&1 &&
> > + cat >expect <<-EOF &&
> > + From $(test-tool path-utils real_path .)/.
> > + * branch HEAD -> FETCH_HEAD
> > + EOF
> > + test_cmp expect actual &&
> > +
> > + git -C head fetch --dry-run origin HEAD:foo >actual 2>&1 &&
> > + cat >expect <<-EOF &&
> > + From $(test-tool path-utils real_path .)/.
> > + * [new ref] HEAD -> foo
> > + EOF
> > + test_cmp expect actual
> > +'
> > +
>
> The test mentions HEAD and --dry-run, but the bug seems to exist
> regardless of whether --dry-run is used. I understand the use of
> --dry-run for testing fetch output so that you can repeatably run git
> fetch and get the same results.
>
> The tests here should probably also have a test that covers fetch
> without --dry-run though.
Makes sense, will amend!
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo
2023-04-26 19:25 ` Glen Choo
@ 2023-04-27 10:58 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-27 10:58 UTC (permalink / raw)
To: Glen Choo; +Cc: git, Jonathan Tan
[-- Attachment #1: Type: text/plain, Size: 3197 bytes --]
On Wed, Apr 26, 2023 at 12:25:53PM -0700, Glen Choo wrote:
> Rearranging the lines slightly,
>
> Patrick Steinhardt <ps@pks.im> writes:
>
> > When displaying reference updates, we print a line that looks similar to
> > the following:
> >
> > ```
> > * branch master -> master
> > ```
> >
> > The "branch" bit changes depending on what kind of reference we're
> > updating, while both of the right-hand references are computed by
> > stripping well-known prefixes like "refs/heads/" or "refs/tags".
> >
> > [...]
> > we also use this value to display reference updates.
> > And while the call to `display_ref_update()` correctly figures out that
> > we meant "HEAD" when `what` is empty, the call to `update_local_ref()`
> > doesn't. `update_local_ref()` will then call `display_ref_update()` with
> > the empty string and cause the following broken output:
> >
> > ```
> > $ git fetch --dry-run origin HEAD:foo
> > From https://github.com/git/git
> > * [new ref] -> foo
> > ```
> >
> > [...]
> >
> > Fix this bug by instead unconditionally passing the full reference name
> > to `display_ref_update()` which learns to call `prettify_refname()` on
> > it. This does fix the above bug and is otherwise functionally the same
> > as `prettify_refname()` would only ever strip the well-known prefixes
> > just as intended. So at the same time, this also simplifies the code a
> > bit.
>
>
> The bug fix is obviously good. I'm surprised we hadn't caught this
> sooner.
>
> As a nitpicky comment, the commit message goes into a lot of detail,
> which makes it tricky to read on its own (though the level of detail
> makes it easy to match to the diff, making the diff quite easy to
> follow). I would have found this easier to read by summarizing the
> high-level mental model before diving into the background, e.g.
>
>
> store_updated_refs() parses the remote ref name to create a 'note' to
> write to FETCH_HEAD. This note is usually the prettified ref name, so
> it is used to diplay ref updates (display_ref_update()). But if the
> remote ref is HEAD, the note is the empty string [insert bug
> description]. Instead, use the note only as a note and have
> display_ref_update() prettify the ref name itself...
I like that and will use a variant of this.
[snip]
> > diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
> > index 0e45c27007..55f0f05b6a 100755
> > --- a/t/t5574-fetch-output.sh
> > +++ b/t/t5574-fetch-output.sh
> > @@ -54,6 +54,25 @@ test_expect_success 'fetch compact output' '
> > test_cmp expect actual
> > '
> >
> > +test_expect_success 'fetch output with HEAD and --dry-run' '
>
> The commit message and diff didn't imply that this is a --dry-run only
> bug. I tested locally, and it seems to reproduce without --dry-run too,
> so I think we should drop "--dry-run" from this test name. In a later
> patch, you also add a test for porcelain output with --dry-run, but
> since this test seems designed for just this bug, I think we can drop
> the later test.
True, I'll amend the test.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 0/8] fetch: introduce machine-parseable output
2023-04-26 20:23 ` Junio C Hamano
2023-04-26 20:30 ` Jacob Keller
@ 2023-04-27 10:58 ` Patrick Steinhardt
2023-04-27 19:46 ` Jacob Keller
2023-04-27 22:49 ` Glen Choo
2 siblings, 1 reply; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-27 10:58 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Jacob Keller, git
[-- Attachment #1: Type: text/plain, Size: 3025 bytes --]
On Wed, Apr 26, 2023 at 01:23:12PM -0700, Junio C Hamano wrote:
> Jacob Keller <jacob.e.keller@intel.com> writes:
>
> >> Yeah, I'd be perfectly happy to rename this to `--format=porcelain`.
> >> I'll wait for the Review Club that discusses this patch set tomorrow and
> >> will send a new version with that change afterwards if nobody disagrees.
> >>
> >> Patrick
> >
> > We had some discussion during review club about this, where the idea of
> > using "--porcelain" came up because many commands use that when
> > switching into a machine readable format.
> >
> > In addition, this format not only changes the output but also moves it
> > from being on stderr to stdout, which is a hint that the intended usage
> > of the command is now a little different.
>
> A little different from what? I do not think the answer would be
> "other program's --porcelain mode", as sending them to stdout would
> be one of the things that make the output easier for programs to
> parse, so it does sound like very much in the same spirit as "git
> status --porcelain" where its output format gets tweaked to be more
> machine friendly.
>
> The output with "--porcelain" option enabled tend to be less human
> friendly and the distinction between Porcelain (for humans) and
> plumbing (for scripts) is reversed in the use of the word there---it
> started as "this is the option for those who write Porcelain
> commands to use", but still it is not a very good name for the
> option.
Ah, thanks for explaining where this reversed meaning actually comes
from. I really wondered why we use "porcelain" in preexisting code to
reflect machine-parseable interface, but that explanation does make
sense.
> I am perfectly OK if the plan is to uniformly use --output-format
> (or something equally more descriptive) and migrate and deprecate
> the "--porcelain" option away from existing commands.
I'm still not quite clear on where the consensus lies now. Personally, I
think that both `--format` and `--output-format` work well and are a bit
more descriptive than simply `--porcelain`, and wouldn't mind also
migrating other binaries to use either of them.
Furthermore, I think that `--[output-]format` has the advantage that you
don't need to handle priorities or mutual exclusion of different options
that all apply to the reference format. To a user, it is not immediately
obvious what `git fetch --format=compact --porcelain` would do, and
which of both options ultimately get respected. But that's likely only
true for future commands, because any migration would create the same
kind of ambiguity for preexisting commands.
If we were to also migrate preexisting code to use `--[output-]format`
then I'd argue that `--output-format` is likely the better name, mostly
because it is less likely to be ambiguous compared to `--format`. The
latter could e.g. easily confused with `--object-format`.
So I think I'll stick with `--output-format` for the time being.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH v2 0/8] fetch: introduce machine-parseable output
2023-04-19 12:31 [PATCH 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (10 preceding siblings ...)
2023-04-26 19:17 ` Jacob Keller
@ 2023-04-27 11:13 ` Patrick Steinhardt
2023-04-27 11:13 ` [PATCH v2 1/8] fetch: split out tests for output format Patrick Steinhardt
` (7 more replies)
2023-05-03 11:34 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (2 subsequent siblings)
14 siblings, 8 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-27 11:13 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 18496 bytes --]
Hi,
this is the second version of my patch series that introduces a
machine-parseable output.
Changes compared to v1:
- Patch 3/8: I've reworded the commit message to hopefully be more
straight-forward. I've also amended tests so that we don't only
test with `--dry-run`, but also without.
- Patch 6/8: I've also moved the `all` and `multiple` variables from
their global scope into `cmd__fetch`.
- Patch 7/8: Fixed a bug where `--output-format=` wasn't honored for
child processes when doing multi-remote fetches. Furthermore, I've
unified parsing of the actual format so that we don't have to
repeat it thrice.
- Patch 8/8: Added a note to the commit message that tries to argue
why I didn't add remote information to the interface. I'm still
open to change this if you disagree with my reasoning here.
Thanks a bunch for all the feedback received so far, really appreciate
it!
Patrick
Patrick Steinhardt (8):
fetch: split out tests for output format
fetch: add a test to exercise invalid output formats
fetch: fix missing from-reference when fetching HEAD:foo
fetch: introduce `display_format` enum
fetch: move display format parsing into main function
fetch: move option related variables into main function
fetch: introduce new `--output-format` option
fetch: introduce machine-parseable "porcelain" output format
Documentation/config/fetch.txt | 4 +-
Documentation/fetch-options.txt | 5 +
Documentation/git-fetch.txt | 17 +-
builtin/fetch.c | 427 ++++++++++++++++++++------------
t/t5510-fetch.sh | 53 ----
t/t5574-fetch-output.sh | 237 ++++++++++++++++++
6 files changed, 521 insertions(+), 222 deletions(-)
create mode 100755 t/t5574-fetch-output.sh
Range-diff against v1:
1: 0d0d50d14c = 1: 0d0d50d14c fetch: split out tests for output format
2: 29d2c58914 = 2: 29d2c58914 fetch: add a test to exercise invalid output formats
3: 596e12f03a ! 3: d1fb6eeae7 fetch: fix missing from-reference when fetching HEAD:foo
@@ Metadata
## Commit message ##
fetch: fix missing from-reference when fetching HEAD:foo
- When displaying reference updates, we print a line that looks similar to
- the following:
+ `store_updated_refs()` parses the remote reference for two purposes:
+
+ - It gets used as a note when writing FETCH_HEAD.
+
+ - It is passed through to `display_ref_update()` to display
+ updated references in the following format:
+
+ ```
+ * branch master -> master
+ ```
+
+ In most cases, the parsed remote reference is the prettified reference
+ name and can thus be used for both cases. But if the remote reference is
+ HEAD, the parsed remote reference becomes empty. This is intended when
+ we write the FETCH_HEAD, where we skip writing the note in that case.
+ But it is not intended when displaying the updated references and would
+ cause us to miss the left-hand side of the displayed reference update:
```
- * branch master -> master
- ```
-
- The "branch" bit changes depending on what kind of reference we're
- updating, while both of the right-hand references are computed by
- stripping well-known prefixes like "refs/heads/" or "refs/tags".
-
- The logic is kind of intertwined though and not easy to follow: we
- precompute both the kind (e.g. "branch") and the what, which is the
- abbreviated remote reference name, in `store_updated_refs()` and then
- pass it down the call chain to `display_ref_update()`.
-
- There is a set of different cases here:
-
- - When the remote reference name is "HEAD" we assume no kind and
- will thus instead print "[new ref]". We keep what at the empty
- string.
-
- - When the remote reference name has a well-known prefix then the
- kind would be "branch", "tag" or "remote-tracking branch". The
- what is the reference with the well-known prefix stripped and in
- fact matches the output that `prettify_refname()` would return.
-
- - Otherwise, we'll again assume no kind and keep the what set to the
- fully qualified reference name.
-
- Now there is a bug with the first case here, where the remote reference
- name is "HEAD". As noted, "what" will be set to the empty string. And
- that seems to be intentional because we also use this information to
- update the FETCH_HEAD, and in case we're updating HEAD we seemingly
- don't want to append that to our FETCH_HEAD value.
-
- But as mentioned, we also use this value to display reference updates.
- And while the call to `display_ref_update()` correctly figures out that
- we meant "HEAD" when `what` is empty, the call to `update_local_ref()`
- doesn't. `update_local_ref()` will then call `display_ref_update()` with
- the empty string and cause the following broken output:
-
- ```
- $ git fetch --dry-run origin HEAD:foo
+ $ git fetch origin HEAD:foo
From https://github.com/git/git
* [new ref] -> foo
```
The HEAD string is clearly missing from the left-hand side of the arrow,
- which is further stressed by the point that the following commands work
- as expected:
+ which is further stressed by the point that the following commands show
+ the left-hand side as expected:
```
- $ git fetch --dry-run origin HEAD
+ $ git fetch origin HEAD
From https://github.com/git/git
* branch HEAD -> FETCH_HEAD
- $ git fetch --dry-run origin master
+ $ git fetch origin master
From https://github.com/git/git
* branch master -> FETCH_HEAD
* branch master -> origin/master
```
- Fix this bug by instead unconditionally passing the full reference name
- to `display_ref_update()` which learns to call `prettify_refname()` on
- it. This does fix the above bug and is otherwise functionally the same
- as `prettify_refname()` would only ever strip the well-known prefixes
- just as intended. So at the same time, this also simplifies the code a
- bit.
+ The logic of how we compute the remote reference name that we ultimately
+ pass to `display_ref_update()` is not easy to follow. There are three
+ different cases here:
+
+ - When the remote reference name is "HEAD" we set the remote
+ reference name to the empty string. This is the case that causes
+ the bug to occur, where we would indeed want to print "HEAD"
+ instead of the empty string. This is what `prettify_refname()`
+ would return.
+
+ - When the remote reference name has a well-known prefix then we
+ strip this prefix. This matches what `prettify_refname()` does.
+
+ - Otherwise, we keep the fully qualified reference name. This also
+ matches what `prettify_refname()` does.
+
+ As the return value of `prettify_refname()` would do the correct thing
+ for us in all three cases, we can fix the bug by passing through the
+ full remote reference name to `display_ref_update()`, which learns to
+ call `prettify_refname()`. At the same time, this also simplifies the
+ code a bit.
Note that this patch also changes formatting of the block that computes
the "kind" and "what" variables. This is done on purpose so that it is
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
+ EOF
+ test_cmp expect actual &&
+
++ git -C head fetch origin HEAD >actual 2>&1 &&
++ test_cmp expect actual &&
++
+ git -C head fetch --dry-run origin HEAD:foo >actual 2>&1 &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * [new ref] HEAD -> foo
+ EOF
++ test_cmp expect actual &&
++
++ git -C head fetch origin HEAD:foo >actual 2>&1 &&
+ test_cmp expect actual
+'
+
4: 8571363be1 = 4: b545bf8bb9 fetch: introduce `display_format` enum
5: d98c3ee0ce = 5: 4990d35998 fetch: move display format parsing into main function
6: 640a8840e1 ! 6: cfe84129ab fetch: move option related variables into main function
@@ Commit message
Signed-off-by: Patrick Steinhardt <ps@pks.im>
## builtin/fetch.c ##
-@@ builtin/fetch.c: static int all, append, dry_run, force, keep, multiple, update_head_ok;
+@@ builtin/fetch.c: static int fetch_prune_tags_config = -1; /* unspecified */
+ static int prune_tags = -1; /* unspecified */
+ #define PRUNE_TAGS_BY_DEFAULT 0 /* do we prune tags by default? */
+
+-static int all, append, dry_run, force, keep, multiple, update_head_ok;
++static int append, dry_run, force, keep, update_head_ok;
static int write_fetch_head = 1;
static int verbosity, deepen_relative, set_upstream, refetch;
static int progress = -1;
@@ builtin/fetch.c: static int fetch_one(struct remote *remote, int argc, const cha
enum display_format display_format = DISPLAY_FORMAT_UNKNOWN;
struct string_list list = STRING_LIST_INIT_DUP;
struct remote *remote = NULL;
++ int all = 0, multiple = 0;
int result = 0;
int prune_tags_ok = 1;
+ int enable_auto_gc = 1;
7: 3b2cad066a ! 7: 0335e5eeb4 fetch: introduce new `--output-format` option
@@ Commit message
current mechanism feels a little bit indirect and rigid.
Introduce a new `--output-format` option that allows the user to change
- the desired output format more directly.
+ the desired format more directly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
@@ Documentation/fetch-options.txt: linkgit:git-config[1].
Write the list of remote refs fetched in the `FETCH_HEAD`
## builtin/fetch.c ##
+@@ builtin/fetch.c: enum display_format {
+ DISPLAY_FORMAT_UNKNOWN = 0,
+ DISPLAY_FORMAT_FULL,
+ DISPLAY_FORMAT_COMPACT,
++ DISPLAY_FORMAT_MAX,
++};
++
++static const char * const display_formats[DISPLAY_FORMAT_MAX] = {
++ NULL,
++ "full",
++ "compact",
+ };
+
+ struct display_state {
+@@ builtin/fetch.c: static int fetch_finished(int result, struct strbuf *out,
+ return 0;
+ }
+
+-static int fetch_multiple(struct string_list *list, int max_children)
++static int fetch_multiple(struct string_list *list, int max_children,
++ enum display_format format)
+ {
+ int i, result = 0;
+ struct strvec argv = STRVEC_INIT;
+@@ builtin/fetch.c: static int fetch_multiple(struct string_list *list, int max_children)
+ "--no-write-commit-graph", NULL);
+ add_options_to_argv(&argv);
+
++ if (format != DISPLAY_FORMAT_UNKNOWN)
++ strvec_pushf(&argv, "--output-format=%s", display_formats[format]);
++
+ if (max_children != 1 && list->nr != 1) {
+ struct parallel_fetch_state state = { argv.v, list, 0, 0 };
+ const struct run_process_parallel_opts opts = {
@@ builtin/fetch.c: static int fetch_one(struct remote *remote, int argc, const char **argv,
return exit_code;
}
++static enum display_format parse_display_format(const char *format)
++{
++ for (int i = 0; i < ARRAY_SIZE(display_formats); i++)
++ if (display_formats[i] && !strcmp(display_formats[i], format))
++ return i;
++ return DISPLAY_FORMAT_UNKNOWN;
++}
++
+static int opt_parse_output_format(const struct option *opt, const char *arg, int unset)
+{
-+ enum display_format *format = opt->value;
++ enum display_format *format = opt->value, parsed;
++
+ if (unset || !arg)
+ return 1;
-+ else if (!strcmp(arg, "full"))
-+ *format = DISPLAY_FORMAT_FULL;
-+ else if (!strcmp(arg, "compact"))
-+ *format = DISPLAY_FORMAT_COMPACT;
-+ else
++
++ parsed = parse_display_format(arg);
++ if (parsed == DISPLAY_FORMAT_UNKNOWN)
+ return error(_("unsupported output format '%s'"), arg);
++ *format = parsed;
+
+ return 0;
+}
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
N_("write fetched references to the FETCH_HEAD file")),
OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
+@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ const char *format = "full";
+
+ git_config_get_string_tmp("fetch.output", &format);
+- if (!strcasecmp(format, "full"))
+- display_format = DISPLAY_FORMAT_FULL;
+- else if (!strcasecmp(format, "compact"))
+- display_format = DISPLAY_FORMAT_COMPACT;
+- else
++
++ display_format = parse_display_format(format);
++ if (display_format == DISPLAY_FORMAT_UNKNOWN)
+ die(_("invalid value for '%s': '%s'"),
+ "fetch.output", format);
+ }
+@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ max_children = fetch_parallel_config;
+
+ /* TODO should this also die if we have a previous partial-clone? */
+- result = fetch_multiple(&list, max_children);
++ result = fetch_multiple(&list, max_children, display_format);
+ }
+
+
## t/t5574-fetch-output.sh ##
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch with invalid output format configuration' '
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch aligned output' '
cat >expect <<-\EOF &&
main -> origin/*
extraaa -> *
+@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
+ test_cmp expect actual
+ '
+
++test_expect_success 'fetch compact output with multiple remotes' '
++ test_when_finished "rm -rf compact-cfg compact-cli" &&
++
++ git clone . compact-cli &&
++ git -C compact-cli remote add second-remote "$PWD" &&
++ git clone . compact-cfg &&
++ git -C compact-cfg remote add second-remote "$PWD" &&
++ test_commit multi-commit &&
++
++ git -C compact-cfg -c fetch.output=compact fetch --all >actual-cfg 2>&1 &&
++ git -C compact-cli fetch --output-format=compact --all >actual-cli 2>&1 &&
++ test_cmp actual-cfg actual-cli &&
++
++ grep -e "->" actual-cfg | cut -c 22- >actual &&
++ cat >expect <<-\EOF &&
++ main -> origin/*
++ multi-commit -> *
++ main -> second-remote/*
++ EOF
++ test_cmp expect actual
++'
++
+ test_expect_success 'fetch output with HEAD and --dry-run' '
+ test_when_finished "rm -rf head" &&
+ git clone . head &&
8: 301138da03 ! 8: d7c1bc1a80 fetch: introduce machine-parseable "porcelain" output format
@@ Commit message
so that other data printed to stderr, like error messages or progress
meters, don't interfere with the parseable data.
+ A notable ommission here is that the output format does not include the
+ remote from which a reference was fetched, which might be important
+ information especially in the context of multi-remote fetches. But as
+ such a format would require us to print the remote for every single
+ reference update due to parallelizable fetches it feels wasteful for the
+ most likely usecase, which is when fetching from a single remote. If
+ usecases come up for this in the future though it is easy enough to add
+ a new "porcelain-v2" format that adds this information.
+
Signed-off-by: Patrick Steinhardt <ps@pks.im>
## Documentation/config/fetch.txt ##
@@ builtin/fetch.c: enum display_format {
DISPLAY_FORMAT_FULL,
DISPLAY_FORMAT_COMPACT,
+ DISPLAY_FORMAT_PORCELAIN,
+ DISPLAY_FORMAT_MAX,
+ };
+
+@@ builtin/fetch.c: static const char * const display_formats[DISPLAY_FORMAT_MAX] = {
+ NULL,
+ "full",
+ "compact",
++ "porcelain",
};
struct display_state {
@@ builtin/fetch.c: static int prune_refs(struct display_state *display_state,
summary_width);
warn_dangling_symref(stderr, dangling_msg, ref->name);
}
-@@ builtin/fetch.c: static int opt_parse_output_format(const struct option *opt, const char *arg, in
- *format = DISPLAY_FORMAT_FULL;
- else if (!strcmp(arg, "compact"))
- *format = DISPLAY_FORMAT_COMPACT;
-+ else if (!strcmp(arg, "porcelain"))
-+ *format = DISPLAY_FORMAT_PORCELAIN;
- else
- return error(_("unsupported output format '%s'"), arg);
-
-@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
- display_format = DISPLAY_FORMAT_FULL;
- else if (!strcasecmp(format, "compact"))
- display_format = DISPLAY_FORMAT_COMPACT;
-+ else if (!strcasecmp(format, "porcelain"))
-+ display_format = DISPLAY_FORMAT_PORCELAIN;
- else
- die(_("invalid value for '%s': '%s'"),
- "fetch.output", format);
## t/t5574-fetch-output.sh ##
-@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
+@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output with multiple remotes' '
test_cmp expect actual
'
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH v2 1/8] fetch: split out tests for output format
2023-04-27 11:13 ` [PATCH v2 " Patrick Steinhardt
@ 2023-04-27 11:13 ` Patrick Steinhardt
2023-04-29 17:34 ` SZEDER Gábor
2023-04-27 11:13 ` [PATCH v2 2/8] fetch: add a test to exercise invalid output formats Patrick Steinhardt
` (6 subsequent siblings)
7 siblings, 1 reply; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-27 11:13 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 4269 bytes --]
We're about to introduce a new porcelain mode for the output of
git-fetch(1). As part of that we'll be introducing a set of new tests
that only relate to the output of this command.
Split out tests that exercise the output format of git-fetch(1) so that
it becomes easier to verify this functionality as a standalone unit. As
the tests assume that the default branch is called "main" we set up the
corresponding GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME environment variable
accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
t/t5510-fetch.sh | 53 ----------------------------------
t/t5574-fetch-output.sh | 63 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 63 insertions(+), 53 deletions(-)
create mode 100755 t/t5574-fetch-output.sh
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index dc44da9c79..4f289063ce 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -1118,59 +1118,6 @@ test_expect_success 'fetching with auto-gc does not lock up' '
)
'
-test_expect_success 'fetch aligned output' '
- git clone . full-output &&
- test_commit looooooooooooong-tag &&
- (
- cd full-output &&
- git -c fetch.output=full fetch origin >actual 2>&1 &&
- grep -e "->" actual | cut -c 22- >../actual
- ) &&
- cat >expect <<-\EOF &&
- main -> origin/main
- looooooooooooong-tag -> looooooooooooong-tag
- EOF
- test_cmp expect actual
-'
-
-test_expect_success 'fetch compact output' '
- git clone . compact &&
- test_commit extraaa &&
- (
- cd compact &&
- git -c fetch.output=compact fetch origin >actual 2>&1 &&
- grep -e "->" actual | cut -c 22- >../actual
- ) &&
- cat >expect <<-\EOF &&
- main -> origin/*
- extraaa -> *
- EOF
- test_cmp expect actual
-'
-
-test_expect_success '--no-show-forced-updates' '
- mkdir forced-updates &&
- (
- cd forced-updates &&
- git init &&
- test_commit 1 &&
- test_commit 2
- ) &&
- git clone forced-updates forced-update-clone &&
- git clone forced-updates no-forced-update-clone &&
- git -C forced-updates reset --hard HEAD~1 &&
- (
- cd forced-update-clone &&
- git fetch --show-forced-updates origin 2>output &&
- test_i18ngrep "(forced update)" output
- ) &&
- (
- cd no-forced-update-clone &&
- git fetch --no-show-forced-updates origin 2>output &&
- test_i18ngrep ! "(forced update)" output
- )
-'
-
for section in fetch transfer
do
test_expect_success "$section.hideRefs affects connectivity check" '
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
new file mode 100755
index 0000000000..f91b654d38
--- /dev/null
+++ b/t/t5574-fetch-output.sh
@@ -0,0 +1,63 @@
+#!/bin/sh
+
+test_description='git fetch output format'
+
+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+
+. ./test-lib.sh
+
+test_expect_success 'fetch aligned output' '
+ git clone . full-output &&
+ test_commit looooooooooooong-tag &&
+ (
+ cd full-output &&
+ git -c fetch.output=full fetch origin >actual 2>&1 &&
+ grep -e "->" actual | cut -c 22- >../actual
+ ) &&
+ cat >expect <<-\EOF &&
+ main -> origin/main
+ looooooooooooong-tag -> looooooooooooong-tag
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success 'fetch compact output' '
+ git clone . compact &&
+ test_commit extraaa &&
+ (
+ cd compact &&
+ git -c fetch.output=compact fetch origin >actual 2>&1 &&
+ grep -e "->" actual | cut -c 22- >../actual
+ ) &&
+ cat >expect <<-\EOF &&
+ main -> origin/*
+ extraaa -> *
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success '--no-show-forced-updates' '
+ mkdir forced-updates &&
+ (
+ cd forced-updates &&
+ git init &&
+ test_commit 1 &&
+ test_commit 2
+ ) &&
+ git clone forced-updates forced-update-clone &&
+ git clone forced-updates no-forced-update-clone &&
+ git -C forced-updates reset --hard HEAD~1 &&
+ (
+ cd forced-update-clone &&
+ git fetch --show-forced-updates origin 2>output &&
+ test_i18ngrep "(forced update)" output
+ ) &&
+ (
+ cd no-forced-update-clone &&
+ git fetch --no-show-forced-updates origin 2>output &&
+ test_i18ngrep ! "(forced update)" output
+ )
+'
+
+test_done
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v2 2/8] fetch: add a test to exercise invalid output formats
2023-04-27 11:13 ` [PATCH v2 " Patrick Steinhardt
2023-04-27 11:13 ` [PATCH v2 1/8] fetch: split out tests for output format Patrick Steinhardt
@ 2023-04-27 11:13 ` Patrick Steinhardt
2023-04-27 11:13 ` [PATCH v2 3/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
` (5 subsequent siblings)
7 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-27 11:13 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 1203 bytes --]
Add a testcase that exercises the logic when an invalid output format is
passed via the `fetch.output` configuration.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
t/t5574-fetch-output.sh | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index f91b654d38..0e45c27007 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -7,6 +7,23 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
. ./test-lib.sh
+test_expect_success 'fetch with invalid output format configuration' '
+ test_when_finished "rm -rf clone" &&
+ git clone . clone &&
+
+ test_must_fail git -C clone -c fetch.output= fetch origin >actual 2>&1 &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}${SQ}
+ EOF
+ test_cmp expect actual &&
+
+ test_must_fail git -C clone -c fetch.output=garbage fetch origin >actual 2>&1 &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}garbage${SQ}
+ EOF
+ test_cmp expect actual
+'
+
test_expect_success 'fetch aligned output' '
git clone . full-output &&
test_commit looooooooooooong-tag &&
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v2 3/8] fetch: fix missing from-reference when fetching HEAD:foo
2023-04-27 11:13 ` [PATCH v2 " Patrick Steinhardt
2023-04-27 11:13 ` [PATCH v2 1/8] fetch: split out tests for output format Patrick Steinhardt
2023-04-27 11:13 ` [PATCH v2 2/8] fetch: add a test to exercise invalid output formats Patrick Steinhardt
@ 2023-04-27 11:13 ` Patrick Steinhardt
2023-04-27 17:26 ` Glen Choo
2023-04-27 19:49 ` Jacob Keller
2023-04-27 11:13 ` [PATCH v2 4/8] fetch: introduce `display_format` enum Patrick Steinhardt
` (4 subsequent siblings)
7 siblings, 2 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-27 11:13 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 9214 bytes --]
`store_updated_refs()` parses the remote reference for two purposes:
- It gets used as a note when writing FETCH_HEAD.
- It is passed through to `display_ref_update()` to display
updated references in the following format:
```
* branch master -> master
```
In most cases, the parsed remote reference is the prettified reference
name and can thus be used for both cases. But if the remote reference is
HEAD, the parsed remote reference becomes empty. This is intended when
we write the FETCH_HEAD, where we skip writing the note in that case.
But it is not intended when displaying the updated references and would
cause us to miss the left-hand side of the displayed reference update:
```
$ git fetch origin HEAD:foo
From https://github.com/git/git
* [new ref] -> foo
```
The HEAD string is clearly missing from the left-hand side of the arrow,
which is further stressed by the point that the following commands show
the left-hand side as expected:
```
$ git fetch origin HEAD
From https://github.com/git/git
* branch HEAD -> FETCH_HEAD
$ git fetch origin master
From https://github.com/git/git
* branch master -> FETCH_HEAD
* branch master -> origin/master
```
The logic of how we compute the remote reference name that we ultimately
pass to `display_ref_update()` is not easy to follow. There are three
different cases here:
- When the remote reference name is "HEAD" we set the remote
reference name to the empty string. This is the case that causes
the bug to occur, where we would indeed want to print "HEAD"
instead of the empty string. This is what `prettify_refname()`
would return.
- When the remote reference name has a well-known prefix then we
strip this prefix. This matches what `prettify_refname()` does.
- Otherwise, we keep the fully qualified reference name. This also
matches what `prettify_refname()` does.
As the return value of `prettify_refname()` would do the correct thing
for us in all three cases, we can fix the bug by passing through the
full remote reference name to `display_ref_update()`, which learns to
call `prettify_refname()`. At the same time, this also simplifies the
code a bit.
Note that this patch also changes formatting of the block that computes
the "kind" and "what" variables. This is done on purpose so that it is
part of the diff, hopefully making the change easier to comprehend.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 37 +++++++++++++++++++------------------
t/t5574-fetch-output.sh | 25 +++++++++++++++++++++++++
2 files changed, 44 insertions(+), 18 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index c310d89878..7c64f0c562 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -918,12 +918,14 @@ static void display_ref_update(struct display_state *display_state, char code,
}
width = (summary_width + strlen(summary) - gettext_width(summary));
+ remote = prettify_refname(remote);
+ local = prettify_refname(local);
strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
if (!display_state->compact_format)
- print_remote_to_local(display_state, remote, prettify_refname(local));
+ print_remote_to_local(display_state, remote, local);
else
- print_compact(display_state, remote, prettify_refname(local));
+ print_compact(display_state, remote, local);
if (error)
strbuf_addf(&display_state->buf, " (%s)", error);
strbuf_addch(&display_state->buf, '\n');
@@ -934,7 +936,7 @@ static void display_ref_update(struct display_state *display_state, char code,
static int update_local_ref(struct ref *ref,
struct ref_transaction *transaction,
struct display_state *display_state,
- const char *remote, const struct ref *remote_ref,
+ const struct ref *remote_ref,
int summary_width)
{
struct commit *current = NULL, *updated;
@@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,
if (oideq(&ref->old_oid, &ref->new_oid)) {
if (verbosity > 0)
display_ref_update(display_state, '=', _("[up to date]"), NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 0;
}
@@ -959,7 +961,7 @@ static int update_local_ref(struct ref *ref,
*/
display_ref_update(display_state, '!', _("[rejected]"),
_("can't fetch into checked-out branch"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 1;
}
@@ -970,12 +972,12 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("updating tag", ref, transaction, 0);
display_ref_update(display_state, r ? '!' : 't', _("[tag update]"),
r ? _("unable to update local ref") : NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"),
_("would clobber existing tag"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 1;
}
}
@@ -1008,7 +1010,7 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref(msg, ref, transaction, 0);
display_ref_update(display_state, r ? '!' : '*', what,
r ? _("unable to update local ref") : NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return r;
}
@@ -1030,7 +1032,7 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("fast-forward", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : ' ', quickref.buf,
r ? _("unable to update local ref") : NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
strbuf_release(&quickref);
return r;
} else if (force || ref->force) {
@@ -1042,12 +1044,12 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("forced-update", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : '+', quickref.buf,
r ? _("unable to update local ref") : _("forced update"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
strbuf_release(&quickref);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"), _("non-fast-forward"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 1;
}
}
@@ -1252,14 +1254,13 @@ static int store_updated_refs(struct display_state *display_state,
if (!strcmp(rm->name, "HEAD")) {
kind = "";
what = "";
- }
- else if (skip_prefix(rm->name, "refs/heads/", &what))
+ } else if (skip_prefix(rm->name, "refs/heads/", &what)) {
kind = "branch";
- else if (skip_prefix(rm->name, "refs/tags/", &what))
+ } else if (skip_prefix(rm->name, "refs/tags/", &what)) {
kind = "tag";
- else if (skip_prefix(rm->name, "refs/remotes/", &what))
+ } else if (skip_prefix(rm->name, "refs/remotes/", &what)) {
kind = "remote-tracking branch";
- else {
+ } else {
kind = "";
what = rm->name;
}
@@ -1277,7 +1278,7 @@ static int store_updated_refs(struct display_state *display_state,
display_state->url_len);
if (ref) {
- rc |= update_local_ref(ref, transaction, display_state, what,
+ rc |= update_local_ref(ref, transaction, display_state,
rm, summary_width);
free(ref);
} else if (write_fetch_head || dry_run) {
@@ -1288,7 +1289,7 @@ static int store_updated_refs(struct display_state *display_state,
*/
display_ref_update(display_state, '*',
*kind ? kind : "branch", NULL,
- *what ? what : "HEAD",
+ rm->name,
"FETCH_HEAD", summary_width);
}
}
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index 0e45c27007..b9dcdade63 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -54,6 +54,31 @@ test_expect_success 'fetch compact output' '
test_cmp expect actual
'
+test_expect_success 'fetch output with HEAD and --dry-run' '
+ test_when_finished "rm -rf head" &&
+ git clone . head &&
+
+ git -C head fetch --dry-run origin HEAD >actual 2>&1 &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * branch HEAD -> FETCH_HEAD
+ EOF
+ test_cmp expect actual &&
+
+ git -C head fetch origin HEAD >actual 2>&1 &&
+ test_cmp expect actual &&
+
+ git -C head fetch --dry-run origin HEAD:foo >actual 2>&1 &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * [new ref] HEAD -> foo
+ EOF
+ test_cmp expect actual &&
+
+ git -C head fetch origin HEAD:foo >actual 2>&1 &&
+ test_cmp expect actual
+'
+
test_expect_success '--no-show-forced-updates' '
mkdir forced-updates &&
(
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v2 4/8] fetch: introduce `display_format` enum
2023-04-27 11:13 ` [PATCH v2 " Patrick Steinhardt
` (2 preceding siblings ...)
2023-04-27 11:13 ` [PATCH v2 3/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
@ 2023-04-27 11:13 ` Patrick Steinhardt
2023-04-27 11:13 ` [PATCH v2 5/8] fetch: move display format parsing into main function Patrick Steinhardt
` (3 subsequent siblings)
7 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-27 11:13 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 5113 bytes --]
We currently have two different display formats in git-fetch(1) with the
"full" and "compact" formats. This is tracked with a boolean value that
simply denotes whether the display format is supposed to be compacted
or not. This works reasonably well while there are only two formats, but
we're about to introduce another format that will make this a bit more
awkward to use.
Introduce a `enum display_format` that is more readily extensible.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 101 ++++++++++++++++++++++++++++++------------------
1 file changed, 64 insertions(+), 37 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 7c64f0c562..e03fcd1b2f 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -48,11 +48,17 @@ enum {
TAGS_SET = 2
};
+enum display_format {
+ DISPLAY_FORMAT_UNKNOWN = 0,
+ DISPLAY_FORMAT_FULL,
+ DISPLAY_FORMAT_COMPACT,
+};
+
struct display_state {
struct strbuf buf;
int refcol_width;
- int compact_format;
+ enum display_format format;
char *url;
int url_len, shown_url;
@@ -784,7 +790,6 @@ static int refcol_width(const struct ref *ref, int compact_format)
static void display_state_init(struct display_state *display_state, struct ref *ref_map,
const char *raw_url)
{
- struct ref *rm;
const char *format = "full";
int i;
@@ -809,31 +814,42 @@ static void display_state_init(struct display_state *display_state, struct ref *
git_config_get_string_tmp("fetch.output", &format);
if (!strcasecmp(format, "full"))
- display_state->compact_format = 0;
+ display_state->format = DISPLAY_FORMAT_FULL;
else if (!strcasecmp(format, "compact"))
- display_state->compact_format = 1;
+ display_state->format = DISPLAY_FORMAT_COMPACT;
else
die(_("invalid value for '%s': '%s'"),
"fetch.output", format);
- display_state->refcol_width = 10;
- for (rm = ref_map; rm; rm = rm->next) {
- int width;
+ switch (display_state->format) {
+ case DISPLAY_FORMAT_FULL:
+ case DISPLAY_FORMAT_COMPACT: {
+ struct ref *rm;
- if (rm->status == REF_STATUS_REJECT_SHALLOW ||
- !rm->peer_ref ||
- !strcmp(rm->name, "HEAD"))
- continue;
+ display_state->refcol_width = 10;
+ for (rm = ref_map; rm; rm = rm->next) {
+ int width;
- width = refcol_width(rm, display_state->compact_format);
+ if (rm->status == REF_STATUS_REJECT_SHALLOW ||
+ !rm->peer_ref ||
+ !strcmp(rm->name, "HEAD"))
+ continue;
- /*
- * Not precise calculation for compact mode because '*' can
- * appear on the left hand side of '->' and shrink the column
- * back.
- */
- if (display_state->refcol_width < width)
- display_state->refcol_width = width;
+ width = refcol_width(rm, display_state->format == DISPLAY_FORMAT_COMPACT);
+
+ /*
+ * Not precise calculation for compact mode because '*' can
+ * appear on the left hand side of '->' and shrink the column
+ * back.
+ */
+ if (display_state->refcol_width < width)
+ display_state->refcol_width = width;
+ }
+
+ break;
+ }
+ default:
+ BUG("unexpected display foramt %d", display_state->format);
}
}
@@ -904,30 +920,41 @@ static void display_ref_update(struct display_state *display_state, char code,
const char *remote, const char *local,
int summary_width)
{
- int width;
-
if (verbosity < 0)
return;
strbuf_reset(&display_state->buf);
- if (!display_state->shown_url) {
- strbuf_addf(&display_state->buf, _("From %.*s\n"),
- display_state->url_len, display_state->url);
- display_state->shown_url = 1;
+ switch (display_state->format) {
+ case DISPLAY_FORMAT_FULL:
+ case DISPLAY_FORMAT_COMPACT: {
+ int width;
+
+ if (!display_state->shown_url) {
+ strbuf_addf(&display_state->buf, _("From %.*s\n"),
+ display_state->url_len, display_state->url);
+ display_state->shown_url = 1;
+ }
+
+ width = (summary_width + strlen(summary) - gettext_width(summary));
+ remote = prettify_refname(remote);
+ local = prettify_refname(local);
+
+ strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
+
+ if (display_state->format != DISPLAY_FORMAT_COMPACT)
+ print_remote_to_local(display_state, remote, local);
+ else
+ print_compact(display_state, remote, local);
+
+ if (error)
+ strbuf_addf(&display_state->buf, " (%s)", error);
+
+ break;
}
-
- width = (summary_width + strlen(summary) - gettext_width(summary));
- remote = prettify_refname(remote);
- local = prettify_refname(local);
-
- strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
- if (!display_state->compact_format)
- print_remote_to_local(display_state, remote, local);
- else
- print_compact(display_state, remote, local);
- if (error)
- strbuf_addf(&display_state->buf, " (%s)", error);
+ default:
+ BUG("unexpected display format %d", display_state->format);
+ };
strbuf_addch(&display_state->buf, '\n');
fputs(display_state->buf.buf, stderr);
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v2 5/8] fetch: move display format parsing into main function
2023-04-27 11:13 ` [PATCH v2 " Patrick Steinhardt
` (3 preceding siblings ...)
2023-04-27 11:13 ` [PATCH v2 4/8] fetch: introduce `display_format` enum Patrick Steinhardt
@ 2023-04-27 11:13 ` Patrick Steinhardt
2023-04-27 11:13 ` [PATCH v2 6/8] fetch: move option related variables " Patrick Steinhardt
` (2 subsequent siblings)
7 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-27 11:13 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 5425 bytes --]
Parsing the display format happens inside of `display_state_init()`. As
we only need to check for a simple config entry, this is a natural
location to put this code as it means that display-state logic is neatly
contained in a single location.
We're about to introduce a output format though that is intended to be
parseable by machines, for example inside of a script. In that case it
becomes a bit awkward of an interface if you have to call git-fetch(1)
with the `fetch.output` config key set. We're thus going to introduce a
new `--output-format` switch for git-fetch(1) so that the output format
can be configured more directly.
This means we'll have to hook parsing of the display format into the
command line options parser. Having the code to determine the actual
output format scattered across two different sites is hard to reason
about though.
Refactor the code such that callers are expected to pass the display
format that is to be used into `display_state_init()`. This allows us to
lift up the code into the main function, where we can then hook it into
command line options parser in a follow-up commit.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 41 ++++++++++++++++++++++++-----------------
1 file changed, 24 insertions(+), 17 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index e03fcd1b2f..bcc156a9ce 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -788,14 +788,13 @@ static int refcol_width(const struct ref *ref, int compact_format)
}
static void display_state_init(struct display_state *display_state, struct ref *ref_map,
- const char *raw_url)
+ const char *raw_url, enum display_format format)
{
- const char *format = "full";
int i;
memset(display_state, 0, sizeof(*display_state));
-
strbuf_init(&display_state->buf, 0);
+ display_state->format = format;
if (raw_url)
display_state->url = transport_anonymize_url(raw_url);
@@ -812,15 +811,6 @@ static void display_state_init(struct display_state *display_state, struct ref *
if (verbosity < 0)
return;
- git_config_get_string_tmp("fetch.output", &format);
- if (!strcasecmp(format, "full"))
- display_state->format = DISPLAY_FORMAT_FULL;
- else if (!strcasecmp(format, "compact"))
- display_state->format = DISPLAY_FORMAT_COMPACT;
- else
- die(_("invalid value for '%s': '%s'"),
- "fetch.output", format);
-
switch (display_state->format) {
case DISPLAY_FORMAT_FULL:
case DISPLAY_FORMAT_COMPACT: {
@@ -1614,7 +1604,8 @@ static int backfill_tags(struct display_state *display_state,
}
static int do_fetch(struct transport *transport,
- struct refspec *rs)
+ struct refspec *rs,
+ enum display_format display_format)
{
struct ref_transaction *transaction = NULL;
struct ref *ref_map = NULL;
@@ -1700,7 +1691,7 @@ static int do_fetch(struct transport *transport,
if (retcode)
goto cleanup;
- display_state_init(&display_state, ref_map, transport->url);
+ display_state_init(&display_state, ref_map, transport->url, display_format);
if (atomic_fetch) {
transaction = ref_transaction_begin(&err);
@@ -2076,7 +2067,8 @@ static inline void fetch_one_setup_partial(struct remote *remote)
}
static int fetch_one(struct remote *remote, int argc, const char **argv,
- int prune_tags_ok, int use_stdin_refspecs)
+ int prune_tags_ok, int use_stdin_refspecs,
+ enum display_format display_format)
{
struct refspec rs = REFSPEC_INIT_FETCH;
int i;
@@ -2143,7 +2135,7 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,
sigchain_push_common(unlock_pack_on_signal);
atexit(unlock_pack_atexit);
sigchain_push(SIGPIPE, SIG_IGN);
- exit_code = do_fetch(gtransport, &rs);
+ exit_code = do_fetch(gtransport, &rs, display_format);
sigchain_pop(SIGPIPE);
refspec_clear(&rs);
transport_disconnect(gtransport);
@@ -2155,6 +2147,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
{
int i;
const char *bundle_uri;
+ enum display_format display_format = DISPLAY_FORMAT_UNKNOWN;
struct string_list list = STRING_LIST_INIT_DUP;
struct remote *remote = NULL;
int result = 0;
@@ -2181,6 +2174,19 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
argc = parse_options(argc, argv, prefix,
builtin_fetch_options, builtin_fetch_usage, 0);
+ if (display_format == DISPLAY_FORMAT_UNKNOWN) {
+ const char *format = "full";
+
+ git_config_get_string_tmp("fetch.output", &format);
+ if (!strcasecmp(format, "full"))
+ display_format = DISPLAY_FORMAT_FULL;
+ else if (!strcasecmp(format, "compact"))
+ display_format = DISPLAY_FORMAT_COMPACT;
+ else
+ die(_("invalid value for '%s': '%s'"),
+ "fetch.output", format);
+ }
+
if (recurse_submodules_cli != RECURSE_SUBMODULES_DEFAULT)
recurse_submodules = recurse_submodules_cli;
@@ -2309,7 +2315,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
} else if (remote) {
if (filter_options.choice || has_promisor_remote())
fetch_one_setup_partial(remote);
- result = fetch_one(remote, argc, argv, prune_tags_ok, stdin_refspecs);
+ result = fetch_one(remote, argc, argv, prune_tags_ok, stdin_refspecs,
+ display_format);
} else {
int max_children = max_jobs;
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v2 6/8] fetch: move option related variables into main function
2023-04-27 11:13 ` [PATCH v2 " Patrick Steinhardt
` (4 preceding siblings ...)
2023-04-27 11:13 ` [PATCH v2 5/8] fetch: move display format parsing into main function Patrick Steinhardt
@ 2023-04-27 11:13 ` Patrick Steinhardt
2023-04-27 21:52 ` Junio C Hamano
2023-04-27 11:13 ` [PATCH v2 7/8] fetch: introduce new `--output-format` option Patrick Steinhardt
2023-04-27 11:13 ` [PATCH v2 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
7 siblings, 1 reply; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-27 11:13 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 12498 bytes --]
The options of git-fetch(1) which we pass to `parse_options()` are
declared globally in `builtin/fetch.c`. This means we're forced to use
global variables for all the options, which is more likely to cause
confusion than explicitly passing state around.
Refactor the code to move the options into `cmd_fetch()`. Move variables
that were previously forced to be declared globally and which are only
used by `cmd_fetch()` into function-local scope.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 197 ++++++++++++++++++++++++------------------------
1 file changed, 100 insertions(+), 97 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index bcc156a9ce..97a510649c 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -75,13 +75,12 @@ static int fetch_prune_tags_config = -1; /* unspecified */
static int prune_tags = -1; /* unspecified */
#define PRUNE_TAGS_BY_DEFAULT 0 /* do we prune tags by default? */
-static int all, append, dry_run, force, keep, multiple, update_head_ok;
+static int append, dry_run, force, keep, update_head_ok;
static int write_fetch_head = 1;
static int verbosity, deepen_relative, set_upstream, refetch;
static int progress = -1;
-static int enable_auto_gc = 1;
-static int tags = TAGS_DEFAULT, unshallow, update_shallow, deepen;
-static int max_jobs = -1, submodule_fetch_jobs_config = -1;
+static int tags = TAGS_DEFAULT, update_shallow, deepen;
+static int submodule_fetch_jobs_config = -1;
static int fetch_parallel_config = 1;
static int atomic_fetch;
static enum transport_family family;
@@ -92,17 +91,11 @@ static struct string_list deepen_not = STRING_LIST_INIT_NODUP;
static struct strbuf default_rla = STRBUF_INIT;
static struct transport *gtransport;
static struct transport *gsecondary;
-static const char *submodule_prefix = "";
static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
-static int recurse_submodules_cli = RECURSE_SUBMODULES_DEFAULT;
-static int recurse_submodules_default = RECURSE_SUBMODULES_ON_DEMAND;
static struct refspec refmap = REFSPEC_INIT_FETCH;
static struct list_objects_filter_options filter_options = LIST_OBJECTS_FILTER_INIT;
static struct string_list server_options = STRING_LIST_INIT_DUP;
static struct string_list negotiation_tip = STRING_LIST_INIT_NODUP;
-static int fetch_write_commit_graph = -1;
-static int stdin_refspecs = 0;
-static int negotiate_only;
static int git_fetch_config(const char *k, const char *v, void *cb)
{
@@ -160,92 +153,6 @@ static int parse_refmap_arg(const struct option *opt, const char *arg, int unset
return 0;
}
-static struct option builtin_fetch_options[] = {
- OPT__VERBOSITY(&verbosity),
- OPT_BOOL(0, "all", &all,
- N_("fetch from all remotes")),
- OPT_BOOL(0, "set-upstream", &set_upstream,
- N_("set upstream for git pull/fetch")),
- OPT_BOOL('a', "append", &append,
- N_("append to .git/FETCH_HEAD instead of overwriting")),
- OPT_BOOL(0, "atomic", &atomic_fetch,
- N_("use atomic transaction to update references")),
- OPT_STRING(0, "upload-pack", &upload_pack, N_("path"),
- N_("path to upload pack on remote end")),
- OPT__FORCE(&force, N_("force overwrite of local reference"), 0),
- OPT_BOOL('m', "multiple", &multiple,
- N_("fetch from multiple remotes")),
- OPT_SET_INT('t', "tags", &tags,
- N_("fetch all tags and associated objects"), TAGS_SET),
- OPT_SET_INT('n', NULL, &tags,
- N_("do not fetch all tags (--no-tags)"), TAGS_UNSET),
- OPT_INTEGER('j', "jobs", &max_jobs,
- N_("number of submodules fetched in parallel")),
- OPT_BOOL(0, "prefetch", &prefetch,
- N_("modify the refspec to place all refs within refs/prefetch/")),
- OPT_BOOL('p', "prune", &prune,
- N_("prune remote-tracking branches no longer on remote")),
- OPT_BOOL('P', "prune-tags", &prune_tags,
- N_("prune local tags no longer on remote and clobber changed tags")),
- OPT_CALLBACK_F(0, "recurse-submodules", &recurse_submodules_cli, N_("on-demand"),
- N_("control recursive fetching of submodules"),
- PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
- OPT_BOOL(0, "dry-run", &dry_run,
- N_("dry run")),
- OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
- N_("write fetched references to the FETCH_HEAD file")),
- OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
- OPT_BOOL('u', "update-head-ok", &update_head_ok,
- N_("allow updating of HEAD ref")),
- OPT_BOOL(0, "progress", &progress, N_("force progress reporting")),
- OPT_STRING(0, "depth", &depth, N_("depth"),
- N_("deepen history of shallow clone")),
- OPT_STRING(0, "shallow-since", &deepen_since, N_("time"),
- N_("deepen history of shallow repository based on time")),
- OPT_STRING_LIST(0, "shallow-exclude", &deepen_not, N_("revision"),
- N_("deepen history of shallow clone, excluding rev")),
- OPT_INTEGER(0, "deepen", &deepen_relative,
- N_("deepen history of shallow clone")),
- OPT_SET_INT_F(0, "unshallow", &unshallow,
- N_("convert to a complete repository"),
- 1, PARSE_OPT_NONEG),
- OPT_SET_INT_F(0, "refetch", &refetch,
- N_("re-fetch without negotiating common commits"),
- 1, PARSE_OPT_NONEG),
- { OPTION_STRING, 0, "submodule-prefix", &submodule_prefix, N_("dir"),
- N_("prepend this to submodule path output"), PARSE_OPT_HIDDEN },
- OPT_CALLBACK_F(0, "recurse-submodules-default",
- &recurse_submodules_default, N_("on-demand"),
- N_("default for recursive fetching of submodules "
- "(lower priority than config files)"),
- PARSE_OPT_HIDDEN, option_fetch_parse_recurse_submodules),
- OPT_BOOL(0, "update-shallow", &update_shallow,
- N_("accept refs that update .git/shallow")),
- OPT_CALLBACK_F(0, "refmap", NULL, N_("refmap"),
- N_("specify fetch refmap"), PARSE_OPT_NONEG, parse_refmap_arg),
- OPT_STRING_LIST('o', "server-option", &server_options, N_("server-specific"), N_("option to transmit")),
- OPT_SET_INT('4', "ipv4", &family, N_("use IPv4 addresses only"),
- TRANSPORT_FAMILY_IPV4),
- OPT_SET_INT('6', "ipv6", &family, N_("use IPv6 addresses only"),
- TRANSPORT_FAMILY_IPV6),
- OPT_STRING_LIST(0, "negotiation-tip", &negotiation_tip, N_("revision"),
- N_("report that we have only objects reachable from this object")),
- OPT_BOOL(0, "negotiate-only", &negotiate_only,
- N_("do not fetch a packfile; instead, print ancestors of negotiation tips")),
- OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
- OPT_BOOL(0, "auto-maintenance", &enable_auto_gc,
- N_("run 'maintenance --auto' after fetching")),
- OPT_BOOL(0, "auto-gc", &enable_auto_gc,
- N_("run 'maintenance --auto' after fetching")),
- OPT_BOOL(0, "show-forced-updates", &fetch_show_forced_updates,
- N_("check for forced-updates on all updated branches")),
- OPT_BOOL(0, "write-commit-graph", &fetch_write_commit_graph,
- N_("write the commit-graph after fetching")),
- OPT_BOOL(0, "stdin", &stdin_refspecs,
- N_("accept refspecs from stdin")),
- OPT_END()
-};
-
static void unlock_pack(unsigned int flags)
{
if (gtransport)
@@ -2145,13 +2052,109 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,
int cmd_fetch(int argc, const char **argv, const char *prefix)
{
- int i;
const char *bundle_uri;
+ const char *submodule_prefix = "";
enum display_format display_format = DISPLAY_FORMAT_UNKNOWN;
struct string_list list = STRING_LIST_INIT_DUP;
struct remote *remote = NULL;
+ int all = 0, multiple = 0;
int result = 0;
int prune_tags_ok = 1;
+ int enable_auto_gc = 1;
+ int unshallow = 0;
+ int max_jobs = -1;
+ int recurse_submodules_cli = RECURSE_SUBMODULES_DEFAULT;
+ int recurse_submodules_default = RECURSE_SUBMODULES_ON_DEMAND;
+ int fetch_write_commit_graph = -1;
+ int stdin_refspecs = 0;
+ int negotiate_only = 0;
+ int i;
+
+ struct option builtin_fetch_options[] = {
+ OPT__VERBOSITY(&verbosity),
+ OPT_BOOL(0, "all", &all,
+ N_("fetch from all remotes")),
+ OPT_BOOL(0, "set-upstream", &set_upstream,
+ N_("set upstream for git pull/fetch")),
+ OPT_BOOL('a', "append", &append,
+ N_("append to .git/FETCH_HEAD instead of overwriting")),
+ OPT_BOOL(0, "atomic", &atomic_fetch,
+ N_("use atomic transaction to update references")),
+ OPT_STRING(0, "upload-pack", &upload_pack, N_("path"),
+ N_("path to upload pack on remote end")),
+ OPT__FORCE(&force, N_("force overwrite of local reference"), 0),
+ OPT_BOOL('m', "multiple", &multiple,
+ N_("fetch from multiple remotes")),
+ OPT_SET_INT('t', "tags", &tags,
+ N_("fetch all tags and associated objects"), TAGS_SET),
+ OPT_SET_INT('n', NULL, &tags,
+ N_("do not fetch all tags (--no-tags)"), TAGS_UNSET),
+ OPT_INTEGER('j', "jobs", &max_jobs,
+ N_("number of submodules fetched in parallel")),
+ OPT_BOOL(0, "prefetch", &prefetch,
+ N_("modify the refspec to place all refs within refs/prefetch/")),
+ OPT_BOOL('p', "prune", &prune,
+ N_("prune remote-tracking branches no longer on remote")),
+ OPT_BOOL('P', "prune-tags", &prune_tags,
+ N_("prune local tags no longer on remote and clobber changed tags")),
+ OPT_CALLBACK_F(0, "recurse-submodules", &recurse_submodules_cli, N_("on-demand"),
+ N_("control recursive fetching of submodules"),
+ PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
+ OPT_BOOL(0, "dry-run", &dry_run,
+ N_("dry run")),
+ OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
+ N_("write fetched references to the FETCH_HEAD file")),
+ OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
+ OPT_BOOL('u', "update-head-ok", &update_head_ok,
+ N_("allow updating of HEAD ref")),
+ OPT_BOOL(0, "progress", &progress, N_("force progress reporting")),
+ OPT_STRING(0, "depth", &depth, N_("depth"),
+ N_("deepen history of shallow clone")),
+ OPT_STRING(0, "shallow-since", &deepen_since, N_("time"),
+ N_("deepen history of shallow repository based on time")),
+ OPT_STRING_LIST(0, "shallow-exclude", &deepen_not, N_("revision"),
+ N_("deepen history of shallow clone, excluding rev")),
+ OPT_INTEGER(0, "deepen", &deepen_relative,
+ N_("deepen history of shallow clone")),
+ OPT_SET_INT_F(0, "unshallow", &unshallow,
+ N_("convert to a complete repository"),
+ 1, PARSE_OPT_NONEG),
+ OPT_SET_INT_F(0, "refetch", &refetch,
+ N_("re-fetch without negotiating common commits"),
+ 1, PARSE_OPT_NONEG),
+ { OPTION_STRING, 0, "submodule-prefix", &submodule_prefix, N_("dir"),
+ N_("prepend this to submodule path output"), PARSE_OPT_HIDDEN },
+ OPT_CALLBACK_F(0, "recurse-submodules-default",
+ &recurse_submodules_default, N_("on-demand"),
+ N_("default for recursive fetching of submodules "
+ "(lower priority than config files)"),
+ PARSE_OPT_HIDDEN, option_fetch_parse_recurse_submodules),
+ OPT_BOOL(0, "update-shallow", &update_shallow,
+ N_("accept refs that update .git/shallow")),
+ OPT_CALLBACK_F(0, "refmap", NULL, N_("refmap"),
+ N_("specify fetch refmap"), PARSE_OPT_NONEG, parse_refmap_arg),
+ OPT_STRING_LIST('o', "server-option", &server_options, N_("server-specific"), N_("option to transmit")),
+ OPT_SET_INT('4', "ipv4", &family, N_("use IPv4 addresses only"),
+ TRANSPORT_FAMILY_IPV4),
+ OPT_SET_INT('6', "ipv6", &family, N_("use IPv6 addresses only"),
+ TRANSPORT_FAMILY_IPV6),
+ OPT_STRING_LIST(0, "negotiation-tip", &negotiation_tip, N_("revision"),
+ N_("report that we have only objects reachable from this object")),
+ OPT_BOOL(0, "negotiate-only", &negotiate_only,
+ N_("do not fetch a packfile; instead, print ancestors of negotiation tips")),
+ OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
+ OPT_BOOL(0, "auto-maintenance", &enable_auto_gc,
+ N_("run 'maintenance --auto' after fetching")),
+ OPT_BOOL(0, "auto-gc", &enable_auto_gc,
+ N_("run 'maintenance --auto' after fetching")),
+ OPT_BOOL(0, "show-forced-updates", &fetch_show_forced_updates,
+ N_("check for forced-updates on all updated branches")),
+ OPT_BOOL(0, "write-commit-graph", &fetch_write_commit_graph,
+ N_("write the commit-graph after fetching")),
+ OPT_BOOL(0, "stdin", &stdin_refspecs,
+ N_("accept refspecs from stdin")),
+ OPT_END()
+ };
packet_trace_identity("fetch");
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v2 7/8] fetch: introduce new `--output-format` option
2023-04-27 11:13 ` [PATCH v2 " Patrick Steinhardt
` (5 preceding siblings ...)
2023-04-27 11:13 ` [PATCH v2 6/8] fetch: move option related variables " Patrick Steinhardt
@ 2023-04-27 11:13 ` Patrick Steinhardt
2023-04-27 22:01 ` Junio C Hamano
2023-04-28 22:31 ` Glen Choo
2023-04-27 11:13 ` [PATCH v2 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
7 siblings, 2 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-27 11:13 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 8241 bytes --]
It is only possible to configure the output format that git-fetch(1)
uses by setting it via a config key. While this interface may be fine as
long as we only have the current "full" and "compact" output formats,
where it is unlikely that the user will have to change them regularly.
But we're about to introduce a new machine-parseable interface where the
current mechanism feels a little bit indirect and rigid.
Introduce a new `--output-format` option that allows the user to change
the desired format more directly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
Documentation/fetch-options.txt | 5 +++
builtin/fetch.c | 48 ++++++++++++++++++----
t/t5574-fetch-output.sh | 72 +++++++++++++++++++++++++++------
3 files changed, 106 insertions(+), 19 deletions(-)
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 622bd84768..654f96f79d 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -78,6 +78,11 @@ linkgit:git-config[1].
--dry-run::
Show what would be done, without making any changes.
+--output-format::
+ Control how ref update status is printed. Valid values are
+ `full` and `compact`. Default value is `full`. See section
+ OUTPUT in linkgit:git-fetch[1] for detail.
+
ifndef::git-pull[]
--[no-]write-fetch-head::
Write the list of remote refs fetched in the `FETCH_HEAD`
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 97a510649c..30099b2ac3 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -52,6 +52,13 @@ enum display_format {
DISPLAY_FORMAT_UNKNOWN = 0,
DISPLAY_FORMAT_FULL,
DISPLAY_FORMAT_COMPACT,
+ DISPLAY_FORMAT_MAX,
+};
+
+static const char * const display_formats[DISPLAY_FORMAT_MAX] = {
+ NULL,
+ "full",
+ "compact",
};
struct display_state {
@@ -1879,7 +1886,8 @@ static int fetch_finished(int result, struct strbuf *out,
return 0;
}
-static int fetch_multiple(struct string_list *list, int max_children)
+static int fetch_multiple(struct string_list *list, int max_children,
+ enum display_format format)
{
int i, result = 0;
struct strvec argv = STRVEC_INIT;
@@ -1894,6 +1902,9 @@ static int fetch_multiple(struct string_list *list, int max_children)
"--no-write-commit-graph", NULL);
add_options_to_argv(&argv);
+ if (format != DISPLAY_FORMAT_UNKNOWN)
+ strvec_pushf(&argv, "--output-format=%s", display_formats[format]);
+
if (max_children != 1 && list->nr != 1) {
struct parallel_fetch_state state = { argv.v, list, 0, 0 };
const struct run_process_parallel_opts opts = {
@@ -2050,6 +2061,29 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,
return exit_code;
}
+static enum display_format parse_display_format(const char *format)
+{
+ for (int i = 0; i < ARRAY_SIZE(display_formats); i++)
+ if (display_formats[i] && !strcmp(display_formats[i], format))
+ return i;
+ return DISPLAY_FORMAT_UNKNOWN;
+}
+
+static int opt_parse_output_format(const struct option *opt, const char *arg, int unset)
+{
+ enum display_format *format = opt->value, parsed;
+
+ if (unset || !arg)
+ return 1;
+
+ parsed = parse_display_format(arg);
+ if (parsed == DISPLAY_FORMAT_UNKNOWN)
+ return error(_("unsupported output format '%s'"), arg);
+ *format = parsed;
+
+ return 0;
+}
+
int cmd_fetch(int argc, const char **argv, const char *prefix)
{
const char *bundle_uri;
@@ -2102,6 +2136,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
OPT_BOOL(0, "dry-run", &dry_run,
N_("dry run")),
+ OPT_CALLBACK(0, "output-format", &display_format, N_("format"), N_("output format"),
+ opt_parse_output_format),
OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
N_("write fetched references to the FETCH_HEAD file")),
OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
@@ -2181,11 +2217,9 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
const char *format = "full";
git_config_get_string_tmp("fetch.output", &format);
- if (!strcasecmp(format, "full"))
- display_format = DISPLAY_FORMAT_FULL;
- else if (!strcasecmp(format, "compact"))
- display_format = DISPLAY_FORMAT_COMPACT;
- else
+
+ display_format = parse_display_format(format);
+ if (display_format == DISPLAY_FORMAT_UNKNOWN)
die(_("invalid value for '%s': '%s'"),
"fetch.output", format);
}
@@ -2339,7 +2373,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
max_children = fetch_parallel_config;
/* TODO should this also die if we have a previous partial-clone? */
- result = fetch_multiple(&list, max_children);
+ result = fetch_multiple(&list, max_children, display_format);
}
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index b9dcdade63..662c960f94 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -24,14 +24,37 @@ test_expect_success 'fetch with invalid output format configuration' '
test_cmp expect actual
'
+test_expect_success 'fetch with invalid output format via command line' '
+ test_must_fail git fetch --output-format >actual 2>&1 &&
+ cat >expect <<-EOF &&
+ error: option \`output-format${SQ} requires a value
+ EOF
+ test_cmp expect actual &&
+
+ test_must_fail git fetch --output-format= origin >actual 2>&1 &&
+ cat >expect <<-EOF &&
+ error: unsupported output format ${SQ}${SQ}
+ EOF
+ test_cmp expect actual &&
+
+ test_must_fail git fetch --output-format=garbage origin >actual 2>&1 &&
+ cat >expect <<-EOF &&
+ error: unsupported output format ${SQ}garbage${SQ}
+ EOF
+ test_cmp expect actual
+'
+
test_expect_success 'fetch aligned output' '
- git clone . full-output &&
+ test_when_finished "rm -rf full-cfg full-cli" &&
+ git clone . full-cfg &&
+ git clone . full-cli &&
test_commit looooooooooooong-tag &&
- (
- cd full-output &&
- git -c fetch.output=full fetch origin >actual 2>&1 &&
- grep -e "->" actual | cut -c 22- >../actual
- ) &&
+
+ git -C full-cfg -c fetch.output=full fetch origin >actual-cfg 2>&1 &&
+ git -C full-cli fetch --output-format=full origin >actual-cli 2>&1 &&
+ test_cmp actual-cfg actual-cli &&
+
+ grep -e "->" actual-cfg | cut -c 22- >actual &&
cat >expect <<-\EOF &&
main -> origin/main
looooooooooooong-tag -> looooooooooooong-tag
@@ -40,13 +63,16 @@ test_expect_success 'fetch aligned output' '
'
test_expect_success 'fetch compact output' '
- git clone . compact &&
+ test_when_finished "rm -rf compact-cfg compact-cli" &&
+ git clone . compact-cli &&
+ git clone . compact-cfg &&
test_commit extraaa &&
- (
- cd compact &&
- git -c fetch.output=compact fetch origin >actual 2>&1 &&
- grep -e "->" actual | cut -c 22- >../actual
- ) &&
+
+ git -C compact-cfg -c fetch.output=compact fetch origin >actual-cfg 2>&1 &&
+ git -C compact-cli fetch --output-format=compact origin >actual-cli 2>&1 &&
+ test_cmp actual-cfg actual-cli &&
+
+ grep -e "->" actual-cfg | cut -c 22- >actual &&
cat >expect <<-\EOF &&
main -> origin/*
extraaa -> *
@@ -54,6 +80,28 @@ test_expect_success 'fetch compact output' '
test_cmp expect actual
'
+test_expect_success 'fetch compact output with multiple remotes' '
+ test_when_finished "rm -rf compact-cfg compact-cli" &&
+
+ git clone . compact-cli &&
+ git -C compact-cli remote add second-remote "$PWD" &&
+ git clone . compact-cfg &&
+ git -C compact-cfg remote add second-remote "$PWD" &&
+ test_commit multi-commit &&
+
+ git -C compact-cfg -c fetch.output=compact fetch --all >actual-cfg 2>&1 &&
+ git -C compact-cli fetch --output-format=compact --all >actual-cli 2>&1 &&
+ test_cmp actual-cfg actual-cli &&
+
+ grep -e "->" actual-cfg | cut -c 22- >actual &&
+ cat >expect <<-\EOF &&
+ main -> origin/*
+ multi-commit -> *
+ main -> second-remote/*
+ EOF
+ test_cmp expect actual
+'
+
test_expect_success 'fetch output with HEAD and --dry-run' '
test_when_finished "rm -rf head" &&
git clone . head &&
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v2 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-04-27 11:13 ` [PATCH v2 " Patrick Steinhardt
` (6 preceding siblings ...)
2023-04-27 11:13 ` [PATCH v2 7/8] fetch: introduce new `--output-format` option Patrick Steinhardt
@ 2023-04-27 11:13 ` Patrick Steinhardt
2023-04-27 19:52 ` Jacob Keller
2023-04-28 22:42 ` Glen Choo
7 siblings, 2 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-27 11:13 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 15116 bytes --]
The output of git-fetch(1) is obviously designed for consumption by
users, only: we neatly columnize data, we abbreviate reference names, we
print neat arrows and we don't provide information about actual object
IDs that have changed. This makes the output format basically unusable
in the context of scripted invocations of git-fetch(1) that want to
learn about the exact changes that the command performs.
Introduce a new machine-parseable "porcelain" output format that is
supposed to fix this shortcoming. This output format is intended to
provide information about every reference that is about to be updated,
the old object ID that the reference has been pointing to and the new
object ID it will be updated to. Furthermore, the output format provides
the same flags as the human-readable format to indicate basic conditions
for each reference update like whether it was a fast-forward update, a
branch deletion, a rejected update or others.
The output format is quite simple:
```
<flag> <old-object-id> <new-object-id> <local-reference>\n
```
We assume two conditions which are generally true:
- The old and new object IDs have fixed known widths and cannot
contain spaces.
- References cannot contain newlines.
With these assumptions, the output format becomes unambiguously
parseable. Furthermore, given that this output is designed to be
consumed by scripts, the machine-readable data is printed to stdout
instead of stderr like the human-readable output is. This is mostly done
so that other data printed to stderr, like error messages or progress
meters, don't interfere with the parseable data.
A notable ommission here is that the output format does not include the
remote from which a reference was fetched, which might be important
information especially in the context of multi-remote fetches. But as
such a format would require us to print the remote for every single
reference update due to parallelizable fetches it feels wasteful for the
most likely usecase, which is when fetching from a single remote. If
usecases come up for this in the future though it is easy enough to add
a new "porcelain-v2" format that adds this information.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
Documentation/config/fetch.txt | 4 +-
Documentation/fetch-options.txt | 4 +-
Documentation/git-fetch.txt | 17 ++++++-
builtin/fetch.c | 47 +++++++++++++-----
t/t5574-fetch-output.sh | 84 +++++++++++++++++++++++++++++++++
5 files changed, 139 insertions(+), 17 deletions(-)
diff --git a/Documentation/config/fetch.txt b/Documentation/config/fetch.txt
index 568f0f75b3..70734226c0 100644
--- a/Documentation/config/fetch.txt
+++ b/Documentation/config/fetch.txt
@@ -52,8 +52,8 @@ fetch.pruneTags::
fetch.output::
Control how ref update status is printed. Valid values are
- `full` and `compact`. Default value is `full`. See section
- OUTPUT in linkgit:git-fetch[1] for detail.
+ `full`, `compact` and `porcelain`. Default value is `full`.
+ See section OUTPUT in linkgit:git-fetch[1] for detail.
fetch.negotiationAlgorithm::
Control how information about the commits in the local repository
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 654f96f79d..5ca8a67fe8 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -80,8 +80,8 @@ linkgit:git-config[1].
--output-format::
Control how ref update status is printed. Valid values are
- `full` and `compact`. Default value is `full`. See section
- OUTPUT in linkgit:git-fetch[1] for detail.
+ `full`, `compact` and `porcelain`. Default value is `full`.
+ See section OUTPUT in linkgit:git-fetch[1] for detail.
ifndef::git-pull[]
--[no-]write-fetch-head::
diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index fba66f1460..efd22cd372 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -197,13 +197,26 @@ The output of "git fetch" depends on the transport method used; this
section describes the output when fetching over the Git protocol
(either locally or via ssh) and Smart HTTP protocol.
-The status of the fetch is output in tabular form, with each line
-representing the status of a single ref. Each line is of the form:
+The output format can be chosen either via the `fetch.output` config
+(see linkgit:git-config[1]), or via the `--output-format` switch.
+Supported values include:
+
+For the `full` and `compact` output formats, the status of the fetch is
+output in tabular, with each line representing the status of a single
+ref. Each line is of the form:
-------------------------------
<flag> <summary> <from> -> <to> [<reason>]
-------------------------------
+The `porcelain` output format is intended to be machine-parseable. In
+contrast to the human-readable output formats it thus prints to standard
+output instead of standard error. Each line is of the form:
+
+-------------------------------
+<flag> <old-object-id> <new-object-id> <local-reference>
+-------------------------------
+
The status of up-to-date refs is shown only if the --verbose option is
used.
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 30099b2ac3..abe6b8879d 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -52,6 +52,7 @@ enum display_format {
DISPLAY_FORMAT_UNKNOWN = 0,
DISPLAY_FORMAT_FULL,
DISPLAY_FORMAT_COMPACT,
+ DISPLAY_FORMAT_PORCELAIN,
DISPLAY_FORMAT_MAX,
};
@@ -59,6 +60,7 @@ static const char * const display_formats[DISPLAY_FORMAT_MAX] = {
NULL,
"full",
"compact",
+ "porcelain",
};
struct display_state {
@@ -752,8 +754,11 @@ static void display_state_init(struct display_state *display_state, struct ref *
break;
}
+ case DISPLAY_FORMAT_PORCELAIN:
+ /* We don't need to precompute anything here. */
+ break;
default:
- BUG("unexpected display foramt %d", display_state->format);
+ BUG("unexpected display format %d", display_state->format);
}
}
@@ -822,8 +827,12 @@ static void print_compact(struct display_state *display_state,
static void display_ref_update(struct display_state *display_state, char code,
const char *summary, const char *error,
const char *remote, const char *local,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
int summary_width)
{
+ FILE *f = stderr;
+
if (verbosity < 0)
return;
@@ -856,12 +865,17 @@ static void display_ref_update(struct display_state *display_state, char code,
break;
}
+ case DISPLAY_FORMAT_PORCELAIN:
+ strbuf_addf(&display_state->buf, "%c %s %s %s", code,
+ oid_to_hex(old_oid), oid_to_hex(new_oid), local);
+ f = stdout;
+ break;
default:
BUG("unexpected display format %d", display_state->format);
};
strbuf_addch(&display_state->buf, '\n');
- fputs(display_state->buf.buf, stderr);
+ fputs(display_state->buf.buf, f);
}
static int update_local_ref(struct ref *ref,
@@ -879,7 +893,8 @@ static int update_local_ref(struct ref *ref,
if (oideq(&ref->old_oid, &ref->new_oid)) {
if (verbosity > 0)
display_ref_update(display_state, '=', _("[up to date]"), NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 0;
}
@@ -892,7 +907,8 @@ static int update_local_ref(struct ref *ref,
*/
display_ref_update(display_state, '!', _("[rejected]"),
_("can't fetch into checked-out branch"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 1;
}
@@ -903,12 +919,14 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("updating tag", ref, transaction, 0);
display_ref_update(display_state, r ? '!' : 't', _("[tag update]"),
r ? _("unable to update local ref") : NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"),
_("would clobber existing tag"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 1;
}
}
@@ -941,7 +959,8 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref(msg, ref, transaction, 0);
display_ref_update(display_state, r ? '!' : '*', what,
r ? _("unable to update local ref") : NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return r;
}
@@ -963,7 +982,8 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("fast-forward", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : ' ', quickref.buf,
r ? _("unable to update local ref") : NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
strbuf_release(&quickref);
return r;
} else if (force || ref->force) {
@@ -975,12 +995,14 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("forced-update", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : '+', quickref.buf,
r ? _("unable to update local ref") : _("forced update"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
strbuf_release(&quickref);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"), _("non-fast-forward"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 1;
}
}
@@ -1221,7 +1243,9 @@ static int store_updated_refs(struct display_state *display_state,
display_ref_update(display_state, '*',
*kind ? kind : "branch", NULL,
rm->name,
- "FETCH_HEAD", summary_width);
+ "FETCH_HEAD",
+ &rm->new_oid, &rm->old_oid,
+ summary_width);
}
}
}
@@ -1361,6 +1385,7 @@ static int prune_refs(struct display_state *display_state,
for (ref = stale_refs; ref; ref = ref->next) {
display_ref_update(display_state, '-', _("[deleted]"), NULL,
_("(none)"), ref->name,
+ &ref->new_oid, &ref->old_oid,
summary_width);
warn_dangling_symref(stderr, dangling_msg, ref->name);
}
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index 662c960f94..d88ad8af31 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -102,6 +102,72 @@ test_expect_success 'fetch compact output with multiple remotes' '
test_cmp expect actual
'
+test_expect_success 'fetch porcelain output' '
+ test_when_finished "rm -rf porcelain-cfg porcelain-cli" &&
+
+ # Set up a bunch of references that we can use to demonstrate different
+ # kinds of flag symbols in the output format.
+ MAIN_OLD=$(git rev-parse HEAD) &&
+ git branch "fast-forward" &&
+ git branch "deleted-branch" &&
+ git checkout -b force-updated &&
+ test_commit --no-tag force-update-old &&
+ FORCE_UPDATED_OLD=$(git rev-parse HEAD) &&
+ git checkout main &&
+
+ # Clone and pre-seed the repositories. We fetch references into two
+ # namespaces so that we can test that rejected and force-updated
+ # references are reported properly.
+ refspecs="refs/heads/*:refs/unforced/* +refs/heads/*:refs/forced/*" &&
+ git clone . porcelain-cli &&
+ git clone . porcelain-cfg &&
+ git -C porcelain-cfg fetch origin $refspecs &&
+ git -C porcelain-cli fetch origin $refspecs &&
+
+ # Now that we have set up the client repositories we can change our
+ # local references.
+ git branch new-branch &&
+ git branch -d deleted-branch &&
+ git checkout fast-forward &&
+ test_commit --no-tag fast-forward-new &&
+ FAST_FORWARD_NEW=$(git rev-parse HEAD) &&
+ git checkout force-updated &&
+ git reset --hard HEAD~ &&
+ test_commit --no-tag force-update-new &&
+ FORCE_UPDATED_NEW=$(git rev-parse HEAD) &&
+
+ cat >expect <<-EOF &&
+ - $MAIN_OLD $ZERO_OID refs/forced/deleted-branch
+ - $MAIN_OLD $ZERO_OID refs/unforced/deleted-branch
+ $MAIN_OLD $FAST_FORWARD_NEW refs/unforced/fast-forward
+ ! $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/unforced/force-updated
+ * $ZERO_OID $MAIN_OLD refs/unforced/new-branch
+ $MAIN_OLD $FAST_FORWARD_NEW refs/forced/fast-forward
+ + $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/forced/force-updated
+ * $ZERO_OID $MAIN_OLD refs/forced/new-branch
+ $MAIN_OLD $FAST_FORWARD_NEW refs/remotes/origin/fast-forward
+ + $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/remotes/origin/force-updated
+ * $ZERO_OID $MAIN_OLD refs/remotes/origin/new-branch
+ EOF
+
+ # Execute a dry-run fetch first. We do this to assert that the dry-run
+ # and non-dry-run fetches produces the same output. Execution of the
+ # fetch is expected to fail as we have a rejected reference update.
+ test_must_fail git -C porcelain-cfg -c fetch.output=porcelain fetch --dry-run --prune origin $refspecs >actual-dry-run-cfg &&
+ test_must_fail git -C porcelain-cli fetch --output-format=porcelain --dry-run --prune origin $refspecs >actual-dry-run-cli &&
+ test_cmp actual-dry-run-cfg actual-dry-run-cli &&
+ test_cmp expect actual-dry-run-cfg &&
+
+ # And now we perform a non-dry-run fetch.
+ test_must_fail git -C porcelain-cfg -c fetch.output=porcelain fetch --prune origin $refspecs >actual-cfg &&
+ test_must_fail git -C porcelain-cli fetch --output-format=porcelain --prune origin $refspecs >actual-cli &&
+ test_cmp actual-cfg actual-cli &&
+ test_cmp expect actual-cfg &&
+
+ # Ensure that the dry-run and non-dry-run output matches.
+ test_cmp actual-dry-run-cfg actual-cfg
+'
+
test_expect_success 'fetch output with HEAD and --dry-run' '
test_when_finished "rm -rf head" &&
git clone . head &&
@@ -127,6 +193,24 @@ test_expect_success 'fetch output with HEAD and --dry-run' '
test_cmp expect actual
'
+test_expect_success 'fetch porcelain output with HEAD and --dry-run' '
+ test_when_finished "rm -rf head" &&
+ git clone . head &&
+ COMMIT_ID=$(git rev-parse HEAD) &&
+
+ git -C head fetch --output-format=porcelain --dry-run origin HEAD >actual &&
+ cat >expect <<-EOF &&
+ * $ZERO_OID $COMMIT_ID FETCH_HEAD
+ EOF
+ test_cmp expect actual &&
+
+ git -C head fetch --output-format=porcelain --dry-run origin HEAD:foo >actual &&
+ cat >expect <<-EOF &&
+ * $ZERO_OID $COMMIT_ID refs/heads/foo
+ EOF
+ test_cmp expect actual
+'
+
test_expect_success '--no-show-forced-updates' '
mkdir forced-updates &&
(
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* Re: [PATCH v2 3/8] fetch: fix missing from-reference when fetching HEAD:foo
2023-04-27 11:13 ` [PATCH v2 3/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
@ 2023-04-27 17:26 ` Glen Choo
2023-04-27 19:49 ` Jacob Keller
1 sibling, 0 replies; 120+ messages in thread
From: Glen Choo @ 2023-04-27 17:26 UTC (permalink / raw)
To: Patrick Steinhardt, git
Cc: Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
For v2, I've only read through this patch for now. I'm still formulating
my thoughts on the output format and the CLI option name.
Patrick Steinhardt <ps@pks.im> writes:
> `store_updated_refs()` parses the remote reference for two purposes:
>
> - It gets used as a note when writing FETCH_HEAD.
>
> - It is passed through to `display_ref_update()` to display
> updated references in the following format:
>
> ```
> * branch master -> master
> ```
>
> In most cases, the parsed remote reference is the prettified reference
> name and can thus be used for both cases. But if the remote reference is
> HEAD, the parsed remote reference becomes empty. This is intended when
> we write the FETCH_HEAD, where we skip writing the note in that case.
> But it is not intended when displaying the updated references and would
> cause us to miss the left-hand side of the displayed reference update:
>
> [...]
>
> As the return value of `prettify_refname()` would do the correct thing
> for us in all three cases, we can fix the bug by passing through the
> full remote reference name to `display_ref_update()`, which learns to
> call `prettify_refname()`. At the same time, this also simplifies the
> code a bit.
This rewritten section explains the bug and the fix very nicely :)
Thanks!
> diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
> index 0e45c27007..b9dcdade63 100755
> --- a/t/t5574-fetch-output.sh
> +++ b/t/t5574-fetch-output.sh
> @@ -54,6 +54,31 @@ test_expect_success 'fetch compact output' '
> test_cmp expect actual
> '
>
> +test_expect_success 'fetch output with HEAD and --dry-run' '
> + test_when_finished "rm -rf head" &&
> + git clone . head &&
> +
> + git -C head fetch --dry-run origin HEAD >actual 2>&1 &&
> + cat >expect <<-EOF &&
> + From $(test-tool path-utils real_path .)/.
> + * branch HEAD -> FETCH_HEAD
> + EOF
> + test_cmp expect actual &&
> +
> + git -C head fetch origin HEAD >actual 2>&1 &&
> + test_cmp expect actual &&
Maybe rename the test to just 'fetch output with HEAD'? The 'and
--dry-run' part seems completely obsolete.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 0/8] fetch: introduce machine-parseable output
2023-04-27 10:58 ` Patrick Steinhardt
@ 2023-04-27 19:46 ` Jacob Keller
0 siblings, 0 replies; 120+ messages in thread
From: Jacob Keller @ 2023-04-27 19:46 UTC (permalink / raw)
To: Patrick Steinhardt, Junio C Hamano; +Cc: git
On 4/27/2023 3:58 AM, Patrick Steinhardt wrote:
> On Wed, Apr 26, 2023 at 01:23:12PM -0700, Junio C Hamano wrote:
> Furthermore, I think that `--[output-]format` has the advantage that you
> don't need to handle priorities or mutual exclusion of different options
> that all apply to the reference format. To a user, it is not immediately
> obvious what `git fetch --format=compact --porcelain` would do, and
> which of both options ultimately get respected. But that's likely only
> true for future commands, because any migration would create the same
> kind of ambiguity for preexisting commands.
>
> If we were to also migrate preexisting code to use `--[output-]format`
> then I'd argue that `--output-format` is likely the better name, mostly
> because it is less likely to be ambiguous compared to `--format`. The
> latter could e.g. easily confused with `--object-format`.
>
> So I think I'll stick with `--output-format` for the time being.
>
I agree. I think using --output-format and migrating existing commands
that have --porcelain to use --output-format is good. I'm not sure
whether to keep using --output-format=porcelain or whether to use a
different term there.
I definitely think avoiding confusion with --format is good, and I don't
think its too much of a burden to have the longer --output-format as the
option name.
Thanks,
Jake
> Patrick
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v2 3/8] fetch: fix missing from-reference when fetching HEAD:foo
2023-04-27 11:13 ` [PATCH v2 3/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
2023-04-27 17:26 ` Glen Choo
@ 2023-04-27 19:49 ` Jacob Keller
1 sibling, 0 replies; 120+ messages in thread
From: Jacob Keller @ 2023-04-27 19:49 UTC (permalink / raw)
To: Patrick Steinhardt, git
Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan
On 4/27/2023 4:13 AM, Patrick Steinhardt wrote:
> index 0e45c27007..b9dcdade63 100755
> --- a/t/t5574-fetch-output.sh
> +++ b/t/t5574-fetch-output.sh
> @@ -54,6 +54,31 @@ test_expect_success 'fetch compact output' '
> test_cmp expect actual
> '
>
> +test_expect_success 'fetch output with HEAD and --dry-run' '
> + test_when_finished "rm -rf head" &&
The test name could drop the --dry-run mention since this now tests both
variants.
Everything else looks good on this patch. The new commit message is much
better to me now!
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v2 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-04-27 11:13 ` [PATCH v2 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
@ 2023-04-27 19:52 ` Jacob Keller
2023-04-28 22:42 ` Glen Choo
1 sibling, 0 replies; 120+ messages in thread
From: Jacob Keller @ 2023-04-27 19:52 UTC (permalink / raw)
To: Patrick Steinhardt, git
Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan
On 4/27/2023 4:13 AM, Patrick Steinhardt wrote:
> A notable ommission here is that the output format does not include the
> remote from which a reference was fetched, which might be important
> information especially in the context of multi-remote fetches. But as
> such a format would require us to print the remote for every single
> reference update due to parallelizable fetches it feels wasteful for the
> most likely usecase, which is when fetching from a single remote. If
> usecases come up for this in the future though it is easy enough to add
> a new "porcelain-v2" format that adds this information.
Thanks for clarifying this. I think generally its ok to leave out since
you can often infer what remote a ref came from (either because of
refs/remotes, or because of how you configured your refspecs for fetching).
We do have the option to extend this in the future if needed, so I think
that's ok. In that case if we wanted an even more verbose output I'd
probably argue for something more structured like adding it as part of a
JSON output.
Thanks,
Jake
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v2 6/8] fetch: move option related variables into main function
2023-04-27 11:13 ` [PATCH v2 6/8] fetch: move option related variables " Patrick Steinhardt
@ 2023-04-27 21:52 ` Junio C Hamano
0 siblings, 0 replies; 120+ messages in thread
From: Junio C Hamano @ 2023-04-27 21:52 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> The options of git-fetch(1) which we pass to `parse_options()` are
> declared globally in `builtin/fetch.c`. This means we're forced to use
> global variables for all the options, which is more likely to cause
> confusion than explicitly passing state around.
>
> Refactor the code to move the options into `cmd_fetch()`. Move variables
> that were previously forced to be declared globally and which are only
> used by `cmd_fetch()` into function-local scope.
Very well done.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v2 7/8] fetch: introduce new `--output-format` option
2023-04-27 11:13 ` [PATCH v2 7/8] fetch: introduce new `--output-format` option Patrick Steinhardt
@ 2023-04-27 22:01 ` Junio C Hamano
2023-04-28 22:03 ` Glen Choo
2023-04-28 22:31 ` Glen Choo
1 sibling, 1 reply; 120+ messages in thread
From: Junio C Hamano @ 2023-04-27 22:01 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> index 97a510649c..30099b2ac3 100644
> --- a/builtin/fetch.c
> +++ b/builtin/fetch.c
> @@ -52,6 +52,13 @@ enum display_format {
> DISPLAY_FORMAT_UNKNOWN = 0,
> DISPLAY_FORMAT_FULL,
> DISPLAY_FORMAT_COMPACT,
> + DISPLAY_FORMAT_MAX,
> +};
> +
> +static const char * const display_formats[DISPLAY_FORMAT_MAX] = {
> + NULL,
> + "full",
> + "compact",
> };
Hmph, the _MAX thing that is only needed to size the array and never
used elsewhere (i.e. parse_display_format() uses ARRAY_SIZE() of the
thing, instead of the constant, and that is just fine) is an eyesore.
I wonder if
static const char *const display_format[] = {
[DISPLAY_FORMAT_UNKNOWN] = NULL,
[DISPLAY_FORMAT_FULL] = "full",
[DISPLAY_FORMAT_COMPACT] = "compact",
};
would be easier to maintain?
I'll omit my usual "name your array in singular" lecture, as I think
you've heard it already.
Thanks.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 0/8] fetch: introduce machine-parseable output
2023-04-26 20:23 ` Junio C Hamano
2023-04-26 20:30 ` Jacob Keller
2023-04-27 10:58 ` Patrick Steinhardt
@ 2023-04-27 22:49 ` Glen Choo
2 siblings, 0 replies; 120+ messages in thread
From: Glen Choo @ 2023-04-27 22:49 UTC (permalink / raw)
To: Junio C Hamano, Jacob Keller; +Cc: Patrick Steinhardt, git
Junio C Hamano <gitster@pobox.com> writes:
>> We had some discussion during review club about this, where the idea of
>> using "--porcelain" came up because many commands use that when
>> switching into a machine readable format.
>>
>> In addition, this format not only changes the output but also moves it
>> from being on stderr to stdout, which is a hint that the intended usage
>> of the command is now a little different.
>
> A little different from what? I do not think the answer would be
> "other program's --porcelain mode", as sending them to stdout would
> be one of the things that make the output easier for programs to
> parse, so it does sound like very much in the same spirit as "git
> status --porcelain" where its output format gets tweaked to be more
> machine friendly.
>
> The output with "--porcelain" option enabled tend to be less human
> friendly and the distinction between Porcelain (for humans) and
> plumbing (for scripts) is reversed in the use of the word there---it
> started as "this is the option for those who write Porcelain
> commands to use", but still it is not a very good name for the
> option.
>
> I am perfectly OK if the plan is to uniformly use --output-format
> (or something equally more descriptive) and migrate and deprecate
> the "--porcelain" option away from existing commands.
I agree that --porcelain is a confusing name that would be nice to
deprecate, but I don't think --output-format captures all of the intent
of "operate in a machine-friendly mode instead of a human-friendly one".
Unfortunately, if we had picked --plumbing from the beginning, I doubt
we would be having this discussion today :/
E.g. machines (Unix ones at least?) like to have output on stdout and to
be able to request NUL-terminated output. It's unfortunate that if we
don't piggyback onto --output-format, we run into option precedence
problems (like Patrick mentioned), but I'd find it more confusing that
--output-format=[porcelain|full|compact] don't behave the same way.
I don't think this puts us in a better spot with regards to option
precedence either. Consider:
git fetch --output-format=full -z <...>
The only way to respect both options is to have -z affect the
human-readable output, which isn't the end of the world, but it seems
unnecessary.
Perhaps something like --machine instead?
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-04-27 10:58 ` Patrick Steinhardt
@ 2023-04-27 23:20 ` Glen Choo
2023-04-28 8:51 ` Patrick Steinhardt
2023-05-02 20:55 ` Felipe Contreras
1 sibling, 1 reply; 120+ messages in thread
From: Glen Choo @ 2023-04-27 23:20 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: git, Jonathan Tan
Patrick Steinhardt <ps@pks.im> writes:
> On Wed, Apr 26, 2023 at 12:52:46PM -0700, Glen Choo wrote:
>> Patrick Steinhardt <ps@pks.im> writes:
>>
>> > The output format is quite simple:
>> >
>> > ```
>> > <flag> <old-object-id> <new-object-id> <local-reference>\n
>> > ```
>>
>> This format doesn't show the remote name or url that was fetched. That
>> seems okay when fetching with a single remote, but it seems necessary
>> with "--all". Perhaps you were planning to add that in a later series?
>> If so, I think it's okay to call the "porcelain" format experimental,
>> and forbid porcelain + --all until then.
>
> The reason is mostly that I didn't find an output format that I really
> liked here. We'd basically have to repeat the remote URL for every
> single reference: just repeating it once per remote doesn't fly because
> with `--parallel` the output could be intermingled. But doing that feels
> wasteful to me, so I bailed. I guess I'm also biased here because it
> just wouldn't be useful to myself.
Yeah, I couldn't think of a good way to output the remote URL either.
> So with that in mind, I'd like to continue ignoring this issue for now
> and just not report the remote that the ref came from. But I'd also
> argue that we don't have to restrict porcelain mode to single-remote
> fetches: it can still be useful to do multi-remote fetches even without
> the information where a certain reference update comes from. So any kind
> of restriction would feel artificial to me here.
>
> Furthermore, I'd argue that it is not necessary to label the format as
> experimental only because of this limitation. With the refactorings done
> in this and the preceding patch series it is easy to add a new format in
> case there indeed is somebody that would have a usecase for this. The
> "porcelain" format should stay stable, and if we decide that we want to
> also report the remote for each reference in a follow-up we can easily
> add a "porcelain-v2" or "porcelain-with-remote" format.
I agree that the current form is useful, but it seems quite likely that
users will want the remote name/url at some point. In the majority of
cases, the user can parse the remote name from the updated local ref and
if they want, they look up the url by asking "git config", but that
breaks down quite quickly if they have a custom refspec.
My suggestion to label it experimental was meant to give us the freedom
to change the "first version" in backwards-incompatible ways that we
otherwise couldn't. We could say "here's porcelain-v2, use this and
ignore porcelain-v1", but it would be even nicer if the user didn't need
to think about porcelain-v1. The idea is that we could call it
experimental, see what users need in the wild, then make changes without
having to create a new "porcelain-v2" mode.
(I realize in hindsight that calling it experimental sounds a bit
insulting. I thought your proposal was quite sensible, actually. i
apologize if it came across that way)
Perhaps part of the reason why you'd prefer not to make it experimental
is that users who might want this feature are too afraid of
backwards-incompatible changes. I think that's fair. A different way of
achieving the same flexibility would be to make the "porcelain" format
extensible (like git status --porcelain=v2). A future, multivalued
"--porcelain-options=show-remote-url,show-remote-ref" would cover
our bases.
I'm happy with either approach as long as we don't have to bikeshed
about the "perfect" porcelain output :)
>> > With these assumptions, the output format becomes unambiguously
>> > parseable. Furthermore, given that this output is designed to be
>> > consumed by scripts, the machine-readable data is printed to stdout
>> > instead of stderr like the human-readable output is. This is mostly done
>> > so that other data printed to stderr, like error messages or progress
>> > meters, don't interfere with the parseable data.
>>
>> Sending the 'main output' to stdout makes sense to me, but this (and
>> possibly respecting -z) sounds like a different mode of operation, not
>> just a matter of formats. It seems different enough that I'd prefer not
>> to piggyback on "fetch.output" for this (even though this adds more
>> surface to the interface...).
>>
>> We could add --porcelain and say that "fetch.output" is ignored if
>> --porcelain is also given. That also eliminates the need for
>> --output-format, I think.
>
> I was thinking about this initially, as well. But ultimately I decided
> against this especially because of your second paragraph: we'd now need
> to think about precedence of options and mutual exclusion, and that to
> me feels like an interface that is less obvious than a single knob that
> works as you'd expect.
My full thoughts on this are in
https://lore.kernel.org/git/kl6lildhlz3i.fsf@chooglen-macbookpro.roam.corp.google.com
but the short version is that I'm not sure if I expect something as
innocuous-looking as --output-format would imply other, machine-friendly
things (like stdout instead of stderr), and using --porcelain might make
option precedence clearer in some situtations (like if -z is given).
>> > +test_expect_success 'fetch porcelain output with HEAD and --dry-run' '
>> > + test_when_finished "rm -rf head" &&
>> > + git clone . head &&
>> > + COMMIT_ID=$(git rev-parse HEAD) &&
>> > +
>> > + git -C head fetch --output-format=porcelain --dry-run origin HEAD >actual &&
>> > + cat >expect <<-EOF &&
>> > + * $ZERO_OID $COMMIT_ID FETCH_HEAD
>> > + EOF
>> > + test_cmp expect actual &&
>> > +
>> > + git -C head fetch --output-format=porcelain --dry-run origin HEAD:foo >actual &&
>> > + cat >expect <<-EOF &&
>> > + * $ZERO_OID $COMMIT_ID refs/heads/foo
>> > + EOF
>> > + test_cmp expect actual
>> > +'
>>
>> As mentioned upthread, I think this test isn't needed because
>> "porcelain" wouldn't run into the bug we are checking for anyway.
>
> The only reason that the other bug was able to survive for so long was
> that we didn't have test coverage there. So I think it makes sense to
> explicitly test this, too, also because it causes us to walk a different
> code path.
>
> Last but not least: this test uncovered a segfault I had in a previous
> version. So I'd rather keep it :)
Ah, okay, both are fair. In that case, it probably makes sense to drop
the "--dry-run".
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-04-27 23:20 ` Glen Choo
@ 2023-04-28 8:51 ` Patrick Steinhardt
2023-04-28 17:20 ` Glen Choo
0 siblings, 1 reply; 120+ messages in thread
From: Patrick Steinhardt @ 2023-04-28 8:51 UTC (permalink / raw)
To: Glen Choo; +Cc: git, Jonathan Tan
[-- Attachment #1: Type: text/plain, Size: 6461 bytes --]
On Thu, Apr 27, 2023 at 04:20:12PM -0700, Glen Choo wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> > On Wed, Apr 26, 2023 at 12:52:46PM -0700, Glen Choo wrote:
> >> Patrick Steinhardt <ps@pks.im> writes:
[snip]
> > So with that in mind, I'd like to continue ignoring this issue for now
> > and just not report the remote that the ref came from. But I'd also
> > argue that we don't have to restrict porcelain mode to single-remote
> > fetches: it can still be useful to do multi-remote fetches even without
> > the information where a certain reference update comes from. So any kind
> > of restriction would feel artificial to me here.
> >
> > Furthermore, I'd argue that it is not necessary to label the format as
> > experimental only because of this limitation. With the refactorings done
> > in this and the preceding patch series it is easy to add a new format in
> > case there indeed is somebody that would have a usecase for this. The
> > "porcelain" format should stay stable, and if we decide that we want to
> > also report the remote for each reference in a follow-up we can easily
> > add a "porcelain-v2" or "porcelain-with-remote" format.
>
> I agree that the current form is useful, but it seems quite likely that
> users will want the remote name/url at some point. In the majority of
> cases, the user can parse the remote name from the updated local ref and
> if they want, they look up the url by asking "git config", but that
> breaks down quite quickly if they have a custom refspec.
>
> My suggestion to label it experimental was meant to give us the freedom
> to change the "first version" in backwards-incompatible ways that we
> otherwise couldn't. We could say "here's porcelain-v2, use this and
> ignore porcelain-v1", but it would be even nicer if the user didn't need
> to think about porcelain-v1. The idea is that we could call it
> experimental, see what users need in the wild, then make changes without
> having to create a new "porcelain-v2" mode.
>
> (I realize in hindsight that calling it experimental sounds a bit
> insulting. I thought your proposal was quite sensible, actually. i
> apologize if it came across that way)
No offense taken, so no need to apologize.
> Perhaps part of the reason why you'd prefer not to make it experimental
> is that users who might want this feature are too afraid of
> backwards-incompatible changes. I think that's fair.
Exactly. I'm implementing this whole feature with a concrete usecase at
GitLab in mind, and if we declare this as experimental then it will
become a whole lot more risky for us to actually adopt it. Because if
the output format may change at any point in time, then the code that
builds on top of it is likely to be broken.
> A different way of
> achieving the same flexibility would be to make the "porcelain" format
> extensible (like git status --porcelain=v2). A future, multivalued
> "--porcelain-options=show-remote-url,show-remote-ref" would cover
> our bases.
Yeah, that would indeed be quite flexible. Theoretically, there is no
reason why we couldn't have `--output-format=porcelain,show-remote-url`
though.
> I'm happy with either approach as long as we don't have to bikeshed
> about the "perfect" porcelain output :)
Agreed, and that's why I'm currently defending the "good enough" format.
It should likely work for most usecases that exist out there. The target
audience is going to be quite small here as this is not a user-directed
feature. Furthermore, I assume that the overlap of machines that want to
parse all reference updates done in a multi-remote fetch while also
being able to exactly tell which remote brought in what updates is going
to be tiny.
> >> > With these assumptions, the output format becomes unambiguously
> >> > parseable. Furthermore, given that this output is designed to be
> >> > consumed by scripts, the machine-readable data is printed to stdout
> >> > instead of stderr like the human-readable output is. This is mostly done
> >> > so that other data printed to stderr, like error messages or progress
> >> > meters, don't interfere with the parseable data.
> >>
> >> Sending the 'main output' to stdout makes sense to me, but this (and
> >> possibly respecting -z) sounds like a different mode of operation, not
> >> just a matter of formats. It seems different enough that I'd prefer not
> >> to piggyback on "fetch.output" for this (even though this adds more
> >> surface to the interface...).
> >>
> >> We could add --porcelain and say that "fetch.output" is ignored if
> >> --porcelain is also given. That also eliminates the need for
> >> --output-format, I think.
> >
> > I was thinking about this initially, as well. But ultimately I decided
> > against this especially because of your second paragraph: we'd now need
> > to think about precedence of options and mutual exclusion, and that to
> > me feels like an interface that is less obvious than a single knob that
> > works as you'd expect.
>
> My full thoughts on this are in
>
> https://lore.kernel.org/git/kl6lildhlz3i.fsf@chooglen-macbookpro.roam.corp.google.com
>
> but the short version is that I'm not sure if I expect something as
> innocuous-looking as --output-format would imply other, machine-friendly
> things (like stdout instead of stderr), and using --porcelain might make
> option precedence clearer in some situtations (like if -z is given).
I'm not even sure that `-z` makes sense in this context. If we see cases
where not using `-z` can cause the machine-readable interface to become
unparseable then this is a bug in the output format, if you ask me.
Mostly because the whole intent of it is to be machine-parseable. So if
we output data that can e.g. contain newlines, then we must not use
newlines as part of the output format or alternatively escape them. Why
let the author of the script shoot themselves into the foot?
Anyway, I'm digressing. It's hard for me to decide what to do right now.
The thread with Junio and Jacob points into the direction of keeping the
`--output-format=` interface, while this thread points into the other
direction. I'm naturally more inclined to keep `--output-format=`,
mostly because I personally feel like it's the more obvious interface.
But I also see your point, so it's not really a choice of right-or-wrong
here, but rather of style.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-04-28 8:51 ` Patrick Steinhardt
@ 2023-04-28 17:20 ` Glen Choo
0 siblings, 0 replies; 120+ messages in thread
From: Glen Choo @ 2023-04-28 17:20 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: git, Jonathan Tan
Patrick Steinhardt <ps@pks.im> writes:
>> I'm happy with either approach as long as we don't have to bikeshed
>> about the "perfect" porcelain output :)
>
> Agreed, and that's why I'm currently defending the "good enough" format.
> It should likely work for most usecases that exist out there. The target
> audience is going to be quite small here as this is not a user-directed
> feature.
Okay, I agree that this is "good enough" for most cases and we can
extend it if needed.
>> My full thoughts on this are in
>>
>> https://lore.kernel.org/git/kl6lildhlz3i.fsf@chooglen-macbookpro.roam.corp.google.com
>>
>> but the short version is that I'm not sure if I expect something as
>> innocuous-looking as --output-format would imply other, machine-friendly
>> things (like stdout instead of stderr), and using --porcelain might make
>> option precedence clearer in some situtations (like if -z is given).
>
> I'm not even sure that `-z` makes sense in this context. If we see cases
> where not using `-z` can cause the machine-readable interface to become
> unparseable then this is a bug in the output format, if you ask me.
> Mostly because the whole intent of it is to be machine-parseable. So if
> we output data that can e.g. contain newlines, then we must not use
> newlines as part of the output format or alternatively escape them. Why
> let the author of the script shoot themselves into the foot?
Agreed. I think the current output format is resilient enough, just that
"-z" is something we choose to support for plumbing anyway.
> Anyway, I'm digressing. It's hard for me to decide what to do right now.
> The thread with Junio and Jacob points into the direction of keeping the
> `--output-format=` interface, while this thread points into the other
> direction. I'm naturally more inclined to keep `--output-format=`,
> mostly because I personally feel like it's the more obvious interface.
> But I also see your point, so it's not really a choice of right-or-wrong
> here, but rather of style.
Yes. All sides seem to understand the tradeoffs, but we value them
differently.
I'd personally prefer to err on the side of consistency, because even if
the existing behavior is less-than-ideal since:
1) This makes it possible for us to fix it in a consistent, well-thought
out way, so we don't have to decide the future for the whole project
in a git-fetch series.
2) At least it still behaves how _some_ users have come to expect it.
As for which is more consistent, a grep for --porcelain over
Documentation/* shows:
- git-push
- git-status
- git-worktree
- git-blame
- git-commit
Most of them already output to stdout. The lone exception is git-push,
which does exactly this "use stdout for the main output, but keep using
stderr for debugging output" behavior you added, so using --porcelain
seems somewhat consistent.
On the other hand, I couldn't find any other option that switches from
stderr to stdout (we sometimes say --stdout to say "use stdout instead
of a file", but that's different), so if we added this behavior to "git
fetch --[output-]format", this might be the first.
(Sidenote: most instances of --format are for templated-style format,
but "git-replace" does accept an enum for it. I don't think this should
weigh heavily in our decision though.)
For the other, non git-push commands, --porcelain is treated like an
output format and respects "last one wins" regular option precedence,
e.g. in the case of "git status --porcelain --short". I find this
behavior somewhat confusing (I'd expect --porcelain to win, or at least
be incompatible), but at least we can consistently change this in the
future [*].
That said, I'm willing to accept that I'm biased toward my own ideas. If
others don't find "--format=porcelain implies stdout" confusing, I can
accept that decision.
[*] In all likelihood, we probably can't make the change to --porcelain
for backwards-compatibility reasons, but we could introduce a synonym
with the behavior we want ("--machine"?) and have OPT_PARSE_MACHINE
handle both --porcelain and --machine.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v2 7/8] fetch: introduce new `--output-format` option
2023-04-27 22:01 ` Junio C Hamano
@ 2023-04-28 22:03 ` Glen Choo
2023-05-03 9:12 ` Patrick Steinhardt
0 siblings, 1 reply; 120+ messages in thread
From: Glen Choo @ 2023-04-28 22:03 UTC (permalink / raw)
To: Junio C Hamano, Patrick Steinhardt
Cc: git, Felipe Contreras, Jonathan Tan, Jacob Keller
Junio C Hamano <gitster@pobox.com> writes:
> I wonder if
>
> static const char *const display_format[] = {
> [DISPLAY_FORMAT_UNKNOWN] = NULL,
> [DISPLAY_FORMAT_FULL] = "full",
> [DISPLAY_FORMAT_COMPACT] = "compact",
> };
>
> would be easier to maintain?
It's easier to read, so I'd think so.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v2 7/8] fetch: introduce new `--output-format` option
2023-04-27 11:13 ` [PATCH v2 7/8] fetch: introduce new `--output-format` option Patrick Steinhardt
2023-04-27 22:01 ` Junio C Hamano
@ 2023-04-28 22:31 ` Glen Choo
2023-05-03 9:43 ` Patrick Steinhardt
1 sibling, 1 reply; 120+ messages in thread
From: Glen Choo @ 2023-04-28 22:31 UTC (permalink / raw)
To: Patrick Steinhardt, git
Cc: Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> @@ -1894,6 +1902,9 @@ static int fetch_multiple(struct string_list *list, int max_children)
> "--no-write-commit-graph", NULL);
> add_options_to_argv(&argv);
>
> + if (format != DISPLAY_FORMAT_UNKNOWN)
> + strvec_pushf(&argv, "--output-format=%s", display_formats[format]);
> +
I think these lines belong inside add_options_to_argv(), since that's
also used to prepare argv for fetch_submodules(), so we'd also get
support for --recurse-submodules. (I wish I had spotted that in v1,
sorry. Thankfully they use the same helper function, so we only have to
do this once.)
----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 422e29a914..7aa385aed5 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1796,8 +1796,11 @@ static int add_remote_or_group(const char *name, struct string_list *list)
return 1;
}
-static void add_options_to_argv(struct strvec *argv)
+static void add_options_to_argv(struct strvec *argv,
+ enum display_format format)
{
/* Maybe this shouldn't be first, idk */
+ if (format != DISPLAY_FORMAT_UNKNOWN)
+ strvec_pushf(argv, "--output-format=%s", display_formats[format]);
if (dry_run)
strvec_push(argv, "--dry-run");
if (prune != -1)
@@ -1908,10 +1911,7 @@ static int fetch_multiple(struct string_list *list, int max_children,
strvec_pushl(&argv, "-c", "fetch.bundleURI=",
"fetch", "--append", "--no-auto-gc",
"--no-write-commit-graph", NULL);
- add_options_to_argv(&argv);
-
- if (format != DISPLAY_FORMAT_UNKNOWN)
- strvec_pushf(&argv, "--output-format=%s", display_formats[format]);
+ add_options_to_argv(&argv, format);
if (max_children != 1 && list->nr != 1) {
struct parallel_fetch_state state = { argv.v, list, 0, 0 };
@@ -2403,7 +2403,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
if (max_children < 0)
max_children = fetch_parallel_config;
- add_options_to_argv(&options);
+ add_options_to_argv(&options, display_format);
result = fetch_submodules(the_repository,
&options,
submodule_prefix,
----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----
I tested the result of that locally with --recurse-submodules, and
it works.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v2 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-04-27 11:13 ` [PATCH v2 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
2023-04-27 19:52 ` Jacob Keller
@ 2023-04-28 22:42 ` Glen Choo
1 sibling, 0 replies; 120+ messages in thread
From: Glen Choo @ 2023-04-28 22:42 UTC (permalink / raw)
To: Patrick Steinhardt, git
Cc: Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> A notable ommission here is that the output format does not include the
> remote from which a reference was fetched, which might be important
> information especially in the context of multi-remote fetches. But as
> such a format would require us to print the remote for every single
> reference update due to parallelizable fetches it feels wasteful for the
> most likely usecase, which is when fetching from a single remote. If
> usecases come up for this in the future though it is easy enough to add
> a new "porcelain-v2" format that adds this information.
We discussed this elsewhere in the thread, but if we are just adding
information (and not omitting existing information or shuffling it
around), I would prefer for us to make the format extensible using flags
than to add a whole new format enum. We can't imagine what other
information users might want (maybe the remote ref name?), and it would
be nice to avoid bumping the 'porcelain version' unnecessarily.
I agree that this format is good enough as a starting point, though.
It's not new to v2, but I've mentioned my reservations on
--output-format on:
https://lore.kernel.org/git/kl6ledo33ovx.fsf@chooglen-macbookpro.roam.corp.google.com/
I will be out of office all of next week. If, in the meantime, everyone
else decides that --output-format=porcelain is good enough, I'm happy to
accept the result.
> @@ -127,6 +193,24 @@ test_expect_success 'fetch output with HEAD and --dry-run' '
> test_cmp expect actual
> '
>
> +test_expect_success 'fetch porcelain output with HEAD and --dry-run' '
> + test_when_finished "rm -rf head" &&
> + git clone . head &&
> + COMMIT_ID=$(git rev-parse HEAD) &&
> +
> + git -C head fetch --output-format=porcelain --dry-run origin HEAD >actual &&
> + cat >expect <<-EOF &&
> + * $ZERO_OID $COMMIT_ID FETCH_HEAD
> + EOF
> + test_cmp expect actual &&
> +
> + git -C head fetch --output-format=porcelain --dry-run origin HEAD:foo >actual &&
> + cat >expect <<-EOF &&
> + * $ZERO_OID $COMMIT_ID refs/heads/foo
> + EOF
> + test_cmp expect actual
> +'
> +
Now that the earlier test also tests without --dry-run, shouldn't this
one too?
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v2 1/8] fetch: split out tests for output format
2023-04-27 11:13 ` [PATCH v2 1/8] fetch: split out tests for output format Patrick Steinhardt
@ 2023-04-29 17:34 ` SZEDER Gábor
2023-05-03 11:21 ` Patrick Steinhardt
0 siblings, 1 reply; 120+ messages in thread
From: SZEDER Gábor @ 2023-04-29 17:34 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
On Thu, Apr 27, 2023 at 01:13:08PM +0200, Patrick Steinhardt wrote:
> We're about to introduce a new porcelain mode for the output of
> git-fetch(1). As part of that we'll be introducing a set of new tests
> that only relate to the output of this command.
>
> Split out tests that exercise the output format of git-fetch(1) so that
> it becomes easier to verify this functionality as a standalone unit. As
> the tests assume that the default branch is called "main" we set up the
> corresponding GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME environment variable
> accordingly.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
> t/t5510-fetch.sh | 53 ----------------------------------
> t/t5574-fetch-output.sh | 63 +++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 63 insertions(+), 53 deletions(-)
> create mode 100755 t/t5574-fetch-output.sh
> diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
> new file mode 100755
> index 0000000000..f91b654d38
> --- /dev/null
> +++ b/t/t5574-fetch-output.sh
> @@ -0,0 +1,63 @@
> +#!/bin/sh
> +
> +test_description='git fetch output format'
> +
> +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
> +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
> +
> +. ./test-lib.sh
> +
> +test_expect_success 'fetch aligned output' '
> + git clone . full-output &&
> + test_commit looooooooooooong-tag &&
> + (
> + cd full-output &&
> + git -c fetch.output=full fetch origin >actual 2>&1 &&
Why is that 2>&1 redirection used here?
If the output the test case is interested in goes to the command's
standard output, then it's unnecessary. However, if it goes to
standard error, then why is standard output redirected as well?
I understand that this patch just moves existing test cases around
as-is, so this is not something you introduced, but I point it out
here, because later patches of this series add several new test cases
following this anti-pattern.
Since this series creates a new test script, perhaps this might be the
right time to clean this up.
> + grep -e "->" actual | cut -c 22- >../actual
> + ) &&
> + cat >expect <<-\EOF &&
> + main -> origin/main
> + looooooooooooong-tag -> looooooooooooong-tag
> + EOF
> + test_cmp expect actual
> +'
> +
> +test_expect_success 'fetch compact output' '
> + git clone . compact &&
> + test_commit extraaa &&
> + (
> + cd compact &&
> + git -c fetch.output=compact fetch origin >actual 2>&1 &&
> + grep -e "->" actual | cut -c 22- >../actual
> + ) &&
> + cat >expect <<-\EOF &&
> + main -> origin/*
> + extraaa -> *
> + EOF
> + test_cmp expect actual
> +'
> +
> +test_expect_success '--no-show-forced-updates' '
> + mkdir forced-updates &&
> + (
> + cd forced-updates &&
> + git init &&
> + test_commit 1 &&
> + test_commit 2
> + ) &&
> + git clone forced-updates forced-update-clone &&
> + git clone forced-updates no-forced-update-clone &&
> + git -C forced-updates reset --hard HEAD~1 &&
> + (
> + cd forced-update-clone &&
> + git fetch --show-forced-updates origin 2>output &&
Oh, look, there are some good examples to follow here as well :)
> + test_i18ngrep "(forced update)" output
> + ) &&
> + (
> + cd no-forced-update-clone &&
> + git fetch --no-show-forced-updates origin 2>output &&
> + test_i18ngrep ! "(forced update)" output
> + )
> +'
> +
> +test_done
> --
> 2.40.1
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-04-27 10:58 ` Patrick Steinhardt
2023-04-27 23:20 ` Glen Choo
@ 2023-05-02 20:55 ` Felipe Contreras
1 sibling, 0 replies; 120+ messages in thread
From: Felipe Contreras @ 2023-05-02 20:55 UTC (permalink / raw)
To: Patrick Steinhardt, Glen Choo; +Cc: git, Jonathan Tan
Patrick Steinhardt wrote:
> On Wed, Apr 26, 2023 at 12:52:46PM -0700, Glen Choo wrote:
> > Patrick Steinhardt <ps@pks.im> writes:
> >
> > > The output format is quite simple:
> > >
> > > ```
> > > <flag> <old-object-id> <new-object-id> <local-reference>\n
> > > ```
> >
> > This format doesn't show the remote name or url that was fetched. That
> > seems okay when fetching with a single remote, but it seems necessary
> > with "--all". Perhaps you were planning to add that in a later series?
> > If so, I think it's okay to call the "porcelain" format experimental,
> > and forbid porcelain + --all until then.
>
> The reason is mostly that I didn't find an output format that I really
> liked here. We'd basically have to repeat the remote URL for every
> single reference: just repeating it once per remote doesn't fly because
> with `--parallel` the output could be intermingled. But doing that feels
> wasteful to me, so I bailed. I guess I'm also biased here because it
> just wouldn't be useful to myself.
Couldn't each URL be assigned an index and then just show `0` in the output? I
think it should be straightforward for a parser to figure out what `0` means
but if not, that's something that could be figured out once the use case
arrives (which very well could be never).
--
Felipe Contreras
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v2 7/8] fetch: introduce new `--output-format` option
2023-04-28 22:03 ` Glen Choo
@ 2023-05-03 9:12 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-03 9:12 UTC (permalink / raw)
To: Glen Choo
Cc: Junio C Hamano, git, Felipe Contreras, Jonathan Tan, Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 453 bytes --]
On Fri, Apr 28, 2023 at 03:03:43PM -0700, Glen Choo wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
> > I wonder if
> >
> > static const char *const display_format[] = {
> > [DISPLAY_FORMAT_UNKNOWN] = NULL,
> > [DISPLAY_FORMAT_FULL] = "full",
> > [DISPLAY_FORMAT_COMPACT] = "compact",
> > };
> >
> > would be easier to maintain?
>
> It's easier to read, so I'd think so.
Yeah, I'll adopt this approach in v3.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v2 7/8] fetch: introduce new `--output-format` option
2023-04-28 22:31 ` Glen Choo
@ 2023-05-03 9:43 ` Patrick Steinhardt
2023-05-03 11:36 ` Patrick Steinhardt
0 siblings, 1 reply; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-03 9:43 UTC (permalink / raw)
To: Glen Choo
Cc: git, Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 3904 bytes --]
On Fri, Apr 28, 2023 at 03:31:08PM -0700, Glen Choo wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
> > @@ -1894,6 +1902,9 @@ static int fetch_multiple(struct string_list *list, int max_children)
> > "--no-write-commit-graph", NULL);
> > add_options_to_argv(&argv);
> >
> > + if (format != DISPLAY_FORMAT_UNKNOWN)
> > + strvec_pushf(&argv, "--output-format=%s", display_formats[format]);
> > +
>
> I think these lines belong inside add_options_to_argv(), since that's
> also used to prepare argv for fetch_submodules(), so we'd also get
> support for --recurse-submodules. (I wish I had spotted that in v1,
> sorry. Thankfully they use the same helper function, so we only have to
> do this once.)
>
> ----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> index 422e29a914..7aa385aed5 100644
> --- a/builtin/fetch.c
> +++ b/builtin/fetch.c
> @@ -1796,8 +1796,11 @@ static int add_remote_or_group(const char *name, struct string_list *list)
> return 1;
> }
>
> -static void add_options_to_argv(struct strvec *argv)
> +static void add_options_to_argv(struct strvec *argv,
> + enum display_format format)
> {
> /* Maybe this shouldn't be first, idk */
> + if (format != DISPLAY_FORMAT_UNKNOWN)
> + strvec_pushf(argv, "--output-format=%s", display_formats[format]);
> if (dry_run)
> strvec_push(argv, "--dry-run");
> if (prune != -1)
> @@ -1908,10 +1911,7 @@ static int fetch_multiple(struct string_list *list, int max_children,
> strvec_pushl(&argv, "-c", "fetch.bundleURI=",
> "fetch", "--append", "--no-auto-gc",
> "--no-write-commit-graph", NULL);
> - add_options_to_argv(&argv);
> -
> - if (format != DISPLAY_FORMAT_UNKNOWN)
> - strvec_pushf(&argv, "--output-format=%s", display_formats[format]);
> + add_options_to_argv(&argv, format);
>
> if (max_children != 1 && list->nr != 1) {
> struct parallel_fetch_state state = { argv.v, list, 0, 0 };
> @@ -2403,7 +2403,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
> if (max_children < 0)
> max_children = fetch_parallel_config;
>
> - add_options_to_argv(&options);
> + add_options_to_argv(&options, display_format);
> result = fetch_submodules(the_repository,
> &options,
> submodule_prefix,
>
> ----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----
>
> I tested the result of that locally with --recurse-submodules, and
> it works.
Unfortunately it doesn't quite work alright: while the porcelain format
does indeed get inherited to the child process correctly, the parallel
process API will cause us to group output per submodule-fetch. This has
the consequence that stdout will be redirected into stderr, and that
then breaks the assumption that all machine-parseable output goes to
stdout.
My initial reflex is to just outright reject porcelain mode when
submodule fetches are enabled. But that would require the caller to
always explicitly pass `--recurse-submodules=off`, which isn't exactly
great usability-wise.
The alternative would be to ungroup the output so that we can continue
to print to the correct output streams. That works alright, and I've got
a working version that does exactly that. But now we have the issue that
the porcelain output is misleading: you cannot tell whether a specific
reference update happens in the parent repository or in the submodule as
that information is not part of the output.
I consider the second option to be much worse than the first option
because it can cause scripts do to the wrong thing. So I'll send v3 with
the first option, even though it's kind of an awful workaround. I'd be
happy to hear any alternative proposals though.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v2 1/8] fetch: split out tests for output format
2023-04-29 17:34 ` SZEDER Gábor
@ 2023-05-03 11:21 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-03 11:21 UTC (permalink / raw)
To: SZEDER Gábor
Cc: git, Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 2746 bytes --]
On Sat, Apr 29, 2023 at 07:34:48PM +0200, SZEDER Gábor wrote:
> On Thu, Apr 27, 2023 at 01:13:08PM +0200, Patrick Steinhardt wrote:
> > We're about to introduce a new porcelain mode for the output of
> > git-fetch(1). As part of that we'll be introducing a set of new tests
> > that only relate to the output of this command.
> >
> > Split out tests that exercise the output format of git-fetch(1) so that
> > it becomes easier to verify this functionality as a standalone unit. As
> > the tests assume that the default branch is called "main" we set up the
> > corresponding GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME environment variable
> > accordingly.
> >
> > Signed-off-by: Patrick Steinhardt <ps@pks.im>
> > ---
> > t/t5510-fetch.sh | 53 ----------------------------------
> > t/t5574-fetch-output.sh | 63 +++++++++++++++++++++++++++++++++++++++++
> > 2 files changed, 63 insertions(+), 53 deletions(-)
> > create mode 100755 t/t5574-fetch-output.sh
>
>
> > diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
> > new file mode 100755
> > index 0000000000..f91b654d38
> > --- /dev/null
> > +++ b/t/t5574-fetch-output.sh
> > @@ -0,0 +1,63 @@
> > +#!/bin/sh
> > +
> > +test_description='git fetch output format'
> > +
> > +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
> > +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
> > +
> > +. ./test-lib.sh
> > +
> > +test_expect_success 'fetch aligned output' '
> > + git clone . full-output &&
> > + test_commit looooooooooooong-tag &&
> > + (
> > + cd full-output &&
> > + git -c fetch.output=full fetch origin >actual 2>&1 &&
>
> Why is that 2>&1 redirection used here?
> If the output the test case is interested in goes to the command's
> standard output, then it's unnecessary. However, if it goes to
> standard error, then why is standard output redirected as well?
>
> I understand that this patch just moves existing test cases around
> as-is, so this is not something you introduced, but I point it out
> here, because later patches of this series add several new test cases
> following this anti-pattern.
>
> Since this series creates a new test script, perhaps this might be the
> right time to clean this up.
I feel like this patch series is big enough on its own already, so for
now I'd like to refrain from touching up the old tests. But you've got a
good point here, and I'll make sure that any new tests don't followw the
same pattern.
That being said, I think we should keep on testing both stdout and
stderr. The tests here are explicitly about the output format, so it
does make sense to verify that both of them match what we expect. We
should treat them as separate things though.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH v3 0/8] fetch: introduce machine-parseable output
2023-04-19 12:31 [PATCH 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (11 preceding siblings ...)
2023-04-27 11:13 ` [PATCH v2 " Patrick Steinhardt
@ 2023-05-03 11:34 ` Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches Patrick Steinhardt
` (9 more replies)
2023-05-09 13:01 ` [PATCH v4 " Patrick Steinhardt
2023-05-10 12:33 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Patrick Steinhardt
14 siblings, 10 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-03 11:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 22706 bytes --]
Hi,
this is the third version of my patch series to introduce a
machine-parseable output for git-fetch(1).
Changes compared to v2:
- A new commit that fixes `--no-recurse-submodules` not being
honored for multi-remote fetches was added. This fixes issues that
would otherwise be hit later in the patch series.
- Tests were adjusted to explicitly verify stdout and stderr as
standalone units instead of redirecting 2>&1.
- Instead of introducing `--output-format` and a new "porcelain"
format I've changed this to instead introduce `--porcelain`. What
finally convinced me to go this way is Glen's list of preexisting
commands that have `--porcelain`. So I decided to not be the only
outlier here.
- The DISPLAY_FORMAT_MAX macro was dropped in favor of Junio's
proposal that uses designated initializers for the array.
- Similar to `--negotiate-only`, the porcelain output is now
incompatible with `--recurse-submodules=[yes|on-demand]`. This is
done to avoid ambiguity when references in both the superproject
and any of its submodules is updated.
Thanks again for all the feedback!
Patrick
Patrick Steinhardt (8):
fetch: fix `--no-recurse-submodules` with multi-remote fetches
fetch: split out tests for output format
fetch: add a test to exercise invalid output formats
fetch: fix missing from-reference when fetching HEAD:foo
fetch: introduce `display_format` enum
fetch: move display format parsing into main function
fetch: move option related variables into main function
fetch: introduce machine-parseable "porcelain" output format
Documentation/fetch-options.txt | 6 +
Documentation/git-fetch.txt | 9 +
builtin/fetch.c | 437 +++++++++++++++++++-------------
t/t5510-fetch.sh | 53 ----
t/t5526-fetch-submodules.sh | 31 +++
t/t5574-fetch-output.sh | 231 +++++++++++++++++
6 files changed, 543 insertions(+), 224 deletions(-)
create mode 100755 t/t5574-fetch-output.sh
Range-diff against v2:
-: ---------- > 1: 4b2b0cfe15 fetch: fix `--no-recurse-submodules` with multi-remote fetches
1: 0d0d50d14c = 2: 6ebc7450ba fetch: split out tests for output format
2: 29d2c58914 ! 3: 78479922ac fetch: add a test to exercise invalid output formats
@@ t/t5574-fetch-output.sh: export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+ test_when_finished "rm -rf clone" &&
+ git clone . clone &&
+
-+ test_must_fail git -C clone -c fetch.output= fetch origin >actual 2>&1 &&
++ test_must_fail git -C clone -c fetch.output= fetch origin >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}${SQ}
+ EOF
-+ test_cmp expect actual &&
++ test_must_be_empty actual.out &&
++ test_cmp expect actual.err &&
+
-+ test_must_fail git -C clone -c fetch.output=garbage fetch origin >actual 2>&1 &&
++ test_must_fail git -C clone -c fetch.output=garbage fetch origin >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}garbage${SQ}
+ EOF
-+ test_cmp expect actual
++ test_must_be_empty actual.out &&
++ test_cmp expect actual.err
+'
+
test_expect_success 'fetch aligned output' '
3: d1fb6eeae7 ! 4: 46e1266ab0 fetch: fix missing from-reference when fetching HEAD:foo
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
test_cmp expect actual
'
-+test_expect_success 'fetch output with HEAD and --dry-run' '
++test_expect_success 'fetch output with HEAD' '
+ test_when_finished "rm -rf head" &&
+ git clone . head &&
+
-+ git -C head fetch --dry-run origin HEAD >actual 2>&1 &&
++ git -C head fetch --dry-run origin HEAD >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * branch HEAD -> FETCH_HEAD
+ EOF
-+ test_cmp expect actual &&
++ test_must_be_empty actual.out &&
++ test_cmp expect actual.err &&
+
-+ git -C head fetch origin HEAD >actual 2>&1 &&
-+ test_cmp expect actual &&
++ git -C head fetch origin HEAD >actual.out 2>actual.err &&
++ test_must_be_empty actual.out &&
++ test_cmp expect actual.err &&
+
-+ git -C head fetch --dry-run origin HEAD:foo >actual 2>&1 &&
++ git -C head fetch --dry-run origin HEAD:foo >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * [new ref] HEAD -> foo
+ EOF
-+ test_cmp expect actual &&
++ test_must_be_empty actual.out &&
++ test_cmp expect actual.err &&
+
-+ git -C head fetch origin HEAD:foo >actual 2>&1 &&
-+ test_cmp expect actual
++ git -C head fetch origin HEAD:foo >actual.out 2>actual.err &&
++ test_must_be_empty actual.out &&
++ test_cmp expect actual.err
+'
+
test_expect_success '--no-show-forced-updates' '
4: b545bf8bb9 ! 5: acc0f7f520 fetch: introduce `display_format` enum
@@ builtin/fetch.c: static void display_state_init(struct display_state *display_st
+ break;
+ }
+ default:
-+ BUG("unexpected display foramt %d", display_state->format);
++ BUG("unexpected display format %d", display_state->format);
}
}
5: 4990d35998 = 6: cd23440128 fetch: move display format parsing into main function
6: cfe84129ab = 7: edbc31013f fetch: move option related variables into main function
7: 0335e5eeb4 < -: ---------- fetch: introduce new `--output-format` option
8: d7c1bc1a80 ! 8: e132d9494e fetch: introduce machine-parseable "porcelain" output format
@@ Commit message
information especially in the context of multi-remote fetches. But as
such a format would require us to print the remote for every single
reference update due to parallelizable fetches it feels wasteful for the
- most likely usecase, which is when fetching from a single remote. If
- usecases come up for this in the future though it is easy enough to add
- a new "porcelain-v2" format that adds this information.
+ most likely usecase, which is when fetching from a single remote.
+
+ In a similar spirit, a second restriction is that this cannot be used
+ with `--recurse-submodules`. This is because any reference updates would
+ be ambiguous without also printing the repository in which the update
+ happens.
+
+ Considering that both multi-remote and submodule fetches are rather
+ niche and likely not going to be useful for the majority of usecases
+ these omissions feel acceptable. If usecases for either of these come up
+ in the future though it is easy enough to add a new "porcelain-v2"
+ format that adds this information.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
- ## Documentation/config/fetch.txt ##
-@@ Documentation/config/fetch.txt: fetch.pruneTags::
-
- fetch.output::
- Control how ref update status is printed. Valid values are
-- `full` and `compact`. Default value is `full`. See section
-- OUTPUT in linkgit:git-fetch[1] for detail.
-+ `full`, `compact` and `porcelain`. Default value is `full`.
-+ See section OUTPUT in linkgit:git-fetch[1] for detail.
-
- fetch.negotiationAlgorithm::
- Control how information about the commits in the local repository
-
## Documentation/fetch-options.txt ##
@@ Documentation/fetch-options.txt: linkgit:git-config[1].
+ --dry-run::
+ Show what would be done, without making any changes.
- --output-format::
- Control how ref update status is printed. Valid values are
-- `full` and `compact`. Default value is `full`. See section
-- OUTPUT in linkgit:git-fetch[1] for detail.
-+ `full`, `compact` and `porcelain`. Default value is `full`.
-+ See section OUTPUT in linkgit:git-fetch[1] for detail.
-
++--porcelain::
++ Print the output to standard output in an easy-to-parse format for
++ scripts. See section OUTPUT in linkgit:git-fetch[1] for details.
+++
++This is incompatible with `--recurse-submodules=[yes|on-demand]`.
++
ifndef::git-pull[]
--[no-]write-fetch-head::
+ Write the list of remote refs fetched in the `FETCH_HEAD`
## Documentation/git-fetch.txt ##
-@@ Documentation/git-fetch.txt: The output of "git fetch" depends on the transport method used; this
- section describes the output when fetching over the Git protocol
- (either locally or via ssh) and Smart HTTP protocol.
-
--The status of the fetch is output in tabular form, with each line
--representing the status of a single ref. Each line is of the form:
-+The output format can be chosen either via the `fetch.output` config
-+(see linkgit:git-config[1]), or via the `--output-format` switch.
-+Supported values include:
-+
-+For the `full` and `compact` output formats, the status of the fetch is
-+output in tabular, with each line representing the status of a single
-+ref. Each line is of the form:
-
- -------------------------------
+@@ Documentation/git-fetch.txt: representing the status of a single ref. Each line is of the form:
<flag> <summary> <from> -> <to> [<reason>]
-------------------------------
-+The `porcelain` output format is intended to be machine-parseable. In
-+contrast to the human-readable output formats it thus prints to standard
-+output instead of standard error. Each line is of the form:
++When using `--porcelain`, the output format is intended to be
++machine-parseable. In contrast to the human-readable output formats it
++thus prints to standard output instead of standard error. Each line is
++of the form:
+
+-------------------------------
+<flag> <old-object-id> <new-object-id> <local-reference>
@@ builtin/fetch.c: enum display_format {
DISPLAY_FORMAT_FULL,
DISPLAY_FORMAT_COMPACT,
+ DISPLAY_FORMAT_PORCELAIN,
- DISPLAY_FORMAT_MAX,
- };
-
-@@ builtin/fetch.c: static const char * const display_formats[DISPLAY_FORMAT_MAX] = {
- NULL,
- "full",
- "compact",
-+ "porcelain",
};
struct display_state {
@@ builtin/fetch.c: static void display_state_init(struct display_state *display_st
+ /* We don't need to precompute anything here. */
+ break;
default:
-- BUG("unexpected display foramt %d", display_state->format);
-+ BUG("unexpected display format %d", display_state->format);
+ BUG("unexpected display format %d", display_state->format);
}
- }
-
@@ builtin/fetch.c: static void print_compact(struct display_state *display_state,
static void display_ref_update(struct display_state *display_state, char code,
const char *summary, const char *error,
@@ builtin/fetch.c: static int prune_refs(struct display_state *display_state,
summary_width);
warn_dangling_symref(stderr, dangling_msg, ref->name);
}
+@@ builtin/fetch.c: static int add_remote_or_group(const char *name, struct string_list *list)
+ return 1;
+ }
+
+-static void add_options_to_argv(struct strvec *argv)
++static void add_options_to_argv(struct strvec *argv,
++ enum display_format format)
+ {
+ if (dry_run)
+ strvec_push(argv, "--dry-run");
+@@ builtin/fetch.c: static void add_options_to_argv(struct strvec *argv)
+ strvec_push(argv, "--ipv6");
+ if (!write_fetch_head)
+ strvec_push(argv, "--no-write-fetch-head");
++ if (format == DISPLAY_FORMAT_PORCELAIN)
++ strvec_pushf(argv, "--porcelain");
+ }
+
+ /* Fetch multiple remotes in parallel */
+@@ builtin/fetch.c: struct parallel_fetch_state {
+ const char **argv;
+ struct string_list *remotes;
+ int next, result;
++ enum display_format format;
+ };
+
+ static int fetch_next_remote(struct child_process *cp,
+@@ builtin/fetch.c: static int fetch_next_remote(struct child_process *cp,
+ strvec_push(&cp->args, remote);
+ cp->git_cmd = 1;
+
+- if (verbosity >= 0)
++ if (verbosity >= 0 && state->format != DISPLAY_FORMAT_PORCELAIN)
+ printf(_("Fetching %s\n"), remote);
+
+ return 1;
+@@ builtin/fetch.c: static int fetch_finished(int result, struct strbuf *out,
+ return 0;
+ }
+
+-static int fetch_multiple(struct string_list *list, int max_children)
++static int fetch_multiple(struct string_list *list, int max_children,
++ enum display_format format)
+ {
+ int i, result = 0;
+ struct strvec argv = STRVEC_INIT;
+@@ builtin/fetch.c: static int fetch_multiple(struct string_list *list, int max_children)
+
+ strvec_pushl(&argv, "fetch", "--append", "--no-auto-gc",
+ "--no-write-commit-graph", NULL);
+- add_options_to_argv(&argv);
++ add_options_to_argv(&argv, format);
+
+ if (max_children != 1 && list->nr != 1) {
+- struct parallel_fetch_state state = { argv.v, list, 0, 0 };
++ struct parallel_fetch_state state = { argv.v, list, 0, 0, format };
+ const struct run_process_parallel_opts opts = {
+ .tr2_category = "fetch",
+ .tr2_label = "parallel/fetch",
+@@ builtin/fetch.c: static int fetch_multiple(struct string_list *list, int max_children)
+
+ strvec_pushv(&cmd.args, argv.v);
+ strvec_push(&cmd.args, name);
+- if (verbosity >= 0)
++ if (verbosity >= 0 && format != DISPLAY_FORMAT_PORCELAIN)
+ printf(_("Fetching %s\n"), name);
+ cmd.git_cmd = 1;
+ if (run_command(&cmd)) {
+@@ builtin/fetch.c: static int fetch_one(struct remote *remote, int argc, const char **argv,
+ return exit_code;
+ }
+
++static int opt_parse_porcelain(const struct option *opt, const char *arg, int unset)
++{
++ enum display_format *format = opt->value;
++ *format = DISPLAY_FORMAT_PORCELAIN;
++ return 0;
++}
++
+ int cmd_fetch(int argc, const char **argv, const char *prefix)
+ {
+ const char *bundle_uri;
+@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
+ OPT_BOOL(0, "dry-run", &dry_run,
+ N_("dry run")),
++ OPT_CALLBACK_F(0, "porcelain", &display_format, NULL, N_("machine-readable output"),
++ PARSE_OPT_NOARG|PARSE_OPT_NONEG, opt_parse_porcelain),
+ OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
+ N_("write fetched references to the FETCH_HEAD file")),
+ OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
+@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ fetch_config_from_gitmodules(sfjc, rs);
+ }
+
++ if (display_format == DISPLAY_FORMAT_PORCELAIN) {
++ switch (recurse_submodules_cli) {
++ case RECURSE_SUBMODULES_OFF:
++ case RECURSE_SUBMODULES_DEFAULT:
++ /*
++ * Reference updates in submodules would be ambiguous
++ * in porcelain mode, so we reject this combination.
++ */
++ recurse_submodules = RECURSE_SUBMODULES_OFF;
++ break;
++
++ default:
++ die(_("options '%s' and '%s' cannot be used together"),
++ "--porcelain", "--recurse-submodules");
++ }
++ }
++
+ if (negotiate_only && !negotiation_tip.nr)
+ die(_("--negotiate-only needs one or more --negotiation-tip=*"));
+
+@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ max_children = fetch_parallel_config;
+
+ /* TODO should this also die if we have a previous partial-clone? */
+- result = fetch_multiple(&list, max_children);
++ result = fetch_multiple(&list, max_children, display_format);
+ }
+
+
+@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ if (max_children < 0)
+ max_children = fetch_parallel_config;
+
+- add_options_to_argv(&options);
++ add_options_to_argv(&options, display_format);
+ result = fetch_submodules(the_repository,
+ &options,
+ submodule_prefix,
## t/t5574-fetch-output.sh ##
-@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output with multiple remotes' '
+@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
test_cmp expect actual
'
+test_expect_success 'fetch porcelain output' '
-+ test_when_finished "rm -rf porcelain-cfg porcelain-cli" &&
++ test_when_finished "rm -rf porcelain" &&
+
+ # Set up a bunch of references that we can use to demonstrate different
+ # kinds of flag symbols in the output format.
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output with multiple
+ # namespaces so that we can test that rejected and force-updated
+ # references are reported properly.
+ refspecs="refs/heads/*:refs/unforced/* +refs/heads/*:refs/forced/*" &&
-+ git clone . porcelain-cli &&
-+ git clone . porcelain-cfg &&
-+ git -C porcelain-cfg fetch origin $refspecs &&
-+ git -C porcelain-cli fetch origin $refspecs &&
++ git clone . porcelain &&
++ git -C porcelain fetch origin $refspecs &&
+
+ # Now that we have set up the client repositories we can change our
+ # local references.
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output with multiple
+ # Execute a dry-run fetch first. We do this to assert that the dry-run
+ # and non-dry-run fetches produces the same output. Execution of the
+ # fetch is expected to fail as we have a rejected reference update.
-+ test_must_fail git -C porcelain-cfg -c fetch.output=porcelain fetch --dry-run --prune origin $refspecs >actual-dry-run-cfg &&
-+ test_must_fail git -C porcelain-cli fetch --output-format=porcelain --dry-run --prune origin $refspecs >actual-dry-run-cli &&
-+ test_cmp actual-dry-run-cfg actual-dry-run-cli &&
-+ test_cmp expect actual-dry-run-cfg &&
++ test_must_fail git -C porcelain fetch \
++ --porcelain --dry-run --prune origin $refspecs >actual &&
++ test_cmp expect actual &&
+
+ # And now we perform a non-dry-run fetch.
-+ test_must_fail git -C porcelain-cfg -c fetch.output=porcelain fetch --prune origin $refspecs >actual-cfg &&
-+ test_must_fail git -C porcelain-cli fetch --output-format=porcelain --prune origin $refspecs >actual-cli &&
-+ test_cmp actual-cfg actual-cli &&
-+ test_cmp expect actual-cfg &&
-+
-+ # Ensure that the dry-run and non-dry-run output matches.
-+ test_cmp actual-dry-run-cfg actual-cfg
++ test_must_fail git -C porcelain fetch \
++ --porcelain --prune origin $refspecs >actual 2>stderr &&
++ test_cmp expect actual &&
++ test_must_be_empty stderr
+'
+
- test_expect_success 'fetch output with HEAD and --dry-run' '
++test_expect_success 'fetch porcelain with multiple remotes' '
++ test_when_finished "rm -rf porcelain" &&
++
++ git clone . porcelain &&
++ git -C porcelain remote add second-remote "$PWD" &&
++ git -C porcelain fetch second-remote &&
++
++ test_commit --no-tag multi-commit &&
++ old_commit=$(git rev-parse HEAD~) &&
++ new_commit=$(git rev-parse HEAD) &&
++
++ cat >expect <<-EOF &&
++ $old_commit $new_commit refs/remotes/origin/force-updated
++ $old_commit $new_commit refs/remotes/second-remote/force-updated
++ EOF
++
++ git -C porcelain fetch --porcelain --all >actual 2>stderr &&
++ test_cmp expect actual &&
++ test_must_be_empty stderr
++'
++
++test_expect_success 'fetch porcelain refuses to work with submodules' '
++ test_when_finished "rm -rf porcelain" &&
++
++ cat >expect <<-EOF &&
++ fatal: options ${SQ}--porcelain${SQ} and ${SQ}--recurse-submodules${SQ} cannot be used together
++ EOF
++
++ git init porcelain &&
++ test_must_fail git -C porcelain fetch --porcelain --recurse-submodules=yes 2>stderr &&
++ test_cmp expect stderr &&
++
++ test_must_fail git -C porcelain fetch --porcelain --recurse-submodules=on-demand 2>stderr &&
++ test_cmp expect stderr
++'
++
+ test_expect_success 'fetch output with HEAD' '
test_when_finished "rm -rf head" &&
git clone . head &&
-@@ t/t5574-fetch-output.sh: test_expect_success 'fetch output with HEAD and --dry-run' '
- test_cmp expect actual
+@@ t/t5574-fetch-output.sh: test_expect_success 'fetch output with HEAD' '
+ test_cmp expect actual.err
'
-+test_expect_success 'fetch porcelain output with HEAD and --dry-run' '
++test_expect_success 'fetch porcelain output with HEAD' '
+ test_when_finished "rm -rf head" &&
+ git clone . head &&
+ COMMIT_ID=$(git rev-parse HEAD) &&
+
-+ git -C head fetch --output-format=porcelain --dry-run origin HEAD >actual &&
++ git -C head fetch --porcelain --dry-run origin HEAD >actual &&
+ cat >expect <<-EOF &&
+ * $ZERO_OID $COMMIT_ID FETCH_HEAD
+ EOF
+ test_cmp expect actual &&
+
-+ git -C head fetch --output-format=porcelain --dry-run origin HEAD:foo >actual &&
++ git -C head fetch --porcelain origin HEAD >actual &&
++ test_cmp expect actual &&
++
++ git -C head fetch --porcelain --dry-run origin HEAD:foo >actual &&
+ cat >expect <<-EOF &&
+ * $ZERO_OID $COMMIT_ID refs/heads/foo
+ EOF
++ test_cmp expect actual &&
++
++ git -C head fetch --porcelain origin HEAD:foo >actual &&
+ test_cmp expect actual
+'
+
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH v3 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches
2023-05-03 11:34 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
@ 2023-05-03 11:34 ` Patrick Steinhardt
2023-05-08 22:51 ` Glen Choo
2023-05-03 11:34 ` [PATCH v3 2/8] fetch: split out tests for output format Patrick Steinhardt
` (8 subsequent siblings)
9 siblings, 1 reply; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-03 11:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 2669 bytes --]
When running `git fetch --no-recurse-submodules`, the exectation is that
we don't fetch any submodules. And while this works for fetches of a
single remote, it doesn't when fetching multiple remotes at once. The
result is that we do recurse into submodules even though the user has
explicitly asked us not to.
This is because while we pass on `--recurse-submodules={yes,on-demand}`
if specified by the user, we don't pass on `--no-recurse-submodules` to
the subprocess spawned to perform the submodule fetch.
Fix this by also forwarding this flag as expected.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 2 ++
t/t5526-fetch-submodules.sh | 31 +++++++++++++++++++++++++++++++
2 files changed, 33 insertions(+)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index c310d89878..08d7fc7233 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1876,6 +1876,8 @@ static void add_options_to_argv(struct strvec *argv)
strvec_push(argv, "--keep");
if (recurse_submodules == RECURSE_SUBMODULES_ON)
strvec_push(argv, "--recurse-submodules");
+ else if (recurse_submodules == RECURSE_SUBMODULES_OFF)
+ strvec_push(argv, "--no-recurse-submodules");
else if (recurse_submodules == RECURSE_SUBMODULES_ON_DEMAND)
strvec_push(argv, "--recurse-submodules=on-demand");
if (tags == TAGS_SET)
diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh
index dcdbe26a08..162e5bac2f 100755
--- a/t/t5526-fetch-submodules.sh
+++ b/t/t5526-fetch-submodules.sh
@@ -1180,4 +1180,35 @@ test_expect_success 'fetch --all with --recurse-submodules with multiple' '
test_line_count = 2 fetch-subs
'
+test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" '
+ test_when_finished "git -C downstream remote remove second" &&
+
+ # We need to add a second remote, otherwise --all falls back to the
+ # normal fetch-one case.
+ git -C downstream remote add second .. &&
+ git -C downstream fetch --all &&
+
+ add_submodule_commits &&
+ add_superproject_commits &&
+ old_commit=$(git rev-parse --short HEAD~) &&
+ new_commit=$(git rev-parse --short HEAD) &&
+
+ git -C downstream fetch --all --no-recurse-submodules >actual.out 2>actual.err &&
+
+ cat >expect.out <<-EOF &&
+ Fetching origin
+ Fetching second
+ EOF
+
+ cat >expect.err <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ $old_commit..$new_commit super -> origin/super
+ From ..
+ $old_commit..$new_commit super -> second/super
+ EOF
+
+ test_cmp expect.out actual.out &&
+ test_cmp expect.err actual.err
+'
+
test_done
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v3 2/8] fetch: split out tests for output format
2023-05-03 11:34 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches Patrick Steinhardt
@ 2023-05-03 11:34 ` Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 3/8] fetch: add a test to exercise invalid output formats Patrick Steinhardt
` (7 subsequent siblings)
9 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-03 11:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 4269 bytes --]
We're about to introduce a new porcelain mode for the output of
git-fetch(1). As part of that we'll be introducing a set of new tests
that only relate to the output of this command.
Split out tests that exercise the output format of git-fetch(1) so that
it becomes easier to verify this functionality as a standalone unit. As
the tests assume that the default branch is called "main" we set up the
corresponding GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME environment variable
accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
t/t5510-fetch.sh | 53 ----------------------------------
t/t5574-fetch-output.sh | 63 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 63 insertions(+), 53 deletions(-)
create mode 100755 t/t5574-fetch-output.sh
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index dc44da9c79..4f289063ce 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -1118,59 +1118,6 @@ test_expect_success 'fetching with auto-gc does not lock up' '
)
'
-test_expect_success 'fetch aligned output' '
- git clone . full-output &&
- test_commit looooooooooooong-tag &&
- (
- cd full-output &&
- git -c fetch.output=full fetch origin >actual 2>&1 &&
- grep -e "->" actual | cut -c 22- >../actual
- ) &&
- cat >expect <<-\EOF &&
- main -> origin/main
- looooooooooooong-tag -> looooooooooooong-tag
- EOF
- test_cmp expect actual
-'
-
-test_expect_success 'fetch compact output' '
- git clone . compact &&
- test_commit extraaa &&
- (
- cd compact &&
- git -c fetch.output=compact fetch origin >actual 2>&1 &&
- grep -e "->" actual | cut -c 22- >../actual
- ) &&
- cat >expect <<-\EOF &&
- main -> origin/*
- extraaa -> *
- EOF
- test_cmp expect actual
-'
-
-test_expect_success '--no-show-forced-updates' '
- mkdir forced-updates &&
- (
- cd forced-updates &&
- git init &&
- test_commit 1 &&
- test_commit 2
- ) &&
- git clone forced-updates forced-update-clone &&
- git clone forced-updates no-forced-update-clone &&
- git -C forced-updates reset --hard HEAD~1 &&
- (
- cd forced-update-clone &&
- git fetch --show-forced-updates origin 2>output &&
- test_i18ngrep "(forced update)" output
- ) &&
- (
- cd no-forced-update-clone &&
- git fetch --no-show-forced-updates origin 2>output &&
- test_i18ngrep ! "(forced update)" output
- )
-'
-
for section in fetch transfer
do
test_expect_success "$section.hideRefs affects connectivity check" '
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
new file mode 100755
index 0000000000..f91b654d38
--- /dev/null
+++ b/t/t5574-fetch-output.sh
@@ -0,0 +1,63 @@
+#!/bin/sh
+
+test_description='git fetch output format'
+
+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+
+. ./test-lib.sh
+
+test_expect_success 'fetch aligned output' '
+ git clone . full-output &&
+ test_commit looooooooooooong-tag &&
+ (
+ cd full-output &&
+ git -c fetch.output=full fetch origin >actual 2>&1 &&
+ grep -e "->" actual | cut -c 22- >../actual
+ ) &&
+ cat >expect <<-\EOF &&
+ main -> origin/main
+ looooooooooooong-tag -> looooooooooooong-tag
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success 'fetch compact output' '
+ git clone . compact &&
+ test_commit extraaa &&
+ (
+ cd compact &&
+ git -c fetch.output=compact fetch origin >actual 2>&1 &&
+ grep -e "->" actual | cut -c 22- >../actual
+ ) &&
+ cat >expect <<-\EOF &&
+ main -> origin/*
+ extraaa -> *
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success '--no-show-forced-updates' '
+ mkdir forced-updates &&
+ (
+ cd forced-updates &&
+ git init &&
+ test_commit 1 &&
+ test_commit 2
+ ) &&
+ git clone forced-updates forced-update-clone &&
+ git clone forced-updates no-forced-update-clone &&
+ git -C forced-updates reset --hard HEAD~1 &&
+ (
+ cd forced-update-clone &&
+ git fetch --show-forced-updates origin 2>output &&
+ test_i18ngrep "(forced update)" output
+ ) &&
+ (
+ cd no-forced-update-clone &&
+ git fetch --no-show-forced-updates origin 2>output &&
+ test_i18ngrep ! "(forced update)" output
+ )
+'
+
+test_done
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v3 3/8] fetch: add a test to exercise invalid output formats
2023-05-03 11:34 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 2/8] fetch: split out tests for output format Patrick Steinhardt
@ 2023-05-03 11:34 ` Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 4/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
` (6 subsequent siblings)
9 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-03 11:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 1309 bytes --]
Add a testcase that exercises the logic when an invalid output format is
passed via the `fetch.output` configuration.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
t/t5574-fetch-output.sh | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index f91b654d38..a09750d225 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -7,6 +7,25 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
. ./test-lib.sh
+test_expect_success 'fetch with invalid output format configuration' '
+ test_when_finished "rm -rf clone" &&
+ git clone . clone &&
+
+ test_must_fail git -C clone -c fetch.output= fetch origin >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}${SQ}
+ EOF
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err &&
+
+ test_must_fail git -C clone -c fetch.output=garbage fetch origin >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}garbage${SQ}
+ EOF
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err
+'
+
test_expect_success 'fetch aligned output' '
git clone . full-output &&
test_commit looooooooooooong-tag &&
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v3 4/8] fetch: fix missing from-reference when fetching HEAD:foo
2023-05-03 11:34 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (2 preceding siblings ...)
2023-05-03 11:34 ` [PATCH v3 3/8] fetch: add a test to exercise invalid output formats Patrick Steinhardt
@ 2023-05-03 11:34 ` Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 5/8] fetch: introduce `display_format` enum Patrick Steinhardt
` (5 subsequent siblings)
9 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-03 11:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 9412 bytes --]
`store_updated_refs()` parses the remote reference for two purposes:
- It gets used as a note when writing FETCH_HEAD.
- It is passed through to `display_ref_update()` to display
updated references in the following format:
```
* branch master -> master
```
In most cases, the parsed remote reference is the prettified reference
name and can thus be used for both cases. But if the remote reference is
HEAD, the parsed remote reference becomes empty. This is intended when
we write the FETCH_HEAD, where we skip writing the note in that case.
But it is not intended when displaying the updated references and would
cause us to miss the left-hand side of the displayed reference update:
```
$ git fetch origin HEAD:foo
From https://github.com/git/git
* [new ref] -> foo
```
The HEAD string is clearly missing from the left-hand side of the arrow,
which is further stressed by the point that the following commands show
the left-hand side as expected:
```
$ git fetch origin HEAD
From https://github.com/git/git
* branch HEAD -> FETCH_HEAD
$ git fetch origin master
From https://github.com/git/git
* branch master -> FETCH_HEAD
* branch master -> origin/master
```
The logic of how we compute the remote reference name that we ultimately
pass to `display_ref_update()` is not easy to follow. There are three
different cases here:
- When the remote reference name is "HEAD" we set the remote
reference name to the empty string. This is the case that causes
the bug to occur, where we would indeed want to print "HEAD"
instead of the empty string. This is what `prettify_refname()`
would return.
- When the remote reference name has a well-known prefix then we
strip this prefix. This matches what `prettify_refname()` does.
- Otherwise, we keep the fully qualified reference name. This also
matches what `prettify_refname()` does.
As the return value of `prettify_refname()` would do the correct thing
for us in all three cases, we can fix the bug by passing through the
full remote reference name to `display_ref_update()`, which learns to
call `prettify_refname()`. At the same time, this also simplifies the
code a bit.
Note that this patch also changes formatting of the block that computes
the "kind" and "what" variables. This is done on purpose so that it is
part of the diff, hopefully making the change easier to comprehend.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 37 +++++++++++++++++++------------------
t/t5574-fetch-output.sh | 29 +++++++++++++++++++++++++++++
2 files changed, 48 insertions(+), 18 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 08d7fc7233..6aecf549e8 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -918,12 +918,14 @@ static void display_ref_update(struct display_state *display_state, char code,
}
width = (summary_width + strlen(summary) - gettext_width(summary));
+ remote = prettify_refname(remote);
+ local = prettify_refname(local);
strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
if (!display_state->compact_format)
- print_remote_to_local(display_state, remote, prettify_refname(local));
+ print_remote_to_local(display_state, remote, local);
else
- print_compact(display_state, remote, prettify_refname(local));
+ print_compact(display_state, remote, local);
if (error)
strbuf_addf(&display_state->buf, " (%s)", error);
strbuf_addch(&display_state->buf, '\n');
@@ -934,7 +936,7 @@ static void display_ref_update(struct display_state *display_state, char code,
static int update_local_ref(struct ref *ref,
struct ref_transaction *transaction,
struct display_state *display_state,
- const char *remote, const struct ref *remote_ref,
+ const struct ref *remote_ref,
int summary_width)
{
struct commit *current = NULL, *updated;
@@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,
if (oideq(&ref->old_oid, &ref->new_oid)) {
if (verbosity > 0)
display_ref_update(display_state, '=', _("[up to date]"), NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 0;
}
@@ -959,7 +961,7 @@ static int update_local_ref(struct ref *ref,
*/
display_ref_update(display_state, '!', _("[rejected]"),
_("can't fetch into checked-out branch"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 1;
}
@@ -970,12 +972,12 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("updating tag", ref, transaction, 0);
display_ref_update(display_state, r ? '!' : 't', _("[tag update]"),
r ? _("unable to update local ref") : NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"),
_("would clobber existing tag"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 1;
}
}
@@ -1008,7 +1010,7 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref(msg, ref, transaction, 0);
display_ref_update(display_state, r ? '!' : '*', what,
r ? _("unable to update local ref") : NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return r;
}
@@ -1030,7 +1032,7 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("fast-forward", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : ' ', quickref.buf,
r ? _("unable to update local ref") : NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
strbuf_release(&quickref);
return r;
} else if (force || ref->force) {
@@ -1042,12 +1044,12 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("forced-update", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : '+', quickref.buf,
r ? _("unable to update local ref") : _("forced update"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
strbuf_release(&quickref);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"), _("non-fast-forward"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 1;
}
}
@@ -1252,14 +1254,13 @@ static int store_updated_refs(struct display_state *display_state,
if (!strcmp(rm->name, "HEAD")) {
kind = "";
what = "";
- }
- else if (skip_prefix(rm->name, "refs/heads/", &what))
+ } else if (skip_prefix(rm->name, "refs/heads/", &what)) {
kind = "branch";
- else if (skip_prefix(rm->name, "refs/tags/", &what))
+ } else if (skip_prefix(rm->name, "refs/tags/", &what)) {
kind = "tag";
- else if (skip_prefix(rm->name, "refs/remotes/", &what))
+ } else if (skip_prefix(rm->name, "refs/remotes/", &what)) {
kind = "remote-tracking branch";
- else {
+ } else {
kind = "";
what = rm->name;
}
@@ -1277,7 +1278,7 @@ static int store_updated_refs(struct display_state *display_state,
display_state->url_len);
if (ref) {
- rc |= update_local_ref(ref, transaction, display_state, what,
+ rc |= update_local_ref(ref, transaction, display_state,
rm, summary_width);
free(ref);
} else if (write_fetch_head || dry_run) {
@@ -1288,7 +1289,7 @@ static int store_updated_refs(struct display_state *display_state,
*/
display_ref_update(display_state, '*',
*kind ? kind : "branch", NULL,
- *what ? what : "HEAD",
+ rm->name,
"FETCH_HEAD", summary_width);
}
}
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index a09750d225..6e0f7e0046 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -56,6 +56,35 @@ test_expect_success 'fetch compact output' '
test_cmp expect actual
'
+test_expect_success 'fetch output with HEAD' '
+ test_when_finished "rm -rf head" &&
+ git clone . head &&
+
+ git -C head fetch --dry-run origin HEAD >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * branch HEAD -> FETCH_HEAD
+ EOF
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err &&
+
+ git -C head fetch origin HEAD >actual.out 2>actual.err &&
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err &&
+
+ git -C head fetch --dry-run origin HEAD:foo >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * [new ref] HEAD -> foo
+ EOF
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err &&
+
+ git -C head fetch origin HEAD:foo >actual.out 2>actual.err &&
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err
+'
+
test_expect_success '--no-show-forced-updates' '
mkdir forced-updates &&
(
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v3 5/8] fetch: introduce `display_format` enum
2023-05-03 11:34 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (3 preceding siblings ...)
2023-05-03 11:34 ` [PATCH v3 4/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
@ 2023-05-03 11:34 ` Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 6/8] fetch: move display format parsing into main function Patrick Steinhardt
` (4 subsequent siblings)
9 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-03 11:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 5113 bytes --]
We currently have two different display formats in git-fetch(1) with the
"full" and "compact" formats. This is tracked with a boolean value that
simply denotes whether the display format is supposed to be compacted
or not. This works reasonably well while there are only two formats, but
we're about to introduce another format that will make this a bit more
awkward to use.
Introduce a `enum display_format` that is more readily extensible.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 101 ++++++++++++++++++++++++++++++------------------
1 file changed, 64 insertions(+), 37 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 6aecf549e8..9e7e45344d 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -48,11 +48,17 @@ enum {
TAGS_SET = 2
};
+enum display_format {
+ DISPLAY_FORMAT_UNKNOWN = 0,
+ DISPLAY_FORMAT_FULL,
+ DISPLAY_FORMAT_COMPACT,
+};
+
struct display_state {
struct strbuf buf;
int refcol_width;
- int compact_format;
+ enum display_format format;
char *url;
int url_len, shown_url;
@@ -784,7 +790,6 @@ static int refcol_width(const struct ref *ref, int compact_format)
static void display_state_init(struct display_state *display_state, struct ref *ref_map,
const char *raw_url)
{
- struct ref *rm;
const char *format = "full";
int i;
@@ -809,31 +814,42 @@ static void display_state_init(struct display_state *display_state, struct ref *
git_config_get_string_tmp("fetch.output", &format);
if (!strcasecmp(format, "full"))
- display_state->compact_format = 0;
+ display_state->format = DISPLAY_FORMAT_FULL;
else if (!strcasecmp(format, "compact"))
- display_state->compact_format = 1;
+ display_state->format = DISPLAY_FORMAT_COMPACT;
else
die(_("invalid value for '%s': '%s'"),
"fetch.output", format);
- display_state->refcol_width = 10;
- for (rm = ref_map; rm; rm = rm->next) {
- int width;
+ switch (display_state->format) {
+ case DISPLAY_FORMAT_FULL:
+ case DISPLAY_FORMAT_COMPACT: {
+ struct ref *rm;
- if (rm->status == REF_STATUS_REJECT_SHALLOW ||
- !rm->peer_ref ||
- !strcmp(rm->name, "HEAD"))
- continue;
+ display_state->refcol_width = 10;
+ for (rm = ref_map; rm; rm = rm->next) {
+ int width;
- width = refcol_width(rm, display_state->compact_format);
+ if (rm->status == REF_STATUS_REJECT_SHALLOW ||
+ !rm->peer_ref ||
+ !strcmp(rm->name, "HEAD"))
+ continue;
- /*
- * Not precise calculation for compact mode because '*' can
- * appear on the left hand side of '->' and shrink the column
- * back.
- */
- if (display_state->refcol_width < width)
- display_state->refcol_width = width;
+ width = refcol_width(rm, display_state->format == DISPLAY_FORMAT_COMPACT);
+
+ /*
+ * Not precise calculation for compact mode because '*' can
+ * appear on the left hand side of '->' and shrink the column
+ * back.
+ */
+ if (display_state->refcol_width < width)
+ display_state->refcol_width = width;
+ }
+
+ break;
+ }
+ default:
+ BUG("unexpected display format %d", display_state->format);
}
}
@@ -904,30 +920,41 @@ static void display_ref_update(struct display_state *display_state, char code,
const char *remote, const char *local,
int summary_width)
{
- int width;
-
if (verbosity < 0)
return;
strbuf_reset(&display_state->buf);
- if (!display_state->shown_url) {
- strbuf_addf(&display_state->buf, _("From %.*s\n"),
- display_state->url_len, display_state->url);
- display_state->shown_url = 1;
+ switch (display_state->format) {
+ case DISPLAY_FORMAT_FULL:
+ case DISPLAY_FORMAT_COMPACT: {
+ int width;
+
+ if (!display_state->shown_url) {
+ strbuf_addf(&display_state->buf, _("From %.*s\n"),
+ display_state->url_len, display_state->url);
+ display_state->shown_url = 1;
+ }
+
+ width = (summary_width + strlen(summary) - gettext_width(summary));
+ remote = prettify_refname(remote);
+ local = prettify_refname(local);
+
+ strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
+
+ if (display_state->format != DISPLAY_FORMAT_COMPACT)
+ print_remote_to_local(display_state, remote, local);
+ else
+ print_compact(display_state, remote, local);
+
+ if (error)
+ strbuf_addf(&display_state->buf, " (%s)", error);
+
+ break;
}
-
- width = (summary_width + strlen(summary) - gettext_width(summary));
- remote = prettify_refname(remote);
- local = prettify_refname(local);
-
- strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
- if (!display_state->compact_format)
- print_remote_to_local(display_state, remote, local);
- else
- print_compact(display_state, remote, local);
- if (error)
- strbuf_addf(&display_state->buf, " (%s)", error);
+ default:
+ BUG("unexpected display format %d", display_state->format);
+ };
strbuf_addch(&display_state->buf, '\n');
fputs(display_state->buf.buf, stderr);
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v3 6/8] fetch: move display format parsing into main function
2023-05-03 11:34 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (4 preceding siblings ...)
2023-05-03 11:34 ` [PATCH v3 5/8] fetch: introduce `display_format` enum Patrick Steinhardt
@ 2023-05-03 11:34 ` Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 7/8] fetch: move option related variables " Patrick Steinhardt
` (3 subsequent siblings)
9 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-03 11:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 5425 bytes --]
Parsing the display format happens inside of `display_state_init()`. As
we only need to check for a simple config entry, this is a natural
location to put this code as it means that display-state logic is neatly
contained in a single location.
We're about to introduce a output format though that is intended to be
parseable by machines, for example inside of a script. In that case it
becomes a bit awkward of an interface if you have to call git-fetch(1)
with the `fetch.output` config key set. We're thus going to introduce a
new `--output-format` switch for git-fetch(1) so that the output format
can be configured more directly.
This means we'll have to hook parsing of the display format into the
command line options parser. Having the code to determine the actual
output format scattered across two different sites is hard to reason
about though.
Refactor the code such that callers are expected to pass the display
format that is to be used into `display_state_init()`. This allows us to
lift up the code into the main function, where we can then hook it into
command line options parser in a follow-up commit.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 41 ++++++++++++++++++++++++-----------------
1 file changed, 24 insertions(+), 17 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 9e7e45344d..e15d43dc1e 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -788,14 +788,13 @@ static int refcol_width(const struct ref *ref, int compact_format)
}
static void display_state_init(struct display_state *display_state, struct ref *ref_map,
- const char *raw_url)
+ const char *raw_url, enum display_format format)
{
- const char *format = "full";
int i;
memset(display_state, 0, sizeof(*display_state));
-
strbuf_init(&display_state->buf, 0);
+ display_state->format = format;
if (raw_url)
display_state->url = transport_anonymize_url(raw_url);
@@ -812,15 +811,6 @@ static void display_state_init(struct display_state *display_state, struct ref *
if (verbosity < 0)
return;
- git_config_get_string_tmp("fetch.output", &format);
- if (!strcasecmp(format, "full"))
- display_state->format = DISPLAY_FORMAT_FULL;
- else if (!strcasecmp(format, "compact"))
- display_state->format = DISPLAY_FORMAT_COMPACT;
- else
- die(_("invalid value for '%s': '%s'"),
- "fetch.output", format);
-
switch (display_state->format) {
case DISPLAY_FORMAT_FULL:
case DISPLAY_FORMAT_COMPACT: {
@@ -1614,7 +1604,8 @@ static int backfill_tags(struct display_state *display_state,
}
static int do_fetch(struct transport *transport,
- struct refspec *rs)
+ struct refspec *rs,
+ enum display_format display_format)
{
struct ref_transaction *transaction = NULL;
struct ref *ref_map = NULL;
@@ -1700,7 +1691,7 @@ static int do_fetch(struct transport *transport,
if (retcode)
goto cleanup;
- display_state_init(&display_state, ref_map, transport->url);
+ display_state_init(&display_state, ref_map, transport->url, display_format);
if (atomic_fetch) {
transaction = ref_transaction_begin(&err);
@@ -2078,7 +2069,8 @@ static inline void fetch_one_setup_partial(struct remote *remote)
}
static int fetch_one(struct remote *remote, int argc, const char **argv,
- int prune_tags_ok, int use_stdin_refspecs)
+ int prune_tags_ok, int use_stdin_refspecs,
+ enum display_format display_format)
{
struct refspec rs = REFSPEC_INIT_FETCH;
int i;
@@ -2145,7 +2137,7 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,
sigchain_push_common(unlock_pack_on_signal);
atexit(unlock_pack_atexit);
sigchain_push(SIGPIPE, SIG_IGN);
- exit_code = do_fetch(gtransport, &rs);
+ exit_code = do_fetch(gtransport, &rs, display_format);
sigchain_pop(SIGPIPE);
refspec_clear(&rs);
transport_disconnect(gtransport);
@@ -2157,6 +2149,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
{
int i;
const char *bundle_uri;
+ enum display_format display_format = DISPLAY_FORMAT_UNKNOWN;
struct string_list list = STRING_LIST_INIT_DUP;
struct remote *remote = NULL;
int result = 0;
@@ -2183,6 +2176,19 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
argc = parse_options(argc, argv, prefix,
builtin_fetch_options, builtin_fetch_usage, 0);
+ if (display_format == DISPLAY_FORMAT_UNKNOWN) {
+ const char *format = "full";
+
+ git_config_get_string_tmp("fetch.output", &format);
+ if (!strcasecmp(format, "full"))
+ display_format = DISPLAY_FORMAT_FULL;
+ else if (!strcasecmp(format, "compact"))
+ display_format = DISPLAY_FORMAT_COMPACT;
+ else
+ die(_("invalid value for '%s': '%s'"),
+ "fetch.output", format);
+ }
+
if (recurse_submodules_cli != RECURSE_SUBMODULES_DEFAULT)
recurse_submodules = recurse_submodules_cli;
@@ -2311,7 +2317,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
} else if (remote) {
if (filter_options.choice || has_promisor_remote())
fetch_one_setup_partial(remote);
- result = fetch_one(remote, argc, argv, prune_tags_ok, stdin_refspecs);
+ result = fetch_one(remote, argc, argv, prune_tags_ok, stdin_refspecs,
+ display_format);
} else {
int max_children = max_jobs;
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v3 7/8] fetch: move option related variables into main function
2023-05-03 11:34 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (5 preceding siblings ...)
2023-05-03 11:34 ` [PATCH v3 6/8] fetch: move display format parsing into main function Patrick Steinhardt
@ 2023-05-03 11:34 ` Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
` (2 subsequent siblings)
9 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-03 11:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 12498 bytes --]
The options of git-fetch(1) which we pass to `parse_options()` are
declared globally in `builtin/fetch.c`. This means we're forced to use
global variables for all the options, which is more likely to cause
confusion than explicitly passing state around.
Refactor the code to move the options into `cmd_fetch()`. Move variables
that were previously forced to be declared globally and which are only
used by `cmd_fetch()` into function-local scope.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 197 ++++++++++++++++++++++++------------------------
1 file changed, 100 insertions(+), 97 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index e15d43dc1e..820ec7285c 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -75,13 +75,12 @@ static int fetch_prune_tags_config = -1; /* unspecified */
static int prune_tags = -1; /* unspecified */
#define PRUNE_TAGS_BY_DEFAULT 0 /* do we prune tags by default? */
-static int all, append, dry_run, force, keep, multiple, update_head_ok;
+static int append, dry_run, force, keep, update_head_ok;
static int write_fetch_head = 1;
static int verbosity, deepen_relative, set_upstream, refetch;
static int progress = -1;
-static int enable_auto_gc = 1;
-static int tags = TAGS_DEFAULT, unshallow, update_shallow, deepen;
-static int max_jobs = -1, submodule_fetch_jobs_config = -1;
+static int tags = TAGS_DEFAULT, update_shallow, deepen;
+static int submodule_fetch_jobs_config = -1;
static int fetch_parallel_config = 1;
static int atomic_fetch;
static enum transport_family family;
@@ -92,17 +91,11 @@ static struct string_list deepen_not = STRING_LIST_INIT_NODUP;
static struct strbuf default_rla = STRBUF_INIT;
static struct transport *gtransport;
static struct transport *gsecondary;
-static const char *submodule_prefix = "";
static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
-static int recurse_submodules_cli = RECURSE_SUBMODULES_DEFAULT;
-static int recurse_submodules_default = RECURSE_SUBMODULES_ON_DEMAND;
static struct refspec refmap = REFSPEC_INIT_FETCH;
static struct list_objects_filter_options filter_options = LIST_OBJECTS_FILTER_INIT;
static struct string_list server_options = STRING_LIST_INIT_DUP;
static struct string_list negotiation_tip = STRING_LIST_INIT_NODUP;
-static int fetch_write_commit_graph = -1;
-static int stdin_refspecs = 0;
-static int negotiate_only;
static int git_fetch_config(const char *k, const char *v, void *cb)
{
@@ -160,92 +153,6 @@ static int parse_refmap_arg(const struct option *opt, const char *arg, int unset
return 0;
}
-static struct option builtin_fetch_options[] = {
- OPT__VERBOSITY(&verbosity),
- OPT_BOOL(0, "all", &all,
- N_("fetch from all remotes")),
- OPT_BOOL(0, "set-upstream", &set_upstream,
- N_("set upstream for git pull/fetch")),
- OPT_BOOL('a', "append", &append,
- N_("append to .git/FETCH_HEAD instead of overwriting")),
- OPT_BOOL(0, "atomic", &atomic_fetch,
- N_("use atomic transaction to update references")),
- OPT_STRING(0, "upload-pack", &upload_pack, N_("path"),
- N_("path to upload pack on remote end")),
- OPT__FORCE(&force, N_("force overwrite of local reference"), 0),
- OPT_BOOL('m', "multiple", &multiple,
- N_("fetch from multiple remotes")),
- OPT_SET_INT('t', "tags", &tags,
- N_("fetch all tags and associated objects"), TAGS_SET),
- OPT_SET_INT('n', NULL, &tags,
- N_("do not fetch all tags (--no-tags)"), TAGS_UNSET),
- OPT_INTEGER('j', "jobs", &max_jobs,
- N_("number of submodules fetched in parallel")),
- OPT_BOOL(0, "prefetch", &prefetch,
- N_("modify the refspec to place all refs within refs/prefetch/")),
- OPT_BOOL('p', "prune", &prune,
- N_("prune remote-tracking branches no longer on remote")),
- OPT_BOOL('P', "prune-tags", &prune_tags,
- N_("prune local tags no longer on remote and clobber changed tags")),
- OPT_CALLBACK_F(0, "recurse-submodules", &recurse_submodules_cli, N_("on-demand"),
- N_("control recursive fetching of submodules"),
- PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
- OPT_BOOL(0, "dry-run", &dry_run,
- N_("dry run")),
- OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
- N_("write fetched references to the FETCH_HEAD file")),
- OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
- OPT_BOOL('u', "update-head-ok", &update_head_ok,
- N_("allow updating of HEAD ref")),
- OPT_BOOL(0, "progress", &progress, N_("force progress reporting")),
- OPT_STRING(0, "depth", &depth, N_("depth"),
- N_("deepen history of shallow clone")),
- OPT_STRING(0, "shallow-since", &deepen_since, N_("time"),
- N_("deepen history of shallow repository based on time")),
- OPT_STRING_LIST(0, "shallow-exclude", &deepen_not, N_("revision"),
- N_("deepen history of shallow clone, excluding rev")),
- OPT_INTEGER(0, "deepen", &deepen_relative,
- N_("deepen history of shallow clone")),
- OPT_SET_INT_F(0, "unshallow", &unshallow,
- N_("convert to a complete repository"),
- 1, PARSE_OPT_NONEG),
- OPT_SET_INT_F(0, "refetch", &refetch,
- N_("re-fetch without negotiating common commits"),
- 1, PARSE_OPT_NONEG),
- { OPTION_STRING, 0, "submodule-prefix", &submodule_prefix, N_("dir"),
- N_("prepend this to submodule path output"), PARSE_OPT_HIDDEN },
- OPT_CALLBACK_F(0, "recurse-submodules-default",
- &recurse_submodules_default, N_("on-demand"),
- N_("default for recursive fetching of submodules "
- "(lower priority than config files)"),
- PARSE_OPT_HIDDEN, option_fetch_parse_recurse_submodules),
- OPT_BOOL(0, "update-shallow", &update_shallow,
- N_("accept refs that update .git/shallow")),
- OPT_CALLBACK_F(0, "refmap", NULL, N_("refmap"),
- N_("specify fetch refmap"), PARSE_OPT_NONEG, parse_refmap_arg),
- OPT_STRING_LIST('o', "server-option", &server_options, N_("server-specific"), N_("option to transmit")),
- OPT_SET_INT('4', "ipv4", &family, N_("use IPv4 addresses only"),
- TRANSPORT_FAMILY_IPV4),
- OPT_SET_INT('6', "ipv6", &family, N_("use IPv6 addresses only"),
- TRANSPORT_FAMILY_IPV6),
- OPT_STRING_LIST(0, "negotiation-tip", &negotiation_tip, N_("revision"),
- N_("report that we have only objects reachable from this object")),
- OPT_BOOL(0, "negotiate-only", &negotiate_only,
- N_("do not fetch a packfile; instead, print ancestors of negotiation tips")),
- OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
- OPT_BOOL(0, "auto-maintenance", &enable_auto_gc,
- N_("run 'maintenance --auto' after fetching")),
- OPT_BOOL(0, "auto-gc", &enable_auto_gc,
- N_("run 'maintenance --auto' after fetching")),
- OPT_BOOL(0, "show-forced-updates", &fetch_show_forced_updates,
- N_("check for forced-updates on all updated branches")),
- OPT_BOOL(0, "write-commit-graph", &fetch_write_commit_graph,
- N_("write the commit-graph after fetching")),
- OPT_BOOL(0, "stdin", &stdin_refspecs,
- N_("accept refspecs from stdin")),
- OPT_END()
-};
-
static void unlock_pack(unsigned int flags)
{
if (gtransport)
@@ -2147,13 +2054,109 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,
int cmd_fetch(int argc, const char **argv, const char *prefix)
{
- int i;
const char *bundle_uri;
+ const char *submodule_prefix = "";
enum display_format display_format = DISPLAY_FORMAT_UNKNOWN;
struct string_list list = STRING_LIST_INIT_DUP;
struct remote *remote = NULL;
+ int all = 0, multiple = 0;
int result = 0;
int prune_tags_ok = 1;
+ int enable_auto_gc = 1;
+ int unshallow = 0;
+ int max_jobs = -1;
+ int recurse_submodules_cli = RECURSE_SUBMODULES_DEFAULT;
+ int recurse_submodules_default = RECURSE_SUBMODULES_ON_DEMAND;
+ int fetch_write_commit_graph = -1;
+ int stdin_refspecs = 0;
+ int negotiate_only = 0;
+ int i;
+
+ struct option builtin_fetch_options[] = {
+ OPT__VERBOSITY(&verbosity),
+ OPT_BOOL(0, "all", &all,
+ N_("fetch from all remotes")),
+ OPT_BOOL(0, "set-upstream", &set_upstream,
+ N_("set upstream for git pull/fetch")),
+ OPT_BOOL('a', "append", &append,
+ N_("append to .git/FETCH_HEAD instead of overwriting")),
+ OPT_BOOL(0, "atomic", &atomic_fetch,
+ N_("use atomic transaction to update references")),
+ OPT_STRING(0, "upload-pack", &upload_pack, N_("path"),
+ N_("path to upload pack on remote end")),
+ OPT__FORCE(&force, N_("force overwrite of local reference"), 0),
+ OPT_BOOL('m', "multiple", &multiple,
+ N_("fetch from multiple remotes")),
+ OPT_SET_INT('t', "tags", &tags,
+ N_("fetch all tags and associated objects"), TAGS_SET),
+ OPT_SET_INT('n', NULL, &tags,
+ N_("do not fetch all tags (--no-tags)"), TAGS_UNSET),
+ OPT_INTEGER('j', "jobs", &max_jobs,
+ N_("number of submodules fetched in parallel")),
+ OPT_BOOL(0, "prefetch", &prefetch,
+ N_("modify the refspec to place all refs within refs/prefetch/")),
+ OPT_BOOL('p', "prune", &prune,
+ N_("prune remote-tracking branches no longer on remote")),
+ OPT_BOOL('P', "prune-tags", &prune_tags,
+ N_("prune local tags no longer on remote and clobber changed tags")),
+ OPT_CALLBACK_F(0, "recurse-submodules", &recurse_submodules_cli, N_("on-demand"),
+ N_("control recursive fetching of submodules"),
+ PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
+ OPT_BOOL(0, "dry-run", &dry_run,
+ N_("dry run")),
+ OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
+ N_("write fetched references to the FETCH_HEAD file")),
+ OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
+ OPT_BOOL('u', "update-head-ok", &update_head_ok,
+ N_("allow updating of HEAD ref")),
+ OPT_BOOL(0, "progress", &progress, N_("force progress reporting")),
+ OPT_STRING(0, "depth", &depth, N_("depth"),
+ N_("deepen history of shallow clone")),
+ OPT_STRING(0, "shallow-since", &deepen_since, N_("time"),
+ N_("deepen history of shallow repository based on time")),
+ OPT_STRING_LIST(0, "shallow-exclude", &deepen_not, N_("revision"),
+ N_("deepen history of shallow clone, excluding rev")),
+ OPT_INTEGER(0, "deepen", &deepen_relative,
+ N_("deepen history of shallow clone")),
+ OPT_SET_INT_F(0, "unshallow", &unshallow,
+ N_("convert to a complete repository"),
+ 1, PARSE_OPT_NONEG),
+ OPT_SET_INT_F(0, "refetch", &refetch,
+ N_("re-fetch without negotiating common commits"),
+ 1, PARSE_OPT_NONEG),
+ { OPTION_STRING, 0, "submodule-prefix", &submodule_prefix, N_("dir"),
+ N_("prepend this to submodule path output"), PARSE_OPT_HIDDEN },
+ OPT_CALLBACK_F(0, "recurse-submodules-default",
+ &recurse_submodules_default, N_("on-demand"),
+ N_("default for recursive fetching of submodules "
+ "(lower priority than config files)"),
+ PARSE_OPT_HIDDEN, option_fetch_parse_recurse_submodules),
+ OPT_BOOL(0, "update-shallow", &update_shallow,
+ N_("accept refs that update .git/shallow")),
+ OPT_CALLBACK_F(0, "refmap", NULL, N_("refmap"),
+ N_("specify fetch refmap"), PARSE_OPT_NONEG, parse_refmap_arg),
+ OPT_STRING_LIST('o', "server-option", &server_options, N_("server-specific"), N_("option to transmit")),
+ OPT_SET_INT('4', "ipv4", &family, N_("use IPv4 addresses only"),
+ TRANSPORT_FAMILY_IPV4),
+ OPT_SET_INT('6', "ipv6", &family, N_("use IPv6 addresses only"),
+ TRANSPORT_FAMILY_IPV6),
+ OPT_STRING_LIST(0, "negotiation-tip", &negotiation_tip, N_("revision"),
+ N_("report that we have only objects reachable from this object")),
+ OPT_BOOL(0, "negotiate-only", &negotiate_only,
+ N_("do not fetch a packfile; instead, print ancestors of negotiation tips")),
+ OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
+ OPT_BOOL(0, "auto-maintenance", &enable_auto_gc,
+ N_("run 'maintenance --auto' after fetching")),
+ OPT_BOOL(0, "auto-gc", &enable_auto_gc,
+ N_("run 'maintenance --auto' after fetching")),
+ OPT_BOOL(0, "show-forced-updates", &fetch_show_forced_updates,
+ N_("check for forced-updates on all updated branches")),
+ OPT_BOOL(0, "write-commit-graph", &fetch_write_commit_graph,
+ N_("write the commit-graph after fetching")),
+ OPT_BOOL(0, "stdin", &stdin_refspecs,
+ N_("accept refspecs from stdin")),
+ OPT_END()
+ };
packet_trace_identity("fetch");
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v3 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-05-03 11:34 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (6 preceding siblings ...)
2023-05-03 11:34 ` [PATCH v3 7/8] fetch: move option related variables " Patrick Steinhardt
@ 2023-05-03 11:34 ` Patrick Steinhardt
2023-05-08 23:42 ` Glen Choo
2023-05-09 0:03 ` Glen Choo
2023-05-03 16:48 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Junio C Hamano
2023-05-09 0:06 ` Glen Choo
9 siblings, 2 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-03 11:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 19534 bytes --]
The output of git-fetch(1) is obviously designed for consumption by
users, only: we neatly columnize data, we abbreviate reference names, we
print neat arrows and we don't provide information about actual object
IDs that have changed. This makes the output format basically unusable
in the context of scripted invocations of git-fetch(1) that want to
learn about the exact changes that the command performs.
Introduce a new machine-parseable "porcelain" output format that is
supposed to fix this shortcoming. This output format is intended to
provide information about every reference that is about to be updated,
the old object ID that the reference has been pointing to and the new
object ID it will be updated to. Furthermore, the output format provides
the same flags as the human-readable format to indicate basic conditions
for each reference update like whether it was a fast-forward update, a
branch deletion, a rejected update or others.
The output format is quite simple:
```
<flag> <old-object-id> <new-object-id> <local-reference>\n
```
We assume two conditions which are generally true:
- The old and new object IDs have fixed known widths and cannot
contain spaces.
- References cannot contain newlines.
With these assumptions, the output format becomes unambiguously
parseable. Furthermore, given that this output is designed to be
consumed by scripts, the machine-readable data is printed to stdout
instead of stderr like the human-readable output is. This is mostly done
so that other data printed to stderr, like error messages or progress
meters, don't interfere with the parseable data.
A notable ommission here is that the output format does not include the
remote from which a reference was fetched, which might be important
information especially in the context of multi-remote fetches. But as
such a format would require us to print the remote for every single
reference update due to parallelizable fetches it feels wasteful for the
most likely usecase, which is when fetching from a single remote.
In a similar spirit, a second restriction is that this cannot be used
with `--recurse-submodules`. This is because any reference updates would
be ambiguous without also printing the repository in which the update
happens.
Considering that both multi-remote and submodule fetches are rather
niche and likely not going to be useful for the majority of usecases
these omissions feel acceptable. If usecases for either of these come up
in the future though it is easy enough to add a new "porcelain-v2"
format that adds this information.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
Documentation/fetch-options.txt | 6 ++
Documentation/git-fetch.txt | 9 +++
builtin/fetch.c | 91 +++++++++++++++++++-----
t/t5574-fetch-output.sh | 120 ++++++++++++++++++++++++++++++++
4 files changed, 208 insertions(+), 18 deletions(-)
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 622bd84768..02a5ae21ef 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -78,6 +78,12 @@ linkgit:git-config[1].
--dry-run::
Show what would be done, without making any changes.
+--porcelain::
+ Print the output to standard output in an easy-to-parse format for
+ scripts. See section OUTPUT in linkgit:git-fetch[1] for details.
++
+This is incompatible with `--recurse-submodules=[yes|on-demand]`.
+
ifndef::git-pull[]
--[no-]write-fetch-head::
Write the list of remote refs fetched in the `FETCH_HEAD`
diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index fba66f1460..f123139c58 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -204,6 +204,15 @@ representing the status of a single ref. Each line is of the form:
<flag> <summary> <from> -> <to> [<reason>]
-------------------------------
+When using `--porcelain`, the output format is intended to be
+machine-parseable. In contrast to the human-readable output formats it
+thus prints to standard output instead of standard error. Each line is
+of the form:
+
+-------------------------------
+<flag> <old-object-id> <new-object-id> <local-reference>
+-------------------------------
+
The status of up-to-date refs is shown only if the --verbose option is
used.
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 820ec7285c..187c4d373c 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -52,6 +52,7 @@ enum display_format {
DISPLAY_FORMAT_UNKNOWN = 0,
DISPLAY_FORMAT_FULL,
DISPLAY_FORMAT_COMPACT,
+ DISPLAY_FORMAT_PORCELAIN,
};
struct display_state {
@@ -745,6 +746,9 @@ static void display_state_init(struct display_state *display_state, struct ref *
break;
}
+ case DISPLAY_FORMAT_PORCELAIN:
+ /* We don't need to precompute anything here. */
+ break;
default:
BUG("unexpected display format %d", display_state->format);
}
@@ -815,8 +819,12 @@ static void print_compact(struct display_state *display_state,
static void display_ref_update(struct display_state *display_state, char code,
const char *summary, const char *error,
const char *remote, const char *local,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
int summary_width)
{
+ FILE *f = stderr;
+
if (verbosity < 0)
return;
@@ -849,12 +857,17 @@ static void display_ref_update(struct display_state *display_state, char code,
break;
}
+ case DISPLAY_FORMAT_PORCELAIN:
+ strbuf_addf(&display_state->buf, "%c %s %s %s", code,
+ oid_to_hex(old_oid), oid_to_hex(new_oid), local);
+ f = stdout;
+ break;
default:
BUG("unexpected display format %d", display_state->format);
};
strbuf_addch(&display_state->buf, '\n');
- fputs(display_state->buf.buf, stderr);
+ fputs(display_state->buf.buf, f);
}
static int update_local_ref(struct ref *ref,
@@ -872,7 +885,8 @@ static int update_local_ref(struct ref *ref,
if (oideq(&ref->old_oid, &ref->new_oid)) {
if (verbosity > 0)
display_ref_update(display_state, '=', _("[up to date]"), NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 0;
}
@@ -885,7 +899,8 @@ static int update_local_ref(struct ref *ref,
*/
display_ref_update(display_state, '!', _("[rejected]"),
_("can't fetch into checked-out branch"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 1;
}
@@ -896,12 +911,14 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("updating tag", ref, transaction, 0);
display_ref_update(display_state, r ? '!' : 't', _("[tag update]"),
r ? _("unable to update local ref") : NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"),
_("would clobber existing tag"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 1;
}
}
@@ -934,7 +951,8 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref(msg, ref, transaction, 0);
display_ref_update(display_state, r ? '!' : '*', what,
r ? _("unable to update local ref") : NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return r;
}
@@ -956,7 +974,8 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("fast-forward", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : ' ', quickref.buf,
r ? _("unable to update local ref") : NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
strbuf_release(&quickref);
return r;
} else if (force || ref->force) {
@@ -968,12 +987,14 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("forced-update", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : '+', quickref.buf,
r ? _("unable to update local ref") : _("forced update"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
strbuf_release(&quickref);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"), _("non-fast-forward"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 1;
}
}
@@ -1214,7 +1235,9 @@ static int store_updated_refs(struct display_state *display_state,
display_ref_update(display_state, '*',
*kind ? kind : "branch", NULL,
rm->name,
- "FETCH_HEAD", summary_width);
+ "FETCH_HEAD",
+ &rm->new_oid, &rm->old_oid,
+ summary_width);
}
}
}
@@ -1354,6 +1377,7 @@ static int prune_refs(struct display_state *display_state,
for (ref = stale_refs; ref; ref = ref->next) {
display_ref_update(display_state, '-', _("[deleted]"), NULL,
_("(none)"), ref->name,
+ &ref->new_oid, &ref->old_oid,
summary_width);
warn_dangling_symref(stderr, dangling_msg, ref->name);
}
@@ -1786,7 +1810,8 @@ static int add_remote_or_group(const char *name, struct string_list *list)
return 1;
}
-static void add_options_to_argv(struct strvec *argv)
+static void add_options_to_argv(struct strvec *argv,
+ enum display_format format)
{
if (dry_run)
strvec_push(argv, "--dry-run");
@@ -1822,6 +1847,8 @@ static void add_options_to_argv(struct strvec *argv)
strvec_push(argv, "--ipv6");
if (!write_fetch_head)
strvec_push(argv, "--no-write-fetch-head");
+ if (format == DISPLAY_FORMAT_PORCELAIN)
+ strvec_pushf(argv, "--porcelain");
}
/* Fetch multiple remotes in parallel */
@@ -1830,6 +1857,7 @@ struct parallel_fetch_state {
const char **argv;
struct string_list *remotes;
int next, result;
+ enum display_format format;
};
static int fetch_next_remote(struct child_process *cp,
@@ -1849,7 +1877,7 @@ static int fetch_next_remote(struct child_process *cp,
strvec_push(&cp->args, remote);
cp->git_cmd = 1;
- if (verbosity >= 0)
+ if (verbosity >= 0 && state->format != DISPLAY_FORMAT_PORCELAIN)
printf(_("Fetching %s\n"), remote);
return 1;
@@ -1881,7 +1909,8 @@ static int fetch_finished(int result, struct strbuf *out,
return 0;
}
-static int fetch_multiple(struct string_list *list, int max_children)
+static int fetch_multiple(struct string_list *list, int max_children,
+ enum display_format format)
{
int i, result = 0;
struct strvec argv = STRVEC_INIT;
@@ -1894,10 +1923,10 @@ static int fetch_multiple(struct string_list *list, int max_children)
strvec_pushl(&argv, "fetch", "--append", "--no-auto-gc",
"--no-write-commit-graph", NULL);
- add_options_to_argv(&argv);
+ add_options_to_argv(&argv, format);
if (max_children != 1 && list->nr != 1) {
- struct parallel_fetch_state state = { argv.v, list, 0, 0 };
+ struct parallel_fetch_state state = { argv.v, list, 0, 0, format };
const struct run_process_parallel_opts opts = {
.tr2_category = "fetch",
.tr2_label = "parallel/fetch",
@@ -1921,7 +1950,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
strvec_pushv(&cmd.args, argv.v);
strvec_push(&cmd.args, name);
- if (verbosity >= 0)
+ if (verbosity >= 0 && format != DISPLAY_FORMAT_PORCELAIN)
printf(_("Fetching %s\n"), name);
cmd.git_cmd = 1;
if (run_command(&cmd)) {
@@ -2052,6 +2081,13 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,
return exit_code;
}
+static int opt_parse_porcelain(const struct option *opt, const char *arg, int unset)
+{
+ enum display_format *format = opt->value;
+ *format = DISPLAY_FORMAT_PORCELAIN;
+ return 0;
+}
+
int cmd_fetch(int argc, const char **argv, const char *prefix)
{
const char *bundle_uri;
@@ -2104,6 +2140,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
OPT_BOOL(0, "dry-run", &dry_run,
N_("dry run")),
+ OPT_CALLBACK_F(0, "porcelain", &display_format, NULL, N_("machine-readable output"),
+ PARSE_OPT_NOARG|PARSE_OPT_NONEG, opt_parse_porcelain),
OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
N_("write fetched references to the FETCH_HEAD file")),
OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
@@ -2222,6 +2260,23 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
fetch_config_from_gitmodules(sfjc, rs);
}
+ if (display_format == DISPLAY_FORMAT_PORCELAIN) {
+ switch (recurse_submodules_cli) {
+ case RECURSE_SUBMODULES_OFF:
+ case RECURSE_SUBMODULES_DEFAULT:
+ /*
+ * Reference updates in submodules would be ambiguous
+ * in porcelain mode, so we reject this combination.
+ */
+ recurse_submodules = RECURSE_SUBMODULES_OFF;
+ break;
+
+ default:
+ die(_("options '%s' and '%s' cannot be used together"),
+ "--porcelain", "--recurse-submodules");
+ }
+ }
+
if (negotiate_only && !negotiation_tip.nr)
die(_("--negotiate-only needs one or more --negotiation-tip=*"));
@@ -2341,7 +2396,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
max_children = fetch_parallel_config;
/* TODO should this also die if we have a previous partial-clone? */
- result = fetch_multiple(&list, max_children);
+ result = fetch_multiple(&list, max_children, display_format);
}
@@ -2363,7 +2418,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
if (max_children < 0)
max_children = fetch_parallel_config;
- add_options_to_argv(&options);
+ add_options_to_argv(&options, display_format);
result = fetch_submodules(the_repository,
&options,
submodule_prefix,
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index 6e0f7e0046..18949e9323 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -56,6 +56,102 @@ test_expect_success 'fetch compact output' '
test_cmp expect actual
'
+test_expect_success 'fetch porcelain output' '
+ test_when_finished "rm -rf porcelain" &&
+
+ # Set up a bunch of references that we can use to demonstrate different
+ # kinds of flag symbols in the output format.
+ MAIN_OLD=$(git rev-parse HEAD) &&
+ git branch "fast-forward" &&
+ git branch "deleted-branch" &&
+ git checkout -b force-updated &&
+ test_commit --no-tag force-update-old &&
+ FORCE_UPDATED_OLD=$(git rev-parse HEAD) &&
+ git checkout main &&
+
+ # Clone and pre-seed the repositories. We fetch references into two
+ # namespaces so that we can test that rejected and force-updated
+ # references are reported properly.
+ refspecs="refs/heads/*:refs/unforced/* +refs/heads/*:refs/forced/*" &&
+ git clone . porcelain &&
+ git -C porcelain fetch origin $refspecs &&
+
+ # Now that we have set up the client repositories we can change our
+ # local references.
+ git branch new-branch &&
+ git branch -d deleted-branch &&
+ git checkout fast-forward &&
+ test_commit --no-tag fast-forward-new &&
+ FAST_FORWARD_NEW=$(git rev-parse HEAD) &&
+ git checkout force-updated &&
+ git reset --hard HEAD~ &&
+ test_commit --no-tag force-update-new &&
+ FORCE_UPDATED_NEW=$(git rev-parse HEAD) &&
+
+ cat >expect <<-EOF &&
+ - $MAIN_OLD $ZERO_OID refs/forced/deleted-branch
+ - $MAIN_OLD $ZERO_OID refs/unforced/deleted-branch
+ $MAIN_OLD $FAST_FORWARD_NEW refs/unforced/fast-forward
+ ! $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/unforced/force-updated
+ * $ZERO_OID $MAIN_OLD refs/unforced/new-branch
+ $MAIN_OLD $FAST_FORWARD_NEW refs/forced/fast-forward
+ + $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/forced/force-updated
+ * $ZERO_OID $MAIN_OLD refs/forced/new-branch
+ $MAIN_OLD $FAST_FORWARD_NEW refs/remotes/origin/fast-forward
+ + $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/remotes/origin/force-updated
+ * $ZERO_OID $MAIN_OLD refs/remotes/origin/new-branch
+ EOF
+
+ # Execute a dry-run fetch first. We do this to assert that the dry-run
+ # and non-dry-run fetches produces the same output. Execution of the
+ # fetch is expected to fail as we have a rejected reference update.
+ test_must_fail git -C porcelain fetch \
+ --porcelain --dry-run --prune origin $refspecs >actual &&
+ test_cmp expect actual &&
+
+ # And now we perform a non-dry-run fetch.
+ test_must_fail git -C porcelain fetch \
+ --porcelain --prune origin $refspecs >actual 2>stderr &&
+ test_cmp expect actual &&
+ test_must_be_empty stderr
+'
+
+test_expect_success 'fetch porcelain with multiple remotes' '
+ test_when_finished "rm -rf porcelain" &&
+
+ git clone . porcelain &&
+ git -C porcelain remote add second-remote "$PWD" &&
+ git -C porcelain fetch second-remote &&
+
+ test_commit --no-tag multi-commit &&
+ old_commit=$(git rev-parse HEAD~) &&
+ new_commit=$(git rev-parse HEAD) &&
+
+ cat >expect <<-EOF &&
+ $old_commit $new_commit refs/remotes/origin/force-updated
+ $old_commit $new_commit refs/remotes/second-remote/force-updated
+ EOF
+
+ git -C porcelain fetch --porcelain --all >actual 2>stderr &&
+ test_cmp expect actual &&
+ test_must_be_empty stderr
+'
+
+test_expect_success 'fetch porcelain refuses to work with submodules' '
+ test_when_finished "rm -rf porcelain" &&
+
+ cat >expect <<-EOF &&
+ fatal: options ${SQ}--porcelain${SQ} and ${SQ}--recurse-submodules${SQ} cannot be used together
+ EOF
+
+ git init porcelain &&
+ test_must_fail git -C porcelain fetch --porcelain --recurse-submodules=yes 2>stderr &&
+ test_cmp expect stderr &&
+
+ test_must_fail git -C porcelain fetch --porcelain --recurse-submodules=on-demand 2>stderr &&
+ test_cmp expect stderr
+'
+
test_expect_success 'fetch output with HEAD' '
test_when_finished "rm -rf head" &&
git clone . head &&
@@ -85,6 +181,30 @@ test_expect_success 'fetch output with HEAD' '
test_cmp expect actual.err
'
+test_expect_success 'fetch porcelain output with HEAD' '
+ test_when_finished "rm -rf head" &&
+ git clone . head &&
+ COMMIT_ID=$(git rev-parse HEAD) &&
+
+ git -C head fetch --porcelain --dry-run origin HEAD >actual &&
+ cat >expect <<-EOF &&
+ * $ZERO_OID $COMMIT_ID FETCH_HEAD
+ EOF
+ test_cmp expect actual &&
+
+ git -C head fetch --porcelain origin HEAD >actual &&
+ test_cmp expect actual &&
+
+ git -C head fetch --porcelain --dry-run origin HEAD:foo >actual &&
+ cat >expect <<-EOF &&
+ * $ZERO_OID $COMMIT_ID refs/heads/foo
+ EOF
+ test_cmp expect actual &&
+
+ git -C head fetch --porcelain origin HEAD:foo >actual &&
+ test_cmp expect actual
+'
+
test_expect_success '--no-show-forced-updates' '
mkdir forced-updates &&
(
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* Re: [PATCH v2 7/8] fetch: introduce new `--output-format` option
2023-05-03 9:43 ` Patrick Steinhardt
@ 2023-05-03 11:36 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-03 11:36 UTC (permalink / raw)
To: Glen Choo
Cc: git, Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 4409 bytes --]
On Wed, May 03, 2023 at 11:43:31AM +0200, Patrick Steinhardt wrote:
> On Fri, Apr 28, 2023 at 03:31:08PM -0700, Glen Choo wrote:
> > Patrick Steinhardt <ps@pks.im> writes:
> >
> > > @@ -1894,6 +1902,9 @@ static int fetch_multiple(struct string_list *list, int max_children)
> > > "--no-write-commit-graph", NULL);
> > > add_options_to_argv(&argv);
> > >
> > > + if (format != DISPLAY_FORMAT_UNKNOWN)
> > > + strvec_pushf(&argv, "--output-format=%s", display_formats[format]);
> > > +
> >
> > I think these lines belong inside add_options_to_argv(), since that's
> > also used to prepare argv for fetch_submodules(), so we'd also get
> > support for --recurse-submodules. (I wish I had spotted that in v1,
> > sorry. Thankfully they use the same helper function, so we only have to
> > do this once.)
> >
> > ----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----
> > diff --git a/builtin/fetch.c b/builtin/fetch.c
> > index 422e29a914..7aa385aed5 100644
> > --- a/builtin/fetch.c
> > +++ b/builtin/fetch.c
> > @@ -1796,8 +1796,11 @@ static int add_remote_or_group(const char *name, struct string_list *list)
> > return 1;
> > }
> >
> > -static void add_options_to_argv(struct strvec *argv)
> > +static void add_options_to_argv(struct strvec *argv,
> > + enum display_format format)
> > {
> > /* Maybe this shouldn't be first, idk */
> > + if (format != DISPLAY_FORMAT_UNKNOWN)
> > + strvec_pushf(argv, "--output-format=%s", display_formats[format]);
> > if (dry_run)
> > strvec_push(argv, "--dry-run");
> > if (prune != -1)
> > @@ -1908,10 +1911,7 @@ static int fetch_multiple(struct string_list *list, int max_children,
> > strvec_pushl(&argv, "-c", "fetch.bundleURI=",
> > "fetch", "--append", "--no-auto-gc",
> > "--no-write-commit-graph", NULL);
> > - add_options_to_argv(&argv);
> > -
> > - if (format != DISPLAY_FORMAT_UNKNOWN)
> > - strvec_pushf(&argv, "--output-format=%s", display_formats[format]);
> > + add_options_to_argv(&argv, format);
> >
> > if (max_children != 1 && list->nr != 1) {
> > struct parallel_fetch_state state = { argv.v, list, 0, 0 };
> > @@ -2403,7 +2403,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
> > if (max_children < 0)
> > max_children = fetch_parallel_config;
> >
> > - add_options_to_argv(&options);
> > + add_options_to_argv(&options, display_format);
> > result = fetch_submodules(the_repository,
> > &options,
> > submodule_prefix,
> >
> > ----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----
> >
> > I tested the result of that locally with --recurse-submodules, and
> > it works.
>
> Unfortunately it doesn't quite work alright: while the porcelain format
> does indeed get inherited to the child process correctly, the parallel
> process API will cause us to group output per submodule-fetch. This has
> the consequence that stdout will be redirected into stderr, and that
> then breaks the assumption that all machine-parseable output goes to
> stdout.
>
> My initial reflex is to just outright reject porcelain mode when
> submodule fetches are enabled. But that would require the caller to
> always explicitly pass `--recurse-submodules=off`, which isn't exactly
> great usability-wise.
>
> The alternative would be to ungroup the output so that we can continue
> to print to the correct output streams. That works alright, and I've got
> a working version that does exactly that. But now we have the issue that
> the porcelain output is misleading: you cannot tell whether a specific
> reference update happens in the parent repository or in the submodule as
> that information is not part of the output.
>
> I consider the second option to be much worse than the first option
> because it can cause scripts do to the wrong thing. So I'll send v3 with
> the first option, even though it's kind of an awful workaround. I'd be
> happy to hear any alternative proposals though.
>
> Patrick
I've gone with a slightly different variant of the first option that is
inspired by `--negotiate-only`: instead of refusing to run, we disable
submodule-fetches unless explicitly specified on the command line. In
that case we return an error.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v3 0/8] fetch: introduce machine-parseable output
2023-05-03 11:34 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (7 preceding siblings ...)
2023-05-03 11:34 ` [PATCH v3 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
@ 2023-05-03 16:48 ` Junio C Hamano
2023-05-03 16:53 ` Junio C Hamano
2023-05-09 0:06 ` Glen Choo
9 siblings, 1 reply; 120+ messages in thread
From: Junio C Hamano @ 2023-05-03 16:48 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> this is the third version of my patch series to introduce a
> machine-parseable output for git-fetch(1).
Which base did you pick to build these 8 patches? It seems that 6/8
is not happy when the series is attempted to be applied on top of
many recent tips of 'master' that I tried (including 667fcf4e15 that
the previous iteration was queued on top of in my tree).
Thanks.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v3 0/8] fetch: introduce machine-parseable output
2023-05-03 16:48 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Junio C Hamano
@ 2023-05-03 16:53 ` Junio C Hamano
2023-05-04 7:57 ` Patrick Steinhardt
0 siblings, 1 reply; 120+ messages in thread
From: Junio C Hamano @ 2023-05-03 16:53 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
Junio C Hamano <gitster@pobox.com> writes:
> Patrick Steinhardt <ps@pks.im> writes:
>
>> this is the third version of my patch series to introduce a
>> machine-parseable output for git-fetch(1).
>
> Which base did you pick to build these 8 patches? It seems that 6/8
> is not happy when the series is attempted to be applied on top of
> many recent tips of 'master' that I tried (including 667fcf4e15 that
> the previous iteration was queued on top of in my tree).
OK, I found e9dffbc7 (Merge branch 'ps/fetch-ref-update-reporting',
2023-04-06). I can manage from here.
Thanks.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v3 0/8] fetch: introduce machine-parseable output
2023-05-03 16:53 ` Junio C Hamano
@ 2023-05-04 7:57 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-04 7:57 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 798 bytes --]
On Wed, May 03, 2023 at 09:53:04AM -0700, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
> > Patrick Steinhardt <ps@pks.im> writes:
> >
> >> this is the third version of my patch series to introduce a
> >> machine-parseable output for git-fetch(1).
> >
> > Which base did you pick to build these 8 patches? It seems that 6/8
> > is not happy when the series is attempted to be applied on top of
> > many recent tips of 'master' that I tried (including 667fcf4e15 that
> > the previous iteration was queued on top of in my tree).
>
> OK, I found e9dffbc7 (Merge branch 'ps/fetch-ref-update-reporting',
> 2023-04-06). I can manage from here.
>
> Thanks.
Yeah, that's where I based it off from. Sorry for not explicitly
mentioning this fact.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v3 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches
2023-05-03 11:34 ` [PATCH v3 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches Patrick Steinhardt
@ 2023-05-08 22:51 ` Glen Choo
2023-05-09 12:41 ` Patrick Steinhardt
0 siblings, 1 reply; 120+ messages in thread
From: Glen Choo @ 2023-05-08 22:51 UTC (permalink / raw)
To: Patrick Steinhardt, git
Cc: Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> When running `git fetch --no-recurse-submodules`, the exectation is that
> we don't fetch any submodules. And while this works for fetches of a
> single remote, it doesn't when fetching multiple remotes at once. The
> result is that we do recurse into submodules even though the user has
> explicitly asked us not to.
>
> This is because while we pass on `--recurse-submodules={yes,on-demand}`
> if specified by the user, we don't pass on `--no-recurse-submodules` to
> the subprocess spawned to perform the submodule fetch.
>
> Fix this by also forwarding this flag as expected.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
> builtin/fetch.c | 2 ++
> t/t5526-fetch-submodules.sh | 31 +++++++++++++++++++++++++++++++
> 2 files changed, 33 insertions(+)
>
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> index c310d89878..08d7fc7233 100644
> --- a/builtin/fetch.c
> +++ b/builtin/fetch.c
> @@ -1876,6 +1876,8 @@ static void add_options_to_argv(struct strvec *argv)
> strvec_push(argv, "--keep");
> if (recurse_submodules == RECURSE_SUBMODULES_ON)
> strvec_push(argv, "--recurse-submodules");
> + else if (recurse_submodules == RECURSE_SUBMODULES_OFF)
> + strvec_push(argv, "--no-recurse-submodules");
> else if (recurse_submodules == RECURSE_SUBMODULES_ON_DEMAND)
> strvec_push(argv, "--recurse-submodules=on-demand");
> if (tags == TAGS_SET)
Makes sense.
I wondered for a bit whether this should have been checking
recurse_submodules_cli (the actual CLI flag) back in 386c076a86 (fetch
--negotiate-only: do not update submodules, 2022-01-18). I think this
current state is correct, though. After we have told the superproject
what submodule recursion mode to use, we want to continue using that
mode when recursing through submodules regardless of whether that mode
originally came from the CLI or config.
> +test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" '
> + test_when_finished "git -C downstream remote remove second" &&
> +
> + # We need to add a second remote, otherwise --all falls back to the
> + # normal fetch-one case.
> + git -C downstream remote add second .. &&
> + git -C downstream fetch --all &&
> +
> + add_submodule_commits &&
> + add_superproject_commits &&
> + old_commit=$(git rev-parse --short HEAD~) &&
> + new_commit=$(git rev-parse --short HEAD) &&
> +
> + git -C downstream fetch --all --no-recurse-submodules >actual.out 2>actual.err &&
> +
> + cat >expect.out <<-EOF &&
> + Fetching origin
> + Fetching second
> + EOF
> +
> + cat >expect.err <<-EOF &&
> + From $(test-tool path-utils real_path .)/.
> + $old_commit..$new_commit super -> origin/super
> + From ..
> + $old_commit..$new_commit super -> second/super
> + EOF
> +
> + test_cmp expect.out actual.out &&
> + test_cmp expect.err actual.err
> +'
The test looks okay, though is there a reason you didn't copy the style
of the previous test? It is nearly exactly what you want, I think, like
(untested)
test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" '
test_when_finished "rm -fr src_clone" &&
git clone --recurse-submodules src src_clone &&
(
cd src_clone &&
git remote add secondary ../src &&
git config submodule.recurse true &&
git config fetch.parallel 0 &&
git fetch --all 2>../fetch-log
) &&
grep "Fetching submodule" fetch-log >fetch-subs &&
test_must_be_empty fetch-subs
'
which has the handy benefit of not needing the test-tools invocation.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v3 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-05-03 11:34 ` [PATCH v3 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
@ 2023-05-08 23:42 ` Glen Choo
2023-05-09 12:41 ` Patrick Steinhardt
2023-05-09 0:03 ` Glen Choo
1 sibling, 1 reply; 120+ messages in thread
From: Glen Choo @ 2023-05-08 23:42 UTC (permalink / raw)
To: Patrick Steinhardt, git
Cc: Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
This version looks great! I only have minor comments.
Patrick Steinhardt <ps@pks.im> writes:
> A notable ommission here is that the output format does not include the
> remote from which a reference was fetched, which might be important
> information especially in the context of multi-remote fetches. But as
> such a format would require us to print the remote for every single
> reference update due to parallelizable fetches it feels wasteful for the
> most likely usecase, which is when fetching from a single remote.
>
> In a similar spirit, a second restriction is that this cannot be used
> with `--recurse-submodules`. This is because any reference updates would
> be ambiguous without also printing the repository in which the update
> happens.
>
> Considering that both multi-remote and submodule fetches are rather
> niche and likely not going to be useful for the majority of usecases
> these omissions feel acceptable. If usecases for either of these come up
> in the future though it is easy enough to add a new "porcelain-v2"
> format that adds this information.
As a point of clarification, I think these options aren't niche in
themselves, but they are more user-facing, so using them _in conjunction
with_ --porcelain is probably pretty niche, so I think this is okay for
now.
> --- a/Documentation/fetch-options.txt
> +++ b/Documentation/fetch-options.txt
> @@ -78,6 +78,12 @@ linkgit:git-config[1].
> --dry-run::
> Show what would be done, without making any changes.
>
> +--porcelain::
> + Print the output to standard output in an easy-to-parse format for
> + scripts. See section OUTPUT in linkgit:git-fetch[1] for details.
> ++
> +This is incompatible with `--recurse-submodules=[yes|on-demand]`.
> +
> ifndef::git-pull[]
> --[no-]write-fetch-head::
> Write the list of remote refs fetched in the `FETCH_HEAD`
> diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
> index fba66f1460..f123139c58 100644
> --- a/Documentation/git-fetch.txt
> +++ b/Documentation/git-fetch.txt
> @@ -204,6 +204,15 @@ representing the status of a single ref. Each line is of the form:
> <flag> <summary> <from> -> <to> [<reason>]
> -------------------------------
>
> +When using `--porcelain`, the output format is intended to be
> +machine-parseable. In contrast to the human-readable output formats it
> +thus prints to standard output instead of standard error. Each line is
> +of the form:
> +
> +-------------------------------
> +<flag> <old-object-id> <new-object-id> <local-reference>
> +-------------------------------
> +
> The status of up-to-date refs is shown only if the --verbose option is
> used.
I hadn't commented on the docs before, but they are very clear. Thanks!
> @@ -1830,6 +1857,7 @@ struct parallel_fetch_state {
> const char **argv;
> struct string_list *remotes;
> int next, result;
> + enum display_format format;
> };
>
> static int fetch_next_remote(struct child_process *cp,
> @@ -1849,7 +1877,7 @@ static int fetch_next_remote(struct child_process *cp,
> strvec_push(&cp->args, remote);
> cp->git_cmd = 1;
>
> - if (verbosity >= 0)
> + if (verbosity >= 0 && state->format != DISPLAY_FORMAT_PORCELAIN)
> printf(_("Fetching %s\n"), remote);
>
> return 1;
Here and elsewhere, I wonder if it's clearer to name the variable
"porcelain" and separate it from "enum display_format". Then we can
check "porcelain" directly instead of using "format ==
DISPLAY_FORMAT_PORCELAIN" as a proxy...
> +static int opt_parse_porcelain(const struct option *opt, const char *arg, int unset)
> +{
> + enum display_format *format = opt->value;
> + *format = DISPLAY_FORMAT_PORCELAIN;
> + return 0;
> +}
> +
> int cmd_fetch(int argc, const char **argv, const char *prefix)
> {
> const char *bundle_uri;
> @@ -2104,6 +2140,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
> PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
> OPT_BOOL(0, "dry-run", &dry_run,
> N_("dry run")),
> + OPT_CALLBACK_F(0, "porcelain", &display_format, NULL, N_("machine-readable output"),
> + PARSE_OPT_NOARG|PARSE_OPT_NONEG, opt_parse_porcelain),
> OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
> N_("write fetched references to the FETCH_HEAD file")),
> OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
e.g. since we are reusing the "display_format" variable, we need to make
sure we parse "--porcelain" after we read "fetch.output". I
double-checked to make sure we were doing the right thing, though it
would be nice to not have to worry about those sorts of things. This
shouldn't hold up the series though.
> +test_expect_success 'fetch porcelain with multiple remotes' '
> + test_when_finished "rm -rf porcelain" &&
> +
> + git clone . porcelain &&
> + git -C porcelain remote add second-remote "$PWD" &&
> + git -C porcelain fetch second-remote &&
> +
> + test_commit --no-tag multi-commit &&
> + old_commit=$(git rev-parse HEAD~) &&
> + new_commit=$(git rev-parse HEAD) &&
> +
> + cat >expect <<-EOF &&
> + $old_commit $new_commit refs/remotes/origin/force-updated
> + $old_commit $new_commit refs/remotes/second-remote/force-updated
> + EOF
The only thing in this test that relies on the previous test is that
HEAD is pointing to "force-updated", and it's hard to tell where HEAD is
since the previous test is so long. Could we create a new branch
instead?
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v3 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-05-03 11:34 ` [PATCH v3 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
2023-05-08 23:42 ` Glen Choo
@ 2023-05-09 0:03 ` Glen Choo
1 sibling, 0 replies; 120+ messages in thread
From: Glen Choo @ 2023-05-09 0:03 UTC (permalink / raw)
To: Patrick Steinhardt, git
Cc: Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> +--porcelain::
> + Print the output to standard output in an easy-to-parse format for
> + scripts. See section OUTPUT in linkgit:git-fetch[1] for details.
> ++
> +This is incompatible with `--recurse-submodules=[yes|on-demand]`.
> +
We should probably specify that this will override any setting of
"fetch.output" (and perhaps add a test for it). Most readers will
probably guess that this is the case, but the extra clarity wouldn't
hurt.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v3 0/8] fetch: introduce machine-parseable output
2023-05-03 11:34 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (8 preceding siblings ...)
2023-05-03 16:48 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Junio C Hamano
@ 2023-05-09 0:06 ` Glen Choo
2023-05-09 12:42 ` Patrick Steinhardt
9 siblings, 1 reply; 120+ messages in thread
From: Glen Choo @ 2023-05-09 0:06 UTC (permalink / raw)
To: Patrick Steinhardt, git
Cc: Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> Hi,
>
> this is the third version of my patch series to introduce a
> machine-parseable output for git-fetch(1).
This version looks great! I only had minor comments this time, mainly on
docs and tests.
Thanks for your patience and receptiveness, and for the fruitful
discussions :)
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v3 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches
2023-05-08 22:51 ` Glen Choo
@ 2023-05-09 12:41 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-09 12:41 UTC (permalink / raw)
To: Glen Choo
Cc: git, Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 2003 bytes --]
On Mon, May 08, 2023 at 03:51:52PM -0700, Glen Choo wrote:
> Patrick Steinhardt <ps@pks.im> writes:
[snip]
> > +test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" '
> > + test_when_finished "git -C downstream remote remove second" &&
> > +
> > + # We need to add a second remote, otherwise --all falls back to the
> > + # normal fetch-one case.
> > + git -C downstream remote add second .. &&
> > + git -C downstream fetch --all &&
> > +
> > + add_submodule_commits &&
> > + add_superproject_commits &&
> > + old_commit=$(git rev-parse --short HEAD~) &&
> > + new_commit=$(git rev-parse --short HEAD) &&
> > +
> > + git -C downstream fetch --all --no-recurse-submodules >actual.out 2>actual.err &&
> > +
> > + cat >expect.out <<-EOF &&
> > + Fetching origin
> > + Fetching second
> > + EOF
> > +
> > + cat >expect.err <<-EOF &&
> > + From $(test-tool path-utils real_path .)/.
> > + $old_commit..$new_commit super -> origin/super
> > + From ..
> > + $old_commit..$new_commit super -> second/super
> > + EOF
> > +
> > + test_cmp expect.out actual.out &&
> > + test_cmp expect.err actual.err
> > +'
>
> The test looks okay, though is there a reason you didn't copy the style
> of the previous test? It is nearly exactly what you want, I think, like
> (untested)
>
> test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" '
> test_when_finished "rm -fr src_clone" &&
> git clone --recurse-submodules src src_clone &&
> (
> cd src_clone &&
> git remote add secondary ../src &&
> git config submodule.recurse true &&
> git config fetch.parallel 0 &&
> git fetch --all 2>../fetch-log
> ) &&
> grep "Fetching submodule" fetch-log >fetch-subs &&
> test_must_be_empty fetch-subs
> '
>
> which has the handy benefit of not needing the test-tools invocation.
That is indeed much simpler, thanks.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v3 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-05-08 23:42 ` Glen Choo
@ 2023-05-09 12:41 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-09 12:41 UTC (permalink / raw)
To: Glen Choo
Cc: git, Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 5179 bytes --]
On Mon, May 08, 2023 at 04:42:46PM -0700, Glen Choo wrote:
> This version looks great! I only have minor comments.
>
> Patrick Steinhardt <ps@pks.im> writes:
>
> > A notable ommission here is that the output format does not include the
> > remote from which a reference was fetched, which might be important
> > information especially in the context of multi-remote fetches. But as
> > such a format would require us to print the remote for every single
> > reference update due to parallelizable fetches it feels wasteful for the
> > most likely usecase, which is when fetching from a single remote.
> >
> > In a similar spirit, a second restriction is that this cannot be used
> > with `--recurse-submodules`. This is because any reference updates would
> > be ambiguous without also printing the repository in which the update
> > happens.
> >
> > Considering that both multi-remote and submodule fetches are rather
> > niche and likely not going to be useful for the majority of usecases
> > these omissions feel acceptable. If usecases for either of these come up
> > in the future though it is easy enough to add a new "porcelain-v2"
> > format that adds this information.
>
> As a point of clarification, I think these options aren't niche in
> themselves, but they are more user-facing, so using them _in conjunction
> with_ --porcelain is probably pretty niche, so I think this is okay for
> now.
Yeah, that's what I indeed intended to say. Will clarify.
[snip]
> > @@ -1830,6 +1857,7 @@ struct parallel_fetch_state {
> > const char **argv;
> > struct string_list *remotes;
> > int next, result;
> > + enum display_format format;
> > };
> >
> > static int fetch_next_remote(struct child_process *cp,
> > @@ -1849,7 +1877,7 @@ static int fetch_next_remote(struct child_process *cp,
> > strvec_push(&cp->args, remote);
> > cp->git_cmd = 1;
> >
> > - if (verbosity >= 0)
> > + if (verbosity >= 0 && state->format != DISPLAY_FORMAT_PORCELAIN)
> > printf(_("Fetching %s\n"), remote);
> >
> > return 1;
>
> Here and elsewhere, I wonder if it's clearer to name the variable
> "porcelain" and separate it from "enum display_format". Then we can
> check "porcelain" directly instead of using "format ==
> DISPLAY_FORMAT_PORCELAIN" as a proxy...
For now I'd like to keep this as-is: it's easier to keep track of this
when there is only a single variable that keeps track of the output
format. But if we were to add additional porcelain formats in the future
I agree that it would be nice to refactor the code as you propose.
I'd rather keep the simpler version for now though where we only have a
single state to worry about.
> > +static int opt_parse_porcelain(const struct option *opt, const char *arg, int unset)
> > +{
> > + enum display_format *format = opt->value;
> > + *format = DISPLAY_FORMAT_PORCELAIN;
> > + return 0;
> > +}
> > +
> > int cmd_fetch(int argc, const char **argv, const char *prefix)
> > {
> > const char *bundle_uri;
> > @@ -2104,6 +2140,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
> > PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
> > OPT_BOOL(0, "dry-run", &dry_run,
> > N_("dry run")),
> > + OPT_CALLBACK_F(0, "porcelain", &display_format, NULL, N_("machine-readable output"),
> > + PARSE_OPT_NOARG|PARSE_OPT_NONEG, opt_parse_porcelain),
> > OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
> > N_("write fetched references to the FETCH_HEAD file")),
> > OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
>
> e.g. since we are reusing the "display_format" variable, we need to make
> sure we parse "--porcelain" after we read "fetch.output". I
> double-checked to make sure we were doing the right thing, though it
> would be nice to not have to worry about those sorts of things. This
> shouldn't hold up the series though.
In fact it is the other way round: we parse `--porcelain` first and then
only read "fetch.output" in the case where the `display_format` variable
is still set to `DISPLAY_FORMAT_UNKNOWN`.
So in the end there is no fragile order dependence here -- it would work
just the same regardless of whether we first parse command line options
or the configuration.
> > +test_expect_success 'fetch porcelain with multiple remotes' '
> > + test_when_finished "rm -rf porcelain" &&
> > +
> > + git clone . porcelain &&
> > + git -C porcelain remote add second-remote "$PWD" &&
> > + git -C porcelain fetch second-remote &&
> > +
> > + test_commit --no-tag multi-commit &&
> > + old_commit=$(git rev-parse HEAD~) &&
> > + new_commit=$(git rev-parse HEAD) &&
> > +
> > + cat >expect <<-EOF &&
> > + $old_commit $new_commit refs/remotes/origin/force-updated
> > + $old_commit $new_commit refs/remotes/second-remote/force-updated
> > + EOF
>
> The only thing in this test that relies on the previous test is that
> HEAD is pointing to "force-updated", and it's hard to tell where HEAD is
> since the previous test is so long. Could we create a new branch
> instead?
Makes sense, done.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v3 0/8] fetch: introduce machine-parseable output
2023-05-09 0:06 ` Glen Choo
@ 2023-05-09 12:42 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-09 12:42 UTC (permalink / raw)
To: Glen Choo
Cc: git, Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 503 bytes --]
On Mon, May 08, 2023 at 05:06:28PM -0700, Glen Choo wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
> > Hi,
> >
> > this is the third version of my patch series to introduce a
> > machine-parseable output for git-fetch(1).
>
> This version looks great! I only had minor comments this time, mainly on
> docs and tests.
>
> Thanks for your patience and receptiveness, and for the fruitful
> discussions :)
Thank _you_ for the extensive feedback and constructive criticism.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH v4 0/8] fetch: introduce machine-parseable output
2023-04-19 12:31 [PATCH 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (12 preceding siblings ...)
2023-05-03 11:34 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
@ 2023-05-09 13:01 ` Patrick Steinhardt
2023-05-09 13:02 ` [PATCH v4 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches Patrick Steinhardt
` (7 more replies)
2023-05-10 12:33 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Patrick Steinhardt
14 siblings, 8 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-09 13:01 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 7818 bytes --]
Hi,
this is the fourth version of my patch series to introduce a
machine-parseable output format for git-fetch(1). It applies on top of
e9dffbc7f1 (Merge branch 'ps/fetch-ref-update-reporting', 2023-04-06).
There's only a small set of change compared to v3:
- Patch 1/8: The test added by this change has been significantly
simplified.
- Patch 8/8: Reworded the commit message to clarify that parallel
fetches and `--recurse-submodules` aren't niche on their own, but
are likely niche combined with `--porcelain` given that they are
user-facing while `--porcelain` is script-facing.
- Patch 8/8: Clarified that `--porcelain` takes precedence over the
`fetch.output` config option in the user-facing documentation.
I've also added a test to verify that this is the case.
- Patch 8/8: Amended one test to not depend on the state that its
preceding test cretaes.
Thanks to Glen for the feedback.
Patrick
Patrick Steinhardt (8):
fetch: fix `--no-recurse-submodules` with multi-remote fetches
fetch: split out tests for output format
fetch: add a test to exercise invalid output formats
fetch: fix missing from-reference when fetching HEAD:foo
fetch: introduce `display_format` enum
fetch: move display format parsing into main function
fetch: move option related variables into main function
fetch: introduce machine-parseable "porcelain" output format
Documentation/fetch-options.txt | 7 +
Documentation/git-fetch.txt | 9 +
builtin/fetch.c | 437 +++++++++++++++++++-------------
t/t5510-fetch.sh | 53 ----
t/t5526-fetch-submodules.sh | 19 ++
t/t5574-fetch-output.sh | 251 ++++++++++++++++++
6 files changed, 552 insertions(+), 224 deletions(-)
create mode 100755 t/t5574-fetch-output.sh
Range-diff against v3:
1: 4b2b0cfe15 ! 1: d82b42ed34 fetch: fix `--no-recurse-submodules` with multi-remote fetches
@@ t/t5526-fetch-submodules.sh: test_expect_success 'fetch --all with --recurse-sub
'
+test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" '
-+ test_when_finished "git -C downstream remote remove second" &&
++ test_when_finished "rm -rf src_clone" &&
+
-+ # We need to add a second remote, otherwise --all falls back to the
-+ # normal fetch-one case.
-+ git -C downstream remote add second .. &&
-+ git -C downstream fetch --all &&
++ git clone --recurse-submodules src src_clone &&
++ (
++ cd src_clone &&
++ git remote add secondary ../src &&
++ git config submodule.recurse true &&
++ git config fetch.parallel 0 &&
++ git fetch --all --no-recurse-submodules 2>../actual
++ ) &&
+
-+ add_submodule_commits &&
-+ add_superproject_commits &&
-+ old_commit=$(git rev-parse --short HEAD~) &&
-+ new_commit=$(git rev-parse --short HEAD) &&
-+
-+ git -C downstream fetch --all --no-recurse-submodules >actual.out 2>actual.err &&
-+
-+ cat >expect.out <<-EOF &&
-+ Fetching origin
-+ Fetching second
++ cat >expect <<-EOF &&
++ From ../src
++ * [new branch] master -> secondary/master
+ EOF
-+
-+ cat >expect.err <<-EOF &&
-+ From $(test-tool path-utils real_path .)/.
-+ $old_commit..$new_commit super -> origin/super
-+ From ..
-+ $old_commit..$new_commit super -> second/super
-+ EOF
-+
-+ test_cmp expect.out actual.out &&
-+ test_cmp expect.err actual.err
++ test_cmp expect actual
+'
+
test_done
2: 6ebc7450ba = 2: 33112dc51a fetch: split out tests for output format
3: 78479922ac = 3: 006ea93afb fetch: add a test to exercise invalid output formats
4: 46e1266ab0 = 4: e599ea6d33 fetch: fix missing from-reference when fetching HEAD:foo
5: acc0f7f520 = 5: 80ac00b0c4 fetch: introduce `display_format` enum
6: cd23440128 = 6: 826b8b7bc0 fetch: move display format parsing into main function
7: edbc31013f = 7: 20f2e061d6 fetch: move option related variables into main function
8: e132d9494e ! 8: 24ae381950 fetch: introduce machine-parseable "porcelain" output format
@@ Commit message
be ambiguous without also printing the repository in which the update
happens.
- Considering that both multi-remote and submodule fetches are rather
- niche and likely not going to be useful for the majority of usecases
- these omissions feel acceptable. If usecases for either of these come up
- in the future though it is easy enough to add a new "porcelain-v2"
- format that adds this information.
+ Considering that both multi-remote and submodule fetches are user-facing
+ features, using them in conjunction with `--porcelain` that is intended
+ for scripting purposes is likely not going to be useful in the majority
+ of cases. With that in mind these restrictions feel acceptable. If
+ usecases for either of these come up in the future though it is easy
+ enough to add a new "porcelain-v2" format that adds this information.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
@@ Documentation/fetch-options.txt: linkgit:git-config[1].
+ Print the output to standard output in an easy-to-parse format for
+ scripts. See section OUTPUT in linkgit:git-fetch[1] for details.
++
-+This is incompatible with `--recurse-submodules=[yes|on-demand]`.
++This is incompatible with `--recurse-submodules=[yes|on-demand]` and takes
++precedence over the `fetch.output` config option.
+
ifndef::git-pull[]
--[no-]write-fetch-head::
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
+test_expect_success 'fetch porcelain with multiple remotes' '
+ test_when_finished "rm -rf porcelain" &&
+
++ git switch --create multiple-remotes &&
+ git clone . porcelain &&
+ git -C porcelain remote add second-remote "$PWD" &&
+ git -C porcelain fetch second-remote &&
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
+ new_commit=$(git rev-parse HEAD) &&
+
+ cat >expect <<-EOF &&
-+ $old_commit $new_commit refs/remotes/origin/force-updated
-+ $old_commit $new_commit refs/remotes/second-remote/force-updated
++ $old_commit $new_commit refs/remotes/origin/multiple-remotes
++ $old_commit $new_commit refs/remotes/second-remote/multiple-remotes
+ EOF
+
+ git -C porcelain fetch --porcelain --all >actual 2>stderr &&
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
+ test_must_fail git -C porcelain fetch --porcelain --recurse-submodules=on-demand 2>stderr &&
+ test_cmp expect stderr
+'
++
++test_expect_success 'fetch porcelain overrides fetch.output config' '
++ test_when_finished "rm -rf porcelain" &&
++
++ git switch --create config-override &&
++ git clone . porcelain &&
++ test_commit new-commit &&
++ old_commit=$(git rev-parse HEAD~) &&
++ new_commit=$(git rev-parse HEAD) &&
++
++ cat >expect <<-EOF &&
++ $old_commit $new_commit refs/remotes/origin/config-override
++ * $ZERO_OID $new_commit refs/tags/new-commit
++ EOF
++
++ git -C porcelain -c fetch.output=compact fetch --porcelain >stdout 2>stderr &&
++ test_must_be_empty stderr &&
++ test_cmp expect stdout
++'
+
test_expect_success 'fetch output with HEAD' '
test_when_finished "rm -rf head" &&
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH v4 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches
2023-05-09 13:01 ` [PATCH v4 " Patrick Steinhardt
@ 2023-05-09 13:02 ` Patrick Steinhardt
2023-05-09 17:49 ` Junio C Hamano
2023-05-09 13:02 ` [PATCH v4 2/8] fetch: split out tests for output format Patrick Steinhardt
` (6 subsequent siblings)
7 siblings, 1 reply; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-09 13:02 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 2231 bytes --]
When running `git fetch --no-recurse-submodules`, the exectation is that
we don't fetch any submodules. And while this works for fetches of a
single remote, it doesn't when fetching multiple remotes at once. The
result is that we do recurse into submodules even though the user has
explicitly asked us not to.
This is because while we pass on `--recurse-submodules={yes,on-demand}`
if specified by the user, we don't pass on `--no-recurse-submodules` to
the subprocess spawned to perform the submodule fetch.
Fix this by also forwarding this flag as expected.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 2 ++
t/t5526-fetch-submodules.sh | 19 +++++++++++++++++++
2 files changed, 21 insertions(+)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index c310d89878..08d7fc7233 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1876,6 +1876,8 @@ static void add_options_to_argv(struct strvec *argv)
strvec_push(argv, "--keep");
if (recurse_submodules == RECURSE_SUBMODULES_ON)
strvec_push(argv, "--recurse-submodules");
+ else if (recurse_submodules == RECURSE_SUBMODULES_OFF)
+ strvec_push(argv, "--no-recurse-submodules");
else if (recurse_submodules == RECURSE_SUBMODULES_ON_DEMAND)
strvec_push(argv, "--recurse-submodules=on-demand");
if (tags == TAGS_SET)
diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh
index dcdbe26a08..ba69cd583f 100755
--- a/t/t5526-fetch-submodules.sh
+++ b/t/t5526-fetch-submodules.sh
@@ -1180,4 +1180,23 @@ test_expect_success 'fetch --all with --recurse-submodules with multiple' '
test_line_count = 2 fetch-subs
'
+test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" '
+ test_when_finished "rm -rf src_clone" &&
+
+ git clone --recurse-submodules src src_clone &&
+ (
+ cd src_clone &&
+ git remote add secondary ../src &&
+ git config submodule.recurse true &&
+ git config fetch.parallel 0 &&
+ git fetch --all --no-recurse-submodules 2>../actual
+ ) &&
+
+ cat >expect <<-EOF &&
+ From ../src
+ * [new branch] master -> secondary/master
+ EOF
+ test_cmp expect actual
+'
+
test_done
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v4 2/8] fetch: split out tests for output format
2023-05-09 13:01 ` [PATCH v4 " Patrick Steinhardt
2023-05-09 13:02 ` [PATCH v4 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches Patrick Steinhardt
@ 2023-05-09 13:02 ` Patrick Steinhardt
2023-05-09 13:02 ` [PATCH v4 3/8] fetch: add a test to exercise invalid output formats Patrick Steinhardt
` (5 subsequent siblings)
7 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-09 13:02 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 4269 bytes --]
We're about to introduce a new porcelain mode for the output of
git-fetch(1). As part of that we'll be introducing a set of new tests
that only relate to the output of this command.
Split out tests that exercise the output format of git-fetch(1) so that
it becomes easier to verify this functionality as a standalone unit. As
the tests assume that the default branch is called "main" we set up the
corresponding GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME environment variable
accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
t/t5510-fetch.sh | 53 ----------------------------------
t/t5574-fetch-output.sh | 63 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 63 insertions(+), 53 deletions(-)
create mode 100755 t/t5574-fetch-output.sh
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index dc44da9c79..4f289063ce 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -1118,59 +1118,6 @@ test_expect_success 'fetching with auto-gc does not lock up' '
)
'
-test_expect_success 'fetch aligned output' '
- git clone . full-output &&
- test_commit looooooooooooong-tag &&
- (
- cd full-output &&
- git -c fetch.output=full fetch origin >actual 2>&1 &&
- grep -e "->" actual | cut -c 22- >../actual
- ) &&
- cat >expect <<-\EOF &&
- main -> origin/main
- looooooooooooong-tag -> looooooooooooong-tag
- EOF
- test_cmp expect actual
-'
-
-test_expect_success 'fetch compact output' '
- git clone . compact &&
- test_commit extraaa &&
- (
- cd compact &&
- git -c fetch.output=compact fetch origin >actual 2>&1 &&
- grep -e "->" actual | cut -c 22- >../actual
- ) &&
- cat >expect <<-\EOF &&
- main -> origin/*
- extraaa -> *
- EOF
- test_cmp expect actual
-'
-
-test_expect_success '--no-show-forced-updates' '
- mkdir forced-updates &&
- (
- cd forced-updates &&
- git init &&
- test_commit 1 &&
- test_commit 2
- ) &&
- git clone forced-updates forced-update-clone &&
- git clone forced-updates no-forced-update-clone &&
- git -C forced-updates reset --hard HEAD~1 &&
- (
- cd forced-update-clone &&
- git fetch --show-forced-updates origin 2>output &&
- test_i18ngrep "(forced update)" output
- ) &&
- (
- cd no-forced-update-clone &&
- git fetch --no-show-forced-updates origin 2>output &&
- test_i18ngrep ! "(forced update)" output
- )
-'
-
for section in fetch transfer
do
test_expect_success "$section.hideRefs affects connectivity check" '
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
new file mode 100755
index 0000000000..f91b654d38
--- /dev/null
+++ b/t/t5574-fetch-output.sh
@@ -0,0 +1,63 @@
+#!/bin/sh
+
+test_description='git fetch output format'
+
+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+
+. ./test-lib.sh
+
+test_expect_success 'fetch aligned output' '
+ git clone . full-output &&
+ test_commit looooooooooooong-tag &&
+ (
+ cd full-output &&
+ git -c fetch.output=full fetch origin >actual 2>&1 &&
+ grep -e "->" actual | cut -c 22- >../actual
+ ) &&
+ cat >expect <<-\EOF &&
+ main -> origin/main
+ looooooooooooong-tag -> looooooooooooong-tag
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success 'fetch compact output' '
+ git clone . compact &&
+ test_commit extraaa &&
+ (
+ cd compact &&
+ git -c fetch.output=compact fetch origin >actual 2>&1 &&
+ grep -e "->" actual | cut -c 22- >../actual
+ ) &&
+ cat >expect <<-\EOF &&
+ main -> origin/*
+ extraaa -> *
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success '--no-show-forced-updates' '
+ mkdir forced-updates &&
+ (
+ cd forced-updates &&
+ git init &&
+ test_commit 1 &&
+ test_commit 2
+ ) &&
+ git clone forced-updates forced-update-clone &&
+ git clone forced-updates no-forced-update-clone &&
+ git -C forced-updates reset --hard HEAD~1 &&
+ (
+ cd forced-update-clone &&
+ git fetch --show-forced-updates origin 2>output &&
+ test_i18ngrep "(forced update)" output
+ ) &&
+ (
+ cd no-forced-update-clone &&
+ git fetch --no-show-forced-updates origin 2>output &&
+ test_i18ngrep ! "(forced update)" output
+ )
+'
+
+test_done
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v4 3/8] fetch: add a test to exercise invalid output formats
2023-05-09 13:01 ` [PATCH v4 " Patrick Steinhardt
2023-05-09 13:02 ` [PATCH v4 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches Patrick Steinhardt
2023-05-09 13:02 ` [PATCH v4 2/8] fetch: split out tests for output format Patrick Steinhardt
@ 2023-05-09 13:02 ` Patrick Steinhardt
2023-05-09 17:58 ` Junio C Hamano
2023-05-09 13:02 ` [PATCH v4 4/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
` (4 subsequent siblings)
7 siblings, 1 reply; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-09 13:02 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 1309 bytes --]
Add a testcase that exercises the logic when an invalid output format is
passed via the `fetch.output` configuration.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
t/t5574-fetch-output.sh | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index f91b654d38..a09750d225 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -7,6 +7,25 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
. ./test-lib.sh
+test_expect_success 'fetch with invalid output format configuration' '
+ test_when_finished "rm -rf clone" &&
+ git clone . clone &&
+
+ test_must_fail git -C clone -c fetch.output= fetch origin >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}${SQ}
+ EOF
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err &&
+
+ test_must_fail git -C clone -c fetch.output=garbage fetch origin >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}garbage${SQ}
+ EOF
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err
+'
+
test_expect_success 'fetch aligned output' '
git clone . full-output &&
test_commit looooooooooooong-tag &&
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v4 4/8] fetch: fix missing from-reference when fetching HEAD:foo
2023-05-09 13:01 ` [PATCH v4 " Patrick Steinhardt
` (2 preceding siblings ...)
2023-05-09 13:02 ` [PATCH v4 3/8] fetch: add a test to exercise invalid output formats Patrick Steinhardt
@ 2023-05-09 13:02 ` Patrick Steinhardt
2023-05-09 19:28 ` Junio C Hamano
2023-05-09 13:02 ` [PATCH v4 5/8] fetch: introduce `display_format` enum Patrick Steinhardt
` (3 subsequent siblings)
7 siblings, 1 reply; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-09 13:02 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 9412 bytes --]
`store_updated_refs()` parses the remote reference for two purposes:
- It gets used as a note when writing FETCH_HEAD.
- It is passed through to `display_ref_update()` to display
updated references in the following format:
```
* branch master -> master
```
In most cases, the parsed remote reference is the prettified reference
name and can thus be used for both cases. But if the remote reference is
HEAD, the parsed remote reference becomes empty. This is intended when
we write the FETCH_HEAD, where we skip writing the note in that case.
But it is not intended when displaying the updated references and would
cause us to miss the left-hand side of the displayed reference update:
```
$ git fetch origin HEAD:foo
From https://github.com/git/git
* [new ref] -> foo
```
The HEAD string is clearly missing from the left-hand side of the arrow,
which is further stressed by the point that the following commands show
the left-hand side as expected:
```
$ git fetch origin HEAD
From https://github.com/git/git
* branch HEAD -> FETCH_HEAD
$ git fetch origin master
From https://github.com/git/git
* branch master -> FETCH_HEAD
* branch master -> origin/master
```
The logic of how we compute the remote reference name that we ultimately
pass to `display_ref_update()` is not easy to follow. There are three
different cases here:
- When the remote reference name is "HEAD" we set the remote
reference name to the empty string. This is the case that causes
the bug to occur, where we would indeed want to print "HEAD"
instead of the empty string. This is what `prettify_refname()`
would return.
- When the remote reference name has a well-known prefix then we
strip this prefix. This matches what `prettify_refname()` does.
- Otherwise, we keep the fully qualified reference name. This also
matches what `prettify_refname()` does.
As the return value of `prettify_refname()` would do the correct thing
for us in all three cases, we can fix the bug by passing through the
full remote reference name to `display_ref_update()`, which learns to
call `prettify_refname()`. At the same time, this also simplifies the
code a bit.
Note that this patch also changes formatting of the block that computes
the "kind" and "what" variables. This is done on purpose so that it is
part of the diff, hopefully making the change easier to comprehend.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 37 +++++++++++++++++++------------------
t/t5574-fetch-output.sh | 29 +++++++++++++++++++++++++++++
2 files changed, 48 insertions(+), 18 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 08d7fc7233..6aecf549e8 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -918,12 +918,14 @@ static void display_ref_update(struct display_state *display_state, char code,
}
width = (summary_width + strlen(summary) - gettext_width(summary));
+ remote = prettify_refname(remote);
+ local = prettify_refname(local);
strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
if (!display_state->compact_format)
- print_remote_to_local(display_state, remote, prettify_refname(local));
+ print_remote_to_local(display_state, remote, local);
else
- print_compact(display_state, remote, prettify_refname(local));
+ print_compact(display_state, remote, local);
if (error)
strbuf_addf(&display_state->buf, " (%s)", error);
strbuf_addch(&display_state->buf, '\n');
@@ -934,7 +936,7 @@ static void display_ref_update(struct display_state *display_state, char code,
static int update_local_ref(struct ref *ref,
struct ref_transaction *transaction,
struct display_state *display_state,
- const char *remote, const struct ref *remote_ref,
+ const struct ref *remote_ref,
int summary_width)
{
struct commit *current = NULL, *updated;
@@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,
if (oideq(&ref->old_oid, &ref->new_oid)) {
if (verbosity > 0)
display_ref_update(display_state, '=', _("[up to date]"), NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 0;
}
@@ -959,7 +961,7 @@ static int update_local_ref(struct ref *ref,
*/
display_ref_update(display_state, '!', _("[rejected]"),
_("can't fetch into checked-out branch"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 1;
}
@@ -970,12 +972,12 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("updating tag", ref, transaction, 0);
display_ref_update(display_state, r ? '!' : 't', _("[tag update]"),
r ? _("unable to update local ref") : NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"),
_("would clobber existing tag"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 1;
}
}
@@ -1008,7 +1010,7 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref(msg, ref, transaction, 0);
display_ref_update(display_state, r ? '!' : '*', what,
r ? _("unable to update local ref") : NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return r;
}
@@ -1030,7 +1032,7 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("fast-forward", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : ' ', quickref.buf,
r ? _("unable to update local ref") : NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
strbuf_release(&quickref);
return r;
} else if (force || ref->force) {
@@ -1042,12 +1044,12 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("forced-update", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : '+', quickref.buf,
r ? _("unable to update local ref") : _("forced update"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
strbuf_release(&quickref);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"), _("non-fast-forward"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 1;
}
}
@@ -1252,14 +1254,13 @@ static int store_updated_refs(struct display_state *display_state,
if (!strcmp(rm->name, "HEAD")) {
kind = "";
what = "";
- }
- else if (skip_prefix(rm->name, "refs/heads/", &what))
+ } else if (skip_prefix(rm->name, "refs/heads/", &what)) {
kind = "branch";
- else if (skip_prefix(rm->name, "refs/tags/", &what))
+ } else if (skip_prefix(rm->name, "refs/tags/", &what)) {
kind = "tag";
- else if (skip_prefix(rm->name, "refs/remotes/", &what))
+ } else if (skip_prefix(rm->name, "refs/remotes/", &what)) {
kind = "remote-tracking branch";
- else {
+ } else {
kind = "";
what = rm->name;
}
@@ -1277,7 +1278,7 @@ static int store_updated_refs(struct display_state *display_state,
display_state->url_len);
if (ref) {
- rc |= update_local_ref(ref, transaction, display_state, what,
+ rc |= update_local_ref(ref, transaction, display_state,
rm, summary_width);
free(ref);
} else if (write_fetch_head || dry_run) {
@@ -1288,7 +1289,7 @@ static int store_updated_refs(struct display_state *display_state,
*/
display_ref_update(display_state, '*',
*kind ? kind : "branch", NULL,
- *what ? what : "HEAD",
+ rm->name,
"FETCH_HEAD", summary_width);
}
}
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index a09750d225..6e0f7e0046 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -56,6 +56,35 @@ test_expect_success 'fetch compact output' '
test_cmp expect actual
'
+test_expect_success 'fetch output with HEAD' '
+ test_when_finished "rm -rf head" &&
+ git clone . head &&
+
+ git -C head fetch --dry-run origin HEAD >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * branch HEAD -> FETCH_HEAD
+ EOF
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err &&
+
+ git -C head fetch origin HEAD >actual.out 2>actual.err &&
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err &&
+
+ git -C head fetch --dry-run origin HEAD:foo >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * [new ref] HEAD -> foo
+ EOF
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err &&
+
+ git -C head fetch origin HEAD:foo >actual.out 2>actual.err &&
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err
+'
+
test_expect_success '--no-show-forced-updates' '
mkdir forced-updates &&
(
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v4 5/8] fetch: introduce `display_format` enum
2023-05-09 13:01 ` [PATCH v4 " Patrick Steinhardt
` (3 preceding siblings ...)
2023-05-09 13:02 ` [PATCH v4 4/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
@ 2023-05-09 13:02 ` Patrick Steinhardt
2023-05-09 20:19 ` Junio C Hamano
2023-05-09 13:02 ` [PATCH v4 6/8] fetch: move display format parsing into main function Patrick Steinhardt
` (2 subsequent siblings)
7 siblings, 1 reply; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-09 13:02 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 5113 bytes --]
We currently have two different display formats in git-fetch(1) with the
"full" and "compact" formats. This is tracked with a boolean value that
simply denotes whether the display format is supposed to be compacted
or not. This works reasonably well while there are only two formats, but
we're about to introduce another format that will make this a bit more
awkward to use.
Introduce a `enum display_format` that is more readily extensible.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 101 ++++++++++++++++++++++++++++++------------------
1 file changed, 64 insertions(+), 37 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 6aecf549e8..9e7e45344d 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -48,11 +48,17 @@ enum {
TAGS_SET = 2
};
+enum display_format {
+ DISPLAY_FORMAT_UNKNOWN = 0,
+ DISPLAY_FORMAT_FULL,
+ DISPLAY_FORMAT_COMPACT,
+};
+
struct display_state {
struct strbuf buf;
int refcol_width;
- int compact_format;
+ enum display_format format;
char *url;
int url_len, shown_url;
@@ -784,7 +790,6 @@ static int refcol_width(const struct ref *ref, int compact_format)
static void display_state_init(struct display_state *display_state, struct ref *ref_map,
const char *raw_url)
{
- struct ref *rm;
const char *format = "full";
int i;
@@ -809,31 +814,42 @@ static void display_state_init(struct display_state *display_state, struct ref *
git_config_get_string_tmp("fetch.output", &format);
if (!strcasecmp(format, "full"))
- display_state->compact_format = 0;
+ display_state->format = DISPLAY_FORMAT_FULL;
else if (!strcasecmp(format, "compact"))
- display_state->compact_format = 1;
+ display_state->format = DISPLAY_FORMAT_COMPACT;
else
die(_("invalid value for '%s': '%s'"),
"fetch.output", format);
- display_state->refcol_width = 10;
- for (rm = ref_map; rm; rm = rm->next) {
- int width;
+ switch (display_state->format) {
+ case DISPLAY_FORMAT_FULL:
+ case DISPLAY_FORMAT_COMPACT: {
+ struct ref *rm;
- if (rm->status == REF_STATUS_REJECT_SHALLOW ||
- !rm->peer_ref ||
- !strcmp(rm->name, "HEAD"))
- continue;
+ display_state->refcol_width = 10;
+ for (rm = ref_map; rm; rm = rm->next) {
+ int width;
- width = refcol_width(rm, display_state->compact_format);
+ if (rm->status == REF_STATUS_REJECT_SHALLOW ||
+ !rm->peer_ref ||
+ !strcmp(rm->name, "HEAD"))
+ continue;
- /*
- * Not precise calculation for compact mode because '*' can
- * appear on the left hand side of '->' and shrink the column
- * back.
- */
- if (display_state->refcol_width < width)
- display_state->refcol_width = width;
+ width = refcol_width(rm, display_state->format == DISPLAY_FORMAT_COMPACT);
+
+ /*
+ * Not precise calculation for compact mode because '*' can
+ * appear on the left hand side of '->' and shrink the column
+ * back.
+ */
+ if (display_state->refcol_width < width)
+ display_state->refcol_width = width;
+ }
+
+ break;
+ }
+ default:
+ BUG("unexpected display format %d", display_state->format);
}
}
@@ -904,30 +920,41 @@ static void display_ref_update(struct display_state *display_state, char code,
const char *remote, const char *local,
int summary_width)
{
- int width;
-
if (verbosity < 0)
return;
strbuf_reset(&display_state->buf);
- if (!display_state->shown_url) {
- strbuf_addf(&display_state->buf, _("From %.*s\n"),
- display_state->url_len, display_state->url);
- display_state->shown_url = 1;
+ switch (display_state->format) {
+ case DISPLAY_FORMAT_FULL:
+ case DISPLAY_FORMAT_COMPACT: {
+ int width;
+
+ if (!display_state->shown_url) {
+ strbuf_addf(&display_state->buf, _("From %.*s\n"),
+ display_state->url_len, display_state->url);
+ display_state->shown_url = 1;
+ }
+
+ width = (summary_width + strlen(summary) - gettext_width(summary));
+ remote = prettify_refname(remote);
+ local = prettify_refname(local);
+
+ strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
+
+ if (display_state->format != DISPLAY_FORMAT_COMPACT)
+ print_remote_to_local(display_state, remote, local);
+ else
+ print_compact(display_state, remote, local);
+
+ if (error)
+ strbuf_addf(&display_state->buf, " (%s)", error);
+
+ break;
}
-
- width = (summary_width + strlen(summary) - gettext_width(summary));
- remote = prettify_refname(remote);
- local = prettify_refname(local);
-
- strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
- if (!display_state->compact_format)
- print_remote_to_local(display_state, remote, local);
- else
- print_compact(display_state, remote, local);
- if (error)
- strbuf_addf(&display_state->buf, " (%s)", error);
+ default:
+ BUG("unexpected display format %d", display_state->format);
+ };
strbuf_addch(&display_state->buf, '\n');
fputs(display_state->buf.buf, stderr);
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v4 6/8] fetch: move display format parsing into main function
2023-05-09 13:01 ` [PATCH v4 " Patrick Steinhardt
` (4 preceding siblings ...)
2023-05-09 13:02 ` [PATCH v4 5/8] fetch: introduce `display_format` enum Patrick Steinhardt
@ 2023-05-09 13:02 ` Patrick Steinhardt
2023-05-09 20:35 ` Junio C Hamano
2023-05-09 22:30 ` Glen Choo
2023-05-09 13:02 ` [PATCH v4 7/8] fetch: move option related variables " Patrick Steinhardt
2023-05-09 13:02 ` [PATCH v4 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
7 siblings, 2 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-09 13:02 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 5425 bytes --]
Parsing the display format happens inside of `display_state_init()`. As
we only need to check for a simple config entry, this is a natural
location to put this code as it means that display-state logic is neatly
contained in a single location.
We're about to introduce a output format though that is intended to be
parseable by machines, for example inside of a script. In that case it
becomes a bit awkward of an interface if you have to call git-fetch(1)
with the `fetch.output` config key set. We're thus going to introduce a
new `--output-format` switch for git-fetch(1) so that the output format
can be configured more directly.
This means we'll have to hook parsing of the display format into the
command line options parser. Having the code to determine the actual
output format scattered across two different sites is hard to reason
about though.
Refactor the code such that callers are expected to pass the display
format that is to be used into `display_state_init()`. This allows us to
lift up the code into the main function, where we can then hook it into
command line options parser in a follow-up commit.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 41 ++++++++++++++++++++++++-----------------
1 file changed, 24 insertions(+), 17 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 9e7e45344d..e15d43dc1e 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -788,14 +788,13 @@ static int refcol_width(const struct ref *ref, int compact_format)
}
static void display_state_init(struct display_state *display_state, struct ref *ref_map,
- const char *raw_url)
+ const char *raw_url, enum display_format format)
{
- const char *format = "full";
int i;
memset(display_state, 0, sizeof(*display_state));
-
strbuf_init(&display_state->buf, 0);
+ display_state->format = format;
if (raw_url)
display_state->url = transport_anonymize_url(raw_url);
@@ -812,15 +811,6 @@ static void display_state_init(struct display_state *display_state, struct ref *
if (verbosity < 0)
return;
- git_config_get_string_tmp("fetch.output", &format);
- if (!strcasecmp(format, "full"))
- display_state->format = DISPLAY_FORMAT_FULL;
- else if (!strcasecmp(format, "compact"))
- display_state->format = DISPLAY_FORMAT_COMPACT;
- else
- die(_("invalid value for '%s': '%s'"),
- "fetch.output", format);
-
switch (display_state->format) {
case DISPLAY_FORMAT_FULL:
case DISPLAY_FORMAT_COMPACT: {
@@ -1614,7 +1604,8 @@ static int backfill_tags(struct display_state *display_state,
}
static int do_fetch(struct transport *transport,
- struct refspec *rs)
+ struct refspec *rs,
+ enum display_format display_format)
{
struct ref_transaction *transaction = NULL;
struct ref *ref_map = NULL;
@@ -1700,7 +1691,7 @@ static int do_fetch(struct transport *transport,
if (retcode)
goto cleanup;
- display_state_init(&display_state, ref_map, transport->url);
+ display_state_init(&display_state, ref_map, transport->url, display_format);
if (atomic_fetch) {
transaction = ref_transaction_begin(&err);
@@ -2078,7 +2069,8 @@ static inline void fetch_one_setup_partial(struct remote *remote)
}
static int fetch_one(struct remote *remote, int argc, const char **argv,
- int prune_tags_ok, int use_stdin_refspecs)
+ int prune_tags_ok, int use_stdin_refspecs,
+ enum display_format display_format)
{
struct refspec rs = REFSPEC_INIT_FETCH;
int i;
@@ -2145,7 +2137,7 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,
sigchain_push_common(unlock_pack_on_signal);
atexit(unlock_pack_atexit);
sigchain_push(SIGPIPE, SIG_IGN);
- exit_code = do_fetch(gtransport, &rs);
+ exit_code = do_fetch(gtransport, &rs, display_format);
sigchain_pop(SIGPIPE);
refspec_clear(&rs);
transport_disconnect(gtransport);
@@ -2157,6 +2149,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
{
int i;
const char *bundle_uri;
+ enum display_format display_format = DISPLAY_FORMAT_UNKNOWN;
struct string_list list = STRING_LIST_INIT_DUP;
struct remote *remote = NULL;
int result = 0;
@@ -2183,6 +2176,19 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
argc = parse_options(argc, argv, prefix,
builtin_fetch_options, builtin_fetch_usage, 0);
+ if (display_format == DISPLAY_FORMAT_UNKNOWN) {
+ const char *format = "full";
+
+ git_config_get_string_tmp("fetch.output", &format);
+ if (!strcasecmp(format, "full"))
+ display_format = DISPLAY_FORMAT_FULL;
+ else if (!strcasecmp(format, "compact"))
+ display_format = DISPLAY_FORMAT_COMPACT;
+ else
+ die(_("invalid value for '%s': '%s'"),
+ "fetch.output", format);
+ }
+
if (recurse_submodules_cli != RECURSE_SUBMODULES_DEFAULT)
recurse_submodules = recurse_submodules_cli;
@@ -2311,7 +2317,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
} else if (remote) {
if (filter_options.choice || has_promisor_remote())
fetch_one_setup_partial(remote);
- result = fetch_one(remote, argc, argv, prune_tags_ok, stdin_refspecs);
+ result = fetch_one(remote, argc, argv, prune_tags_ok, stdin_refspecs,
+ display_format);
} else {
int max_children = max_jobs;
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v4 7/8] fetch: move option related variables into main function
2023-05-09 13:01 ` [PATCH v4 " Patrick Steinhardt
` (5 preceding siblings ...)
2023-05-09 13:02 ` [PATCH v4 6/8] fetch: move display format parsing into main function Patrick Steinhardt
@ 2023-05-09 13:02 ` Patrick Steinhardt
2023-05-09 13:02 ` [PATCH v4 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
7 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-09 13:02 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 12498 bytes --]
The options of git-fetch(1) which we pass to `parse_options()` are
declared globally in `builtin/fetch.c`. This means we're forced to use
global variables for all the options, which is more likely to cause
confusion than explicitly passing state around.
Refactor the code to move the options into `cmd_fetch()`. Move variables
that were previously forced to be declared globally and which are only
used by `cmd_fetch()` into function-local scope.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 197 ++++++++++++++++++++++++------------------------
1 file changed, 100 insertions(+), 97 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index e15d43dc1e..820ec7285c 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -75,13 +75,12 @@ static int fetch_prune_tags_config = -1; /* unspecified */
static int prune_tags = -1; /* unspecified */
#define PRUNE_TAGS_BY_DEFAULT 0 /* do we prune tags by default? */
-static int all, append, dry_run, force, keep, multiple, update_head_ok;
+static int append, dry_run, force, keep, update_head_ok;
static int write_fetch_head = 1;
static int verbosity, deepen_relative, set_upstream, refetch;
static int progress = -1;
-static int enable_auto_gc = 1;
-static int tags = TAGS_DEFAULT, unshallow, update_shallow, deepen;
-static int max_jobs = -1, submodule_fetch_jobs_config = -1;
+static int tags = TAGS_DEFAULT, update_shallow, deepen;
+static int submodule_fetch_jobs_config = -1;
static int fetch_parallel_config = 1;
static int atomic_fetch;
static enum transport_family family;
@@ -92,17 +91,11 @@ static struct string_list deepen_not = STRING_LIST_INIT_NODUP;
static struct strbuf default_rla = STRBUF_INIT;
static struct transport *gtransport;
static struct transport *gsecondary;
-static const char *submodule_prefix = "";
static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
-static int recurse_submodules_cli = RECURSE_SUBMODULES_DEFAULT;
-static int recurse_submodules_default = RECURSE_SUBMODULES_ON_DEMAND;
static struct refspec refmap = REFSPEC_INIT_FETCH;
static struct list_objects_filter_options filter_options = LIST_OBJECTS_FILTER_INIT;
static struct string_list server_options = STRING_LIST_INIT_DUP;
static struct string_list negotiation_tip = STRING_LIST_INIT_NODUP;
-static int fetch_write_commit_graph = -1;
-static int stdin_refspecs = 0;
-static int negotiate_only;
static int git_fetch_config(const char *k, const char *v, void *cb)
{
@@ -160,92 +153,6 @@ static int parse_refmap_arg(const struct option *opt, const char *arg, int unset
return 0;
}
-static struct option builtin_fetch_options[] = {
- OPT__VERBOSITY(&verbosity),
- OPT_BOOL(0, "all", &all,
- N_("fetch from all remotes")),
- OPT_BOOL(0, "set-upstream", &set_upstream,
- N_("set upstream for git pull/fetch")),
- OPT_BOOL('a', "append", &append,
- N_("append to .git/FETCH_HEAD instead of overwriting")),
- OPT_BOOL(0, "atomic", &atomic_fetch,
- N_("use atomic transaction to update references")),
- OPT_STRING(0, "upload-pack", &upload_pack, N_("path"),
- N_("path to upload pack on remote end")),
- OPT__FORCE(&force, N_("force overwrite of local reference"), 0),
- OPT_BOOL('m', "multiple", &multiple,
- N_("fetch from multiple remotes")),
- OPT_SET_INT('t', "tags", &tags,
- N_("fetch all tags and associated objects"), TAGS_SET),
- OPT_SET_INT('n', NULL, &tags,
- N_("do not fetch all tags (--no-tags)"), TAGS_UNSET),
- OPT_INTEGER('j', "jobs", &max_jobs,
- N_("number of submodules fetched in parallel")),
- OPT_BOOL(0, "prefetch", &prefetch,
- N_("modify the refspec to place all refs within refs/prefetch/")),
- OPT_BOOL('p', "prune", &prune,
- N_("prune remote-tracking branches no longer on remote")),
- OPT_BOOL('P', "prune-tags", &prune_tags,
- N_("prune local tags no longer on remote and clobber changed tags")),
- OPT_CALLBACK_F(0, "recurse-submodules", &recurse_submodules_cli, N_("on-demand"),
- N_("control recursive fetching of submodules"),
- PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
- OPT_BOOL(0, "dry-run", &dry_run,
- N_("dry run")),
- OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
- N_("write fetched references to the FETCH_HEAD file")),
- OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
- OPT_BOOL('u', "update-head-ok", &update_head_ok,
- N_("allow updating of HEAD ref")),
- OPT_BOOL(0, "progress", &progress, N_("force progress reporting")),
- OPT_STRING(0, "depth", &depth, N_("depth"),
- N_("deepen history of shallow clone")),
- OPT_STRING(0, "shallow-since", &deepen_since, N_("time"),
- N_("deepen history of shallow repository based on time")),
- OPT_STRING_LIST(0, "shallow-exclude", &deepen_not, N_("revision"),
- N_("deepen history of shallow clone, excluding rev")),
- OPT_INTEGER(0, "deepen", &deepen_relative,
- N_("deepen history of shallow clone")),
- OPT_SET_INT_F(0, "unshallow", &unshallow,
- N_("convert to a complete repository"),
- 1, PARSE_OPT_NONEG),
- OPT_SET_INT_F(0, "refetch", &refetch,
- N_("re-fetch without negotiating common commits"),
- 1, PARSE_OPT_NONEG),
- { OPTION_STRING, 0, "submodule-prefix", &submodule_prefix, N_("dir"),
- N_("prepend this to submodule path output"), PARSE_OPT_HIDDEN },
- OPT_CALLBACK_F(0, "recurse-submodules-default",
- &recurse_submodules_default, N_("on-demand"),
- N_("default for recursive fetching of submodules "
- "(lower priority than config files)"),
- PARSE_OPT_HIDDEN, option_fetch_parse_recurse_submodules),
- OPT_BOOL(0, "update-shallow", &update_shallow,
- N_("accept refs that update .git/shallow")),
- OPT_CALLBACK_F(0, "refmap", NULL, N_("refmap"),
- N_("specify fetch refmap"), PARSE_OPT_NONEG, parse_refmap_arg),
- OPT_STRING_LIST('o', "server-option", &server_options, N_("server-specific"), N_("option to transmit")),
- OPT_SET_INT('4', "ipv4", &family, N_("use IPv4 addresses only"),
- TRANSPORT_FAMILY_IPV4),
- OPT_SET_INT('6', "ipv6", &family, N_("use IPv6 addresses only"),
- TRANSPORT_FAMILY_IPV6),
- OPT_STRING_LIST(0, "negotiation-tip", &negotiation_tip, N_("revision"),
- N_("report that we have only objects reachable from this object")),
- OPT_BOOL(0, "negotiate-only", &negotiate_only,
- N_("do not fetch a packfile; instead, print ancestors of negotiation tips")),
- OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
- OPT_BOOL(0, "auto-maintenance", &enable_auto_gc,
- N_("run 'maintenance --auto' after fetching")),
- OPT_BOOL(0, "auto-gc", &enable_auto_gc,
- N_("run 'maintenance --auto' after fetching")),
- OPT_BOOL(0, "show-forced-updates", &fetch_show_forced_updates,
- N_("check for forced-updates on all updated branches")),
- OPT_BOOL(0, "write-commit-graph", &fetch_write_commit_graph,
- N_("write the commit-graph after fetching")),
- OPT_BOOL(0, "stdin", &stdin_refspecs,
- N_("accept refspecs from stdin")),
- OPT_END()
-};
-
static void unlock_pack(unsigned int flags)
{
if (gtransport)
@@ -2147,13 +2054,109 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,
int cmd_fetch(int argc, const char **argv, const char *prefix)
{
- int i;
const char *bundle_uri;
+ const char *submodule_prefix = "";
enum display_format display_format = DISPLAY_FORMAT_UNKNOWN;
struct string_list list = STRING_LIST_INIT_DUP;
struct remote *remote = NULL;
+ int all = 0, multiple = 0;
int result = 0;
int prune_tags_ok = 1;
+ int enable_auto_gc = 1;
+ int unshallow = 0;
+ int max_jobs = -1;
+ int recurse_submodules_cli = RECURSE_SUBMODULES_DEFAULT;
+ int recurse_submodules_default = RECURSE_SUBMODULES_ON_DEMAND;
+ int fetch_write_commit_graph = -1;
+ int stdin_refspecs = 0;
+ int negotiate_only = 0;
+ int i;
+
+ struct option builtin_fetch_options[] = {
+ OPT__VERBOSITY(&verbosity),
+ OPT_BOOL(0, "all", &all,
+ N_("fetch from all remotes")),
+ OPT_BOOL(0, "set-upstream", &set_upstream,
+ N_("set upstream for git pull/fetch")),
+ OPT_BOOL('a', "append", &append,
+ N_("append to .git/FETCH_HEAD instead of overwriting")),
+ OPT_BOOL(0, "atomic", &atomic_fetch,
+ N_("use atomic transaction to update references")),
+ OPT_STRING(0, "upload-pack", &upload_pack, N_("path"),
+ N_("path to upload pack on remote end")),
+ OPT__FORCE(&force, N_("force overwrite of local reference"), 0),
+ OPT_BOOL('m', "multiple", &multiple,
+ N_("fetch from multiple remotes")),
+ OPT_SET_INT('t', "tags", &tags,
+ N_("fetch all tags and associated objects"), TAGS_SET),
+ OPT_SET_INT('n', NULL, &tags,
+ N_("do not fetch all tags (--no-tags)"), TAGS_UNSET),
+ OPT_INTEGER('j', "jobs", &max_jobs,
+ N_("number of submodules fetched in parallel")),
+ OPT_BOOL(0, "prefetch", &prefetch,
+ N_("modify the refspec to place all refs within refs/prefetch/")),
+ OPT_BOOL('p', "prune", &prune,
+ N_("prune remote-tracking branches no longer on remote")),
+ OPT_BOOL('P', "prune-tags", &prune_tags,
+ N_("prune local tags no longer on remote and clobber changed tags")),
+ OPT_CALLBACK_F(0, "recurse-submodules", &recurse_submodules_cli, N_("on-demand"),
+ N_("control recursive fetching of submodules"),
+ PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
+ OPT_BOOL(0, "dry-run", &dry_run,
+ N_("dry run")),
+ OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
+ N_("write fetched references to the FETCH_HEAD file")),
+ OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
+ OPT_BOOL('u', "update-head-ok", &update_head_ok,
+ N_("allow updating of HEAD ref")),
+ OPT_BOOL(0, "progress", &progress, N_("force progress reporting")),
+ OPT_STRING(0, "depth", &depth, N_("depth"),
+ N_("deepen history of shallow clone")),
+ OPT_STRING(0, "shallow-since", &deepen_since, N_("time"),
+ N_("deepen history of shallow repository based on time")),
+ OPT_STRING_LIST(0, "shallow-exclude", &deepen_not, N_("revision"),
+ N_("deepen history of shallow clone, excluding rev")),
+ OPT_INTEGER(0, "deepen", &deepen_relative,
+ N_("deepen history of shallow clone")),
+ OPT_SET_INT_F(0, "unshallow", &unshallow,
+ N_("convert to a complete repository"),
+ 1, PARSE_OPT_NONEG),
+ OPT_SET_INT_F(0, "refetch", &refetch,
+ N_("re-fetch without negotiating common commits"),
+ 1, PARSE_OPT_NONEG),
+ { OPTION_STRING, 0, "submodule-prefix", &submodule_prefix, N_("dir"),
+ N_("prepend this to submodule path output"), PARSE_OPT_HIDDEN },
+ OPT_CALLBACK_F(0, "recurse-submodules-default",
+ &recurse_submodules_default, N_("on-demand"),
+ N_("default for recursive fetching of submodules "
+ "(lower priority than config files)"),
+ PARSE_OPT_HIDDEN, option_fetch_parse_recurse_submodules),
+ OPT_BOOL(0, "update-shallow", &update_shallow,
+ N_("accept refs that update .git/shallow")),
+ OPT_CALLBACK_F(0, "refmap", NULL, N_("refmap"),
+ N_("specify fetch refmap"), PARSE_OPT_NONEG, parse_refmap_arg),
+ OPT_STRING_LIST('o', "server-option", &server_options, N_("server-specific"), N_("option to transmit")),
+ OPT_SET_INT('4', "ipv4", &family, N_("use IPv4 addresses only"),
+ TRANSPORT_FAMILY_IPV4),
+ OPT_SET_INT('6', "ipv6", &family, N_("use IPv6 addresses only"),
+ TRANSPORT_FAMILY_IPV6),
+ OPT_STRING_LIST(0, "negotiation-tip", &negotiation_tip, N_("revision"),
+ N_("report that we have only objects reachable from this object")),
+ OPT_BOOL(0, "negotiate-only", &negotiate_only,
+ N_("do not fetch a packfile; instead, print ancestors of negotiation tips")),
+ OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
+ OPT_BOOL(0, "auto-maintenance", &enable_auto_gc,
+ N_("run 'maintenance --auto' after fetching")),
+ OPT_BOOL(0, "auto-gc", &enable_auto_gc,
+ N_("run 'maintenance --auto' after fetching")),
+ OPT_BOOL(0, "show-forced-updates", &fetch_show_forced_updates,
+ N_("check for forced-updates on all updated branches")),
+ OPT_BOOL(0, "write-commit-graph", &fetch_write_commit_graph,
+ N_("write the commit-graph after fetching")),
+ OPT_BOOL(0, "stdin", &stdin_refspecs,
+ N_("accept refspecs from stdin")),
+ OPT_END()
+ };
packet_trace_identity("fetch");
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v4 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-05-09 13:01 ` [PATCH v4 " Patrick Steinhardt
` (6 preceding siblings ...)
2023-05-09 13:02 ` [PATCH v4 7/8] fetch: move option related variables " Patrick Steinhardt
@ 2023-05-09 13:02 ` Patrick Steinhardt
2023-05-09 20:43 ` Junio C Hamano
7 siblings, 1 reply; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-09 13:02 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 20347 bytes --]
The output of git-fetch(1) is obviously designed for consumption by
users, only: we neatly columnize data, we abbreviate reference names, we
print neat arrows and we don't provide information about actual object
IDs that have changed. This makes the output format basically unusable
in the context of scripted invocations of git-fetch(1) that want to
learn about the exact changes that the command performs.
Introduce a new machine-parseable "porcelain" output format that is
supposed to fix this shortcoming. This output format is intended to
provide information about every reference that is about to be updated,
the old object ID that the reference has been pointing to and the new
object ID it will be updated to. Furthermore, the output format provides
the same flags as the human-readable format to indicate basic conditions
for each reference update like whether it was a fast-forward update, a
branch deletion, a rejected update or others.
The output format is quite simple:
```
<flag> <old-object-id> <new-object-id> <local-reference>\n
```
We assume two conditions which are generally true:
- The old and new object IDs have fixed known widths and cannot
contain spaces.
- References cannot contain newlines.
With these assumptions, the output format becomes unambiguously
parseable. Furthermore, given that this output is designed to be
consumed by scripts, the machine-readable data is printed to stdout
instead of stderr like the human-readable output is. This is mostly done
so that other data printed to stderr, like error messages or progress
meters, don't interfere with the parseable data.
A notable ommission here is that the output format does not include the
remote from which a reference was fetched, which might be important
information especially in the context of multi-remote fetches. But as
such a format would require us to print the remote for every single
reference update due to parallelizable fetches it feels wasteful for the
most likely usecase, which is when fetching from a single remote.
In a similar spirit, a second restriction is that this cannot be used
with `--recurse-submodules`. This is because any reference updates would
be ambiguous without also printing the repository in which the update
happens.
Considering that both multi-remote and submodule fetches are user-facing
features, using them in conjunction with `--porcelain` that is intended
for scripting purposes is likely not going to be useful in the majority
of cases. With that in mind these restrictions feel acceptable. If
usecases for either of these come up in the future though it is easy
enough to add a new "porcelain-v2" format that adds this information.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
Documentation/fetch-options.txt | 7 ++
Documentation/git-fetch.txt | 9 ++
builtin/fetch.c | 91 +++++++++++++++++----
t/t5574-fetch-output.sh | 140 ++++++++++++++++++++++++++++++++
4 files changed, 229 insertions(+), 18 deletions(-)
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 622bd84768..41fc7ca3c6 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -78,6 +78,13 @@ linkgit:git-config[1].
--dry-run::
Show what would be done, without making any changes.
+--porcelain::
+ Print the output to standard output in an easy-to-parse format for
+ scripts. See section OUTPUT in linkgit:git-fetch[1] for details.
++
+This is incompatible with `--recurse-submodules=[yes|on-demand]` and takes
+precedence over the `fetch.output` config option.
+
ifndef::git-pull[]
--[no-]write-fetch-head::
Write the list of remote refs fetched in the `FETCH_HEAD`
diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index fba66f1460..f123139c58 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -204,6 +204,15 @@ representing the status of a single ref. Each line is of the form:
<flag> <summary> <from> -> <to> [<reason>]
-------------------------------
+When using `--porcelain`, the output format is intended to be
+machine-parseable. In contrast to the human-readable output formats it
+thus prints to standard output instead of standard error. Each line is
+of the form:
+
+-------------------------------
+<flag> <old-object-id> <new-object-id> <local-reference>
+-------------------------------
+
The status of up-to-date refs is shown only if the --verbose option is
used.
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 820ec7285c..187c4d373c 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -52,6 +52,7 @@ enum display_format {
DISPLAY_FORMAT_UNKNOWN = 0,
DISPLAY_FORMAT_FULL,
DISPLAY_FORMAT_COMPACT,
+ DISPLAY_FORMAT_PORCELAIN,
};
struct display_state {
@@ -745,6 +746,9 @@ static void display_state_init(struct display_state *display_state, struct ref *
break;
}
+ case DISPLAY_FORMAT_PORCELAIN:
+ /* We don't need to precompute anything here. */
+ break;
default:
BUG("unexpected display format %d", display_state->format);
}
@@ -815,8 +819,12 @@ static void print_compact(struct display_state *display_state,
static void display_ref_update(struct display_state *display_state, char code,
const char *summary, const char *error,
const char *remote, const char *local,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
int summary_width)
{
+ FILE *f = stderr;
+
if (verbosity < 0)
return;
@@ -849,12 +857,17 @@ static void display_ref_update(struct display_state *display_state, char code,
break;
}
+ case DISPLAY_FORMAT_PORCELAIN:
+ strbuf_addf(&display_state->buf, "%c %s %s %s", code,
+ oid_to_hex(old_oid), oid_to_hex(new_oid), local);
+ f = stdout;
+ break;
default:
BUG("unexpected display format %d", display_state->format);
};
strbuf_addch(&display_state->buf, '\n');
- fputs(display_state->buf.buf, stderr);
+ fputs(display_state->buf.buf, f);
}
static int update_local_ref(struct ref *ref,
@@ -872,7 +885,8 @@ static int update_local_ref(struct ref *ref,
if (oideq(&ref->old_oid, &ref->new_oid)) {
if (verbosity > 0)
display_ref_update(display_state, '=', _("[up to date]"), NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 0;
}
@@ -885,7 +899,8 @@ static int update_local_ref(struct ref *ref,
*/
display_ref_update(display_state, '!', _("[rejected]"),
_("can't fetch into checked-out branch"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 1;
}
@@ -896,12 +911,14 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("updating tag", ref, transaction, 0);
display_ref_update(display_state, r ? '!' : 't', _("[tag update]"),
r ? _("unable to update local ref") : NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"),
_("would clobber existing tag"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 1;
}
}
@@ -934,7 +951,8 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref(msg, ref, transaction, 0);
display_ref_update(display_state, r ? '!' : '*', what,
r ? _("unable to update local ref") : NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return r;
}
@@ -956,7 +974,8 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("fast-forward", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : ' ', quickref.buf,
r ? _("unable to update local ref") : NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
strbuf_release(&quickref);
return r;
} else if (force || ref->force) {
@@ -968,12 +987,14 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("forced-update", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : '+', quickref.buf,
r ? _("unable to update local ref") : _("forced update"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
strbuf_release(&quickref);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"), _("non-fast-forward"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 1;
}
}
@@ -1214,7 +1235,9 @@ static int store_updated_refs(struct display_state *display_state,
display_ref_update(display_state, '*',
*kind ? kind : "branch", NULL,
rm->name,
- "FETCH_HEAD", summary_width);
+ "FETCH_HEAD",
+ &rm->new_oid, &rm->old_oid,
+ summary_width);
}
}
}
@@ -1354,6 +1377,7 @@ static int prune_refs(struct display_state *display_state,
for (ref = stale_refs; ref; ref = ref->next) {
display_ref_update(display_state, '-', _("[deleted]"), NULL,
_("(none)"), ref->name,
+ &ref->new_oid, &ref->old_oid,
summary_width);
warn_dangling_symref(stderr, dangling_msg, ref->name);
}
@@ -1786,7 +1810,8 @@ static int add_remote_or_group(const char *name, struct string_list *list)
return 1;
}
-static void add_options_to_argv(struct strvec *argv)
+static void add_options_to_argv(struct strvec *argv,
+ enum display_format format)
{
if (dry_run)
strvec_push(argv, "--dry-run");
@@ -1822,6 +1847,8 @@ static void add_options_to_argv(struct strvec *argv)
strvec_push(argv, "--ipv6");
if (!write_fetch_head)
strvec_push(argv, "--no-write-fetch-head");
+ if (format == DISPLAY_FORMAT_PORCELAIN)
+ strvec_pushf(argv, "--porcelain");
}
/* Fetch multiple remotes in parallel */
@@ -1830,6 +1857,7 @@ struct parallel_fetch_state {
const char **argv;
struct string_list *remotes;
int next, result;
+ enum display_format format;
};
static int fetch_next_remote(struct child_process *cp,
@@ -1849,7 +1877,7 @@ static int fetch_next_remote(struct child_process *cp,
strvec_push(&cp->args, remote);
cp->git_cmd = 1;
- if (verbosity >= 0)
+ if (verbosity >= 0 && state->format != DISPLAY_FORMAT_PORCELAIN)
printf(_("Fetching %s\n"), remote);
return 1;
@@ -1881,7 +1909,8 @@ static int fetch_finished(int result, struct strbuf *out,
return 0;
}
-static int fetch_multiple(struct string_list *list, int max_children)
+static int fetch_multiple(struct string_list *list, int max_children,
+ enum display_format format)
{
int i, result = 0;
struct strvec argv = STRVEC_INIT;
@@ -1894,10 +1923,10 @@ static int fetch_multiple(struct string_list *list, int max_children)
strvec_pushl(&argv, "fetch", "--append", "--no-auto-gc",
"--no-write-commit-graph", NULL);
- add_options_to_argv(&argv);
+ add_options_to_argv(&argv, format);
if (max_children != 1 && list->nr != 1) {
- struct parallel_fetch_state state = { argv.v, list, 0, 0 };
+ struct parallel_fetch_state state = { argv.v, list, 0, 0, format };
const struct run_process_parallel_opts opts = {
.tr2_category = "fetch",
.tr2_label = "parallel/fetch",
@@ -1921,7 +1950,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
strvec_pushv(&cmd.args, argv.v);
strvec_push(&cmd.args, name);
- if (verbosity >= 0)
+ if (verbosity >= 0 && format != DISPLAY_FORMAT_PORCELAIN)
printf(_("Fetching %s\n"), name);
cmd.git_cmd = 1;
if (run_command(&cmd)) {
@@ -2052,6 +2081,13 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,
return exit_code;
}
+static int opt_parse_porcelain(const struct option *opt, const char *arg, int unset)
+{
+ enum display_format *format = opt->value;
+ *format = DISPLAY_FORMAT_PORCELAIN;
+ return 0;
+}
+
int cmd_fetch(int argc, const char **argv, const char *prefix)
{
const char *bundle_uri;
@@ -2104,6 +2140,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
OPT_BOOL(0, "dry-run", &dry_run,
N_("dry run")),
+ OPT_CALLBACK_F(0, "porcelain", &display_format, NULL, N_("machine-readable output"),
+ PARSE_OPT_NOARG|PARSE_OPT_NONEG, opt_parse_porcelain),
OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
N_("write fetched references to the FETCH_HEAD file")),
OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
@@ -2222,6 +2260,23 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
fetch_config_from_gitmodules(sfjc, rs);
}
+ if (display_format == DISPLAY_FORMAT_PORCELAIN) {
+ switch (recurse_submodules_cli) {
+ case RECURSE_SUBMODULES_OFF:
+ case RECURSE_SUBMODULES_DEFAULT:
+ /*
+ * Reference updates in submodules would be ambiguous
+ * in porcelain mode, so we reject this combination.
+ */
+ recurse_submodules = RECURSE_SUBMODULES_OFF;
+ break;
+
+ default:
+ die(_("options '%s' and '%s' cannot be used together"),
+ "--porcelain", "--recurse-submodules");
+ }
+ }
+
if (negotiate_only && !negotiation_tip.nr)
die(_("--negotiate-only needs one or more --negotiation-tip=*"));
@@ -2341,7 +2396,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
max_children = fetch_parallel_config;
/* TODO should this also die if we have a previous partial-clone? */
- result = fetch_multiple(&list, max_children);
+ result = fetch_multiple(&list, max_children, display_format);
}
@@ -2363,7 +2418,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
if (max_children < 0)
max_children = fetch_parallel_config;
- add_options_to_argv(&options);
+ add_options_to_argv(&options, display_format);
result = fetch_submodules(the_repository,
&options,
submodule_prefix,
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index 6e0f7e0046..695624ac08 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -56,6 +56,122 @@ test_expect_success 'fetch compact output' '
test_cmp expect actual
'
+test_expect_success 'fetch porcelain output' '
+ test_when_finished "rm -rf porcelain" &&
+
+ # Set up a bunch of references that we can use to demonstrate different
+ # kinds of flag symbols in the output format.
+ MAIN_OLD=$(git rev-parse HEAD) &&
+ git branch "fast-forward" &&
+ git branch "deleted-branch" &&
+ git checkout -b force-updated &&
+ test_commit --no-tag force-update-old &&
+ FORCE_UPDATED_OLD=$(git rev-parse HEAD) &&
+ git checkout main &&
+
+ # Clone and pre-seed the repositories. We fetch references into two
+ # namespaces so that we can test that rejected and force-updated
+ # references are reported properly.
+ refspecs="refs/heads/*:refs/unforced/* +refs/heads/*:refs/forced/*" &&
+ git clone . porcelain &&
+ git -C porcelain fetch origin $refspecs &&
+
+ # Now that we have set up the client repositories we can change our
+ # local references.
+ git branch new-branch &&
+ git branch -d deleted-branch &&
+ git checkout fast-forward &&
+ test_commit --no-tag fast-forward-new &&
+ FAST_FORWARD_NEW=$(git rev-parse HEAD) &&
+ git checkout force-updated &&
+ git reset --hard HEAD~ &&
+ test_commit --no-tag force-update-new &&
+ FORCE_UPDATED_NEW=$(git rev-parse HEAD) &&
+
+ cat >expect <<-EOF &&
+ - $MAIN_OLD $ZERO_OID refs/forced/deleted-branch
+ - $MAIN_OLD $ZERO_OID refs/unforced/deleted-branch
+ $MAIN_OLD $FAST_FORWARD_NEW refs/unforced/fast-forward
+ ! $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/unforced/force-updated
+ * $ZERO_OID $MAIN_OLD refs/unforced/new-branch
+ $MAIN_OLD $FAST_FORWARD_NEW refs/forced/fast-forward
+ + $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/forced/force-updated
+ * $ZERO_OID $MAIN_OLD refs/forced/new-branch
+ $MAIN_OLD $FAST_FORWARD_NEW refs/remotes/origin/fast-forward
+ + $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/remotes/origin/force-updated
+ * $ZERO_OID $MAIN_OLD refs/remotes/origin/new-branch
+ EOF
+
+ # Execute a dry-run fetch first. We do this to assert that the dry-run
+ # and non-dry-run fetches produces the same output. Execution of the
+ # fetch is expected to fail as we have a rejected reference update.
+ test_must_fail git -C porcelain fetch \
+ --porcelain --dry-run --prune origin $refspecs >actual &&
+ test_cmp expect actual &&
+
+ # And now we perform a non-dry-run fetch.
+ test_must_fail git -C porcelain fetch \
+ --porcelain --prune origin $refspecs >actual 2>stderr &&
+ test_cmp expect actual &&
+ test_must_be_empty stderr
+'
+
+test_expect_success 'fetch porcelain with multiple remotes' '
+ test_when_finished "rm -rf porcelain" &&
+
+ git switch --create multiple-remotes &&
+ git clone . porcelain &&
+ git -C porcelain remote add second-remote "$PWD" &&
+ git -C porcelain fetch second-remote &&
+
+ test_commit --no-tag multi-commit &&
+ old_commit=$(git rev-parse HEAD~) &&
+ new_commit=$(git rev-parse HEAD) &&
+
+ cat >expect <<-EOF &&
+ $old_commit $new_commit refs/remotes/origin/multiple-remotes
+ $old_commit $new_commit refs/remotes/second-remote/multiple-remotes
+ EOF
+
+ git -C porcelain fetch --porcelain --all >actual 2>stderr &&
+ test_cmp expect actual &&
+ test_must_be_empty stderr
+'
+
+test_expect_success 'fetch porcelain refuses to work with submodules' '
+ test_when_finished "rm -rf porcelain" &&
+
+ cat >expect <<-EOF &&
+ fatal: options ${SQ}--porcelain${SQ} and ${SQ}--recurse-submodules${SQ} cannot be used together
+ EOF
+
+ git init porcelain &&
+ test_must_fail git -C porcelain fetch --porcelain --recurse-submodules=yes 2>stderr &&
+ test_cmp expect stderr &&
+
+ test_must_fail git -C porcelain fetch --porcelain --recurse-submodules=on-demand 2>stderr &&
+ test_cmp expect stderr
+'
+
+test_expect_success 'fetch porcelain overrides fetch.output config' '
+ test_when_finished "rm -rf porcelain" &&
+
+ git switch --create config-override &&
+ git clone . porcelain &&
+ test_commit new-commit &&
+ old_commit=$(git rev-parse HEAD~) &&
+ new_commit=$(git rev-parse HEAD) &&
+
+ cat >expect <<-EOF &&
+ $old_commit $new_commit refs/remotes/origin/config-override
+ * $ZERO_OID $new_commit refs/tags/new-commit
+ EOF
+
+ git -C porcelain -c fetch.output=compact fetch --porcelain >stdout 2>stderr &&
+ test_must_be_empty stderr &&
+ test_cmp expect stdout
+'
+
test_expect_success 'fetch output with HEAD' '
test_when_finished "rm -rf head" &&
git clone . head &&
@@ -85,6 +201,30 @@ test_expect_success 'fetch output with HEAD' '
test_cmp expect actual.err
'
+test_expect_success 'fetch porcelain output with HEAD' '
+ test_when_finished "rm -rf head" &&
+ git clone . head &&
+ COMMIT_ID=$(git rev-parse HEAD) &&
+
+ git -C head fetch --porcelain --dry-run origin HEAD >actual &&
+ cat >expect <<-EOF &&
+ * $ZERO_OID $COMMIT_ID FETCH_HEAD
+ EOF
+ test_cmp expect actual &&
+
+ git -C head fetch --porcelain origin HEAD >actual &&
+ test_cmp expect actual &&
+
+ git -C head fetch --porcelain --dry-run origin HEAD:foo >actual &&
+ cat >expect <<-EOF &&
+ * $ZERO_OID $COMMIT_ID refs/heads/foo
+ EOF
+ test_cmp expect actual &&
+
+ git -C head fetch --porcelain origin HEAD:foo >actual &&
+ test_cmp expect actual
+'
+
test_expect_success '--no-show-forced-updates' '
mkdir forced-updates &&
(
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* Re: [PATCH v4 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches
2023-05-09 13:02 ` [PATCH v4 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches Patrick Steinhardt
@ 2023-05-09 17:49 ` Junio C Hamano
2023-05-09 18:27 ` Glen Choo
0 siblings, 1 reply; 120+ messages in thread
From: Junio C Hamano @ 2023-05-09 17:49 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> When running `git fetch --no-recurse-submodules`, the exectation is that
> we don't fetch any submodules. And while this works for fetches of a
> single remote, it doesn't when fetching multiple remotes at once. The
> result is that we do recurse into submodules even though the user has
> explicitly asked us not to.
>
> This is because while we pass on `--recurse-submodules={yes,on-demand}`
> if specified by the user, we don't pass on `--no-recurse-submodules` to
> the subprocess spawned to perform the submodule fetch.
>
> Fix this by also forwarding this flag as expected.
Makes sense.
> +test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" '
> + test_when_finished "rm -rf src_clone" &&
> +
> + git clone --recurse-submodules src src_clone &&
> + (
> + cd src_clone &&
> + git remote add secondary ../src &&
> + git config submodule.recurse true &&
The above two is essential to this test; we are interested in making
sure that --no-recurse-submodules is propagated down even when the
"--all" option is used, and we want another remote for that. We set
the default to recurse, so that passing "--no-recurse-submodules"
would defeat it, but just refraining to pass "--recurse-submodules"
would cause us to recurse.
> + git config fetch.parallel 0 &&
Is this necessary for the purpose of the test, though? It should
not hurt, but we do not require the end-users to set it in real life
for the parallel fetching to work, either, right?
Just being curious.
> + git fetch --all --no-recurse-submodules 2>../actual
> + ) &&
> +
> + cat >expect <<-EOF &&
> + From ../src
> + * [new branch] master -> secondary/master
> + EOF
> + test_cmp expect actual
> +'
In the context of a series that attempts to introduce a new stable
output format for machine consumption, which implies the traditional
output can change to match human users' preference, this test feels
a bit brittle, but let's wait until the end of the series to judge
that.
Looking good. Thanks.
> test_done
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v4 3/8] fetch: add a test to exercise invalid output formats
2023-05-09 13:02 ` [PATCH v4 3/8] fetch: add a test to exercise invalid output formats Patrick Steinhardt
@ 2023-05-09 17:58 ` Junio C Hamano
2023-05-10 12:34 ` Patrick Steinhardt
0 siblings, 1 reply; 120+ messages in thread
From: Junio C Hamano @ 2023-05-09 17:58 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: git
Patrick Steinhardt <ps@pks.im> writes:
> Add a testcase that exercises the logic when an invalid output format is
> passed via the `fetch.output` configuration.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
> t/t5574-fetch-output.sh | 19 +++++++++++++++++++
> 1 file changed, 19 insertions(+)
It makes perfect sense to make sure that invalid input gets rejected
and the command exits with non-zero status, and it is probably a
good thing that the end-user sees a message that explains why the
particular input is rejected (even though it adds one more thing
that needs to be updated when the message gets reworded).
But do we need to insist on no output to the standard output stream
when the command errors out?
Other than that, looking good.
THanks.
> diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
> index f91b654d38..a09750d225 100755
> --- a/t/t5574-fetch-output.sh
> +++ b/t/t5574-fetch-output.sh
> @@ -7,6 +7,25 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
>
> . ./test-lib.sh
>
> +test_expect_success 'fetch with invalid output format configuration' '
> + test_when_finished "rm -rf clone" &&
> + git clone . clone &&
> +
> + test_must_fail git -C clone -c fetch.output= fetch origin >actual.out 2>actual.err &&
> + cat >expect <<-EOF &&
> + fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}${SQ}
> + EOF
> + test_must_be_empty actual.out &&
> + test_cmp expect actual.err &&
> +
> + test_must_fail git -C clone -c fetch.output=garbage fetch origin >actual.out 2>actual.err &&
> + cat >expect <<-EOF &&
> + fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}garbage${SQ}
> + EOF
> + test_must_be_empty actual.out &&
> + test_cmp expect actual.err
> +'
> +
> test_expect_success 'fetch aligned output' '
> git clone . full-output &&
> test_commit looooooooooooong-tag &&
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v4 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches
2023-05-09 17:49 ` Junio C Hamano
@ 2023-05-09 18:27 ` Glen Choo
2023-05-10 12:34 ` Patrick Steinhardt
0 siblings, 1 reply; 120+ messages in thread
From: Glen Choo @ 2023-05-09 18:27 UTC (permalink / raw)
To: Junio C Hamano, Patrick Steinhardt
Cc: git, Felipe Contreras, Jonathan Tan, Jacob Keller
Junio C Hamano <gitster@pobox.com> writes:
>> + git config fetch.parallel 0 &&
>
> Is this necessary for the purpose of the test, though? It should
> not hurt, but we do not require the end-users to set it in real life
> for the parallel fetching to work, either, right?
IIUC it would make the test output deterministic if we fetched from both
remotes. That doesn't happen here though, so I guess it's not doing
anything right now.
>> + git fetch --all --no-recurse-submodules 2>../actual
>> + ) &&
>> +
>> + cat >expect <<-EOF &&
>> + From ../src
>> + * [new branch] master -> secondary/master
>> + EOF
>> + test_cmp expect actual
>> +'
>
> In the context of a series that attempts to introduce a new stable
> output format for machine consumption, which implies the traditional
> output can change to match human users' preference, this test feels
> a bit brittle, but let's wait until the end of the series to judge
> that.
I also find it a bit brittle to assert on the whole output when this
test is about checking that we do not fetch the superproject.
Is there a reason you didn't go with the "grep for submodule lines"
approach in the previous tests? If it's about catching regressions, IMO
your PATCH 2/8 does a good enough job of doing that.
Wondering out loud, I wonder if it makes sense for us to make a bigger
distinction between "tests whose purpose is to guard against unexpected
changes in output" (i.e. snapshot tests) vs "tests that happen to use
output as a way to assert behavior" (i.e. 'regular' behavioral tests).
Many GUI app codebases have such a distinction and have different best
practices around them.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v4 4/8] fetch: fix missing from-reference when fetching HEAD:foo
2023-05-09 13:02 ` [PATCH v4 4/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
@ 2023-05-09 19:28 ` Junio C Hamano
2023-05-10 12:34 ` Patrick Steinhardt
0 siblings, 1 reply; 120+ messages in thread
From: Junio C Hamano @ 2023-05-09 19:28 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> But it is not intended when displaying the updated references and would
> cause us to miss the left-hand side of the displayed reference update:
>
> ```
> $ git fetch origin HEAD:foo
> From https://github.com/git/git
> * [new ref] -> foo
> ```
> The HEAD string is clearly missing from the left-hand side of the arrow,
> which is further stressed by the point that the following commands show
> the left-hand side as expected:
>
> ```
> $ git fetch origin HEAD
> From https://github.com/git/git
> * branch HEAD -> FETCH_HEAD
I do not mind being explicit and showing HEAD in this case for the
sake of consistency.
But speaking for the past developers, it was deliberate to omit what
is common from the output to make it more terse, IIRC, and I think
it is unfair to call it a "BUG".
Back when we wrote git-fetch-script, the output was a lot more
verbose, and through efforts like 165f3902 (git-fetch: more terse
fetch output, 2007-11-03) and numerous others over time, we got to
the current output.
> Note that this patch also changes formatting of the block that computes
> the "kind" and "what" variables. This is done on purpose so that it is
> part of the diff, hopefully making the change easier to comprehend.
Just to help readers, "kind" is the category like branch, tag,
etc. and "what" is the concrete name like 'master' and 'foo'.
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> index 08d7fc7233..6aecf549e8 100644
> --- a/builtin/fetch.c
> +++ b/builtin/fetch.c
> @@ -918,12 +918,14 @@ static void display_ref_update(struct display_state *display_state, char code,
> }
>
> width = (summary_width + strlen(summary) - gettext_width(summary));
> + remote = prettify_refname(remote);
> + local = prettify_refname(local);
>
> strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
> if (!display_state->compact_format)
> - print_remote_to_local(display_state, remote, prettify_refname(local));
> + print_remote_to_local(display_state, remote, local);
> else
> - print_compact(display_state, remote, prettify_refname(local));
> + print_compact(display_state, remote, local);
> if (error)
> strbuf_addf(&display_state->buf, " (%s)", error);
> strbuf_addch(&display_state->buf, '\n');
> @@ -934,7 +936,7 @@ static void display_ref_update(struct display_state *display_state, char code,
> static int update_local_ref(struct ref *ref,
> struct ref_transaction *transaction,
> struct display_state *display_state,
> - const char *remote, const struct ref *remote_ref,
> + const struct ref *remote_ref,
> int summary_width)
> {
> struct commit *current = NULL, *updated;
> @@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,
> if (oideq(&ref->old_oid, &ref->new_oid)) {
> if (verbosity > 0)
> display_ref_update(display_state, '=', _("[up to date]"), NULL,
> - remote, ref->name, summary_width);
> + remote_ref->name, ref->name, summary_width);
Makes sense. The variable "remote" (now removed) holds what to
write to FETCH_HEAD to be used to formulate a merge message by the
caller, but this function is purely to report the ref updates and
has no need to have access to that information.
> @@ -1252,14 +1254,13 @@ static int store_updated_refs(struct display_state *display_state,
> if (!strcmp(rm->name, "HEAD")) {
> kind = "";
> what = "";
> - }
> - else if (skip_prefix(rm->name, "refs/heads/", &what))
> + } else if (skip_prefix(rm->name, "refs/heads/", &what)) {
> kind = "branch";
> - else if (skip_prefix(rm->name, "refs/tags/", &what))
> + } else if (skip_prefix(rm->name, "refs/tags/", &what)) {
> kind = "tag";
> - else if (skip_prefix(rm->name, "refs/remotes/", &what))
> + } else if (skip_prefix(rm->name, "refs/remotes/", &what)) {
> kind = "remote-tracking branch";
> - else {
> + } else {
> kind = "";
> what = rm->name;
> }
This is a bit noisier than necessary. It took me a while until I
realized that this hunk is a no-op.
> @@ -1277,7 +1278,7 @@ static int store_updated_refs(struct display_state *display_state,
> display_state->url_len);
>
> if (ref) {
> - rc |= update_local_ref(ref, transaction, display_state, what,
> + rc |= update_local_ref(ref, transaction, display_state,
> rm, summary_width);
> free(ref);
Good.
> @@ -1288,7 +1289,7 @@ static int store_updated_refs(struct display_state *display_state,
> */
> display_ref_update(display_state, '*',
> *kind ? kind : "branch", NULL,
> - *what ? what : "HEAD",
> + rm->name,
> "FETCH_HEAD", summary_width);
Good, too. The original cleared "what" and then to compensate for
that had a logic to turn it back to "HEAD", but that is all gone by
passing rm->name down.
I think we could pass "rm" and leave it to display_ref_update() what
string to use, if we wanted to further refine the output later.
Then somebody in the future may even want to see "HEAD" to be shown
as an empty string and that can all be done in display_ref_update().
It would fix the inconsistency that "git fetch origin HEAD" reports
"HEAD -> FETCH_HEAD" by hiding "HEAD" just like the case where
fetching "HEAD:foo" does, going in the other direction, I would
think.
Thanks.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v4 5/8] fetch: introduce `display_format` enum
2023-05-09 13:02 ` [PATCH v4 5/8] fetch: introduce `display_format` enum Patrick Steinhardt
@ 2023-05-09 20:19 ` Junio C Hamano
2023-05-10 12:35 ` Patrick Steinhardt
0 siblings, 1 reply; 120+ messages in thread
From: Junio C Hamano @ 2023-05-09 20:19 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> We currently have two different display formats in git-fetch(1) with the
> "full" and "compact" formats. This is tracked with a boolean value that
> simply denotes whether the display format is supposed to be compacted
> or not. This works reasonably well while there are only two formats, but
> we're about to introduce another format that will make this a bit more
> awkward to use.
>
> Introduce a `enum display_format` that is more readily extensible.
Makes sense.
> +enum display_format {
> + DISPLAY_FORMAT_UNKNOWN = 0,
> + DISPLAY_FORMAT_FULL,
> + DISPLAY_FORMAT_COMPACT,
> +};
>
> struct display_state {
> struct strbuf buf;
>
> int refcol_width;
> - int compact_format;
> + enum display_format format;
OK. Preparatory conversion without adding anything new.
> @@ -809,31 +814,42 @@ static void display_state_init(struct display_state *display_state, struct ref *
>
> git_config_get_string_tmp("fetch.output", &format);
> if (!strcasecmp(format, "full"))
> - display_state->compact_format = 0;
> + display_state->format = DISPLAY_FORMAT_FULL;
> else if (!strcasecmp(format, "compact"))
> - display_state->compact_format = 1;
> + display_state->format = DISPLAY_FORMAT_COMPACT;
> else
> die(_("invalid value for '%s': '%s'"),
> "fetch.output", format);
Naturally.
> - display_state->refcol_width = 10;
> - for (rm = ref_map; rm; rm = rm->next) {
> - int width;
> + switch (display_state->format) {
> + case DISPLAY_FORMAT_FULL:
> + case DISPLAY_FORMAT_COMPACT: {
> + struct ref *rm;
>
> - if (rm->status == REF_STATUS_REJECT_SHALLOW ||
> - !rm->peer_ref ||
> - !strcmp(rm->name, "HEAD"))
> - continue;
> + display_state->refcol_width = 10;
> + for (rm = ref_map; rm; rm = rm->next) {
> + int width;
>
> - width = refcol_width(rm, display_state->compact_format);
> + if (rm->status == REF_STATUS_REJECT_SHALLOW ||
> + !rm->peer_ref ||
> + !strcmp(rm->name, "HEAD"))
> + continue;
>
> - /*
> - * Not precise calculation for compact mode because '*' can
> - * appear on the left hand side of '->' and shrink the column
> - * back.
> - */
> - if (display_state->refcol_width < width)
> - display_state->refcol_width = width;
> + width = refcol_width(rm, display_state->format == DISPLAY_FORMAT_COMPACT);
> +
> + /*
> + * Not precise calculation for compact mode because '*' can
> + * appear on the left hand side of '->' and shrink the column
> + * back.
> + */
> + if (display_state->refcol_width < width)
> + display_state->refcol_width = width;
> + }
> +
> + break;
> + }
> + default:
> + BUG("unexpected display format %d", display_state->format);
> }
Due to reindentation, the patch is noisier than what it does (which
should be "nothing, other than allowing another value in the .format
member").
It makes me wonder if it would make it easier to read to move the
bulk of this code to a helper function. If we are to give a name to
what is being done in the above hunk, what would it be? It computes
display->refcol_width in which all records would fit, but presumably
if we are to add more things to be shown per ref and align them in a
simlar way, we would compute widths for these other things there as
well. Perhaps compute_display_alignment() or somesuch?
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v4 6/8] fetch: move display format parsing into main function
2023-05-09 13:02 ` [PATCH v4 6/8] fetch: move display format parsing into main function Patrick Steinhardt
@ 2023-05-09 20:35 ` Junio C Hamano
2023-05-09 22:30 ` Glen Choo
1 sibling, 0 replies; 120+ messages in thread
From: Junio C Hamano @ 2023-05-09 20:35 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> We're about to introduce a output format though that is intended to be
> parseable by machines, for example inside of a script. In that case it
> becomes a bit awkward of an interface if you have to call git-fetch(1)
> with the `fetch.output` config key set. We're thus going to introduce a
> new `--output-format` switch for git-fetch(1) so that the output format
> can be configured more directly.
Good. I was wondering about that code in the context of the
previous patch, especially the error message had a hard-coded
assumption that it comes from a configuration variable.
And the changes to display_state_init() to lift the responsibility
of finding and validating .format out of the function, and the
changes to intermediate functions to pass the .format through the
callchain, are all expected and there was nothing questionable.
> @@ -2157,6 +2149,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
> {
> int i;
> const char *bundle_uri;
> + enum display_format display_format = DISPLAY_FORMAT_UNKNOWN;
> struct string_list list = STRING_LIST_INIT_DUP;
> struct remote *remote = NULL;
> int result = 0;
> @@ -2183,6 +2176,19 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
> argc = parse_options(argc, argv, prefix,
> builtin_fetch_options, builtin_fetch_usage, 0);
>
> + if (display_format == DISPLAY_FORMAT_UNKNOWN) {
> + const char *format = "full";
> +
> + git_config_get_string_tmp("fetch.output", &format);
> + if (!strcasecmp(format, "full"))
> + display_format = DISPLAY_FORMAT_FULL;
> + else if (!strcasecmp(format, "compact"))
> + display_format = DISPLAY_FORMAT_COMPACT;
> + else
> + die(_("invalid value for '%s': '%s'"),
> + "fetch.output", format);
> + }
OK, but isn't the usual way to do this to have configuration parser
before parse_options() and then let parse_options() override
whatever display_format set by it?
That way, we do not have to worry about DISPLAY_FORMAT_UNKNOWN at
all. Just initialize the variable to whatever default format at the
beginning of this function, read "fetch.output" to override it if
the configuration exists, and then let parse_options() to handle
"--output-format" or "--porcelain" or whatever to further override
it.
Thanks.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v4 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-05-09 13:02 ` [PATCH v4 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
@ 2023-05-09 20:43 ` Junio C Hamano
2023-05-10 12:35 ` Patrick Steinhardt
0 siblings, 1 reply; 120+ messages in thread
From: Junio C Hamano @ 2023-05-09 20:43 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> Considering that both multi-remote and submodule fetches are user-facing
> features, using them in conjunction with `--porcelain` that is intended
> for scripting purposes is likely not going to be useful in the majority
> of cases. With that in mind these restrictions feel acceptable. If
> usecases for either of these come up in the future though it is easy
> enough to add a new "porcelain-v2" format that adds this information.
Two steps ago, the proposed log message still mentioned "--output-format",
which may want to be proofread again and revised.
> @@ -1786,7 +1810,8 @@ static int add_remote_or_group(const char *name, struct string_list *list)
> return 1;
> }
>
> -static void add_options_to_argv(struct strvec *argv)
> +static void add_options_to_argv(struct strvec *argv,
> + enum display_format format)
> {
> if (dry_run)
> strvec_push(argv, "--dry-run");
> @@ -1822,6 +1847,8 @@ static void add_options_to_argv(struct strvec *argv)
> strvec_push(argv, "--ipv6");
> if (!write_fetch_head)
> strvec_push(argv, "--no-write-fetch-head");
> + if (format == DISPLAY_FORMAT_PORCELAIN)
> + strvec_pushf(argv, "--porcelain");
> }
Hmph.
[PATCH 9/8] may want to also introduce and pass down the
"--output-format=full/compact" option, but that is clearly outside
of the scope of this step.
> +static int opt_parse_porcelain(const struct option *opt, const char *arg, int unset)
> +{
> + enum display_format *format = opt->value;
> + *format = DISPLAY_FORMAT_PORCELAIN;
> + return 0;
> +}
Lack of "if (unset) ..." worries me. Shouldn't the code allow
git fetch --porcelain --no-porcelain
to defeat an earlier one and revert back to the default?
Other than that the changes in this step were mostly expected and I
didn't spot anything glaringly wrong, but I didn't spend as much
time as I did for the other steps.
Thanks.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v4 6/8] fetch: move display format parsing into main function
2023-05-09 13:02 ` [PATCH v4 6/8] fetch: move display format parsing into main function Patrick Steinhardt
2023-05-09 20:35 ` Junio C Hamano
@ 2023-05-09 22:30 ` Glen Choo
2023-05-10 12:35 ` Patrick Steinhardt
1 sibling, 1 reply; 120+ messages in thread
From: Glen Choo @ 2023-05-09 22:30 UTC (permalink / raw)
To: Patrick Steinhardt, git
Cc: Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> with the `fetch.output` config key set. We're thus going to introduce a
> new `--output-format` switch for git-fetch(1) so that the output format
> can be configured more directly.
This is stale as of v3, since isn't named --output-format any more. Let
me see if there are other instances of this.
(I should have caught this earlier; I only saw it because Junio quoted
it in a reply of his.)
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH v5 0/9] fetch: introduce machine-parseable output
2023-04-19 12:31 [PATCH 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
` (13 preceding siblings ...)
2023-05-09 13:01 ` [PATCH v4 " Patrick Steinhardt
@ 2023-05-10 12:33 ` Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 1/9] fetch: fix `--no-recurse-submodules` with multi-remote fetches Patrick Steinhardt
` (10 more replies)
14 siblings, 11 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-10 12:33 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 26181 bytes --]
Hi,
this is the fourth version of my patch series to introduce a
machine-parseable output format for git-fetch(1). It applies on top of
e9dffbc7f1 (Merge branch 'ps/fetch-ref-update-reporting', 2023-04-06).
Changes compared to v4:
- Patch 1/9: Simplified the test as proposed by Junio and Glen.
- Patch 3/9: Added a test to verify that `git fetch -c fetch.output`
without a value set fails as expected. Also dropped the tests that
checked whether stdout was empty.
- Patch 4/9: Reformulated the commit message to treat the missing
left-hand side of displayed references as an inconsistency instead
of a bug. I've also added a testcase to verify that direct OID
fetches continue to work as expected.
- Patch 5/9: New patch that makes calculation of the table width for
displayed reference updates self-contained in `refcol_width()`.
This is a preparatory refactoring that should make patch 6/9
easier to review.
- Patch 7/9: Refactored the code to parse the "fetch.output" config
variable inside of `git_fetch_config()` before we parse command
line options. Also fixed that the commit message was still
referring to `--output-format=porcelain` instead of the new
`--porcelain` switch.
- Patch 9/9: The `--porcelain` option is now a simple `OPT_BOOL()`
that can be negated. Added a test that `--no-porcelain` works as
expected.
Thanks for your feedback, Junio and Glen!
Patrick
Patrick Steinhardt (9):
fetch: fix `--no-recurse-submodules` with multi-remote fetches
fetch: split out tests for output format
fetch: add a test to exercise invalid output formats
fetch: print left-hand side when fetching HEAD:foo
fetch: refactor calculation of the display table width
fetch: introduce `display_format` enum
fetch: lift up parsing of "fetch.output" config variable
fetch: move option related variables into main function
fetch: introduce machine-parseable "porcelain" output format
Documentation/fetch-options.txt | 7 +
Documentation/git-fetch.txt | 9 +
builtin/fetch.c | 490 +++++++++++++++++++-------------
t/t5510-fetch.sh | 53 ----
t/t5526-fetch-submodules.sh | 13 +
t/t5574-fetch-output.sh | 293 +++++++++++++++++++
6 files changed, 611 insertions(+), 254 deletions(-)
create mode 100755 t/t5574-fetch-output.sh
Range-diff against v4:
1: d82b42ed34 ! 1: 02ee4fab7e fetch: fix `--no-recurse-submodules` with multi-remote fetches
@@ t/t5526-fetch-submodules.sh: test_expect_success 'fetch --all with --recurse-sub
+ cd src_clone &&
+ git remote add secondary ../src &&
+ git config submodule.recurse true &&
-+ git config fetch.parallel 0 &&
-+ git fetch --all --no-recurse-submodules 2>../actual
++ git fetch --all --no-recurse-submodules 2>../fetch-log
+ ) &&
-+
-+ cat >expect <<-EOF &&
-+ From ../src
-+ * [new branch] master -> secondary/master
-+ EOF
-+ test_cmp expect actual
++ ! grep "Fetching submodule" fetch-log
+'
+
test_done
2: 33112dc51a = 2: e459d8a1ec fetch: split out tests for output format
3: 006ea93afb ! 3: d503c425fe fetch: add a test to exercise invalid output formats
@@ t/t5574-fetch-output.sh: export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+ test_when_finished "rm -rf clone" &&
+ git clone . clone &&
+
-+ test_must_fail git -C clone -c fetch.output= fetch origin >actual.out 2>actual.err &&
++ test_must_fail git -C clone -c fetch.output fetch origin 2>actual.err &&
++ cat >expect <<-EOF &&
++ error: missing value for ${SQ}fetch.output${SQ}
++ fatal: unable to parse ${SQ}fetch.output${SQ} from command-line config
++ EOF
++ test_cmp expect actual.err &&
++
++ test_must_fail git -C clone -c fetch.output= fetch origin 2>actual.err &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}${SQ}
+ EOF
-+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err &&
+
-+ test_must_fail git -C clone -c fetch.output=garbage fetch origin >actual.out 2>actual.err &&
++ test_must_fail git -C clone -c fetch.output=garbage fetch origin 2>actual.err &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}garbage${SQ}
+ EOF
-+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err
+'
+
4: e599ea6d33 ! 4: 2cc7318697 fetch: fix missing from-reference when fetching HEAD:foo
@@ Metadata
Author: Patrick Steinhardt <ps@pks.im>
## Commit message ##
- fetch: fix missing from-reference when fetching HEAD:foo
+ fetch: print left-hand side when fetching HEAD:foo
`store_updated_refs()` parses the remote reference for two purposes:
@@ Commit message
name and can thus be used for both cases. But if the remote reference is
HEAD, the parsed remote reference becomes empty. This is intended when
we write the FETCH_HEAD, where we skip writing the note in that case.
- But it is not intended when displaying the updated references and would
- cause us to miss the left-hand side of the displayed reference update:
+ But when displaying the updated references this leads to inconsistent
+ output where the left-hand side of reference updates is missing in some
+ cases:
```
- $ git fetch origin HEAD:foo
- From https://github.com/git/git
- * [new ref] -> foo
- ```
-
- The HEAD string is clearly missing from the left-hand side of the arrow,
- which is further stressed by the point that the following commands show
- the left-hand side as expected:
-
- ```
- $ git fetch origin HEAD
+ $ git fetch origin HEAD HEAD:explicit-head :implicit-head main
From https://github.com/git/git
* branch HEAD -> FETCH_HEAD
-
- $ git fetch origin master
- From https://github.com/git/git
- * branch master -> FETCH_HEAD
- * branch master -> origin/master
+ * [new ref] -> explicit-head
+ * [new ref] -> implicit-head
+ * branch main -> FETCH_HEAD
```
+ This behaviour has existed ever since the table-based output has been
+ introduced for git-fetch(1) via 165f390250 (git-fetch: more terse fetch
+ output, 2007-11-03) and was never explicitly documented either in the
+ commit message or in any of our tests. So while it may not be a bug per
+ se, it feels like a weird inconsistency and not like it was a concious
+ design decision.
+
The logic of how we compute the remote reference name that we ultimately
pass to `display_ref_update()` is not easy to follow. There are three
different cases here:
- When the remote reference name is "HEAD" we set the remote
reference name to the empty string. This is the case that causes
- the bug to occur, where we would indeed want to print "HEAD"
- instead of the empty string. This is what `prettify_refname()`
- would return.
+ the left-hand side to go missing, where we would indeed want to
+ print "HEAD" instead of the empty string. This is what
+ `prettify_refname()` would return.
- When the remote reference name has a well-known prefix then we
strip this prefix. This matches what `prettify_refname()` does.
@@ Commit message
matches what `prettify_refname()` does.
As the return value of `prettify_refname()` would do the correct thing
- for us in all three cases, we can fix the bug by passing through the
- full remote reference name to `display_ref_update()`, which learns to
- call `prettify_refname()`. At the same time, this also simplifies the
- code a bit.
+ for us in all three cases, we can thus fix the inconsistency by passing
+ through the full remote reference name to `display_ref_update()`, which
+ learns to call `prettify_refname()`. At the same time, this also
+ simplifies the code a bit.
Note that this patch also changes formatting of the block that computes
- the "kind" and "what" variables. This is done on purpose so that it is
- part of the diff, hopefully making the change easier to comprehend.
+ the "kind" (which is the category like "branch" or "tag") and "what"
+ (which is the prettified reference name like "master" or "v1.0")
+ variables. This is done on purpose so that it is part of the diff,
+ hopefully making the change easier to comprehend.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err
+'
++
++test_expect_success 'fetch output with object ID' '
++ test_when_finished "rm -rf object-id" &&
++ git clone . object-id &&
++ commit=$(git rev-parse HEAD) &&
++
++ git -C object-id fetch --dry-run origin $commit:object-id >actual.out 2>actual.err &&
++ cat >expect <<-EOF &&
++ From $(test-tool path-utils real_path .)/.
++ * [new ref] $commit -> object-id
++ EOF
++ test_must_be_empty actual.out &&
++ test_cmp expect actual.err &&
++
++ git -C object-id fetch origin $commit:object-id >actual.out 2>actual.err &&
++ test_must_be_empty actual.out &&
++ test_cmp expect actual.err
++'
+
test_expect_success '--no-show-forced-updates' '
mkdir forced-updates &&
-: ---------- > 5: bb1a591c2f fetch: refactor calculation of the display table width
5: 80ac00b0c4 ! 6: 3cac552f5f fetch: introduce `display_format` enum
@@ builtin/fetch.c: enum {
char *url;
int url_len, shown_url;
-@@ builtin/fetch.c: static int refcol_width(const struct ref *ref, int compact_format)
- static void display_state_init(struct display_state *display_state, struct ref *ref_map,
- const char *raw_url)
- {
-- struct ref *rm;
- const char *format = "full";
- int i;
-
@@ builtin/fetch.c: static void display_state_init(struct display_state *display_state, struct ref *
git_config_get_string_tmp("fetch.output", &format);
@@ builtin/fetch.c: static void display_state_init(struct display_state *display_st
die(_("invalid value for '%s': '%s'"),
"fetch.output", format);
-- display_state->refcol_width = 10;
-- for (rm = ref_map; rm; rm = rm->next) {
-- int width;
+- display_state->refcol_width = refcol_width(ref_map, display_state->compact_format);
+ switch (display_state->format) {
+ case DISPLAY_FORMAT_FULL:
-+ case DISPLAY_FORMAT_COMPACT: {
-+ struct ref *rm;
-
-- if (rm->status == REF_STATUS_REJECT_SHALLOW ||
-- !rm->peer_ref ||
-- !strcmp(rm->name, "HEAD"))
-- continue;
-+ display_state->refcol_width = 10;
-+ for (rm = ref_map; rm; rm = rm->next) {
-+ int width;
-
-- width = refcol_width(rm, display_state->compact_format);
-+ if (rm->status == REF_STATUS_REJECT_SHALLOW ||
-+ !rm->peer_ref ||
-+ !strcmp(rm->name, "HEAD"))
-+ continue;
-
-- /*
-- * Not precise calculation for compact mode because '*' can
-- * appear on the left hand side of '->' and shrink the column
-- * back.
-- */
-- if (display_state->refcol_width < width)
-- display_state->refcol_width = width;
-+ width = refcol_width(rm, display_state->format == DISPLAY_FORMAT_COMPACT);
-+
-+ /*
-+ * Not precise calculation for compact mode because '*' can
-+ * appear on the left hand side of '->' and shrink the column
-+ * back.
-+ */
-+ if (display_state->refcol_width < width)
-+ display_state->refcol_width = width;
-+ }
-+
++ case DISPLAY_FORMAT_COMPACT:
++ display_state->refcol_width = refcol_width(ref_map,
++ display_state->format == DISPLAY_FORMAT_COMPACT);
+ break;
-+ }
+ default:
+ BUG("unexpected display format %d", display_state->format);
- }
++ }
}
+ static void display_state_release(struct display_state *display_state)
@@ builtin/fetch.c: static void display_ref_update(struct display_state *display_state, char code,
const char *remote, const char *local,
int summary_width)
6: 826b8b7bc0 ! 7: 0c3dbcd09f fetch: move display format parsing into main function
@@ Metadata
Author: Patrick Steinhardt <ps@pks.im>
## Commit message ##
- fetch: move display format parsing into main function
+ fetch: lift up parsing of "fetch.output" config variable
Parsing the display format happens inside of `display_state_init()`. As
we only need to check for a simple config entry, this is a natural
location to put this code as it means that display-state logic is neatly
contained in a single location.
- We're about to introduce a output format though that is intended to be
- parseable by machines, for example inside of a script. In that case it
- becomes a bit awkward of an interface if you have to call git-fetch(1)
- with the `fetch.output` config key set. We're thus going to introduce a
- new `--output-format` switch for git-fetch(1) so that the output format
- can be configured more directly.
-
- This means we'll have to hook parsing of the display format into the
- command line options parser. Having the code to determine the actual
- output format scattered across two different sites is hard to reason
- about though.
+ We're about to introduce a new "porcelain" output format though that is
+ intended to be parseable by machines, for example inside of a script.
+ This format can be enabled by passing the `--porcelain` switch to
+ git-fetch(1). As a consequence, we'll have to add a second callsite that
+ influences the output format, which will become awkward to handle.
Refactor the code such that callers are expected to pass the display
format that is to be used into `display_state_init()`. This allows us to
@@ Commit message
Signed-off-by: Patrick Steinhardt <ps@pks.im>
## builtin/fetch.c ##
-@@ builtin/fetch.c: static int refcol_width(const struct ref *ref, int compact_format)
+@@ builtin/fetch.c: static int fetch_write_commit_graph = -1;
+ static int stdin_refspecs = 0;
+ static int negotiate_only;
+
++struct fetch_config {
++ enum display_format display_format;
++};
++
+ static int git_fetch_config(const char *k, const char *v, void *cb)
+ {
++ struct fetch_config *fetch_config = cb;
++
+ if (!strcmp(k, "fetch.prune")) {
+ fetch_prune_config = git_config_bool(k, v);
+ return 0;
+@@ builtin/fetch.c: static int git_fetch_config(const char *k, const char *v, void *cb)
+ return 0;
+ }
+
++ if (!strcmp(k, "fetch.output")) {
++ if (!v)
++ return config_error_nonbool(k);
++ else if (!strcasecmp(v, "full"))
++ fetch_config->display_format = DISPLAY_FORMAT_FULL;
++ else if (!strcasecmp(v, "compact"))
++ fetch_config->display_format = DISPLAY_FORMAT_COMPACT;
++ else
++ die(_("invalid value for '%s': '%s'"),
++ "fetch.output", v);
++ }
++
+ return git_default_config(k, v, cb);
+ }
+
+@@ builtin/fetch.c: static int refcol_width(const struct ref *ref_map, int compact_format)
}
static void display_state_init(struct display_state *display_state, struct ref *ref_map,
@@ builtin/fetch.c: static void display_state_init(struct display_state *display_st
-
switch (display_state->format) {
case DISPLAY_FORMAT_FULL:
- case DISPLAY_FORMAT_COMPACT: {
+ case DISPLAY_FORMAT_COMPACT:
@@ builtin/fetch.c: static int backfill_tags(struct display_state *display_state,
}
@@ builtin/fetch.c: static int fetch_one(struct remote *remote, int argc, const cha
sigchain_pop(SIGPIPE);
refspec_clear(&rs);
transport_disconnect(gtransport);
-@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+@@ builtin/fetch.c: static int fetch_one(struct remote *remote, int argc, const char **argv,
+
+ int cmd_fetch(int argc, const char **argv, const char *prefix)
{
++ struct fetch_config config = {
++ .display_format = DISPLAY_FORMAT_FULL,
++ };
int i;
const char *bundle_uri;
-+ enum display_format display_format = DISPLAY_FORMAT_UNKNOWN;
struct string_list list = STRING_LIST_INIT_DUP;
- struct remote *remote = NULL;
- int result = 0;
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
- argc = parse_options(argc, argv, prefix,
- builtin_fetch_options, builtin_fetch_usage, 0);
-
-+ if (display_format == DISPLAY_FORMAT_UNKNOWN) {
-+ const char *format = "full";
-+
-+ git_config_get_string_tmp("fetch.output", &format);
-+ if (!strcasecmp(format, "full"))
-+ display_format = DISPLAY_FORMAT_FULL;
-+ else if (!strcasecmp(format, "compact"))
-+ display_format = DISPLAY_FORMAT_COMPACT;
-+ else
-+ die(_("invalid value for '%s': '%s'"),
-+ "fetch.output", format);
-+ }
-+
- if (recurse_submodules_cli != RECURSE_SUBMODULES_DEFAULT)
- recurse_submodules = recurse_submodules_cli;
+ free(anon);
+ }
+- git_config(git_fetch_config, NULL);
++ git_config(git_fetch_config, &config);
+ if (the_repository->gitdir) {
+ prepare_repo_settings(the_repository);
+ the_repository->settings.command_requires_full_index = 0;
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
} else if (remote) {
if (filter_options.choice || has_promisor_remote())
fetch_one_setup_partial(remote);
- result = fetch_one(remote, argc, argv, prune_tags_ok, stdin_refspecs);
+ result = fetch_one(remote, argc, argv, prune_tags_ok, stdin_refspecs,
-+ display_format);
++ config.display_format);
} else {
int max_children = max_jobs;
7: 20f2e061d6 ! 8: 8e33a08c35 fetch: move option related variables into main function
@@ builtin/fetch.c: static struct string_list deepen_not = STRING_LIST_INIT_NODUP;
-static int stdin_refspecs = 0;
-static int negotiate_only;
- static int git_fetch_config(const char *k, const char *v, void *cb)
- {
+ struct fetch_config {
+ enum display_format display_format;
@@ builtin/fetch.c: static int parse_refmap_arg(const struct option *opt, const char *arg, int unset
return 0;
}
@@ builtin/fetch.c: static int parse_refmap_arg(const struct option *opt, const cha
static void unlock_pack(unsigned int flags)
{
if (gtransport)
-@@ builtin/fetch.c: static int fetch_one(struct remote *remote, int argc, const char **argv,
-
- int cmd_fetch(int argc, const char **argv, const char *prefix)
- {
+@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ struct fetch_config config = {
+ .display_format = DISPLAY_FORMAT_FULL,
+ };
- int i;
- const char *bundle_uri;
+ const char *submodule_prefix = "";
- enum display_format display_format = DISPLAY_FORMAT_UNKNOWN;
+ const char *bundle_uri;
struct string_list list = STRING_LIST_INIT_DUP;
struct remote *remote = NULL;
+ int all = 0, multiple = 0;
8: 24ae381950 ! 9: d49152c220 fetch: introduce machine-parseable "porcelain" output format
@@ builtin/fetch.c: enum display_format {
struct display_state {
@@ builtin/fetch.c: static void display_state_init(struct display_state *display_state, struct ref *
-
+ display_state->refcol_width = refcol_width(ref_map,
+ display_state->format == DISPLAY_FORMAT_COMPACT);
break;
- }
+ case DISPLAY_FORMAT_PORCELAIN:
+ /* We don't need to precompute anything here. */
+ break;
@@ builtin/fetch.c: static int fetch_multiple(struct string_list *list, int max_chi
printf(_("Fetching %s\n"), name);
cmd.git_cmd = 1;
if (run_command(&cmd)) {
-@@ builtin/fetch.c: static int fetch_one(struct remote *remote, int argc, const char **argv,
- return exit_code;
- }
+@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ int fetch_write_commit_graph = -1;
+ int stdin_refspecs = 0;
+ int negotiate_only = 0;
++ int porcelain = 0;
+ int i;
-+static int opt_parse_porcelain(const struct option *opt, const char *arg, int unset)
-+{
-+ enum display_format *format = opt->value;
-+ *format = DISPLAY_FORMAT_PORCELAIN;
-+ return 0;
-+}
-+
- int cmd_fetch(int argc, const char **argv, const char *prefix)
- {
- const char *bundle_uri;
+ struct option builtin_fetch_options[] = {
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
OPT_BOOL(0, "dry-run", &dry_run,
N_("dry run")),
-+ OPT_CALLBACK_F(0, "porcelain", &display_format, NULL, N_("machine-readable output"),
-+ PARSE_OPT_NOARG|PARSE_OPT_NONEG, opt_parse_porcelain),
++ OPT_BOOL(0, "porcelain", &porcelain, N_("machine-readable output")),
OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
N_("write fetched references to the FETCH_HEAD file")),
OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
fetch_config_from_gitmodules(sfjc, rs);
}
-+ if (display_format == DISPLAY_FORMAT_PORCELAIN) {
++
++ if (porcelain) {
+ switch (recurse_submodules_cli) {
+ case RECURSE_SUBMODULES_OFF:
+ case RECURSE_SUBMODULES_DEFAULT:
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ die(_("options '%s' and '%s' cannot be used together"),
+ "--porcelain", "--recurse-submodules");
+ }
++
++ config.display_format = DISPLAY_FORMAT_PORCELAIN;
+ }
+
if (negotiate_only && !negotiation_tip.nr)
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
/* TODO should this also die if we have a previous partial-clone? */
- result = fetch_multiple(&list, max_children);
-+ result = fetch_multiple(&list, max_children, display_format);
++ result = fetch_multiple(&list, max_children, config.display_format);
}
-
+-
+ /*
+ * This is only needed after fetch_one(), which does not fetch
+ * submodules by itself.
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
if (max_children < 0)
max_children = fetch_parallel_config;
- add_options_to_argv(&options);
-+ add_options_to_argv(&options, display_format);
++ add_options_to_argv(&options, config.display_format);
result = fetch_submodules(the_repository,
&options,
submodule_prefix,
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
+ test_must_be_empty stderr &&
+ test_cmp expect stdout
+'
++
++test_expect_success 'fetch --no-porcelain overrides previous --porcelain' '
++ test_when_finished "rm -rf no-porcelain" &&
++
++ git switch --create no-porcelain &&
++ git clone . no-porcelain &&
++ test_commit --no-tag no-porcelain &&
++ old_commit=$(git rev-parse --short HEAD~) &&
++ new_commit=$(git rev-parse --short HEAD) &&
++
++ cat >expect <<-EOF &&
++ From $(test-tool path-utils real_path .)/.
++ $old_commit..$new_commit no-porcelain -> origin/no-porcelain
++ EOF
++
++ git -C no-porcelain fetch --porcelain --no-porcelain >stdout 2>stderr &&
++ test_cmp expect stderr &&
++ test_must_be_empty stdout
++'
+
test_expect_success 'fetch output with HEAD' '
test_when_finished "rm -rf head" &&
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch output with HEAD' '
+ test_cmp expect actual
+'
+
- test_expect_success '--no-show-forced-updates' '
- mkdir forced-updates &&
- (
+ test_expect_success 'fetch output with object ID' '
+ test_when_finished "rm -rf object-id" &&
+ git clone . object-id &&
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH v5 1/9] fetch: fix `--no-recurse-submodules` with multi-remote fetches
2023-05-10 12:33 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Patrick Steinhardt
@ 2023-05-10 12:34 ` Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 2/9] fetch: split out tests for output format Patrick Steinhardt
` (9 subsequent siblings)
10 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-10 12:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 2103 bytes --]
When running `git fetch --no-recurse-submodules`, the exectation is that
we don't fetch any submodules. And while this works for fetches of a
single remote, it doesn't when fetching multiple remotes at once. The
result is that we do recurse into submodules even though the user has
explicitly asked us not to.
This is because while we pass on `--recurse-submodules={yes,on-demand}`
if specified by the user, we don't pass on `--no-recurse-submodules` to
the subprocess spawned to perform the submodule fetch.
Fix this by also forwarding this flag as expected.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 2 ++
t/t5526-fetch-submodules.sh | 13 +++++++++++++
2 files changed, 15 insertions(+)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index c310d89878..08d7fc7233 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1876,6 +1876,8 @@ static void add_options_to_argv(struct strvec *argv)
strvec_push(argv, "--keep");
if (recurse_submodules == RECURSE_SUBMODULES_ON)
strvec_push(argv, "--recurse-submodules");
+ else if (recurse_submodules == RECURSE_SUBMODULES_OFF)
+ strvec_push(argv, "--no-recurse-submodules");
else if (recurse_submodules == RECURSE_SUBMODULES_ON_DEMAND)
strvec_push(argv, "--recurse-submodules=on-demand");
if (tags == TAGS_SET)
diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh
index dcdbe26a08..26e933f93a 100755
--- a/t/t5526-fetch-submodules.sh
+++ b/t/t5526-fetch-submodules.sh
@@ -1180,4 +1180,17 @@ test_expect_success 'fetch --all with --recurse-submodules with multiple' '
test_line_count = 2 fetch-subs
'
+test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" '
+ test_when_finished "rm -rf src_clone" &&
+
+ git clone --recurse-submodules src src_clone &&
+ (
+ cd src_clone &&
+ git remote add secondary ../src &&
+ git config submodule.recurse true &&
+ git fetch --all --no-recurse-submodules 2>../fetch-log
+ ) &&
+ ! grep "Fetching submodule" fetch-log
+'
+
test_done
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v5 2/9] fetch: split out tests for output format
2023-05-10 12:33 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 1/9] fetch: fix `--no-recurse-submodules` with multi-remote fetches Patrick Steinhardt
@ 2023-05-10 12:34 ` Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 3/9] fetch: add a test to exercise invalid output formats Patrick Steinhardt
` (8 subsequent siblings)
10 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-10 12:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 4269 bytes --]
We're about to introduce a new porcelain mode for the output of
git-fetch(1). As part of that we'll be introducing a set of new tests
that only relate to the output of this command.
Split out tests that exercise the output format of git-fetch(1) so that
it becomes easier to verify this functionality as a standalone unit. As
the tests assume that the default branch is called "main" we set up the
corresponding GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME environment variable
accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
t/t5510-fetch.sh | 53 ----------------------------------
t/t5574-fetch-output.sh | 63 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 63 insertions(+), 53 deletions(-)
create mode 100755 t/t5574-fetch-output.sh
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index dc44da9c79..4f289063ce 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -1118,59 +1118,6 @@ test_expect_success 'fetching with auto-gc does not lock up' '
)
'
-test_expect_success 'fetch aligned output' '
- git clone . full-output &&
- test_commit looooooooooooong-tag &&
- (
- cd full-output &&
- git -c fetch.output=full fetch origin >actual 2>&1 &&
- grep -e "->" actual | cut -c 22- >../actual
- ) &&
- cat >expect <<-\EOF &&
- main -> origin/main
- looooooooooooong-tag -> looooooooooooong-tag
- EOF
- test_cmp expect actual
-'
-
-test_expect_success 'fetch compact output' '
- git clone . compact &&
- test_commit extraaa &&
- (
- cd compact &&
- git -c fetch.output=compact fetch origin >actual 2>&1 &&
- grep -e "->" actual | cut -c 22- >../actual
- ) &&
- cat >expect <<-\EOF &&
- main -> origin/*
- extraaa -> *
- EOF
- test_cmp expect actual
-'
-
-test_expect_success '--no-show-forced-updates' '
- mkdir forced-updates &&
- (
- cd forced-updates &&
- git init &&
- test_commit 1 &&
- test_commit 2
- ) &&
- git clone forced-updates forced-update-clone &&
- git clone forced-updates no-forced-update-clone &&
- git -C forced-updates reset --hard HEAD~1 &&
- (
- cd forced-update-clone &&
- git fetch --show-forced-updates origin 2>output &&
- test_i18ngrep "(forced update)" output
- ) &&
- (
- cd no-forced-update-clone &&
- git fetch --no-show-forced-updates origin 2>output &&
- test_i18ngrep ! "(forced update)" output
- )
-'
-
for section in fetch transfer
do
test_expect_success "$section.hideRefs affects connectivity check" '
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
new file mode 100755
index 0000000000..f91b654d38
--- /dev/null
+++ b/t/t5574-fetch-output.sh
@@ -0,0 +1,63 @@
+#!/bin/sh
+
+test_description='git fetch output format'
+
+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+
+. ./test-lib.sh
+
+test_expect_success 'fetch aligned output' '
+ git clone . full-output &&
+ test_commit looooooooooooong-tag &&
+ (
+ cd full-output &&
+ git -c fetch.output=full fetch origin >actual 2>&1 &&
+ grep -e "->" actual | cut -c 22- >../actual
+ ) &&
+ cat >expect <<-\EOF &&
+ main -> origin/main
+ looooooooooooong-tag -> looooooooooooong-tag
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success 'fetch compact output' '
+ git clone . compact &&
+ test_commit extraaa &&
+ (
+ cd compact &&
+ git -c fetch.output=compact fetch origin >actual 2>&1 &&
+ grep -e "->" actual | cut -c 22- >../actual
+ ) &&
+ cat >expect <<-\EOF &&
+ main -> origin/*
+ extraaa -> *
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success '--no-show-forced-updates' '
+ mkdir forced-updates &&
+ (
+ cd forced-updates &&
+ git init &&
+ test_commit 1 &&
+ test_commit 2
+ ) &&
+ git clone forced-updates forced-update-clone &&
+ git clone forced-updates no-forced-update-clone &&
+ git -C forced-updates reset --hard HEAD~1 &&
+ (
+ cd forced-update-clone &&
+ git fetch --show-forced-updates origin 2>output &&
+ test_i18ngrep "(forced update)" output
+ ) &&
+ (
+ cd no-forced-update-clone &&
+ git fetch --no-show-forced-updates origin 2>output &&
+ test_i18ngrep ! "(forced update)" output
+ )
+'
+
+test_done
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v5 3/9] fetch: add a test to exercise invalid output formats
2023-05-10 12:33 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 1/9] fetch: fix `--no-recurse-submodules` with multi-remote fetches Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 2/9] fetch: split out tests for output format Patrick Steinhardt
@ 2023-05-10 12:34 ` Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 4/9] fetch: print left-hand side when fetching HEAD:foo Patrick Steinhardt
` (7 subsequent siblings)
10 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-10 12:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 1487 bytes --]
Add a testcase that exercises the logic when an invalid output format is
passed via the `fetch.output` configuration.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
t/t5574-fetch-output.sh | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index f91b654d38..8a344e6790 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -7,6 +7,30 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
. ./test-lib.sh
+test_expect_success 'fetch with invalid output format configuration' '
+ test_when_finished "rm -rf clone" &&
+ git clone . clone &&
+
+ test_must_fail git -C clone -c fetch.output fetch origin 2>actual.err &&
+ cat >expect <<-EOF &&
+ error: missing value for ${SQ}fetch.output${SQ}
+ fatal: unable to parse ${SQ}fetch.output${SQ} from command-line config
+ EOF
+ test_cmp expect actual.err &&
+
+ test_must_fail git -C clone -c fetch.output= fetch origin 2>actual.err &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}${SQ}
+ EOF
+ test_cmp expect actual.err &&
+
+ test_must_fail git -C clone -c fetch.output=garbage fetch origin 2>actual.err &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}garbage${SQ}
+ EOF
+ test_cmp expect actual.err
+'
+
test_expect_success 'fetch aligned output' '
git clone . full-output &&
test_commit looooooooooooong-tag &&
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v5 4/9] fetch: print left-hand side when fetching HEAD:foo
2023-05-10 12:33 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Patrick Steinhardt
` (2 preceding siblings ...)
2023-05-10 12:34 ` [PATCH v5 3/9] fetch: add a test to exercise invalid output formats Patrick Steinhardt
@ 2023-05-10 12:34 ` Patrick Steinhardt
2023-05-12 0:16 ` Glen Choo
2023-05-13 16:59 ` Jeff King
2023-05-10 12:34 ` [PATCH v5 5/9] fetch: refactor calculation of the display table width Patrick Steinhardt
` (6 subsequent siblings)
10 siblings, 2 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-10 12:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 10282 bytes --]
`store_updated_refs()` parses the remote reference for two purposes:
- It gets used as a note when writing FETCH_HEAD.
- It is passed through to `display_ref_update()` to display
updated references in the following format:
```
* branch master -> master
```
In most cases, the parsed remote reference is the prettified reference
name and can thus be used for both cases. But if the remote reference is
HEAD, the parsed remote reference becomes empty. This is intended when
we write the FETCH_HEAD, where we skip writing the note in that case.
But when displaying the updated references this leads to inconsistent
output where the left-hand side of reference updates is missing in some
cases:
```
$ git fetch origin HEAD HEAD:explicit-head :implicit-head main
From https://github.com/git/git
* branch HEAD -> FETCH_HEAD
* [new ref] -> explicit-head
* [new ref] -> implicit-head
* branch main -> FETCH_HEAD
```
This behaviour has existed ever since the table-based output has been
introduced for git-fetch(1) via 165f390250 (git-fetch: more terse fetch
output, 2007-11-03) and was never explicitly documented either in the
commit message or in any of our tests. So while it may not be a bug per
se, it feels like a weird inconsistency and not like it was a concious
design decision.
The logic of how we compute the remote reference name that we ultimately
pass to `display_ref_update()` is not easy to follow. There are three
different cases here:
- When the remote reference name is "HEAD" we set the remote
reference name to the empty string. This is the case that causes
the left-hand side to go missing, where we would indeed want to
print "HEAD" instead of the empty string. This is what
`prettify_refname()` would return.
- When the remote reference name has a well-known prefix then we
strip this prefix. This matches what `prettify_refname()` does.
- Otherwise, we keep the fully qualified reference name. This also
matches what `prettify_refname()` does.
As the return value of `prettify_refname()` would do the correct thing
for us in all three cases, we can thus fix the inconsistency by passing
through the full remote reference name to `display_ref_update()`, which
learns to call `prettify_refname()`. At the same time, this also
simplifies the code a bit.
Note that this patch also changes formatting of the block that computes
the "kind" (which is the category like "branch" or "tag") and "what"
(which is the prettified reference name like "master" or "v1.0")
variables. This is done on purpose so that it is part of the diff,
hopefully making the change easier to comprehend.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 37 ++++++++++++++++----------------
t/t5574-fetch-output.sh | 47 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 66 insertions(+), 18 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 08d7fc7233..6aecf549e8 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -918,12 +918,14 @@ static void display_ref_update(struct display_state *display_state, char code,
}
width = (summary_width + strlen(summary) - gettext_width(summary));
+ remote = prettify_refname(remote);
+ local = prettify_refname(local);
strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
if (!display_state->compact_format)
- print_remote_to_local(display_state, remote, prettify_refname(local));
+ print_remote_to_local(display_state, remote, local);
else
- print_compact(display_state, remote, prettify_refname(local));
+ print_compact(display_state, remote, local);
if (error)
strbuf_addf(&display_state->buf, " (%s)", error);
strbuf_addch(&display_state->buf, '\n');
@@ -934,7 +936,7 @@ static void display_ref_update(struct display_state *display_state, char code,
static int update_local_ref(struct ref *ref,
struct ref_transaction *transaction,
struct display_state *display_state,
- const char *remote, const struct ref *remote_ref,
+ const struct ref *remote_ref,
int summary_width)
{
struct commit *current = NULL, *updated;
@@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,
if (oideq(&ref->old_oid, &ref->new_oid)) {
if (verbosity > 0)
display_ref_update(display_state, '=', _("[up to date]"), NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 0;
}
@@ -959,7 +961,7 @@ static int update_local_ref(struct ref *ref,
*/
display_ref_update(display_state, '!', _("[rejected]"),
_("can't fetch into checked-out branch"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 1;
}
@@ -970,12 +972,12 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("updating tag", ref, transaction, 0);
display_ref_update(display_state, r ? '!' : 't', _("[tag update]"),
r ? _("unable to update local ref") : NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"),
_("would clobber existing tag"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 1;
}
}
@@ -1008,7 +1010,7 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref(msg, ref, transaction, 0);
display_ref_update(display_state, r ? '!' : '*', what,
r ? _("unable to update local ref") : NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return r;
}
@@ -1030,7 +1032,7 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("fast-forward", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : ' ', quickref.buf,
r ? _("unable to update local ref") : NULL,
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
strbuf_release(&quickref);
return r;
} else if (force || ref->force) {
@@ -1042,12 +1044,12 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("forced-update", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : '+', quickref.buf,
r ? _("unable to update local ref") : _("forced update"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
strbuf_release(&quickref);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"), _("non-fast-forward"),
- remote, ref->name, summary_width);
+ remote_ref->name, ref->name, summary_width);
return 1;
}
}
@@ -1252,14 +1254,13 @@ static int store_updated_refs(struct display_state *display_state,
if (!strcmp(rm->name, "HEAD")) {
kind = "";
what = "";
- }
- else if (skip_prefix(rm->name, "refs/heads/", &what))
+ } else if (skip_prefix(rm->name, "refs/heads/", &what)) {
kind = "branch";
- else if (skip_prefix(rm->name, "refs/tags/", &what))
+ } else if (skip_prefix(rm->name, "refs/tags/", &what)) {
kind = "tag";
- else if (skip_prefix(rm->name, "refs/remotes/", &what))
+ } else if (skip_prefix(rm->name, "refs/remotes/", &what)) {
kind = "remote-tracking branch";
- else {
+ } else {
kind = "";
what = rm->name;
}
@@ -1277,7 +1278,7 @@ static int store_updated_refs(struct display_state *display_state,
display_state->url_len);
if (ref) {
- rc |= update_local_ref(ref, transaction, display_state, what,
+ rc |= update_local_ref(ref, transaction, display_state,
rm, summary_width);
free(ref);
} else if (write_fetch_head || dry_run) {
@@ -1288,7 +1289,7 @@ static int store_updated_refs(struct display_state *display_state,
*/
display_ref_update(display_state, '*',
*kind ? kind : "branch", NULL,
- *what ? what : "HEAD",
+ rm->name,
"FETCH_HEAD", summary_width);
}
}
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index 8a344e6790..9890f6f381 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -61,6 +61,53 @@ test_expect_success 'fetch compact output' '
test_cmp expect actual
'
+test_expect_success 'fetch output with HEAD' '
+ test_when_finished "rm -rf head" &&
+ git clone . head &&
+
+ git -C head fetch --dry-run origin HEAD >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * branch HEAD -> FETCH_HEAD
+ EOF
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err &&
+
+ git -C head fetch origin HEAD >actual.out 2>actual.err &&
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err &&
+
+ git -C head fetch --dry-run origin HEAD:foo >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * [new ref] HEAD -> foo
+ EOF
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err &&
+
+ git -C head fetch origin HEAD:foo >actual.out 2>actual.err &&
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err
+'
+
+test_expect_success 'fetch output with object ID' '
+ test_when_finished "rm -rf object-id" &&
+ git clone . object-id &&
+ commit=$(git rev-parse HEAD) &&
+
+ git -C object-id fetch --dry-run origin $commit:object-id >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * [new ref] $commit -> object-id
+ EOF
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err &&
+
+ git -C object-id fetch origin $commit:object-id >actual.out 2>actual.err &&
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err
+'
+
test_expect_success '--no-show-forced-updates' '
mkdir forced-updates &&
(
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v5 5/9] fetch: refactor calculation of the display table width
2023-05-10 12:33 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Patrick Steinhardt
` (3 preceding siblings ...)
2023-05-10 12:34 ` [PATCH v5 4/9] fetch: print left-hand side when fetching HEAD:foo Patrick Steinhardt
@ 2023-05-10 12:34 ` Patrick Steinhardt
2023-05-12 0:49 ` Glen Choo
2023-05-10 12:34 ` [PATCH v5 6/9] fetch: introduce `display_format` enum Patrick Steinhardt
` (5 subsequent siblings)
10 siblings, 1 reply; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-10 12:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 4182 bytes --]
When displaying reference updates, we try to print the references in a
neat table. As the table's width is determined its contents we thus need
to precalculate the overall width before we can start printing updated
references.
The calculation is driven by `display_state_init()`, which invokes
`refcol_width()` for every reference that is to be printed. This split
is somewhat confusing. For one, we filter references that shall be
attributed to the overall width in both places. And second, we
needlessly recalculate the maximum line length based on the terminal
columns and display format for every reference.
Refactor the code so that the complete width calculations are neatly
contained in `refcol_width()`.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 81 ++++++++++++++++++++++---------------------------
1 file changed, 37 insertions(+), 44 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 6aecf549e8..007eb3693d 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -751,40 +751,51 @@ static int s_update_ref(const char *action,
return ret;
}
-static int refcol_width(const struct ref *ref, int compact_format)
+static int refcol_width(const struct ref *ref_map, int compact_format)
{
- int max, rlen, llen, len;
+ const struct ref *ref;
+ int max, width = 10;
- /* uptodate lines are only shown on high verbosity level */
- if (verbosity <= 0 && oideq(&ref->peer_ref->old_oid, &ref->old_oid))
- return 0;
-
- max = term_columns();
- rlen = utf8_strwidth(prettify_refname(ref->name));
-
- llen = utf8_strwidth(prettify_refname(ref->peer_ref->name));
-
- /*
- * rough estimation to see if the output line is too long and
- * should not be counted (we can't do precise calculation
- * anyway because we don't know if the error explanation part
- * will be printed in update_local_ref)
- */
- if (compact_format) {
- llen = 0;
+ max = term_columns();
+ if (compact_format)
max = max * 2 / 3;
- }
- len = 21 /* flag and summary */ + rlen + 4 /* -> */ + llen;
- if (len >= max)
- return 0;
- return rlen;
+ for (ref = ref_map; ref; ref = ref->next) {
+ int rlen, llen = 0, len;
+
+ if (ref->status == REF_STATUS_REJECT_SHALLOW ||
+ !ref->peer_ref ||
+ !strcmp(ref->name, "HEAD"))
+ continue;
+
+ /* uptodate lines are only shown on high verbosity level */
+ if (verbosity <= 0 && oideq(&ref->peer_ref->old_oid, &ref->old_oid))
+ continue;
+
+ rlen = utf8_strwidth(prettify_refname(ref->name));
+ if (!compact_format)
+ llen = utf8_strwidth(prettify_refname(ref->peer_ref->name));
+
+ /*
+ * rough estimation to see if the output line is too long and
+ * should not be counted (we can't do precise calculation
+ * anyway because we don't know if the error explanation part
+ * will be printed in update_local_ref)
+ */
+ len = 21 /* flag and summary */ + rlen + 4 /* -> */ + llen;
+ if (len >= max)
+ continue;
+
+ if (width < rlen)
+ width = rlen;
+ }
+
+ return width;
}
static void display_state_init(struct display_state *display_state, struct ref *ref_map,
const char *raw_url)
{
- struct ref *rm;
const char *format = "full";
int i;
@@ -816,25 +827,7 @@ static void display_state_init(struct display_state *display_state, struct ref *
die(_("invalid value for '%s': '%s'"),
"fetch.output", format);
- display_state->refcol_width = 10;
- for (rm = ref_map; rm; rm = rm->next) {
- int width;
-
- if (rm->status == REF_STATUS_REJECT_SHALLOW ||
- !rm->peer_ref ||
- !strcmp(rm->name, "HEAD"))
- continue;
-
- width = refcol_width(rm, display_state->compact_format);
-
- /*
- * Not precise calculation for compact mode because '*' can
- * appear on the left hand side of '->' and shrink the column
- * back.
- */
- if (display_state->refcol_width < width)
- display_state->refcol_width = width;
- }
+ display_state->refcol_width = refcol_width(ref_map, display_state->compact_format);
}
static void display_state_release(struct display_state *display_state)
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v5 6/9] fetch: introduce `display_format` enum
2023-05-10 12:33 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Patrick Steinhardt
` (4 preceding siblings ...)
2023-05-10 12:34 ` [PATCH v5 5/9] fetch: refactor calculation of the display table width Patrick Steinhardt
@ 2023-05-10 12:34 ` Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 7/9] fetch: lift up parsing of "fetch.output" config variable Patrick Steinhardt
` (4 subsequent siblings)
10 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-10 12:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 3976 bytes --]
We currently have two different display formats in git-fetch(1) with the
"full" and "compact" formats. This is tracked with a boolean value that
simply denotes whether the display format is supposed to be compacted
or not. This works reasonably well while there are only two formats, but
we're about to introduce another format that will make this a bit more
awkward to use.
Introduce a `enum display_format` that is more readily extensible.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 69 +++++++++++++++++++++++++++++++++----------------
1 file changed, 47 insertions(+), 22 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 007eb3693d..4c58ee503a 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -48,11 +48,17 @@ enum {
TAGS_SET = 2
};
+enum display_format {
+ DISPLAY_FORMAT_UNKNOWN = 0,
+ DISPLAY_FORMAT_FULL,
+ DISPLAY_FORMAT_COMPACT,
+};
+
struct display_state {
struct strbuf buf;
int refcol_width;
- int compact_format;
+ enum display_format format;
char *url;
int url_len, shown_url;
@@ -820,14 +826,22 @@ static void display_state_init(struct display_state *display_state, struct ref *
git_config_get_string_tmp("fetch.output", &format);
if (!strcasecmp(format, "full"))
- display_state->compact_format = 0;
+ display_state->format = DISPLAY_FORMAT_FULL;
else if (!strcasecmp(format, "compact"))
- display_state->compact_format = 1;
+ display_state->format = DISPLAY_FORMAT_COMPACT;
else
die(_("invalid value for '%s': '%s'"),
"fetch.output", format);
- display_state->refcol_width = refcol_width(ref_map, display_state->compact_format);
+ switch (display_state->format) {
+ case DISPLAY_FORMAT_FULL:
+ case DISPLAY_FORMAT_COMPACT:
+ display_state->refcol_width = refcol_width(ref_map,
+ display_state->format == DISPLAY_FORMAT_COMPACT);
+ break;
+ default:
+ BUG("unexpected display format %d", display_state->format);
+ }
}
static void display_state_release(struct display_state *display_state)
@@ -897,30 +911,41 @@ static void display_ref_update(struct display_state *display_state, char code,
const char *remote, const char *local,
int summary_width)
{
- int width;
-
if (verbosity < 0)
return;
strbuf_reset(&display_state->buf);
- if (!display_state->shown_url) {
- strbuf_addf(&display_state->buf, _("From %.*s\n"),
- display_state->url_len, display_state->url);
- display_state->shown_url = 1;
+ switch (display_state->format) {
+ case DISPLAY_FORMAT_FULL:
+ case DISPLAY_FORMAT_COMPACT: {
+ int width;
+
+ if (!display_state->shown_url) {
+ strbuf_addf(&display_state->buf, _("From %.*s\n"),
+ display_state->url_len, display_state->url);
+ display_state->shown_url = 1;
+ }
+
+ width = (summary_width + strlen(summary) - gettext_width(summary));
+ remote = prettify_refname(remote);
+ local = prettify_refname(local);
+
+ strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
+
+ if (display_state->format != DISPLAY_FORMAT_COMPACT)
+ print_remote_to_local(display_state, remote, local);
+ else
+ print_compact(display_state, remote, local);
+
+ if (error)
+ strbuf_addf(&display_state->buf, " (%s)", error);
+
+ break;
}
-
- width = (summary_width + strlen(summary) - gettext_width(summary));
- remote = prettify_refname(remote);
- local = prettify_refname(local);
-
- strbuf_addf(&display_state->buf, " %c %-*s ", code, width, summary);
- if (!display_state->compact_format)
- print_remote_to_local(display_state, remote, local);
- else
- print_compact(display_state, remote, local);
- if (error)
- strbuf_addf(&display_state->buf, " (%s)", error);
+ default:
+ BUG("unexpected display format %d", display_state->format);
+ };
strbuf_addch(&display_state->buf, '\n');
fputs(display_state->buf.buf, stderr);
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v5 7/9] fetch: lift up parsing of "fetch.output" config variable
2023-05-10 12:33 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Patrick Steinhardt
` (5 preceding siblings ...)
2023-05-10 12:34 ` [PATCH v5 6/9] fetch: introduce `display_format` enum Patrick Steinhardt
@ 2023-05-10 12:34 ` Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 8/9] fetch: move option related variables into main function Patrick Steinhardt
` (3 subsequent siblings)
10 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-10 12:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 5779 bytes --]
Parsing the display format happens inside of `display_state_init()`. As
we only need to check for a simple config entry, this is a natural
location to put this code as it means that display-state logic is neatly
contained in a single location.
We're about to introduce a new "porcelain" output format though that is
intended to be parseable by machines, for example inside of a script.
This format can be enabled by passing the `--porcelain` switch to
git-fetch(1). As a consequence, we'll have to add a second callsite that
influences the output format, which will become awkward to handle.
Refactor the code such that callers are expected to pass the display
format that is to be used into `display_state_init()`. This allows us to
lift up the code into the main function, where we can then hook it into
command line options parser in a follow-up commit.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 50 +++++++++++++++++++++++++++++++------------------
1 file changed, 32 insertions(+), 18 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 4c58ee503a..439d3c667b 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -104,8 +104,14 @@ static int fetch_write_commit_graph = -1;
static int stdin_refspecs = 0;
static int negotiate_only;
+struct fetch_config {
+ enum display_format display_format;
+};
+
static int git_fetch_config(const char *k, const char *v, void *cb)
{
+ struct fetch_config *fetch_config = cb;
+
if (!strcmp(k, "fetch.prune")) {
fetch_prune_config = git_config_bool(k, v);
return 0;
@@ -144,6 +150,18 @@ static int git_fetch_config(const char *k, const char *v, void *cb)
return 0;
}
+ if (!strcmp(k, "fetch.output")) {
+ if (!v)
+ return config_error_nonbool(k);
+ else if (!strcasecmp(v, "full"))
+ fetch_config->display_format = DISPLAY_FORMAT_FULL;
+ else if (!strcasecmp(v, "compact"))
+ fetch_config->display_format = DISPLAY_FORMAT_COMPACT;
+ else
+ die(_("invalid value for '%s': '%s'"),
+ "fetch.output", v);
+ }
+
return git_default_config(k, v, cb);
}
@@ -800,14 +818,13 @@ static int refcol_width(const struct ref *ref_map, int compact_format)
}
static void display_state_init(struct display_state *display_state, struct ref *ref_map,
- const char *raw_url)
+ const char *raw_url, enum display_format format)
{
- const char *format = "full";
int i;
memset(display_state, 0, sizeof(*display_state));
-
strbuf_init(&display_state->buf, 0);
+ display_state->format = format;
if (raw_url)
display_state->url = transport_anonymize_url(raw_url);
@@ -824,15 +841,6 @@ static void display_state_init(struct display_state *display_state, struct ref *
if (verbosity < 0)
return;
- git_config_get_string_tmp("fetch.output", &format);
- if (!strcasecmp(format, "full"))
- display_state->format = DISPLAY_FORMAT_FULL;
- else if (!strcasecmp(format, "compact"))
- display_state->format = DISPLAY_FORMAT_COMPACT;
- else
- die(_("invalid value for '%s': '%s'"),
- "fetch.output", format);
-
switch (display_state->format) {
case DISPLAY_FORMAT_FULL:
case DISPLAY_FORMAT_COMPACT:
@@ -1605,7 +1613,8 @@ static int backfill_tags(struct display_state *display_state,
}
static int do_fetch(struct transport *transport,
- struct refspec *rs)
+ struct refspec *rs,
+ enum display_format display_format)
{
struct ref_transaction *transaction = NULL;
struct ref *ref_map = NULL;
@@ -1691,7 +1700,7 @@ static int do_fetch(struct transport *transport,
if (retcode)
goto cleanup;
- display_state_init(&display_state, ref_map, transport->url);
+ display_state_init(&display_state, ref_map, transport->url, display_format);
if (atomic_fetch) {
transaction = ref_transaction_begin(&err);
@@ -2069,7 +2078,8 @@ static inline void fetch_one_setup_partial(struct remote *remote)
}
static int fetch_one(struct remote *remote, int argc, const char **argv,
- int prune_tags_ok, int use_stdin_refspecs)
+ int prune_tags_ok, int use_stdin_refspecs,
+ enum display_format display_format)
{
struct refspec rs = REFSPEC_INIT_FETCH;
int i;
@@ -2136,7 +2146,7 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,
sigchain_push_common(unlock_pack_on_signal);
atexit(unlock_pack_atexit);
sigchain_push(SIGPIPE, SIG_IGN);
- exit_code = do_fetch(gtransport, &rs);
+ exit_code = do_fetch(gtransport, &rs, display_format);
sigchain_pop(SIGPIPE);
refspec_clear(&rs);
transport_disconnect(gtransport);
@@ -2146,6 +2156,9 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,
int cmd_fetch(int argc, const char **argv, const char *prefix)
{
+ struct fetch_config config = {
+ .display_format = DISPLAY_FORMAT_FULL,
+ };
int i;
const char *bundle_uri;
struct string_list list = STRING_LIST_INIT_DUP;
@@ -2165,7 +2178,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
free(anon);
}
- git_config(git_fetch_config, NULL);
+ git_config(git_fetch_config, &config);
if (the_repository->gitdir) {
prepare_repo_settings(the_repository);
the_repository->settings.command_requires_full_index = 0;
@@ -2302,7 +2315,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
} else if (remote) {
if (filter_options.choice || has_promisor_remote())
fetch_one_setup_partial(remote);
- result = fetch_one(remote, argc, argv, prune_tags_ok, stdin_refspecs);
+ result = fetch_one(remote, argc, argv, prune_tags_ok, stdin_refspecs,
+ config.display_format);
} else {
int max_children = max_jobs;
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v5 8/9] fetch: move option related variables into main function
2023-05-10 12:33 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Patrick Steinhardt
` (6 preceding siblings ...)
2023-05-10 12:34 ` [PATCH v5 7/9] fetch: lift up parsing of "fetch.output" config variable Patrick Steinhardt
@ 2023-05-10 12:34 ` Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 9/9] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
` (2 subsequent siblings)
10 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-10 12:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 12424 bytes --]
The options of git-fetch(1) which we pass to `parse_options()` are
declared globally in `builtin/fetch.c`. This means we're forced to use
global variables for all the options, which is more likely to cause
confusion than explicitly passing state around.
Refactor the code to move the options into `cmd_fetch()`. Move variables
that were previously forced to be declared globally and which are only
used by `cmd_fetch()` into function-local scope.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fetch.c | 197 ++++++++++++++++++++++++------------------------
1 file changed, 100 insertions(+), 97 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 439d3c667b..7a3d620c4c 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -75,13 +75,12 @@ static int fetch_prune_tags_config = -1; /* unspecified */
static int prune_tags = -1; /* unspecified */
#define PRUNE_TAGS_BY_DEFAULT 0 /* do we prune tags by default? */
-static int all, append, dry_run, force, keep, multiple, update_head_ok;
+static int append, dry_run, force, keep, update_head_ok;
static int write_fetch_head = 1;
static int verbosity, deepen_relative, set_upstream, refetch;
static int progress = -1;
-static int enable_auto_gc = 1;
-static int tags = TAGS_DEFAULT, unshallow, update_shallow, deepen;
-static int max_jobs = -1, submodule_fetch_jobs_config = -1;
+static int tags = TAGS_DEFAULT, update_shallow, deepen;
+static int submodule_fetch_jobs_config = -1;
static int fetch_parallel_config = 1;
static int atomic_fetch;
static enum transport_family family;
@@ -92,17 +91,11 @@ static struct string_list deepen_not = STRING_LIST_INIT_NODUP;
static struct strbuf default_rla = STRBUF_INIT;
static struct transport *gtransport;
static struct transport *gsecondary;
-static const char *submodule_prefix = "";
static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
-static int recurse_submodules_cli = RECURSE_SUBMODULES_DEFAULT;
-static int recurse_submodules_default = RECURSE_SUBMODULES_ON_DEMAND;
static struct refspec refmap = REFSPEC_INIT_FETCH;
static struct list_objects_filter_options filter_options = LIST_OBJECTS_FILTER_INIT;
static struct string_list server_options = STRING_LIST_INIT_DUP;
static struct string_list negotiation_tip = STRING_LIST_INIT_NODUP;
-static int fetch_write_commit_graph = -1;
-static int stdin_refspecs = 0;
-static int negotiate_only;
struct fetch_config {
enum display_format display_format;
@@ -178,92 +171,6 @@ static int parse_refmap_arg(const struct option *opt, const char *arg, int unset
return 0;
}
-static struct option builtin_fetch_options[] = {
- OPT__VERBOSITY(&verbosity),
- OPT_BOOL(0, "all", &all,
- N_("fetch from all remotes")),
- OPT_BOOL(0, "set-upstream", &set_upstream,
- N_("set upstream for git pull/fetch")),
- OPT_BOOL('a', "append", &append,
- N_("append to .git/FETCH_HEAD instead of overwriting")),
- OPT_BOOL(0, "atomic", &atomic_fetch,
- N_("use atomic transaction to update references")),
- OPT_STRING(0, "upload-pack", &upload_pack, N_("path"),
- N_("path to upload pack on remote end")),
- OPT__FORCE(&force, N_("force overwrite of local reference"), 0),
- OPT_BOOL('m', "multiple", &multiple,
- N_("fetch from multiple remotes")),
- OPT_SET_INT('t', "tags", &tags,
- N_("fetch all tags and associated objects"), TAGS_SET),
- OPT_SET_INT('n', NULL, &tags,
- N_("do not fetch all tags (--no-tags)"), TAGS_UNSET),
- OPT_INTEGER('j', "jobs", &max_jobs,
- N_("number of submodules fetched in parallel")),
- OPT_BOOL(0, "prefetch", &prefetch,
- N_("modify the refspec to place all refs within refs/prefetch/")),
- OPT_BOOL('p', "prune", &prune,
- N_("prune remote-tracking branches no longer on remote")),
- OPT_BOOL('P', "prune-tags", &prune_tags,
- N_("prune local tags no longer on remote and clobber changed tags")),
- OPT_CALLBACK_F(0, "recurse-submodules", &recurse_submodules_cli, N_("on-demand"),
- N_("control recursive fetching of submodules"),
- PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
- OPT_BOOL(0, "dry-run", &dry_run,
- N_("dry run")),
- OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
- N_("write fetched references to the FETCH_HEAD file")),
- OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
- OPT_BOOL('u', "update-head-ok", &update_head_ok,
- N_("allow updating of HEAD ref")),
- OPT_BOOL(0, "progress", &progress, N_("force progress reporting")),
- OPT_STRING(0, "depth", &depth, N_("depth"),
- N_("deepen history of shallow clone")),
- OPT_STRING(0, "shallow-since", &deepen_since, N_("time"),
- N_("deepen history of shallow repository based on time")),
- OPT_STRING_LIST(0, "shallow-exclude", &deepen_not, N_("revision"),
- N_("deepen history of shallow clone, excluding rev")),
- OPT_INTEGER(0, "deepen", &deepen_relative,
- N_("deepen history of shallow clone")),
- OPT_SET_INT_F(0, "unshallow", &unshallow,
- N_("convert to a complete repository"),
- 1, PARSE_OPT_NONEG),
- OPT_SET_INT_F(0, "refetch", &refetch,
- N_("re-fetch without negotiating common commits"),
- 1, PARSE_OPT_NONEG),
- { OPTION_STRING, 0, "submodule-prefix", &submodule_prefix, N_("dir"),
- N_("prepend this to submodule path output"), PARSE_OPT_HIDDEN },
- OPT_CALLBACK_F(0, "recurse-submodules-default",
- &recurse_submodules_default, N_("on-demand"),
- N_("default for recursive fetching of submodules "
- "(lower priority than config files)"),
- PARSE_OPT_HIDDEN, option_fetch_parse_recurse_submodules),
- OPT_BOOL(0, "update-shallow", &update_shallow,
- N_("accept refs that update .git/shallow")),
- OPT_CALLBACK_F(0, "refmap", NULL, N_("refmap"),
- N_("specify fetch refmap"), PARSE_OPT_NONEG, parse_refmap_arg),
- OPT_STRING_LIST('o', "server-option", &server_options, N_("server-specific"), N_("option to transmit")),
- OPT_SET_INT('4', "ipv4", &family, N_("use IPv4 addresses only"),
- TRANSPORT_FAMILY_IPV4),
- OPT_SET_INT('6', "ipv6", &family, N_("use IPv6 addresses only"),
- TRANSPORT_FAMILY_IPV6),
- OPT_STRING_LIST(0, "negotiation-tip", &negotiation_tip, N_("revision"),
- N_("report that we have only objects reachable from this object")),
- OPT_BOOL(0, "negotiate-only", &negotiate_only,
- N_("do not fetch a packfile; instead, print ancestors of negotiation tips")),
- OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
- OPT_BOOL(0, "auto-maintenance", &enable_auto_gc,
- N_("run 'maintenance --auto' after fetching")),
- OPT_BOOL(0, "auto-gc", &enable_auto_gc,
- N_("run 'maintenance --auto' after fetching")),
- OPT_BOOL(0, "show-forced-updates", &fetch_show_forced_updates,
- N_("check for forced-updates on all updated branches")),
- OPT_BOOL(0, "write-commit-graph", &fetch_write_commit_graph,
- N_("write the commit-graph after fetching")),
- OPT_BOOL(0, "stdin", &stdin_refspecs,
- N_("accept refspecs from stdin")),
- OPT_END()
-};
-
static void unlock_pack(unsigned int flags)
{
if (gtransport)
@@ -2159,12 +2066,108 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
struct fetch_config config = {
.display_format = DISPLAY_FORMAT_FULL,
};
- int i;
+ const char *submodule_prefix = "";
const char *bundle_uri;
struct string_list list = STRING_LIST_INIT_DUP;
struct remote *remote = NULL;
+ int all = 0, multiple = 0;
int result = 0;
int prune_tags_ok = 1;
+ int enable_auto_gc = 1;
+ int unshallow = 0;
+ int max_jobs = -1;
+ int recurse_submodules_cli = RECURSE_SUBMODULES_DEFAULT;
+ int recurse_submodules_default = RECURSE_SUBMODULES_ON_DEMAND;
+ int fetch_write_commit_graph = -1;
+ int stdin_refspecs = 0;
+ int negotiate_only = 0;
+ int i;
+
+ struct option builtin_fetch_options[] = {
+ OPT__VERBOSITY(&verbosity),
+ OPT_BOOL(0, "all", &all,
+ N_("fetch from all remotes")),
+ OPT_BOOL(0, "set-upstream", &set_upstream,
+ N_("set upstream for git pull/fetch")),
+ OPT_BOOL('a', "append", &append,
+ N_("append to .git/FETCH_HEAD instead of overwriting")),
+ OPT_BOOL(0, "atomic", &atomic_fetch,
+ N_("use atomic transaction to update references")),
+ OPT_STRING(0, "upload-pack", &upload_pack, N_("path"),
+ N_("path to upload pack on remote end")),
+ OPT__FORCE(&force, N_("force overwrite of local reference"), 0),
+ OPT_BOOL('m', "multiple", &multiple,
+ N_("fetch from multiple remotes")),
+ OPT_SET_INT('t', "tags", &tags,
+ N_("fetch all tags and associated objects"), TAGS_SET),
+ OPT_SET_INT('n', NULL, &tags,
+ N_("do not fetch all tags (--no-tags)"), TAGS_UNSET),
+ OPT_INTEGER('j', "jobs", &max_jobs,
+ N_("number of submodules fetched in parallel")),
+ OPT_BOOL(0, "prefetch", &prefetch,
+ N_("modify the refspec to place all refs within refs/prefetch/")),
+ OPT_BOOL('p', "prune", &prune,
+ N_("prune remote-tracking branches no longer on remote")),
+ OPT_BOOL('P', "prune-tags", &prune_tags,
+ N_("prune local tags no longer on remote and clobber changed tags")),
+ OPT_CALLBACK_F(0, "recurse-submodules", &recurse_submodules_cli, N_("on-demand"),
+ N_("control recursive fetching of submodules"),
+ PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
+ OPT_BOOL(0, "dry-run", &dry_run,
+ N_("dry run")),
+ OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
+ N_("write fetched references to the FETCH_HEAD file")),
+ OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
+ OPT_BOOL('u', "update-head-ok", &update_head_ok,
+ N_("allow updating of HEAD ref")),
+ OPT_BOOL(0, "progress", &progress, N_("force progress reporting")),
+ OPT_STRING(0, "depth", &depth, N_("depth"),
+ N_("deepen history of shallow clone")),
+ OPT_STRING(0, "shallow-since", &deepen_since, N_("time"),
+ N_("deepen history of shallow repository based on time")),
+ OPT_STRING_LIST(0, "shallow-exclude", &deepen_not, N_("revision"),
+ N_("deepen history of shallow clone, excluding rev")),
+ OPT_INTEGER(0, "deepen", &deepen_relative,
+ N_("deepen history of shallow clone")),
+ OPT_SET_INT_F(0, "unshallow", &unshallow,
+ N_("convert to a complete repository"),
+ 1, PARSE_OPT_NONEG),
+ OPT_SET_INT_F(0, "refetch", &refetch,
+ N_("re-fetch without negotiating common commits"),
+ 1, PARSE_OPT_NONEG),
+ { OPTION_STRING, 0, "submodule-prefix", &submodule_prefix, N_("dir"),
+ N_("prepend this to submodule path output"), PARSE_OPT_HIDDEN },
+ OPT_CALLBACK_F(0, "recurse-submodules-default",
+ &recurse_submodules_default, N_("on-demand"),
+ N_("default for recursive fetching of submodules "
+ "(lower priority than config files)"),
+ PARSE_OPT_HIDDEN, option_fetch_parse_recurse_submodules),
+ OPT_BOOL(0, "update-shallow", &update_shallow,
+ N_("accept refs that update .git/shallow")),
+ OPT_CALLBACK_F(0, "refmap", NULL, N_("refmap"),
+ N_("specify fetch refmap"), PARSE_OPT_NONEG, parse_refmap_arg),
+ OPT_STRING_LIST('o', "server-option", &server_options, N_("server-specific"), N_("option to transmit")),
+ OPT_SET_INT('4', "ipv4", &family, N_("use IPv4 addresses only"),
+ TRANSPORT_FAMILY_IPV4),
+ OPT_SET_INT('6', "ipv6", &family, N_("use IPv6 addresses only"),
+ TRANSPORT_FAMILY_IPV6),
+ OPT_STRING_LIST(0, "negotiation-tip", &negotiation_tip, N_("revision"),
+ N_("report that we have only objects reachable from this object")),
+ OPT_BOOL(0, "negotiate-only", &negotiate_only,
+ N_("do not fetch a packfile; instead, print ancestors of negotiation tips")),
+ OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
+ OPT_BOOL(0, "auto-maintenance", &enable_auto_gc,
+ N_("run 'maintenance --auto' after fetching")),
+ OPT_BOOL(0, "auto-gc", &enable_auto_gc,
+ N_("run 'maintenance --auto' after fetching")),
+ OPT_BOOL(0, "show-forced-updates", &fetch_show_forced_updates,
+ N_("check for forced-updates on all updated branches")),
+ OPT_BOOL(0, "write-commit-graph", &fetch_write_commit_graph,
+ N_("write the commit-graph after fetching")),
+ OPT_BOOL(0, "stdin", &stdin_refspecs,
+ N_("accept refspecs from stdin")),
+ OPT_END()
+ };
packet_trace_identity("fetch");
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* [PATCH v5 9/9] fetch: introduce machine-parseable "porcelain" output format
2023-05-10 12:33 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Patrick Steinhardt
` (7 preceding siblings ...)
2023-05-10 12:34 ` [PATCH v5 8/9] fetch: move option related variables into main function Patrick Steinhardt
@ 2023-05-10 12:34 ` Patrick Steinhardt
2023-05-12 1:02 ` Glen Choo
2023-05-10 18:05 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Junio C Hamano
2023-05-12 1:09 ` Glen Choo
10 siblings, 1 reply; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-10 12:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 21024 bytes --]
The output of git-fetch(1) is obviously designed for consumption by
users, only: we neatly columnize data, we abbreviate reference names, we
print neat arrows and we don't provide information about actual object
IDs that have changed. This makes the output format basically unusable
in the context of scripted invocations of git-fetch(1) that want to
learn about the exact changes that the command performs.
Introduce a new machine-parseable "porcelain" output format that is
supposed to fix this shortcoming. This output format is intended to
provide information about every reference that is about to be updated,
the old object ID that the reference has been pointing to and the new
object ID it will be updated to. Furthermore, the output format provides
the same flags as the human-readable format to indicate basic conditions
for each reference update like whether it was a fast-forward update, a
branch deletion, a rejected update or others.
The output format is quite simple:
```
<flag> <old-object-id> <new-object-id> <local-reference>\n
```
We assume two conditions which are generally true:
- The old and new object IDs have fixed known widths and cannot
contain spaces.
- References cannot contain newlines.
With these assumptions, the output format becomes unambiguously
parseable. Furthermore, given that this output is designed to be
consumed by scripts, the machine-readable data is printed to stdout
instead of stderr like the human-readable output is. This is mostly done
so that other data printed to stderr, like error messages or progress
meters, don't interfere with the parseable data.
A notable ommission here is that the output format does not include the
remote from which a reference was fetched, which might be important
information especially in the context of multi-remote fetches. But as
such a format would require us to print the remote for every single
reference update due to parallelizable fetches it feels wasteful for the
most likely usecase, which is when fetching from a single remote.
In a similar spirit, a second restriction is that this cannot be used
with `--recurse-submodules`. This is because any reference updates would
be ambiguous without also printing the repository in which the update
happens.
Considering that both multi-remote and submodule fetches are user-facing
features, using them in conjunction with `--porcelain` that is intended
for scripting purposes is likely not going to be useful in the majority
of cases. With that in mind these restrictions feel acceptable. If
usecases for either of these come up in the future though it is easy
enough to add a new "porcelain-v2" format that adds this information.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
Documentation/fetch-options.txt | 7 ++
Documentation/git-fetch.txt | 9 ++
builtin/fetch.c | 88 ++++++++++++++----
t/t5574-fetch-output.sh | 159 ++++++++++++++++++++++++++++++++
4 files changed, 244 insertions(+), 19 deletions(-)
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 622bd84768..41fc7ca3c6 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -78,6 +78,13 @@ linkgit:git-config[1].
--dry-run::
Show what would be done, without making any changes.
+--porcelain::
+ Print the output to standard output in an easy-to-parse format for
+ scripts. See section OUTPUT in linkgit:git-fetch[1] for details.
++
+This is incompatible with `--recurse-submodules=[yes|on-demand]` and takes
+precedence over the `fetch.output` config option.
+
ifndef::git-pull[]
--[no-]write-fetch-head::
Write the list of remote refs fetched in the `FETCH_HEAD`
diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index fba66f1460..f123139c58 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -204,6 +204,15 @@ representing the status of a single ref. Each line is of the form:
<flag> <summary> <from> -> <to> [<reason>]
-------------------------------
+When using `--porcelain`, the output format is intended to be
+machine-parseable. In contrast to the human-readable output formats it
+thus prints to standard output instead of standard error. Each line is
+of the form:
+
+-------------------------------
+<flag> <old-object-id> <new-object-id> <local-reference>
+-------------------------------
+
The status of up-to-date refs is shown only if the --verbose option is
used.
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 7a3d620c4c..462fc86b99 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -52,6 +52,7 @@ enum display_format {
DISPLAY_FORMAT_UNKNOWN = 0,
DISPLAY_FORMAT_FULL,
DISPLAY_FORMAT_COMPACT,
+ DISPLAY_FORMAT_PORCELAIN,
};
struct display_state {
@@ -754,6 +755,9 @@ static void display_state_init(struct display_state *display_state, struct ref *
display_state->refcol_width = refcol_width(ref_map,
display_state->format == DISPLAY_FORMAT_COMPACT);
break;
+ case DISPLAY_FORMAT_PORCELAIN:
+ /* We don't need to precompute anything here. */
+ break;
default:
BUG("unexpected display format %d", display_state->format);
}
@@ -824,8 +828,12 @@ static void print_compact(struct display_state *display_state,
static void display_ref_update(struct display_state *display_state, char code,
const char *summary, const char *error,
const char *remote, const char *local,
+ const struct object_id *old_oid,
+ const struct object_id *new_oid,
int summary_width)
{
+ FILE *f = stderr;
+
if (verbosity < 0)
return;
@@ -858,12 +866,17 @@ static void display_ref_update(struct display_state *display_state, char code,
break;
}
+ case DISPLAY_FORMAT_PORCELAIN:
+ strbuf_addf(&display_state->buf, "%c %s %s %s", code,
+ oid_to_hex(old_oid), oid_to_hex(new_oid), local);
+ f = stdout;
+ break;
default:
BUG("unexpected display format %d", display_state->format);
};
strbuf_addch(&display_state->buf, '\n');
- fputs(display_state->buf.buf, stderr);
+ fputs(display_state->buf.buf, f);
}
static int update_local_ref(struct ref *ref,
@@ -881,7 +894,8 @@ static int update_local_ref(struct ref *ref,
if (oideq(&ref->old_oid, &ref->new_oid)) {
if (verbosity > 0)
display_ref_update(display_state, '=', _("[up to date]"), NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 0;
}
@@ -894,7 +908,8 @@ static int update_local_ref(struct ref *ref,
*/
display_ref_update(display_state, '!', _("[rejected]"),
_("can't fetch into checked-out branch"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 1;
}
@@ -905,12 +920,14 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("updating tag", ref, transaction, 0);
display_ref_update(display_state, r ? '!' : 't', _("[tag update]"),
r ? _("unable to update local ref") : NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"),
_("would clobber existing tag"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 1;
}
}
@@ -943,7 +960,8 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref(msg, ref, transaction, 0);
display_ref_update(display_state, r ? '!' : '*', what,
r ? _("unable to update local ref") : NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return r;
}
@@ -965,7 +983,8 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("fast-forward", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : ' ', quickref.buf,
r ? _("unable to update local ref") : NULL,
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
strbuf_release(&quickref);
return r;
} else if (force || ref->force) {
@@ -977,12 +996,14 @@ static int update_local_ref(struct ref *ref,
r = s_update_ref("forced-update", ref, transaction, 1);
display_ref_update(display_state, r ? '!' : '+', quickref.buf,
r ? _("unable to update local ref") : _("forced update"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
strbuf_release(&quickref);
return r;
} else {
display_ref_update(display_state, '!', _("[rejected]"), _("non-fast-forward"),
- remote_ref->name, ref->name, summary_width);
+ remote_ref->name, ref->name,
+ &ref->old_oid, &ref->new_oid, summary_width);
return 1;
}
}
@@ -1223,7 +1244,9 @@ static int store_updated_refs(struct display_state *display_state,
display_ref_update(display_state, '*',
*kind ? kind : "branch", NULL,
rm->name,
- "FETCH_HEAD", summary_width);
+ "FETCH_HEAD",
+ &rm->new_oid, &rm->old_oid,
+ summary_width);
}
}
}
@@ -1363,6 +1386,7 @@ static int prune_refs(struct display_state *display_state,
for (ref = stale_refs; ref; ref = ref->next) {
display_ref_update(display_state, '-', _("[deleted]"), NULL,
_("(none)"), ref->name,
+ &ref->new_oid, &ref->old_oid,
summary_width);
warn_dangling_symref(stderr, dangling_msg, ref->name);
}
@@ -1795,7 +1819,8 @@ static int add_remote_or_group(const char *name, struct string_list *list)
return 1;
}
-static void add_options_to_argv(struct strvec *argv)
+static void add_options_to_argv(struct strvec *argv,
+ enum display_format format)
{
if (dry_run)
strvec_push(argv, "--dry-run");
@@ -1831,6 +1856,8 @@ static void add_options_to_argv(struct strvec *argv)
strvec_push(argv, "--ipv6");
if (!write_fetch_head)
strvec_push(argv, "--no-write-fetch-head");
+ if (format == DISPLAY_FORMAT_PORCELAIN)
+ strvec_pushf(argv, "--porcelain");
}
/* Fetch multiple remotes in parallel */
@@ -1839,6 +1866,7 @@ struct parallel_fetch_state {
const char **argv;
struct string_list *remotes;
int next, result;
+ enum display_format format;
};
static int fetch_next_remote(struct child_process *cp,
@@ -1858,7 +1886,7 @@ static int fetch_next_remote(struct child_process *cp,
strvec_push(&cp->args, remote);
cp->git_cmd = 1;
- if (verbosity >= 0)
+ if (verbosity >= 0 && state->format != DISPLAY_FORMAT_PORCELAIN)
printf(_("Fetching %s\n"), remote);
return 1;
@@ -1890,7 +1918,8 @@ static int fetch_finished(int result, struct strbuf *out,
return 0;
}
-static int fetch_multiple(struct string_list *list, int max_children)
+static int fetch_multiple(struct string_list *list, int max_children,
+ enum display_format format)
{
int i, result = 0;
struct strvec argv = STRVEC_INIT;
@@ -1903,10 +1932,10 @@ static int fetch_multiple(struct string_list *list, int max_children)
strvec_pushl(&argv, "fetch", "--append", "--no-auto-gc",
"--no-write-commit-graph", NULL);
- add_options_to_argv(&argv);
+ add_options_to_argv(&argv, format);
if (max_children != 1 && list->nr != 1) {
- struct parallel_fetch_state state = { argv.v, list, 0, 0 };
+ struct parallel_fetch_state state = { argv.v, list, 0, 0, format };
const struct run_process_parallel_opts opts = {
.tr2_category = "fetch",
.tr2_label = "parallel/fetch",
@@ -1930,7 +1959,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
strvec_pushv(&cmd.args, argv.v);
strvec_push(&cmd.args, name);
- if (verbosity >= 0)
+ if (verbosity >= 0 && format != DISPLAY_FORMAT_PORCELAIN)
printf(_("Fetching %s\n"), name);
cmd.git_cmd = 1;
if (run_command(&cmd)) {
@@ -2081,6 +2110,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
int fetch_write_commit_graph = -1;
int stdin_refspecs = 0;
int negotiate_only = 0;
+ int porcelain = 0;
int i;
struct option builtin_fetch_options[] = {
@@ -2115,6 +2145,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
OPT_BOOL(0, "dry-run", &dry_run,
N_("dry run")),
+ OPT_BOOL(0, "porcelain", &porcelain, N_("machine-readable output")),
OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
N_("write fetched references to the FETCH_HEAD file")),
OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
@@ -2220,6 +2251,26 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
fetch_config_from_gitmodules(sfjc, rs);
}
+
+ if (porcelain) {
+ switch (recurse_submodules_cli) {
+ case RECURSE_SUBMODULES_OFF:
+ case RECURSE_SUBMODULES_DEFAULT:
+ /*
+ * Reference updates in submodules would be ambiguous
+ * in porcelain mode, so we reject this combination.
+ */
+ recurse_submodules = RECURSE_SUBMODULES_OFF;
+ break;
+
+ default:
+ die(_("options '%s' and '%s' cannot be used together"),
+ "--porcelain", "--recurse-submodules");
+ }
+
+ config.display_format = DISPLAY_FORMAT_PORCELAIN;
+ }
+
if (negotiate_only && !negotiation_tip.nr)
die(_("--negotiate-only needs one or more --negotiation-tip=*"));
@@ -2339,10 +2390,9 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
max_children = fetch_parallel_config;
/* TODO should this also die if we have a previous partial-clone? */
- result = fetch_multiple(&list, max_children);
+ result = fetch_multiple(&list, max_children, config.display_format);
}
-
/*
* This is only needed after fetch_one(), which does not fetch
* submodules by itself.
@@ -2361,7 +2411,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
if (max_children < 0)
max_children = fetch_parallel_config;
- add_options_to_argv(&options);
+ add_options_to_argv(&options, config.display_format);
result = fetch_submodules(the_repository,
&options,
submodule_prefix,
diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh
index 9890f6f381..90e6dcb9a7 100755
--- a/t/t5574-fetch-output.sh
+++ b/t/t5574-fetch-output.sh
@@ -61,6 +61,141 @@ test_expect_success 'fetch compact output' '
test_cmp expect actual
'
+test_expect_success 'fetch porcelain output' '
+ test_when_finished "rm -rf porcelain" &&
+
+ # Set up a bunch of references that we can use to demonstrate different
+ # kinds of flag symbols in the output format.
+ MAIN_OLD=$(git rev-parse HEAD) &&
+ git branch "fast-forward" &&
+ git branch "deleted-branch" &&
+ git checkout -b force-updated &&
+ test_commit --no-tag force-update-old &&
+ FORCE_UPDATED_OLD=$(git rev-parse HEAD) &&
+ git checkout main &&
+
+ # Clone and pre-seed the repositories. We fetch references into two
+ # namespaces so that we can test that rejected and force-updated
+ # references are reported properly.
+ refspecs="refs/heads/*:refs/unforced/* +refs/heads/*:refs/forced/*" &&
+ git clone . porcelain &&
+ git -C porcelain fetch origin $refspecs &&
+
+ # Now that we have set up the client repositories we can change our
+ # local references.
+ git branch new-branch &&
+ git branch -d deleted-branch &&
+ git checkout fast-forward &&
+ test_commit --no-tag fast-forward-new &&
+ FAST_FORWARD_NEW=$(git rev-parse HEAD) &&
+ git checkout force-updated &&
+ git reset --hard HEAD~ &&
+ test_commit --no-tag force-update-new &&
+ FORCE_UPDATED_NEW=$(git rev-parse HEAD) &&
+
+ cat >expect <<-EOF &&
+ - $MAIN_OLD $ZERO_OID refs/forced/deleted-branch
+ - $MAIN_OLD $ZERO_OID refs/unforced/deleted-branch
+ $MAIN_OLD $FAST_FORWARD_NEW refs/unforced/fast-forward
+ ! $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/unforced/force-updated
+ * $ZERO_OID $MAIN_OLD refs/unforced/new-branch
+ $MAIN_OLD $FAST_FORWARD_NEW refs/forced/fast-forward
+ + $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/forced/force-updated
+ * $ZERO_OID $MAIN_OLD refs/forced/new-branch
+ $MAIN_OLD $FAST_FORWARD_NEW refs/remotes/origin/fast-forward
+ + $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/remotes/origin/force-updated
+ * $ZERO_OID $MAIN_OLD refs/remotes/origin/new-branch
+ EOF
+
+ # Execute a dry-run fetch first. We do this to assert that the dry-run
+ # and non-dry-run fetches produces the same output. Execution of the
+ # fetch is expected to fail as we have a rejected reference update.
+ test_must_fail git -C porcelain fetch \
+ --porcelain --dry-run --prune origin $refspecs >actual &&
+ test_cmp expect actual &&
+
+ # And now we perform a non-dry-run fetch.
+ test_must_fail git -C porcelain fetch \
+ --porcelain --prune origin $refspecs >actual 2>stderr &&
+ test_cmp expect actual &&
+ test_must_be_empty stderr
+'
+
+test_expect_success 'fetch porcelain with multiple remotes' '
+ test_when_finished "rm -rf porcelain" &&
+
+ git switch --create multiple-remotes &&
+ git clone . porcelain &&
+ git -C porcelain remote add second-remote "$PWD" &&
+ git -C porcelain fetch second-remote &&
+
+ test_commit --no-tag multi-commit &&
+ old_commit=$(git rev-parse HEAD~) &&
+ new_commit=$(git rev-parse HEAD) &&
+
+ cat >expect <<-EOF &&
+ $old_commit $new_commit refs/remotes/origin/multiple-remotes
+ $old_commit $new_commit refs/remotes/second-remote/multiple-remotes
+ EOF
+
+ git -C porcelain fetch --porcelain --all >actual 2>stderr &&
+ test_cmp expect actual &&
+ test_must_be_empty stderr
+'
+
+test_expect_success 'fetch porcelain refuses to work with submodules' '
+ test_when_finished "rm -rf porcelain" &&
+
+ cat >expect <<-EOF &&
+ fatal: options ${SQ}--porcelain${SQ} and ${SQ}--recurse-submodules${SQ} cannot be used together
+ EOF
+
+ git init porcelain &&
+ test_must_fail git -C porcelain fetch --porcelain --recurse-submodules=yes 2>stderr &&
+ test_cmp expect stderr &&
+
+ test_must_fail git -C porcelain fetch --porcelain --recurse-submodules=on-demand 2>stderr &&
+ test_cmp expect stderr
+'
+
+test_expect_success 'fetch porcelain overrides fetch.output config' '
+ test_when_finished "rm -rf porcelain" &&
+
+ git switch --create config-override &&
+ git clone . porcelain &&
+ test_commit new-commit &&
+ old_commit=$(git rev-parse HEAD~) &&
+ new_commit=$(git rev-parse HEAD) &&
+
+ cat >expect <<-EOF &&
+ $old_commit $new_commit refs/remotes/origin/config-override
+ * $ZERO_OID $new_commit refs/tags/new-commit
+ EOF
+
+ git -C porcelain -c fetch.output=compact fetch --porcelain >stdout 2>stderr &&
+ test_must_be_empty stderr &&
+ test_cmp expect stdout
+'
+
+test_expect_success 'fetch --no-porcelain overrides previous --porcelain' '
+ test_when_finished "rm -rf no-porcelain" &&
+
+ git switch --create no-porcelain &&
+ git clone . no-porcelain &&
+ test_commit --no-tag no-porcelain &&
+ old_commit=$(git rev-parse --short HEAD~) &&
+ new_commit=$(git rev-parse --short HEAD) &&
+
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ $old_commit..$new_commit no-porcelain -> origin/no-porcelain
+ EOF
+
+ git -C no-porcelain fetch --porcelain --no-porcelain >stdout 2>stderr &&
+ test_cmp expect stderr &&
+ test_must_be_empty stdout
+'
+
test_expect_success 'fetch output with HEAD' '
test_when_finished "rm -rf head" &&
git clone . head &&
@@ -90,6 +225,30 @@ test_expect_success 'fetch output with HEAD' '
test_cmp expect actual.err
'
+test_expect_success 'fetch porcelain output with HEAD' '
+ test_when_finished "rm -rf head" &&
+ git clone . head &&
+ COMMIT_ID=$(git rev-parse HEAD) &&
+
+ git -C head fetch --porcelain --dry-run origin HEAD >actual &&
+ cat >expect <<-EOF &&
+ * $ZERO_OID $COMMIT_ID FETCH_HEAD
+ EOF
+ test_cmp expect actual &&
+
+ git -C head fetch --porcelain origin HEAD >actual &&
+ test_cmp expect actual &&
+
+ git -C head fetch --porcelain --dry-run origin HEAD:foo >actual &&
+ cat >expect <<-EOF &&
+ * $ZERO_OID $COMMIT_ID refs/heads/foo
+ EOF
+ test_cmp expect actual &&
+
+ git -C head fetch --porcelain origin HEAD:foo >actual &&
+ test_cmp expect actual
+'
+
test_expect_success 'fetch output with object ID' '
test_when_finished "rm -rf object-id" &&
git clone . object-id &&
--
2.40.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply related [flat|nested] 120+ messages in thread
* Re: [PATCH v4 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches
2023-05-09 18:27 ` Glen Choo
@ 2023-05-10 12:34 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-10 12:34 UTC (permalink / raw)
To: Glen Choo
Cc: Junio C Hamano, git, Felipe Contreras, Jonathan Tan, Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 1923 bytes --]
On Tue, May 09, 2023 at 11:27:35AM -0700, Glen Choo wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
> >> + git config fetch.parallel 0 &&
> >
> > Is this necessary for the purpose of the test, though? It should
> > not hurt, but we do not require the end-users to set it in real life
> > for the parallel fetching to work, either, right?
>
> IIUC it would make the test output deterministic if we fetched from both
> remotes. That doesn't happen here though, so I guess it's not doing
> anything right now.
Right, will drop.
> >> + git fetch --all --no-recurse-submodules 2>../actual
> >> + ) &&
> >> +
> >> + cat >expect <<-EOF &&
> >> + From ../src
> >> + * [new branch] master -> secondary/master
> >> + EOF
> >> + test_cmp expect actual
> >> +'
> >
> > In the context of a series that attempts to introduce a new stable
> > output format for machine consumption, which implies the traditional
> > output can change to match human users' preference, this test feels
> > a bit brittle, but let's wait until the end of the series to judge
> > that.
>
> I also find it a bit brittle to assert on the whole output when this
> test is about checking that we do not fetch the superproject.
>
> Is there a reason you didn't go with the "grep for submodule lines"
> approach in the previous tests? If it's about catching regressions, IMO
> your PATCH 2/8 does a good enough job of doing that.
>
> Wondering out loud, I wonder if it makes sense for us to make a bigger
> distinction between "tests whose purpose is to guard against unexpected
> changes in output" (i.e. snapshot tests) vs "tests that happen to use
> output as a way to assert behavior" (i.e. 'regular' behavioral tests).
> Many GUI app codebases have such a distinction and have different best
> practices around them.
Fair enough, will switch to `! grep "Fetching submodule"`.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v4 3/8] fetch: add a test to exercise invalid output formats
2023-05-09 17:58 ` Junio C Hamano
@ 2023-05-10 12:34 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-10 12:34 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
[-- Attachment #1: Type: text/plain, Size: 928 bytes --]
On Tue, May 09, 2023 at 10:58:33AM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
> > Add a testcase that exercises the logic when an invalid output format is
> > passed via the `fetch.output` configuration.
> >
> > Signed-off-by: Patrick Steinhardt <ps@pks.im>
> > ---
> > t/t5574-fetch-output.sh | 19 +++++++++++++++++++
> > 1 file changed, 19 insertions(+)
>
> It makes perfect sense to make sure that invalid input gets rejected
> and the command exits with non-zero status, and it is probably a
> good thing that the end-user sees a message that explains why the
> particular input is rejected (even though it adds one more thing
> that needs to be updated when the message gets reworded).
>
> But do we need to insist on no output to the standard output stream
> when the command errors out?
>
> Other than that, looking good.
Not really, no. Will change.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v4 4/8] fetch: fix missing from-reference when fetching HEAD:foo
2023-05-09 19:28 ` Junio C Hamano
@ 2023-05-10 12:34 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-10 12:34 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 2320 bytes --]
On Tue, May 09, 2023 at 12:28:16PM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
> > But it is not intended when displaying the updated references and would
> > cause us to miss the left-hand side of the displayed reference update:
> >
> > ```
> > $ git fetch origin HEAD:foo
> > From https://github.com/git/git
> > * [new ref] -> foo
> > ```
> > The HEAD string is clearly missing from the left-hand side of the arrow,
> > which is further stressed by the point that the following commands show
> > the left-hand side as expected:
> >
> > ```
> > $ git fetch origin HEAD
> > From https://github.com/git/git
> > * branch HEAD -> FETCH_HEAD
>
> I do not mind being explicit and showing HEAD in this case for the
> sake of consistency.
>
> But speaking for the past developers, it was deliberate to omit what
> is common from the output to make it more terse, IIRC, and I think
> it is unfair to call it a "BUG".
>
> Back when we wrote git-fetch-script, the output was a lot more
> verbose, and through efforts like 165f3902 (git-fetch: more terse
> fetch output, 2007-11-03) and numerous others over time, we got to
> the current output.
That's fair. It's still not quite clear whether this behaviour is in
fact intentional though. Quoting 165f3902 (git-fetch: more terse
fetch output, 2007-11-03), the weird corner case is not documented:
This makes the fetch output much more terse and prettier on a 80 column
display, based on a consensus reached on the mailing list. Here's an
example output:
Receiving objects: 100% (5439/5439), 1.60 MiB | 636 KiB/s, done.
Resolving deltas: 100% (4604/4604), done.
From git://git.kernel.org/pub/scm/git/git
! [rejected] html -> origin/html (non fast forward)
136e631..f45e867 maint -> origin/maint (fast forward)
9850e2e..44dd7e0 man -> origin/man (fast forward)
3e4bb08..e3d6d56 master -> origin/master (fast forward)
fa3665c..536f64a next -> origin/next (fast forward)
+ 4f6d9d6...768326f pu -> origin/pu (forced update)
* [new branch] todo -> origin/todo
I've reformulated the commit message to talk about an inconsistency
instead of a bug.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v4 5/8] fetch: introduce `display_format` enum
2023-05-09 20:19 ` Junio C Hamano
@ 2023-05-10 12:35 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-10 12:35 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 2822 bytes --]
On Tue, May 09, 2023 at 01:19:29PM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
[snip]
> > - display_state->refcol_width = 10;
> > - for (rm = ref_map; rm; rm = rm->next) {
> > - int width;
> > + switch (display_state->format) {
> > + case DISPLAY_FORMAT_FULL:
> > + case DISPLAY_FORMAT_COMPACT: {
> > + struct ref *rm;
> >
> > - if (rm->status == REF_STATUS_REJECT_SHALLOW ||
> > - !rm->peer_ref ||
> > - !strcmp(rm->name, "HEAD"))
> > - continue;
> > + display_state->refcol_width = 10;
> > + for (rm = ref_map; rm; rm = rm->next) {
> > + int width;
> >
> > - width = refcol_width(rm, display_state->compact_format);
> > + if (rm->status == REF_STATUS_REJECT_SHALLOW ||
> > + !rm->peer_ref ||
> > + !strcmp(rm->name, "HEAD"))
> > + continue;
> >
> > - /*
> > - * Not precise calculation for compact mode because '*' can
> > - * appear on the left hand side of '->' and shrink the column
> > - * back.
> > - */
> > - if (display_state->refcol_width < width)
> > - display_state->refcol_width = width;
> > + width = refcol_width(rm, display_state->format == DISPLAY_FORMAT_COMPACT);
> > +
> > + /*
> > + * Not precise calculation for compact mode because '*' can
> > + * appear on the left hand side of '->' and shrink the column
> > + * back.
> > + */
> > + if (display_state->refcol_width < width)
> > + display_state->refcol_width = width;
> > + }
> > +
> > + break;
> > + }
> > + default:
> > + BUG("unexpected display format %d", display_state->format);
> > }
>
> Due to reindentation, the patch is noisier than what it does (which
> should be "nothing, other than allowing another value in the .format
> member").
>
> It makes me wonder if it would make it easier to read to move the
> bulk of this code to a helper function. If we are to give a name to
> what is being done in the above hunk, what would it be? It computes
> display->refcol_width in which all records would fit, but presumably
> if we are to add more things to be shown per ref and align them in a
> simlar way, we would compute widths for these other things there as
> well. Perhaps compute_display_alignment() or somesuch?
The code already has such a function and calls it `refcol_width()`, but
it only computes the width for a single reference. The most natural
thing to do here would thus be to merge the loop over the references
into that function. This would also allow us to skip some weirdness,
like the fact that we skip some references inside `refcol_width()` while
we skip other references in the `ref_map` loop.
This refactoring is also quite noisy, but it makes the code simpler
overall and will make the patch introducing the enum less so.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v4 6/8] fetch: move display format parsing into main function
2023-05-09 22:30 ` Glen Choo
@ 2023-05-10 12:35 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-10 12:35 UTC (permalink / raw)
To: Glen Choo
Cc: git, Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 566 bytes --]
On Tue, May 09, 2023 at 03:30:33PM -0700, Glen Choo wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
> > with the `fetch.output` config key set. We're thus going to introduce a
> > new `--output-format` switch for git-fetch(1) so that the output format
> > can be configured more directly.
>
> This is stale as of v3, since isn't named --output-format any more. Let
> me see if there are other instances of this.
>
> (I should have caught this earlier; I only saw it because Junio quoted
> it in a reply of his.)
Good catch, fixed now.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v4 8/8] fetch: introduce machine-parseable "porcelain" output format
2023-05-09 20:43 ` Junio C Hamano
@ 2023-05-10 12:35 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-10 12:35 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 2463 bytes --]
On Tue, May 09, 2023 at 01:43:41PM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
> > Considering that both multi-remote and submodule fetches are user-facing
> > features, using them in conjunction with `--porcelain` that is intended
> > for scripting purposes is likely not going to be useful in the majority
> > of cases. With that in mind these restrictions feel acceptable. If
> > usecases for either of these come up in the future though it is easy
> > enough to add a new "porcelain-v2" format that adds this information.
>
> Two steps ago, the proposed log message still mentioned "--output-format",
> which may want to be proofread again and revised.
It's a relict from previous iterations. I've fixed the preceding patch
to talk about `--porcelain` instead.
> > @@ -1786,7 +1810,8 @@ static int add_remote_or_group(const char *name, struct string_list *list)
> > return 1;
> > }
> >
> > -static void add_options_to_argv(struct strvec *argv)
> > +static void add_options_to_argv(struct strvec *argv,
> > + enum display_format format)
> > {
> > if (dry_run)
> > strvec_push(argv, "--dry-run");
> > @@ -1822,6 +1847,8 @@ static void add_options_to_argv(struct strvec *argv)
> > strvec_push(argv, "--ipv6");
> > if (!write_fetch_head)
> > strvec_push(argv, "--no-write-fetch-head");
> > + if (format == DISPLAY_FORMAT_PORCELAIN)
> > + strvec_pushf(argv, "--porcelain");
> > }
>
> Hmph.
>
> [PATCH 9/8] may want to also introduce and pass down the
> "--output-format=full/compact" option, but that is clearly outside
> of the scope of this step.
We don't have `--output-format` now though, but only the `--porcelain`
switch. The only way to configure "full"/"compact" output formats is via
the "fetch.output" config variable right now.
> > +static int opt_parse_porcelain(const struct option *opt, const char *arg, int unset)
> > +{
> > + enum display_format *format = opt->value;
> > + *format = DISPLAY_FORMAT_PORCELAIN;
> > + return 0;
> > +}
>
> Lack of "if (unset) ..." worries me. Shouldn't the code allow
>
> git fetch --porcelain --no-porcelain
>
> to defeat an earlier one and revert back to the default?
We pass `PARSE_OPT_NOARG|PARSE_OPT_NONEG`, so this isn't really much of
a concern. But yeah, it wouldn't allow `--no-porcelain`, and supporting
it is trivial enough to do. I'll change this to a `OPT_BOOL()`.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v5 0/9] fetch: introduce machine-parseable output
2023-05-10 12:33 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Patrick Steinhardt
` (8 preceding siblings ...)
2023-05-10 12:34 ` [PATCH v5 9/9] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
@ 2023-05-10 18:05 ` Junio C Hamano
2023-05-11 11:05 ` Patrick Steinhardt
2023-05-12 1:09 ` Glen Choo
10 siblings, 1 reply; 120+ messages in thread
From: Junio C Hamano @ 2023-05-10 18:05 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> - Patch 3/9: Added a test to verify that `git fetch -c fetch.output`
> without a value set fails as expected. Also dropped the tests that
> checked whether stdout was empty.
Thank you for the attention to this small detail of valueless
configuration variable definition.
> - Patch 4/9: Reformulated the commit message to treat the missing
> left-hand side of displayed references as an inconsistency instead
> of a bug. I've also added a testcase to verify that direct OID
> fetches continue to work as expected.
Again, the direct OID fetch is a good thing to test here. I noticed
that the test added here insists that the standard output stream is
empty when the command errors out, which is not consistent with [3/9]
above.
> - Patch 5/9: New patch that makes calculation of the table width for
> displayed reference updates self-contained in `refcol_width()`.
> This is a preparatory refactoring that should make patch 6/9
> easier to review.
It is an excellent idea to avoid calls to refcol_width() for each
ref that gets shown and make the helper responsible for computing
the required maxwidth. The result indeed has become easier to
follow as you mentioned in your response to my review on the
previous round.
> - Patch 7/9: Refactored the code to parse the "fetch.output" config
> variable inside of `git_fetch_config()` before we parse command
> line options. Also fixed that the commit message was still
> referring to `--output-format=porcelain` instead of the new
> `--porcelain` switch.
As a standalone step, it leaves an impression that the step makes
the way we handle the output-format configuration variable
inconsistent with the way we handle the other configuration
variables, but I think it is a good place to stop for the purpose of
this topic. It lays a good foundation for future clean-up after the
dust settles from this topic---we may want to move global variables
assigned in the git_fetch_config() into the fetch_config structure.
> - Patch 9/9: The `--porcelain` option is now a simple `OPT_BOOL()`
> that can be negated. Added a test that `--no-porcelain` works as
> expected.
OK, this time the familiar "prepare a variable to its default, let
config callback to overwrite it by reading configuration variables,
and then let the command line option override it" is used and the
result easy to follow. I do not think .display_format is never
assigned DISPLAY_FORMAT_UNKNOWN with this change, so [6/9] could
lose the value from the enum, I think. The defensive switch
statement that has BUG() to notice an erroneous caller that pass
values other than DISPLAY_FORMAT_{FULL,COMPACT} is still a good
idea.
Looking good.
Thanks.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v5 0/9] fetch: introduce machine-parseable output
2023-05-10 18:05 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Junio C Hamano
@ 2023-05-11 11:05 ` Patrick Steinhardt
2023-05-11 16:53 ` Junio C Hamano
2023-05-11 17:24 ` Felipe Contreras
0 siblings, 2 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-11 11:05 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 2401 bytes --]
On Wed, May 10, 2023 at 11:05:19AM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
[snip]
> > - Patch 4/9: Reformulated the commit message to treat the missing
> > left-hand side of displayed references as an inconsistency instead
> > of a bug. I've also added a testcase to verify that direct OID
> > fetches continue to work as expected.
>
> Again, the direct OID fetch is a good thing to test here. I noticed
> that the test added here insists that the standard output stream is
> empty when the command errors out, which is not consistent with [3/9]
> above.
I think you misread the test: yes, we do test stdout and stderr
separately and in many cases assert that stdout is in fact empty. But
none of the added tests are about failing commands. So given that:
- The added tests explicitly are about verifying the output format.
- The distinction between stdout and stderr matters.
- The distinction matters even more with the addition of
`--porcelain`.
I think that explicitly verifing both output streams is the correct
thing to do.
> > - Patch 9/9: The `--porcelain` option is now a simple `OPT_BOOL()`
> > that can be negated. Added a test that `--no-porcelain` works as
> > expected.
>
> OK, this time the familiar "prepare a variable to its default, let
> config callback to overwrite it by reading configuration variables,
> and then let the command line option override it" is used and the
> result easy to follow. I do not think .display_format is never
> assigned DISPLAY_FORMAT_UNKNOWN with this change, so [6/9] could
> lose the value from the enum, I think. The defensive switch
> statement that has BUG() to notice an erroneous caller that pass
> values other than DISPLAY_FORMAT_{FULL,COMPACT} is still a good
> idea.
Ah, right, `DISPLAY_FORMAT_UNKNOWN` isn't really needed anymore. I think
it's still good to have valid values of the enum start with `1` so that
it becomes easier to detect cases where we accidentally use a default
initialized variable. But that can be achieved without giving the
default value an explicit name.
I'll refrain from sending a new version just to remove this constant as
it doesn't really feel worth it, though. But please, let me know in case
you disagree and I'll send an updated version.
Thanks
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v5 0/9] fetch: introduce machine-parseable output
2023-05-11 11:05 ` Patrick Steinhardt
@ 2023-05-11 16:53 ` Junio C Hamano
2023-05-11 17:24 ` Felipe Contreras
1 sibling, 0 replies; 120+ messages in thread
From: Junio C Hamano @ 2023-05-11 16:53 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> On Wed, May 10, 2023 at 11:05:19AM -0700, Junio C Hamano wrote:
>> Patrick Steinhardt <ps@pks.im> writes:
> [snip]
>> > - Patch 4/9: Reformulated the commit message to treat the missing
>> > left-hand side of displayed references as an inconsistency instead
>> > of a bug. I've also added a testcase to verify that direct OID
>> > fetches continue to work as expected.
>>
>> Again, the direct OID fetch is a good thing to test here. I noticed
>> that the test added here insists that the standard output stream is
>> empty when the command errors out, which is not consistent with [3/9]
>> above.
>
> I think you misread the test: yes, we do test stdout and stderr
> separately and in many cases assert that stdout is in fact empty. But
> none of the added tests are about failing commands.
Ah, you are absolutely right. Sorry for the noise---I do not have an
objection to ensure that program output in the successful case is
predictable. My main concern is to avoid giving unneeded assurance
in the failing case.
>> assigned DISPLAY_FORMAT_UNKNOWN with this change, so [6/9] could
>> lose the value from the enum, I think. The defensive switch
>> statement that has BUG() to notice an erroneous caller that pass
>> values other than DISPLAY_FORMAT_{FULL,COMPACT} is still a good
>> idea.
>
> Ah, right, `DISPLAY_FORMAT_UNKNOWN` isn't really needed anymore. I think
> it's still good to have valid values of the enum start with `1` so that
> it becomes easier to detect cases where we accidentally use a default
> initialized variable. But that can be achieved without giving the
> default value an explicit name.
Unless we explicitly take advantage of UNKNOWN being 0 and perform
clever defaulting, the result is far safer if you get rid of it, and
instead give 0 to a real choice that is used as the default, for two
reasons: (1) naturally 0 initialization work fine, (2) we have one
less enum constant people (or editor's auto-completer) assign to a
variable by mistake instead of a real one they intended to. I agree
with the last sentence of your paragraph above.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v5 0/9] fetch: introduce machine-parseable output
2023-05-11 11:05 ` Patrick Steinhardt
2023-05-11 16:53 ` Junio C Hamano
@ 2023-05-11 17:24 ` Felipe Contreras
1 sibling, 0 replies; 120+ messages in thread
From: Felipe Contreras @ 2023-05-11 17:24 UTC (permalink / raw)
To: Patrick Steinhardt, Junio C Hamano
Cc: git, Felipe Contreras, Glen Choo, Jonathan Tan, Jacob Keller
Patrick Steinhardt wrote:
> On Wed, May 10, 2023 at 11:05:19AM -0700, Junio C Hamano wrote:
> > OK, this time the familiar "prepare a variable to its default, let
> > config callback to overwrite it by reading configuration variables,
> > and then let the command line option override it" is used and the
> > result easy to follow. I do not think .display_format is never
> > assigned DISPLAY_FORMAT_UNKNOWN with this change, so [6/9] could
> > lose the value from the enum, I think. The defensive switch
> > statement that has BUG() to notice an erroneous caller that pass
> > values other than DISPLAY_FORMAT_{FULL,COMPACT} is still a good
> > idea.
>
> Ah, right, `DISPLAY_FORMAT_UNKNOWN` isn't really needed anymore. I think
> it's still good to have valid values of the enum start with `1` so that
> it becomes easier to detect cases where we accidentally use a default
> initialized variable. But that can be achieved without giving the
> default value an explicit name.
I think it's standard to have an element like that. It could be UNKNOWN, NULL,
INVALID, or even DEFAULT.
For example, you could have code like this:
if (format == DISPLAY_FORMAT_DEFAULT)
format = DISPLAY_FORMAT_FULL;
This would distinguish cases in which the user did not specify a display format
and we choose a default, from those where the user explicitely chose the format
that happens to be the default.
--
Felipe Contreras
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v5 4/9] fetch: print left-hand side when fetching HEAD:foo
2023-05-10 12:34 ` [PATCH v5 4/9] fetch: print left-hand side when fetching HEAD:foo Patrick Steinhardt
@ 2023-05-12 0:16 ` Glen Choo
2023-05-13 16:59 ` Jeff King
1 sibling, 0 replies; 120+ messages in thread
From: Glen Choo @ 2023-05-12 0:16 UTC (permalink / raw)
To: Patrick Steinhardt, git
Cc: Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> This behaviour has existed ever since the table-based output has been
> introduced for git-fetch(1) via 165f390250 (git-fetch: more terse fetch
> output, 2007-11-03) and was never explicitly documented either in the
> commit message or in any of our tests. So while it may not be a bug per
> se, it feels like a weird inconsistency and not like it was a concious
> design decision.
The change seems well-justified. I agree that it does feel strange and
unintended. I was really surprised to learn that it was not a bug.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v5 5/9] fetch: refactor calculation of the display table width
2023-05-10 12:34 ` [PATCH v5 5/9] fetch: refactor calculation of the display table width Patrick Steinhardt
@ 2023-05-12 0:49 ` Glen Choo
0 siblings, 0 replies; 120+ messages in thread
From: Glen Choo @ 2023-05-12 0:49 UTC (permalink / raw)
To: Patrick Steinhardt, git
Cc: Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> When displaying reference updates, we try to print the references in a
> neat table. As the table's width is determined its contents we thus need
> to precalculate the overall width before we can start printing updated
> references.
>
> The calculation is driven by `display_state_init()`, which invokes
> `refcol_width()` for every reference that is to be printed. This split
> is somewhat confusing. For one, we filter references that shall be
> attributed to the overall width in both places. And second, we
> needlessly recalculate the maximum line length based on the terminal
> columns and display format for every reference.
>
> Refactor the code so that the complete width calculations are neatly
> contained in `refcol_width()`.
Through no fault of yours, I have to admit that I found this refactor
quite hard to read. I ended up redoing the refactor and ended up with a
result very similar to yours. That was probably overkill since we have
pretty extensive tests in this area, but I'm quite happy with the change
since it's far more readable.
> - /*
> - * rough estimation to see if the output line is too long and
> - * should not be counted (we can't do precise calculation
> - * anyway because we don't know if the error explanation part
> - * will be printed in update_local_ref)
> - */
> - if (compact_format) {
> - llen = 0;
> + max = term_columns();
> + if (compact_format)
> max = max * 2 / 3;
> - }
> - len = 21 /* flag and summary */ + rlen + 4 /* -> */ + llen;
> - if (len >= max)
> - return 0;
The one thing that changed for the better (vs keeping the lines in the
same order as before) is that this comment that used to be anchored on
"if (compact_format) {"...
> + /*
> + * rough estimation to see if the output line is too long and
> + * should not be counted (we can't do precise calculation
> + * anyway because we don't know if the error explanation part
> + * will be printed in update_local_ref)
> + */
> + len = 21 /* flag and summary */ + rlen + 4 /* -> */ + llen;
> + if (len >= max)
> + continue;
is now more accurately anchored to the check that throws away the line.
Looks good.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v5 9/9] fetch: introduce machine-parseable "porcelain" output format
2023-05-10 12:34 ` [PATCH v5 9/9] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
@ 2023-05-12 1:02 ` Glen Choo
0 siblings, 0 replies; 120+ messages in thread
From: Glen Choo @ 2023-05-12 1:02 UTC (permalink / raw)
To: Patrick Steinhardt, git
Cc: Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> +test_expect_success 'fetch --no-porcelain overrides previous --porcelain' '
> + test_when_finished "rm -rf no-porcelain" &&
> +
> + git switch --create no-porcelain &&
> + git clone . no-porcelain &&
> + test_commit --no-tag no-porcelain &&
> + old_commit=$(git rev-parse --short HEAD~) &&
> + new_commit=$(git rev-parse --short HEAD) &&
> +
> + cat >expect <<-EOF &&
> + From $(test-tool path-utils real_path .)/.
> + $old_commit..$new_commit no-porcelain -> origin/no-porcelain
> + EOF
> +
> + git -C no-porcelain fetch --porcelain --no-porcelain >stdout 2>stderr &&
> + test_cmp expect stderr &&
> + test_must_be_empty stdout
> +'
(This shouldn't block the series) As a matter of taste, it feels like
overkill to construct the whole output to check if --no-porcelain wins.
Maybe in a future cleanup, we would go through the tests in this file
and see where it is really important that we have the exact output and
where it is not.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v5 0/9] fetch: introduce machine-parseable output
2023-05-10 12:33 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Patrick Steinhardt
` (9 preceding siblings ...)
2023-05-10 18:05 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Junio C Hamano
@ 2023-05-12 1:09 ` Glen Choo
2023-05-12 7:16 ` Patrick Steinhardt
10 siblings, 1 reply; 120+ messages in thread
From: Glen Choo @ 2023-05-12 1:09 UTC (permalink / raw)
To: Patrick Steinhardt, git
Cc: Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
Patrick Steinhardt <ps@pks.im> writes:
> Changes compared to v4:
>
> - Patch 1/9: Simplified the test as proposed by Junio and Glen.
>
> - Patch 3/9: Added a test to verify that `git fetch -c fetch.output`
> without a value set fails as expected. Also dropped the tests that
> checked whether stdout was empty.
>
> - Patch 4/9: Reformulated the commit message to treat the missing
> left-hand side of displayed references as an inconsistency instead
> of a bug. I've also added a testcase to verify that direct OID
> fetches continue to work as expected.
>
> - Patch 5/9: New patch that makes calculation of the table width for
> displayed reference updates self-contained in `refcol_width()`.
> This is a preparatory refactoring that should make patch 6/9
> easier to review.
>
> - Patch 7/9: Refactored the code to parse the "fetch.output" config
> variable inside of `git_fetch_config()` before we parse command
> line options. Also fixed that the commit message was still
> referring to `--output-format=porcelain` instead of the new
> `--porcelain` switch.
>
> - Patch 9/9: The `--porcelain` option is now a simple `OPT_BOOL()`
> that can be negated. Added a test that `--no-porcelain` works as
> expected.
I didn't spot any blocking issues in this version, and the various
improvements (especially 6-7/9) are really welcome. I also read through
Junio's comments, but I didn't spot anything that I thought should block
the series, so I'm happy to leave
Reviewed-by: Glen Choo <chooglen@google.com>
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v5 0/9] fetch: introduce machine-parseable output
2023-05-12 1:09 ` Glen Choo
@ 2023-05-12 7:16 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-12 7:16 UTC (permalink / raw)
To: Glen Choo
Cc: git, Junio C Hamano, Felipe Contreras, Jonathan Tan, Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 1829 bytes --]
On Thu, May 11, 2023 at 06:09:03PM -0700, Glen Choo wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
> > Changes compared to v4:
> >
> > - Patch 1/9: Simplified the test as proposed by Junio and Glen.
> >
> > - Patch 3/9: Added a test to verify that `git fetch -c fetch.output`
> > without a value set fails as expected. Also dropped the tests that
> > checked whether stdout was empty.
> >
> > - Patch 4/9: Reformulated the commit message to treat the missing
> > left-hand side of displayed references as an inconsistency instead
> > of a bug. I've also added a testcase to verify that direct OID
> > fetches continue to work as expected.
> >
> > - Patch 5/9: New patch that makes calculation of the table width for
> > displayed reference updates self-contained in `refcol_width()`.
> > This is a preparatory refactoring that should make patch 6/9
> > easier to review.
> >
> > - Patch 7/9: Refactored the code to parse the "fetch.output" config
> > variable inside of `git_fetch_config()` before we parse command
> > line options. Also fixed that the commit message was still
> > referring to `--output-format=porcelain` instead of the new
> > `--porcelain` switch.
> >
> > - Patch 9/9: The `--porcelain` option is now a simple `OPT_BOOL()`
> > that can be negated. Added a test that `--no-porcelain` works as
> > expected.
>
> I didn't spot any blocking issues in this version, and the various
> improvements (especially 6-7/9) are really welcome. I also read through
> Junio's comments, but I didn't spot anything that I thought should block
> the series, so I'm happy to leave
>
> Reviewed-by: Glen Choo <chooglen@google.com>
Thanks a lot for your reviews!
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v5 4/9] fetch: print left-hand side when fetching HEAD:foo
2023-05-10 12:34 ` [PATCH v5 4/9] fetch: print left-hand side when fetching HEAD:foo Patrick Steinhardt
2023-05-12 0:16 ` Glen Choo
@ 2023-05-13 16:59 ` Jeff King
2023-05-15 5:15 ` Patrick Steinhardt
1 sibling, 1 reply; 120+ messages in thread
From: Jeff King @ 2023-05-13 16:59 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: git, Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
On Wed, May 10, 2023 at 02:34:15PM +0200, Patrick Steinhardt wrote:
> @@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,
> if (oideq(&ref->old_oid, &ref->new_oid)) {
> if (verbosity > 0)
> display_ref_update(display_state, '=', _("[up to date]"), NULL,
> - remote, ref->name, summary_width);
> + remote_ref->name, ref->name, summary_width);
> return 0;
> }
Here (and in other hunks) we now dereference remote_ref unconditionally.
But in existing parts of the code, we guard against remote_ref being
NULL. E.g., later on:
if (!current || !updated) {
const char *msg;
const char *what;
int r;
/*
* Nicely describe the new ref we're fetching.
* Base this on the remote's ref name, as it's
* more likely to follow a standard layout.
*/
const char *name = remote_ref ? remote_ref->name : "";
[...]
I'm not sure if the old code was being overly defensive, or if the new
code is ripe for a segfault. But it's probably worth looking into (it
was noticed by Coverity).
Looking at the caller, it is always store_update_refs() which passes its
own "rm", a pointer iterating over ref_map. And it should always be
non-NULL, since that's the loop condition.
So I think your code is fine, but you might want to double-check my
logic. (And it may be worth cleaning up the existing redundant check to
prevent confusion).
-Peff
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH v5 4/9] fetch: print left-hand side when fetching HEAD:foo
2023-05-13 16:59 ` Jeff King
@ 2023-05-15 5:15 ` Patrick Steinhardt
0 siblings, 0 replies; 120+ messages in thread
From: Patrick Steinhardt @ 2023-05-15 5:15 UTC (permalink / raw)
To: Jeff King
Cc: git, Junio C Hamano, Felipe Contreras, Glen Choo, Jonathan Tan,
Jacob Keller
[-- Attachment #1: Type: text/plain, Size: 2146 bytes --]
On Sat, May 13, 2023 at 12:59:25PM -0400, Jeff King wrote:
> On Wed, May 10, 2023 at 02:34:15PM +0200, Patrick Steinhardt wrote:
>
> > @@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,
> > if (oideq(&ref->old_oid, &ref->new_oid)) {
> > if (verbosity > 0)
> > display_ref_update(display_state, '=', _("[up to date]"), NULL,
> > - remote, ref->name, summary_width);
> > + remote_ref->name, ref->name, summary_width);
> > return 0;
> > }
>
> Here (and in other hunks) we now dereference remote_ref unconditionally.
> But in existing parts of the code, we guard against remote_ref being
> NULL. E.g., later on:
>
> if (!current || !updated) {
> const char *msg;
> const char *what;
> int r;
> /*
> * Nicely describe the new ref we're fetching.
> * Base this on the remote's ref name, as it's
> * more likely to follow a standard layout.
> */
> const char *name = remote_ref ? remote_ref->name : "";
> [...]
>
> I'm not sure if the old code was being overly defensive, or if the new
> code is ripe for a segfault. But it's probably worth looking into (it
> was noticed by Coverity).
>
> Looking at the caller, it is always store_update_refs() which passes its
> own "rm", a pointer iterating over ref_map. And it should always be
> non-NULL, since that's the loop condition.
>
> So I think your code is fine, but you might want to double-check my
> logic. (And it may be worth cleaning up the existing redundant check to
> prevent confusion).
I really think that the code is overly defensive. As you mention, there
is a single caller of `update_local_ref()`, only, and that caller
already dereferences the remote reference unconditionally anyway. So if
there was any way for `rm` to become `NULL` we'd already have a segfault
earlier than in `update_local_ref()`.
I'll send a follow-up patch series after the dust has settled that
removes the check.
Thanks!
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread
end of thread, other threads:[~2023-05-15 5:15 UTC | newest]
Thread overview: 120+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-04-19 12:31 [PATCH 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 1/8] fetch: split out tests for output format Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 2/8] fetch: add a test to exercise invalid output formats Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
2023-04-26 19:20 ` Jacob Keller
2023-04-27 10:58 ` Patrick Steinhardt
2023-04-26 19:21 ` Jacob Keller
2023-04-27 10:58 ` Patrick Steinhardt
2023-04-26 19:25 ` Glen Choo
2023-04-27 10:58 ` Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 4/8] fetch: introduce `display_format` enum Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 5/8] fetch: move display format parsing into main function Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 6/8] fetch: move option related variables " Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 7/8] fetch: introduce new `--output-format` option Patrick Steinhardt
2023-04-26 19:40 ` Glen Choo
2023-04-27 10:58 ` Patrick Steinhardt
2023-04-19 12:31 ` [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
2023-04-26 19:52 ` Glen Choo
2023-04-27 10:58 ` Patrick Steinhardt
2023-04-27 23:20 ` Glen Choo
2023-04-28 8:51 ` Patrick Steinhardt
2023-04-28 17:20 ` Glen Choo
2023-05-02 20:55 ` Felipe Contreras
2023-04-24 20:17 ` [PATCH 0/8] fetch: introduce machine-parseable output Felipe Contreras
2023-04-25 9:58 ` Patrick Steinhardt
2023-04-26 19:14 ` Jacob Keller
2023-04-26 20:23 ` Junio C Hamano
2023-04-26 20:30 ` Jacob Keller
2023-04-27 10:58 ` Patrick Steinhardt
2023-04-27 19:46 ` Jacob Keller
2023-04-27 22:49 ` Glen Choo
2023-04-26 20:24 ` Junio C Hamano
2023-04-26 18:54 ` Glen Choo
2023-04-26 21:14 ` Glen Choo
2023-04-26 19:17 ` Jacob Keller
2023-04-27 11:13 ` [PATCH v2 " Patrick Steinhardt
2023-04-27 11:13 ` [PATCH v2 1/8] fetch: split out tests for output format Patrick Steinhardt
2023-04-29 17:34 ` SZEDER Gábor
2023-05-03 11:21 ` Patrick Steinhardt
2023-04-27 11:13 ` [PATCH v2 2/8] fetch: add a test to exercise invalid output formats Patrick Steinhardt
2023-04-27 11:13 ` [PATCH v2 3/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
2023-04-27 17:26 ` Glen Choo
2023-04-27 19:49 ` Jacob Keller
2023-04-27 11:13 ` [PATCH v2 4/8] fetch: introduce `display_format` enum Patrick Steinhardt
2023-04-27 11:13 ` [PATCH v2 5/8] fetch: move display format parsing into main function Patrick Steinhardt
2023-04-27 11:13 ` [PATCH v2 6/8] fetch: move option related variables " Patrick Steinhardt
2023-04-27 21:52 ` Junio C Hamano
2023-04-27 11:13 ` [PATCH v2 7/8] fetch: introduce new `--output-format` option Patrick Steinhardt
2023-04-27 22:01 ` Junio C Hamano
2023-04-28 22:03 ` Glen Choo
2023-05-03 9:12 ` Patrick Steinhardt
2023-04-28 22:31 ` Glen Choo
2023-05-03 9:43 ` Patrick Steinhardt
2023-05-03 11:36 ` Patrick Steinhardt
2023-04-27 11:13 ` [PATCH v2 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
2023-04-27 19:52 ` Jacob Keller
2023-04-28 22:42 ` Glen Choo
2023-05-03 11:34 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches Patrick Steinhardt
2023-05-08 22:51 ` Glen Choo
2023-05-09 12:41 ` Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 2/8] fetch: split out tests for output format Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 3/8] fetch: add a test to exercise invalid output formats Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 4/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 5/8] fetch: introduce `display_format` enum Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 6/8] fetch: move display format parsing into main function Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 7/8] fetch: move option related variables " Patrick Steinhardt
2023-05-03 11:34 ` [PATCH v3 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
2023-05-08 23:42 ` Glen Choo
2023-05-09 12:41 ` Patrick Steinhardt
2023-05-09 0:03 ` Glen Choo
2023-05-03 16:48 ` [PATCH v3 0/8] fetch: introduce machine-parseable output Junio C Hamano
2023-05-03 16:53 ` Junio C Hamano
2023-05-04 7:57 ` Patrick Steinhardt
2023-05-09 0:06 ` Glen Choo
2023-05-09 12:42 ` Patrick Steinhardt
2023-05-09 13:01 ` [PATCH v4 " Patrick Steinhardt
2023-05-09 13:02 ` [PATCH v4 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches Patrick Steinhardt
2023-05-09 17:49 ` Junio C Hamano
2023-05-09 18:27 ` Glen Choo
2023-05-10 12:34 ` Patrick Steinhardt
2023-05-09 13:02 ` [PATCH v4 2/8] fetch: split out tests for output format Patrick Steinhardt
2023-05-09 13:02 ` [PATCH v4 3/8] fetch: add a test to exercise invalid output formats Patrick Steinhardt
2023-05-09 17:58 ` Junio C Hamano
2023-05-10 12:34 ` Patrick Steinhardt
2023-05-09 13:02 ` [PATCH v4 4/8] fetch: fix missing from-reference when fetching HEAD:foo Patrick Steinhardt
2023-05-09 19:28 ` Junio C Hamano
2023-05-10 12:34 ` Patrick Steinhardt
2023-05-09 13:02 ` [PATCH v4 5/8] fetch: introduce `display_format` enum Patrick Steinhardt
2023-05-09 20:19 ` Junio C Hamano
2023-05-10 12:35 ` Patrick Steinhardt
2023-05-09 13:02 ` [PATCH v4 6/8] fetch: move display format parsing into main function Patrick Steinhardt
2023-05-09 20:35 ` Junio C Hamano
2023-05-09 22:30 ` Glen Choo
2023-05-10 12:35 ` Patrick Steinhardt
2023-05-09 13:02 ` [PATCH v4 7/8] fetch: move option related variables " Patrick Steinhardt
2023-05-09 13:02 ` [PATCH v4 8/8] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
2023-05-09 20:43 ` Junio C Hamano
2023-05-10 12:35 ` Patrick Steinhardt
2023-05-10 12:33 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 1/9] fetch: fix `--no-recurse-submodules` with multi-remote fetches Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 2/9] fetch: split out tests for output format Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 3/9] fetch: add a test to exercise invalid output formats Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 4/9] fetch: print left-hand side when fetching HEAD:foo Patrick Steinhardt
2023-05-12 0:16 ` Glen Choo
2023-05-13 16:59 ` Jeff King
2023-05-15 5:15 ` Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 5/9] fetch: refactor calculation of the display table width Patrick Steinhardt
2023-05-12 0:49 ` Glen Choo
2023-05-10 12:34 ` [PATCH v5 6/9] fetch: introduce `display_format` enum Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 7/9] fetch: lift up parsing of "fetch.output" config variable Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 8/9] fetch: move option related variables into main function Patrick Steinhardt
2023-05-10 12:34 ` [PATCH v5 9/9] fetch: introduce machine-parseable "porcelain" output format Patrick Steinhardt
2023-05-12 1:02 ` Glen Choo
2023-05-10 18:05 ` [PATCH v5 0/9] fetch: introduce machine-parseable output Junio C Hamano
2023-05-11 11:05 ` Patrick Steinhardt
2023-05-11 16:53 ` Junio C Hamano
2023-05-11 17:24 ` Felipe Contreras
2023-05-12 1:09 ` Glen Choo
2023-05-12 7:16 ` Patrick Steinhardt
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).