The output of git-fetch(1) is obviously designed for consumption by users, only: we neatly columnize data, we abbreviate reference names, we print neat arrows and we don't provide information about actual object IDs that have changed. This makes the output format basically unusable in the context of scripted invocations of git-fetch(1) that want to learn about the exact changes that the command performs. Introduce a new machine-parseable "porcelain" output format that is supposed to fix this shortcoming. This output format is intended to provide information about every reference that is about to be updated, the old object ID that the reference has been pointing to and the new object ID it will be updated to. Furthermore, the output format provides the same flags as the human-readable format to indicate basic conditions for each reference update like whether it was a fast-forward update, a branch deletion, a rejected update or others. The output format is quite simple: ``` \n ``` We assume two conditions which are generally true: - The old and new object IDs have fixed known widths and cannot contain spaces. - References cannot contain newlines. With these assumptions, the output format becomes unambiguously parseable. Furthermore, given that this output is designed to be consumed by scripts, the machine-readable data is printed to stdout instead of stderr like the human-readable output is. This is mostly done so that other data printed to stderr, like error messages or progress meters, don't interfere with the parseable data. Signed-off-by: Patrick Steinhardt --- Documentation/config/fetch.txt | 4 +- Documentation/fetch-options.txt | 4 +- Documentation/git-fetch.txt | 17 ++++++- builtin/fetch.c | 50 +++++++++++++++----- t/t5574-fetch-output.sh | 84 +++++++++++++++++++++++++++++++++ 5 files changed, 142 insertions(+), 17 deletions(-) diff --git a/Documentation/config/fetch.txt b/Documentation/config/fetch.txt index 568f0f75b3..70734226c0 100644 --- a/Documentation/config/fetch.txt +++ b/Documentation/config/fetch.txt @@ -52,8 +52,8 @@ fetch.pruneTags:: fetch.output:: Control how ref update status is printed. Valid values are - `full` and `compact`. Default value is `full`. See section - OUTPUT in linkgit:git-fetch[1] for detail. + `full`, `compact` and `porcelain`. Default value is `full`. + See section OUTPUT in linkgit:git-fetch[1] for detail. fetch.negotiationAlgorithm:: Control how information about the commits in the local repository diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt index 654f96f79d..5ca8a67fe8 100644 --- a/Documentation/fetch-options.txt +++ b/Documentation/fetch-options.txt @@ -80,8 +80,8 @@ linkgit:git-config[1]. --output-format:: Control how ref update status is printed. Valid values are - `full` and `compact`. Default value is `full`. See section - OUTPUT in linkgit:git-fetch[1] for detail. + `full`, `compact` and `porcelain`. Default value is `full`. + See section OUTPUT in linkgit:git-fetch[1] for detail. ifndef::git-pull[] --[no-]write-fetch-head:: diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt index fba66f1460..efd22cd372 100644 --- a/Documentation/git-fetch.txt +++ b/Documentation/git-fetch.txt @@ -197,13 +197,26 @@ The output of "git fetch" depends on the transport method used; this section describes the output when fetching over the Git protocol (either locally or via ssh) and Smart HTTP protocol. -The status of the fetch is output in tabular form, with each line -representing the status of a single ref. Each line is of the form: +The output format can be chosen either via the `fetch.output` config +(see linkgit:git-config[1]), or via the `--output-format` switch. +Supported values include: + +For the `full` and `compact` output formats, the status of the fetch is +output in tabular, with each line representing the status of a single +ref. Each line is of the form: ------------------------------- -> [] ------------------------------- +The `porcelain` output format is intended to be machine-parseable. In +contrast to the human-readable output formats it thus prints to standard +output instead of standard error. Each line is of the form: + +------------------------------- + +------------------------------- + The status of up-to-date refs is shown only if the --verbose option is used. diff --git a/builtin/fetch.c b/builtin/fetch.c index 22ba75a958..a99e24448b 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -52,6 +52,7 @@ enum display_format { DISPLAY_FORMAT_UNKNOWN = 0, DISPLAY_FORMAT_FULL, DISPLAY_FORMAT_COMPACT, + DISPLAY_FORMAT_PORCELAIN, }; struct display_state { @@ -745,8 +746,11 @@ static void display_state_init(struct display_state *display_state, struct ref * break; } + case DISPLAY_FORMAT_PORCELAIN: + /* We don't need to precompute anything here. */ + break; default: - BUG("unexpected display foramt %d", display_state->format); + BUG("unexpected display format %d", display_state->format); } } @@ -815,8 +819,12 @@ static void print_compact(struct display_state *display_state, static void display_ref_update(struct display_state *display_state, char code, const char *summary, const char *error, const char *remote, const char *local, + const struct object_id *old_oid, + const struct object_id *new_oid, int summary_width) { + FILE *f = stderr; + if (verbosity < 0) return; @@ -849,12 +857,17 @@ static void display_ref_update(struct display_state *display_state, char code, break; } + case DISPLAY_FORMAT_PORCELAIN: + strbuf_addf(&display_state->buf, "%c %s %s %s", code, + oid_to_hex(old_oid), oid_to_hex(new_oid), local); + f = stdout; + break; default: BUG("unexpected display format %d", display_state->format); }; strbuf_addch(&display_state->buf, '\n'); - fputs(display_state->buf.buf, stderr); + fputs(display_state->buf.buf, f); } static int update_local_ref(struct ref *ref, @@ -872,7 +885,8 @@ static int update_local_ref(struct ref *ref, if (oideq(&ref->old_oid, &ref->new_oid)) { if (verbosity > 0) display_ref_update(display_state, '=', _("[up to date]"), NULL, - remote_ref->name, ref->name, summary_width); + remote_ref->name, ref->name, + &ref->old_oid, &ref->new_oid, summary_width); return 0; } @@ -885,7 +899,8 @@ static int update_local_ref(struct ref *ref, */ display_ref_update(display_state, '!', _("[rejected]"), _("can't fetch into checked-out branch"), - remote_ref->name, ref->name, summary_width); + remote_ref->name, ref->name, + &ref->old_oid, &ref->new_oid, summary_width); return 1; } @@ -896,12 +911,14 @@ static int update_local_ref(struct ref *ref, r = s_update_ref("updating tag", ref, transaction, 0); display_ref_update(display_state, r ? '!' : 't', _("[tag update]"), r ? _("unable to update local ref") : NULL, - remote_ref->name, ref->name, summary_width); + remote_ref->name, ref->name, + &ref->old_oid, &ref->new_oid, summary_width); return r; } else { display_ref_update(display_state, '!', _("[rejected]"), _("would clobber existing tag"), - remote_ref->name, ref->name, summary_width); + remote_ref->name, ref->name, + &ref->old_oid, &ref->new_oid, summary_width); return 1; } } @@ -934,7 +951,8 @@ static int update_local_ref(struct ref *ref, r = s_update_ref(msg, ref, transaction, 0); display_ref_update(display_state, r ? '!' : '*', what, r ? _("unable to update local ref") : NULL, - remote_ref->name, ref->name, summary_width); + remote_ref->name, ref->name, + &ref->old_oid, &ref->new_oid, summary_width); return r; } @@ -956,7 +974,8 @@ static int update_local_ref(struct ref *ref, r = s_update_ref("fast-forward", ref, transaction, 1); display_ref_update(display_state, r ? '!' : ' ', quickref.buf, r ? _("unable to update local ref") : NULL, - remote_ref->name, ref->name, summary_width); + remote_ref->name, ref->name, + &ref->old_oid, &ref->new_oid, summary_width); strbuf_release(&quickref); return r; } else if (force || ref->force) { @@ -968,12 +987,14 @@ static int update_local_ref(struct ref *ref, r = s_update_ref("forced-update", ref, transaction, 1); display_ref_update(display_state, r ? '!' : '+', quickref.buf, r ? _("unable to update local ref") : _("forced update"), - remote_ref->name, ref->name, summary_width); + remote_ref->name, ref->name, + &ref->old_oid, &ref->new_oid, summary_width); strbuf_release(&quickref); return r; } else { display_ref_update(display_state, '!', _("[rejected]"), _("non-fast-forward"), - remote_ref->name, ref->name, summary_width); + remote_ref->name, ref->name, + &ref->old_oid, &ref->new_oid, summary_width); return 1; } } @@ -1214,7 +1235,9 @@ static int store_updated_refs(struct display_state *display_state, display_ref_update(display_state, '*', *kind ? kind : "branch", NULL, rm->name, - "FETCH_HEAD", summary_width); + "FETCH_HEAD", + &rm->new_oid, &rm->old_oid, + summary_width); } } } @@ -1354,6 +1377,7 @@ static int prune_refs(struct display_state *display_state, for (ref = stale_refs; ref; ref = ref->next) { display_ref_update(display_state, '-', _("[deleted]"), NULL, _("(none)"), ref->name, + &ref->new_oid, &ref->old_oid, summary_width); warn_dangling_symref(stderr, dangling_msg, ref->name); } @@ -2059,6 +2083,8 @@ static int opt_parse_output_format(const struct option *opt, const char *arg, in *format = DISPLAY_FORMAT_FULL; else if (!strcmp(arg, "compact")) *format = DISPLAY_FORMAT_COMPACT; + else if (!strcmp(arg, "porcelain")) + *format = DISPLAY_FORMAT_PORCELAIN; else return error(_("unsupported output format '%s'"), arg); @@ -2201,6 +2227,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) display_format = DISPLAY_FORMAT_FULL; else if (!strcasecmp(format, "compact")) display_format = DISPLAY_FORMAT_COMPACT; + else if (!strcasecmp(format, "porcelain")) + display_format = DISPLAY_FORMAT_PORCELAIN; else die(_("invalid value for '%s': '%s'"), "fetch.output", format); diff --git a/t/t5574-fetch-output.sh b/t/t5574-fetch-output.sh index 5144d5ed21..28d02a93e4 100755 --- a/t/t5574-fetch-output.sh +++ b/t/t5574-fetch-output.sh @@ -80,6 +80,72 @@ test_expect_success 'fetch compact output' ' test_cmp expect actual ' +test_expect_success 'fetch porcelain output' ' + test_when_finished "rm -rf porcelain-cfg porcelain-cli" && + + # Set up a bunch of references that we can use to demonstrate different + # kinds of flag symbols in the output format. + MAIN_OLD=$(git rev-parse HEAD) && + git branch "fast-forward" && + git branch "deleted-branch" && + git checkout -b force-updated && + test_commit --no-tag force-update-old && + FORCE_UPDATED_OLD=$(git rev-parse HEAD) && + git checkout main && + + # Clone and pre-seed the repositories. We fetch references into two + # namespaces so that we can test that rejected and force-updated + # references are reported properly. + refspecs="refs/heads/*:refs/unforced/* +refs/heads/*:refs/forced/*" && + git clone . porcelain-cli && + git clone . porcelain-cfg && + git -C porcelain-cfg fetch origin $refspecs && + git -C porcelain-cli fetch origin $refspecs && + + # Now that we have set up the client repositories we can change our + # local references. + git branch new-branch && + git branch -d deleted-branch && + git checkout fast-forward && + test_commit --no-tag fast-forward-new && + FAST_FORWARD_NEW=$(git rev-parse HEAD) && + git checkout force-updated && + git reset --hard HEAD~ && + test_commit --no-tag force-update-new && + FORCE_UPDATED_NEW=$(git rev-parse HEAD) && + + cat >expect <<-EOF && + - $MAIN_OLD $ZERO_OID refs/forced/deleted-branch + - $MAIN_OLD $ZERO_OID refs/unforced/deleted-branch + $MAIN_OLD $FAST_FORWARD_NEW refs/unforced/fast-forward + ! $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/unforced/force-updated + * $ZERO_OID $MAIN_OLD refs/unforced/new-branch + $MAIN_OLD $FAST_FORWARD_NEW refs/forced/fast-forward + + $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/forced/force-updated + * $ZERO_OID $MAIN_OLD refs/forced/new-branch + $MAIN_OLD $FAST_FORWARD_NEW refs/remotes/origin/fast-forward + + $FORCE_UPDATED_OLD $FORCE_UPDATED_NEW refs/remotes/origin/force-updated + * $ZERO_OID $MAIN_OLD refs/remotes/origin/new-branch + EOF + + # Execute a dry-run fetch first. We do this to assert that the dry-run + # and non-dry-run fetches produces the same output. Execution of the + # fetch is expected to fail as we have a rejected reference update. + test_must_fail git -C porcelain-cfg -c fetch.output=porcelain fetch --dry-run --prune origin $refspecs >actual-dry-run-cfg && + test_must_fail git -C porcelain-cli fetch --output-format=porcelain --dry-run --prune origin $refspecs >actual-dry-run-cli && + test_cmp actual-dry-run-cfg actual-dry-run-cli && + test_cmp expect actual-dry-run-cfg && + + # And now we perform a non-dry-run fetch. + test_must_fail git -C porcelain-cfg -c fetch.output=porcelain fetch --prune origin $refspecs >actual-cfg && + test_must_fail git -C porcelain-cli fetch --output-format=porcelain --prune origin $refspecs >actual-cli && + test_cmp actual-cfg actual-cli && + test_cmp expect actual-cfg && + + # Ensure that the dry-run and non-dry-run output matches. + test_cmp actual-dry-run-cfg actual-cfg +' + test_expect_success 'fetch output with HEAD and --dry-run' ' test_when_finished "rm -rf head" && git clone . head && @@ -99,6 +165,24 @@ test_expect_success 'fetch output with HEAD and --dry-run' ' test_cmp expect actual ' +test_expect_success 'fetch porcelain output with HEAD and --dry-run' ' + test_when_finished "rm -rf head" && + git clone . head && + COMMIT_ID=$(git rev-parse HEAD) && + + git -C head fetch --output-format=porcelain --dry-run origin HEAD >actual && + cat >expect <<-EOF && + * $ZERO_OID $COMMIT_ID FETCH_HEAD + EOF + test_cmp expect actual && + + git -C head fetch --output-format=porcelain --dry-run origin HEAD:foo >actual && + cat >expect <<-EOF && + * $ZERO_OID $COMMIT_ID refs/heads/foo + EOF + test_cmp expect actual +' + test_expect_success '--no-show-forced-updates' ' mkdir forced-updates && ( -- 2.40.0