From: "blanet via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: blanet <bupt_xingxin@163.com>, Xing Xin <xingxin.xx@bytedance.com>
Subject: [PATCH] midx: disable replace objects
Date: Sun, 07 Apr 2024 13:11:47 +0000 [thread overview]
Message-ID: <pull.1711.git.1712495507815.gitgitgadget@gmail.com> (raw)
From: Xing Xin <xingxin.xx@bytedance.com>
We observed a series of clone failures arose in a specific set of
repositories after we fully enabled the MIDX bitmap feature within our
Codebase service. These failures were accompanied with error messages
such as:
fatal: did not receive expected object ...
fatal: fetch-pack: invalid index-pack output
Temporarily disabling the MIDX feature eliminated the reported issues.
After some investigation we found that all repositories experiencing
failures contain replace references, which seem to be improperly
acknowledged by the MIDX bitmap generation logic. During cloning or
fetching, git-pack-objects, which may make use of MIDX bitmap to find
objects to pack, would give wrong objects even if we explicitly
specified not to enable replace refs by GIT_NO_REPLACE_OBJECTS=1.
Indeed, this issue appears to have persisted since the introduction of
MIDX.
This patch updates the MIDX logic to disable replace objects during
operations, mirroring the handling seen in single pack index scenarios,
i.e. git-index-pack and git-pack-objects. The added test uses
git-rev-list to give a more intuitive check.
Signed-off-by: Xing Xin <xingxin.xx@bytedance.com>
---
midx: disable replace objects
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1711%2Fblanet%2Fxx%2Fmidx-ignore-replace-objects-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1711/blanet/xx/midx-ignore-replace-objects-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1711
builtin/multi-pack-index.c | 3 +++
t/t5326-multi-pack-bitmaps.sh | 24 ++++++++++++++++++++++++
2 files changed, 27 insertions(+)
diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c
index a72aebecaa2..8360932d2e7 100644
--- a/builtin/multi-pack-index.c
+++ b/builtin/multi-pack-index.c
@@ -8,6 +8,7 @@
#include "strbuf.h"
#include "trace2.h"
#include "object-store-ll.h"
+#include "replace-object.h"
#define BUILTIN_MIDX_WRITE_USAGE \
N_("git multi-pack-index [<options>] write [--preferred-pack=<pack>]" \
@@ -273,6 +274,8 @@ int cmd_multi_pack_index(int argc, const char **argv,
};
struct option *options = parse_options_concat(builtin_multi_pack_index_options, common_opts);
+ disable_replace_refs();
+
git_config(git_default_config, NULL);
if (the_repository &&
diff --git a/t/t5326-multi-pack-bitmaps.sh b/t/t5326-multi-pack-bitmaps.sh
index 70d1b58709a..5e4cdef6a8b 100755
--- a/t/t5326-multi-pack-bitmaps.sh
+++ b/t/t5326-multi-pack-bitmaps.sh
@@ -434,6 +434,30 @@ test_expect_success 'tagged commits are selected for bitmapping' '
)
'
+test_expect_success 'do not follow replace objects for MIDX bitmap' '
+ rm -fr repo &&
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+
+ test_commit A &&
+ A=$(git rev-parse HEAD) &&
+ test_commit B &&
+ B=$(git rev-parse HEAD) &&
+ git checkout --orphan=orphan $A &&
+ test_commit orphan &&
+ C=$(git rev-parse HEAD) &&
+ git rev-list --objects --no-object-names $B |sort >expected &&
+
+ git replace $A $C &&
+ git repack -ad &&
+ git multi-pack-index write --bitmap &&
+ git rev-list --objects --no-object-names --use-bitmap-index $B |sort >actual &&
+ test_cmp expected actual
+ )
+'
+
corrupt_file () {
chmod a+w "$1" &&
printf "bogus" | dd of="$1" bs=1 seek="12" conv=notrunc
base-commit: 3c2a3fdc388747b9eaf4a4a4f2035c1c9ddb26d0
--
gitgitgadget
next reply other threads:[~2024-04-07 13:11 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-07 13:11 blanet via GitGitGadget [this message]
2024-04-07 14:16 ` [PATCH] midx: disable replace objects Taylor Blau
2024-04-07 18:02 ` Taylor Blau
2024-04-07 18:04 ` Taylor Blau
2024-04-08 5:45 ` [External] " 鑫邢
2024-04-08 5:26 ` [PATCH v2] " blanet via GitGitGadget
2024-04-17 19:34 ` Junio C Hamano
2024-04-18 13:22 ` Taylor Blau
2024-04-18 16:06 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=pull.1711.git.1712495507815.gitgitgadget@gmail.com \
--to=gitgitgadget@gmail.com \
--cc=bupt_xingxin@163.com \
--cc=git@vger.kernel.org \
--cc=xingxin.xx@bytedance.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).