* git fetch recursion problem
@ 2023-05-27 17:57 Fraser Hanson
2023-05-30 19:11 ` Fraser Hanson
2023-06-01 22:40 ` Glen Choo
0 siblings, 2 replies; 5+ messages in thread
From: Fraser Hanson @ 2023-05-27 17:57 UTC (permalink / raw)
To: git
I have some git repositories in a state where `git fetch` recursively
spawns subprocesses until all system resources are used up and the OS
locks up.
This has been reproduced on debian with git 2.39.2, and on macos with
git 2.40.1.
The repositories being tracked are mirrors of github repos, within a
secure corporate environment.
During the problem, the `ps aux` output looks like this:
fhanson 3187 0.0 0.0 11400 5092 ? S 12:58 0:00
\_ git fetch --recurse-submodules --tags --force
--progress
fhanson 3195 0.0 0.0 9932 3372 ? S 12:58 0:00
| \_ /usr/lib/git-core/git remote-https origin
https://repomirror.eng.corporate.com/github-neovim/lewis6991/gitsigns.nvim.git
fhanson 3207 0.3 0.0 101608 16004 ? S 12:58 0:00
| \_ /usr/lib/git-core/git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/lewis6991/gitsigns.nvim.git
fhanson 3555 0.0 0.0 10292 4788 ? S 12:58 0:00
| \_ /usr/lib/git-core/git -c
fetch.negotiationAlgorithm=noop fetch origin --no-tags
--no-write-fetch-head --recurse-submodules=no --filter=blob:none
--stdin
fhanson 3583 0.0 0.0 9932 3956 ? S 12:58 0:00
| \_ /usr/lib/git-core/git remote-https
origin https://repomirror.eng.corporate.com/github-neovim/lewis6991/gitsigns.nvim.git
fhanson 3592 0.3 0.0 101600 16156 ? S 12:58 0:00
| \_ /usr/lib/git-core/git-remote-https
origin https://repomirror.eng.corporate.com/github-neovim/lewis6991/gitsigns.nvim.git
... repeating forever!
The git config for these repositories looks like this:
$ cat .git/config
[core]
repositoryformatversion = 1
filemode = true
bare = false
logallrefupdates = true
[submodule]
active = .
[remote "origin"]
url = https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
fetch = +refs/heads/*:refs/remotes/origin/*
promisor = true
partialclonefilter = blob:none
[branch "master"]
remote = origin
merge = refs/heads/master
The git trace output looks like this (some server names redacted):
$ export GIT_TRACE=1
$ git fetch
07:41:19.325652 git.c:439 trace: built-in: git fetch
07:41:19.330118 run-command.c:655 trace: run_command:
GIT_DIR=.git git remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:19.337765 git.c:725 trace: exec:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:19.338220 run-command.c:655 trace: run_command:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:20.664527 run-command.c:655 trace: run_command: git -c
fetch.negotiationAlgorithm=noop fetch origin --no-tags
--no-write-fetch-head --recurse-submodules=no --filter=blob:none
--stdin
07:41:20.679814 git.c:439 trace: built-in: git fetch
origin --no-tags --no-write-fetch-head --recurse-submodules=no
--filter=blob:none --stdin
07:41:20.681735 run-command.c:655 trace: run_command: git
remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:20.692494 git.c:725 trace: exec:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:20.693132 run-command.c:655 trace: run_command:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:21.695677 run-command.c:655 trace: run_command: git -c
fetch.negotiationAlgorithm=noop fetch origin --no-tags
--no-write-fetch-head --recurse-submodules=no --filter=blob:none
--stdin
07:41:21.715621 git.c:439 trace: built-in: git fetch
origin --no-tags --no-write-fetch-head --recurse-submodules=no
--filter=blob:none --stdin
07:41:21.717230 run-command.c:655 trace: run_command: git
remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:21.727616 git.c:725 trace: exec:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:21.728337 run-command.c:655 trace: run_command:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:22.801117 run-command.c:655 trace: run_command: git -c
fetch.negotiationAlgorithm=noop fetch origin --no-tags
--no-write-fetch-head --recurse-submodules=no --filter=blob:none
--stdin
07:41:22.816026 git.c:439 trace: built-in: git fetch
origin --no-tags --no-write-fetch-head --recurse-submodules=no
--filter=blob:none --stdin
07:41:22.817972 run-command.c:655 trace: run_command: git
remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:22.828436 git.c:725 trace: exec:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:22.829219 run-command.c:655 trace: run_command:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:24.242220 run-command.c:655 trace: run_command: git -c
fetch.negotiationAlgorithm=noop fetch origin --no-tags
--no-write-fetch-head --recurse-submodules=no --filter=blob:none
--stdin
07:41:24.257211 git.c:439 trace: built-in: git fetch
origin --no-tags --no-write-fetch-head --recurse-submodules=no
--filter=blob:none --stdin
07:41:24.259264 run-command.c:655 trace: run_command: git
remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:24.269718 git.c:725 trace: exec:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:24.270612 run-command.c:655 trace: run_command:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:25.306129 run-command.c:655 trace: run_command: git -c
fetch.negotiationAlgorithm=noop fetch origin --no-tags
--no-write-fetch-head --recurse-submodules=no --filter=blob:none
--stdin
07:41:25.320842 git.c:439 trace: built-in: git fetch
origin --no-tags --no-write-fetch-head --recurse-submodules=no
--filter=blob:none --stdin
07:41:25.322801 run-command.c:655 trace: run_command: git
remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:25.332775 git.c:725 trace: exec:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:25.333522 run-command.c:655 trace: run_command:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:26.332580 run-command.c:655 trace: run_command: git -c
fetch.negotiationAlgorithm=noop fetch origin --no-tags
--no-write-fetch-head --recurse-submodules=no --filter=blob:none
--stdin
07:41:26.348349 git.c:439 trace: built-in: git fetch
origin --no-tags --no-write-fetch-head --recurse-submodules=no
--filter=blob:none --stdin
07:41:26.350225 run-command.c:655 trace: run_command: git
remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:26.361171 git.c:725 trace: exec:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:26.361969 run-command.c:655 trace: run_command:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:27.585641 run-command.c:655 trace: run_command: git -c
fetch.negotiationAlgorithm=noop fetch origin --no-tags
--no-write-fetch-head --recurse-submodules=no --filter=blob:none
--stdin
07:41:27.600316 git.c:439 trace: built-in: git fetch
origin --no-tags --no-write-fetch-head --recurse-submodules=no
--filter=blob:none --stdin
07:41:27.602370 run-command.c:655 trace: run_command: git
remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:27.611823 git.c:725 trace: exec:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:27.612607 run-command.c:655 trace: run_command:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:28.666477 run-command.c:655 trace: run_command: git -c
fetch.negotiationAlgorithm=noop fetch origin --no-tags
--no-write-fetch-head --recurse-submodules=no --filter=blob:none
--stdin
07:41:28.678956 git.c:439 trace: built-in: git fetch
origin --no-tags --no-write-fetch-head --recurse-submodules=no
--filter=blob:none --stdin
07:41:28.680665 run-command.c:655 trace: run_command: git
remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:28.690534 git.c:725 trace: exec:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
07:41:28.691277 run-command.c:655 trace: run_command:
git-remote-https origin
https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
^C
Random clues and experiments:
A `git clone` from our corporate mirrors always succeeds. Immediately
after a `git clone`, I am always able to `git fetch`, so far.
Something else must cause the repository to enter a broken state,
possibly the addition of upstream commits. I can't verify that since
I can't complete `git fetch.`
I can fix a broken repository by deleting these settings from `.git/config`:
promisor = true
partialclonefilter = blob:none
After this, `git fetch` succeeds.
Deleting just one of these settings is not sufficient, it must be both.
I copied one such broken repository out of our secure environment onto
a mac laptop that has access to both our mirror sites and to the
internet.
I verified that `git fetch` from our mirror is still broken on the mac.
Then I modified .git/config to point to the https://github.com/ url
instead of our mirror site, leaving the 'promisor' and
'partialclonefilter' settings in place.
Running `git fetch` in this state succeeded.
This suggests that our internal mirror site is returning something
different from github itself.
Here is the trace from successfully fetching from github.com:
$ git fetch
07:43:30.901275 git.c:439 trace: built-in: git fetch
07:43:30.904188 run-command.c:655 trace: run_command:
GIT_DIR=.git git remote-https origin
https://github.com/nvim-treesitter/nvim-treesitter-textobjects.git
07:43:30.917345 git.c:725 trace: exec:
git-remote-https origin
https://github.com/nvim-treesitter/nvim-treesitter-textobjects.git
07:43:30.918560 run-command.c:655 trace: run_command:
git-remote-https origin
https://github.com/nvim-treesitter/nvim-treesitter-textobjects.git
remote: Enumerating objects: 8, done.
remote: Counting objects: 100% (8/8), done.
remote: Compressing objects: 100% (5/5), done.
07:43:31.714337 run-command.c:655 trace: run_command: git
index-pack --stdin -v --fix-thin --promisor --pack_header=2,8
remote: Total 8 (delta 2), reused 5 (delta 1), pack-reused 0
07:43:31.729793 git.c:439 trace: built-in: git
index-pack --stdin -v --fix-thin --promisor --pack_header=2,8
Receiving objects: 100% (8/8), 3.36 KiB | 3.36 MiB/s, done.
Resolving deltas: 100% (2/2), completed with 1 local object.
07:43:31.738815 run-command.c:655 trace: run_command: git
rev-list --objects --stdin --exclude-promisor-objects --not --all
--quiet --alternate-refs
07:43:31.748286 git.c:439 trace: built-in: git
rev-list --objects --stdin --exclude-promisor-objects --not --all
--quiet --alternate-refs
From https://github.com/nvim-treesitter/nvim-treesitter-textobjects
9c5237b..95b76b9 master -> origin/master
9c5237b..11ae4c7 update-readme -> origin/update-readme
07:43:31.757504 run-command.c:1524 run_processes_parallel:
preparing to run up to 1 tasks
07:43:31.757515 run-command.c:1551 run_processes_parallel: done
07:43:31.757522 run-command.c:655 trace: run_command: git
maintenance run --auto --no-quiet
07:43:31.765282 git.c:439 trace: built-in: git
maintenance run --auto --no-quiet
The difference starts just a few lines in, my failed fetches never see this:
remote: Counting objects: 100% (8/8), done.
remote: Compressing objects: 100% (5/
Next I enabled both GIT_TRACE and GIT_CURL_VERBOSE to identify
differences in the HTTP conversation between a failed fetch versus a
successful one.
The successful run was made by deleting the `promisor` and
`partialclonefilter` settings from .git/config.
I'm having some some trouble attaching these to this email, I'll have
to include them in a follow up.
Nothing jumped out at me here but I'm no expert.
Is this a git bug, or a server misconfiguration, or both?
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: git fetch recursion problem
2023-05-27 17:57 git fetch recursion problem Fraser Hanson
@ 2023-05-30 19:11 ` Fraser Hanson
2023-06-01 22:40 ` Glen Choo
1 sibling, 0 replies; 5+ messages in thread
From: Fraser Hanson @ 2023-05-30 19:11 UTC (permalink / raw)
To: git
Following is output from `GIT_CURL_VERBOSE=1 git fetch`.
Each git-remote-https process spawns a sub-process that calls GET
/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git/info/refs?service=git-upload-pack
HTTP/1.1
There is no error, only an HTTP 200 OK response so it is unclear why
git is unsatisfied with the results of this call.
09:24:59.990256 git.c:439 trace: built-in: git fetch
09:24:59.992794 run-command.c:655 trace: run_command:
GIT_DIR=.git git remote-https origin
https://repomirror.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
09:25:00.006532 git.c:725 trace: exec: git-remote-https
origin https://repomirror.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
09:25:00.007191 run-command.c:655 trace: run_command:
git-remote-https origin
https://repomirror.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
09:25:00.023290 http.c:699 == Info: Couldn't find host
repomirror.corporate.com in the (nil) file; using defaults
09:25:00.029420 http.c:699 == Info: Trying 10.60.16.190:443...
09:25:00.137589 http.c:699 == Info: Connected to
repomirror.corporate.com (10.60.16.190) port 443 (#0)
09:25:00.137756 http.c:699 == Info: ALPN: offers h2
09:25:00.137774 http.c:699 == Info: ALPN: offers http/1.1
09:25:00.144804 http.c:699 == Info: CAfile: /etc/ssl/cert.pem
09:25:00.144822 http.c:699 == Info: CApath: none
09:25:00.145105 http.c:699 == Info: [CONN-0-0][CF-SSL]
(304) (OUT), TLS handshake, Client hello (1):
09:25:00.247453 http.c:699 == Info: [CONN-0-0][CF-SSL]
(304) (IN), TLS handshake, Server hello (2):
09:25:00.247781 http.c:699 == Info: [CONN-0-0][CF-SSL]
TLSv1.2 (IN), TLS handshake, Certificate (11):
09:25:00.250670 http.c:699 == Info: [CONN-0-0][CF-SSL]
TLSv1.2 (IN), TLS handshake, Server key exchange (12):
09:25:00.251240 http.c:699 == Info: [CONN-0-0][CF-SSL]
TLSv1.2 (IN), TLS handshake, Server finished (14):
09:25:00.257051 http.c:699 == Info: [CONN-0-0][CF-SSL]
TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
09:25:00.257084 http.c:699 == Info: [CONN-0-0][CF-SSL]
TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
09:25:00.257145 http.c:699 == Info: [CONN-0-0][CF-SSL]
TLSv1.2 (OUT), TLS handshake, Finished (20):
09:25:00.353682 http.c:699 == Info: [CONN-0-0][CF-SSL]
TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
09:25:00.354049 http.c:699 == Info: [CONN-0-0][CF-SSL]
TLSv1.2 (IN), TLS handshake, Finished (20):
09:25:00.354143 http.c:699 == Info: SSL connection using
TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
09:25:00.354190 http.c:699 == Info: ALPN: server did not
agree on a protocol. Uses default.
09:25:00.354222 http.c:699 == Info: Server certificate:
09:25:00.354274 http.c:699 == Info: subject: C=US;
ST=California; O=Company, Inc.; CN=proxy.corporate.com
09:25:00.354303 http.c:699 == Info: start date: Jun 11
00:00:00 2022 GMT
09:25:00.354327 http.c:699 == Info: expire date: Jun 11
23:59:59 2024 GMT
09:25:00.354400 http.c:699 == Info: subjectAltName: host
"repomirror.corporate.com" matched cert's "repomirror.corporate.com"
09:25:00.354452 http.c:699 == Info: issuer: C=GB;
ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA
Organization Validation Secure Server CA
09:25:00.354476 http.c:699 == Info: SSL certificate verify ok.
09:25:00.354624 http.c:646 => Send header, 0000000295
bytes (0x00000127)
09:25:00.354660 http.c:658 => Send header: GET
/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git/info/refs?service=git-upload-pack
HTTP/1.1
09:25:00.354680 http.c:658 => Send header: Host:
repomirror.corporate.com
09:25:00.354699 http.c:658 => Send header: User-Agent: git/2.40.1
09:25:00.354717 http.c:658 => Send header: Accept: */*
09:25:00.354736 http.c:658 => Send header:
Accept-Encoding: deflate, gzip
09:25:00.354755 http.c:658 => Send header:
Accept-Language: en-CA, *;q=0.9
09:25:00.354827 http.c:658 => Send header: Pragma: no-cache
09:25:00.354921 http.c:658 => Send header: Git-Protocol: version=2
09:25:00.354940 http.c:658 => Send header:
09:25:00.453443 http.c:699 == Info: Mark bundle as not
supporting multiuse
09:25:00.453568 http.c:646 <= Recv header, 0000000017
bytes (0x00000011)
09:25:00.453586 http.c:658 <= Recv header: HTTP/1.1 200 OK
09:25:00.453602 http.c:646 <= Recv header, 0000000037
bytes (0x00000025)
09:25:00.453617 http.c:658 <= Recv header: Date: Mon, 29
May 2023 16:25:00 GMT
09:25:00.453632 http.c:646 <= Recv header, 0000000033
bytes (0x00000021)
09:25:00.453646 http.c:658 <= Recv header: Server:
Apache/2.2.15 (Red Hat)
09:25:00.453661 http.c:646 <= Recv header, 0000000046
bytes (0x0000002e)
09:25:00.453674 http.c:658 <= Recv header: Last-Modified:
Sun, 28 May 2023 10:15:35 GMT
09:25:00.453689 http.c:646 <= Recv header, 0000000037
bytes (0x00000025)
09:25:00.453703 http.c:658 <= Recv header: ETag:
"42e2b247-4a13-5fcbe403f497b"
09:25:00.453750 http.c:646 <= Recv header, 0000000022
bytes (0x00000016)
09:25:00.453808 http.c:658 <= Recv header: Accept-Ranges: bytes
09:25:00.453851 http.c:646 <= Recv header, 0000000023
bytes (0x00000017)
09:25:00.453874 http.c:658 <= Recv header: Content-Length: 18963
09:25:00.453890 http.c:646 <= Recv header, 0000000019
bytes (0x00000013)
09:25:00.453904 http.c:658 <= Recv header: Connection: close
09:25:00.453919 http.c:646 <= Recv header, 0000000041
bytes (0x00000029)
09:25:00.453932 http.c:658 <= Recv header: Content-Type:
text/plain; charset=UTF-8
09:25:00.453949 http.c:646 <= Recv header, 0000000002
bytes (0x00000002)
09:25:00.453964 http.c:658 <= Recv header:
09:25:00.551101 http.c:699 == Info: Closing connection 0
09:25:00.551165 http.c:699 == Info: [CONN-0-0][CF-SSL]
TLSv1.2 (IN), TLS alert, close notify (256):
09:25:00.551212 http.c:699 == Info: [CONN-0-0][CF-SSL]
TLSv1.2 (OUT), TLS alert, close notify (256):
09:25:00.551491 http.c:699 == Info: Couldn't find host
repomirror.corporate.com in the (nil) file; using defaults
09:25:00.551509 http.c:699 == Info: Hostname
repomirror.corporate.com was found in DNS cache
09:25:00.551561 http.c:699 == Info: Trying 10.60.16.190:443...
09:25:00.658191 http.c:699 == Info: Connected to
repomirror.corporate.com (10.60.16.190) port 443 (#1)
09:25:00.658452 http.c:699 == Info: ALPN: offers h2
09:25:00.658479 http.c:699 == Info: ALPN: offers http/1.1
09:25:00.658552 http.c:699 == Info: SSL re-using session ID
09:25:00.658784 http.c:699 == Info: [CONN-1-0][CF-SSL]
TLSv1.2 (OUT), TLS handshake, Client hello (1):
09:25:00.755274 http.c:699 == Info: [CONN-1-0][CF-SSL]
TLSv1.2 (IN), TLS handshake, Server hello (2):
09:25:00.755437 http.c:699 == Info: [CONN-1-0][CF-SSL]
TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
09:25:00.755535 http.c:699 == Info: [CONN-1-0][CF-SSL]
TLSv1.2 (IN), TLS handshake, Finished (20):
09:25:00.755555 http.c:699 == Info: [CONN-1-0][CF-SSL]
TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
09:25:00.755586 http.c:699 == Info: [CONN-1-0][CF-SSL]
TLSv1.2 (OUT), TLS handshake, Finished (20):
09:25:00.755660 http.c:699 == Info: SSL connection using
TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
09:25:00.755679 http.c:699 == Info: ALPN: server did not
agree on a protocol. Uses default.
09:25:00.755693 http.c:699 == Info: Server certificate:
09:25:00.755694 http.c:699 == Info: subject: C=US;
ST=California; O=Company, Inc.; CN=proxy.corporate.com
09:25:00.755740 http.c:699 == Info: start date: Jun 11
00:00:00 2022 GMT
09:25:00.755749 http.c:699 == Info: expire date: Jun 11
23:59:59 2024 GMT
09:25:00.755848 http.c:699 == Info: subjectAltName: host
"repomirror.corporate.com" matched cert's "repomirror.corporate.com"
09:25:00.755879 http.c:699 == Info: issuer: C=GB;
ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA
Organization Validation Secure Server CA
09:25:00.755888 http.c:699 == Info: SSL certificate verify ok.
09:25:00.755959 http.c:646 => Send header, 0000000241
bytes (0x000000f1)
09:25:00.755976 http.c:658 => Send header: GET
/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git/HEAD
HTTP/1.1
09:25:00.755983 http.c:658 => Send header: Host:
repomirror.corporate.com
09:25:00.755989 http.c:658 => Send header: User-Agent: git/2.40.1
09:25:00.755995 http.c:658 => Send header: Accept: */*
09:25:00.756001 http.c:658 => Send header:
Accept-Encoding: deflate, gzip
09:25:00.756008 http.c:658 => Send header:
Accept-Language: en-CA, *;q=0.9
09:25:00.756015 http.c:658 => Send header: Pragma: no-cache
09:25:00.756022 http.c:658 => Send header:
09:25:00.852762 http.c:699 == Info: Mark bundle as not
supporting multiuse
09:25:00.852921 http.c:646 <= Recv header, 0000000017
bytes (0x00000011)
09:25:00.852996 http.c:658 <= Recv header: HTTP/1.1 200 OK
09:25:00.853018 http.c:646 <= Recv header, 0000000037
bytes (0x00000025)
09:25:00.853033 http.c:658 <= Recv header: Date: Mon, 29
May 2023 16:25:00 GMT
09:25:00.853049 http.c:646 <= Recv header, 0000000033
bytes (0x00000021)
09:25:00.853063 http.c:658 <= Recv header: Server:
Apache/2.2.15 (Red Hat)
09:25:00.853078 http.c:646 <= Recv header, 0000000046
bytes (0x0000002e)
09:25:00.853091 http.c:658 <= Recv header: Last-Modified:
Tue, 09 May 2023 13:33:51 GMT
09:25:00.853106 http.c:646 <= Recv header, 0000000035
bytes (0x00000023)
09:25:00.853184 http.c:658 <= Recv header: ETag:
"4319f275-17-5fb42ce4c3e23"
09:25:00.853209 http.c:646 <= Recv header, 0000000022
bytes (0x00000016)
09:25:00.853357 http.c:658 <= Recv header: Accept-Ranges: bytes
09:25:00.853513 http.c:646 <= Recv header, 0000000020
bytes (0x00000014)
09:25:00.853535 http.c:658 <= Recv header: Content-Length: 23
09:25:00.853551 http.c:646 <= Recv header, 0000000019
bytes (0x00000013)
09:25:00.853565 http.c:658 <= Recv header: Connection: close
09:25:00.853582 http.c:646 <= Recv header, 0000000041
bytes (0x00000029)
09:25:00.853595 http.c:658 <= Recv header: Content-Type:
text/plain; charset=UTF-8
09:25:00.853613 http.c:646 <= Recv header, 0000000002
bytes (0x00000002)
09:25:00.853626 http.c:658 <= Recv header:
09:25:00.853701 http.c:699 == Info: Closing connection 1
09:25:00.853746 http.c:699 == Info: [CONN-1-0][CF-SSL]
TLSv1.2 (IN), TLS alert, close notify (256):
09:25:00.853818 http.c:699 == Info: [CONN-1-0][CF-SSL]
TLSv1.2 (OUT), TLS alert, close notify (256):
09:25:00.860085 run-command.c:655 trace: run_command: git -c
fetch.negotiationAlgorithm=noop fetch origin --no-tags
--no-write-fetch-head --recurse-submodules=no --filter=blob:none
--stdin
09:25:00.872999 git.c:439 trace: built-in: git fetch
origin --no-tags --no-write-fetch-head --recurse-submodules=no
--filter=blob:none --stdin
09:25:00.874628 run-command.c:655 trace: run_command: git
remote-https origin
https://repomirror.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
09:25:00.884027 git.c:725 trace: exec: git-remote-https
origin https://repomirror.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
...
From this point it repeats until killed (start over from line 3, git.c:725...)
I put together what I think is an equivalent curl command. Running
this command does return a list of refs.
curl -isSL --user-agent 'git/2.40.1' -v
https://repomirror.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git/info/refs\?service\=git-upload-pack
The log output shows a TLS result code 256 which looked suspicious to
me. However the curl command shows an identical TLS notify code after
fetching the list of refs.
Any clues as to how to debug this further would be appreciated, I'm stuck.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: git fetch recursion problem
2023-05-27 17:57 git fetch recursion problem Fraser Hanson
2023-05-30 19:11 ` Fraser Hanson
@ 2023-06-01 22:40 ` Glen Choo
2023-06-01 22:57 ` Fraser Hanson
1 sibling, 1 reply; 5+ messages in thread
From: Glen Choo @ 2023-06-01 22:40 UTC (permalink / raw)
To: Fraser Hanson, git
Fraser Hanson <fraser.hanson@gmail.com> writes:
> The git trace output looks like this (some server names redacted):
> $ export GIT_TRACE=1
> $ git fetch
> 07:41:19.325652 git.c:439 trace: built-in: git fetch
> 07:41:19.330118 run-command.c:655 trace: run_command:
> GIT_DIR=.git git remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:19.337765 git.c:725 trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:19.338220 run-command.c:655 trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:20.664527 run-command.c:655 trace: run_command: git -c
> fetch.negotiationAlgorithm=noop fetch origin --no-tags
> --no-write-fetch-head --recurse-submodules=no --filter=blob:none
> --stdin
> 07:41:20.679814 git.c:439 trace: built-in: git fetch
> origin --no-tags --no-write-fetch-head --recurse-submodules=no
> --filter=blob:none --stdin
> 07:41:20.681735 run-command.c:655 trace: run_command: git
> remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:20.692494 git.c:725 trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:20.693132 run-command.c:655 trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:21.695677 run-command.c:655 trace: run_command: git -c
> fetch.negotiationAlgorithm=noop fetch origin --no-tags
> --no-write-fetch-head --recurse-submodules=no --filter=blob:none
> --stdin
> 07:41:21.715621 git.c:439 trace: built-in: git fetch
> origin --no-tags --no-write-fetch-head --recurse-submodules=no
> --filter=blob:none --stdin
> 07:41:21.717230 run-command.c:655 trace: run_command: git
> remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:21.727616 git.c:725 trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:21.728337 run-command.c:655 trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:22.801117 run-command.c:655 trace: run_command: git -c
> fetch.negotiationAlgorithm=noop fetch origin --no-tags
> --no-write-fetch-head --recurse-submodules=no --filter=blob:none
> --stdin
> 07:41:22.816026 git.c:439 trace: built-in: git fetch
> origin --no-tags --no-write-fetch-head --recurse-submodules=no
> --filter=blob:none --stdin
> 07:41:22.817972 run-command.c:655 trace: run_command: git
> remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:22.828436 git.c:725 trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:22.829219 run-command.c:655 trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:24.242220 run-command.c:655 trace: run_command: git -c
> fetch.negotiationAlgorithm=noop fetch origin --no-tags
> --no-write-fetch-head --recurse-submodules=no --filter=blob:none
> --stdin
> 07:41:24.257211 git.c:439 trace: built-in: git fetch
> origin --no-tags --no-write-fetch-head --recurse-submodules=no
> --filter=blob:none --stdin
> 07:41:24.259264 run-command.c:655 trace: run_command: git
> remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:24.269718 git.c:725 trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:24.270612 run-command.c:655 trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:25.306129 run-command.c:655 trace: run_command: git -c
> fetch.negotiationAlgorithm=noop fetch origin --no-tags
> --no-write-fetch-head --recurse-submodules=no --filter=blob:none
> --stdin
> 07:41:25.320842 git.c:439 trace: built-in: git fetch
> origin --no-tags --no-write-fetch-head --recurse-submodules=no
> --filter=blob:none --stdin
> 07:41:25.322801 run-command.c:655 trace: run_command: git
> remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:25.332775 git.c:725 trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:25.333522 run-command.c:655 trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:26.332580 run-command.c:655 trace: run_command: git -c
> fetch.negotiationAlgorithm=noop fetch origin --no-tags
> --no-write-fetch-head --recurse-submodules=no --filter=blob:none
> --stdin
> 07:41:26.348349 git.c:439 trace: built-in: git fetch
> origin --no-tags --no-write-fetch-head --recurse-submodules=no
> --filter=blob:none --stdin
> 07:41:26.350225 run-command.c:655 trace: run_command: git
> remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:26.361171 git.c:725 trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:26.361969 run-command.c:655 trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:27.585641 run-command.c:655 trace: run_command: git -c
> fetch.negotiationAlgorithm=noop fetch origin --no-tags
> --no-write-fetch-head --recurse-submodules=no --filter=blob:none
> --stdin
> 07:41:27.600316 git.c:439 trace: built-in: git fetch
> origin --no-tags --no-write-fetch-head --recurse-submodules=no
> --filter=blob:none --stdin
> 07:41:27.602370 run-command.c:655 trace: run_command: git
> remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:27.611823 git.c:725 trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:27.612607 run-command.c:655 trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:28.666477 run-command.c:655 trace: run_command: git -c
> fetch.negotiationAlgorithm=noop fetch origin --no-tags
> --no-write-fetch-head --recurse-submodules=no --filter=blob:none
> --stdin
> 07:41:28.678956 git.c:439 trace: built-in: git fetch
> origin --no-tags --no-write-fetch-head --recurse-submodules=no
> --filter=blob:none --stdin
> 07:41:28.680665 run-command.c:655 trace: run_command: git
> remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:28.690534 git.c:725 trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:41:28.691277 run-command.c:655 trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
> ^C
>
>
> Random clues and experiments:
>
> A `git clone` from our corporate mirrors always succeeds. Immediately
> after a `git clone`, I am always able to `git fetch`, so far.
> Something else must cause the repository to enter a broken state,
> possibly the addition of upstream commits. I can't verify that since
> I can't complete `git fetch.`
>
> I can fix a broken repository by deleting these settings from `.git/config`:
> promisor = true
> partialclonefilter = blob:none
> After this, `git fetch` succeeds.
> Deleting just one of these settings is not sufficient, it must be both.
>
> I copied one such broken repository out of our secure environment onto
> a mac laptop that has access to both our mirror sites and to the
> internet.
> I verified that `git fetch` from our mirror is still broken on the mac.
> Then I modified .git/config to point to the https://github.com/ url
> instead of our mirror site, leaving the 'promisor' and
> 'partialclonefilter' settings in place.
> Running `git fetch` in this state succeeded.
>
> This suggests that our internal mirror site is returning something
> different from github itself.
>
>
> Here is the trace from successfully fetching from github.com:
> $ git fetch
> 07:43:30.901275 git.c:439 trace: built-in: git fetch
> 07:43:30.904188 run-command.c:655 trace: run_command:
> GIT_DIR=.git git remote-https origin
> https://github.com/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:43:30.917345 git.c:725 trace: exec:
> git-remote-https origin
> https://github.com/nvim-treesitter/nvim-treesitter-textobjects.git
> 07:43:30.918560 run-command.c:655 trace: run_command:
> git-remote-https origin
> https://github.com/nvim-treesitter/nvim-treesitter-textobjects.git
> remote: Enumerating objects: 8, done.
> remote: Counting objects: 100% (8/8), done.
> remote: Compressing objects: 100% (5/5), done.
> 07:43:31.714337 run-command.c:655 trace: run_command: git
> index-pack --stdin -v --fix-thin --promisor --pack_header=2,8
> remote: Total 8 (delta 2), reused 5 (delta 1), pack-reused 0
> 07:43:31.729793 git.c:439 trace: built-in: git
> index-pack --stdin -v --fix-thin --promisor --pack_header=2,8
> Receiving objects: 100% (8/8), 3.36 KiB | 3.36 MiB/s, done.
> Resolving deltas: 100% (2/2), completed with 1 local object.
> 07:43:31.738815 run-command.c:655 trace: run_command: git
> rev-list --objects --stdin --exclude-promisor-objects --not --all
> --quiet --alternate-refs
> 07:43:31.748286 git.c:439 trace: built-in: git
> rev-list --objects --stdin --exclude-promisor-objects --not --all
> --quiet --alternate-refs
> From https://github.com/nvim-treesitter/nvim-treesitter-textobjects
> 9c5237b..95b76b9 master -> origin/master
> 9c5237b..11ae4c7 update-readme -> origin/update-readme
> 07:43:31.757504 run-command.c:1524 run_processes_parallel:
> preparing to run up to 1 tasks
> 07:43:31.757515 run-command.c:1551 run_processes_parallel: done
> 07:43:31.757522 run-command.c:655 trace: run_command: git
> maintenance run --auto --no-quiet
> 07:43:31.765282 git.c:439 trace: built-in: git
> maintenance run --auto --no-quiet
>
> The difference starts just a few lines in, my failed fetches never see this:
> remote: Counting objects: 100% (8/8), done.
> remote: Compressing objects: 100% (5/
>
> Next I enabled both GIT_TRACE and GIT_CURL_VERBOSE to identify
> differences in the HTTP conversation between a failed fetch versus a
> successful one.
> The successful run was made by deleting the `promisor` and
> `partialclonefilter` settings from .git/config.
> I'm having some some trouble attaching these to this email, I'll have
> to include them in a follow up.
This looks a lot like the partial clone fetch is recursing infinitely.
This could happen if Git encountered a missing object, initiated a
partial clone fetch to get the missing object, then in the process of
that fetch it encountered a missing object, initiated a partial clone
fetch to get the missing object, then (etc etc etc).
I don't have good clues as to why that might be the case though.
Wild speculation: Does this reproduce if you clone without submodules,
or if you pass --no-recurse-submodules to "git fetch"? In a non-partial
clone fetch, "git fetch" may read .gitmodules to get submodule
information. Perhaps something similar is happening in the partial clone
fetch.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: git fetch recursion problem
2023-06-01 22:40 ` Glen Choo
@ 2023-06-01 22:57 ` Fraser Hanson
2023-06-19 18:40 ` Fraser Hanson
0 siblings, 1 reply; 5+ messages in thread
From: Fraser Hanson @ 2023-06-01 22:57 UTC (permalink / raw)
To: Glen Choo; +Cc: git
Thanks for the theory!
This repo has no .gitmodules file.
I tried running `git fetch --no-recurse-submodules` in one of my
broken repos, but it does not prevent the bug from happening.
I also tried removing the submodule configuration in .git/config, that
had no effect either.
Maybe something is unusual on the server side. I've set up my own git
server attempting to reproduce similar apache and git versions as on
our corporate mirror but so far cannot reproduce the bug.
I've compared the GIT_CURL_VERBOSE output to what I get for the same
repo from github, but it's not very helpful.
Github uses HTTP2 and my git client sends different service api calls
to github so there's not much basis for comparison here.
My main theory was that maybe the server-side git version was too old
to handle partial clones properly.
Partial clones were introduced in git 2.25. On my test git server, I
set the server-side git version to 2.7.2.
This did not reproduce the bug -- on the client side using a recent
git version, I was still able to pull partial clones and then fetch
from origin after my local copy went out of date.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: git fetch recursion problem
2023-06-01 22:57 ` Fraser Hanson
@ 2023-06-19 18:40 ` Fraser Hanson
0 siblings, 0 replies; 5+ messages in thread
From: Fraser Hanson @ 2023-06-19 18:40 UTC (permalink / raw)
To: Glen Choo; +Cc: git
I figured out what is going on here.
This problem is caused by the server configuration, but git should
handle this more gracefully because it is easy to hit this problem and
the consequence are severe.
The root cause is that the git server is using the git "dumb http" protocol.
The git server runs apache configured like this:
<Directory "/srv/git">
Options +Indexes
AllowOverride None
Order allow,deny
Allow from all
</Directory>
Apache does not pass requests to git-http-backend.
The server's /srv/git/ dir contains bare git repositories created with
`git clone --mirror`.
The repositories on the server must be prepared before download with
`git update-server-info`.
This is how the bug develops.
### git client requests a partial clone from the server:
$ git clone --filter=blob:none http://172.20.208.191/git/kmarius/jsregexp.git
### git clone succeeds, but there are no *.promisor files (these
should exist for a partial clone)
$ cd jsregexp
$ find .git -name \*.promisor | wc -l
0
### the cloned repository is still configured as a partial clone
$ cat .git/config
[core]
repositoryformatversion = 1
filemode = true
bare = false
logallrefupdates = true
[remote "origin"]
url = http://172.20.208.191/git/kmarius/jsregexp.git
fetch = +refs/heads/*:refs/remotes/origin/*
promisor = true
partialclonefilter = blob:none
[branch "master"]
remote = origin
merge = refs/heads/master
### git pull works, as long as there are no new commits added to the
server's git repo
$ git pull
Already up to date.
### Next, add a commit into the server's git repository.
### Then update the server's git repository with this:
server# cd /srv/git/kmarius/jsregexp.git
server# git update-server-info
### Back on the client side, git pull is now broken:
$ GIT_TRACE=1 git fetch 2>&1 | head -20
14:21:03.574765 git.c:460 trace: built-in: git fetch
14:21:03.575455 run-command.c:655 trace: run_command:
GIT_DIR=.git git remote-http origin
http://172.20.208.191/git/kmarius/jsregexp.git
14:21:03.578239 git.c:750 trace: exec: git-remote-http
origin http://172.20.208.191/git/kmarius/jsregexp.git
14:21:03.578349 run-command.c:655 trace: run_command:
git-remote-http origin http://172.20.208.191/git/kmarius/jsregexp.git
14:21:03.590589 run-command.c:655 trace: run_command: git -c
fetch.negotiationAlgorithm=noop fetch origin --no-tags
--no-write-fetch-head --recurse-submodules=no --filter=blob:none
--stdin
14:21:03.593654 git.c:460 trace: built-in: git fetch
origin --no-tags --no-write-fetch-head --recurse-submodules=no
--filter=blob:none --stdin
14:21:03.594362 run-command.c:655 trace: run_command: git
remote-http origin http://172.20.208.191/git/kmarius/jsregexp.git
14:21:03.597148 git.c:750 trace: exec: git-remote-http
origin http://172.20.208.191/git/kmarius/jsregexp.git
14:21:03.597253 run-command.c:655 trace: run_command:
git-remote-http origin http://172.20.208.191/git/kmarius/jsregexp.git
14:21:03.609592 run-command.c:655 trace: run_command: git -c
fetch.negotiationAlgorithm=noop fetch origin --no-tags
--no-write-fetch-head --recurse-submodules=no --filter=blob:none
--stdin
14:21:03.612749 git.c:460 trace: built-in: git fetch
origin --no-tags --no-write-fetch-head --recurse-submodules=no
--filter=blob:none --stdin
14:21:03.613392 run-command.c:655 trace: run_command: git
remote-http origin http://172.20.208.191/git/kmarius/jsregexp.git
14:21:03.616123 git.c:750 trace: exec: git-remote-http
origin http://172.20.208.191/git/kmarius/jsregexp.git
14:21:03.616235 run-command.c:655 trace: run_command:
git-remote-http origin http://172.20.208.191/git/kmarius/jsregexp.git
14:21:03.630921 run-command.c:655 trace: run_command: git -c
fetch.negotiationAlgorithm=noop fetch origin --no-tags
--no-write-fetch-head --recurse-submodules=no --filter=blob:none
--stdin
14:21:03.634074 git.c:460 trace: built-in: git fetch
origin --no-tags --no-write-fetch-head --recurse-submodules=no
--filter=blob:none --stdin
14:21:03.634746 run-command.c:655 trace: run_command: git
remote-http origin http://172.20.208.191/git/kmarius/jsregexp.git
14:21:03.636388 git.c:750 trace: exec: git-remote-http
origin http://172.20.208.191/git/kmarius/jsregexp.git
14:21:03.636443 run-command.c:655 trace: run_command:
git-remote-http origin http://172.20.208.191/git/kmarius/jsregexp.git
14:21:03.649168 run-command.c:655 trace: run_command: git -c
fetch.negotiationAlgorithm=noop fetch origin --no-tags
--no-write-fetch-head --recurse-submodules=no --filter=blob:none
--stdin
...
This will repeat forever, repeatedly spawning sub-processes.
The impact can be severe. I discovered this while using Neovim's
lazy.nvim plugin manager within a secure, firewalled environment
without internet access. The editor is configured to pull packages
from an intranet mirror site containing bare clones of GitHub
projects. The multi-threaded lazy.nvim plugin manager attempts to
update all 30 or so repositories in my configuration simultaneously.
All of the repos with fresh commits on our dumb http mirror server hit
the bug, causes many git processes to spawn very fast. The system
locks up in seconds, the OOM killer shows up too late to save it.
Git should handle this situation better. The following would be nice:
* when the http dumb protocol is used for clone or fetch, log an
info-level message which is visible with GIT_TRACE
* when a git partial clone is performed over dumb http protocol, log a
warning-level message explaining that the resulting repo may be broken
* when a 'git fetch' is done from a broken repository, don't
repeatedly spawn git-remote-http processes forever
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-06-19 18:40 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-05-27 17:57 git fetch recursion problem Fraser Hanson
2023-05-30 19:11 ` Fraser Hanson
2023-06-01 22:40 ` Glen Choo
2023-06-01 22:57 ` Fraser Hanson
2023-06-19 18:40 ` Fraser Hanson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).