From: Jeff King <peff@peff.net>
To: Toon Claes <toon@iotcl.com>
Cc: git@vger.kernel.org
Subject: Re: [PATCH 4/4] bundle-uri: enable git-remote-https progress
Date: Fri, 21 Feb 2025 02:36:05 -0500 [thread overview]
Message-ID: <20250221073605.GA1988395@coredump.intra.peff.net> (raw)
In-Reply-To: <87o6z43gz8.fsf@iotcl.com>
On Fri, Feb 14, 2025 at 12:26:03PM +0100, Toon Claes wrote:
> I've been playing around with things and I haven't found a good way
> forward with this. We could have the parent process ingest stderr of
> git-remote-https and swallow messages that match `/^fatal:/`, but that
> feels like a hack and not foolproof.
Yeah, agree that feels quite hacky (and you'd probably want to swallow
/^error:/, /^warning:/, etc, too).
> I was thinking if we could override `die()` in the child process to have
> it not print anything, but because git-remote-http basically can call
> die() basically from anywhere in the codebase, I don't think we can
> ensure the silenced die() function is called.
>
> Or what do you mean by "squelch non-progress errors"?
I was thinking of having some kind of "very quiet" mode where
git-remote-http would not print any errors (except for progress). But I
agree that doing it is non-trivial. Our die/warning/error functions are
all pluggable, so remote-http could add its own implementations using
set_die_routine(), etc.
But that does feel pretty heavyweight, and you'd still have to pass
through the "please suppress all your die calls" option into
remote-http. Plus it wouldn't catch any spots in the code that happen to
call fprintf(stderr), etc.
> And yes, sending progress logging over a separate fd seems like the
> ideal approach, but I haven't tried it yet. I'm afraid it's not worth
> attempting so.
>
> So I think that leaves us with your suggestion to "ferry
> machine-readable output back to the parent". If I understand correctly
> you mean the child process will not write progress logging to stderr but
> to stdout (with some kind of command prefix the parent process knows
> what to do with this)?
Exactly.
> I imagine communication between parent and child will then look
> something like this:
>
> -> capabilities
> <- stateless-connect
> <- fetch
> <- get
> <- option
> <- push
> <- check-connectivity
> <- object-format
> -> option progress true
> <- ok
> -> get http://example.com git.bundle
> <- progress 123 345 40
> <- progress 234 345 50
> <- progress 345 345 40
>
> ~fin~
>
> But then we need to decide on the format the child sends back to the
> parent. In the above example it's something like `progress <size>
> <total> <throughput>`. An alternative proposal could be:
>
> <- log Downloading via HTTP:
> <- log Downloading via HTTP: 200.00 KiB | 100.00 KiB/s
> <- log Downloading via HTTP: 300.00 KiB | 100.00 KiB/s
> <- log Downloading via HTTP: 400.00 KiB | 100.00 KiB/s
> <- log Downloading via HTTP: 400.00 KiB | 100.00 KiB/s, done.
>
> So the child sends the progress text with a `log` prefix, which the
> parent simply has to send that logging to where it wants it to go.
>
> Or am I completely misunderstanding your proposal? Do you maybe happen
> to have any examples of a similar solution?
No, I think you understand it perfectly.
Your "log" example with arbitrary text seems like the simplest approach,
and might be enough. Then we wouldn't have to define a schema for
progress numbers (and we have at least two types of progress: counts and
throughput, though I guess maybe this would always be throughput?).
But I do wonder if we'd want the flexibility of the machine-readable
numbers. In particular, would we ever fetch multiple bundle-uri files in
parallel? If so, then we wouldn't want them stomping on each other's
progress. You'd probably want the caller to present a unified view based
on the progress reports from all of the child processes.
-Peff
next prev parent reply other threads:[~2025-02-21 7:36 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-08 12:44 [PATCH 0/4] bundle-uri: show progress when downloading from bundle URIs Toon Claes
2024-05-08 12:44 ` [PATCH 1/4] progress: add function to set total Toon Claes
2024-05-08 12:44 ` [PATCH 2/4] http: add the ability to log progress Toon Claes
2024-05-08 16:52 ` Eric Sunshine
2024-05-09 16:34 ` Jeff King
2024-05-09 16:51 ` Junio C Hamano
2024-05-09 17:04 ` Jeff King
2024-05-09 16:52 ` Jeff King
2024-05-08 12:44 ` [PATCH 3/4] remote-curl: optionally show progress for HTTP get Toon Claes
2024-05-08 22:29 ` Junio C Hamano
2024-05-08 12:44 ` [PATCH 4/4] bundle-uri: enable git-remote-https progress Toon Claes
2024-05-09 16:46 ` Jeff King
2025-02-14 11:26 ` Toon Claes
2025-02-21 7:36 ` Jeff King [this message]
2024-05-08 23:49 ` [PATCH 0/4] bundle-uri: show progress when downloading from bundle URIs Junio C Hamano
2025-02-19 14:30 ` [PATCH v2 0/7] Show " Toon Claes
2025-02-19 14:30 ` [PATCH v2 1/7] progress: add function to set total Toon Claes
2025-02-21 7:43 ` Jeff King
2025-02-19 14:30 ` [PATCH v2 2/7] progress: allow pure-throughput progress meters Toon Claes
2025-02-19 14:30 ` [PATCH v2 3/7] http: turn off curl signals Toon Claes
2025-02-19 14:30 ` [PATCH v2 4/7] http: add the ability to log progress Toon Claes
2025-02-19 14:30 ` [PATCH v2 5/7] remote-curl: optionally show progress for HTTP get Toon Claes
2025-02-19 14:30 ` [PATCH v2 6/7] bundle-uri: enable git-remote-https progress Toon Claes
2025-02-19 14:30 ` [PATCH v2 7/7] http: silence stderr when progress is enabled Toon Claes
2025-02-21 7:48 ` Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250221073605.GA1988395@coredump.intra.peff.net \
--to=peff@peff.net \
--cc=git@vger.kernel.org \
--cc=toon@iotcl.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).