From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 2AAB71F487 for ; Fri, 6 Oct 2023 09:07:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1696583247; bh=JsddhBpxdAjP2nAZkJOu0J21naliQqX7PbZsOZsjngk=; h=From:To:Subject:Date:In-Reply-To:References:From; b=FumzSU5qX09d7clm+aNmsVw0gYqUDjksn1sUErJOdT7C6oX0MFMWd2IQvFKQFcM2C 1pEaR/kli7VJGnnKyErg2CIPU0WsqgdtbrmRu0PVhXxUu6/gL+R/DXJK0ADi6gjRDm XHysmocdVxFdVIXIqEiXs62CuAh73YL7vtSEuM0c= From: Eric Wong To: spew@80x24.org Subject: [PATCH 4/6] import: use autodie, rely on PerlIO for retries Date: Fri, 6 Oct 2023 09:07:24 +0000 Message-ID: <20231006090726.3936839-4-e@80x24.org> In-Reply-To: <20231006090726.3936839-1-e@80x24.org> References: <20231006090726.3936839-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: As documented in perlipc(1), the default :perlio layer retries the `read' perlop on EINTR. The :perlio layer also makes `read' perform read-in-full behavior; so there's no need to loop ourselves. Our code only needs to detect short reads in case fast-import is killed mid-stream. --- lib/PublicInbox/Import.pm | 40 ++++++++++++++------------------------- 1 file changed, 14 insertions(+), 26 deletions(-) diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm index 2386983a..44f355ec 100644 --- a/lib/PublicInbox/Import.pm +++ b/lib/PublicInbox/Import.pm @@ -17,13 +17,15 @@ use PublicInbox::ContentHash qw(content_digest); use PublicInbox::MDA; use PublicInbox::Eml; use POSIX qw(strftime); +use autodie qw(read close); +use Carp qw(croak); sub default_branch () { state $default_branch = do { my $r = popen_rd([qw(git config --global init.defaultBranch)], { GIT_CONFIG => undef }); chomp(my $h = <$r> // ''); - close $r; + CORE::close $r; $h eq '' ? 'refs/heads/master' : "refs/heads/$h"; } } @@ -106,20 +108,10 @@ sub _cat_blob ($$$) { local $/ = "\n"; my $info = <$r> // die "EOF from fast-import / cat-blob: $!"; $info =~ /\A[a-f0-9]{40,} blob ([0-9]+)\n\z/ or return; - my $left = $1; - my $offset = 0; - my $buf = ''; - my $n; - while ($left > 0) { - $n = read($r, $buf, $left, $offset) // - die "read cat-blob failed: $!"; - $n == 0 and die 'fast-export (cat-blob) died'; - $left -= $n; - $offset += $n; - } - $n = read($r, my $lf, 1) // - die "read final byte of cat-blob failed: $!"; - die "bad read on final byte: <$lf>" if $lf ne "\n"; + my $n = read($r, my $buf, my $len = $1 + 1); + $n == $len or croak "cat-blob: short read: $n < $len"; + my $lf = chop $buf; + croak "bad read on final byte: <$lf>" if $lf ne "\n"; # fixup some bugginess in old versions: $buf =~ s/\A[\r\n]*From [^\r\n]*\r?\n//s; @@ -472,9 +464,9 @@ EOM while (my ($fn, $contents) = splice(@fn_contents, 0, 2)) { my $f = $dir.'/'.$fn; next if -f $f; - open my $fh, '>', $f or die "open $f: $!"; - print $fh $contents or die "print $f: $!"; - close $fh or die "close $f: $!"; + open my $fh, '>', $f; + print $fh $contents; + close $fh; } } @@ -501,10 +493,7 @@ sub done { sub atfork_child { my ($self) = @_; - for my $f (qw(pp_rd pp_wr)) { - next unless defined($self->{$f}); - close $self->{$f} or die "failed to close import[$f]: $!\n"; - } + close($_) for (grep defined, delete(@$self{qw(pp_rd pp_wr)})); } sub digest2mid ($$;$) { @@ -575,10 +564,9 @@ sub replace_oids { push @buf, "commit $tmp\n"; } elsif (/^data ([0-9]+)/) { # only commit message, so $len is small: - my $len = $1; # + 1 for trailing "\n" push @buf, $_; - my $n = read($rd, my $buf, $len) or die "read: $!"; - $len == $n or die "short read ($n < $len)"; + my $n = read($rd, my $buf, my $len = $1); + $len == $n or croak "short read ($n < $len)"; push @buf, $buf; } elsif (/^M 100644 ([a-f0-9]+) (\w+)/) { my ($oid, $path) = ($1, $2); @@ -617,7 +605,7 @@ sub replace_oids { push @buf, $_; } } - close $rd or die "close fast-export failed: $?"; + close $rd; if (@buf) { print $w @buf or wfail; }