about summary refs log tree commit homepage
path: root/lib/PublicInbox/LeiExportKw.pm
diff options
context:
space:
mode:
authorEric Wong <e@80x24.org>2021-09-02 10:17:58 +0000
committerEric Wong <e@80x24.org>2021-09-02 21:22:50 +0000
commit4166c482b710acea785063de2c69a8a370c6d373 (patch)
treeb6dfb983908d485b66f6e6140b45f71a6fcae251 /lib/PublicInbox/LeiExportKw.pm
parent4f9c44e3fff9c413fc54050dcc633692d33f6968 (diff)
downloadpublic-inbox-4166c482b710acea785063de2c69a8a370c6d373.tar.gz
This works with existing inotify/EVFILT_VNODE functionality to
propagate changes made from one Maildir to another Maildir.

I chose the lei/store worker process to handle this since
propagating changes back into lei-daemon on a massive scale
could lead to dead-locking while both processes are attempting
to write to each other.  Eliminating IPC overhead is a nice
side effect, but could hurt performance if Maildirs are slow.

The code for "lei export-kw" is significantly revamped to match
the new code used in the "lei/store" daemon.  It should be more
correct w.r.t. corner-cases and stale entries, but perhaps
better tests need to be written.

squashed:
  t/lei-auto-watch: increase delay for FreeBSD kevent

  My FreeBSD VM seems to need longer for this test than inotify
  under Linux, likely because the kevent support code needs to be
  more complicated.
Diffstat (limited to 'lib/PublicInbox/LeiExportKw.pm')
-rw-r--r--lib/PublicInbox/LeiExportKw.pm24
1 files changed, 11 insertions, 13 deletions
diff --git a/lib/PublicInbox/LeiExportKw.pm b/lib/PublicInbox/LeiExportKw.pm
index 42a5ff22..78c6c6f9 100644
--- a/lib/PublicInbox/LeiExportKw.pm
+++ b/lib/PublicInbox/LeiExportKw.pm
@@ -25,12 +25,11 @@ sub export_kw_md { # LeiMailSync->each_src callback
         }
         $bn .= ':2,'.
                 PublicInbox::LeiToMail::kw2suffix([keys %$sto_kw], @$unknown);
+        return if $bn eq $$id;
         my $dst = "$mdir/cur/$bn";
-        my @fail;
         my $lei = $self->{lei};
         for my $d (@try) {
                 my $src = "$mdir/$d/$$id";
-                next if $src eq $dst;
 
                 # we use link(2) + unlink(2) since rename(2) may
                 # inadvertently clobber if the "uniquefilename" part wasn't
@@ -44,20 +43,19 @@ sub export_kw_md { # LeiMailSync->each_src callback
                         $lei->{sto}->ipc_do('lms_mv_src', "maildir:$mdir",
                                                 $oidbin, $id, $bn);
                         return; # success anyways if link(2) worked
-                }
-                if ($! == ENOENT && !-e $src) { # some other process moved it
-                        $lei->{sto}->ipc_do('lms_clear_src',
-                                                "maildir:$mdir", $id);
-                        next;
-                }
-                push @fail, $src if $! != EEXIST;
+                } elsif ($! == EEXIST) { # lost race with lei/store?
+                        return;
+                } elsif ($! != ENOENT) {
+                        $lei->child_error(1, "E: link($src -> $dst): $!");
+                } # else loop @try
         }
-        return unless @fail;
-        # both tries failed
         my $e = $!;
-        my $orig = '['.join('|', @fail).']';
+        # both tries failed
         my $oidhex = unpack('H*', $oidbin);
-        $lei->child_error(1, "link($orig, $dst) ($oidhex): $e");
+        my $src = "$mdir/{".join(',', @try)."}/$$id";
+        $lei->child_error(1, "link($src -> $dst) ($oidhex): $e");
+        for (@try) { return if -e "$mdir/$_/$$id" }
+        $lei->{sto}->ipc_do('lms_clear_src', "maildir:$mdir", $id);
 }
 
 sub export_kw_imap { # LeiMailSync->each_src callback