From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS12876 163.172.0.0/16 X-Spam-Status: No, score=-1.1 required=3.0 tests=AWL,BAYES_00, RCVD_IN_MSPIKE_BL,RCVD_IN_MSPIKE_ZBI,RCVD_IN_XBL,SPF_FAIL,SPF_HELO_FAIL shortcircuit=no autolearn=no autolearn_force=no version=3.4.0 Received: from 80x24.org (torrelay6.tomhek.net [163.172.38.175]) by dcvr.yhbt.net (Postfix) with ESMTP id 944BA1F859 for ; Sun, 14 Aug 2016 10:21:21 +0000 (UTC) From: Eric Wong To: spew@80x24.org Subject: [PATCH 02/11] searchidx: do not release Xapian lock while (only) Msgmap is indexing Date: Sun, 14 Aug 2016 10:21:08 +0000 Message-Id: <20160814102117.11456-2-e@80x24.org> In-Reply-To: <20160814102117.11456-1-e@80x24.org> References: <20160814102117.11456-1-e@80x24.org> List-Id: SQLite might index quickly, so we hold the lock used by Xapian for the duration. This probably needs to be reworked entirely, actually. --- lib/PublicInbox/SearchIdx.pm | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/lib/PublicInbox/SearchIdx.pm b/lib/PublicInbox/SearchIdx.pm index 0eb07a1..f8155ec 100644 --- a/lib/PublicInbox/SearchIdx.pm +++ b/lib/PublicInbox/SearchIdx.pm @@ -436,19 +436,24 @@ sub _index_sync { my $mm = _msgmap_init($self); my $dbh = $mm->{dbh} if $mm; + my $mm_only; my $cb = sub { my ($commit, $more) = @_; if ($dbh) { $mm->last_commit($commit) if $commit; $dbh->commit; } - $xdb->set_metadata($mkey, $commit) if $mkey && $commit; - $xdb->commit_transaction; - $xdb = _xdb_release($self); + if (!$mm_only) { + $xdb->set_metadata($mkey, $commit) if $mkey && $commit; + $xdb->commit_transaction; + $xdb = _xdb_release($self); + } # let another process do some work... < if ($more) { - $xdb = _xdb_acquire($self); - $xdb->begin_transaction; + if (!$mm_only) { + $xdb = _xdb_acquire($self); + $xdb->begin_transaction; + } $dbh->begin_work if $dbh; } }; @@ -472,14 +477,13 @@ sub _index_sync { my $mkey_prev = $mkey; $mkey = undef; # ignore xapian, for now my $mlog = _git_log($self, $r); + $mm_only = 1; rlog($self, $mlog, *index_mm, *unindex_mm, $cb); - $mlog = undef; + $mm_only = $mlog = undef; # now deal with Xapian $mkey = $mkey_prev; $dbh = undef; - $xdb = _xdb_acquire($self); - $xdb->begin_transaction; rlog($self, $xlog, *index_mm2, *unindex_mm2, $cb); } } else { -- EW