From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.0 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 3812E1F404; Sun, 29 Apr 2018 09:02:50 +0000 (UTC) Date: Sun, 29 Apr 2018 09:02:50 +0000 From: Eric Wong To: spew@80x24.org Subject: [PATCH 3/2] thread.c (do_select): perform GC if idle Message-ID: <20180429090250.GA15634@dcvr> References: <20180429035007.6499-1-e@80x24.org> <20180429035007.6499-3-e@80x24.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20180429035007.6499-3-e@80x24.org> List-Id: Continuing on the GC-on-idle idea, the process may perform useful GC work if we are waiting on the select(2) system call. Typical use of IO.select on Linux systems is NOT after EAGAIN (unlike rb_wait_for_single_fd-with-ppoll), so we have fewer heuristics on whether or not initial select(2) will succeed. Hence, we make initial select() call before attempting GC. Also, explain the difference in ordering for the previous ppoll patch. The following script goes from 56MB to around 32MB depending on entropy in the system. Thread.abort_on_exception = true len = 100_000_000 rd, wr = IO.pipe readers = 10.times.map { rd.dup } th = Thread.new do IO.copy_stream('/dev/urandom', wr, len) wr.close end until readers.empty? ready = IO.select(readers) ready[0].each do |r| r.read(16384) or readers.clear end end th.join --- thread.c | 38 +++++++++++++++++++++++++++++--------- 1 file changed, 29 insertions(+), 9 deletions(-) diff --git a/thread.c b/thread.c index 4725c809b8..b361b6dd26 100644 --- a/thread.c +++ b/thread.c @@ -3825,6 +3825,7 @@ do_select(int n, rb_fdset_t *const readfds, rb_fdset_t *const writefds, rb_fdset_t MAYBE_UNUSED(orig_except); struct timespec ts, end, *tsp; rb_thread_t *th = GET_THREAD(); + int do_gc = 1; /* we call select() before rb_gc_step() anyways */ timeout_prepare(&tsp, &ts, &end, timeout); #define do_select_update() \ @@ -3842,15 +3843,29 @@ do_select(int n, rb_fdset_t *const readfds, rb_fdset_t *const writefds, do { lerrno = 0; - - BLOCKING_REGION({ - result = native_fd_select(n, readfds, writefds, exceptfds, - timeval_for(timeout, tsp), th); - if (result < 0) lerrno = errno; - }, ubf_select, th, FALSE); - - RUBY_VM_CHECK_INTS_BLOCKING(th->ec); - } while (result < 0 && retryable(errno = lerrno) && do_select_update()); + if (!do_gc || gvl_contended_p(th->vm)) { + BLOCKING_REGION({ + result = native_fd_select(n, readfds, writefds, exceptfds, + timeval_for(timeout, tsp), th); + if (result < 0) lerrno = errno; + }, ubf_select, th, FALSE); + RUBY_VM_CHECK_INTS_BLOCKING(th->ec); + } + else { /* no need to release GVL if nobody is waiting for it */ + static struct timeval zero; + + /* + * For IO.select callers sometimes do NOT hit EAGAIN before + * calling this function, so native_fd_select may succeed + * on the first try before we get to GC. + */ + result = native_fd_select(n, readfds, writefds, exceptfds, + &zero, th); + if (result < 0) lerrno = errno; + if (result == 0) do_gc = rb_gc_step(th->ec); + } + } while ((result == 0 || (result < 0 && retryable(errno = lerrno))) && + do_select_update()); #define fd_term(f) if (f##fds) rb_fd_term(&orig_##f) fd_term(read); @@ -3984,6 +3999,11 @@ rb_wait_for_single_fd(int fd, int events, struct timeval *timeout) RUBY_VM_CHECK_INTS_BLOCKING(th->ec); } else { /* no need to release GVL if nobody is waiting for it */ + /* + * we typically enter this function on EAGAIN, so we have + * a low likelyhood of ppoll succeeding right away, thus + * we do GC before initial ppoll + */ do_gc = rb_gc_step(th->ec); result = ppoll(&fds, 1, &zero, NULL); if (result < 0) lerrno = errno; -- EW