about summary refs log tree commit homepage
path: root/script/public-inbox-watch
DateCommit message (Collapse)
2020-09-14sigfd: fix typos and scoping on systems w/o epoll+kqueue
Unfortunately, I'm not sure how easy catching these at compile-time, is. Prototypes do not seem to check these at compile time when crossing packages (not even with exported subroutines).
2020-09-02watch: add --help/-h support
And avoid unnecessary POD markup in the man page.
2020-09-01watch: log signal activities to STDERR
Sometimes it may not be apparent when/if a signal is processed, this hopefully improves the situation. We'll also change the process title when we're quitting to better inform users.
2020-09-01rename WatchMaildir => Watch
This is no longer limited to Maildirs now that IMAP and NNTP support exist; so give it a shorter name.
2020-08-07syscall: support sparc64 (and maybe other big-endian systems)
Thanks to the GCC compile farm project, we can wire up syscalls for sparc64 and set system-specific SFD_* constants properly. I've FINALLY figured out how to use POSIX::SigSet to generate a usable buffer for the syscall perlfunc. This is required for endian-neutral behavior and relevant to sparc64, at least. There's no need for signalfd-related stuff to be constants, either. signalfd initialization is never a hot path and a stub subroutine for constants uses several KB of memory in the interpreter. We'll drop the needless SEEK_CUR import while we're importing O_NONBLOCK, too.
2020-06-28watch: enable autoflush for STDOUT and STDERR
In case output is redirected to a pipe, ensure stdout and stderr are always unbuffered, as -watch may go long periods without any output to fill up buffers.
2020-06-28watch: wire up IMAP IDLE reapers to DS
We can avoid synchronous `waitpid(-1, 0)' and save a process when simultaneously watching Maildirs. One DS bug is fixed: ->Reset needs to clear the DS $in_loop flag in forked children so dwaitpid() fails and allows git processes to be reaped synchronously. TestCommon also calls DS->Reset when spawning new processes, since t/imapd.t uses DS->EventLoop while waiting on -watch to write.
2020-06-28watch: use signalfd for Maildir watching
We can get rid of the janky wannabe self-using-a-directory-instead-of-pipe thing we needed to workaround Filesys::Notify::Simple being blocking. For existing Maildir users, this should be more robust and immune to missed wakeups for signalfd and kqueue-enabled systems; as well as being immune to BOFHs clearing $TMPDIR and preventing notifications from firing. The IMAP IDLE code still uses normal Perl signals, so it's still vulnerable to missed wakeups. That will be addressed in future commits.
2020-06-28watch: remove Filesys::Notify::Simple dependency
Since we already use inotify and EVFILT_VNODE (kqueue) in -imapd, we might as well use them directly in -watch, too. This will allow public-inbox-watch to use PublicInbox::DS for timers to watch newsgroups/mailboxes and have saner signal handling in future commits.
2020-02-06treewide: run update-copyrights from gnulib for 2019
I didn't wait until September to do it, this year!
2019-09-09run update-copyrights from gnulib for 2019
2018-02-07update copyrights for 2018
Using update-copyrights from gnulib While we're at it, use the SPDX identifier for AGPL-3.0+ to ease mechanical processing.
2017-06-26watch: use "self-inotify-tempfile trick" for quit
This should be more reliable and safer as it'll ensure existing fast-import instances are shut down properly.
2017-06-26watch: improve fairness during full rescans
We need to ensure new messages are being processed fairly during full rescans, so have the ->scan subroutine yield and reschedule itself. Additionally, having a long-running task inside the signal handler is dangerous and subject to reentrancy bugs. Due to the limitations of the Filesys::Notify::Simple interface, we cannot rely on multiplexing I/O interfaces (select, IO::Poll, Danga::Socket, etc...) for this. Forking a separate process was considered, but it is more expensive for a mostly-idle process. So, we use a variant of the "self-pipe trick" via inotify (or whatever Filesys::Notify::Simple gives us). Instead of writing to our own pipe, we write to a file in our own temporary directory watched by Filesys::Notify::Simple to trigger events in signal handlers.
2017-06-26watch: ensure HUP causes the scanner to be reloaded
Otherwise the old watcher may run indefinitely
2016-08-12public-inbox-watch: support reloading config with SIGHUP
This can be useful for adding new lists, as restarting is expensive (but still non-lossy).
2016-06-17watch: introduce watch directive
This will allow users to run importers off existing mail accounts where they may not have access to run -mda. Currently, we only support Maildirs, but IMAP ought to be doable.