about summary refs log tree commit homepage
DateCommit message (Collapse)
2023-08-02f3: fix test with updated executable name fuse3
Only noticed since this was on a new PERL_INLINE_DIRECTORY; not sure if there's a good way to test this without blowing away caches and slowing down tests even more :<
2023-08-02f3: define F3_NS in cflags rather than C code
This enhances reusability in other projects which use the same C shim. The executable will also be `lei.fuse' rather than `leifs', because `leifs' is likely confused as `leafs' which makes no sense.
2023-08-02f3: safer fixed buffer size for alternate F3_NS
This accounts for longer F3_NS values if we need them.
2023-02-27f3: opendir: set keep_cache, too
cache_readdir and keep_cache are independent flags, so we honor the global keep_cache.
2023-02-22fuse3: less error prone inode_acq/inode_rel API
We can return the locked mutex to the caller which makes it more apparent we need to unlock the mutex.
2023-02-20preliminary FUSE 3 support shim
This shim is imported from another (as-yet-unannounced) AGPL-3+ project I started way back in May 2021. It turns out the small C99 + libfuse3 shim is pretty generic aside from its name and can be easily repurposed for any language or FUSE project. I'm avoiding existing Perl FUSE bindings since they're out-of-date w.r.t. FUSE 3 support, and Perl5 has a thread-unfriendly standard library. FUSE 3 adds readdirplus which is beneficial for Maildir, and FUSE FSes are basically unusable without threads.
2023-02-20README: add POP3 bits
Maybe this can make our newish support of POP3 more noticeable...
2023-02-20searchidx: do not index quoted Base-85 patches
Base-85 binary patches were a source of false-positives in results and we've filtered out in non-quoted text since July 2022. Unfortunately, people were quoting binary patch contents in replies (*sigh*) and triggering false positives in search results. So we must filter out base-85-looking contents from quoted text, too. Followup-to: 8fda04081acde705 (search: do not index base-85 binary patches, 2022-06-20) Followup-to: 840785917bc74c8e (searchidx: skip "delta $N" sections for base-85, 2022-07-19)
2023-02-20multi_git: do not set include.path if already set
The epoch may already be read-only, and we don't need to cause more I/O traffic and disk wear for no-op stuff. This fixes idempotent use of public-inbox-clone to update multi-epoch inboxes.
2023-02-20git_async_cat: don't mis-abort replaced process
When a git process gets replaced (e.g. due to new epochs/alternates), we must be careful and not abort the wrong one. I suspect this fixes the problem exacerbated by --batch-command. It was theoretically possible w/o --batch-command, but it seems to have made it surface more readily. This should fix "Failed to retrieve generated blob" errors from PublicInbox/ViewVCS.pm appearing in syslog Link: https://public-inbox.org/meta/20230209012932.M934961@dcvr/
2023-02-19search: translate d: to dt: in query
dt: is higher resolution and the YYYYMMDD column will be dropped if there's ever another SCHEMA_VERSION update. While the upcoming code repo index is independent of the mail schemas, it'll use similar query prefixes and likely use d:/dt: for Author Date of git commits.
2023-02-17search: move query transform + enquire setup out of retry loop
The Xapian query transformation and Enquire object setup aren't subject to MVCC and retries, so move it outside the retry loop to save some cycles in case we need to retry on a busy DB.
2023-02-17public-inbox.cgi(1): Mention AllowEncodedSlashes for Apache setups
When AllowEncodedSlashes is Off (the default setting), URLs containing %2f are replied with a 404 error without calling the CGI. To (maybe) prevent others debugging this issue add a hint with the solution.
2023-02-17TODO: handle more cases of unencoded slashes
Nowadays, mutt defaults to Message-IDs with `/' in them :<
2023-02-16Makefile.PL: drop update-copyrights rule
I'm no longer updating them since it's noisy and acceptable to not have them: https://www.linuxfoundation.org/blog/copyright-notices-in-open-source-software-projects/ I'm tired of being reminded what year it is :<
2023-02-16doc: extindex update on configuration and union section
The coderepo indexer will use similar ideas, I think...
2023-02-16doc: flow: update with newer tools, note forkability
public-inbox-{clone,fetch,netd} are all relatively new developments which we can document, here. We'll also update the generator Makefile snippet since there may be more Graph::Easy-based docs coming.
2023-02-16doc: WWW + cgi: favor -netd over -httpd
-netd is strictly more powerful and a gateway drug for imapd/nntpd/pop3d instances :>
2023-02-15www_coderepo: handle unborn/dead branches in summary
We need to account for `git log' showing nothing for invalid branches and continue to render properly. We'll also quiet down `git log' stderr to avoid cluttering stderr, too.
2023-02-15www_coderepo: quiet 404s on Atom feeds for dead branches
No need to clutter up logs when a request hits a dead branch.
2023-02-15lei q: do not collapse threads with `-tt'
While having Xapian collapse threads is an easy way to reduce the amount of deduplication work we need to do when writing out threads; we can't rely on it when using `lei q -tt` since that needs to flag all hits. Reported-by: Maxim Mikityanskiy <maxtram95@gmail.com> Link: https://public-inbox.org/git/Y+pgBmj0jxR+cVkD@mail.gmail.com/
2023-02-13imap: quiet Parse::RecDescent errors on bad search queries
Parse::RecDescent emits giant errors to STDERR by default (bypassing $SIG{__WARN__}, even). Shut it up since there's no good way to pass those back to a client, and we don't want clients flooding logs with bogus requests.
2023-02-13lei_mirror: fetch most-recently-updated repos, first
Within the same forkgroup, we can assume the most recently updated repo has the most data, so fetch those, first. We'll save new clones for last since we can preserve {reference} ordering for them.
2023-02-13lei_mirror: further reduce `git config' calls
We can parse the config at once and avoid clobbering variables which do not need changing. We'll also do some prep work for fetch.hideRefs proposal being discussed at <https://public-inbox.org/git/20230209122857.M669733@dcvr/>
2023-02-12t/lei-refresh-mail-sync: avoid kill+sleep loop
While we can't waitpid() on daemonized process, we can abuse the lack of FD_CLOEXEC to detect a process death. This saves roughly 400ms for this slow test.
2023-02-10git_async_cat: use awaitpid
While awaitpid already registered a no-op callback in _bidi_pipe, we can still call it again when registering it into our event loop to ensure EPOLL_CTL_DEL fires.
2023-02-10lei_mirror: avoid dir/file conflicts in update-ref
Using the files ref backend for git, `delete' and `create' operations for `update-ref --stdin' need to be processed in separate transactions to avoid conflicts in cases where a file becomes a directory (or presumably, vice versa).
2023-02-09spawn_pp: fix incorrect `use'
We can't `use PublicInbox::Spawn' from SpawnPP because PublicInbox::Spawn loads SpawnPP from BEGIN. Fixes: 9eb8baf199cd148b (spawn_pp: use `which()' properly for pure-Perl spawn, 2023-01-29)
2023-02-09lei_mirror: show non-ASCII owner properly w/ --verbose
This makes the verbose progress output look nicer, but doesn't affect the actual config file generation.
2023-02-06lei_mirror: reduce `git config' usage
We can use `git -c $KEY=$VAL fetch' with a random remote name that never makes it to a config file.
2023-02-04www: sort all /$INBOX/ topics by Received: timestamp
Our previous pinning prevention only worked to prevent older (non-most-recent) topics from being pinned to the landing page, but not the most recent window of messages. We still sort messages within threads by Date: because that makes git-send-email patchsets display more nicely, but we don't want recent topics pinned due to future Date: headers. I nearly switched sort_ds() back to sorting by Received: until I looked back on commit 8e52e5fdea416d6fda0b8d301144af0c043a5a76 (use both Date: and Received: times, 2018-03-21) and was reminded git-send-email relies on Date: for large series, so I added a note about it for sort_ds(). Reported-by: Kyle Meyer <kyle@kyleam.com> Tested-by: Kyle Meyer <kyle@kyleam.com> Link: https://public-inbox.org/meta/87edr5gx63.fsf@kyleam.com/
2023-02-03lei_mirror: use --no-write-fetch-head on git 2.29+
This avoids unnecessary writes to the FETCH_HEAD file, which is worthless in multi-remote mirrors. Actually, I haven't found FETCH_HEAD useful anywhere since the `/remotes/' namespace became popular...
2023-01-31www: diff: fix encoding problems when showing diff
We need to use the utf8 layer when writing files to be diffed, and utf8::decode the `git diff' output. Furthermore, do the CRLF > LF conversion early to avoid showing CRLF vs LF differences in the diff, since that doesn't matter to MUAs (nor our normal HTML views)
2023-01-31lei: drop -watches and -lei_note_event from workers
I noticed these while tracking down circular refs for commit 7b654d175cf2e31b (ipc: drop awaitpid_init to avoid circular refs, 2023-01-30). While they're not the cause of circular refs, they're still a waste of memory in worker processes.
2023-01-31tests: make require_git and require_cmd easier-to-use
We'll rely on defined(wantarray) to implicitly skip subtests, and memoize these to reduce syscalls, since tests should be short-lived enough to not be affected by new installations or removals of git/xapian-compact/curl/etc...
2023-01-30tests: make slow tests easier-to-find
t/run.perl now prints slowest 10 tests at startup, and I've added ./devel/longest-tests to print all tests sorted by elapsed time. This should allow us to notice outliers more quickly in the future.
2023-01-30ipc: drop awaitpid_init to avoid circular refs
This brings t/lei-index.t back down from ~8 to ~3s. I didn't notice this before was because the LeiNoteEvent timer was firing every 5s and clearing circular refs and parallel testing meant the delay got hidden. Fixes: 4a2a95bbc78f99c8 (ipc+lei: switch to awaitpid, 2023-01-17)
2023-01-30xt/lei-auth-fail: use valid label name
Uppercase characters aren't allowed for labels due to Xapian boolean limitations, so we need to use lowercase labels. Fixes: 27015c3365fd0690 (lei_input: disallow uppercase characters for labels, 2021-10-31)
2023-01-30lei_input: give a hint for upper-case in labels
I just encountered this error in xt/lei-auth-fail.t
2023-01-30content_digest_dbg: convert to arrayref and limit to lei
Since it's an extremely small class and not subclassed or anything, we'll make it even smaller as an arrayref. We also don't load this for PublicInbox::WWW or anything that runs in public-facing daemons.
2023-01-30use Net::SSLeay (OpenSSL) for SHA-(1|256) if installed
On my x86-64 machine, OpenSSL SHA-256 is nearly twice as fast as the Digest::SHA implementation from Perl, most likely due to an optimized assembly implementation. SHA-1 is a few percent faster, too.
2023-01-30spawn_pp: use `which()' properly for pure-Perl spawn
I have no idea if mod_perl/mod_perl2 is used nowadays, but we're stuck supporting it as long as mod_perl exists. So add some tests and make minor updates to existing ones to ensure it stays working.
2023-01-28www_coderepo: summary: fix mis-linkification of `...'
We need to use the ternary operator in assignments to clobber previous values of `$last'.
2023-01-28www_coderepo: support $REPO/refs/{heads,tags}/ endpoints
These are also in cgit, but we'll include CLI hints to show viewers how our data is generated. We don't have "$REPO/refs/" without (heads|tags) yet, though...
2023-01-28repo_atom: translate: account for multiple args
->translate should handle unlimited args, even if we don't currently use it that way...
2023-01-28www_coderepo: reduce utf8::decode calls
It's safe to call utf8::decode on data where "\0" exists.
2023-01-28www_coderepo: fix snapshot link generation
Do not assume ".git" exists as a suffix in the repo nickname, and filter out all trailing slashes in case it didn't get filtered from Config.
2023-01-28www_coderepo: support /$REPO/tags.atom endpoint
Providing an Atom feed for tags can be a nice way for users to subscribe to new releases without excessive noise.
2023-01-28www_coderepo: tree: quiet and 404 on non-existent refs
Clients should see 404s when attempting to hit files for deleted branches or tags.
2023-01-27git: drop needless checks for old git
`ambiguous' was added in git 2.21, and `dangling' was the only other possible phrase which was inadvertantly slipped in prior to 2.21. Thus there's no need to check for `notdir' or `loop' responses since we aren't using `git cat-file --follow-symlinks' anywhere.