Git Mailing List Archive mirror
 help / color / mirror / Atom feed
From: Phillip Wood <phillip.wood123@gmail.com>
To: Glen Choo <chooglen@google.com>,
	steadmon@google.com, git@vger.kernel.org
Cc: szeder.dev@gmail.com, avarab@gmail.com, gitster@pobox.com,
	sandals@crustytoothpaste.net, Calvin Wan <calvinwan@google.com>
Subject: Re: [PATCH RFC v2 2/4] unit tests: Add a project plan document
Date: Thu, 1 Jun 2023 10:19:23 +0100	[thread overview]
Message-ID: <73580406-f2cd-12ff-954a-2adb73b0edc8@gmail.com> (raw)
In-Reply-To: <kl6lzg61xuox.fsf@chooglen-macbookpro.roam.corp.google.com>

Hi Glen

On 18/05/2023 21:15, Glen Choo wrote:
> steadmon@google.com writes:
 >
> A third option would be to pick another, more mature third party testing
> library. As I mentioned in
> 
>    https://lore.kernel.org/git/kl6lpm76zcg7.fsf@chooglen-macbookpro.roam.corp.google.com
> 
> my primary concern is the maintainability and extensibility of a third
> party library that (no offense to the original author) is not used very
> widely, is relatively underdocumented, is missing features that we want,
> and whose maintenance policy is relatively unknown to us. I'm not
> opposed to taking in a third party testing framework, but we need to be
> sure that we can rely on it instead of being something that requires
> active upkeep.
> 
> I don't what sorts of testing libraries exist for C or how widely they
> are used, but a quick web search gives some candidates that seem like
> plausible alternatives to C TAP Harness:
> 
> - cmocka https://cmocka.org/ supports TAP, assert macros and mocking,
>    and is used by other projects (their website indicates Samba, OpenVPN,
>    etc). The documentation is a bit lacking IMO. It's apparently a fork
>    of cmockery, which I'm not familiar with.
> 
> - Check https://libcheck.github.io/check/ supports TAP and has
>    relatively good documentation, though the last release seems to have
>    been 3 years ago.
> 
> - µnit https://nemequ.github.io/munit/ has a shiny website with nice
>    docs (and a handy list of other unit test frameworks we can look at).
>    The last release also seems to be ~3 years ago. Not sure if this
>    supports TAP.

Thanks for finding those. I'd add

  - libtap https://github.com/zorgnax/libtap which seems to have nice
      assertions and TAP output. It is licensed under LGPL3 but used
      to be GPL2 and there don't seem to have been many changes since
      the license change so we could just use the older version.

I think it would be helpful to have a section in the plan which clearly 
states the features we want (TAP output, helpers that make it easy to 
write tests with good diagnostic output, ...) in our unit test framework 
and explains how the selected solution meets those criteria.

It would also be helpful to have a guide to writing effective tests that 
sets out some guidelines on where unit tests are appropriate and 
outlines our expectations in terms of style, coverage, leak cleanliness etc.

Best Wishes

Phillip

> For flexibility, I also think it's reasonable for us to roll our own
> testing library. I think it is perfectly fine for our unit test
> framework to be suboptimal at the beginning, but owning the code makes
> it relatively easy to fix bugs and extend it to our liking.

  parent reply	other threads:[~2023-06-01  9:19 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-05-17 23:56 [PATCH RFC v2 0/4] Add an external testing library for unit tests steadmon
2023-05-17 23:56 ` [PATCH RFC v2 1/4] common-main: split common_exit() into a new file steadmon
2023-05-18 17:17   ` Junio C Hamano
2023-07-14 23:38     ` Splitting common-main (Was: Re: [PATCH RFC v2 1/4] common-main: split common_exit() into a new file) Josh Steadmon
2023-07-15  0:34       ` Splitting common-main Junio C Hamano
2023-08-14 13:09       ` Splitting common-main (Was: Re: [PATCH RFC v2 1/4] common-main: split common_exit() into a new file) Jeff Hostetler
2023-05-17 23:56 ` [PATCH RFC v2 2/4] unit tests: Add a project plan document steadmon
2023-05-18 13:13   ` Phillip Wood
2023-05-18 20:15   ` Glen Choo
2023-05-24 17:40     ` Josh Steadmon
2023-06-01  9:19     ` Phillip Wood [this message]
2023-05-17 23:56 ` [PATCH RFC v2 3/4] Add C TAP harness steadmon
2023-05-18 13:15   ` Phillip Wood
2023-05-18 20:50     ` Josh Steadmon
2023-05-17 23:56 ` [PATCH RFC v2 4/4] unit test: add basic example and build rules steadmon
2023-05-18 13:32   ` Phillip Wood
2023-06-09 23:25 ` [RFC PATCH v3 0/1] Add a project document for adding unit tests Josh Steadmon
2023-06-09 23:25 ` [RFC PATCH v3 1/1] unit tests: Add a project plan document Josh Steadmon
2023-06-13 22:30   ` Junio C Hamano
2023-06-30 22:18     ` Josh Steadmon
2023-06-29 19:42   ` Linus Arver
2023-06-29 20:48     ` Josh Steadmon
2023-06-30 19:31       ` Linus Arver
2023-07-06 18:24         ` Glen Choo
2023-07-06 19:02           ` Junio C Hamano
2023-07-06 22:48             ` Glen Choo
2023-06-30 21:33       ` Josh Steadmon
2023-06-29 21:21     ` Junio C Hamano
2023-06-30  0:11       ` Linus Arver
2023-06-30 14:07   ` Phillip Wood
2023-06-30 18:47     ` K Wan
2023-06-30 22:35     ` Josh Steadmon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=73580406-f2cd-12ff-954a-2adb73b0edc8@gmail.com \
    --to=phillip.wood123@gmail.com \
    --cc=avarab@gmail.com \
    --cc=calvinwan@google.com \
    --cc=chooglen@google.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=phillip.wood@dunelm.org.uk \
    --cc=sandals@crustytoothpaste.net \
    --cc=steadmon@google.com \
    --cc=szeder.dev@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).