From: Phillip Wood <phillip.wood123@gmail.com>
To: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>, phillip.wood@dunelm.org.uk
Cc: Calvin Wan <calvinwan@google.com>, git@vger.kernel.org
Subject: Re: [RFC PATCH 1/2] Add C TAP harness
Date: Wed, 10 May 2023 09:18:35 +0100 [thread overview]
Message-ID: <3808fc35-6d3d-5a91-3eac-9428ff0e8045@gmail.com> (raw)
In-Reply-To: <230502.86wn1qhemd.gmgdl@evledraar.gmail.com>
Hi Ævar
On 02/05/2023 17:34, Ævar Arnfjörð Bjarmason wrote:
>
> On Thu, Apr 27 2023, Phillip Wood wrote:
>
>> Hi Calvin
>>
>> On 27/04/2023 18:50, Calvin Wan wrote:
>>> Introduces the C TAP harness from https://github.com/rra/c-tap-harness/
>>> There is also more complete documentation at
>>> https://www.eyrie.org/~eagle/software/c-tap-harness/
>>
>> I'm afraid this reply is rather briefer than I'd like but I'm short of
>> time and about to go off-list for a couple of weeks. My ideal unit
>> test library would
>>
>> - print the file and line number of failed assertions
>> - allow the test plan to be omitted by calling test_done() at the end
>> of the test file as we do in our main test suite.
>> - support the TODO directive
>> - allow named tests (this maybe more trouble that it is worth as I
>> think it inevitably leads to more boilerplate code calling the named
>> tests)
>>
>> Unfortunately this library doesn't seem to offer any of those
>> features. It does support a lazy test plan but uses atexit() so will
>> not detect if the test program exits before all the tests have run. I
>> think it would be useful to add some unit tests to our test suite and
>> maybe this library could form the basis of that but I think printing
>> the file and line number of failed assertions is pretty essential.
>
> Other things aside, I prefer our explicit "test_done", but I don't see
> why you think an atexit() isn't enough to catch incomplete tests.
>
> For a C program you'd just do something like this (somewhat pseudocode,
> I didn't check if it compiled etc):
>
> static int done; /* read by atexit() handler */
>
> void on_atexit(void)
> {
> if (!done)
> BUG();
> print_plan_line();
> }
>
> int main(void)
> {
> int ret;
>
> setup_atexit(a_handler);
> ret = do_tests();
> done = 1;
>
> return ret;
> }
>
> If I'm understanding you correctly you're concerned that if some user
> code within do_test() calls exit() we won't return from "do_test()", but
> we *would* call print_plan_line().
Exactly
> That's a valid concern, we want to distinguish such "early return" from
> cases where we run to completion, that's why we use "test_done" in the
> shell code.
>
> But in the C case I think just using something like the "done" variable
> pattern above should cover that, without the need for an explicit
> "test_done".
We could do that. My complaint is that the code being proposed does not
and so prints a valid plan if any code being tested calls exit()
Best Wishes
Phillip
next prev parent reply other threads:[~2023-05-10 8:18 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-04-27 17:50 [RFC PATCH 0/2] add an external testing library for unit tests Calvin Wan
2023-04-27 17:50 ` [RFC PATCH 1/2] Add C TAP harness Calvin Wan
2023-04-27 18:29 ` SZEDER Gábor
2023-04-27 18:38 ` Calvin Wan
2023-04-27 20:15 ` Phillip Wood
2023-04-28 16:31 ` Calvin Wan
2023-05-02 15:46 ` Felipe Contreras
2023-05-10 15:46 ` Phillip Wood
2023-05-11 23:16 ` Glen Choo
2023-05-18 10:04 ` Phillip Wood
2023-06-21 15:57 ` Linus Arver
2023-06-26 13:15 ` Phillip Wood
2023-06-28 21:17 ` Linus Arver
2023-06-29 5:52 ` Oswald Buddenhagen
2023-06-30 9:48 ` Phillip Wood
2023-05-02 15:54 ` Felipe Contreras
2023-05-02 16:39 ` Ævar Arnfjörð Bjarmason
2023-05-02 18:11 ` Felipe Contreras
2023-05-02 16:34 ` Ævar Arnfjörð Bjarmason
2023-05-10 8:18 ` Phillip Wood [this message]
2023-04-27 17:50 ` [RFC PATCH 2/2] unit test: add basic example Calvin Wan
2023-04-27 18:47 ` Junio C Hamano
2023-05-02 15:58 ` Felipe Contreras
2023-04-27 18:39 ` [RFC PATCH 0/2] add an external testing library for unit tests Junio C Hamano
2023-04-27 18:46 ` Calvin Wan
2023-04-27 21:35 ` brian m. carlson
2023-05-02 4:18 ` Felipe Contreras
2023-05-02 13:52 ` Ævar Arnfjörð Bjarmason
2023-05-02 15:28 ` Felipe Contreras
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3808fc35-6d3d-5a91-3eac-9428ff0e8045@gmail.com \
--to=phillip.wood123@gmail.com \
--cc=avarab@gmail.com \
--cc=calvinwan@google.com \
--cc=git@vger.kernel.org \
--cc=phillip.wood@dunelm.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).