Support for shared per-task setup and teardown in mochitest tests and xpcshell tests
58 views
Skip to first unread message
Beth Rennie
Apr 7, 2025, 7:24:02 PMApr 7
to [email protected], [email protected], Nimbus
Hi y'all,
Last week I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1957513 for investigating ways to improve mochitest and xpcshell-test tests for Firefox Desktop. I initially was proposing a decorator based approach (à la NormandyTestUtils [1]) but I've also implemented a mocha-ish to add_task wrapper which is entirely opt-in. I've attached patches for both patterns to the bug, as well as some additional patches which migrate a single test (test_FirefoxLabs.js) to each of these patterns.
I'm hoping this drives some discussion on these patterns (or other potential patterns). Feedback is much appreciated.
Thanks!
Beth
Nick Alexander
Apr 11, 2025, 8:03:13 PMApr 11
to Beth Rennie, [email protected], [email protected], Nimbus
Beth, others,
Hi y'all,Last week I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1957513 for investigating ways to improve mochitest and xpcshell-test tests for Firefox Desktop. I initially was proposing a decorator based approach (à la NormandyTestUtils [1]) but I've also implemented a mocha-ish to add_task wrapper which is entirely opt-in. I've attached patches for both patterns to the bug, as well as some additional patches which migrate a single test (test_FirefoxLabs.js) to each of these patterns.I'm hoping this drives some discussion on these patterns (or other potential patterns). Feedback is much appreciated.
Commenting here rather than on BZ in the hope that we spur more and broader discussion.
I have often lamented not having richer testing primitives and support doing almost anything in the area to fill the gap. I am not familiar with Mocha but I am familiar with JUnit, which groups tests by classes in the namespace, and then exposes `{Before,After}Each` and `{Before,After}All` decorators (for per-test and per-suite setup and teardown, respectively). Right now we only have `BeforeAll` in the form of `add_setup`. I am not a fan of the `add_task` decorators that we currently have; they make it really quite awkward to add conditions to tests, so I would be happier if we exposed `add_cleanup`(for `AfterAll`) and perhaps `add_{before,after}_each` for the per-test variants. It's annoying to have to split tests across files in order to have fine-grained before/after configurations, but I'd prefer that to having to decorate every test and have many more awkward definitions than we already do.
Thanks for making this better!
Nick
--
You received this message because you are subscribed to the Google Groups "[email protected]" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion visit https://groups.google.com/a/mozilla.org/d/msgid/dev-platform/CA%2BSxwbfJv-1sUmTJhF8Cm9Bfo%3DjRYtnL7kKg30X13pM0W24XEA%40mail.gmail.com.
Mark Banner
Apr 24, 2025, 4:08:33 PMApr 24
to [email protected], [email protected], Nimbus
On 11/04/2025 18:02, 'Nick Alexander'
via [email protected] wrote:
Hi y'all,
Last week I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1957513 for investigating ways to improve mochitest and xpcshell-test tests for Firefox Desktop. I initially was proposing a decorator based approach (à la NormandyTestUtils [1]) but I've also implemented a mocha-ish to add_task wrapper which is entirely opt-in. I've attached patches for both patterns to the bug, as well as some additional patches which migrate a single test (test_FirefoxLabs.js) to each of these patterns.
I'm hoping this drives some discussion on these patterns (or other potential patterns). Feedback is much appreciated.
Commenting here rather than on BZ in the hope that we spur more and broader discussion.
I have often lamented not having richer testing primitives and support doing almost anything in the area to fill the gap. I am not familiar with Mocha but I am familiar with JUnit, which groups tests by classes in the namespace, and then exposes `{Before,After}Each` and `{Before,After}All` decorators (for per-test and per-suite setup and teardown, respectively). Right now we only have `BeforeAll` in the form of `add_setup`. I am not a fan of the `add_task` decorators that we currently have; they make it really quite awkward to add conditions to tests, so I would be happier if we exposed `add_cleanup`(for `AfterAll`) and perhaps `add_{before,after}_each` for the per-test variants. It's annoying to have to split tests across files in order to have fine-grained before/after configurations, but I'd prefer that to having to decorate every test and have many more awkward definitions than we already do.
I'm not quite sure I understand the issue you're describing here.
The main issue that I believe that this is intended to solve, is
the case where we have multiple small tests which require set-up
and tear-down for each individual test. Splitting those across
files could mean many small files that are testing a single case
only. Having before/afterEach would allow for those related tests
to be in one file. I think you are saying that you would prefer
those to be split across many files? If so, I think that there's
probably a balance to be made as if it is many short tasks, then I
think that would be harder to manage across multiple files.
The reason that I suggested a Mocha (Jest is also an option) like interface for enabling this, is that this will help move our test infrastructure towards standard JavaScript testing mechanisms, rather than our own custom functions. Eventually we could potentially replace our current harnesses with an appropriate library.
I think heading towards this style of decorators should help with
onboarding new developers, as we would be providing a test
infrastructure that's more familiar. There may also be tooling
benefits we can get from it as well, e.g. both Mocha & Jest
have associated ESLint plugins.
Mark
Nick Alexander
Apr 25, 2025, 1:01:02 AMApr 25
to Mark Banner, [email protected], [email protected], Nimbus
On 11/04/2025 18:02, 'Nick Alexander' via [email protected] wrote:
Hi y'all,
Last week I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1957513 for investigating ways to improve mochitest and xpcshell-test tests for Firefox Desktop. I initially was proposing a decorator based approach (à la NormandyTestUtils [1]) but I've also implemented a mocha-ish to add_task wrapper which is entirely opt-in. I've attached patches for both patterns to the bug, as well as some additional patches which migrate a single test (test_FirefoxLabs.js) to each of these patterns.
I'm hoping this drives some discussion on these patterns (or other potential patterns). Feedback is much appreciated.
Commenting here rather than on BZ in the hope that we spur more and broader discussion.
I have often lamented not having richer testing primitives and support doing almost anything in the area to fill the gap. I am not familiar with Mocha but I am familiar with JUnit, which groups tests by classes in the namespace, and then exposes `{Before,After}Each` and `{Before,After}All` decorators (for per-test and per-suite setup and teardown, respectively). Right now we only have `BeforeAll` in the form of `add_setup`. I am not a fan of the `add_task` decorators that we currently have; they make it really quite awkward to add conditions to tests, so I would be happier if we exposed `add_cleanup`(for `AfterAll`) and perhaps `add_{before,after}_each` for the per-test variants. It's annoying to have to split tests across files in order to have fine-grained before/after configurations, but I'd prefer that to having to decorate every test and have many more awkward definitions than we already do.I'm not quite sure I understand the issue you're describing here. The main issue that I believe that this is intended to solve, is the case where we have multiple small tests which require set-up and tear-down for each individual test. Splitting those across files could mean many small files that are testing a single case only. Having before/afterEach would allow for those related tests to be in one file. I think you are saying that you would prefer those to be split across many files? If so, I think that there's probably a balance to be made as if it is many short tasks, then I think that would be harder to manage across multiple files.
The issue I'm describing is frustration using the existing condition decorators, and not wanting to add more decorators when the existing ones frustrate me. My experience with the condition decorators is that it's not easy to compose them and leads to hard to read source files, i.e., adding conditions tends to indent function bodies in difficult to control ways. (Surely we could make it easier to compose conditions, but right now, we haven't.). If beforeEach and afterEach are shared between many tests, then having them in a separate file with function names in the namespace isn't a great burden. If they're not shared, then there's less value in harness support (but not no value).
If Mocha and/or Jest have set a pattern in this space, then I am all for following suit, for all of the reasons that you mention. I should be clear that I will support and use almost any plausible expression of this idea.
Glad to have spurred a little more discussion here :)
Nick
Dan Mosedale
Apr 25, 2025, 8:00:55 PMApr 25
to Nick Alexander, Mark Banner, [email protected], [email protected], Nimbus
I'm generally a fan of moving in the direction of using well-established, documented frameworks in order to reap network-effect benefits (good docs, tutorials, surprising behaviors/bugs are easy to find info about, etc). That said, one thing that people sometimes do when using mocha/jest-style nesting is to over-rotate on code-sharing at the cost of code readability (https://kentcdodds.com/blog/avoid-nesting-when-youre-testing is worth a read), and I've certainly fallen into that trap. If we do go in a mocha/jest-like direction, I'd suggest we document a few best practices here (or maybe even just link to the blog post above) in our docs.
Dan
--
You received this message because you are subscribed to the Google Groups "[email protected]" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
Reply all
Reply to author
Forward
0 new messages