Page MenuHomePhabricator

Implement ClassNameTests in Scribunto to enhance central modules stability
Closed, DeclinedPublic

Description

Why and how Implement ClassNameTests in Scribunto?

  • To enhance central modules stability.
  • In a way which insure a stric protocol, easy to use, in the respect of ClassNameTests specifications.
  • Display the same tests cases for users in p.tests(frame) mode to permit at them to better understand functions, to verify if tests are well made and well checked, and to propose new tests for missing case types.

Proposed functions in Mediawiki and central modules, to adjust in cooperation, in advanced debug state:

* mw.ClassNameTests(mw_tests, nmaxi) -- To initialise, to limit all tests in time and to limit cases in coherence with nmaxi.
* mw.ClassNameCase(case, n), used inside mw.ClassNameTests() to run each test case of number 1 to nmaxi.
* The case object contains fields: count, OK="OK"..., n, name, type, args{}, expect{}, NameTest{}, func(), provide(), run()
* Functions used in central modules: tests_cases.ClassNameTests(), .init(), .expectdiffs(), .gettestsgroups(), .getcases(), .expectdiffs(), .normalcase(case), viewer.form9en()
  • In the tests_cases library, tests cases are in recursive groups of groups of cases.
  • Groups levels: main module, sub modules or libraries, functions and their tests cases.
  • An anti loop system protect against mistakes in groups names declarations.

Event Timeline

The only reference I can find to a "ClassNameTests specification" is https://www.mediawiki.org/wiki/Extension:Scribunto/Lua_reference_manual#Test_cases, which is about writing test cases in Lua to be run from MediaWiki's existing PHPUnit test framework. That exists because running Lua code from PHPUnit requires a lot of boilerplate to set things up.

When you're running in Lua already via a Scribunto module, you don't need all that. You just call the functions and compare the output with the expected output, much like how PHPUnit tests of PHP code work. There's certainly room for an infrastructure to make that more convenient, but basing it on Scribunto's infrastructure for running tests from PHPUnit is probably not a good plan.

I see enwiki already has Module:UnitTests and Module:ScribuntoUnit, either of those would be a better starting point. Personally I like the look of the latter, but that's probably because it resembles PHPUnit syntax that I'm already familiar with.

My motivation was to check all tests cases on each re-compilation, but in new conditions for each new use in new main modules, to enhanse the stability.
My implementation compare also tables, and any types, between case.result{ } and case.expect{ }.
Lua-coders cannot check hundreds of cases after each compilation. Only automatic repeats of tests are efficient.
Functions provide(), run()... are a good guide to structure tests.
Repeat all cases and report detected errors to lua-coders in p.tests(frame) mode seems a minimum to insure a good stability for central modules.
Then I continue in my strategy, with or whithout Mediawiki.