feat(python): Overhaul parametric test implementations and update Hypothesis to latest version #16062
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #16062 +/- ##
=======================================
Coverage 80.99% 80.99%
=======================================
Files 1387 1392 +5
Lines 178832 178884 +52
Branches 2877 2893 +16
=======================================
+ Hits 144839 144887 +48
- Misses 33500 33501 +1
- Partials 493 496 +3 ☔ View full report in Codecov by Sentry. |
|
Nice; happy to see these methods getting some love 😎👍 |
4025ad3 to
cef503e
Compare
|
This got a little out of hand 😅 but I think we have something clean now that works well. @alexander-beedie would you mind taking a look at this if you have the time? |
If you can wait until Sunday then I can over it thoroughly, with pleasure ;)) |
|
@alexander-beedie I'm going to go ahead with this one, there's a few more things I want to build on top of this. If you have any comments I'd be happy to address them in a follow-up! |
@stinodego: Been working my way through it slowly, heh; looks good to me so far, and I'm really happy to see the foundations being built out and incorporated in more places! Will be poking at some of the updated API design to give it a proper test shortly ✌️ |
The newest Hypothesis version is stricter about randomness - using Python's built-in
randommodule in strategies is not allowed (for good reason), and there is additional detection for tests that do not function properly.This is great, but it prohibited us from upgrading. A thorough revision of the code was needed to adhere to the new requirements. Some minor user-facing changes were necessary, but since this concerns code that is only used in test suites, I think we can go ahead with these changes without a major version increase. More details below.
Changes
Randomness is now restricted to hypothesis-controlled randomness within strategies. This affects the following functions:
columnno longer selects adtypeupon creation. This now happens within thedataframesstrategy. Use ofcolumnpassed todataframesis unaltered, but users who used thecolumnoutside of this intended usage will notice this as a breaking change.columnshad built-in functionality to randomly determine a number of columns. This is no longer possible. The function has been deprecated, with the recommendation to build your own columns usingcolumnin conjunction with a list comprehension. It continues to function for now by leveraging.example().create_list_strategyneeds to determine the exact inner data type to select an appropriate strategy. This is no longer possible in the same way. It has been deprecated. Users can use thelistsstrategy to do something similar, but they should supply a fully instantiated data type or defaults will be used - this avoids the randomness. The function continues to work for now by leveraging.example().New strategy
dtypeshas been added which generates a random Polars data type.Various improvements to the data generation strategies
Update
hypothesisto the latest version.Change the default row/column limit from 10/8 to 5/5. This should be enough for parametrized testing in general. It can be overwritten by the user if they require more rows/cols.
Various refactorings / cleanups to make everything work nicely.