Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unittest: if FunctionTestCase is imported, the loader loads "tests" from it #84867

Closed
abetkin mannequin opened this issue May 19, 2020 · 6 comments
Closed

unittest: if FunctionTestCase is imported, the loader loads "tests" from it #84867

abetkin mannequin opened this issue May 19, 2020 · 6 comments
Labels
3.8 (EOL) end of life 3.9 only security fixes 3.10 only security fixes tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error

Comments

@abetkin
Copy link
Mannequin

abetkin mannequin commented May 19, 2020

BPO 40690
Nosy @cjerdonek, @remilapeyre, @abetkin
PRs
  • bpo-40690: Check that testcase.__module__ != 'unittest.case' #20237
  • Files
  • test_functiontest.py
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2020-05-19.20:17:05.761>
    labels = ['3.8', 'type-bug', 'tests', '3.9', '3.10']
    title = 'unittest: if FunctionTestCase is imported, the loader loads "tests" from it'
    updated_at = <Date 2020-05-27.21:04:36.962>
    user = 'https://github.com/abetkin'

    bugs.python.org fields:

    activity = <Date 2020-05-27.21:04:36.962>
    actor = 'remi.lapeyre'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Tests']
    creation = <Date 2020-05-19.20:17:05.761>
    creator = 'pwtail'
    dependencies = []
    files = ['49196']
    hgrepos = []
    issue_num = 40690
    keywords = ['patch']
    message_count = 6.0
    messages = ['369386', '369524', '369547', '369571', '369575', '370135']
    nosy_count = 3.0
    nosy_names = ['chris.jerdonek', 'remi.lapeyre', 'pwtail']
    pr_nums = ['20237']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue40690'
    versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

    Linked PRs

    @abetkin
    Copy link
    Mannequin Author

    abetkin mannequin commented May 19, 2020

    In [2]: from unittest import FunctionTestCase

    In [3]: loader.loadTestsFromTestCase(FunctionTestCase)
    Out[3]: <unittest.suite.TestSuite tests=[<unittest.case.FunctionTestCase tec=runTest>]>

    In [4]: test = _._tests[0]; test
    Out[4]: <unittest.case.FunctionTestCase tec=runTest>

    In [5]: test._testFunc
    Out[5]: 'runTest'

    In [6]: test._testFunc.__name__
    *** AttributeError: 'str' object has no attribute '__name__'

    @abetkin abetkin mannequin added 3.8 (EOL) end of life 3.9 only security fixes tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error labels May 19, 2020
    @remilapeyre
    Copy link
    Mannequin

    remilapeyre mannequin commented May 21, 2020

    Hi Vitalii, can you give more context about why you are writing this code, what you expect it to do and why is there a bug?

    @abetkin
    Copy link
    Mannequin Author

    abetkin mannequin commented May 21, 2020

    If you make an import in a module with your tests

    From unittest import FunctionTestCase

    then you will have 1 extra test in that module (the imported one), moreover, that test will be broken.

    @cjerdonek
    Copy link
    Member

    then you will have 1 extra test in that module (the imported one), moreover, that test will be broken.

    If this is true, then how is anyone able to be using FunctionTestCase in their tests today? Is the feature broken?

    @abetkin
    Copy link
    Mannequin Author

    abetkin mannequin commented May 22, 2020

    I don't think anyone is using FunctionTestCase

    @remilapeyre
    Copy link
    Mannequin

    remilapeyre mannequin commented May 27, 2020

    I checked and FunctionTestCase seems to completely break the loader. The tests for FunctionTestCase in the standard library instantiate the class from inside the method of a TestCase so the loader never see them but even the simple test file I attached completely breaks:

    ✗ python3 -m unittest
    E
    ======================================================================

    Traceback (most recent call last):
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 193, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 86, in _run_code
        exec(code, run_globals)
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/__main__.py", line 18, in <module>
        main(module=None)
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/main.py", line 101, in __init__
        self.runTests()
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/main.py", line 271, in runTests
        self.result = testRunner.run(self.test)
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/runner.py", line 183, in run
        result.printErrors()
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/runner.py", line 109, in printErrors
        self.printErrorList('ERROR', self.errors)
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/runner.py", line 115, in printErrorList
        self.stream.writeln("%s: %s" % (flavour,self.getDescription(test)))
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/runner.py", line 47, in getDescription
        return '\n'.join((str(test), doc_first_line))
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/case.py", line 1472, in __str__
        self._testFunc.__name__)
    AttributeError: 'str' object has no attribute '__name__'

    I look at plenty of usages of FunctionTestCase on Github and all of them seemed to be false positive, they were copies of the unittest/test/test_functiontestcase.py file

    The patch in the attached PR is not correct thought, it only fixes one of the loader and all of them suffer from the same issue.

    @remilapeyre remilapeyre mannequin added 3.10 only security fixes labels May 27, 2020
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    sobolevn added a commit to sobolevn/cpython that referenced this issue Dec 24, 2022
    miss-islington pushed a commit to miss-islington/cpython that referenced this issue Sep 12, 2023
    miss-islington pushed a commit to miss-islington/cpython that referenced this issue Sep 12, 2023
    serhiy-storchaka pushed a commit that referenced this issue Sep 12, 2023
    Oct 2, 2023
    …GH-100497) (#109327)
    
    gh-84867: Do not load tests from TestCase and FunctionTestCase (GH-100497)
    (cherry picked from commit 66d1d7e)
    
    Co-authored-by: Nikita Sobolev <[email protected]>
    @hugovk hugovk closed this as completed Jan 11, 2024
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 (EOL) end of life 3.9 only security fixes 3.10 only security fixes tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error
    Projects
    Status: Done
    Development

    No branches or pull requests

    2 participants