Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tools/freeze needs tests in the test suite. #89792

Closed
ericsnowcurrently opened this issue Oct 27, 2021 · 3 comments
Closed

Tools/freeze needs tests in the test suite. #89792

ericsnowcurrently opened this issue Oct 27, 2021 · 3 comments
Assignees
@ericsnowcurrently
Labels
3.11 only security fixes

Comments

@ericsnowcurrently
Copy link
Member

ericsnowcurrently commented Oct 27, 2021 •
edited by bedevere-bot
Loading

BPO 45629
Nosy @malemburg, @gpshead, @ericsnowcurrently
PRs
  • bpo-45629: Add a test for the "freeze" tool. #29222
  • bpo-45629: Improve test.support.skip_if_buildbot #31094
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/ericsnowcurrently'
closed_at = <Date 2021-10-28.16:15:24.348>
created_at = <Date 2021-10-27.15:42:12.541>
labels = ['3.11']
title = 'Tools/freeze needs tests in the test suite.'
updated_at = <Date 2022-02-03.05:36:25.402>
user = 'https://github.com/ericsnowcurrently'

    bugs.python.org fields:

    activity = <Date 2022-02-03.05:36:25.402>
actor = 'gregory.p.smith'
assignee = 'eric.snow'
closed = True
closed_date = <Date 2021-10-28.16:15:24.348>
closer = 'eric.snow'
components = ['Demos and Tools']
creation = <Date 2021-10-27.15:42:12.541>
creator = 'eric.snow'
dependencies = []
files = []
hgrepos = []
issue_num = 45629
keywords = ['patch']
message_count = 3.0
messages = ['405105', '405197', '412421']
nosy_count = 3.0
nosy_names = ['lemburg', 'gregory.p.smith', 'eric.snow']
pr_nums = ['29222', '31094']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue45629'
versions = ['Python 3.11']

    Linked PRs
    @ericsnowcurrently
    Copy link
    Member Author

    I found that Tools/freeze doesn't get tested. It has a "test" directory but it's more of a rudimentary sanity check that must be run manually. Since the tool isn't tested through our test suite, there's a high likelihood it can get broken without anyone realizing it. The risk has increased with recent changes to frozen modules. So such tests would be valuable.

    The tool requires that Python be installed. In the context of our test suite, this means configuring with a temp dir prefix and then re-building. We must do it in a separate directory to avoid affecting other tests.

    I'm planning on adding a test that does the following:

    1. create a temp dir (TMPDIR)
    2. copy the repo to TMPDIR/cpython
    3. run configure there with --prefix=TMPDIR/python-installation
    4. run make -j
    5. run make install -j
    6. write a simple script to TMPDIR/app.py
    7. run the "freeze" tool on it, with minimal options
    8. run the frozen app and verify it worked

    Note that there needs to be enough disk space for all that (which not all buildbots have). Also, the test will be relatively slow due to having to re-build.

    There aren't any other tests that re-build Python or the like, so I'm a little hesitant to set precedent. Perhaps that's why there weren't any tests already. However, I don't see any significant reasons to not add the test (as long as it doesn't lead to spurious failures).

    @ericsnowcurrently ericsnowcurrently added the 3.11 only security fixes label Oct 27, 2021
    @ericsnowcurrently ericsnowcurrently self-assigned this Oct 27, 2021
    @ericsnowcurrently ericsnowcurrently added the 3.11 only security fixes label Oct 27, 2021
    @ericsnowcurrently ericsnowcurrently self-assigned this Oct 27, 2021
    @ericsnowcurrently
    Copy link
    Member Author

    New changeset 13d9205 by Eric Snow in branch 'main':
    bpo-45629: Add a test for the "freeze" tool. (gh-29222)
    13d9205

    @gpshead
    Copy link
    Member

    gpshead commented Feb 3, 2022

    New changeset 8726067 by Gregory P. Smith in branch 'main':
    bpo-45629: Improve test.support.skip_if_buildbot (GH-31094)
    8726067

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @bedevere-bot bedevere-bot mentioned this issue Feb 12, 2023
    Merged
    gpshead added a commit that referenced this issue Feb 12, 2023
    @gpshead
    gh-89792: Prevent test_tools from copying 1000M of "source" in freeze…
    1d19423 
    … test (#101837)

Prevent test_tools from copying 1000M of "source"
    
It doesn't need a git repo, just the checkout.  We skip .git metadata, Doc/build, Doc/venv, and `__pycache__` subdirs, that developers often have in their clients to reduce the size of the source tree copy ten-fold.

This should significantly reduce IO and presumably time on buildbots during this long test.
    @bedevere-bot bedevere-bot mentioned this issue Feb 12, 2023
    Merged
    miss-islington pushed a commit to miss-islington/cpython that referenced this issue Feb 12, 2023
    @gpshead @miss-islington
    pythongh-89792: Prevent test_tools from copying 1000M of "source" in …
    7362387 
    …freeze test (pythonGH-101837)

Prevent test_tools from copying 1000M of "source"

It doesn't need a git repo, just the checkout.  We skip .git metadata, Doc/build, Doc/venv, and `__pycache__` subdirs, that developers often have in their clients to reduce the size of the source tree copy ten-fold.

This should significantly reduce IO and presumably time on buildbots during this long test.
(cherry picked from commit 1d19423)

Co-authored-by: Gregory P. Smith <[email protected]>
    miss-islington added a commit that referenced this issue Feb 12, 2023
    @miss-islington @gpshead
    gh-89792: Prevent test_tools from copying 1000M of "source" in freeze…
    d17cc3d 
    … test (GH-101837)

Prevent test_tools from copying 1000M of "source"

It doesn't need a git repo, just the checkout.  We skip .git metadata, Doc/build, Doc/venv, and `__pycache__` subdirs, that developers often have in their clients to reduce the size of the source tree copy ten-fold.

This should significantly reduce IO and presumably time on buildbots during this long test.
(cherry picked from commit 1d19423)

Co-authored-by: Gregory P. Smith <[email protected]>
    gpshead added a commit to gpshead/cpython that referenced this issue Feb 12, 2023
    @gpshead
    pythongh-89792: unhardcode freeze test build parallelism.
    81ce27c 
    base it on the number of cpus, don't use more than max(2, 2/3).
    @bedevere-bot bedevere-bot mentioned this issue Feb 12, 2023
    Merged
    gpshead added a commit that referenced this issue Feb 12, 2023
    @gpshead
    gh-89792: Limit test_tools freeze test build parallelism based on the…
    dfc2e06 
    … number of cores (#101841)

unhardcode freeze test build parallelism. base it on the number of cpus, don't use more than max(2, os.cpu_count()/3).
    @bedevere-bot bedevere-bot mentioned this issue Feb 12, 2023
    Merged
    miss-islington pushed a commit to miss-islington/cpython that referenced this issue Feb 12, 2023
    @gpshead @miss-islington
    pythongh-89792: Limit test_tools freeze test build parallelism based …
    08048dd 
    …on the number of cores (pythonGH-101841)

unhardcode freeze test build parallelism. base it on the number of cpus, don't use more than max(2, os.cpu_count()/3).
(cherry picked from commit dfc2e06)

Co-authored-by: Gregory P. Smith <[email protected]>
    miss-islington added a commit that referenced this issue Feb 12, 2023
    @miss-islington @gpshead
    gh-89792: Limit test_tools freeze test build parallelism based on the…
    cec99ed 
    … number of cores (GH-101841)

unhardcode freeze test build parallelism. base it on the number of cpus, don't use more than max(2, os.cpu_count()/3).
(cherry picked from commit dfc2e06)

Co-authored-by: Gregory P. Smith <[email protected]>
    carljm added a commit to carljm/cpython that referenced this issue Feb 13, 2023
    @carljm
    Merge branch 'main' into inlinecomp2
    ae0bd02 
    * main:
  pythongh-101810: Remove duplicated st_ino calculation (pythonGH-101811)
  pythongh-92547: Purge sqlite3_enable_shared_cache() detection from configure (python#101873)
  pythonGH-100987: Refactor `_PyInterpreterFrame` a bit, to assist generator improvement. (pythonGH-100988)
  pythonGH-87849: Simplify stack effect of SEND and specialize it for generators and coroutines. (pythonGH-101788)
  Correct trivial grammar in reset_mock docs (python#101861)
  pythongh-101845: pyspecific: Fix i18n for availability directive (pythonGH-101846)
  pythongh-89792: Limit test_tools freeze test build parallelism based on the number of cores (python#101841)
  pythongh-85984: Utilize new "winsize" functions from termios in pty tests. (python#101831)
  pythongh-89792: Prevent test_tools from copying 1000M of "source" in freeze test (python#101837)
  Fix typo in test_fstring.py (python#101823)
  pythonGH-101797: allocate `PyExpat_CAPI` capsule on heap (python#101798)
  pythongh-101390: Fix docs for `imporlib.util.LazyLoader.factory` to properly call it a class method (pythonGH-101391)
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @ericsnowcurrently ericsnowcurrently

    Labels
    3.11 only security fixes
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @gpshead @ericsnowcurrently