Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup initial end-to-end tests, end-to-end testing infrastructure #42

Open
tmillr opened this issue Dec 26, 2022 · 0 comments
Open

Setup initial end-to-end tests, end-to-end testing infrastructure #42

tmillr opened this issue Dec 26, 2022 · 0 comments

Comments

@tmillr
Copy link
Owner

tmillr commented Dec 26, 2022

Why?

  • to make sure that action.yml is still valid on and into the future (e.g. its schema may change, the chosen/selected node version may become outdated/invalid, etc.)
  • to make sure the action does not break when the gh runners (or their interface with js actions) change
  • for end-to-end testing, if wanted or needed
  • provides a way to capture/view the action's actual output from an actual/realistic run (e.g. for docs: include screenshot and link to example of posted comments #1)

How?

  • create a scratch branch or scratch repo for pushing test commits to
    • if branch, the ci yaml itself must still be on the default branch
  • choose any events necessary, but also at least run on a schedule (e.g. daily) since it's possible for this action (or its manifest: action.yml) to break even if the pkg/repo itself doesn't actually change
  • observe workflow runs for failures, esp those due to an invalid action.yml. This can also be used as a way to verify that the action.yml is (still) valid before releasing/tagging new versions.
    • can use gh api to retrieve workflow run results
    • can view results manually in browser anywhere workflow run results are displayed (e.g. Actions tab, commit list/history, branch landing page, pr checks), or in user notifications (if configured)
    • maybe auto-open an issue or pr on failure for extra visibility

NOTE: remember that pushing commits from within gh actions/ci will not trigger further events/workflows unless pushed using a custom token/pat

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant