Page MenuHomePhabricator

Migration of browsertests* Jenkins jobs to selenium* jobs
Closed, ResolvedPublic

Description

The table below is only for my reference, the table that I need populated is at T134492: Ownership of Selenium tests.

Repositorybrowsertests*selenium*contact from browsertests.yamlcontact from T94150contact from gerrit
mediawiki/extensions/CentralAuth+BryanDavis Jforrester Legoktm
mediawiki/extensions/CentralNotice+[email protected]AndyRussGAwight Ejegg
mediawiki/extensions/CirrusSearch+dcausseChad DCausse EBernhardson
mediawiki/core+Legoktm
mediawiki/extensions/Echo+[email protected] [email protected] [email protected]Mattflaschen JdlrobsonCatrope Jforrester Sbisson
mediawiki/extensions/Flow+[email protected] [email protected] [email protected]Mattflaschen SBisson DannyHEBernhardson Jforrester Mattflaschen Sbisson
mediawiki/extensions/Gather+Jdlrobson Legoktm
mediawiki/extensions/GettingStarted+[email protected] [email protected] [email protected]Mattflaschen phuedxJforrester Mattflaschen
mediawiki/extensions/Math+Jforrester Physikerwelt
mediawiki/extensions/MobileFrontend+JdlrobsonFlorianschmidtwelzow JdlrobsonJforrester
mediawiki/extensions/MultimediaViewer+[email protected]Gilles Jdlrobson TgrJdlrobson Jforrester
mediawiki/extensions/PageTriage+[email protected] [email protected] [email protected]Jforrester
mediawiki/extensions/PdfHandler+Jforrester
mediawiki/extensions/QuickSurveys+Jdlrobson
mediawiki/extensions/RelatedArticles+Bmansurov Jdlrobson
mediawiki/extensions/VisualEditor+[email protected]Amire80 Jforrester Ryasmeen
mediawiki/extensions/WikiLove+[email protected] [email protected] [email protected]Jforrester
wmde/WikidataBrowserTests+[email protected] [email protected] [email protected] [email protected]adrianheine hoo JanZerebecki WMDE-Fisch

selenium* jobs are a new way of running Ruby based Selenium tests. The jobs are triggered daily. Configuration evolved in being too complicated and not flexible enough.

Missing for Wikidata:
running tests against test.wikidata.org by adding that to ci.yml
possibly a new RAKE to define tags that are included; maybe this can be avoided by using exclude https://phabricator.wikimedia.org/diffusion/EMMV/browse/master/tests/browser/ci.yml ; alternative just run all tests in one job

Details

SubjectRepoBranchLines +/-
integration/configmaster+0 -338
mediawiki/extensions/MobileFrontendREL1_27+40 -34
mediawiki/extensions/MobileFrontendmaster+40 -34
wikidata/browsertestsmaster+1 -0
integration/configmaster+8 -0
mediawiki/extensions/Wikibasemaster+20 -14
integration/configmaster+8 -0
wikidata/browsertestsmaster+43 -31
integration/configmaster+18 -90
mediawiki/extensions/Flowmaster+27 -65
mediawiki/extensions/MultimediaViewermaster+4 -62
mediawiki/extensions/Flowmaster+6 -4
integration/configmaster+290 -272
mediawiki/extensions/Flowmaster+4 -0
mediawiki/extensions/Mathmaster+10 -13
mediawiki/extensions/Echomaster+1 -0
mediawiki/extensions/Gathermaster+31 -19
mediawiki/extensions/Echomaster+9 -7
mediawiki/extensions/PdfHandlerwmf/1.27.0-wmf.21+26 -13
mediawiki/extensions/MultimediaViewerwmf/1.27.0-wmf.21+113 -22
mediawiki/extensions/VisualEditormaster+38 -20
mediawiki/extensions/QuickSurveysmaster+28 -18
mediawiki/extensions/WikiLovemaster+22 -12
mediawiki/extensions/RelatedArticlesmaster+36 -22
mediawiki/extensions/PdfHandlermaster+26 -13
mediawiki/extensions/MultimediaViewermaster+112 -17
mediawiki/extensions/Flowmaster+26 -12
mediawiki/extensions/PageTriagemaster+26 -12
mediawiki/extensions/Mathmaster+24 -12
mediawiki/extensions/GettingStartedmaster+25 -12
mediawiki/extensions/Echomaster+26 -12
mediawiki/extensions/CirrusSearchmaster+0 -3
mediawiki/coremaster+29 -13
mediawiki/extensions/CirrusSearchmaster+25 -12
mediawiki/extensions/CentralNoticemaster+0 -3
mediawiki/extensions/CentralAuthmaster+0 -3
mediawiki/extensions/CentralNoticemaster+28 -12
mediawiki/extensions/CentralAuthmaster+29 -13
mediawiki/seleniummaster+5 -1
mediawiki/seleniummaster+17 -3
mediawiki/seleniummaster+1 -0
mediawiki/seleniummaster+100 -5
mediawiki/seleniummaster+109 -15
mediawiki/extensions/MultimediaViewermaster+3 -77
integration/configmaster+7 -35
mediawiki/extensions/Mathmaster+15 -0
Show related patches Customize query in gerrit

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 281960 merged by jenkins-bot:
[BrowserTest] Migration of browsertests* Jenkins jobs to selenium* jobs

https://gerrit.wikimedia.org/r/281960

Change 285372 had a related patch set uploaded (by Zfilipin):
Migration of browsertests* Jenkins jobs to selenium* jobs

https://gerrit.wikimedia.org/r/285372

Change 285370 merged by jenkins-bot:
Migration of browsertests* Jenkins jobs to selenium* jobs

https://gerrit.wikimedia.org/r/285370

Change 285372 merged by jenkins-bot:
Migration of browsertests* Jenkins jobs to selenium* jobs

https://gerrit.wikimedia.org/r/285372

Change 285397 had a related patch set uploaded (by Zfilipin):
Do not run broken Selenium test in CI

https://gerrit.wikimedia.org/r/285397

Change 285397 abandoned by Zfilipin:
WIP Do not run broken Selenium test in CI

Reason:
The problem was in page-object library, see https://gerrit.wikimedia.org/r/#/c/285410/

https://gerrit.wikimedia.org/r/285397

Change 285410 had a related patch set uploaded (by Zfilipin):
page-object 1.1.1 causes a scenario to fail

https://gerrit.wikimedia.org/r/285410

Change 279359 merged by jenkins-bot:
Migration of browsertests* Jenkins jobs to selenium* jobs

https://gerrit.wikimedia.org/r/279359

Change 285410 abandoned by Zfilipin:
page-object 1.1.1 causes a scenario to fail

Reason:
Moar testing showed that this patch does not fix broken scenario reliably. :| Back to https://gerrit.wikimedia.org/r/#/c/285397/

https://gerrit.wikimedia.org/r/285410

Change 285397 restored by Zfilipin:
WIP Do not run broken Selenium test in CI

Reason:
Moar testing showed that https://gerrit.wikimedia.org/r/#/c/285410/ does not fix broken scenario reliably. :|

https://gerrit.wikimedia.org/r/285397

Change 285397 merged by jenkins-bot:
Do not run broken Selenium test in CI

https://gerrit.wikimedia.org/r/285397

Change 285431 had a related patch set uploaded (by Zfilipin):
Fixed broken Selenium test

https://gerrit.wikimedia.org/r/285431

Change 285431 merged by jenkins-bot:
Fixed broken Selenium test

https://gerrit.wikimedia.org/r/285431

Change 285657 had a related patch set uploaded (by Zfilipin):
WIP Cleanup Cucumber tags

https://gerrit.wikimedia.org/r/285657

Change 285657 abandoned by Zfilipin:
WIP Cleanup Cucumber tags

Reason:
just a test

https://gerrit.wikimedia.org/r/285657

Change 285658 had a related patch set uploaded (by Zfilipin):
WIP Cleanup Cucumber tags

https://gerrit.wikimedia.org/r/285658

Change 285925 had a related patch set uploaded (by Zfilipin):
Do not run failing Cucumber scenarios daily

https://gerrit.wikimedia.org/r/285925

Change 274136 merged by jenkins-bot:
Migration of browsertests* Jenkins jobs to selenium* jobs

https://gerrit.wikimedia.org/r/274136

Change 285925 merged by jenkins-bot:
Do not run failing Cucumber scenarios daily

https://gerrit.wikimedia.org/r/285925

Change 286674 had a related patch set uploaded (by Zfilipin):
Do not run "Download menu" scenario in Internet Explorer

https://gerrit.wikimedia.org/r/286674

Change 279364 merged by jenkins-bot:
Migration of browsertests* Jenkins jobs to selenium* jobs

https://gerrit.wikimedia.org/r/279364

Change 286674 merged by jenkins-bot:
Do not run unstable tests daily

https://gerrit.wikimedia.org/r/286674

Change 287195 had a related patch set uploaded (by Zfilipin):
MobileFrontend and MultimediaViewer are moved from browsertests* to selenium* jobs.

https://gerrit.wikimedia.org/r/287195

Change 285658 merged by jenkins-bot:
Cleanup Cucumber tags

https://gerrit.wikimedia.org/r/285658

Change 287195 merged by jenkins-bot:
MobileFrontend and MultimediaViewer are moved from browsertests* to selenium* jobs.

https://gerrit.wikimedia.org/r/287195

Change 289396 had a related patch set uploaded (by Zfilipin):
WIP selenium-Wikidata Jenkins job

https://gerrit.wikimedia.org/r/289396

Change 289399 had a related patch set uploaded (by Zfilipin):
WIP Migration of browsertests* Jenkins jobs to selenium* jobs

https://gerrit.wikimedia.org/r/289399

Change 289878 had a related patch set uploaded (by JanZerebecki):
Migration of browsertests* Jenkins jobs to selenium* jobs

https://gerrit.wikimedia.org/r/289878

Change 289399 merged by jenkins-bot:
Migration of browsertests* Jenkins jobs to selenium* jobs

https://gerrit.wikimedia.org/r/289399

Change 289396 merged by jenkins-bot:
Created selenium-Wikidata Jenkins job

https://gerrit.wikimedia.org/r/289396

Change 289878 merged by Tobias Gritschacher:
Migration of browsertests* Jenkins jobs to selenium* jobs

https://gerrit.wikimedia.org/r/289878

Change 290904 had a related patch set uploaded (by JanZerebecki):
Add Wikibase selenium job

https://gerrit.wikimedia.org/r/290904

Change 290904 merged by jenkins-bot:
Add Wikibase selenium job

https://gerrit.wikimedia.org/r/290904

Change 291890 had a related patch set uploaded (by Zfilipin):
Fix selenium-Wikibase Jenkins job by fixing path to a file

https://gerrit.wikimedia.org/r/291890

Change 291907 had a related patch set uploaded (by Zfilipin):
WIP Fix selenium-Wikibase Jenkins job

https://gerrit.wikimedia.org/r/291907

Change 293106 had a related patch set uploaded (by JanZerebecki):
Also run on test to replace group0 Browsertest job

https://gerrit.wikimedia.org/r/293106

Change 293106 merged by jenkins-bot:
Also run on test to replace group0 Browsertest job

https://gerrit.wikimedia.org/r/293106

Change 296522 had a related patch set uploaded (by Gergő Tisza):
Migration of browsertests* Jenkins jobs to selenium* jobs

https://gerrit.wikimedia.org/r/296522

Change 296522 merged by Gergő Tisza:
Migration of browsertests* Jenkins jobs to selenium* jobs

https://gerrit.wikimedia.org/r/296522

@adrianheine @JanZerebecki @Addshore: since selenium-Wikibase and selenium-Wikidata jobs are created, can we delete browsertests-Wikidata* jobs?

I would like to close this task. If there is anything left to be done, let's create specific subtasks.

According to my information you pinged the wrong people. I'll instead refer you to @Tobi_WMDE_SW .

A quick look at existing browsertests-Wikidata* jobs (12 of them):

  • browsertests-Wikidata-SmokeTests-linux-firefox
    • broken for the last 30 days
    • Selenium::WebDriver::Error::WebDriverError
  • browsertests-Wikidata-SmokeTests-linux-firefox-sauce
    • broken for the last 4 days
    • Timeout::Error
  • browsertests-Wikidata-Wikibase-Beta-linux-chrome-sauce
    • green
  • browsertests-Wikidata-Wikibase-Beta-linux-firefox-sauce
    • green
  • browsertests-Wikidata-Wikibase-Group0-linux-chrome-sauce
    • green
  • browsertests-Wikidata-Wikibase-Group0-linux-firefox-sauce
    • green
  • browsertests-Wikidata-WikidataTests-Group0-linux-chrome-sauce
    • failed 29 of the last 30 runs
    • Timeout::Error
  • browsertests-Wikidata-WikidataTests-Group0-linux-firefox-sauce
    • broken for 30 days
    • Timeout::Error
  • browsertests-Wikidata-WikidataTests-Group0-SmokeTests-linux-firefox-sauce
    • broken for the last 4 days
    • Timeout::Error
  • browsertests-Wikidata-WikidataTests-linux-chrome-sauce
    • broken for 30 days
    • Timeout::Error
  • browsertests-Wikidata-WikidataTests-linux-firefox
    • broken for 30 days
    • Selenium::WebDriver::Error::WebDriverError
  • browsertests-Wikidata-WikidataTests-linux-firefox-sauce
    • broken for 30 days
    • Timeout::Error

A quick look at:

  • selenium-Wikibase
    • broken for 30 days
    • no such file to load -- features/support/pages
  • selenium-Wikidata
    • failed 28 times in the last 30 days
    • beta: Timeout::Error
    • test: MediawikiSelenium::ConfigurationError: unknown environment test

Can I suggest to Wikidata people (cc @Tobi_WMDE_SW) that we delete browsertests-Wikidata* jobs, fix selenium-Wikibase and selenium-Wikidata and then see if anything else should be done?

Can I delete browsertests-Wikidata* jobs?

Do you need help fixing selenium-Wikibase and selenium-Wikidata jobs?

@zeljkofilipin yes, please delete those tests. I can spend some time to work on fixing Wikidata tests next week.

Change 298448 had a related patch set uploaded (by Zfilipin):
Deleted browsertests-Wikidata* Jenkins jobs

https://gerrit.wikimedia.org/r/298448

Change 298448 merged by jenkins-bot:
Deleted browsertests-Wikidata* Jenkins jobs

https://gerrit.wikimedia.org/r/298448