20/21 models were trained successfully in the 9th round of wikis.

The Armenian Wikipedia (hywiki) pipeline did not complete successfully and is being investigated in T327371.

Model evaluation has been completed and below are the backtesting results:

CCing @MGerlach, in case he would like to add comments on the backtesting evaluation.

The conclusion on the backtesting results is that most of the languages look fine besides:

• hywwiki has a low precision (0.63) and recall (0.16).
• igwiki (0.72), jbowiki (0.70), and both hawiki, inhwiki (0.68) have a precision that is slightly lower than the recommended one (0.75).

Talked to @MGerlach about these results and agreed: not to deploy hywwiki; but deploy igwiki, jbowiki, hawiki, and inhwiki since their precision is not too low and the recall is good.

@kostajh, we published datasets for all 19/21 models that passed the evaluation in this round.

In T308134#8553530, @kevinbazira wrote:

@kostajh, we published datasets for all 19/21 models that passed the evaluation in this round.

🎉 thanks!

I ran this script for adding the link-recommendation task type and and populating the excluded sections:

PHAB=T308134
for WIKI in hawiki hakwiki hawwiki hifwiki hrwiki hsbwiki htwiki hywwiki iawiki iewiki igwiki ikwiki ilowiki inhwiki iowiki iswiki iuwiki jamwiki jbowiki jvwiki; do
    ORIGIN=`mwscript getConfiguration.php $WIKI --settings 'wgCanonicalServer' --format json | jq --raw-output '.wgCanonicalServer'`
    mwscript extensions/GrowthExperiments/maintenance/changeWikiConfig.php $WIKI \
            --page MediaWiki:NewcomerTasks.json \
            --create-only \
            --json \
            --summary "Growth features configuration boilerplate ([[phab:$PHAB]])" \
            link-recommendation \
            '{ "type": "link-recommendation", "group": "easy" }'
    jq "select(.wiki==\"$WIKI\" and .probability > 0.25) | .section" wiki_sections.jsonl \
        | jq --slurp --compact-output "unique" \
        | mwscript extensions/GrowthExperiments/maintenance/changeWikiConfig.php $WIKI \
            --page MediaWiki:NewcomerTasks.json \
            --json \
            --summary "machine-generated configuration for excluding sections from link recommendations ([[phab:$PHAB]]), feel free to improve" \
            link-recommendation.excludedSections \
            "`cat`"
    echo "$ORIGIN/wiki/MediaWiki:NewcomerTasks.json"
    echo "$ORIGIN/w/index.php?title=MediaWiki:NewcomerTasks.json&diff=next"
    echo "Press <Enter> to continue"
    read # give time for manual verification
done

I checked the configuration and it seemed to be correctly updated in all wikis. The only mentions worth are ikwiki and ilo which didn't get any excluded section.

Change 892365 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[operations/mediawiki-config@master] GrowthExperiments: Enable link recommendation for 9th round wikis

https://gerrit.wikimedia.org/r/892365

Change 892363 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[operations/mediawiki-config@master] GrowthExperiments: Enable backend of link recommendation for 7,8,9th round wikis

https://gerrit.wikimedia.org/r/892363

Change 892365 abandoned by Sergio Gimeno:

[operations/mediawiki-config@master] GrowthExperiments: Enable link recommendation for 9th round wikis

Reason:

squashed in I81293b799ec5afe62a19ac2d79e0434047cf1be2

https://gerrit.wikimedia.org/r/892365

Change 892363 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: Enable backend of link recommendation for 7, 8, 9th round wikis

https://gerrit.wikimedia.org/r/892363

Mentioned in SAL (#wikimedia-operations) [2023-03-15T20:13:23Z] <samtar@deploy2002> Started scap: Backport for [[gerrit:899673|GrowthExperiments: enable frontend of link recommendation for 6th round wikis (T304550)]], [[gerrit:892363|GrowthExperiments: Enable backend of link recommendation for 7, 8, 9th round wikis (T304551 T308133 T308134)]]

Mentioned in SAL (#wikimedia-operations) [2023-03-15T20:14:55Z] <samtar@deploy2002> sgimeno and samtar: Backport for [[gerrit:899673|GrowthExperiments: enable frontend of link recommendation for 6th round wikis (T304550)]], [[gerrit:892363|GrowthExperiments: Enable backend of link recommendation for 7, 8, 9th round wikis (T304551 T308133 T308134)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-03-15T20:23:36Z] <samtar@deploy2002> Finished scap: Backport for [[gerrit:899673|GrowthExperiments: enable frontend of link recommendation for 6th round wikis (T304550)]], [[gerrit:892363|GrowthExperiments: Enable backend of link recommendation for 7, 8, 9th round wikis (T304551 T308133 T308134)]] (duration: 10m 12s)

@Sgs, based on the fact that a majority of models aren't working for the previous round (T308133), I sampled this one, and all samples returned "There were no results matching the query.". Could you check again?

Change 905950 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[operations/mediawiki-config@master] GrowthExperiments: enable add link frontend and backend

https://gerrit.wikimedia.org/r/905950

In T308134#8718876, @Trizek-WMF wrote:

@Sgs, based on the fact that a majority of models aren't working for the previous round (T308133), I sampled this one, and all samples returned "There were no results matching the query.". Could you check again?

I checked all wikis from this round, this is what I found:

hrwiki and hywiki were correctly enabled but they were missing in the wikis.txt file so not being populated. I've manually added them, cc @kevinbazira

From iewiki to jvwiki were not enabled in 892363, apologies for this, I must have missed them when squashing changes. They will be enabled today at 15h UTC+2.

We should check again all these wikis in a couple of days.

Change 905950 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: enable add link backend in wiki rounds (8,9th)

https://gerrit.wikimedia.org/r/905950

Mentioned in SAL (#wikimedia-operations) [2023-04-05T13:08:58Z] <lucaswerkmeister-wmde@deploy2002> Started scap: Backport for [[gerrit:905950|GrowthExperiments: enable add link backend in wiki rounds (8,9th) (T308133 T308134)]]

Mentioned in SAL (#wikimedia-operations) [2023-04-05T13:10:28Z] <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde and sgimeno: Backport for [[gerrit:905950|GrowthExperiments: enable add link backend in wiki rounds (8,9th) (T308133 T308134)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-04-05T13:16:58Z] <lucaswerkmeister-wmde@deploy2002> Finished scap: Backport for [[gerrit:905950|GrowthExperiments: enable add link backend in wiki rounds (8,9th) (T308133 T308134)]] (duration: 08m 00s)

I tested the models, and everything works fine, except:

• hr.wp returns "There were no results matching the query."
• ie.wp returns "There were no results matching the query."
• ig.wp returns "There were no results matching the query."
• ik.wp returns "There were no results matching the query."
• ilo.wp returns "There were no results matching the query."
• inh.wp returns "There were no results matching the query."
• io.wp returns "There were no results matching the query."
• is.wp returns "There were no results matching the query."
• iu.wp returns "There were no results matching the query."
• is.wp returns "There were no results matching the query."
• jam.wp returns "There were no results matching the query."
• jbo.wp returns "There were no results matching the query."
• jv.wp returns "There were no results matching the query."

In T308134#8759334, @Trizek-WMF wrote:

I tested the models, and everything works fine, except:

• hr.wp returns "There were no results matching the query."
• ie.wp returns "There were no results matching the query."
• ig.wp returns "There were no results matching the query."
• ik.wp returns "There were no results matching the query."
• ilo.wp returns "There were no results matching the query."
• inh.wp returns "There were no results matching the query."
• io.wp returns "There were no results matching the query."
• is.wp returns "There were no results matching the query."
• iu.wp returns "There were no results matching the query."
• is.wp returns "There were no results matching the query."
• jam.wp returns "There were no results matching the query."
• jbo.wp returns "There were no results matching the query."
• jv.wp returns "There were no results matching the query."

We're still investigating, I've added some more details in T308133#8774966 since it seems the same problem affecting gorwiki could also explain why there aren't results on this batch of wikis.

Any update?

In T308134#8790079, @Trizek-WMF wrote:

Any update?

All wikis have now results except jbowiki where link recommendations have been disabled on site. Also worth mentioning that ikwiki is only producing 5 results, so we might want to check that before enabling the frontend.

We can proceed informing communities for the 9th round (except jbo and ik) and add gorwiki from the 8th if you agree. I'll follow up with the issues in jbowiki and ikwiki.

Let's go then with gor + all round 9 (except jbo and ik). Can we deploy next Wednesday, May 17?

I'm adding it to Tech News.

In T308134#8841112, @Trizek-WMF wrote:

Let's go then with gor + all round 9 (except jbo and ik). Can we deploy next Wednesday, May 17?

I'm adding it to Tech News.

Alright.

Change 920722 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[operations/mediawiki-config@master] GrowthExperiments: enable add link frontend in 9th round wikis

https://gerrit.wikimedia.org/r/920722

Change 920722 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: enable add link frontend in 9th round wikis

https://gerrit.wikimedia.org/r/920722

Mentioned in SAL (#wikimedia-operations) [2023-05-17T20:03:10Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:920722|GrowthExperiments: enable add link frontend in 9th round wikis (T308134)]]

Mentioned in SAL (#wikimedia-operations) [2023-05-17T20:04:46Z] <urbanecm@deploy1002> sgimeno and urbanecm: Backport for [[gerrit:920722|GrowthExperiments: enable add link frontend in 9th round wikis (T308134)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet

Change 920784 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[operations/mediawiki-config@master] GrowthExperiments: amend wrong wiki prefix for jbowiki

https://gerrit.wikimedia.org/r/920784

Mentioned in SAL (#wikimedia-operations) [2023-05-17T20:15:17Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:920722|GrowthExperiments: enable add link frontend in 9th round wikis (T308134)]] (duration: 12m 06s)

Change 920784 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: amend wrong wiki prefix for jbowiki

https://gerrit.wikimedia.org/r/920784

Mentioned in SAL (#wikimedia-operations) [2023-05-17T20:17:29Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:920784|GrowthExperiments: amend wrong wiki prefix for jbowiki (T308134)]], [[gerrit:920732|NewTopicOptOutActiveUsers: Skip bot users etc. (T317375)]], [[gerrit:920386|Enable zebra ab test in hewiki (T335972)]]

Mentioned in SAL (#wikimedia-operations) [2023-05-17T20:19:04Z] <urbanecm@deploy1002> urbanecm and matmarex and ksarabia and sgimeno: Backport for [[gerrit:920784|GrowthExperiments: amend wrong wiki prefix for jbowiki (T308134)]], [[gerrit:920732|NewTopicOptOutActiveUsers: Skip bot users etc. (T317375)]], [[gerrit:920386|Enable zebra ab test in hewiki (T335972)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.

Mentioned in SAL (#wikimedia-operations) [2023-05-17T20:29:06Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:920784|GrowthExperiments: amend wrong wiki prefix for jbowiki (T308134)]], [[gerrit:920732|NewTopicOptOutActiveUsers: Skip bot users etc. (T317375)]], [[gerrit:920386|Enable zebra ab test in hewiki (T335972)]] (duration: 11m 36s)

Checked Hawaiian Wikipedia haw, Icelandic Wikipedia is, Inuktitut Wikipedia iu, and Javanese Wikipedia jv - all works as expected.

In T308134#8759334, @Trizek-WMF wrote:

I tested the models, and everything works fine, except:

• hr.wp returns "There were no results matching the query."
• ie.wp returns "There were no results matching the query."
• ig.wp returns "There were no results matching the query."
• ik.wp returns "There were no results matching the query."
• ilo.wp returns "There were no results matching the query."
• inh.wp returns "There were no results matching the query."
• io.wp returns "There were no results matching the query."
• is.wp returns "There were no results matching the query."
• iu.wp returns "There were no results matching the query."
• is.wp returns "There were no results matching the query."
• jam.wp returns "There were no results matching the query."
• jbo.wp returns "There were no results matching the query."
• jv.wp returns "There were no results matching the query."

Checked those - only jbo.wp returns "There were no results matching the query." The Homepage is enabled on jbo.wp (there are options to enable it in Preferences) - no available Suggested edit tasks are displayed.

ikwiki - ikwiki Special:NewcomerTasksInfo does display 5 link-recommendation tasks available, but they are not present in SE module:

@Trizek-WMF , @Sgs - no follow-ups for jbo.wp and ikwiki ?