@Trizek-WMF is the list in the description finalized? If not, please let us know when it is, so we can begin training models, generating the datasets, etc.

In T290011#7347717, @kostajh wrote:

@Trizek-WMF is the list in the description finalized? If not, please let us know when it is, so we can begin training models, generating the datasets, etc.

@kostajh, you can start the work.

Regarding community announcement, I'm off on week 38. We would announce it to the communities starting on Monday September 27.

In T290011#7348334, @Trizek-WMF wrote:

In T290011#7347717, @kostajh wrote:

@Trizek-WMF is the list in the description finalized? If not, please let us know when it is, so we can begin training models, generating the datasets, etc.

@kostajh, you can start the work.

@kostajh Should we help with running the training pipeline to generate datasets and models? I am currently working with @kevinbazira to transition the algorithm to the ML-team. Therefore, going through these steps together to train for a new set of languages might be a good exercise.

In T290011#7358977, @MGerlach wrote:

In T290011#7348334, @Trizek-WMF wrote:

In T290011#7347717, @kostajh wrote:

@Trizek-WMF is the list in the description finalized? If not, please let us know when it is, so we can begin training models, generating the datasets, etc.

@kostajh, you can start the work.

@kostajh Should we help with running the training pipeline to generate datasets and models? I am currently working with @kevinbazira to transition the algorithm to the ML-team. Therefore, going through these steps together to train for a new set of languages might be a good exercise.

@MGerlach, sounds good, please work with @kevinbazira to do the steps on stats1008. Do you think you would be able to do it early next week?

Thanks @kostajh. @MGerlach and I have started running the training pipelines on stat1008. We shall keep you updated on the progress.

Change 722623 had a related patch set uploaded (by Kosta Harlan; author: MGerlach):

[research/mwaddlink@main] Fix parsing error when generating the backtesting data

https://gerrit.wikimedia.org/r/722623

Change 722623 merged by jenkins-bot:

[research/mwaddlink@main] Fix parsing error when generating the backtesting data

https://gerrit.wikimedia.org/r/722623

In T290011#7362447, @kostajh wrote:

@MGerlach, sounds good, please work with @kevinbazira to do the steps on stats1008. Do you think you would be able to do it early next week?

@kevinbazira generated the models and the datasets for all languages mentioned above on stat1008.

The backtesting results are in this spreadsheet (you can also compare to the numbers in previous rounds).
For the default threshold (0.5) we get the following results:

• overall, results are comparable to previous rounds and no major red flags
• cawiki, nowiki, ptwiki, simplewiki, svwiki, ukwiki have high precision (>=79%) and pretty good recall numbers (>=40%) so I am very confident in these models
• hewiki, hiwiki, kowiki have a bit lower precision (72-75%) and quite a bit lower recall (25-28%) so there might be issues in generating enough recommendations; however, I dont think it is extremely concerning as this is comparable to the backtesting results we got for arwiki and bnwiki in the previous round (see the next tab in the spreadsheet).

@kostajh would you like us to continue with publishing the datasets?

In T290011#7376009, @MGerlach wrote:

In T290011#7362447, @kostajh wrote:

@MGerlach, sounds good, please work with @kevinbazira to do the steps on stats1008. Do you think you would be able to do it early next week?

@kevinbazira generated the models and the datasets for all languages mentioned above on stat1008.

The backtesting results are in this spreadsheet (you can also compare to the numbers in previous rounds).
For the default threshold (0.5) we get the following results:

• overall, results are comparable to previous rounds and no major red flags
• cawiki, nowiki, ptwiki, simplewiki, svwiki, ukwiki have high precision (>=79%) and pretty good recall numbers (>=40%) so I am very confident in these models
• hewiki, hiwiki, kowiki have a bit lower precision (72-75%) and quite a bit lower recall (25-28%) so there might be issues in generating enough recommendations; however, I dont think it is extremely concerning as this is comparable to the backtesting results we got for arwiki and bnwiki in the previous round (see the next tab in the spreadsheet).

@kostajh would you like us to continue with publishing the datasets?

@MGerlach yes, please go ahead and publish.

Thank you for the confirmation @kostajh.

@MGerlach and I have published the models and datasets for the 9 languages mentioned above.

Change 723517 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/mediawiki-config@master] GrowthExperiments: Enable AddLink for next round of wikis

https://gerrit.wikimedia.org/r/723517

Change 723518 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] Remove unnecessary GELinkREcommendationsFrontendEnabled config

https://gerrit.wikimedia.org/r/723518

In T290011#7376403, @gerritbot wrote:

Change 723518 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] Remove unnecessary GELinkREcommendationsFrontendEnabled config

https://gerrit.wikimedia.org/r/723518

I ended up moving this to T291687: Remove GELinkREcommendationsFrontendEnabled config flag

In T290011#7388085, @Trizek-WMF wrote:

Given the feedback we received from German Wikipedia and Arabic Wikipedia, we have decided to stall this deployment until we work on limiting the number of links each newcomer can add. The work will resume when "Add an image" will be done.

Do you mean: 1) number of add link tasks the user can complete per day/week? or 2) number of links that can be added per task?

If it's the latter, then implementing that is relatively simple (and is also not directly related or scheduled to be done with the Image-Suggestions work) – we could use the existing maximumLinksPerTask option (this can be adjusted in NewcomerTasks.json) so that the user can only see e.g. a maximum of 5 links on a task, instead of the current default of 10. We would still want to fetch an extra amount of tasks in case we're not able to find some of the results in the UI, though.

@kostajh, the plan is to limit the number of tasks. We will discuss about this in a forthcoming team meeting.

In T290011#7394325, @Trizek-WMF wrote:

@kostajh, the plan is to limit the number of tasks. We will discuss about this in a forthcoming team meeting.

Did we talk about this in a meeting? We now have the ability to limit number of tasks done per day by a user; that functionality was added for Image-Suggestions in T293316: Add an image: minimal quality gate, so it would be trivial to introduce for Add-Link as well.

Change 737402 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] [WIP] Implement dailyLimit quality gate for add link

https://gerrit.wikimedia.org/r/737402

In T290011#7489410, @kostajh wrote:

In T290011#7394325, @Trizek-WMF wrote:

@kostajh, the plan is to limit the number of tasks. We will discuss about this in a forthcoming team meeting.

Did we talk about this in a meeting?

I can't remember which meeting covered it, but I think we have.

We now have the ability to limit number of tasks done per day by a user; that functionality was added for #image-suggestion in T293316: Add an image: minimal quality gate, so it would be trivial to introduce for Add-Link as well.

Oh, good! :)

Change 737402 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Implement dailyLimit quality gate for add link

https://gerrit.wikimedia.org/r/737402

Change 723517 abandoned by Kosta Harlan:

[operations/mediawiki-config@master] GrowthExperiments: Enable AddLink for next round of wikis

Reason:

Outdated

https://gerrit.wikimedia.org/r/723517