Tagging GrowthExperiments-LevelingUp since it's come up in the context of that project.

@Cyndymediawiksim is this something you can take up?

In T332348#9230582, @DMburugu wrote:

@Cyndymediawiksim is this something you can take up?

@DMburugu , I can take a look at it

Change 972406 had a related patch set uploaded (by Cyndywikime; author: Cyndywikime):

[mediawiki/extensions/GrowthExperiments@master] Clean up Add Link / Add Image A/B test logic in GrowthExperiments

https://gerrit.wikimedia.org/r/972406

Hello everyone, I was asked by @Cyndymediawiksim to provide guidance with this task. I had a look at this and here's my current understanding of the task, based on conversations with Tgr and Kirsten.

In T278123: Provide capability for A/B testing task types, we introduced modules/ext.growthExperiments.DataStore/TaskTypesAbFilter.js (and related changes) to provide task type mapping (when an user has copyedit,links enabled and link recommendations is enabled, it gets silently converted into copyedit,link-recommendation), to support A/B testing of the Add Link feature. This addition was labeled as a "temporary hack", and this task was filled to get it removed. As mentioned in the task description, this resulted in production errors such as T332309.

According to @Tgr, we improved the filtering code to the point where the mapping could be just made configurable. One of the projects considered by the team is the Copyedit structured task, but apart from the fact that there will be no capacity to work on it this FY, it is unclear when we could eventually work on that project.

As such, it is a purely engineering judgement whether we want to keep the filtering code in (written properly with configuration, etc.), or whether we want to get rid of it and possibly re-do properly it in the future, for example when/if we get to the copyedit structured task.

With the above in my mind, I think the immediate next step to tackle here is to make an engineering decision whether the task type filtering capability should be kept or made configurable. Based on the decision, we can reformulate the task description to accurately record the decision made.

While making the decision, I think we should keep at least the following factors in our minds:

• Deployment of Add Link to additional wikis: Is the unstructured links task enabled on any of the wikis we're not yet deployed to? If it is, what would be the impact of removing the filtering code?
• Current state of the database: On wikis that have Add Link enabled, is the links task type enabled for any of the active users? What impact would removing the filtering code have on those users?

In T332348#9326118, @Urbanecm_WMF wrote:

Hello everyone, I was asked by @Cyndymediawiksim to provide guidance with this task. I had a look at this and here's my current understanding of the task, based on conversations with Tgr and Kirsten.

In T278123: Provide capability for A/B testing task types, we introduced modules/ext.growthExperiments.DataStore/TaskTypesAbFilter.js (and related changes) to provide task type mapping (when an user has copyedit,links enabled and link recommendations is enabled, it gets silently converted into copyedit,link-recommendation), to support A/B testing of the Add Link feature. This addition was labeled as a "temporary hack", and this task was filled to get it removed. As mentioned in the task description, this resulted in production errors such as T332309.

According to @Tgr, we improved the filtering code to the point where the mapping could be just made configurable. One of the projects considered by the team is the Copyedit structured task, but apart from the fact that there will be no capacity to work on it this FY, it is unclear when we could eventually work on that project.

As such, it is a purely engineering judgement whether we want to keep the filtering code in (written properly with configuration, etc.), or whether we want to get rid of it and possibly re-do properly it in the future, for example when/if we get to the copyedit structured task.

With the above in my mind, I think the immediate next step to tackle here is to make an engineering decision whether the task type filtering capability should be kept or made configurable. Based on the decision, we can reformulate the task description to accurately record the decision made.

While making the decision, I think we should keep at least the following factors in our minds:

• Deployment of Add Link to additional wikis: Is the unstructured links task enabled on any of the wikis we're not yet deployed to? If it is, what would be the impact of removing the filtering code?
• Current state of the database: On wikis that have Add Link enabled, is the links task type enabled for any of the active users? What impact would removing the filtering code have on those users?

Based on the context provided by @Urbanecm, and a discussion with @Sgs, my recommendation would be to refactor and keep the task type filtering code. My reasons for this recommendation are:

1. New features and requirements emerge regularly. Keeping a configurable and well-refactored version of the task type filtering code would provide flexibility for future developments, such as the potential introduction of new task types like the Copyedit structured task as Martin highlighted above.

2. While removing the code might seem like a quick fix to the current technical debt, it could lead to higher effort in the future if similar functionality is needed again. A well-refactored codebase can reduce future development time and effort, even if it requires more resources in the short term.

3. Considering the deployment of the Add Link feature across various wikis, removing the code might have unforeseen impacts, especially on wikis where the unstructured links task is active. Refactoring ensures that existing functionalities are not disrupted.

As Martin highlighted, it is crucial to ensure that the refactored code is well-documented, easily configurable, and thoroughly tested to prevent issues similar to those encountered in https://phabricator.wikimedia.org/T332309.

@KStoller-WMF , should we move this to the Needs Discussion column?

In T332348#9329828, @Cyndymediawiksim wrote:
@KStoller-WMF , should we move this to the Needs Discussion column?

Does this need further discussion? Assuming @Sgs is OK with your recommendation, then I think we can move this task forward. Any concerns, @Sgs?

In T332348#9332844, @KStoller-WMF wrote:

In T332348#9329828, @Cyndymediawiksim wrote:
@KStoller-WMF , should we move this to the Needs Discussion column?

Does this need further discussion? Assuming @Sgs is OK with your recommendation, then I think we can move this task forward. Any concerns, @Sgs?

I understand that Martin's and Cynthia's comments are leaning towards keeping the filtering capability and make it configurable based on some mapping of "variant name" to "task type". I agree the feature is nice to have and can be relatively easy achieved in terms of code but the questions Martin raised still need to be taken in account:

In T332348#9326118, @Urbanecm_WMF wrote:

• Deployment of Add Link to additional wikis: Is the unstructured links task enabled on any of the wikis we're not yet deployed to? If it is, what would be the impact of removing the filtering code?
• Current state of the database: On wikis that have Add Link enabled, is the links task type enabled for any of the active users? What impact would removing the filtering code have on those users?

My first guess is that it would result into incorrect display of both tasks or incorrect display of just one (no replacement). If instead of removing the filtering we decide to make A/B test task types configurable we would need to provide graceful fallbacks when a task type is not available for whatever reason (eg: community disabled it)

As Martin mentioned the most promising use case for this feature would be the Copyedit structured task and that's not an urgent one. In my opinion we can live with the technical debt this created for some more time. If we all agree we could reformulate the task description to capture better the A/B configuration requirements, including the Growth config form graying out or change of texts. Is this work something we can postpone a bit while we work on CC2.0? cc @KStoller-WMF @DMburugu

@Sgs I understand your proposition and I'm okay living with the tech debt if it'll be cleaned. I see that the work is already tracked under the under the Maintenance Epic which is okay.