This task is about implementing the changes necessary to ensure people are being bucketed in the Usability Improvements A/B test (T302358) as expected.
Bucketing requirements
- Bucketing should include
both logged-in and logged-out userslogged-in users - Everyone who is accessing a desktop talk page on any of the wikis listed in the Participating wikis, logged in
and logged out, should have a 50% chance of being included/bucketed into the A/B test's control or test group - Bucketing should be done on a per-Wikipedia basis. 50% of people within a given wiki should be placed within the control group; 50% should be bucketed in the test group
- People should remain in the same group for the duration of the test (even if they explicitly turn on or off discussion tool preferences).
- Testing should be done across various desktop skins (not limited to just Vector 2022)
Instrumentation-Related Requirements
- Within the A/B test, we are able to distinguish all events logged for the control group and the test group.
- For all wikis in the test, we are able to join events from DesktopWebUIActionsTracking to EditAttemptStep (confirmed in T330938)
- We are able to track if/when someone in the test group adjusts any of the discussion tool-related preferences.
- We can track the desktop skin type being viewed by a user in the AB test (confirmed in T335159)
Participating wikis
Defined in T332946 (still being finalized)