Maniphest T281928

No event fired for clicking the language button
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Jdlrobson
	May 4 2021, 10:53 PM

Description

We currently have no way to detect when the language button is clicked.

The existing schema has no event for this action.
https://meta.wikimedia.org/wiki/Schema:UniversalLanguageSelector

TODO:

Option 1) A new event is added to the schema (schema change) and we log this
~~Option 2) We use the DesktopUIClickTracking to capture this event by adding data-event-name="ui.language-btn"~~ See T281928#7063815

QA Results - Beta

AC	Status	Details
1	✅	T281928#7115830

QA Results - Prod

AC	Status	Details
1	✅	T281928#7115852

Details

Subject	Repo	Branch	Lines +/-
Fire hook when compact language links are opened	mediawiki/extensions/UniversalLanguageSelector	master	+2 -2
universalLanguageSelector: Log when mw.uls.compact_language_links.open hook fires	mediawiki/extensions/WikimediaEvents	master	+9 -1
universalLanguageSelector: Add compact-language-links-open action	schemas/event/secondary	master	+651 -4

Customize query in gerrit

Related Objects

Mentioned In: T275762: Instrument clicks to links in the Languages list in the sidebar
T280770: Instrumentation QA for language switching
T269093: Deploy new language switching location to test wikis and begin A/B test pt 1
Mentioned Here: T280770: Instrumentation QA for language switching
T274869: Instrumentation QA for new search widget
T269093: Deploy new language switching location to test wikis and begin A/B test pt 1
T275762: Instrument clicks to links in the Languages list in the sidebar

Event Timeline

Jdlrobson created this task.May 4 2021, 10:53 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 4 2021, 10:53 PM

Change 684984 had a related patch set uploaded (by Jdlrobson; author: Phuedx):

[mediawiki/extensions/UniversalLanguageSelector@master] Fire hook when opened via language button

https://gerrit.wikimedia.org/r/684984

gerritbot added a project: Patch-For-Review.May 4 2021, 10:53 PM

ovasileva triaged this task as High priority.May 5 2021, 9:02 AM

ovasileva subscribed.

LGoto assigned this task to Jdlrobson.May 5 2021, 5:27 PM

LGoto moved this task from Upcoming to Needs Analysis on the Web-Team-Backlog (Kanbanana-FY-2020-21) board.

@MNeisler could you advise us here on how this data is going to be used? We assume it will be wanted as part of the UniversalLanguageSelector, but were hoping you could confirm.

@jrobson - Yes, this should be included as part of the UniversalLanguageSelector Schema if possible.

This data will be used in the AB test to determine sessions with clicks to the language button in the header (from users that receive the treatment) and compare to sessions with clicks to the ULS settings (currently logged as event.action = "settings-open" AND event.context = "interlanguage" in ULS) and language links in the sidebar (to be added to ULS as part of T275762 ) from users in the control.

Jdlrobson removed MNeisler as the assignee of this task.May 5 2021, 10:49 PM

Jdlrobson moved this task from Needs Analysis to Upcoming on the Web-Team-Backlog (Kanbanana-FY-2020-21) board.

phuedx claimed this task.May 6 2021, 1:12 PM

phuedx moved this task from Upcoming to Needs Analysis on the Web-Team-Backlog (Kanbanana-FY-2020-21) board.

phuedx updated the task description. (Show Details)May 6 2021, 1:20 PM

Change 684984 abandoned by Phuedx:

[mediawiki/extensions/UniversalLanguageSelector@master] Fire hook when opened via language button

Reason:

See Nray and Jdlrobson's comments inline.

https://gerrit.wikimedia.org/r/684984

Maintenance_bot removed a project: Patch-For-Review.May 6 2021, 4:11 PM

Notes from today's investigation into what is logged and when:

When I click the ULS link in the personal tools (either $wgULSPosition = 'personal' in your LocalSettings.php or on commonswiki and wikidatawiki), I see:
- Almost the same treatment as that triggered by the button in Vector2
- An event.action = "settings-open" AND event.context = "personal" event logged
When I click either the "Display settings" or "Input settings" buttons at the bottom of the ULS, I don't see an event logged

@phuedx the ULS link in personal tools is the UniversalLanguageSelector. The language button we have created in the top right of articles is the "Compact Languages Button". Although they share lots of similiarities they should be considered two different features. In the desktop refresh version of the compact languages button the links to "display settings" and "input settings" at the bottom launch the same think as the button in the personal tools.

Screen Shot 2021-05-07 at 10.08.24 AM.png (736×864 px, 57 KB)

It seems like the solution here is to modify the schema to support this new event to avoid ambiguity betwen the two in analysis. It looks to me like the compact language button feature was never instrumented.

In T281928#7069948, @Jdlrobson wrote:

It seems like the solution here is to modify the schema to support this new event to avoid ambiguity betwen the two in analysis. It looks to me like the compact language button feature was never instrumented.

For sure. I just found it noteworthy that the ULS link in the personal tools would trigger an event.action = "settings-open" AND event.context = "personal" event and not, say, event.action = "uls-open" (remembering that clicking the cog in the sidebar also triggers an event.action = "settings-open" AND event.context = "interlanguage" event).

Change 688217 had a related patch set uploaded (by Phuedx; author: Phuedx):

[schemas/event/secondary@master] universalLanguageSelector: Add compact-language-links-open action

https://gerrit.wikimedia.org/r/688217

Change 688218 had a related patch set uploaded (by Phuedx; author: Phuedx):

[mediawiki/extensions/WikimediaEvents@master] universalLanguageSelector: Log when mw.uls.compact_language_links.open hook fires

https://gerrit.wikimedia.org/r/688218

Change 684984 restored by Phuedx:

[mediawiki/extensions/UniversalLanguageSelector@master] Fire hook when opened via language button

https://gerrit.wikimedia.org/r/684984

ovasileva removed phuedx as the assignee of this task.May 10 2021, 12:58 PM

ovasileva added a subscriber: phuedx.

The above three patches (https://gerrit.wikimedia.org/r/688217, https://gerrit.wikimedia.org/r/688218, and https://gerrit.wikimedia.org/r/684984) cover logging the case where the user clicks the language button.

LGoto assigned this task to Jdlrobson.May 10 2021, 5:35 PM

Change 688217 merged by jenkins-bot:

[schemas/event/secondary@master] universalLanguageSelector: Add compact-language-links-open action

https://gerrit.wikimedia.org/r/688217

Change 688218 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] universalLanguageSelector: Log when mw.uls.compact_language_links.open hook fires

https://gerrit.wikimedia.org/r/688218

The above patches have been merged (thanks @nray for your help here!)

While reviewing we noticed that we are only firing an event for the first click. Is this enough for the purpose of the A/B test?

If we want to capture subsequent clicks, we also have to consider whether the click is to open or to close the language overlay... if the language overlay is open a second click will close it... unless I click outside the overlay and then click the button again ... @MNeisler can you advise?

Moving to "needs more work" but with an inflection on the word "work" to make it a question.

ReleaseTaggerBot added a project: MW-1.37-notes (1.37.0-wmf.5; 2021-05-11).May 11 2021, 1:00 AM

In T281928#7076572, @Jdlrobson wrote:

While reviewing we noticed that we are only firing an event for the first click. Is this enough for the purpose of the A/B test?

If we want to capture subsequent clicks, we also have to consider whether the click is to open or to close the language overlay... if the language overlay is open a second click will close it... unless I click outside the overlay and then click the button again ... @MNeisler can you advise?

Good catch. FYI the ULS link in the personal tools fires an event only when the ULS is shown, i.e.

User clicks the ULS link in the personal tools and an event.action = "settings-open" AND event.context = "personal" event is fired
User clicks and the ULS is hidden
User clicks and an event.action = "settings-open" AND event.context = "personal" event is fired
User clicks and the ULS is hidden
…

The same is true if the user is clicking the cog in the sidebar.

Change 688994 had a related patch set uploaded (by Phuedx; author: Phuedx):

[mediawiki/extensions/UniversalLanguageSelector@master] Fire hook when compact language links is opened

https://gerrit.wikimedia.org/r/688994

While reviewing we noticed that we are only firing an event for the first click. Is this enough for the purpose of the A/B test?

The first click would be sufficient to determine which ULS treatment the user is receiving for the AB test and compare the number of sessions that include a click to the language button in the header vs sessions that include a click to one of the links/buttons available in the control. With the session id, I should also be able to link users where a click to the language button was recorded with subsequent actions such as switching languages.

However, if feasible, I'd still recommend recording subsequent clicks as well. This would allow us to understand the frequency of usage of the button and be able to directly compare to the usage of ULS setting and language clicks in the control group.

If we want to capture subsequent clicks, we also have to consider whether the click is to open or to close the language overlay... if the language overlay is open a second click will close it... unless I click outside the overlay and then click the button again ... @MNeisler can you advise?

If possible, I think we'd only need to fire an event when the button is opened/shown (not closed) to be consistent with how the other ULS events are instrumented.

Jdlrobson mentioned this in T269093: Deploy new language switching location to test wikis and begin A/B test pt 1.May 11 2021, 10:34 PM

Change 688994 merged by jenkins-bot:

[mediawiki/extensions/UniversalLanguageSelector@master] Fire hook when compact language links are opened

https://gerrit.wikimedia.org/r/688994

ReleaseTaggerBot edited projects, added MW-1.37-notes (1.37.0-wmf.6; 2021-05-18); removed MW-1.37-notes (1.37.0-wmf.5; 2021-05-11).May 11 2021, 11:00 PM

Thanks @phuedx and @MNeisler .

Note, we made need to backport the above change as part of enabling the A/B test (included in https://phabricator.wikimedia.org/T269093)

In T281928#7080329, @Jdlrobson wrote:

Thanks @phuedx and @MNeisler .

Note, we made need to backport the above change as part of enabling the A/B test (included in https://phabricator.wikimedia.org/T269093)

Is this in all cases or only in case we want to begin the A/B test prior to the next train?

In T281928#7081182, @ovasileva wrote:

In T281928#7080329, @Jdlrobson wrote:

Note, we made need to backport the above change as part of enabling the A/B test (included in https://phabricator.wikimedia.org/T269093)

Is this in all cases or only in case we want to begin the A/B test prior to the next train?

Only in the case where we want to begin the A/B test prior to the next train completely rolling out (i.e. EOD next Thursday, 20th May).

MNeisler mentioned this in T280770: Instrumentation QA for language switching .May 12 2021, 11:51 PM

@phuedx The only language selector related event I get is the following one. And it only fires when I select a language. I don't get anything when I open/close the language button. If this is what you are expecting let me know and I'll make a nicely formatted testing comment and table.

{

"event": {
  "version": 2,
  "token": "",
  "contentLanguage": "en",
  "interfaceLanguage": "en",
  "web_session_id": "ceced467437627ce23c4",
  "isAnon": true,
  "action": "compact-language-links-open"
},
"schema": "UniversalLanguageSelector",
"webHost": "en.wikipedia.beta.wmflabs.org",
"wiki": "enwiki",
"$schema": "/analytics/legacy/universallanguageselector/1.3.0",
"client_dt": "2021-05-20T01:49:16.144Z",
"meta": {
  "stream": "eventlogging_UniversalLanguageSelector",
  "domain": "en.wikipedia.beta.wmflabs.org"
}

}

In T281928#7099811, @Edtadros wrote:

@phuedx The only language selector related event I get is the following one. And it only fires when I select a language. I don't get anything when I open/close the language button. If this is what you are expecting let me know and I'll make a nicely formatted testing comment and table.

...

Could you double check whether that event was fired when you selected a language as I'm seeing a similar event fired when I click the language button (which I what I expect). Here's what I did:

Navigate to https://en.wikipedia.beta.wmflabs.org/wiki/Barack_Obama?useskinversion=2
Click on "1 language" to the right of the title
Observe that a action="compact-language-links-open" event has been logged

@phuedx I see it now. It takes a good number of seconds to appear though.

Screen Shot 2021-05-25 at 6.10.15 AM.png (1×1 px, 563 KB)

Is this good enough to move to Needs QA in Prod?

In T281928#7111127, @Edtadros wrote:

Is this good enough to move to Needs QA in Prod?

Yes. Being bold and moving it.

ovasileva mentioned this in T275762: Instrument clicks to links in the Languages list in the sidebar.May 26 2021, 1:16 PM

Very bold, thanks @phuedx

Test Result - Beta

Status: ✅ PASS
Environment: beta
OS: macOS Big Sur
Browser: Chrome
Device: MBP
Emulated Device: NA

Test Artifact(s):

QA Steps

✅ AC1: An event is fired when the ULS button is clicked
T281928#7111127
T281928#7111487

Test Result - Prod

Status: ✅ PASS
Environment: enwiki
OS: macOS Big Sur
Browser: Chrome
Device: MBP
Emulated Device: NA

Test Artifact(s):

QA Steps

✅ AC1: An event is fired when the ULS button is clicked

Screen Shot 2021-05-26 at 6.40.24 AM.png (1×1 px, 597 KB)

Edtadros updated the task description. (Show Details)May 26 2021, 1:52 PM

Edtadros updated the task description. (Show Details)

Change 695351 had a related patch set uploaded (by Phuedx; author: Phuedx):

[mediawiki/extensions/UniversalLanguageSelector@master] Pass context to compact_language_links.open hook

https://gerrit.wikimedia.org/r/695351

Please ignore the above.

ovasileva removed ovasileva as the assignee of this task.May 27 2021, 2:07 PM

Being bold and signing this off and broader instrumentation QA and sign off is happening over at T280770: Instrumentation QA for language switching .

	F34469613: Screen Shot 2021-05-26 at 6.40.24 AM.png
	May 26 2021, 1:42 PM

	F34467407: Screen Shot 2021-05-25 at 6.10.15 AM.png
	May 25 2021, 1:14 PM

	F34445347: Screen Shot 2021-05-07 at 10.08.24 AM.png
	May 7 2021, 5:09 PM

No event fired for clicking the language buttonClosed, ResolvedPublicActions

Description

TODO:

QA Results - Beta

QA Results - Prod

Details

Related Objects

Event Timeline

Test Result - Beta

QA Steps

Test Result - Prod

QA Steps

No event fired for clicking the language button
Closed, ResolvedPublic
Actions