Maniphest T196868

Track maxlag for wikidata on grafana dashboard
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Addshore
	Jun 11 2018, 7:36 AM

Description

This metric should be tracked along with the 'real' dispatch values, so that we can see what clients see with the modified dispatch factor.

This will require updating the script in gerrit/analytics/wmde/scripts that calls the api looking for the dispatch lag and also updating the dashboard on grafana.

Details

	Subject	Repo	Branch	Lines +/-
	Track new API reported maxlag for wikidata on grafana dashboard	analytics/wmde/scripts	production	+29 -1
	Track new API reported maxlag for wikidata on grafana dashboard	analytics/wmde/scripts	master	+29 -1

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		• Jonas	T196868 Track maxlag for wikidata on grafana dashboard
		Resolved		Ladsgroup	T194950 Include Wikibase dispatch lag in API "maxlag" enforcing

Event Timeline

Addshore created this task.Jun 11 2018, 7:36 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 11 2018, 7:36 AM

Addshore added a subtask: T194950: Include Wikibase dispatch lag in API "maxlag" enforcing.Jun 11 2018, 7:36 AM

Addshore updated the task description. (Show Details)

Lucas_Werkmeister_WMDE triaged this task as Medium priority.Jun 12 2018, 1:34 PM

Lucas_Werkmeister_WMDE moved this task from Incoming to Ready to estimate on the Wikidata-Campsite board.

So, the script is https://github.com/wikimedia/analytics-wmde-scripts/blob/master/src/wikidata/dispatch.php

We will need to:

Alter the script to make a new API call to get the new maxlag value.
Send that data to graphite using sendGauge with a sensibly named metric name
Merge patch on master of the repo on gerrit
Cherry-pick patch to production branch on gerrit & merge
Update the relevant dashboard to also show this new lag value somewhere

GoranSMilovanovic added a subscriber: User-GoranSMilovanovic.Jun 25 2018, 10:14 AM

GoranSMilovanovic edited subscribers, added: GoranSMilovanovic; removed: User-GoranSMilovanovic.

• Jonas claimed this task.Jun 25 2018, 11:34 AM

• Jonas moved this task from Ready to estimate to In Progress on the Wikidata-Campsite board.

Change 441838 had a related patch set uploaded (by Jonas Kress (WMDE); owner: Jonas Kress (WMDE)):
[analytics/wmde/scripts@master] Track new API reported maxlag for wikidata on grafana dashboard

https://gerrit.wikimedia.org/r/441838

gerritbot added a project: Patch-For-Review.Jun 25 2018, 11:56 AM

• Jonas mentioned this in rADESf70e3c5906a6: Track new API reported maxlag for wikidata on grafana dashboard.Jun 25 2018, 11:58 AM

• Jonas mentioned this in rADESf6a6add2e471: Track new API reported maxlag for wikidata on grafana dashboard.

• Jonas mentioned this in rADES9d90029cafe6: Track new API reported maxlag for wikidata on grafana dashboard.

• Jonas moved this task from In Progress to Peer Review on the Wikidata-Campsite board.Jun 25 2018, 11:58 AM

Addshore mentioned this in rADES0250fbd0c26b: Track new API reported maxlag for wikidata on grafana dashboard.Jun 25 2018, 12:09 PM

I guess this is stalled until T194950 is complete as the new max lag is not yet enabled anywhere

Per T194950#4314787 it is impossible to specifically track our new max lag without a new api module, so instead this task can just be about tracking the wikidata maxlag in general.

Addshore renamed this task from Track new API reported maxlag for wikidata on grafana dashboard to Track maxlag for wikidata on grafana dashboard.Jun 26 2018, 10:43 AM

• Jonas mentioned this in rADESb5e40db6cde1: Track new API reported maxlag for wikidata on grafana dashboard.Jun 26 2018, 4:03 PM

• Jonas mentioned this in rADESb656c82be3ee: Track new API reported maxlag for wikidata on grafana dashboard.Jun 27 2018, 9:22 AM

Change 442827 had a related patch set uploaded (by Addshore; owner: Jonas Kress (WMDE)):
[analytics/wmde/scripts@production] Track new API reported maxlag for wikidata on grafana dashboard

https://gerrit.wikimedia.org/r/442827

Change 441838 merged by jenkins-bot:
[analytics/wmde/scripts@master] Track new API reported maxlag for wikidata on grafana dashboard

https://gerrit.wikimedia.org/r/441838

Change 442827 merged by jenkins-bot:
[analytics/wmde/scripts@production] Track new API reported maxlag for wikidata on grafana dashboard

https://gerrit.wikimedia.org/r/442827

• Jonas mentioned this in rADES15f8f6fe60ac: Track new API reported maxlag for wikidata on grafana dashboard.Jun 28 2018, 11:04 AM

Addshore mentioned this in rADESacb2dd701ae1: Track new API reported maxlag for wikidata on grafana dashboard.Jun 28 2018, 11:04 AM

I added it to the board https://grafana.wikimedia.org/dashboard/db/wikidata-dispatch

• Jonas moved this task from Peer Review to Test (Verification) on the Wikidata-Campsite board.Jun 28 2018, 1:34 PM

Ladsgroup moved this task from incoming to in progress on the Wikidata board.Jun 28 2018, 3:14 PM

Looks good!
I filed a small follow up patch, but only vaguely related to this patch as the issue already existed with the dispatch script that was copied for this ticket.
https://gerrit.wikimedia.org/r/#/c/analytics/wmde/scripts/+/443036/

• Vvjjkkii renamed this task from Track maxlag for wikidata on grafana dashboard to rabaaaaaaa.Jul 1 2018, 1:04 AM

• Vvjjkkii removed • Jonas as the assignee of this task.

• Vvjjkkii raised the priority of this task from Medium to High.