Page MenuHomePhabricator
Create Task
Maniphest T222954

Imageinfo queries to non-Commons wikis about Commons files return incomplete extmetadata
Closed, ResolvedPublicBUG REPORT
Actions

Description

This is affecting our /page/media results

Steps to Reproduce:

for File:Female_Anna's_Hummingbird.JPG:
https://en.wikipedia.org/w/api.php?action=query&prop=imageinfo&iiprop=extmetadata&iiextmetadatamultilang=&titles=File:Female_Anna%27s_Hummingbird.JPG

Actual Results:

Only a small subset of extmetadata fields (DateTime, ObjectName, CommonsMetadataExtension, Categories, Assessments) are returned.

Important fields like License and ImageDescription are missing.

Expected Results:

All extmetadata is returned.

Details

SubjectRepoBranchLines +/-
Invalidate CommonsMetadata cache for entries affected by T222935operations/mediawiki-configmaster+6 -0
Add ability for manual cache resetsmediawiki/extensions/CommonsMetadatamaster+86 -7
Revert "Revert "Bifurcate imageinfo queries to improve performance""mediawiki/services/mobileappsmaster+90 -38
Customize query in gerrit

Related Objects

Event Timeline

• Mholloway created this task.May 10 2019, 1:42 PM
Restricted Application added a project: Multimedia. · View Herald TranscriptMay 10 2019, 1:42 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
gerritbot added a comment.May 10 2019, 1:43 PM

Change 509398 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[mediawiki/services/mobileapps@master] Revert "Revert "Bifurcate imageinfo queries to improve performance""

https://gerrit.wikimedia.org/r/509398

gerritbot added a project: Patch-For-Review.May 10 2019, 1:43 PM
• Mholloway claimed this task.May 10 2019, 1:44 PM
• Mholloway moved this task from Doing to Code Review on the Product-Infrastructure-Team-Backlog-Deprecated (Kanban) board.
• Mholloway added a comment.May 10 2019, 1:53 PM

Hmm... this diagnosis isn't quite right. Removing the iiextmetadatamultilang= argument from the query results in the full set being returned.

https://en.wikipedia.org/w/api.php?format=json&formatversion=2&action=query&prop=imageinfo|pageprops&iiprop=url|mime|size|thumbmime|metadata|extmetadata&iiurlwidth=320&iimetadataversion=latest&continue=&titles=File%3AFemale_Anna%27s_Hummingbird.JPG

• Mholloway renamed this task from Imageinfo queries to non-Commons wikis return incomplete extmetadata to Imageinfo queries to non-Commons wikis return incomplete extmetadata when iiextmetadatamultilang=true.May 10 2019, 1:55 PM
• Mholloway added a comment.May 10 2019, 2:01 PM

Weirdly, it's also only true for certain files.

Compare

https://en.wikipedia.org/w/api.php?action=query&prop=imageinfo&iiprop=extmetadata&iiextmetadatamultilang=&titles=File:Female_Anna%27s_Hummingbird.JPG (bad)

with

https://en.wikipedia.org/w/api.php?action=query&prop=imageinfo&iiprop=extmetadata&iiextmetadatamultilang=&titles=File:Trinidad_and_Tobago_hummingbirds_composite.jpg (good)

• Mholloway renamed this task from Imageinfo queries to non-Commons wikis return incomplete extmetadata when iiextmetadatamultilang=true to Imageinfo queries to non-Commons wikis return incomplete extmetadata for some files when iiextmetadatamultilang=true.May 10 2019, 2:01 PM
• Mholloway updated the task description. (Show Details)
• Mholloway updated the task description. (Show Details)May 10 2019, 2:03 PM
Tgr updated the task description. (Show Details)May 10 2019, 3:15 PM
TJH2018 subscribed.May 10 2019, 3:17 PM
Tgr subscribed.EditedMay 10 2019, 3:19 PM

I don't see the Commons description of either File:Female_Anna's_Hummingbird.JPG or File:Trinidad_and_Tobago_hummingbirds_composite.jpg show up on their enwiki page, which is a (possibly unrelated) bug in itself.

TJH2018 added a comment.EditedMay 10 2019, 3:34 PM

Could this possibly be related to the implementation of the new Commons description/Wikidata template?

Tgr added a comment.May 10 2019, 4:16 PM
In T222954#5172688, @Tgr wrote:

I don't see the Commons description of either File:Female_Anna's_Hummingbird.JPG or File:Trinidad_and_Tobago_hummingbirds_composite.jpg show up on their enwiki page, which is a (possibly unrelated) bug in itself.

That's T222935: HttpRequestFactory get method always return null (was: All local file description pages pointing to Commons do not display description locally from 1.34.0-wmf.4), apparently.

CommonsMetadata parses the local description page so it would make sense for that to be the cause, but then it should expect all the files. Maybe it's just a caching issue and it will eventually...

• Mholloway added a project: Commons.May 10 2019, 10:16 PM
• Mholloway updated the task description. (Show Details)May 10 2019, 10:19 PM
Tgr merged a task: T223029: Incorrect and misleading license and author information on Android and desktop.May 13 2019, 10:58 AM
Tgr added a subscriber: Bencemac.
Tgr added a comment.May 13 2019, 11:03 AM

Indeed caused by T222935, but the wrong results are still cached. A purge (on Commons) fixes it.
The cache expires only after 30 days, so we should probably do something about it.

gerritbot added a comment.May 13 2019, 1:12 PM

Change 509398 abandoned by Mholloway:
Revert "Revert "Bifurcate imageinfo queries to improve performance""

Reason:
Underlying issue (T222935) is fixed, so there's no need anymore to revive this as a band-aid.

https://gerrit.wikimedia.org/r/509398

gerritbot added a comment.May 13 2019, 2:47 PM

Change 509869 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/extensions/CommonsMetadata@master] Add ability for manual cache resets

https://gerrit.wikimedia.org/r/509869

Tgr renamed this task from Imageinfo queries to non-Commons wikis return incomplete extmetadata for some files when iiextmetadatamultilang=true to Imageinfo queries to non-Commons wikis about Commons files return incomplete extmetadata.May 13 2019, 4:18 PM
• Mholloway reassigned this task from • Mholloway to Tgr.May 13 2019, 4:20 PM

Passing this to @Tgr for the cache busting piece.

Tgr added a comment.May 13 2019, 4:20 PM

(FWIW iiextmetadatamultilang=true is only relevant in that it acted as a cache breaker.)

Tgr mentioned this in T222935: HttpRequestFactory get method always return null (was: All local file description pages pointing to Commons do not display description locally from 1.34.0-wmf.4).May 13 2019, 4:22 PM
gerritbot added a comment.May 13 2019, 5:57 PM

Change 509905 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[operations/mediawiki-config@master] Invalidate CommonsMetadata cache for entries affected by T222935

https://gerrit.wikimedia.org/r/509905

gerritbot added a comment.May 13 2019, 6:00 PM

Change 509869 abandoned by Gergő Tisza:
Add ability for manual cache resets

Reason:
Abandoning in favor of using the hook directly (see I1fa3407a2 for current issue)

https://gerrit.wikimedia.org/r/509869

gerritbot added a comment.May 13 2019, 6:23 PM

Change 509905 merged by jenkins-bot:
[operations/mediawiki-config@master] Invalidate CommonsMetadata cache for entries affected by T222935

https://gerrit.wikimedia.org/r/509905

Stashbot added a comment.May 13 2019, 6:25 PM

Mentioned in SAL (#wikimedia-operations) [2019-05-13T18:25:44Z] <reedy@deploy1001> Synchronized wmf-config/CommonSettings.php: T222954 (duration: 00m 49s)

• Mholloway closed this task as Resolved.May 13 2019, 9:13 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits