Page MenuHomePhabricator

Do not load the entire item into the Lua memory without the need
Closed, InvalidPublic

Description

Wikibase Lua methods for working with item properties like mw.wikibase.getAllStatements() and mw.wikibase.getBestStatements() should not load an entire item into the Lua memory.

The difference between these functions and the methods of the mw.wikibase.entity object is that here we do not initially get all element statements. Therefore, instead of working in the same way as the methods of the mw.wikibase.entity object, they should only load into memory those data that were requested.

A real example: if you want to display a list of country names with flags, then approximately 25 elements are enough to occupy all 50 MB of memory in Lua.

Possibly related tasks: T179638, T76156.

Event Timeline

Vvjjkkii renamed this task from Do not load the entire item into the Lua memory without the need to 3baaaaaaaa.Jul 1 2018, 1:02 AM
Vvjjkkii triaged this task as High priority.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
JJMC89 renamed this task from 3baaaaaaaa to Do not load the entire item into the Lua memory without the need.Jul 1 2018, 4:19 AM
JJMC89 raised the priority of this task from High to Needs Triage.
JJMC89 updated the task description. (Show Details)
JJMC89 added a subscriber: Aklapper.

This has always been the case: Ever since that was introduced with 9f25055cd98293aa0b49fe16acd9490b231b29c1 (the function was later renamed, but this is about the implementation).

mw.wikibase.getBestStatements/mw.wikibase.getAllStatements call getEntityStatements (which is local to mw.wikibase.lua) which in turn uses local statements = php.getEntityStatements( … ) to get just the statements we care about.

There's still room for optimization here on the PHP side (we still load the whole entity there to get statements), but that's a whole different issue.