Wikidata

Wikidata:Infobox Tutorial

Tracked in Phabricator
Task T76229

This is a tutorial that teaches you how to create Wikidata-powered infoboxes or other templates for Wikipedia and other projects connected to Wikidata. This is a living document. Participate in the discussions and edit it with confidence. And please complain on the talk page if you get stuck anywhere.

What you need to know to follow the tutorial

edit
 
Lua is a programming language used for Wikidata templates. It is easy and fun to learn.

All you need to follow the tutorial is some basic programming knowledge. You don't need to be an expert with any of these technologies, but learning some basics might help you get started. The following subsections give you an overview over all the technologies used in this tutorial.

Mediawiki

edit

Mediawiki is the software behind Wikipedia. Usually when you edit pages they will have titles like "Vienna". Mediawiki also has pages with prefixes that serve special purposes. In this guide we will visit pages that have the "Template:" and "Module:" prefix (called "namespace"). These namespaces might have different names in your language, but will work just the same. Templates are an older but very powerful way Mediawiki allows you to use text on many pages. Modules are the place where Lua code is stored that is used on Wikipedia pages.

Mediawiki templates

edit

Templates are not part of this tutorial, but you should have a basic understanding how they work. Take some time to see how the template language works.

For this tutorial also look at Template:Infobox (Q5626735), which on many Wikipedias (and other projects) only calls Module:Infobox (Q13107716) to set up the Infobox. The details will be explained in the Module:Infobox section below.


Mediawiki modules

edit

The first chapter of this guide explains how modules work.

HTML & CSS

edit

HTML and CSS are easy to learn and tell the Internet browser what is on an Internet page, how it is structured and how it should look like. For this guide you only need to know how to create a HTML table and use CSS to format the table and the text that is inside it.

Lua

edit

You only need to know the basics of Lua. Save a string into a variable and print it. Try to create an if statement and a for loop. Then come back and continue with this tutorial. This is an example of a place where you can learn Lua: Wikiversity course on Lua

If you get more interested how Lua is used in the Wikimedia ecosystem you can also look at this guide:

Modules

edit
 
Creating a new module on en-wiki will show a code editor instead of a Wikitext editor.

This section will teach you basic knowledge about modules on Wikipedia and related projects.

Basics

edit

Modules are a special namespace (a set of pages with special properties) in Mediawiki and have the prefix "Module:". While the content namespace of Wikipedia handles Wikitext-content, the Module namespace will interpret any text written as Lua. In order to create a Wikidata-powered infobox we will have to create and edit some of the modules, so we need to understand how they work.

The module namespace uses the Scribunto-Extension which allows you to embed Lua code in Mediawiki. For this tutorial you don't need to know much of the background, but you can read the Scribunto documentation in case you want to learn more about it.

For us it is just important that if we visit a Module page we will see a slightly different interface. If you, for example visit this page: https://test.wikidata.org/wiki/Module:Wikidata and click on "Edit" you will see that a code editor will be shown. On other Wikipedia languages you will also see some helpful information about what a module is, and where to find more documentation (See the figure to the right).

Demonstration modules to see how Modules work:

Debug console

edit
 
The debug console can be used to execute Lua statements to check the written code

Below the code editor the Module-page has a debug console. The console can execute any Lua commands and will also parse the code that you have written in the code editor. That means that you can check if your code returns the right value. We can for example write this in the code editor: 

local p = {}

function p.hello()
    return 'This is a hold-up, not a botany lesson.'
end

return p

and run the following commands in the console: 

> print(p.hello())

This is a hold-up, not a botany lesson.
> print(p.hello() == True)
false

Lua is actually not disagreeing with our bold statement, but only evaluating that string == boolean is false. If you didn't know this before you just learned that programming languages can have "types". Most programming languages for example have types for text, numbers and booleans (a boolean is basically a variable that can only be true or false). When comparing a string to a boolean most programming languages will first check if the types match. If the types don't match, the statement is automatically false.

Importing other modules

edit

Modules can also use code from other modules. This is done by "requiring" another module at the beginning of the module you are editing.

When you import another module into your module you can also inspect it:

local p = {}
local wd = require('Module:Wikidata')

function p.returnModule()
    return wd
end

return p

and in the console we can inspect the object (or table as Lua calls them): 

> wd_obj = p.returnModule()
> table.foreach(wd_obj, print)
pageId	function
getImages	function
labelIn	function
getQualifierDateValue	function
...

Using modules in wikitext

edit

A module that already contains functions can be called anywhere in Wikitext. This is called invoking the module and uses the invoke-template (Examples: Module:Wikidata). The version of the Wikidata:Module here on Wikidata contains a function p.getLabel() (a method of the p object/table). If we invoke the function here, using {{#invoke:Wikidata|getLabel|entity=Q212730}}, it will display the following:

Output: Steven Pinker

It is easy to reconstruct that we are calling the Wikidata module and within it, the getLabel function. This function needs to know at least which item we are interested in which we pass as an argument |entity=Q212730. We are not requesting a specific language, so the Module will default to English.

Using preview on module

edit

The preview function in Module namespace allows to view a page running the module.

Example 1:

  • On Module:NewModule write your code. The code should include an option to use an entity other than the one associated with the page (e.g. Q24045706 in the sample). You could try the below sample.
local p = {}

function p.functionname(frame)
    local frame = mw.getCurrentFrame()
	local id = frame.args.id

	local data = mw.wikibase.getEntityObject(id)
	if not data then
		return "no data"
	end
	local name = data:getLabel( "en" )
	return '<b>' .. name .. '</b>'
end

return p
  • In "Preview page with this template" add the name of the page
    • (e.g. (for the first sample above) "Wikidata:Sandbox/test1"
    • or (for here, the second sample above) "Wikidata:Infobox Tutorial")
  • The red error text "Script error: No such module "NewModule"" should now be replaced by the label of Q24045706

Module:Wikidata

edit

It is also recommended to read and understand the basic workings of your projects Wikidata-Module. The modules are developed separately and can have different approaches of retrieving certain information (e.g. local-specific date formats).

Take some time to review the Module and read the module in one of the languages and projects: Module:Wikidata (Q12069631).

Wikidata also has a Module:Wikidata which we will use as an example in this section. Depending on your local module the function names might have different names, arguments and return values. Module:Wikidata can be used on any Wikitext page on Wikidata. You can find examples here (Module:Wikidata#Examples) which you can try out in your sandbox. We will continue this tutorial by using Module:Wikidata from another Lua module. You can now visit test.wikidata.org and edit this page: https://test.wikidata.org/wiki/Module:Wikidata/Test. Module:Wikidata on test.wikidata is less documented (https://test.wikidata.org/wiki/Module:Wikidata) but has similar code to the normal Wikidata. And we are less likely to break anything while learning how to create infoboxes.

The first thing we will check is if we can load the Wikidata Module: 

-- https://test.wikidata.org/wiki/Module:Wikidata/Test

local p = {}
local wd = require('Module:Wikidata')

function p.returnModule()
    return wd
end

return p

And in the console we inspect the p variable: 

> wd_obj = p.returnModule()
> table.foreach(wd_obj, print)

stringTable	function
sortclaims	function
_formatAndCat	function
getEntityFromId	function
formatAndCat	function
...

We see that we now have access to all the methods of Module:Wikidata.

Accessing frame object

edit

This is also a good time to introduce the frame object (mw:Extension:Scribunto/Lua_reference_manual#Frame_object). The frame object holds among other things arguments about the Wikidata item that is connected to the page (entity) and the language of the current page (lang). We can simulate this behaviour by pasting the following code at https://test.wikidata.org/wiki/Module:Wikidata/Test: 

-- https://test.wikidata.org/wiki/Module:Wikidata/Test
local p = {}
local wd = require('Module:Wikidata')

function p.getFinchLabel()
    local frame = mw.getCurrentFrame()
    frame.args.lang = 'en'
    frame.args.entity = 'Q167'
    finchLabel = wd.getLabel(frame)
    return finchLabel
end

return p

The code gets the current frame, which does not have any values in args and adds the lang and entity manually. Then we can run this command in the console: 

> print(p.getFinchLabel())
Common chaffinch

We see that we get the English label from https://test.wikidata.org/wiki/Q167 which currently reads "Common chaffinch".

Module:Infobox

edit

Similar to Module:Wikidata there is usually a Module:Infobox on each Wikipedia (and other projects). This module is also developed separately on each project, so you will have to adapt your code to work with the local module (or adapt the Module to be more similar to the others).

There are basically two implementations. The first receives the infobox structure from any infobox template (e.g. Template:Infobox book (Q5858283)), which calls Template:Infobox (Q5626735), which invokes Module:Infobox (Q13107716) (usually calling the infobox function within it). The template is then parsed and the headers, rows and other elements are styled accordingly.

The second implementation of Module:Infobox (Q13107716) also allows the infobox to be constructed from another Lua module. An example of this is fr:Module:Infobox/Tapis persan (Persian rug). The template fr:Modèle:Infobox Tapis persan invokes the following code: {{#invoke:Infobox|build|nom=Tapis persan}}. This calls Module:Infobox and the build function in that module. It only passes the name of the infobox, which the Module:Infobox uses to call the structure at fr:Modèle:Infobox Tapis persan. This return the structure to Module:Infobox and then constructs the layout for it.

Links to the different implementations can be found here:

Introduction to Wikidata infoboxes

edit

The tutorial will show how to create a simple infobox that will retrieve different data-types and their citations. The information will come from the item directly connected to that item and also connected items.

Best practices

edit
 
The 2012 and 2016 version of the Gout Infobox. The right version gives the reader a much better summary of the topic.

Infoboxes have a very prominent spot on the users screen and should therefore be carefully designed. This is especially true on small screens, where the infobox comes right below the headline. If an infobox is very long a user will have to scroll 2 or even more screen height until they reach the first paragraph of text. Therefore when rebuilding an infobox it is a good idea to check if any updates are needed.

Especially in the times before Wikidata Infoboxes became storages for all kinds of identifiers that usually offer little benefit to the reader. The figure to the right shows the transformation the Gout infobox has made between 2012 and 2016. It is of course necessary to discuss these changes with the respective community.

See also: Help:Designing infoboxes (Q8614811)

How traditional infoboxes work

edit

Infoboxes are templates that can be called from any other page using Wikitext templates (the double-curly brackets). If you look for example at the page about the book en:Capital in the Twenty-First Century you will see that this code is used to call the infobox: 

{{infobox book
| name         = Capital in the Twenty-First Century
| title_orig   = Le Capital au XXIe siècle
| translator   = [[Arthur Goldhammer]]
| image        = File:Capital in the Twenty-First Century (front cover).jpg
| caption      = <small>Hardcover edition</small>
| author       = [[Thomas Piketty]]
| language     = French
| subject      = [[Capitalism]], [[economic history]], [[economic inequality]]
| genre        = Non fiction
| publisher    = [[Éditions du Seuil]],<br />[[Harvard University Press|Belknap Press]]
| pub_date     = August 2013
| english_pub_date = April 15, 2014
| media_type   = Print ([[Hardcover|Hardback]])
| pages        = 696 pp.
| isbn =  978-0674430006
}}

We see that the curly brackets call the en:Template:Infobox book. The keys (e.g. |name) and values (Capital in the Twenty-First Century) are stored on the page itself. Only keys that are supported by the template can be used. When looking at the code of en:Template:Infobox book we see that it itself uses en:Template:Infobox which depends on en:Module:Infobox. This was the only way to fill inboxes with information before Wikidata.

How do Wikidata infoboxes work

edit

Wikidata-powered infoboxes are either a mix of data stored on a page and Wikidata information or even completely generated from Wikidata. That means that the Infobox-Book example from above could even look like this: 

{{infobox book}}

Or just provide a minimum amount of local data: 

{{infobox book
|language = French and others
}}

For traditional infoboxes that would result in an empty or near empty infobox. But for Wikidata infoboxes it is possible to store all the keys in the template definition and grab all the values from the connected Wikidata item.

How are Wikidata infoboxes made

edit

There are a number of ways how Wikidata infoboxes can be created. Some approaches work almost completely in Lua, while others retain most of the code that traditional infobox templates use. The following two sections will show how to build an infobox based on existing templates and one that is done in Lua.

Wikidata infoboxes using existing templates

edit
 
Wikidata infoboxes largely based on existing templates function like this: When a Wikipedia page loads, it will parse the infobox code and look for local data (and keywords opting in or out of Wikidata). Then (1) it will check the Template:Infobox for Wikidata invokations ({{#invoke:Wikidata|..}}) and check which values need to be retrieved. For each value (2) Module:Wikidata is called which calls the connected Wikidata item (3), which then sends the data back to the Wikipedia page (4).

Wikidata-powered Infoboxes can be made using the existing templates and adding calls to Module:Wikidata to certain rows. This is the simplest approach to retrieve information from Wikidata. There are of course limitations to what can be done using the Template language.

All that needs to be done for this approach is usually to add a {{#invoke:}}, which usually calls Module:Wikidata which would look like this: {{#invoke:Wikidata}}. Calling the Module alone will not do anything unless you call a specific function of the module, which can also take function-arguments. {{#invoke:Wikidata|getRawValue|argument1|argument2|argument3}}. Looking at this function in Module:Wikidata we can see how it works: 

p.getRawValue = function(frame)
	local propertyID = mw.text.trim(frame.args[1] or "")
	local input_parm = mw.text.trim(frame.args[2] or "")
	if input_parm == "FETCH_WIKIDATA" then
		local entity = mw.wikibase.getEntityObject()
		local claims
		if entity and entity.claims then claims = entity.claims[propertyID] end
		if claims then
			local result = entity:formatPropertyValues(propertyID, mw.wikibase.entity.claimRanks).value
		
			-- if number type: remove thousand separators, bounds and units
			if (claims[1] and claims[1].mainsnak.snaktype == "value" and claims[1].mainsnak.datavalue.type == "quantity") then
				result = mw.ustring.gsub(result, "(%d),(%d)", "%1%2")
				result = mw.ustring.gsub(result, "(%d)±.*", "%1")
			end
			return result
		else
			return ""
		end
	else
		return input_parm
	end
end

We see that the function takes one argument, the Frame object. The arguments we pass in with the {{#invoke:}} template are sent along inside the frame object and the first one can be accessed as frame.args[1]. The function then checks whether the Wikidata value is requested or not. If the value is anything different from FETCH_WIKIDATA the original value will be returned (that is the local value stored on the Wikipedia page). If the Wikidata value is requested the function fetches the claim from Wikidata.

In case you haven't read your target Module:Wikidata yet, this is the time to look at it (Module:Wikidata (Q12069631)) and if documentation is missing. Also remember to update the local documentation and discuss unclear things on the talk page.

Wikidata on demand infobox

edit

The most conservative approach are infoboxes that only get a Wikidata value if the article author uses a keyword to retrieve the value. The person infobox is an example of this kind of infobox. When we click on "Edit Source" and scroll down we can see how this works: 

<!-- Template:Infobox person/Wikidata -->

| label26    = Occupation
| class26    = role
| data26     = {{#invoke:Wikidata|getRawValue|P106|{{{occupation|FETCH_WIKIDATA}}}}}

We see that only if FETCH_WIKIDATA is written in the infobox, will the code fetch the Wikidata statement. 

<!-- A Wikipedia Page using the template -->

{{Infobox person/Wikidata
| occupation = FETCH_WIKIDATA
}}

Wikidata by default infobox

edit

Some infoboxes also choose to always display a Wikidata value when the field is empty. That usually means that a keyword is used to hide the field where no local data is present, but the Wikidata value is not desired. Using the same example as above the infobox could look like this: 

<!-- Template:Infobox person/Wikidata -->

| label26    = Occupation
| class26    = role
| data26     = {{#invoke:Wikidata|getRawValue|P106|{{{occupation|}}}}}

Now the following code on any Wikipedia-page will retrieve the Wikidata statement: 

<!-- A Wikipedia Page using the template -->

{{Infobox person/Wikidata
| occupation =
}}

Examples:

Currently, the only way to suppress a Wikidata value in a Wikidata by default infobox in a specific article is by entering an invisible custom value such as a non-breaking space: 

<!-- A Wikipedia Page using the template -->

{{Infobox person/Wikidata
| occupation = &nbsp;
}}

Lua-only infoboxes

edit

Lua also allows it to build infoboxes completely (just 1 line in the template to invoke the module). This has some advantages and disadvantages.

Advantages:

  • Easier to maintain because Lua has functions and other concepts to make it easier to deal with a lot of logic
  • Easier to read than Mediawiki templates (although that might be different for you)

Disadvantages:

  • Editors with only knowledge of templates are excluded from editing modules (But if you know templates well, modules are also very easy)

Basic example

edit

The most basic version of a Lua-only infobox only uses 1 line of Mediawiki template code that is required to register the template and invoke the Lua module:

<includeonly>{{#invoke:MyNewInfobox|my_new_infobox}}</includeonly>

The first word after "#invoke:" is the name of the template. So if you want to use this template on a Wikipedia page you would use:

{{MyNewInfobox}}
This is the start of a Wikipedia page.

The second word (after the "|") is the name of the main function that will be called in the Module. In the module you need at least:

local p = {}

function p.my_new_infobox(frame)
    return "This is message from Lua"
end

return p

If you now would look at the Wikipedia page that uses the template it would simply show "This is message from Lua". All we need to do now is return a more useful string than this example. We will do that in the following section.

Here is what we have so far on test.wikidata:

Styling in Lua

edit

Now that we know how to set up a basic Lua module that returns a text we should learn how we can create a stylish box to contain all the information we want to display on the page. You can reuse the Mediawiki-Template and -Module if you want to. For the example I will create a new one:

<includeonly>{{#invoke:FancyInfobox|fancy_infobox}}</includeonly>

Just like the previous example you can use the template on any page using:

{{FancyInfobox}}
This is some text below the fancy infobox.

The secret to creating HTML from Lua is the "mw.html" Library. To create a "<div>" you would use:

mw.html.create('div')

Let's try to build a simple HTML table using Lua, so we get some practice. Remember that our main Lua function has to return the string we are building so that it appears on the page.

local p = {}

function p.fancy_infobox(frame)
	local fancy_table = mw.html.create('table')
		:tag('tr')
			:tag('th')
				:wikitext('This is the table header')
	    	:done()
		:done()
		:tag('tr')
			:tag('td')
				:wikitext('This is a table body cell')
			:done()
		:done()
	
	return tostring(fancy_table)
end

return p

The result will be a little disappointing because the default styling of tables on Mediawiki is without lines between the table cells. This is a good opportunity to learn how to use CSS to style our table:

local p = {}

function p.fancy_infobox(frame)
	local fancy_table = mw.html.create('table')
	    :css({
				['background-color'] = '#94ff94',
				padding = '10px',
		})
		:tag('tr')
			:tag('th')
			:css({
				color = 'white',
				['font-size'] = '300%',
				['background-color'] = '#f56a3d',
				padding = '10px',
			})
				:wikitext('This is the table header')
	    	:done()
		:done()
		:tag('tr')
			:tag('td')
			:css({
				border = '5px dashed blue',
				['font-weight'] = 900,
				['text-align'] = 'right',
				['background-color'] = 'white',
				padding = '10px',
			})
				:wikitext('This is a table body cell')
			:done()
		:done()
	
	return tostring(fancy_table)
end

return p

The resulting table does not look very encyclopedic but it will teach you a lot about CSS styles. Take some time to play around with the styling before proceeding.

Here is what we have so far on test.wikidata:

Fetching information from Wikidata

edit

The only thing left to do is fetching information from Wikidata and allowing users to add additional data if it is not possible to store the data on Wikidata. This requires that we learn some of the Lua-API that is used to fetch information form Wikibase. You don't need to know everything right away, but this reference is a good place to look for specific functions: https://www.mediawiki.org/wiki/Extension:Wikibase_Client/Lua

Here is what we have so far on test.wikidata:

The template stays the same: 

<includeonly>{{#invoke:SimpleWikidataInfobox|simple_wikidata_infobox}}</includeonly>

Also the invocation on the page: 

{{SimpleWikidataInfobox}}

In the module we add code that fetches data from the connected Wikidata item using the Lua-Wikidata-API. The following code fetches the data of https://test.wikidata.org/wiki/Q175163, which is an item on test.wikidata.org. The item is set manually, because the page on test.wikidata is not connected with the item by default. Fetching an item manually can be done using: mw.wikibase.getEntity("Q175163"). 

local p = {}

function p.simple_wikidata_infobox(frame)
	local item = mw.wikibase.getEntity("Q175163")
	
	local fancy_table = mw.html.create('table')
	    :css({
				['background-color'] = '#eee',
				border = '1px solid black',
		})
		:tag('tr')
			:tag('th')
			:attr('colspan', 2)
			:css({
				['text-align'] = 'center',
				['font-weight'] = 900,
				padding = '10px',
				border = '1px solid black',
			})
				:wikitext(item:getLabel())
	    	:done()
		:done()
		:tag('tr')
			:tag('td')
				:wikitext(item:formatStatements("P174").label)
			:done()
			:tag('td')
				:wikitext(item:getBestStatements('P174')[1].mainsnak.datavalue.value)
			:done()
		:done()
	
	return tostring(fancy_table)
end

return p

What properties to use?

edit

There are several lists at Wikidata:List of properties that can help you find applicable properties.

Some WikiProjects provide more targeted lists of properties (Category:Properties list in a WikiProject) and sometimes even mappings for Wikipedia infoboxes to Wikidata properties.

Sample: WikiProject Movies with its list of properties and Wikipedia infobox mapping.

To assess what properties are currently in use for a specific field, Related Properties (Q25337203) can be useful.

Sample: for a list of properties in use on items with instance of (P31) = lighthouse (Q39715), try claim[31:39715] on related_properties.php. Note that this may be slow and/or time-out.

Resources

edit

Documentation

edit

Other Tutorials

edit

Examples

edit

Many examples can be found in this category:

For specific fields

Presentations

edit
Retrieved from "https://www.wikidata.org/w/index.php?title=Wikidata:Infobox_Tutorial&oldid=2283520121"