Wikipedia:Requests for comment/Deployment of Vector (2022)/More about Vector (2022)
Currently, the Vector 2022 skin is the second most-popular non-default skin on English Wikipedia. It is also in use as the default across over 20 community-led projects accounting for more than 1 billion pageviews per month, such as French Wikipedia, where it has been the default since 2020.
The objective for the Vector 2022 skin is to make the interface more welcoming and comfortable for readers and useful for advanced users on desktop. To do this, the skin introduces changes to the navigation and layout of the site, adds persistent elements such as a sticky header and Table of Contents to make frequently used actions easier to access, and makes some changes to the overall styling of the page. The analysis of the data collected concluded that these changes improve readability and usability, and save time currently spent in scrolling, searching, and navigating – all of which can be interpreted to create an easier reading experience. The new skin does not remove any functionality currently available on the Vector skin.
Graphs are unavailable due to technical issues. There is more info on Phabricator and on MediaWiki.org. |
Below, you'll find more details on how the skin was built over the past three years, what has been changed, and why these changes are considered to be improvements based on the completed research and data collected.
About
[edit][Why is a change necessary] The current skin, Vector, has been in use since 2010. When it was developed, it reflected the needs of the readers and editors of the Wikimedia sites in that year. (See the Wikimedia Usability Initiative wiki for more information.) Since then, vast new audiences have begun using the Internet and Wikimedia projects. Research done with these audiences showed that the current default skin doesn't meet their needs. The Vector 2022 skin aims to change the interface in ways which include the needs of all of the current audiences – both those who have been using the projects for a long time, as well as those who have joined more recently, or have yet to join.
[Improvement in usability] The objective for the skin is to make the interface more welcoming and comfortable for readers and useful for advanced users on desktop. It builds upon the existing Vector skin to introduce a series of feature changes that aim to improve problems new and existing readers and editors were having with the old skin. It also draws inspiration from previous user requests, the Community Wishlist Surveys, and existing gadgets and scripts across different projects.
[Technical improvement] The renewed product focus allowed to bring Vector in line with the development standards and improve all legacy skins. PHP code in Wikimedia deployed skins has been reduced by 75%. The project has also focused on improving gadget support across all skins by making sure skins generate consistent HTML and have improved APIs (for example handling icons in menus).
[Try it out]
- To preview what the skin looks like, go to this article. When you close the tab or go to any other page, you'll be back to your default skin. If you test it in incognito mode, bear in mind that the settings are slightly different for logged-out users.
- You may also switch to the new skin. Go to the appearance tab in your preferences and select Vector 2022 from the list of skins there.
- There's also an option to create a bookmark in your browser with the following URL:
javascript:new mw.Api().saveOption( 'skin', mw.user.options.get( 'skin' ) === 'vector' ? 'vector-2022' : 'vector' ).then( () => location.reload() )
. Clicking the bookmark will switch you to the other version of Vector and reload the page (provided you are logged in).
[Effectiveness] The results and analysis of A/B testing and qualitative testing concluded that these changes make it easier to read and learn, navigate within the page, search, switch between languages, use page and user tools, and more, without negative effects to pageviews, account creation, or edit rates. The team has been working on the new skin for the past three years, ensuring that every change is tested and proven to work. For more details, look at the Findings and results.
Process for developing the new skin
[edit]The new Vector 2022 skin has been in development for the past three years, and tested in collaboration with different audiences and communities on English Wikipedia and other wikis.
- Problem identification research with both readers and editors – During this phase, in 2019, the Web team studied the way people used the site and identified the largest usability issues as well as issues to exploring the site further, becoming more engaged with reading or editing. The team did this by interviewing readers and editors across multiple countries and locations. They also surveyed existing gadgets and use scripts across different wikis and communities to identify what types of changes and alterations to Vector legacy (2010) were made to improve readability and usability (See the links: Research and design: Phase 1, Research and design: Phase 2.)
- Prototype development and testing – Over the course of 2019–2022, the team built out the ideas for individual features and began showing potential solutions to our audiences. These ideas were built in the forms of usable prototypes. Each feature was tested with readers and editors through interviews and wider rounds of prototype testing. Generally, for testing with editors the team used Central Notice banners across multiple languages and Wikimedia projects, including English Wikipedia, so that the widest audience possible could be involved. Each prototype was tested by approximately 200 editors on average. (Example)
- Refining and building – During 2020–2022, the feedback from the prototype testing was taken, and the prototype was refined or changed based on what needs were identified in the prototype testing from both readers and editors. In some cases, the team asked for additional feedback once changes and iterations to ensure they were making the right decisions.
- A/B testing and other quantitative testing on pilot (early adopter) wikis – For each feature, the team performed quantitative tests for whether the features worked as expected based on the previously defined criteria of success. For example, the sticky header was designed to decrease scrolling to the top of the page. The team gave the sticky header to 50% of users and compared them to the other 50% for two weeks. After two weeks the results were compared and it turned out that people who had the sticky header were indeed scrolling less to the top of the page in order to select any of the tools available there. If the team received negative results from the test, they focused on identifying the reasons for these results across different audiences and made changes to the feature which addressed them. Then, they would test the feature again. This was the "beta" phase. During this phase, the team also monitored usage across all wikis, including English Wikipedia, where many account holders have been already using the new skin.
- Finally, now, the team would like to deploy Vector (2022) on more wikis and continue monitoring the way people are using it so that they can flag any issues.
New features
[edit]The skin includes changes to the layout of the site, location and prominence of some features, the overall readability, and addition of sticky features. Among the best-received by the communities, there are the new Table of Contents, sticky header, and the search widget. No existing features or tools were removed as a result of the new skin.
- The table of contents makes it easier to explore pages. It helps identify the current section and remains persistent while scrolling down the page, making it easier to navigate to various sections. On talk pages, it provides context on each discussion by adding the number of comments in each discussion.
- Through more prominently placed language-switching tools, multilingual readers and editors can more easily find their preferred language and switch between languages as needed.
- The content on the main namespace will now have a maximum line width. Research has shown that limiting the width of longform text leads to a more comfortable reading experience, and better retention of the content itself.
- The new user menu collects user links in a single menu. Research showed that collecting these links makes it easier for new editors to understand our overall navigation and distinguish between page navigation and personal tools.
- The search widget is placed in a more prominent location. It includes additional information about each query by adding images and descriptions, making it easier to find articles on Wikipedia.
- A new collapsible sidebar allows for collapsing and opening the main menu of the page, depending on the context and task.
- The sticky header allows for commonly used tools such as access to the history and talk page, search, and user tools to be available while scrolling down the page.
- The visual refinements changed the look and feel of the interface to make sure that all tools, navigation, and content look cohesive and easily recognizable as Wikipedia.
Findings and results
[edit]The results of testing and subsequent analysis concluded that these changes result in improved usage of the skin, and save time currently spent in scrolling, searching, and navigating – all of which can be interpreted to create an easier and more comfortable reading experience. There are no negative effects to pageviews, account creation, or edit rates. Please visit the repository page for the full versions of research and reports.
Research | Usability | Usage | Sentiment | Health Metrics and Overall Performance | |
---|---|---|---|---|---|
Why | This helped identify what needed to be fixed | This helped verify that the suggested solutions were correct | This helped verify that the solutions performed better than their previous iterations | This helped estimate the types of opinions would be heard about the skin immediately after deployment | This helped estimate the long-term effects and benefits of using the new skin at scale |
What | Research across multiple languages and wikis, including English Wikipedia, was performed to determine the main issues with the old Vector skin | User testing, prototype testing, and interviews were performed for individual features as well as the overall skin | A/B testing and subsequent monitoring was performed to validate the results of qualitative testing on wikis, to show how features and the skin were used in reality | Surveys were taken to estimate the subjective reactions to the new skin at the moment of deployment | In addition to testing individual features, health metrics were identified and monitored to ensure the usage of the new skin was on-par or better than the old Vector skin |
Type | Qualitative | Qualitative | Quantitative | Survey | Qualitative |
Results |
|
Header | Long-term affinity surveys | Opt-in rates | |
In prototype testing with editors, the large majority of users reported positive to neutral experiences with the new header and logo configuration. 8% of editors tested reported negative experiences with the new logo. | There was a survey on the general sentiment towards the site in its current (August 2022) form. A few months after deployment, this survey will be run again to identify the ways people will feel towards the new experience. | On the initial partner communities, 96% of all logged-in users and 88% of active editors decided to keep the changes once the default was changed (the target was 60%). | |||
Collapsible sidebar | First look sentiment surveys | Usage | |||
|
|
There was a survey identifying the sentiment of people upon their first interaction with the new skin.
These results are expected and on-par with similar design changes made by other websites. They will be used to prepare communities and internal communications for some amount of negative feedback upon deployment. |
| ||
Limited content width | Pageviews | ||||
|
Limiting the content width did not show any significant effects of the health metrics: pageviews, edit count, and account creation across the pilot wikis. |
| |||
Search | Edit rates | ||||
In user testing performed on usertesting.com, all participants were able to find the result they were looking for using the new search. |
|
Wikis with the Vector 2022 skin as default did not experience any decreases in edit rates attributed to the Vector 2022 skin. | |||
Language switching | Account creation | ||||
|
|
| |||
Sticky header | |||||
|
| ||||
User menu | |||||
| |||||
Table of contents | |||||
|
| ||||
References | Features |
Tasks which will be completed before the deployment
[edit]- [DONE] Table of Contents collapsing and narrow screens behavior – This work is now completed. The table of contents is collapsible, and can be accessed from both the title of the page as well as from the sticky header.
- [DONE] Visual refinements – The team is currently wrapping up the core parts of the visual refinements work. Please see the Qualitative Testing section on the project page dedicated to this part of the project for a full list of the changes Web plans on making.
- [In Progress] Coordinates – Web is continuing to explore different solutions for coordinate alignment, including potentially adding coordinates directly into the styles of the skin in the future.
Support for Vector 2022 after deployment
[edit]We plan to continue work on the desktop experience after this change is made, and into the future. Here's a snapshot of the projects we will be working on within the next six months.
- October–November 2022: Once the skin is deployed, we will be focusing on the page tools feature, which will allow easier access to page tools and further customization of our menus and tools overall. We encourage you to check out the prototype for this work.
- March 2023: Navigation improvements – we are currently exploring ways to make it easier and clearer to understand how wiks and the content creation process works for readers, and specifically surfacing the fact that wikis are collaborative spaces anyone can get involved in. Our goal is to enable people to read content more critically and be able to recognize what is trustworthy, as well as to make it easier for those who might be interested to know that they can contribute and to begin their contributions journey. We will be collaborating on this work with the Growth and Editing teams (see: Core Experiences).