This task will be filled out later, but getting it as a place-holder for the on-going internal conversations.
Description
Details
Subject | Repo | Branch | Lines +/- | |
---|---|---|---|---|
Update point of contact for Thumbor | operations/puppet | production | +1 -1 |
Event Timeline
Will:
- need to update python
- old hardware
Can we define a product owner?
- not clear on if any product team has owned this in the past
- originally owned by the multimedia team
- we plan to ask for a multimedia team for 2022 fiscal
Greg to have conversation with SRE on what a potential timeline is
From my perspective, the largest threats to the continued stability of Thumbor are:
1 and 2 are relatively simple, and it makes sense to fix them at the same time (preferably with T233196: Migrate thumbor to Kubernetes ). It's pretty much only blocked on resourcing. Some work toward this goal was done in T267327. Most of the tasks in Thumbor are upstream library issues, and many are fixed in the versions of the libraries packaged in Buster or Bullseye.
The Python 3 upgrade is trickier. Upstream released an alpha version with Python 3 support (Thumbor 7) in February 2020, but has had very little activity since (none in the last 6 months and no releases of any kind in the last 18 months). T267327 got the Wikimedia customizations working with Thumbor 7, at least well enough to pass the Wikimedia test suite. From my perspective, most of the work remaining there is testing and dealing with whatever upstream may decide to do.
To be clear, thumbor is just one part of the problem, the entire file management stack needs a dedicated owner. Other people have just been covering for issues when it gets really bad. File metadata had been causing significant problems until Tim and Amir moved it into external storage. MediaWiki had been uselessly creating thumbnails of all images that Thumbor served, until Tim disabled it after realizing this was happening during Shellbox work. I ended up doing all of the file-related Shellbox porting in MediaWiki even though it's really not "ServiceOps" responsibility.
There's legacy cruft like T290759: Undeploy VipsScaler from Wikimedia wikis, which is both unused and used at the same time. T40010: RFC: Re-evaluate librsvg as SVG renderer on Wikimedia wikis is a very sad story of people putting in a lot of work to the point of building a new SVG renderer that is really just blocked on someone packaging stuff up and getting it installed (and then dealing with ensuing bugs and updates, etc.).
I believe the bug counts listed in the Google Doc are underselling the scope of the work, because many issues are scattered across MediaWiki-File-management and Wikimedia-SVG-rendering too.
For emphasis, let me add that the effects of the lack of ownership there is visible to the community / end users. Speaking from the perspective of a community member, having a Multimedia team again would be good but there also needs to be somewhere the buck stops on the entirety of the file management stack. That can certainly be a "Multimedia team", but then the scope needs to be very clear so that the necessary resources are allocated. Because it's not obvious just from the name that things like Thumbor, or file metadata storage, or rearchitecting file operations to not require a database lock, or… fall within the scope of a "multimedia" team.
This stuff is fundamentals, and can't be left to random acts of kindness by people on other teams.
(removing @EChetty as assignee, was that a mis-click assignment? I can't find your name on https://office.wikimedia.org/wiki/Contact_list)
This is also blocking the work on adding support for chemical markup files: T18491: Support for Chemical Markup Language
7.0.0 was upgraded from Alpha to Beta in November 2021, and was upgraded to a full release about a week ago.
March 11, 2022 Update
- As per internal discussions, the PET team will take ownership of Thumbor until we resolve the larger media support question with Product Ownership falling Platform PM team.
We have outlined the following items to be worked on:
- Upgrade Thumbor to Buster: https://phabricator.wikimedia.org/T216815
- Migrate thumbor to Kubernetes: https://phabricator.wikimedia.org/T233196
- Upgrade thumbor to Thumbor 7 and python3: https://phabricator.wikimedia.org/T252719
We are currently aligning resources to pick up this work and will provide update on when we expect to start.
Change 770910 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):
[operations/puppet@production] Update point of contact for Thumbor
Change 770910 merged by Muehlenhoff:
[operations/puppet@production] Update point of contact for Thumbor
April 28, 2022 Update
- Migrating test suite from Nosetests to pytest
- Removed dependency on defunct Thumbor plugin lib
- Working on helm chart for k8s
For everyone's info, currently no Code-Stewardship-Reviews are taking place as there is no clear path forward and as this is not prioritized work.
(Entirely personal opinion: I also assume lack of decision authority due to WMF not having a CTO currently. However, discussing this is off-topic for this task.)