Page MenuHomePhabricator
Create Task
Maniphest T285345

Python3 scap breaks mediawiki
Closed, ResolvedPublic
Actions

Description

Recently reported on beta:

In T285118#7170467, @Majavah wrote:

my best guess is caused by a scap upgrade that's on beta but not on production.

In T285125#7165578, @LarsWirzenius wrote:

We've been porting Scap to Python3 latetly, and this _might_ be Unicode string confusion problem somewhere. I don't know that it is, though, nor can I see where it might be.

If this guess is correct then we need to revert Scap in beta and/or fix ASAP. Right now we're bypassing even more QA steps than we normally do for this weeks' deployment T281152.

Details

SubjectRepoBranchLines +/-
cdblib.py: Python 3 fixesmediawiki/tools/scapmaster+25 -3
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolveddancyT279628 Porting scap to Python 3
 ResolveddancyT285345 Python3 scap breaks mediawiki

Event Timeline

Krinkle created this task.Jun 22 2021, 7:01 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 22 2021, 7:01 PM
Krinkle updated the task description. (Show Details)Jun 22 2021, 7:02 PM
Krinkle updated the task description. (Show Details)
Restricted Application added a subscriber: Stang. · View Herald TranscriptJun 22 2021, 7:03 PM
Krinkle updated the task description. (Show Details)Jun 22 2021, 7:03 PM
hashar added a project: Beta-Cluster-Infrastructure.Jun 23 2021, 9:41 AM
hashar subscribed.Jun 23 2021, 9:46 AM

Looks like most of the investigation so far is T285125#7163052 and following comments.

thiemowmde added a project: WMDE-TechWish-Sprint-2021-06-23.Jun 24 2021, 8:12 AM
thiemowmde moved this task from Sprint Backlog to Watching (Stalled / Epic) on the WMDE-TechWish-Sprint-2021-06-23 board.
thiemowmde subscribed.
hashar moved this task from INBOX to Seen on the Release-Engineering-Team board.Jun 25 2021, 11:58 AM
hashar edited projects, added Release-Engineering-Team (Seen); removed Release-Engineering-Team.
thcipriani renamed this task from Localisation cache is broken in beta to Python3 scap breaks localisation cache.Jun 28 2021, 9:10 PM
thcipriani triaged this task as Medium priority.
thcipriani edited projects, added Release-Engineering-Team (Next); removed Release-Engineering-Team (Seen), Beta-Cluster-Infrastructure.
thcipriani updated the task description. (Show Details)
thcipriani added a parent task: T279628: Porting scap to Python 3.
taavi merged a task: T285283: [beta cluster-regression] Several controls display MediaWiki labels not text.Jun 29 2021, 9:31 AM
taavi added subscribers: Etonkovidova, Tgr, • mewoph, kostajh.
Jdforrester-WMF subscribed.Jun 29 2021, 10:37 AM

Last merged patch in scap was rMSCAd6663929dbba: Revert some uses of yaml.safe_load that break "scap deploy", landing 12 days ago. Presumably that's not the direct cause, but more likely something like rMSCAc49e432818a8: fix: Python 3 str/bytes translations?

Jdforrester-WMF added a comment.Jun 29 2021, 1:57 PM

OK, it looks like the issue might be that prod machines generally seem to have Python 3.7, but the Beta Cluster machines have Python 3.5. Joy.

taavi added a comment.Jun 29 2021, 2:59 PM
In T285345#7183782, @Jdforrester-WMF wrote:

OK, it looks like the issue might be that prod machines generally seem to have Python 3.7, but the Beta Cluster machines have Python 3.5. Joy.

Which machines? Deployment hosts (blocked on T277078) and the maintenance script runner are on Stretch/Python 3.5, but app servers (including mw11 in T285125#7180648) have 3.7 (as they're on Buster)

Jdforrester-WMF added a comment.Jun 29 2021, 3:00 PM
In T285345#7183925, @Majavah wrote:
In T285345#7183782, @Jdforrester-WMF wrote:

OK, it looks like the issue might be that prod machines generally seem to have Python 3.7, but the Beta Cluster machines have Python 3.5. Joy.

Which machines? Deployment hosts (blocked on T277078) and the maintenance script runner are on Stretch/Python 3.5, but app servers (including mw11 in T285125#7180648) have 3.7 (as they're on Buster)

The deployment hosts are the ones that count for scap, sadly (well, both do, but…).

dancy subscribed.Jun 29 2021, 3:22 PM
thcipriani renamed this task from Python3 scap breaks localisation cache to Python3 scap breaks mediawiki.Jun 29 2021, 5:20 PM
thcipriani merged a task: T285125: image sizes not displayed on beta.
thcipriani added subscribers: AlexisJazz, • Jpita, • dpifke and 4 others.
brennen subscribed.Jun 29 2021, 7:03 PM
Bas_dehaan subscribed.Jun 29 2021, 7:09 PM
dduvall claimed this task.Jun 29 2021, 7:18 PM
dduvall edited projects, added Release-Engineering-Team (Doing); removed Release-Engineering-Team (Next).
gerritbot added a comment.Jun 29 2021, 8:49 PM

Change 702233 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[mediawiki/tools/scap@master] cdblib.py: Python 3 fixes

https://gerrit.wikimedia.org/r/702233

gerritbot added a project: Patch-For-Review.Jun 29 2021, 8:49 PM
dancy added a comment.Jun 29 2021, 9:17 PM

Scap has been reverted on all beta notes and CDB files have been regenerated and redeployed.

gerritbot added a comment.Jun 29 2021, 9:28 PM

Change 702233 merged by jenkins-bot:

[mediawiki/tools/scap@master] cdblib.py: Python 3 fixes

https://gerrit.wikimedia.org/r/702233

Maintenance_bot removed a project: Patch-For-Review.Jun 29 2021, 10:10 PM
hashar added a comment.Jun 30 2021, 7:54 AM

@dduvall that is an excellent patch and I definitely love the round trip write/read test :-]

Jdforrester-WMF closed this task as Resolved.Jun 30 2021, 8:54 AM
Jdforrester-WMF reassigned this task from dduvall to dancy.
Jdforrester-WMF added a subscriber: dduvall.
dduvall added a comment.Jun 30 2021, 2:57 PM
In T285345#7186175, @hashar wrote:

@dduvall that is an excellent patch and I definitely love the round trip write/read test :-]

@dancy patched the implementation, I wrote the test. Extreeeeeeeme! :)

Stang unsubscribed.Nov 1 2021, 8:40 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits