Page MenuHomePhabricator
Create Task
Maniphest T307616

Move SQL building code from Database class to SQLPlatform
Closed, ResolvedPublic
Actions

Details

Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

Ladsgroup created this task.May 4 2022, 6:46 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 4 2022, 6:46 PM
Ladsgroup triaged this task as Medium priority.May 4 2022, 6:46 PM
Ladsgroup moved this task from Triage to In progress on the DBA board.
gerritbot added a comment.May 4 2022, 7:24 PM

Change 789251 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] rdbms: Move out more functions from Database to SQLPlatform

https://gerrit.wikimedia.org/r/789251

gerritbot added a project: Patch-For-Review.May 4 2022, 7:24 PM
gerritbot added a comment.May 4 2022, 10:41 PM

Change 789317 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/extensions/CheckUser@master] Mock db platform

https://gerrit.wikimedia.org/r/789317

Krinkle edited projects, added Performance-Team; removed Performance-Team (Radar).May 11 2022, 4:52 PM
Krinkle moved this task from Inbox, needs triage to Backlog: Maintenance, non-prioritized on the Performance-Team board.
Krinkle moved this task from Untriaged to Rdbms library on the MediaWiki-libs-Rdbms board.
gerritbot added a comment.May 11 2022, 5:21 PM

Change 789317 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] tests: Mock db platform

https://gerrit.wikimedia.org/r/789317

gerritbot added a comment.May 11 2022, 5:21 PM

Change 789251 merged by jenkins-bot:

[mediawiki/core@master] rdbms: Move out more functions from Database to SQLPlatform

https://gerrit.wikimedia.org/r/789251

gerritbot added a comment.May 11 2022, 6:33 PM

Change 791065 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] rdbms: Prepare for moving Database::selectSQLText to SQLPlatform

https://gerrit.wikimedia.org/r/791065

ReleaseTaggerBot added a project: MW-1.39-notes (1.39.0-wmf.12; 2022-05-16).May 11 2022, 11:30 PM
gerritbot added a comment.May 17 2022, 2:20 AM

Change 791065 merged by jenkins-bot:

[mediawiki/core@master] rdbms: Prepare for moving Database::selectSQLText to SQLPlatform

https://gerrit.wikimedia.org/r/791065

Maintenance_bot removed a project: Patch-For-Review.May 17 2022, 2:30 AM
ReleaseTaggerBot edited projects, added MW-1.39-notes (1.39.0-wmf.13; 2022-05-23); removed MW-1.39-notes (1.39.0-wmf.12; 2022-05-16).May 17 2022, 8:00 AM
gerritbot added a comment.May 17 2022, 6:24 PM

Change 792689 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] rdbms: Move four more functions from Database to SQLPlatform

https://gerrit.wikimedia.org/r/792689

gerritbot added a project: Patch-For-Review.May 17 2022, 6:24 PM

Change 792690 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] rdbms: Move handling index and table aliases to SQLPlatform

https://gerrit.wikimedia.org/r/792690

gerritbot added a comment.May 17 2022, 7:29 PM

Change 792704 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] [WIP] rdbms: Move selectSQLText to SQLPlatform

https://gerrit.wikimedia.org/r/792704

gerritbot added a comment.May 22 2022, 6:06 PM

Change 792689 merged by jenkins-bot:

[mediawiki/core@master] rdbms: Move four more functions from Database to SQLPlatform

https://gerrit.wikimedia.org/r/792689

gerritbot added a comment.May 25 2022, 10:29 PM

Change 792690 merged by jenkins-bot:

[mediawiki/core@master] rdbms: Move handling index and table aliases to SQLPlatform

https://gerrit.wikimedia.org/r/792690

ReleaseTaggerBot edited projects, added MW-1.39-notes (1.39.0-wmf.14; 2022-05-30); removed MW-1.39-notes (1.39.0-wmf.13; 2022-05-23).May 25 2022, 11:00 PM
gerritbot added a comment.Jun 2 2022, 7:54 PM

Change 792704 merged by jenkins-bot:

[mediawiki/core@master] rdbms: Move selectSQLText to SQLPlatform

https://gerrit.wikimedia.org/r/792704

Maintenance_bot removed a project: Patch-For-Review.Jun 2 2022, 8:30 PM
gerritbot added a comment.Jun 3 2022, 12:25 PM

Change 802763 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] [WIP] rdbms: Move assertion logic in sql building to sql platform

https://gerrit.wikimedia.org/r/802763

gerritbot added a project: Patch-For-Review.Jun 3 2022, 12:25 PM
Nikerabbit added a subtask: T310141: Some SemanticMediaWiki queries fail due to duplicated table prefix.Jun 8 2022, 11:57 AM
Krinkle mentioned this in T310214: Wikimedia\Rdbms\DBQueryError: Error 1146: Table 'enwikinews.`categorylinks`' doesn't exist.Jun 8 2022, 7:24 PM
Krinkle subscribed.Jun 8 2022, 10:07 PM

@Ladsgroup Per T310214, we could probably do with a test that ensures the double escaping issue doesn't happen again. If we already have a case of e.g. select( [ 'x' => tableName('x') ], .. ), then maybe the issue is expanding it to include coverage to sqlite/mysql/postgres for the jobs that enable those, either with sql text based assertions or perhaps a simple end-to-end test that selects something simple from a table.

Ladsgroup added a comment.Jun 8 2022, 10:15 PM

Sure, please remind me when I'm back. I want to focus on unblocking train atm (I have a very little time rn)

Universal_Omega subscribed.Jun 10 2022, 8:02 PM
gerritbot added a comment.Jun 27 2022, 9:59 AM

Change 808844 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] rdbms: Move methods delegated to platform to one central place

https://gerrit.wikimedia.org/r/808844

gerritbot added a comment.Jun 27 2022, 10:19 AM

Change 808852 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] rdbms: Migrate buildGroupConcatField and buildSelectSubquery to platform

https://gerrit.wikimedia.org/r/808852

gerritbot added a comment.Jun 28 2022, 2:25 AM

Change 808844 merged by jenkins-bot:

[mediawiki/core@master] rdbms: Move methods delegated to platform to one central place

https://gerrit.wikimedia.org/r/808844

gerritbot added a comment.Jun 28 2022, 2:28 AM

Change 808852 merged by jenkins-bot:

[mediawiki/core@master] rdbms: Migrate buildGroupConcatField and buildSelectSubquery to platform

https://gerrit.wikimedia.org/r/808852

gerritbot added a comment.Jun 28 2022, 6:34 AM

Change 809054 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] rdbms: Migrate insert sql building code to sql platform

https://gerrit.wikimedia.org/r/809054

ReleaseTaggerBot edited projects, added MW-1.39-notes (1.39.0-wmf.21; 2022-07-18); removed MW-1.39-notes (1.39.0-wmf.14; 2022-05-30).Jul 11 2022, 6:32 PM
gerritbot added a comment.Jul 22 2022, 10:31 AM

Change 809054 merged by jenkins-bot:

[mediawiki/core@master] rdbms: Migrate insert sql building code to sql platform

https://gerrit.wikimedia.org/r/809054

ReleaseTaggerBot edited projects, added MW-1.39-notes (1.39.0-wmf.22; 2022-07-25); removed MW-1.39-notes (1.39.0-wmf.21; 2022-07-18).Jul 22 2022, 11:00 AM
gerritbot added a comment.Jul 22 2022, 1:38 PM

Change 816169 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] rdbms: Migrate delete and update sql building to SQLPlatform

https://gerrit.wikimedia.org/r/816169

gerritbot added a comment.Jul 22 2022, 4:58 PM

Change 816169 merged by jenkins-bot:

[mediawiki/core@master] rdbms: Migrate delete and update sql building to SQLPlatform

https://gerrit.wikimedia.org/r/816169

gerritbot added a comment.Jul 27 2022, 1:50 PM

Change 817782 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] rdbms: Migrate dropTable sql building to SQLPlatform

https://gerrit.wikimedia.org/r/817782

gerritbot added a comment.Jul 27 2022, 2:47 PM

Change 817782 merged by jenkins-bot:

[mediawiki/core@master] rdbms: Migrate dropTable sql building to SQLPlatform

https://gerrit.wikimedia.org/r/817782

ReleaseTaggerBot edited projects, added MW-1.39-notes (1.39.0-wmf.23; 2022-08-01); removed MW-1.39-notes (1.39.0-wmf.22; 2022-07-25).Jul 27 2022, 3:00 PM
gerritbot added a comment.Jul 27 2022, 3:50 PM

Change 817807 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] rdbms: Migrate SQL anlysis code pieces to SQLPlatform

https://gerrit.wikimedia.org/r/817807

gerritbot added a comment.Jul 28 2022, 4:54 PM

Change 802763 abandoned by Ladsgroup:

[mediawiki/core@master] [WIP] rdbms: Move assertion logic in sql building to sql platform

Reason:

done in other patches, there is no hope to bring this back without practically rewriting it.

https://gerrit.wikimedia.org/r/802763

gerritbot added a comment.Jul 29 2022, 8:59 AM

Change 817807 merged by jenkins-bot:

[mediawiki/core@master] rdbms: Migrate SQL analysis code pieces to SQLPlatform

https://gerrit.wikimedia.org/r/817807

gerritbot added a comment.Jul 29 2022, 1:28 PM

Change 818459 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] rdbms: Migrate several more Database::do* to SqlPlatform

https://gerrit.wikimedia.org/r/818459

gerritbot added a comment.Jul 29 2022, 1:36 PM

Change 818463 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] rdbms: Migrate delegated protected functions of Database and drop them

https://gerrit.wikimedia.org/r/818463

gerritbot added a comment.Jul 29 2022, 2:09 PM

Change 818473 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] rdbms: Bump the minimum required version of PG to 9.5

https://gerrit.wikimedia.org/r/818473

gerritbot added a comment.Jul 29 2022, 3:02 PM

Change 818459 merged by jenkins-bot:

[mediawiki/core@master] rdbms: Migrate several more Database::do* to SqlPlatform

https://gerrit.wikimedia.org/r/818459

gerritbot added a comment.Jul 29 2022, 4:00 PM

Change 818463 merged by jenkins-bot:

[mediawiki/core@master] rdbms: Migrate delegated protected functions of Database and drop them

https://gerrit.wikimedia.org/r/818463

gerritbot added a comment.Aug 1 2022, 6:08 PM

Change 818473 merged by jenkins-bot:

[mediawiki/core@master] rdbms: Bump the minimum required version of PG to 9.5

https://gerrit.wikimedia.org/r/818473

gerritbot added a comment.Aug 1 2022, 8:29 PM

Change 819157 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] rdbms: Migrate SchemaVar and replaceVars from Database to SQLPlatform

https://gerrit.wikimedia.org/r/819157

gerritbot added a comment.Aug 2 2022, 8:14 PM

Change 819157 merged by jenkins-bot:

[mediawiki/core@master] rdbms: Migrate SchemaVar and replaceVars from Database to SQLPlatform

https://gerrit.wikimedia.org/r/819157

Ladsgroup closed this task as Resolved.EditedAug 2 2022, 8:56 PM

Now there is no large chunk of code left in Database class that could be moved to SQLPlatform.

It is much cleaner and the complexity score of the class has fallen to ~600 out of ~1000 and it's no longer the most complex class of MediaWiki. We deprecated and removed six Database::do* methods.

Of course way more work could to be done. There is a long-tail of small clean ups. Improving mocking in tests (now that we can rely on the mocked addQuotes(), lots of tests use new SQLPlatformTestHelper( new AddQuoterMock() ); for platform) and small code pieces could be refactored to move to platform but that can happen in its own time.

As far as I'm concerned, this is done. 🎉

ReleaseTaggerBot edited projects, added MW-1.39-notes (1.39.0-wmf.25; 2022-08-15); removed MW-1.39-notes (1.39.0-wmf.23; 2022-08-01).Aug 2 2022, 9:00 PM
Maintenance_bot moved this task from In progress to Done on the DBA board.Aug 2 2022, 9:29 PM
Jdforrester-WMF mentioned this in T299691: Break down monster class: Database.Aug 5 2022, 8:34 PM
Krinkle mentioned this in T190396: Consider splitting the IDatabase interface.Aug 6 2022, 2:38 AM
Krinkle mentioned this in T297633: <Tech Initiative> Improving Databases in MediaWiki.Aug 6 2022, 2:42 AM
ashley mentioned this in T322959: QuizGame: Call to undefined method Wikimedia\Rdbms\DatabaseMysqli.Nov 13 2022, 2:53 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits