Page MenuHomePhabricator
Create Task
Maniphest T145947

TransactionProfiler to suppress violations on TEMPORARY TABLE operations
Closed, ResolvedPublic
Actions

Description

Based on the [0] discussion, avoid having the TransactionProfiler to report violations for the "CREATE TEMPORARY TABLE, INSERT, DROP TEMPORARY TABLE" operations.

[0] https://gerrit.wikimedia.org/r/#/c/311298/

Details

SubjectRepoBranchLines +/-
Make registerTempTableOperation() detect TRUNCATE operationsmediawiki/coremaster+20 -6
Avoid TransactionProfiler notices for TEMPORARY table writesmediawiki/coremaster+37 -4
Customize query in gerrit

Related Objects

Event Timeline

mwjames created this task.Sep 18 2016, 1:09 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 18 2016, 1:09 AM
aaron closed this task as Resolved.Sep 19 2016, 6:44 PM
aaron claimed this task.
aaron subscribed.

Fixed by https://gerrit.wikimedia.org/r/#/c/311476/

aaron reopened this task as Open.Sep 19 2016, 6:44 PM
aaron reassigned this task from aaron to hoo.
aaron claimed this task.
aaron added a subscriber: hoo.

Wrong task, I meant T146019.

gerritbot subscribed.Sep 19 2016, 9:15 PM

Change 311591 had a related patch set uploaded (by Aaron Schulz):
Avoid TransactionProfiler notices for TEMPORARY table writes

https://gerrit.wikimedia.org/r/311591

gerritbot added a project: Patch-For-Review.Sep 19 2016, 9:15 PM
aaron added a comment.Sep 19 2016, 9:16 PM
In T145947#2650402, @gerritbot wrote:

Change 311591 had a related patch set uploaded (by Aaron Schulz):
Avoid TransactionProfiler notices for TEMPORARY table writes

https://gerrit.wikimedia.org/r/311591

Does this cover your usage?

mwjames added a comment.Sep 19 2016, 11:05 PM

I did a quick check against the PR and things like CREATE TEMPORARY TABLE t0` [0] are no longer reported but resulting INSERT/DELETE operations on a temp table are still visible (computed values are inserted into the temp table and kept until they are dropped) such as:

[DBPerformance] Expectation (writes <= 0) by MediaWiki::main not met:
query-m: INSERT IGNORE INTO `t0` SELECT DISTINCT t1.s_id FROM `smw_fpt_inst` AS t1 WHERE t1.o_id='X' [TRX#c0045d]
#0 ...\mw-master\includes\libs\rdbms\TransactionProfiler.php(211): TransactionProfiler->reportExpectationViolated('writes', 'query-m: INSERT...')
#1 ...\mw-master\includes\libs\rdbms\database\Database.php(936): TransactionProfiler->recordQueryCompletion('query-m: INSERT...', 1474325613.1235, true, 16)
#2 ...\mw-master\includes\libs\rdbms\database\Database.php(847): Database->doProfiledQuery('INSERT IGNORE I...', 'INSERT /* SMW\\S...', true, 'SMW\\SQLStore\\Qu...')
#3 ...\mw-master\extensions\SemanticMediaWiki\src\MediaWiki\Database.php(258): Database->query('INSERT IGNORE I...', 'SMW\\SQLStore\\Qu...', false)
#4 ...\mw-master\extensions\SemanticMediaWiki\src\SQLStore\QueryEngine\QuerySegmentListProcessor.php(223): SMW\MediaWiki\Database->query('INSERT IGNORE I...', 'SMW\\SQLStore\\Qu...')
#5 ...\mw-master\extensions\SemanticMediaWiki\src\SQLStore\QueryEngine\QuerySegmentListProcessor.php(123): SMW\SQLStore\QueryEngine\QuerySegmentListProcessor->doResolveBySegment(Object(

[0] Previous report:

query-m: CREATE TEMPORARY TABLE `t0`( id INT UNSIGNED KEY ) ENGINE=MEMORY [TRX#7e0f56]
#0 ...\mw-master\includes\libs\rdbms\TransactionProfiler.php(211): TransactionProfiler->reportExpectationViolated('writes', 'query-m: CREATE...')
#1 ...\mw-master\includes\libs\rdbms\database\Database.php(905): TransactionProfiler->recordQueryCompletion('query-m: CREATE...', 1474325733.7617, true, 0)
#2 ...\mw-master\includes\libs\rdbms\database\Database.php(816): Database->doProfiledQuery('CREATE TEMPORAR...', 'CREATE /* SMW\\S...', true, 'SMW\\SQLStore\\Ta...')
#3 ...\mw-master\extensions\SemanticMediaWiki\src\MediaWiki\Database.php(258): Database->query('CREATE TEMPORAR...', 'SMW\\SQLStore\\Ta...', false)
#4 ...\mw-master\extensions\SemanticMediaWiki\src\SQLStore\TableBuilder\TemporaryTableBuilder.php(60): SMW\MediaWiki\Database->query('CREATE TEMPORAR...', 'SMW\\SQLStore\\Ta...')
#5 ...\mw-master\extensions\SemanticMediaWiki\src\SQLStore\QueryEngine\QuerySegmentListProcessor.php(186): SMW\SQLStore\TableBuilder\TemporaryTableBuilder->createTable('`t0`')
aaron added a comment.Sep 19 2016, 11:50 PM

I added "IGNORE" to the query check.

gerritbot added a comment.Sep 20 2016, 10:45 PM

Change 311591 merged by jenkins-bot:
Avoid TransactionProfiler notices for TEMPORARY table writes

https://gerrit.wikimedia.org/r/311591

ReleaseTaggerBot added projects: MW-1.28-release-notes, MW-1.28-release (WMF-deploy-2016-10-04_(1.28.0-wmf.21)).Sep 20 2016, 11:00 PM
aaron closed this task as Resolved.Sep 21 2016, 5:29 AM
mwjames added a comment.Apr 29 2017, 6:26 PM

The TRUNCATE on temporary tables is still reported by the TransactionProfiler.

query-m: TRUNCATE TABLE smw_new [TRX#f380cd]
#0 ...\libs\rdbms\TransactionProfiler.php(218): Wikimedia\Rdbms\TransactionProfiler->reportExpectationViolated('writes', 'query-m: TRUNCA...')
#1 ...\libs\rdbms\database\Database.php(979): Wikimedia\Rdbms\TransactionProfiler->recordQueryCompletion('query-m: TRUNCA...', 1493489625.2314, true, 0)
#2 ...\libs\rdbms\database\Database.php(891): Wikimedia\Rdbms\Database->doProfiledQuery('TRUNCATE TABLE ...', 'TRUNCATE /* SMW...', true, 'SMW\\SQLStore\\Qu...')
#3 ...\extensions\SemanticMediaWiki\src\MediaWiki\Database.php(302): Wikimedia\Rdbms\Database->query('TRUNCATE TABLE ...', 'SMW\\SQLStore\\Qu...', false)
#4 ...\extensions\SemanticMediaWiki\src\SQLStore\QueryEngine\HierarchyTempTableBuilder.php(185): SMW\MediaWiki\Database->query('TRUNCATE TABLE ...', 'SMW\\SQLStore\\Qu...')
#5 ...\extensions\SemanticMediaWiki\src\SQLStore\QueryEngine\HierarchyTempTableBuilder.php(130): SMW\SQLStore\QueryEngine\HierarchyTempTableBuilder->buildTempTableFor('`t4`', '('125')', '`smw_fpt_subc`', 10)
#6 ...\extensions\SemanticMediaWiki\src\SQLStore\QueryEngine\QuerySegmentListProcessor.php(297): SMW\SQLStore\QueryEngine\HierarchyTempTableBuilder->createHierarchyTempTableFor('class', '`t4`', '('125')')
#7 ...\extensions\SemanticMediaWiki\src\SQLStore\QueryEngine\QuerySegmentListProcessor.php(239): SMW\SQLStore\QueryEngine\QuerySegmentListProcessor->
mwjames added a subscriber: Krinkle.May 9 2017, 2:27 PM

@Krinkle Do I need to create another task for this or can that be done in a swift (meaning to add TRUNCATE to the query check for temp tables)?

gerritbot added a comment.May 9 2017, 8:22 PM

Change 351120 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] Make registerTempTableOperation() detect TRUNCATE operations

https://gerrit.wikimedia.org/r/351120

gerritbot added a comment.May 10 2017, 12:29 AM

Change 351120 merged by jenkins-bot:
[mediawiki/core@master] Make registerTempTableOperation() detect TRUNCATE operations

https://gerrit.wikimedia.org/r/351120

ReleaseTaggerBot added a project: MW-1.30-release-notes (WMF-deploy-2017-05-23_(1.30.0-wmf.2)).May 10 2017, 1:00 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits