Performance Tuning Waits Queues

SQL Server 2005 Waits and Queues
SQL Server Best Practices Article
Writers: Tom Davidson

Updated by: Danny Tambs
Technical Reviewer: Sanjay Mishra
Published: November 2006

Applies To: Microsoft SQL Server 2005
Copyright
The information contained in this document represents the current view of Microsoft Corporation on the issues
discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it
should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the
accuracy of any information presented after the date of publication.
This White Paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS,
IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT.
Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under
copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or
transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or
for any purpose, without the express written permission of Microsoft Corporation.
Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights
covering subject matter in this document. Except as expressly provided in any written license agreement
from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks,
copyrights, or other intellectual property.
Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses,
logos, people, places and events depicted herein are fictitious, and no association with any real company,
organization, product, domain name, email address, logo, person, place or event is intended or should be
inferred.
 2006 Microsoft Corporation. All rights reserved.
Microsoft, SQL Server, Windows, Window Server are either registered trademarks or trademarks of Microsoft
Corporation in the United States and/or other countries.
The names of actual companies and products mentioned herein may be the trademarks of their respective
owners.
Table of Contents
Introduction ......................................................................................................1
Overview ...........................................................................................................1
Purpose .................................................................................................... 1
Audience: Who should read this paper .......................................................... 2
Waits and Queues: A Performance Methodology ...............................................2
Execution Model (simplified) ............................................................................. 3
Waiter List and Wait Types ............................................................................... 5
Dynamic Management Views (DMVs) and Functions (DMFs).................................. 5
Sys.dm_exec_requests ............................................................................... 5
Sys.dm_os_waiting_tasks ........................................................................... 5
Sys.dm_os_wait_stats ................................................................................ 6
Track_waitstats_2005 stored procedure ........................................................ 6
Get_waitstats_2005 stored procedure ........................................................... 6
Performance and Tuning Blue Prints .................................................................. 7
OLTP blueprint ........................................................................................... 7
Common scenarios to avoid in OLTP ............................................................. 7
DataWarehouse blueprint .......................................................................... 12
Common Scenarios to avoid with DataWarehousing ...................................... 12
Typical resource bottlenecks ........................................................................... 16
Memory pressure and IO subsystem issues ................................................. 16
IO Stalls ................................................................................................. 16
Missing or poorly formed indexes ............................................................... 17
Largest IO queries. ................................................................................... 18
Query optimizer, query plans and statistics ................................................. 18
Query plan reuse and DMVs ...................................................................... 19
Query plan reuse and Performance counters ................................................ 19
Statement level recompilation and DMVs..................................................... 20
Tracking down blocking issues. .................................................................. 21
Retrieving statements in the waiter list ....................................................... 22
SQL Server 2005 Wait Types and correlation to other Performance information .... 22
QUEUES (Perfmon Counters) .......................................................................... 79
PERFMON Counters, correlation, possible conclusions and actions .................. 79
Interesting PERFMON Ratios and comparisons ............................................. 89
Memory Issues ............................................................................................. 91
Comparison of 32-bit memory architecture vs. 64-bit flat memory ................. 91
64-bit flat memory vs. higher 32-bit clock speeds ........................................ 92
Application Design issues ............................................................................... 93
Recommendations...........................................................................................93
Conclusion.......................................................................................................93
SQL Server 2005 Waits and Queues 1
Introduction
This paper approaches the complex area of Microsoft® SQL Server™ performance
tuning using a methodology called Waits and Queues. By using this methodology one
can identify the best opportunities to improve performance, the so called “biggest bang
for the buck”. These performance improvements are likely to have a significant return
on the performance tuning time investment. The methodology helps identify the areas
of slow performance by looking at the problem from two directions. You, or another
troubleshooter, can use this to pinpoint problem areas by correlating data from two
sources: Waits and Queues. An analysis of Waits indicates where SQL Server is
spending lots of time waiting. In addition, the biggest waits point out the most
important or relevant Queues (that is, Performance Monitor counters and other data)
for this workload. The cross validation of the waits analysis enables us to eliminate all
except the most significant performance counters, and provides a strong indication of
pressure on specific resources.
In sum, Performance Tuning using the Waits and Queues methodology is an effective
way to quickly identify and resolve application performance problems because it lets the
user discover new and potentially unexpected problem areas, within applications or
solutions without the typical guesswork that can accompany such work.
Overview
Performance tuning of applications and solutions has been around for many years. The
performance of SQL Server 2005 database applications should be evaluated from
several different perspectives. Each perspective tells a different section of the complete
performance story. Together they paint a detailed performance picture of the whole and
also cross validate observations in each specific perspective.
We outline a methodology that considers performance from the perspective of
application, SQL Server, and correlates this to the system or resource perspective.
These perspectives are the primary inputs to the waits and queues methodology.
Be aware that some bottlenecks are more easily correctable than other bottlenecks. For
example, a lack of query plan reuse for ad hoc SQL can be resolved by either of two
methods: (1) using sp_executesql to parameterize the ad hoc SQL or (2) by replacing
ad hoc SQL with stored procedures. However, completing these application changes
takes time depending on the extent of the coding and testing requirements.
Purpose
The purpose of this document is to help developers and database administrators in
pinpointing areas in applications and solutions that interact with SQL Server 2005 and
can benefit from improved performance. This paper outlines a best practice
methodology and provides guidelines and thought processes to identify poor performing
applications in addition to providing insight into improvement regimes. It should be
noted that, although the concepts outlined in this paper can apply to all versions of
Microsoft SQL Server, the included examples are specific to SQL Server 2005 as they
use some new features not available in earlier versions.
This methodology can be put to most effective use in order to discover some of the
non-obvious performance issues and help in quickly identifying the root cause. There
are many papers and books on performance tuning and optimization techniques for
Microsoft Corporation ©2006

various versions of Microsoft SQL Server. This paper demonstrates features that are
specific to SQL Server 2005. It embodies knowledge from a range or sources including
the SQL Server development team in addition to specialist consultants working with
customers.
Audience: Who should read this paper

This paper is intended for developers, testers and database administrators (DBAs) that
are involved with development or performance optimization of solutions that are based
on the Microsoft SQL Server platform. This paper assumes some knowledge of
SQL Server commands and a basic foundation in application performance tuning. This
methodology is not intended to be a substitute for application performance testing
during the development phase nor is it meant to be a substitute for other papers in this
area.
Waits and Queues: A Performance Methodology

As an application, SQL Server may request system resources as it executes a user
query and waits for its request to be completed. Waits are represented by SQL Server
wait statistics. SQL Server 2005 tracks wait information any time that a user connection
or session_id is waiting. This wait information is summarized and categorized across all
connections so that a performance profile can be obtained for a given work load.
Therefore, SQL Server wait types identify and categorize user (or thread) waits from an
application workload or user perspective.
Queues measure system resources and utilization. The queues part of performance is
represented by Performance Monitor objects and counters and other sources of
information. Performance Monitor counters measure various aspects of performance
such as transfer rates for disks or the processor time consumed. SQL Server object
counters are exposed to Performance Monitor using the dynamic management view
(DMV) sys.dm_os_performance_counters. Thus, Performance Monitor counters show
performance from a resource point of view.
Associations or correlations of wait types to performance counters, and interesting

performance counter ratios and comparisons round out the picture. The association of
waits and queues allows one to eliminate irrelevant counters insofar as the performance
bottleneck is concerned and focus effectively on the problem area. Comparisons of one
counter to another provide perspective in order to draw the right conclusion. For
example, say you encounter 1000 lock waits during a production workload. Does this
cause a serious performance problem? To determine what role these play in overall
performance you need to also consider the total number of lock requests (how many
locks are requested overall vs. how many result in lock waits), the wait time or duration
of the lock waits, and the time span of the test. If the total number of lock requests
was 10 million, perhaps 1000 waits is insignificant. Further, if the associated lock wait
time is 50 seconds overall, and the workload is over 8 hours, this too is insignificant.
On the other hand, if you average a 50 second wait for EACH of the 1000 lock waits,
this IS significant. In sum, associations or correlations allow us to determine relevancy
to overall performance.

Application performance can be easily explained by looking at SQL Server waits and
System or Resource queues. In SQL Server 2005, the dynamic management view
(DMV) sys.dm_os_wait_stats provides a valuable source of wait information from an
application perspective. The system or resource perspective uses Performance Monitor
counters and other data sources to provide a breakdown of system resource usage
according to resource queues. Taken together, the value of the application and resource
perspectives used by the waits and queues methodology enables us to narrow in on the
bottleneck and eliminate the irrelevant performance data.
Execution Model (simplified)

The best analogy to depict the execution model for SQL Server is the grocery store
checkout line. The cashier is the CPU. The customer who is currently being checked out
by the cashier is the running session. The customers who are waiting in line represent
the runnable queue. If customer1 who is being checked out requires a price check on a
product, customer1 must wait until the price check is completed. Meanwhile, the next
in line, customer2, is immediately checked out by the cashier until the price check is
completed for customer1. When the price check is completed, the cashier can resume
the check out of customer1. This is the simplest illustration of the SQL Server
execution model called SQLOS.
The SQL Server SQLOS uses schedulers to manage the execution of user requests.
SQLOS Schedulers map to CPUs. Assuming a 4-CPU Server, there would be 4 SQLOS
schedulers by default. The following diagrams depict a simplified version of execution
model using a single SQLOS scheduler. The execution model in Figure 1 depicts how
SQL Server user requests or sessions (denoted by SPIDs) are scheduled for execution.
Figure 1: Execution Model – Running, runnable and suspended status, Runnable Queue
and Wait List
Figure 2 shows how SQL Server sessions rotate between the following statuses:
Running (only one session can be running or executing, per scheduler), Runnable
(sessions waiting for CPU), or Suspended. SPIDs with suspended statuses are placed
in Waiter List until the requested resources are available. If a running session needs a
data page that is not in cache, or needs a page that is blocked by another user’s lock,
the session is moved to the wait list. The next SPID or session_id in the runnable queue
is scheduled to start running.
Figure 2: Execution Model – How status changes affect SPIDs

The status change sequence of events is as follows:

1. SPID60 needs a page not in cache. Thus its status changes from Running to
Suspended with wait type IO_Completion
2. SPID60 moved to Waiter List
3. SPID51 moves from Runnable queue with a runnable status to Running status,
SPID64 then moves to the top of the Runnable queue
4. SPID56 is waiting for a parallel process to complete. When the parallel process
is completed, the status for SPID56 changes from Suspended with wait type
CXPACKET to Runnable
5. SPID56 moved to the bottom of the Runnable queue
Figure 3 depicts execution “after” session_ids (or SPIDs) have rotated clockwise due to
status changes.
Figure 3: Execution Model – After status change
The wait list means a thread has to wait for a resource. Example of resource waits
includes IOs to complete, a lock to be released, a memory grant, and so on. When the
session is moved to the wait list, a wait type is assigned and time is accumulated. When
the resource becomes available, the thread is moved to the runnable queue and it
executes as soon as the CPU is available. The clockwise rotation between running,
runnable and suspended states continues until the user request is completed.

Waiter List and Wait Types

When a SQL Server 2005 session_id goes into a suspended status, a wait state is
assigned indicating the reason why the session_id is suspended. The waiter list, shown
in a DMV called sys.dm_os_waiting_tasks, contains currently suspended sessions and
reasons for the suspension including the session_id, wait_type and the session’s
accumulated wait time for this wait type in the column wait_duration_ms. If the wait is
due to blocking where a lock cannot be obtained until another session releases their
lock, the session holding as the lock, also known as the blocker and blocked resource
are shown in the columns blocking_session_id and resource.
The current wait list can be seen in sys.dm_os_waiting_tasks. The current runnable
queue is found in sys.dm_exec_requests where the status is “runnable”. The total time
that is spent waiting in sys.dm_os_waiting_tasks is found in the column wait_time_ms
and the time that is spent waiting for CPU in the runnable queue is called signal_wait
_time_ms. Resource waits can be computed by subtracting signal_wait_time_ms from
wait_time_ms. A runnable queue is unavoidable with an OLTP workload because there
are large volumes of identical transactions. The key question is not the length of the
runnable queue but rather how much time is spent waiting for CPU compared to the
resource waits of the waiter list. The difference between resource and signal waits
shows the extent of CPU pressure, if any, on overall performance. A low signal (where
signal is less than 25% of the total waits) to resource wait ratio indicates there is little
CPU pressure.
Dynamic Management Views (DMVs) and

Functions (DMFs)
Dynamic Management Views (DMVs) and Dynamic Management Functions (DMFs)
expose changing server state information that typically spans many sessions, many
transactions, and many requests. Dynamic management views and functions reflect
what’s going on inside the server process itself or across all sessions in the server. They
are useful for diagnostics, memory and process tuning, and monitoring potentially
across all sessions in the server.
Useful DMVs for performance tuning purposes include sys.dm_exec_requests,
sys.dm_os_waiting_tasks, and sys.dm_os_wait_stats.
Sys.dm_exec_requests
Each SQL Server session has a unique session_id in the system DMV
sys.dm_exec_requests. The stored procedure sp_who2 provides a list of these sessions
in addition to other connection information such as command, resource, wait types,
wait time, and status. User queries will have a session_id > 50. Common status
values are ‘running’, ‘runnable’ and ‘suspended’, as described in the Execution Model
discussion. A session status of ‘Sleeping’ indicates SQL Server is waiting for the next
SQL Server command.
Sys.dm_os_waiting_tasks
The waiter list that shows all waiting sessions and the reasons for the waits can be
found in the DMV sys.dm_os_waiting_tasks. The session_id, wait type, and associated
wait time can be seen. In addition, if the session is blocked from acquiring a lock, the

session holding (known as blocking) the lock as well as the blocked resource is shown in
the columns blocking_session_id and resource.
Sys.dm_os_wait_stats
Sys.dm_os_wait_stats is the DMV that contains wait statistics, which are aggregated
across all session ids since the last restart of SQL Server or since the last time that the
wait statistics were reset manually using DBCC SQLPERF ('sys.dm_os_wait_stats',
CLEAR). Resetting wait statistics can be helpful before running a test or workload.
Anytime a session_id waits for a resource, the session_id is moved to the waiter list
along with an associated wait type. The DMV sys.dm_os_waiting_tasks shows the
waiter list at a given moment in time. Waits for all session_ids are aggregated in
sys.dm_os_wait_stats.
The stored procedures track_waitstats_2005 and get_waitstats_2005 can be used to

measure the wait statistics for a given workload.
Track_waitstats_2005 stored procedure

Track_waitstats_2005 is a stored procedure that captures wait statistics from the
DMV sys.dm_os_wait_stats and provides a ranking of descending order based on
percentage. You can use this ranking to identify the greatest opportunities for
performance improvements.
The script location is:
http://www.microsoft.com/technet/scriptcenter/scripts/sql/sql2005/waitstats/sql05vb04
9.mspx
Get_waitstats_2005 stored procedure

The stored procedure get_waitstats_2005 reports the wait types that are collected by
track_waitstats_2005. The get_waitstats_2005 procedure can be run during the
execution of track_waitstats or after track_waitstats is completed. Running
get_waitstats_2005 during the execution of track_waitstats_2005 will return a report of
intermediate results while running get_waitstats_2005 at the conclusion of
track_waitstats_2005 will return the final wait statistics report. The report provides a
detailed picture of different wait types during the time measured, and the accumulated
wait time for each.
Get_waitstats_2005 reports information about waits. Total wait time is composed of
resource waits and signal waits. Resource waits are computed by subtracting signal
waits from total waits. Because signal waits represent the amount of time spent
waiting in the runnable queue for CPU resources, they are a measure of CPU pressure.
The application blueprints identify the significance CPU pressure by comparing signal
waits with total waits.


http://www.microsoft.com/technet/scriptcenter/scripts/sql/sql2005/perf/sql05vb021.ms
px
Performance and Tuning Blue Prints

In this white paper, we examine different types of applications, how they use resources,
and how you can performance tune the applications for different workloads. An OLTP
workload differs significantly from a relational data warehouse or reporting application
workload and it is useful to understand these differences and how it affects the
objectives for high performance.
Although resource bottlenecks differ from application to application, the performance
and tuning methodology called Waits and Queues is highly accurate and the results are
reproducible. Once you resolve one bottleneck, there will be another as you scale the
application larger.
An OLTP workload is generally characterized by high numbers of small identical
transactions. In contrast, a data warehouse or reporting application is characterized by
a few large transactions, each very different. These distinctions result in very different
objectives and resource usage profiles. The blueprints reflect these distinctions.
OLTP blueprint
An OLTP application is characterized by a high volume of small identical transactions,
which frequently include SELECT, INSERT, UPDATE, and DELETE operations.
Unlike large data warehouse or reporting transactions where multiple CPUs work in
parallel, dividing up the query into smaller pieces, the small transactions of OLTP do not
require parallelism. Parallelism is multiple CPUs working in parallel, dividing up the
query into smaller pieces.
While a query is divided across multiple CPUs it will run faster, but it does so by
sacrificing CPU resources, as it requires merges and sorts of the smaller pieces before
presenting the final result set. An OLTP transaction is small to begin with so there is no
need for parallel operations that basically sacrifice CPU and memory resources for
speed of execution. Plus, with the high transaction volumes of OLTP, it is important not
to waste CPU resources. Parallelism is most appropriate for the big, low volume
transactions of data warehouse or reporting applications.
The implications are significant for database design, resource usage and system
performance.
OLTP Performance blue print objectives: There are likely to be performance and
scalability problems if any of resource issues the following tables are true.
Note The values in Value column are good starting point. The actual values will vary.
Common scenarios to avoid in OLTP

Database Design
Rule Description Value Source Problem

Description

1 High >4 Sys.dm_exec_sql_text High

Frequency Sys.dm_exec_cached_plans frequency
queries queries with
having a lots of joins
high number can be too
of table normalized for
joins. high OLTP
scalability.
2 Frequently >3 Sys.indexes Excessive
updated sys.dm_db_operational_index_stats index
tables maintenance
having # for OLTP.
indexes.
3 Big IOs >1 Perfmon object A missing
Table Scans SQL Server Access Methods index flushes
the cache.
Range Sys.dm_exec_query_stats
Scans
4 Unused Index not in Sys.dm_db_index_usage_stats. If Avoid Index
Indexes. an index is NEVER used, it will not appear in maintenance
the DMV sys.dm_db_index_usage_stats for unused
indexes.
CPU

Description
1 Signal Waits >25% Sys.dm_os_wait_stats Time in runnable
queue is pure CPU
wait.
2 Plan reuse <90% Perfmon object OLTP identical
SQL Server Statistics transactions
should ideally
have >95% plan
reuse.
3 Parallelism: >5% Sys.dm_os_wait_stats Parallelism
Cxpacket waits reduces OLTP
throughput.
CXPACKET
indicates that
multiple CPUs are
working in
parallel, dividing
up the query in
smaller pieces.
Ordinarily a well
tuned OLTP

application would
not parallelize
unless an index is
missing, there is
an incomplete
WHERE clause, or
the query is not a
true OLTP
transaction.
Memory
Rule Description Value Source Problem Description

1 Page life <300 Perfmon object Page life expectancy is
expectancy sec SQL Server Buffer the average number of
Manager seconds a data page
stays in cache. Low
SQL Server Buffer Nodes
values could indicate a
cache flush that is caused
by a big read. Pure OLTP
workloads do NOT issue
big reads, thus possible
missing index.
2 Page life Drops by Perfmon object Page life expectancy is
expectancy 50% SQL Server Buffer the average number of
Manager seconds a data page
stays in cache. Low
values could indicate a
cache flush that is caused
by a big read. Pure OLTP
workloads do NOT issue
big reads, thus possible
missing index.
3 Memory >1 Perfmon object Current number of
Grants SQL Server Memory processes waiting for a
Pending Manager workspace memory grant.
4 SQL cache <90% SQL cache hit ratio falls It is likely that large
hit ratio under 90% for sustained scans have to be
periods of time greater performed, which in turn
than 60 sec. flushes out the buffer
cache.
IO

1 Average Disk >20 ms Perfmon object Reads should take 4-8
ms without any IO

sec/read Physical Disk pressure.

2 Average Disk >20 ms Perfmon object Writes (sequential) can
sec/write Physical Disk be as fast as 1 ms for
transaction log.
3 Big IOs >1 Perfmon object A missing index flushes
Table Scans SQL Server Access the cache.
Range Scans Methods
4 If Top 2 values Top 2 Sys.dm_os_wait_stats If top 2 wait_stats

for wait stats are values include IO, there
any of the is an IO bottleneck.
following:
ASYNCH_IO_CO
MPLETION
IO_COMPLETION
LOGMGR
WRITELOG
PAGEIOLATCH_x
5 Low bytes per Perfmon object
sec. Physical Disk
Blocking

1 Block >2% Sys.dm_db_index_operati Frequency of blocks.
percentage onal_stats
2 Block process 30 sec Sp_configure profiler Report of statements.
report
3 Average Row >100ms Sys.dm_db_index_operati Duration of blocks.
Lock Waits onal_stats
4 If Top 2 Top 2 Sys.dm_os_wait_stats If top 2 wait_stats
values for values include locking,
wait stats are there is a blocking
any of the bottleneck.
following:
LCK_M_BU
LCK_M_IS
LCK_M_IU
LCK_M_IX
LCK_M_RIn_NL
LCK_M_RIn_S
LCK_M_RIn_U

LCK_M_RIn_X
LCK_M_RS_S
LCK_M_RS_U
LCK_M_RX_S
LCK_M_RX_U
LCK_M_RX_X
LCK_M_S
LCK_M_SCH_M
LCK_M_SCH_S
LCK_M_SIU
LCK_M_SIX
LCK_M_U
LCK_M_UIX
LCK_M_X
5 High number >5 per Trace flag 1204 to display If the deadlock occurs
of deadlocks hour in the errorlog and or the with the same
profiler deadlock graph. participant SQL
commands or
operations multiple
times, it is likely that
there is a locking
problem.
Network

1 High network Output queue Perfmon Indicates that the
latency coupled length >2 object: latency between the
with an Network application server and
application that Interface the database is high.
has many round Could be caused by
trips to the significant network
database. infrastructure between
the application and the
instance of SQL Server.
2 Network Packets Perfmon Dropped packets are
bandwidth is Outbound object: detected.
used up. Discarded Network
Packets Interface
Outbound
Errors
Packets
Received
Discarded

Packets
Received Errors
In summary, given the high volume of identical small transactions that characterize
OLTP, transactions per second and resource usage can be improved as follows:
1. Database designs usually keep the number of indexes to a functional minimum as
every insert, update, and delete incurs index maintenance.
2. CPU can be reduced with plan reuse and join reduction.
3. IO performance can be reduced with good indexing, join reduction, and high page
life expectancy.
4. Memory is optimal when there are no sudden drops in Page Life Expectancy.
5. Sorts can be limited with index usage. That is, a certain sort order is supported by
an index that is sorted the same way, either ascending or descending.
6. Blocking can be reduced with index design and short transactions.
DataWarehouse blueprint
Compared to OLTP, data warehouse applications are characterized by low volumes of
large transactions. This workload profile is exactly the opposite of OLTP. Data
warehouse and reporting applications typically include big SELECT or read operations.
The implications are significant for database design, resource usage, and system
performance. Index fragmentation, cache turnover, and IO performance are often
important factors for these workloads.
Data warehouse performance blue print objectives: There are performance problems if
any of the resource issues in the following tables are true.
Note While the actual value in the value column could be debated, it is a good
starting point when identifying common performance problems with DataWarehousing
and Reporting applications.
Common Scenarios to avoid with DataWarehousing

Database Design

1 Excessive Sys.dm_exec_sql_tex Large data warehouse can
sorting and t benefit from more indexes.
RID lookup Sys.dm_exec_cached Indexes can be used to cover
operations _plans queries and avoid sorting.
should be The cost of index overhead is
reduced with only paid when data is
covered loaded.
indexes.
2 Excessive >25% sys.dm_db Reducing index
fragmentation: _index_physical_stats fragmentation through index
Average rebuilds can benefit big
fragmentation_ range scans, common in data
in_percent warehouse and Reporting

should be scenarios.
<25%
3 Scans and >= 1 Perfmon object A missing index flushes the
ranges are SQL Server Access cache.
common. Look Methods
for missing
Sys.dm_db_missing_i
indexes
ndex_group_stats
Sys.dm_db_missing_i
ndex_groups
Sys.dm_db_missing_i
ndex_details
4 Unused If an index is NEVER used, it Index maintenance for
Indexes should will not appear in the DMV unused indexes should be
be avoided sys.dm_db_index_usage_stats avoided.
Resource issue: CPU

1 Signal Waits > 25% Sys.dm_os_wait_st Time in runnable queue is pure
ats CPU wait.
2 Avoid plan > 25% Perfmon object Data warehouse has fewer
reuse SQL Server transactions than OLTP, each
Statistics with significantly bigger IO.
Therefore, having the correct
plan is more important than
reusing a plan. Unlike OLTP,
data warehouse queries are not
identical.
3 Parallelism: <10% Sys.dm_os_wait_st Parallelism is desirable in data
Cxpacket waits ats warehouse or reporting
workloads.
Resource issue: Memory

1 Memory grants >1 Perfmon object Memory grant not available for
pending SQL Server query to run. Check for
Memory Manager Sufficient memory and page
life expectancy.
2 Page life Drops Perfmon object Page life expectancy is the
expectancy by 50% SQL Server Buffer average number of seconds a
Manager data page stays in cache. Low
values could indicate a cache
flush that is caused by a big
read.

Look for possible missing

index.
Resource issue: IO

Description
1 Average Disk sec/read >20 ms Perfmon object Reads should
Physical Disk take 4-8ms
without any IO
pressure.
2 Average Disk sec/write >20 ms Perfmon object Writes
Physical Disk (sequential) can
be as fast as 1
ms for
transaction log.
3 Big scans >1 Perfmon object A missing index
SQL Server Access flushes the
Methods cache.
4 If Top 2 values for wait Top 2 Sys.dm_os_wait_stats If top 2

stats are any of the wait_stats
following: values include
ASYNCH_IO_COMPLETION IO, there is an
IO bottleneck
IO_COMPLETION
LOGMGR
WRITELOG
PAGEIOLATCH_x
Resource issue: Blocking

1 Block >2% Sys.dm_db_index_operation Frequency of blocks.
percentage al_stats
2 Block 30 sec Sp_configure, profiler Report of statements.
process
report
3 Average >100ms Sys.dm_db_index_operation Duration of blocks.
Row Lock al_stats
Waits
4 If Top 2 values Top 2 Sys.dm_os_wait_stats If top 2 wait_stats values
for wait stats include IO, there is a blocking
are any of the bottleneck.
following: Consider using row versioning
LCK_M_BU to minimize shared locking

LCK_M_IS blocks.
LCK_M_IU
LCK_M_IX
LCK_M_RIn_NL
LCK_M_RIn_S
LCK_M_RIn_U
LCK_M_RIn_X
LCK_M_RS_S
LCK_M_RS_U
LCK_M_RX_S
LCK_M_RX_U
LCK_M_RX_X
LCK_M_S
LCK_M_SCH_M
LCK_M_SCH_S
LCK_M_SIU
LCK_M_SIX
LCK_M_U
LCK_M_UIX
LCK_M_X
Exactly the opposite of OLTP applications, reporting or relational data warehouse

applications are characterized by small numbers of (different) big transactions. These
are frequently SELECT intensive operations. The implications are significant for
database design, resource usage, and system performance.
Reporting and data warehouse performance objectives are as follows:
1. Data warehouse and relational data warehouse designs can have more indexes as
the cost of index maintenance is paid only one time, during the batch update
process.
2. Plan reuse should generally be avoided. Plan reuse may result in picking up a plan
that was good for some other query (with different data distribution), but may not
be good for this query. The time taken for plan generation of a large
DataWarehouse query is not nearly as important as having the right plan.
3. Sorts can and should be minimized with correct index usage.
4. Missing index situations should be investigated and corrected.
5. Large IOs such as range scans benefits from on disk contiguity. Index fragmentation
should be frequently monitored and kept to a minimum with index rebuilds.
6. Blocking is generally uncommon as most data warehouse transactions are read
operations.
7. Parallelism is generally desirable for data warehouse applications.

Typical resource bottlenecks

Resource bottlenecks can be identified by correlating waits and queues information.
Typical bottlenecks include memory pressure, IO, CPU, network, and blocking.
Depending on the application, resources can be used differently and frequently have
different performance bottlenecks. An analysis of the application profile helps identify
objectives for database design, resource usage, and performance.
For more information about using the waits and queues methodology, see the
http://www.microsoft.com/technet/scriptcenter/scripts/sql/sql2005/default.mspx, which
includes helpful DMV scripts and samples.
Memory pressure and IO subsystem issues

Latencies caused by disk to memory transfers frequently surface as PageIOLatch waits.
Memory pressure or disk IO subsystem issues can also increase PageIOLatch waits.
When a user needs a page that is not in buffer cache, SQL Server has to first allocate a
buffer page, and then puts a exclusive PageIOLatch_ex latch on the buffer while the
page is transferred from disk to cache. Meanwhile, SQL Server puts a PageIOLatch_sh
request on the buffer on behalf of the user. After the write to cache finishes, the
PageIOLatch_ex latch is released. This allows the user to read the buffer page after
which the PageIOLatch_sh is released. Consequently, high values for both
PageIOLatch_ex and PageIOLatch_sh wait types can indicate IO subsystem issues.
Pertinent performance counters include Physical disk: disk seconds/read and Physical
disk: disk seconds/write and SQL Server Buffer Manager: Page Life Expectancy. See
counters for more information.
IO Stalls
The table valued dynamic management function, sys.dm_io_virtual_file_stats provides
a breakdown of SQL Server reads, writes, and io_stalls for a particular database or
transaction log file. IO_stalls is the total cumulative time, in milliseconds, that users
waited for I/O to be completed on the file since the last restart of SQL Server.
• Select * from sys.dm_io_virtual_file_stats (dbid,file#)
• Select * from sys.dm_io_virtual_file_stats (dbid,NULL) to list all files for a database.
If IO_stalls is inordinately high for one or more files, it is possible that there is either a
disk bottleneck or that high reads and writes are occurring on one drive. Average IO
Waits per read or write can distinguish between consistently high IO queues or a
temporary IO spike. A significantly higher average value for IO stalls on one particular
drive indicates consistently high IO requests. This should be corroborated with
Performance Monitor counters Physical Disk: Average Disk Seconds/Read and Average
Disk Seconds/Write. The following script can also compute the Average Disk
Seconds/Read and Average Disk Seconds/Write using sys.dm_io_virtual_file_stats.
---- average stalls per read, write and total
---- adding 1.0 to avoid division by zero errors
select database_id, file_id
,io_stall_read_ms
,num_of_reads

,cast(io_stall_read_ms/(1.0+num_of_reads) as numeric(10,1)) as
'avg_read_stall_ms'
,io_stall_write_ms
,num_of_writes
,cast(io_stall_write_ms/(1.0+num_of_writes) as numeric(10,1)) as
'avg_write_stall_ms'
,io_stall_read_ms + io_stall_write_ms as io_stalls
,num_of_reads + num_of_writes as total_io
,cast((io_stall_read_ms+io_stall_write_ms)/(1.0+num_of_reads +
num_of_writes) as numeric(10,1)) as 'avg_io_stall_ms'
from sys.dm_io_virtual_file_stats(null,null)
order by avg_io_stall_ms desc
Missing or poorly formed indexes

Missing or poorly formed indexes can also cause excessive memory pressure or cache
flushes. In some cases, SQL Server 2005 optimizer identifies potentially useful indexes
to benefit a specific query (figure 1). The computed benefit of the index can be seen in
the column avg_user_impact (percentage improvement with suggested index). It
should be noted that this benefit applies to the individual query only where the
maintenance cost is borne by inserts, updates, and delete operations.
The following is a list of useful indexes.
-- Potentially Useful Indexes

select d.*
, s.avg_total_user_cost
, s.avg_user_impact
, s.last_user_seek
,s.unique_compiles
from sys.dm_db_missing_index_group_stats s
,sys.dm_db_missing_index_groups g
,sys.dm_db_missing_index_details d
where s.group_handle = g.index_group_handle
and d.index_handle = g.index_handle
order by s.avg_user_impact desc
go
--- suggested index columns and usage
declare @handle int

select @handle = d.index_handle

from sys.dm_db_missing_index_group_stats s
,sys.dm_db_missing_index_groups g
,sys.dm_db_missing_index_details d
where s.group_handle = g.index_group_handle
and d.index_handle = g.index_handle
select *
from sys.dm_db_missing_index_columns(@handle)
order by column_id
Largest IO queries.
The underlying purpose of an index suggested by sys.dm_db_missing_index_columns,
is to avoid doing large amounts of IO for the query in question. Therefore, you can
expect such queries to rank among the highest IO queries. To find the highest IO
queries, you can use the following sample code:
--- top 50 statements by IO
SELECT TOP 50
(qs.total_logical_reads + qs.total_logical_writes)
/qs.execution_count as [Avg IO],
substring (qt.text,qs.statement_start_offset/2,
(case when qs.statement_end_offset = -1
then len(convert(nvarchar(max), qt.text)) * 2
else qs.statement_end_offset end -
qs.statement_start_offset)/2)
as query_text,
qt.dbid,
qt.objectid
FROM sys.dm_exec_query_stats qs
cross apply sys.dm_exec_sql_text (qs.sql_handle) as qt
ORDER BY [Avg IO] DESC
Query optimizer, query plans and statistics

The SQL Server 2005 Query Optimizer compiles a query plan for a user’s query. It is
the job of the SQL Server Query Optimizer to determine the lowest cost strategy that
will be used to retrieve or modify the data. The query plan contains the strategy or
series of steps to be executed in the query plan.

For more information about SQL Server 2005 Query Optimizer, Query plans, and
Statistics refer to
http://www.microsoft.com/technet/prodtechnol/sql/2005/qrystats.mspx.
Query plan reuse and DMVs

High query plan reuse is important for OLTP applications where there are many identical
transactions. The advantage of plan reuse means you will not incur the CPU cost of
optimization for each execution of the same plan. The statements with the lowest plan
reuse can be found using DMVs as follows:
--- DMV reports statements with lowest plan reuse
---
SELECT TOP 50
qs.sql_handle
,qs.plan_handle
,cp.cacheobjtype
,cp.usecounts
,cp.size_in_bytes
,qs.statement_start_offset
,qs.statement_end_offset
,qt.dbid
,qt.objectid
,qt.text
,SUBSTRING(qt.text,qs.statement_start_offset/2,
then len(convert(nvarchar(max), qt.text)) * 2
as statement
FROM sys.dm_exec_query_stats qs
cross apply sys.dm_exec_sql_text(qs.sql_handle) as qt
inner join sys.dm_exec_cached_plans as cp on qs.plan_handle=cp.plan_handle
where cp.plan_handle=qs.plan_handle
and qt.dbid = db_id() ----- put the database ID here
ORDER BY [Usecounts] ASC
Query plan reuse and Performance counters

The Perfmon object SQL Server:SQL Statistics contains counters that can be used to
compute plan reuse. The idea is to compare batch requests to initial compilations. An

initial compilation occurs when a plan is not found in cache. OLTP applications should
have high plan reuse, > 90%.
Initial Compilations = SQL Compilations/sec – SQL Re-Compilations/sec
Plan reuse = (Batch requests/sec – Initial Compilations/sec) / Batch
requests/sec
Memory pressure can cause query plans to be discarded and therefore result in reduced
plan reuse. See OLTP Blueprints for memory pressure.
Statement level recompilation and DMVs

In SQL Server 2005, individual statements inside a stored procedure can be recompiled.
The recompilation feature can be beneficial. For example, assume that you have a
stored procedure that creates a table called MyTable, populates the table, and then
joins MyTable to other tables. Because these operations occur AFTER the initial compile,
the final row size and rowcount of MyTable is not known until run time. It would be
possible that when MyTable is joined, it can contain 1 million rows. SQL Server tracks
the statistics of MyTable and recompiles the join statement to take advantage of the
new statistics for MyTable. For more information about SQL Server 2005 Optimizer and
Statistics refer to
http://www.microsoft.com/technet/prodtechnol/sql/2005/qrystats.mspx
Recompilation is not always a good idea; for example, when the recomplied plan is the
same as the orginal plan. In these cases, you will want to identify the recompiled
statements. For more information about SQL Server 2005 recompilation refer to
http://www.microsoft.com/technet/prodtechnol/sql/2005/recomp.mspx.
The following script returns statements that have been recompiled.
---- Recompilation and SQL.sql
---- (plan_generation_num) and sql statements
---- A statement has been recompiled WHEN the plan generation number is
incremented
----
select top 25
--sql_text.text,
sql_handle,
plan_generation_num,
substring(text,qs.statement_start_offset/2,
then len(convert(nvarchar(max), text)) * 2
as stmt_executing,
execution_count,
dbid,

objectid
from sys.dm_exec_query_stats as qs
Cross apply sys.dm_exec_sql_text(sql_handle) sql_text
where plan_generation_num >1
order by sql_handle, plan_generation_num
Tracking down blocking issues.

Long blocks can be trapped with Profiler and sp_configure. The blocked process
threshold, set with sp_configure “blocked process threshold”, is the mechanism for
reporting any blocks that exceed this configured number of seconds. Make sure that the
threshold is not set too low as it can capture false positives. After setting the blocked
process threshold using sp_configure, Profiler is then used to capture the blocker and
blocked statements using the Errors and Warnings object, Blocked Process Report
event.
In order to see the main objects of blocking contention, the following code lists the
table and index with most blocks:
----Find Row lock waits
declare @dbid int
select @dbid = db_id()
Select dbid=database_id, objectname=object_name(s.object_id)
, indexname=i.name, i.index_id --, partition_number
, row_lock_count, row_lock_wait_count
, [block %]=cast (100.0 * row_lock_wait_count / (1 + row_lock_count) as
numeric(15,2))
, row_lock_wait_in_ms
, [avg row lock waits in ms]=cast (1.0 * row_lock_wait_in_ms / (1 +
row_lock_wait_count) as numeric(15,2))
from sys.dm_db_index_operational_stats (@dbid, NULL, NULL, NULL) s,
sys.indexes i
where objectproperty(s.object_id,'IsUserTable') = 1
and i.object_id = s.object_id
and i.index_id = s.index_id
order by row_lock_wait_count desc
Notice the average block time reported in the above script is in milliseconds. You must
convert the average block time to seconds in order to set the ‘blocked process
threshold’ with sp_configure. This should give you a hint on how to set the
sp_configure ‘blocked process threshold’ if you are not sure where to start. Remember
not to set the blocked process threshold (seconds) too low as this will generate false
positives. The blocked process threshold fires a trace event (Blocked Process Report)
for any block that exceeds the configured number of seconds.

Retrieving statements in the waiter list

The stored procedure get_statements_in_waiter_list reports statements in the
waiter list that matches the optional parameter @wait_list. If @wait_list is NULL, it lists
ALL statements in the waiter list. It can be run at any time to capture statements
waiting in the waiter list. For example, you can list any statement waiting for
parallelism as follows:
Exec get_statements_in_waiter_list @wait_type = ‘CXPACKET’
http://www.microsoft.com/technet/scriptcenter/scripts/sql/sql2005/waitstats/default.m
spx. See section "SQL Server 2005 Wait Types" for a list of all wait types.
SQL Server 2005 Wait Types and correlation to

other Performance information
Wait Type Category In Description Correlation to Other
Wait information
stats
Table
ASYNC_DIS IO True Occurs when there is Possible disk bottleneck.
KPOOL_LO an attempt to See the disk performance
CK synchronize parallel counters for confirmation.
threads that are
performing tasks
such as creating or
initializing a file.
SQL Server 2000:
During Backup and
Restore (for
example, including
zeroing out pages)
threads written in
parallel.
SQL Server 2005:
no longer initializes
(for example, zeros
out) data files before
a restore.


Wait information
stats
Table
ASYNC_IO_ IO True Occurs when a task See section titled
COMPLETI is waiting for “Memory pressure and
ON asynchronous I/Os Disk IO subsystem issues”
to finish. See PERFMON Physical
Identify disk Disk performance
bottlenecks, by counters:
using Perfmon 1. Disk sec/read
Counters, Profiler,
2. Disk sec/write
sys.dm_io_virtual_fil
e_stats and 3. Disk queues
SHOWPLAN. See PERFMON
Any of the following SQLServer:Buffer
reduces these waits: Manager performance
counters for memory
1. Adding additional
pressure:
IO bandwidth.
1. Page Life Expectancy
2. Balancing IO
across other 2. Checkpoint pages/sec
drives. 3. Lazy writes/sec
3. Reducing IO with See PERFMON
appropriate SQLServer:Access
indexing. Methods for correct
4. Check for bad indexing:
query plans. 1. Full Scans/sec
5. Check for 2. Index seeks/sec
memory SQL Profiler can be used
pressure. to identify which
Transact-SQL statements
do scans. Select the scans
event class and events
scan:started and
scan:completed. Include
the object Id data
column. Save the profiler
trace to a trace table, and
then search for the scans
event. The
scan:completed event
provides associated IO so
that you can also search
for high reads, writes, and
duration.
Check SHOWPLAN for bad
query plans


Wait information
stats
Table
ASYNC_NE Network True Occurs on network Check network adapter
TWORK_IO writes when the task bandwidth.
New is blocked behind 1 Gigabit is better than
the network. Verify 100 megabits.
that the client is
100 megabits is better
processing data
than 10 megabits.
from SQL Server.
BACKUP Backup True Occurs when a task
New is blocked as part of
backup processing.
BACKUP_C Backup True Internal Only.
LIENTLOCK
New
BACKUP_O Backup True Occurs when a task Check backup tape drive.
PERATOR is waiting for a tape
New mount. To view the
tape status, query
sys.dm_io_backup_t
apes. If a mount
operation is not
pending, this wait
type can indicate a
hardware problem
with the tape drive.
BACKUPBU Backup True Occurs when a Check backup tape drive.
FFER backup task is
New waiting for data, or
is waiting for a
buffer in which to
store data. This type
is not typical, except
when a task is
waiting for a tape
mount.


Wait information
stats
Table
BACKUPIO Backup True Occurs when a Check backup tape drive.
backup task is
waiting for data, or
is waiting for a
buffer in which to
store data. This type
is not typical, except
when a task is
waiting for a tape
mount.
BACKUPTH Backup True Occurs when a task
READ is waiting for a
backup task to
finish. Wait times
can be long, from
several minutes to
several hours. If the
task that is being
waited on is in an
I/O process, this
type does not
indicate a problem.
BAD_PAGE Memory True Occurs when the Suspect pages are
_PROCESS background suspect captured in the msdb
page logger is trying database system table
to avoid running dbo.suspect_pages.
more than every five Suspect pages can be
seconds which restored using online
occurs when many page level restore.
suspect pages are
encountered.
BROKER_C Service False Occurs when waiting
ONNECTIO Broker for access to receive
N_RECEIVE a message on a
_TASK connection endpoint.
New Receive access to
the endpoint is
serialized.


Wait information
stats
Table
BROKER_E Service False Occurs when there is
NDPOINT_ Broker contention to access
STATE_MU the state of a service
TEX broker connection
New endpoint. Access to
the state for
changes is
serialized.
BROKER_E Service False Occurs when a task
VENTHAND Broker is waiting in the
LER primary event
New handler of the
Service Broker. This
should occur very
briefly.
BROKER_I Service False Occurs when
NIT Broker initializing Service
New Broker in each
active database.
This should rarely
occur.
BROKER_M Service False Occurs when a task
ASTERSTA Broker is waiting for the
RT primary event
New handler of the
Service Broker to
start. This should
occur very briefly.
BROKER_R Service True Occurs when the
ECEIVE_W Broker RECEIVE WAITFOR
AITFOR is waiting. This is
New typical if no
messages are ready
to be received.
BROKER_R Service False Occurs during the
EGISTERAL Broker initialization of a
LENDPOINT Service Broker
S connection endpoint.
New This should occur
very briefly.


Wait information
stats
Table
BROKER_S Service False Occurs when there is
HUTDOWN Broker a planned shutdown
of Service Broker.
This should occur
very briefly, if at all.
BROKER_T Service False Occurs when the
RANSMITTE Broker Service Broker
R message transmitter
New is waiting for work
to do.
BUILTIN_H True Can occur after
ASHKEY_M instance startup
UTEX when internal
New datastructures are
initialized. Does not
reoccur after
datastructures have
been initialized.
CHECKPOI False Occurs while the Checkpoint writes out
NT_QUEUE checkpoint task is dirty (for example,
waiting for the next changed) data and log
checkpoint request. pages. Check for disk
issues. See PERFMON
Physical Disk performance
counters
CHKPT True Occurs at server
startup to tell the
checkpoint thread
that it can start
CLR_AUTO CLR True Occurs when a task
_EVENT is currently
New performing common
language runtime
(CLR) execution and
is waiting for a
particular autoevent
to be initiated.


Wait information
stats
Table
CLR_CRST CLR True Occurs when a task
New is currently
performing CLR
execution and is
waiting to enter a
critical section of the
task that is currently
being used by
another task.
CLR_JOIN CLR True Occurs when a task
New is currently
performing CLR
execution and
waiting for another
task to end. This
wait state occurs
when there is a join
between tasks.
CLR_MANU CLR True Occurs when a task
AL_EVENT is currently
New performing CLR
execution and is
waiting for a specific
manual event to be
initiated.
CLR_MONI CLR True Occurs when a task
TOR is currently
New performing CLR
execution and is
waiting to obtain a
lock on the monitor.
CLR_RWLO CLR True Occurs when a task
CK_READE is currently
R performing CLR
New execution and is
waiting for a reader
lock.


Wait information
stats
Table
CLR_RWLO CLR True Occurs when a task
CK_WRITE is currently
R performing CLR
New execution and is
waiting for a writer
lock.
CLR_SEMA CLR True Occurs when a task
PHORE is currently
New performing CLR
execution and is
waiting for a
semaphore.
CLR_TASK_ CLR False Occurs while waiting
START for a CLR task to
New complete startup.


Wait information
stats
Table
CMEMTHRE Memory True Occurs when a task The serialization makes
AD is waiting for a sure that as long as the
thread-safe memory users are allocating or
object. The wait freeing the memory from
time might increase the memory object, any
when there is other server process IDs
contention caused (SPIDs) that are trying to
by multiple tasks perform the same task
trying to allocate have to wait, and the
memory from the CMEMTHREAD waittype is
same memory set when the SPIDs are
object. waiting.
You might notice this
waittype in many
scenarios. However, this
waittype is most
frequently logged when
the ad hoc query plans
are being quickly inserted
into a procedure cache
from many different
connections to the
instance of SQL Server.
You can address this
bottleneck by limiting the
data that must be
inserted or removed from
the procedure cache, such
as explicitly
parameterizing the
queries so that the
queries can be reused or
using stored procedures
where appropriate.
CURSOR True Asynch Cursor
thread.
CURSOR_A True Internal only.
SYNC


Wait information
stats
Table
CXPACKET True Occurs when trying Check for parallelism:
to synchronize the sp_Configure “max
query processor degree of parallelism”.
exchange iterator. If max degree of
Consider lowering parallelism = 0, you
the degree of might want to use one of
parallelism if the following options:
contention on this
1. turn off parallelism
wait type becomes a
completely for OLTP
problem.
workloads: set max
Parallel process degree of parallelism
waits can sometimes to 1
occur when data is
2. limit parallelism by
skewed. In such
setting max degree of
cases, one parallel
parallelism to some
thread may process
number less than the
a larger number of
total number of CPUs.
rows while another
For example if you
may process a
have 8 processors, set
smaller number of
max degree of
rows and so on.
parallelism to <=4.
In an OLTP
environment,
excessive CXPACKET
waits can affect the
throughput of other
OLTP traffic.
In a data warehouse
environment,
CXPACKET waits are
expected for
multiple proc
environments.
DBMIRROR DBM True Internal only.
_DBM_EVE
NT
New
DBMIRROR DBM True Internal only.
_DBM_MUT
EX
New


Wait information
stats
Table
DBMIRROR DBM False Occurs when
_EVENTS_ database mirroring
QUEUE waits for events to
New process.
DBMIRROR DBM True Occurs when a task

_SEND is waiting for a
New communications
backlog at the
network layer to
clear to be able to
send messages.
Indicates that the
communications
layer is starting to
become overloaded
and affect the
database mirroring
data throughput.
DBMIRROR DBM False Indicates that the
_WORKER_ database mirroring
QUEUE worker task is
New waiting for more
work.
DBMIRROR DBM True Occurs when a task
ING_CMD is waiting for log
New records to be
flushed to disk. This
wait state is
expected to be held
for long periods of
time.
DBTABLE Internal only. See SQL Buffer Manager
New Checkpoint performance counters:
request that is 1. Page Life Expectancy
waiting for 2. Checkpoint pages/sec
outstanding
3. Lazy writes/sec
checkpoint request
to complete


Wait information
stats
Table
DEADLOCK Lock True Occurs when the
_ENUM_MU deadlock monitor
TEX and
sys.dm_os_waiting_
tasks try to make
sure that SQL Server
is not running
multiple deadlock
searches at the
same time.
DEADLOCK Lock True Large waiting time
_TASK_SE on this resource
ARCH indicates that server
is executing queries
in addition to
sys.dm_os_waiting_
tasks and these
queries are blocking
deadlock monitor
from running
deadlock search
(only one query or
deadlock monitor
can examine task
state at any moment
of time).
DEADLOCK_TASK_S
EARCH wait type is
used by deadlock
monitor only,
queries in addition
to
sys.dm_os_waiting_
tasks use wait type
DEADLOCK_ENUM_
MUTEX.
DEBUG True Occurs during
Transact-SQL and
CLR debugging for
internal
synchronization.


Wait information
stats
Table
DISABLE_V True Occurs when
ERSIONING SQL Server polls the
New version transaction
manager to see
whether the
timestamp of the
earliest active
transaction is later
than the timestamp
of when the state
started changing. If
this is this case, all
the snapshot
transactions that
were started before
the ALTER
DATABASE
statement was run
have finished. This
wait state is used
when SQL Server
disables versioning
by using the ALTER
DATABASE
statement.
DISKIO_SU IO True Occurs when a task
SPEND is waiting to access
a file when an
external backup is
active. This is
reported for each
waiting user
process. A count
larger than five per
user process can
indicate that the
external backup is
taking too much
time to finish.
DLL_LOADI XML False Occurs one time
NG_MUTEX while waiting for the
loading of the XML
parser DLL.


Wait information
stats
Table
DROPTEMP True Occurs between
attempts to drop a
temporary object if
the previous try
failed. The wait
duration grows
exponentially with
each failed drop try.
DTC DTC True Occurs when a task Check transaction
is waiting for an isolation level
event that is used to
manage state
transition. This state
controls when the
recovery of Microsoft
Distributed
Transaction
Coordinator (MS
DTC) transactions
occurs after
SQL Server receives
notification that the
MS DTC service has
become unavailable.
This state also
describes a task that
is waiting when a
commit of a MS DTC
transaction is
initiated by
SQL Server and
SQL Server is
waiting for the MS
DTC commit to
finish.
Waiting for
Distributed
Transaction
Coordinator


Wait information
stats
Table
DTC_ABOR DTC True Occurs in a MS DTC
T_REQUES worker session when
T the session is
waiting to take
ownership of a MS
DTC transaction.
After MS DTC owns
the transaction, the
session can roll back
the transaction.
Generally, the
session waits for
another session that
is using the
transaction
DTC_RESO DTC True Occurs when a
LVE recovery task is
waiting for the
master database in
a cross-database
transaction so that
the task can query
the outcome of the
transaction.
DTC_STAT DTC True Occurs when a task
E is waiting for an
event that protects
changes to the
internal MS DTC
global state object.
The state should be
held for very short
periods of time.


Wait information
stats
Table
DTC_TMDO DTC True Occurs in a MS DTC
WN_REQUE worker session when
ST SQL Server receives
notification that the
MS DTC service is
not available. First
the worker waits for
the MS DTC
recovery process to
start. Then the
worker waits to
obtain the outcome
of the distributed
transaction that the
worker is working
on. This can
continue until the
connection with the
MS DTC service has
been reestablished.
DTC_WAIT DTC True Occurs when
FOR_OUTC recovery tasks wait
OME for MS DTC to
become active to
enable the resolution
of prepared
transactions.
DUMP_LOG True Occurs when a main
_COORDIN task is waiting for a
ATOR subtask to generate
data. Ordinarily, this
state does not occur.
A long wait indicates
an unexpected
blockage. The
subtask should be
investigated.
EC Internal Only.


Wait information
stats
Table
EE_PMOLO True Occurs during
CK synchronization of
certain memory
allocation during
statement
execution.
EE_SPECPR False Occurs during
OC_MAP_I synchronization of
NIT internal procedure
hash table creation.
This wait can only
occur during the first
accesses of the hash
table after the
SQL Server 2005
instance started.
ENABLE_VE True Occurs when
RSIONING SQL Server waits for
all update
transactions in this
database to finish
before declaring the
database ready to
transition to
snapshot isolation
enabled state. This
state is used when
SQL Server enables
snapshot isolation
by using the ALTER
DATABASE
statement.
ERROR_RE
PORTING_
MANAGER


Wait information
stats
Table
EXCHANGE True Occurs during Check for parallelism:
synchronization in sp_Configure “max
the query processor degree of parallelism”.
exchange iterator If max degree of
during parallel parallelism = 0, you
queries. might want to use one of
the following options:
1. turn off parallelism
completely: set max
degree of parallelism
to 1
2. limit parallelism by
setting max degree of
parallelism to some
number less than the
total number of CPUs.
For example if you
have 8 processors, set
max degree of
parallelism to <=4.
EXECSYNC True Occurs during
parallel queries
while synchronizing
in query processor in
areas not related to
the exchange
iterator. Examples of
such area are
bitmaps, large
binary objects
(BLOBs) and the
spool iterator. LOBs
can frequently use
this wait state.
Bitmap and spool
use should not cause
contention.
Failpoint True Internal only.


Wait information
stats
Table
FCB_REPLI True Occurs when the
CA_READ reads of a snapshot
(or a temporary
snapshot created by
DBCC) sparse file
are synchronized.
FCB_REPLI True Occurs when the
CA_WRITE pushing or pulling of
a page to a snapshot
(or a temporary
snapshot created by
DBCC) sparse file
are synchronized.
FT_RESTAR True Occurs when a full-
T_CRAWL text crawl
(population) must
restart from a last
known good point to
recover from a
transient failure. The
wait is for letting the
worker tasks
currently working on
that population to
complete/exit the
current step.
FT_RESUM True Occurs when
E_CRAWL throttled full-text
crawls (population)
pause to wait for
existing activity to
finish.
HTTP_END True Internal only.
POINT_COL
LCREATE
HTTP_ENU True Occurs at startup to
MERATION enumerate the HTTP
endpoints to start
HTTP.
IMP_IMPOR True Internal only.
T_MUTEX


Wait information
stats
Table
IMPPROV_I True Occurs when
OWAIT SQL Server waits for
a bulkload I/O to
finish.
INDEX_US True Internal only.
AGE_STAT
S_MUTEX
IO_AUDIT_ True Occurs during
MUTEX synchronization of
trace event buffers.


Wait information
stats
Table
IO_COMPL True Occurs while waiting See Disk performance
ETION for I/O operations to counters:
finish. This wait type 1. Disk sec/read
generally represents
2. Disk sec/write
non-data page I/Os.
Data page I/O 3. Disk queues
completion waits See SQL Buffer Manager
appear as performance counters:
PAGEIOLATCH_* 1. Page Life Expectancy
waits. 2. Checkpoint pages/sec
Identify disk 3. Lazy writes/sec
bottlenecks by using
See SQL Access Methods
Performance
for correct indexing:
Counters, Profiler,
sys.dm_io_virtual_fil 1. Full Scans/sec
e_stats and 2. Index seeks/sec
SHOWPLAN See memory performance
Any of the following counter
reduces these waits: • Page faults/sec
1. Adding additional Refer to Io_stalls section
IO bandwidth, to identify IO bottlenecks.
2. Balancing IO SQL Profiler can be used
across other to identify which
drives Transact-SQL statements
3. Reducing IO with do scan. Select the scans
appropriate event class and events
indexing scan:started and
4. Check for bad scan:completed. Include
query plans the object Id data
column. Save the profiler
trace to a trace table, and
then search for the scans
event. The
scan:completed event
provides the associated
IO so that you can also
search for high reads,
writes, and duration.
Check SHOWPLAN for bad
query plans
KTM_ENLIS True Internal Only.
TMENT


Wait information
stats
Table
KTM_RECO True Internal Only.
VERY_MAN
AGER
KTM_RECO True Internal Only.
VERY_RES
OLUTION
LATCH_x Latches are short If high, check PERFMON
term light weight for
synchronization 1. memory pressure
objects. Latches are
2. SQL Latch waits (ms)
not held for the
duration of a Look for LOG and
transaction. Pagelatch_UP wait types.
“Plain” latches are Latch_x waits can

generally not related frequently be reduced by
to IO. These latches solving LOG and
can be used for a PAGELATCH_UP
variety of things, but contention. If there is no
they are not used to LOG or PAGELATCH_UP
synchronize access contention, the only other
to buffer pages option is to partition the
(PAGELATCH_x is table/index in question in
used for that). order to create multiple
caches (the caches are
Possibly the most
per-index).
common case is
contention on
internal caches (not
the buffer pool
pages), especially
when using heaps or
text.


Wait information
stats
Table
LATCH_DT True Occurs when waiting See LATCH_x
for a DT (destroy)
latch. This does not
include buffer
latches or
transaction mark
latches. A
breakdown of
LATCH_* waits is
available in
sys.dm_os_latch_sta
ts. Notice that
sys.dm_os_latch_sta
ts group LATCH_NL,
LATCH_SH,
LATCH_UP,
LATCH_EX and
LATCH_DT waits
together.
LATCH_EX True Occurs when waiting See LATCH_x
for a EX (exclusive)
include buffer
latches or
transaction mark
latches. A
breakdown of
LATCH_* waits is
available in
sys.dm_os_latch_sta
ts. Notice that
sys.dm_os_latch_sta
ts groups
LATCH_NL,
LATCH_SH,
LATCH_UP,
LATCH_EX, and
LATCH_DT waits
together.


Wait information
stats
Table
LATCH_KP True Occurs when waiting See LATCH_x
for a KP (keep)
include buffer
latches or
transaction mark
latches. A break of
latch_* waits is
available in
sys.dm_os_latch_sta
ts. Notice that
sys.dm_os_latch_sta
ts groups
LATCH_NL,
LATCH_SH,
LATCH_UP,
LATCH_EX, and
LATCH_DT waits
together.
LATCH_NL True Internal Only. See LATCH_x
LATCH_SH True Occurs when waiting See LATCH_x
for a SH (share)
include buffer
latches or
transaction mark
latches. A break of
latch_* waits is
available in
sys.dm_os_latch_sta
ts. Notice that
sys.dm_os_latch_sta
ts groups
LATCH_NL,
LATCH_SH,
LATCH_UP,
LATCH_EX, and
LATCH_DT waits
together.


Wait information
stats
Table
LATCH_UP True Occurs when waiting See LATCH_x
for a UP (update)
include buffer
latches or
transaction mark
latches. A break of
latch_* waits is
available in
sys.dm_os_latch_sta
ts. Notice that
sys.dm_os_latch_sta
ts groups
LATCH_NL,
LATCH_SH,
LATCH_UP,
LATCH_EX, and
LATCH_DT waits
together.
LAZYWRITE True Occurs when lazy
R_SLEEP writer tasks are
suspended. In a
measure of the time
that is spent by
background tasks
that are waiting. Do
not consider this
state when you are
looking for user
stalls.
LCK_x Possible transaction See SQL Locks
management issue. performance counters
1. For shared locks, • Lock wait time (ms)
check Isolation Hint: check for memory
level for pressure, which causes
transaction. more physical IO,
2. Keep transaction therefore prolonging the
as short as duration of transactions
possible and locks.


Wait information
stats
Table
LCK_M_BU Occurs when a task See Lck_x
is waiting to acquire
a Bulk update lock.
See the
sys.dm_tran_locks
topc for a lock
compatibility matrix.
LCK_M_IS Occurs when a task See Lck_x
an Intend Share
lock. See the
sys.dm_tran_locks
topc for a lock
LCK_M_IU Occurs when a task See Lck_x
an Intend Update
lock. See the
sys.dm_tran_locks
topc for a lock
LCK_M_IX Occurs when a task See Lck_x
an Intent Exclusive
lock. See the
sys.dm_tran_locks
topc for a lock
LCK_M_RIn Occurs when a task See Lck_x
_NL is waiting to acquire
a NULL lock on the
current key value
and an insert range
lock between the
current and previous
key. See the
sys.dm_tran_locks
topic for a lock


Wait information
stats
Table
_S is waiting to acquire
a shared lock on the
current key value
and an insert range
lock between the
key. See the
sys.dm_tran_locks
topic for a lock
_U is waiting to acquire
an Update lock on
the current key
value and an insert
range lock between
the current and
previous key. See
the
sys.dm_tran_locks
topic for a lock
LCK_M_RIn True Occurs when a task See Lck_x
_X is waiting to acquire
an exclusive lock on
the current key
value and an insert
range lock between
the current and
previous key. See
the
sys.dm_tran_locks
topic for a lock


Wait information
stats
Table
LCK_M_RS True Occurs when a task See Lck_x
a Shared lock on the
current key value
and a shared range
lock between the
key. See the
sys.dm_tran_locks
topic for a lock
LCK_M_RS True Occurs when a task See Lck_x
a Update Range lock
on the current key
value and a shared
range lock between
the current and
previous key. See
the
sys.dm_tran_locks
topic for a lock
LCK_M_RX True Occurs when a task See Lck_x
a Shared lock on the
current key value
and an Exclusive
Range lock between
the current and
previous key. See
the
sys.dm_tran_locks
topic for a lock


Wait information
stats
Table
an Update lock on
the current key
value and an
Exclusive range lock
between the current
and previous key.
See the
sys.dm_tran_locks
topic for a lock
_X is waiting to acquire
an Exclusive lock on
the current key
value and an
Exclusive Range lock
between the current
and previous key.
See the
sys.dm_tran_locks
topic for a lock
LCK_M_S True Occurs when a task See Lck_x
a Shared lock. See
the
sys.dm_tran_locks
topic for a lock
LCK_M_SC True Occurs when a task See Lck_x
H_M is waiting to acquire
a Schema Modify
lock. See the
sys.dm_tran_locks
topic for a lock


Wait information
stats
Table
LCK_M_SC True Occurs when a task See Lck_x
H_S is waiting to acquire
a Schema Modify
lock. See the
sys.dm_tran_locks
topic for a lock
LCK_M_SIU True Occurs when a task See Lck_x
a Shared With Intent
Updated lock. See
the
sys.dm_tran_locks
topic for a lock
LCK_M_SIX True Occurs when a task See Lck_x
a Shared With Intent
Exclusive lock. See
the
sys.dm_tran_locks
topic for a lock
LCK_M_U True Occurs when a task See Lck_x
an Update lock. See
the
sys.dm_tran_locks
topic for a lock
LCK_M_UIX True Occurs when a task See Lck_x
an Update With
Intent Exclusive
lock. See the
sys.dm_tran_locks
topic for a lock


Wait information
stats
Table
LCK_M_X True Occurs when a task See Lck_x
an Exclusive lock.
See the
sys.dm_tran_locks
topic for a lock
LOGBUFFE True Occurs when a task See Disk performance
R is waiting for space counters:
in the log buffer to 1. Disk sec/read
store a log record.
2. Disk sec/write
Consistently high
values can indicate 3. Disk queues
that the log devices
cannot keep up with
the logging
information being
generated by the
server.
LOGMGR True Occurs when a task See Disk performance
is waiting for any counters:
outstanding log I/Os 1. Disk sec/read
to finish before it
2. Disk sec/write
shuts down the log.
3. Disk queues
Identify disk
bottlenecks, by See SQL Buffer Manager
using Performance performance counters:
Counters, Profiler, 1. Page Life Expectancy
sys.dm_io_virtual_fil 2. Checkpoint pages/sec
e_stats and 3. Lazy writes/sec
SHOWPLAN
Check Io_stall for tranlog
Any of the following
• select * from
reduces these waits: sys.dm_io_virtual_file
1. Adding additional _stats(dbid,file#)
IO bandwidth,
2. Balancing IO
across other
drives
3. Moving /
Isolating the
transaction log
on its own drive


Wait information
stats
Table
LOGMGR_F True Internal Only.
LUSH
LOGMGR_R True Occurs when a task
ESERVE_AP is waiting to see
PEND whether log
truncation frees log
space to enable the
task to write a new
log record. Consider
increasing the size
of the log file(s) for
the affected
database to reduce
this wait.
LOWFAIL_ True Occurs while waiting
MEMMGR_ for memory to be
QUEUE available for use.
MIRROR_S True Internal Only.
END_MESS
AGE
MISCELLAN True Catch all wait type
EOUS
MSQL_DQ True Occurs when a task
is waiting for a
distributed query
operation to finish.
This is used to
detect potential
Multiple Active
Result Set (MARS)
application
deadlocks. The wait
ends when the
distributed query
call finishes.
MSQL_SYN True Internal Only.
C_PIPE


Wait information
stats
Table
MSQL_XAC True Occurs when a task
T_MGR_MU is waiting to obtain
TEX ownership of the
session transaction
manager to perform
a session level
transaction
operation
MSQL_XAC True Occurs during
T_MUTEX synchronization of
usage of a
transaction. A
request must
successfully acquire
the mutex before it
can use the
transaction.
MSQL_XP True Occurs when a task
is waiting for an
extended stored
procedure to end.
SQL Server uses this
wait state to detect
potential MARS
application
deadlocks. The wait
stops when the
extended stored
procedure call ends.
MSSEARCH True Occurs during Full-
Text search calls.
This wait ends when
the full-text
operation is finished.
It does not indicate
contention, but the
duration of full-text
operations.


Wait information
stats
Table
NET_WAITF True Occurs when a
OR_PACKE connection is waiting
T for a network packet
during a network
read.
OLEDB Occurs when 1. Check placement of
SQL Server calls the client applications
Microsoft SQL Native including any file input
Client OLE DB read by the client and
Provider. This state SQL Server data and
is not used for log files. See
synchronization, PERFMON disk
instead it indicates secs/read and disk
the duration of calls secs/write. If disk
to the OLE DB secs/read are high,
provider. It can also you can add IO
include the bandwidth, balance IO
following: across other drives,
Linked server calls and move or isolate
including four part the database and
name calls, remote transaction log to its
procedure calls, own drives
openquery, 2. Inspect Transact-SQL
openrowset and so code for RPC,
on. Distributed (Linked
Queries that access Server) and Full Text
DMVs, because Search. Although SQL
these are Server supports these
implemented as OLE type queries, they are
DB rowset providers. sometimes
performance
Heavy Profiler
bottlenecks.
tracing
3. To retrieve the SQL
statement involved in
OLE DB waits, refer to
section “Retrieving
statements in the
waiter list”.


Wait information
stats
Table
PAGEIOLAT Latches are short If this is significant in
CH_x term synchronization percentage, it typically
objects. used to suggests disk IO
synchronize access subsystem issues. Check
to buffer pages. disk counters.
PageIOLatch is used
for disk to memory
transfers.
PAGEIOLAT True Occurs when a task See PAGEIOLATCH_x
CH_DT is waiting for a latch
for a buffer that is in
an I/O request. The
latch request is in
Destroy mode. Long
waits of this kind
indicate a problem
with the disk
subsystem.
True Occurs when a task See PAGEIOLATCH_x
PAGEIOLAT is waiting for a latch
CH_EX for a buffer that is in
an I/O request. The
latch request is in
Exclusive mode.
Long waits of this
kind indicate a
problem with the
disk subsystem.
PAGEIOLAT True Occurs when a task See PAGEIOLATCH_x
CH_KP is waiting for a latch
an I/O request. The
latch request is in
Keep mode. Long
waits of this kind
indicate a problem
with the disk
subsystem.
PAGEIOLAT True Internal Only. See PAGEIOLATCH_x
CH_NL


Wait information
stats
Table
PAGEIOLAT Occurs when a task See PAGEIOLATCH_x
CH_SH is waiting for a latch
an I/O request. The
latch request is in
Shared mode. Long
waits of this kind
indicate a problem
with the disk
subsystem.
PAGEIOLAT Occurs when a task See PAGEIOLATCH_x
CH_UP is waiting for a latch
an I/O request. The
latch request is in
Update mode. Long
waits of this kind
indicate a problem
with the disk
subsystem.
PAGELATC Latches are short If this is significant in
H_x term light weight percentage, it typically
synchronization indicates cache
objects. Latches are contention.
not held for the
duration of a
transaction. Typical
latching operations
during row transfers
to memory,
controlling
modifications to row
offset table, and so
on. Therefore, the
duration of latches is
typically sensitive to
available memory.


Wait information
stats
Table
PAGELATC True Occurs when a task See PAGELATCH_x
H_DT is waiting for a latch
for a buffer that is
not in an I/O
request. The latch
request is in Destroy
mode.


Wait information
stats
Table
H_EX is waiting for a latch
not in an I/O
request. The latch
request is in
Exclusive mode.
Contention can be
caused by issues
other than IO or
memory
performance, for
example, heavy
concurrent inserts
into the same index
range can cause this
kind of contention. If
many inserts must
be added on the
same page, they are
serialized using the
latch. Lots of inserts
into the same range
can also cause page
splits in the index
which holds onto the
latch while allocating
a new page (this can
take time). Any read
accesses to the
same range as the
inserts would also
conflict on the
latches. The solution
in these cases is to
distribute the inserts
using a more
appropriate index.


Wait information
stats
Table
H_KP is waiting for a latch
not in an I/O
request. The latch
request is in Keep
mode.
PAGELATC True Internal Only. See PAGELATCH_x
H_NL


Wait information
stats
Table
H_SH is waiting for a latch
not in an I/O
request. The latch
request is in Shared
mode.
Contention can be
caused by issues
other than IO or
memory
performance, for
example, heavy
concurrent inserts
into the same index
range can cause this
kind of contention. If
many inserts must
be added on the
same page, they are
serialized using the
latch. Lots of inserts
into the same range
can also cause page
splits in the index
which holds onto the
latch while allocating
a new page (this can
take a time). Any
read accesses to the
same range as the
inserts would also
conflict on the
latches. The solution
in these cases is to
distribute the inserts
using a more
appropriate


Wait information
stats
Table
H_UP is waiting for a latch
not in an I/O
request. The latch
request is in Update
mode.
Page latch Update is
used only for
allocation related
pages, and
contention on it is
frequently a sign
that more files are
needed. With
multiple files,
allocations can be
distributed across
multiple files
therefore reducing
demand on the per-
file data structures
stored on these
pages. The
contention is not IO
performance, but
internal allocation
contention to access
the pages. Adding
more spindles to a
file or moving the
file to a faster disk
does not help, nor
does adding more
memory.


Wait information
stats
Table
PRINT_ROL Used to wait while
LBACK_PR user processes are
OGRESS ended in a database
that has been
transitioned by using
the ALTER
DATABASE
termination clause.
For more
information, see
ALTER DATABASE
(Transact-SQL).
QNMANAGE Internal Only.
R_ACQUIR
E
QPJOB_KIL Indicates that an
L asynchronous
automatic statistics
update was canceled
by a call to KILL as
the update was
starting to run. The
terminating thread is
suspended, waiting
for it to start
listening for KILL
commands. A good
value is less than
one second.


Wait information
stats
Table
QPJOB_WA Indicates that an
ITFOR_ABO asynchronous
RT automatic statistics
update was canceled
by a call to KILL
when it was running.
The update has now
completed but is
suspended until the
terminating thread
message
coordination is
finished. This is an
ordinary but rare
state, and should be
very short. A good
value is less than
one second.
QRY_MEM_ True Occurs when Query
GRANT_INF Execution memory
O_MUTEX management tries to
control access to
static grant
information list. This
state lists
information about
the current granted
and waiting memory
requests. This state
is a simple access
control state. There
should never be a
long wait for this
state. If this mutex
is not released, all
new memory-using
queries will stop
responding.


Wait information
stats
Table
QUERY_NO True Occurs during
TIFICATIO synchronization of
N_MGR_MU the garbage
TEX collection queue in
the Query
Notification
Manager.
QUERY_NO True Occurs during state
TIFICATIO synchronization for
N_SUBSCR transactions in
IPTION_MU Query Notifications.
TEX
QUERY_NO True Occurs during
TIFICATIO internal
N_TABLE_ synchronization
MGR_MUTE within the Query
X Notification
Manager.
QUERY_NO True Internal Only.
TIFICATIO
N_UNITTES
T_MUTEX
QUERY_OP False Occurs during
TIMIZER_P synchronization of
RINT_MUTE production of query
X optimizer diagnostic
output. This wait
type only occurs if
diagnostic settings
have been enabled
under direction of
Microsoft Product
Support.
QUERY_TR True Internal Only.
ACEOUT
RECOVER_ True Occurs during
CHANGEDB synchronization of
database warm
standby databases.


Wait information
stats
Table
REPL_CAC True Occurs during
HE_ACCES synchronization on a
S replications article
cache. During these
waits the replication
log reader stalls and
DDL on a published
table is blocked.
REPL_SCHE Yes Occurs during
MA_ACCES synchronization on a
S replications article
cache. During these
waits the replication
log reader stalls and
DDL on a published
table is blocked
REPLICA_W True Occurs while a task
RITES waits for page writes
to database
snapshots or DBCC
replicas to finish.
REQUEST_ Occurs when a task
DISPENSER is waiting for all
_PAUSE outstanding I/O to
complete so that I/O
to a file can be
frozen for snapshot
backup.
RESOURCE Occurs during Synchronization object
_QUEUE synchronization on
various internal
resource queues.


Wait information
stats
Table
RESOURCE True Occurs when a query See SQL Memory Mgr
_SEMAPHO memory request performance counters
RE cannot be granted 1. Memory Grants
immediately because Pending
of other concurrent
2. Memory Grants
queries. High waits
Outstanding
and wait times can
indicate excessive
number of
concurrent queries
or excessive
memory request
amount.
COMMON for DSS
like workload and
large queries such
as hash joins; must
wait for memory
quota (grant) before
it is executed.
RESOURCE True Occurs while a query
_SEMAPHO waits for its request
RE_MUTEX for a thread
reservation to be
fulfilled. It also
occurs when
synchronizing query
compile and memory
grant requests
RESOURCE True Occurs when the
_SEMAPHO number of
RE_QUERY concurrent query
_COMPILE compiles hit a
throttling limit in
order to avoid over-
burdening the
system with
compiles. High waits
and wait times can
indicate of excessive
compilations,
recompiles or
uncachable plans.


Wait information
stats
Table
RESOURCE True Occurs when
_SEMAPHO memory request by
RE_SMALL_ small query cannot
QUERY be granted
immediately because
of other concurrent
queries. Wait time
should not exceed
several seconds
because the server
transfers the request
to the mainquery
memory pool if it
cannot grant the
requested memory
within a few
seconds. High waits
can indicate too
many concurrent
small queries when
the main memory
pool is blocked by
waiting queries.
SEC_DROP True Occurs after failed
_TEMP_KEY attempt to drop a
temporary security
key before a retry
attempt.
SERVER_ID True Occurs during
LE_CHECK synchronization of
an instance of
SQL Server idle
status when a
resource monitor is
trying to declare an
instance of
SQL Server as idle
or trying wake it up.


Wait information
stats
Table
SLEEP_BPO True Occurs during
OL_FLUSH checkpoints when
checkpoint is
throttling the issuing
of new I/Os in order
to avoid flooding the
disk subsystem.
SLEEP_SYS True Occurs during start
TEMTASK of background task
while waiting for
tempdb to complete
startup.
SLEEP_TAS True Occurs when a task
K sleeps while waiting
for a generic event
to occur.
SNI_HTTP_ True Internal Only.
ACCEPT
SNI_HTTP_ True Occurs during
WAITFOR_ SQL Server
0_DISCON shutdown while
waiting for
outstanding http
connections to exit.
SOAP_REA True Occurs when waiting
D for an HTTP network
read to finish.
SOAP_WRI True Occurs when waiting
TE for an HTTP network
write to finish.
SOS_CALL True Occurs when
BACK_REM synchronization on a
OVAL callback list in order
to remove a
callback. It is not
expected for this
counter to change
after server
initialization is
completed


Wait information
stats
Table
SOS_LOCA True Occurs during
LALLOCATO internal
RLIST synchronization in
the SQL Server
memory manager.
SOS_OBJE True Occurs during
CT_STORE internal
_DESTROY synchronization in
_MUTEX memory pools when
destroying objects
from the pool
SOS_PROC True Occurs during
ESS_AFFIN synchronizing of
ITY_MUTEX access to process
affinity settings
SOS_RESE True Occurs during
RVEDMEMB internal
LOCKLIST synchronization in
the SQL Server
memory manager.
SOS_SCHE True Occurs when a task
DULER_YIE voluntarily yields the
LD scheduler for other
tasks to execute.
During this wait the
task is waiting for its
quantum to be
renewed.
SOS_STAC True Occurs during
KSTORE_IN synchronization of
IT_MUTEX internal store
initialization.


Wait information
stats
Table
SOS_SYNC True Occurs when a task
_TASK_EN is started in a
QUEUE_EV synchronous
ENT manner. Most tasks
in SQL Server 2005
are started in an
asynchronous
manner and control
returns to the
starter immediately
after the task
request has been
put on the work
queue.
SOS_VIRTU True Occurs when a
ALMEMORY memory allocation
_LOW waits for a resource
manager to free
virtual memory.
SOSHOST_ SOS True Occurs when a
EVENT hosted component,
such as CLR, waits
for a SQL Server
2005 event
synchronization
object.
SOSHOST_ SOS True Occurs during
INTERNAL synchronization of
memory manager
callbacks used by
hosted components,
such as CLR.
MUTEX hosted component,
such as CLR, waits
for a SQL Server
2005 mutex
synchronization


Wait information
stats
Table
RWLOCK hosted component,
such as CLR, waits
for a SQL Server
2005 reader-writer
synchronization
SEMAPHOR hosted component,
E such as CLR, waits
for a SQL Server
2005 semaphore
synchronization
object
SLEEP hosted task sleeps
when waiting for a
generic event to
occur Hosted tasks
are used by hosted
components such as
CLR.
SOSHOST_ SOS True Occurs during
TRACELOC synchronization of
K access to trace
streams.
WAITFORD hosted component,
ONE such as CLR, waits
for a task to finish.
SQLCLR_AP CLR True Occurs while CLR
PDOMAIN waits for an
application domain
to complete startup
SQLCLR_A CLR True Occurs while waiting
SSEMBLY for access to the
loaded assembly list
in the sql appdomain
SQLCLR_D CLR True Occurs while CLR
EADLOCK_ waits for deadlock
DETECTION detection to finish.


Wait information
stats
Table
SQLCLR_Q CLR True Occurs when a CLR
UANTUM_P task is throttled
UNISHMEN because it has
T exceeded its
execution quantum.
This throttling is
done in order to
reduce the effect of
this greedy task on
other tasks.
SQLSORT_ True Occurs during
NORMMUTE internal
X synchronization
when initializing
internal sorting
structures.
SQLSORT_ True Occurs during
SORTMUTE internal
X synchronization
when initializing
internal sorting
structures.
SQLTRACE True Occurs when the
_BUFFER_F SQL Trace flush task
LUSH pauses between
flushes. This wait is
expected and long
waits do not indicate
a problem
SQLTRACE True Occurs during
_LOCK synchronization on
trace buffers during
a file trace.
SQLTRACE True Occurs when a trace
_SHUTDO shutdown waits for
WN outstanding trace
events to finish
SQLTRACE True Occurs when a SQL
_WAIT_EN Trace event queue
TRIES waits for packets to
arrive on the queue.


Wait information
stats
Table
SRVPROC_ True Occurs when the
SHUTDOW shutdown process
N waits for internal
resources to be
released to
shutdown cleanly.
TEMPOBJ True Occurs when
temporary object
drops are
synchronized. This
wait is rare and only
occurs if a task has
requested exclusive
access for temp
table drops.
THREADPO True Occurs when a task
OL is waiting for a
worker to run on.
This can indicate
that the max worker
setting is too low or
that batch
executions are
taking unusually
long therefore
reducing the number
of worker available
to satisfy other
batches.


Wait information
stats
Table
TRAN_MAR True Occurs when waiting
KLATCH_D for a destroy mode
T latch on a
transaction mark
latch. Transaction
mark latches are
used for
synchronization of
commits with
marked
transactions. Marked
transaction enable
restore to specific
marked
transactions.
KLATCH_E for an exclusive
X mode latch on a
transaction mark
latch. Transaction
mark latches are
used for
synchronization of
commits with
marked
transaction enable
restore to specific
marked
transactions.


Wait information
stats
Table
KLATCH_KP for a keep mode
latch on a
transaction mark
latch. Transaction
mark latches are
used for
synchronization of
commits with
marked
transactions enable
restore to specific
marked
transactions.
TRAN_MAR True Internal Only.
KLATCH_NL
KLATCH_S for a share mode
H latch on a
transaction mark
latch. Transaction
mark latches are
used for
synchronization of
commits with
marked
transactions enable
restore to specific
marked
transactions.


Wait information
stats
Table
KLATCH_U for an update mode
P latch on a
transaction mark
latch. Transaction
mark latches are
used for
synchronization of
commits with
marked
transactions enable
restore to specific
marked
transactions.
TRANSACTI True Occurs during
ON_MUTEX synchronization of
access to a
transaction by
multiple batches.
UTIL_PAGE True Occurs when
_ALLOC transaction log scans
wait for memory to
be available during
memory pressure.
VIEW_DEFI True Occurs during
NITION_M synchronization on
UTEX access to cached
view definitions.
WAIT_FOR True Occurs when waiting
_RESULTS for a query
notification to be
triggered.
WAITFOR True Occurs because of a Inspect Transact-SQL
WaitFor Transact- code for “waitfor delay”
SQL statement. The statement
duration of the wait
is determined by the
parameters to the
statement. This is a
user initiated wait.


Wait information
stats
Table
WORKTBL_ True Occurs when
DROP pausing before
retrying after a
failed worktable
drop.
WRITELOG Occurs when waiting See Disk performance
for a log flush to counters:
finish. Common 1. Disk sec/read
operations that
2. Disk sec/write
cause log flushes are
checkpoints and 3. Disk queues
transaction commits. See SQL Buffer Manager
Identify disk counters:
bottlenecks, by 1. Page Life Expectancy
using Performance 2. Checkpoint pages/sec
Counters, Profiler,
3. Lazy writes/sec
sys.dm_io_virtual_fil
Check Io_stall for tranlog
e_stats and
SHOWPLAN • select * from
Any of the following sys.dm_io_virtual_file
_stats(dbid,file#)
reduces these waits:
1. Adding additional
IO bandwidth,
2. Balancing IO
across other
drives
3. Moving or
Isolating the
transaction log
on its own drive
XACT_OWN True Occurs when waiting
_TRANSAC to acquire ownership
TION of a transaction.
XACT_RECL True Occurs when waiting
AIM_SESSI for the current
ON owner of a session
to release ownership
of the session.


Wait information
stats
Table
XACTLOCKI True Occurs during
NFO synchronization of
access to a
transaction's list of
locks. In addition to
the transaction
itself, a transactions
list of locks is
accessed by
operations such as
deadlock detection
and lock migration
during page splits.
XACTWORK True Occurs during
SPACE_MU synchronization of
TEX defections from a
transactions in
addition to the
transfer of database
locks between enlist
members of a
transaction.
QUEUES (Perfmon Counters)

The Queues aspect of the Waits and Queues approach to performance analysis refers to
PERFMON counters and other sources of information that measures resource usage.
PERFMON counters provide a view of system performance from a resource perspective.
PERFMON Counters, correlation, possible

conclusions and actions
Resource Component Disk
Perfmon Object: Physical Disk
Counters to Description Possible conclusions / actions

Monitor
Current Disk Sustained high queues mean Confirm IO issues with disk sec/read
Queue your IO subsystem is not and disk sec/write.
Length keeping up. Waitstats correlation:
1. IO_COMPLETION
2. ASYNC_IO_COMPLETION

3. WRITELOG
4. LOGMGR
Average Average of disk queues over Confirm IO issues with disk sec/read
Disk Queue time. If this number is and disk sec/write. Waitstats
Length consistently high, disk correlation:
sec/read and disk sec/write is 1. IO_COMPLETION
also high indicating IO
bandwidth issues.
3. WRITELOG
4. LOGMGR
Disk Under typical circumstances, If disk sec/read > normal read time
Sec/Read reads should take 4-8 ms (ask vendor for typical read time) you
(confirm with hardware vendor can consider the following options:
for exact read time). Sustained 1. Resolve IO bottleneck by adding
queues skew this number more drives; spreading IO across new
higher because disk sec/read drives if it is possible. For example,
factors in the effects of disk move files such as database,
queues. High numbers mean transaction log, other application files
your IO subsystem is not that are being written to or read from.
keeping up with requests
2. Check for memory pressure, see
Check individual drive memory component.
performance if there are
3. Check for appropriate indexing of
multiple drives. If it is a broad
SQL tables. Correct indexing can save
problem affecting all drives,
IO. Check SQL query plans looking for
the IO subsystem is not
scans and sorts, and so on. Showplan
keeping up. More drives could
identifies sorting steps.
be useful. If there is ONE very
hot drive, examine disk activity 4. Run SQL Profiler to identify
such as location of paging file, Transact-SQL statements doing scans.
database, transaction log, and In Profiler, select the scans event
other read/write activity. class and scan stopped event. Click
the data column tab and add object
Id. Run the trace. Save the profiler
trace to a trace table, and then search
for the scans event. Alternatively, you
can search for high duration, reads,
and writes.
Waitstats correlation:
1. IO_COMPLETION
3. WRITELOG
4. LOGMGR
Disk Under typical circumstances, See disk sec/read.
Sec/Write reads should take 4-8 ms High performance (significant insert,
(confirm with hardware update, and delete activity) requires
vendor). Sustained queues the transaction log to be on a
skew this disk sec/write higher

because this counter factors in separate drive from the database.

the effects of disk queues. Waitstats correlation:
High numbers mean your IO
1. IO_COMPLETION
subsystem is not keeping up
with requests. In some SAN 2. ASYNC_IO_COMPLETION
environments, writes can be as 3. WRITELOG
low as 1-2 ms. 4. LOGMGR
Resource Component: Memory / Cache

Perfmon Object: Memory
Counters Description Possible conclusions /

to actions
Monitor
Page Includes both hard faults (those that Check for memory pressure
Faults/sec require disk access) and soft faults (see SQL Server buffer
(where the faulted page is found manager), low data page hit
elsewhere in physical memory.) Most rates, and memory grants
processors can handle a large numbers pending.
of soft faults without significant
consequences. However, hard faults that
require disk access can cause significant
delays. See the disk component for
more information.
Pages/sec Number of pages read from or written to Compare with Page Faults/sec.
disk to resolve hard page faults. Check for memory pressure
These are hard faults that require (see SQL Server buffer
physical IO to fetch the page. manager), low data page hit
rates, and memory grants
pending.
Resource Component: CPU

Perfmon Object: Processor
Counters Description Possible conclusions / actions

to
Monitor
% User Percentage of time SQL Server Make sure % user time >70%. Check
Time runs in User mode. Privileged task manager (taskmgr.exe) to see how
mode is designed for operating much CPU sqlserver.exe is getting. If
system components and user time <70%, check on %Processor
enables direct access to Time and % Privileged activity.
hardware and all memory.
% The operating system switches Should be <20%. Check task manager
Privileged application threads to privileged (taskmgr.exe) to see how much CPU
Time mode to access operating sqlserver.exe is getting. If % privileged
time >20%, check on % Processor Time

system services and % User Time.

% Percentage of time the CPU is Common uses of CPU resources:
Processor executing over sample interval. 1. Compilation and recompilation use
Time CPU resources. Plan reuse and
parameterization minimizes CPU
consumption because of compilation.
For more information about compilation,
recompilation, parameterization, and
plan reuse, see
http://www.microsoft.com/technet/prod
technol/sql/2005/recomp.mspx
Plan reuse is where usecounts are > 1
Select cacheobjtype, objtype,
usecounts, or refcounts from
sys.dm_exec_cached_plans and order
by usecounts
Matches to PERFMON counters:
1. System: Processor Queue length
2. SQL Statistics: Compilations/sec
3. SQL Statistics: Re-Compilations/sec
4. SQL Statistics: Requests/sec
If both of the following are true, you are
CPU bound:
1. Proc time >85% on average
2. Context switches (see system object)
>20K / sec
Light weight pooling can provide a 15%
boost. Light weight pooling (also known
as fiber mode) divides a thread into 10
fibers. Overhead per fiber is less than
that of individual threads.
% Idle Percentage of time CPU is idle
Time over sample interval
Interrupts Interrupts/sec is the average Correlate with other perfmon counters
/sec rate, in incidents per second, at such as IO, Network.
which the processor received
and serviced hardware
interrupts.
Resource Component Thread

Perfmon Object: Process
Counters Description Possible conclusions /

to actions
Monitor

Page This counter includes both hard faults Check for memory pressure
Faults/sec (those that require disk access) and soft (see SQL Server buffer
faults (where the faulted page is found manager), low data page hit
elsewhere in physical memory.) Most rates, and memory grants
processors can handle large numbers of pending, page life expectancy.
soft faults without significant
consequences. However, hard faults,
which require disk access, can cause
significant delays. See the disk
component for more information.
Resource Component: System

Perfmon Object: System

Monitor
Processor Queue Number of threads waiting to be scheduled for CPU
Length time. Some common uses of CPU resources that
can be avoided:
1. Unnecessary compilation and recompilation.
Parameterization and plan reuse would reduce CPU
consumption. See
http://www.microsoft.com/technet/prodtechnol/sql
/2005/recomp.mspx
2. memory pressure
3. lack of appropriate indexing
Context
Switches/sec
Resource Component: SQL Server

Perfmon Object: SQLServer:Access Method

to Monitor
Forwarded Number of records fetched Look at code to determine where the
Records/sec through forwarded record short row is inserted followed by an
pointers. update.
Tables without a clustered Can be avoided by:
index. If you start with a short 1. Using Default values (so that an
row, and update the row update does not result in a longer row
creating a wider row, the row that is the root cause of forwarded
might not fit on the data page. records).
A pointer is put in its location
2. Using Char instead of varchar (fixes
and the row is forwarded to
length so that an update does not
another page.
result in a longer row

Full The entire table or index is SQL Profiler can be used to identify
Scan/sec scanned. Scans can cause which Transact-SQL statements do
excessive IO if an index would scan. Select the scans event class and
be useful. events scan:started and
scan:completed. Include the object Id
data column. Save the profiler trace to
a trace table, and then search for the
scans event.
The scan:completed event provides
associated IO so that you can also
search for high reads, writes, and
duration.
Index Number of index searches. Compare to Full Scan/sec. You want to
Searches/ Index searches are used to see high values for index searches.
sec start range scans, single index
record fetches, and to
reposition in an index.
Page Number of page splits Page splits are extra IO overhead that
Splits/sec occurring as the result of index results from random inserts.
pages overflowing. Typically When there is no room on a data page,
associated with leaf pages of and the row must be inserted on the
clustered indexes and non- page (because of index order), SQL
clustered indexes. splits the page by inserting a new row
and moving the rests of the rows to a
new page.
Compare to Disk: page sec/write. If
this is very high, you might want to
reorganize the indexes on the tables
causing the page splits, to reduce page
splits temporarily. Fillfactor can be
used to leave space for inserts.

Perfmon Object: SQLServer:Memory Mgr

to
Monitor
Memory Memory resources are Compare with Memory grants outstanding.
Grants required for each user If grants pending increases, you can do the
Pending request. If sufficient following:
memory is not available, 1. add more memory to SQL Server
the user waits until there is
2. add more physical memory to the box.
adequate memory for the
query to run. 3. check for memory pressure. See and
correct indexing if you experience “out of
memory” conditions.

Correlate to Waittype
1. RESOURCE_SEMAPHORE

Perfmon Object: SQLServer:Buffer Manager

to Monitor
Buffer Percentage of time that the Check for memory pressure. See
cache hit pages requested are Checkpoint pages/sec, Lazy writes/sec and
ratio already in cache Page life expectancy.
Checkpoint Pages written to disk during Memory pressure is indicated if this
pages/sec the checkpoint process. counter is high together with high lazy
This frees the SQL cache writes/sec and low page life expectancy
(<300 seconds)
Lazy Pages written to disk by the Memory pressure is indicated if this
writes/sec lazy writer, This frees the counter is high together with high lazy
SQL cache writes/sec and low page life expectancy
(<300 seconds)
Page life Time in seconds the data Memory pressure is indicated if this
expectancy pages, on average, stay in counter is low (<300) together with high
SQL cache. Low page life lazy writes/sec and checkpoint pages/sec.
<300 can indicate (1) SQL Check for missing indexes and bad query
cache is cold, (2) memory plans (scans in profiler)
problems or (3) missing
Check for high page faults/sec.
indexes. Correlate to Lazy
writes/sec and Checkpoint
pages/sec
Readahead If there are memory Correlate to counters for SQL buffer mgr:
pages/sec shortages, a cold cache, or buffer cache hit ratio, page life expectancy,
low hit rates, SQL Server lazy writes, and checkpoint pages for
might use worker threads memory pressure.
to readahead (bring in Check for appropriate indexing and bad
pages beforehand) to raise query plans (scans in profiler)
hit rates. By itself
readahead is not a problem
unless users are flushing
each other’s pages
consistently.

Perfmon Object: SQLServer:Plan Cache

to
Monitor

Cache Hit Percentage of time Check for memory pressure. See Checkpoint
Ratio that the procedure pages/sec, Lazy writes/sec and Page life expectancy.
plan pages are See SQL Profiler: Stored Procedure: CacheHit,
already in cache. CacheMiss, and CacheInsert to see what stored
For example,. procedure query plans are already in cache (Hit) vs.
procedure cache those not in cache (Miss,Insert)
hits. That is, how
Check for appropriate plan reuse. See section “Query
frequently a
Plan Reuse”. It is frequently desirable for query plans
compiled
to be reused for similar SQL although not always.
procedure is found
in the procedure See SQL Statistics: Compilations/sec for discussion
cache (therefore of plan reuse.
avoiding the need If there is memory pressure, plans are discarded to
to recompile). make room for other data or procedure plans.

Perfmon Object: SQLServer:Databases

Monitor
Log Flush Waiting for transaction See disk performance counters
Wait Time log writes (ms) Check transaction log file
sys.dm_io_virtual_file_stats(dbid, file#) for
Io_stall (waits in ms)
Log Flush This is the number of See disk performance counters,
Waits/sec commits waiting on a sys.dm_io_virtual_file_stats for Io_stall.
log flush.
Log Growths Microsoft Windows® Generally, growths of the transaction log
automatically extends temporarily freezes writes to the transaction
the transaction log to log when Windows extends the transaction
accommodate insert, log file. Check to see that the extend
update, and delete increment is large enough. If not,
activity. performance will decrease when the log is
extending frequently.
Transactions SQL Server transactions
/sec per second

Perfmon Object: SQLServer:General Statistics

to Monitor
Logins/sec Number of logins per User connections
second
Logout/sec Number of logouts per

second
User Number of user
connections connections

Perfmon Object: SQLServer:Latches

to
Monitor
Average Latches are short term light If high, check PERFMON DISK and MEMORY
Latch Wait weight synchronization objects for
Time(ms) object. Latches are not held 1. IO bottlenecks
for the duration of a
2. memory pressure
transaction. Typical latching
operations during row Typically reduced with more memory or IO
transfers to memory, capacity
controlling modifications to
row offset table, and so on.
Latch See Average Latch Wait
Waits/sec Time(ms)
Total Short term light weight If high, check PERFMON DISK and MEMORY
Latch Wait synchronization object. objects for
Time(ms) Latches are not held for the 1. IO bottlenecks
duration of a transaction.
2. memory pressure
Typical latching operations
during row transfers to Typically reduced with more memory or IO
memory, controlling capacity
modifications to row offset
table, and so on.

Perfmon Object: SQLServer:Locks

to
Monitor
Average Transactions should be as Hint: check for memory pressure, which
Wait short as possible to limit causes more physical IO, therefore
Time(ms) the blocking of other users. prolonging the duration of transactions and
locks.
Lock Wait Transactions should be as Hint: check for memory pressure, which
Time(ms) short as possible to limit causes more physical IO, therefore
the blocking of other users. prolonging the duration of transactions and
locks

Lock Transactions should be as Hint: check for memory pressure, which

Waits/sec short as possible to limit causes more physical IO, therefore
the blocking of other users. prolonging the duration of transactions and
locks

Perfmon Object: SQLServer:SQL Statistics
Counters to Possible conclusions / actions

Monitor /
Description
SQL Compilations/sec Before a SQL statement can be executed, the query optimizer
must have created a query plan. A query plan consists of
steps that return the results for a given SQL statement.
For more information about SQL Server 2005 Optimizer and
Statistics refer to
http://www.microsoft.com/technet/prodtechnol/sql/2005/qryst
ats.mspx
Compilations/sec Includes both initial compiles and subsequent
recompiles. Compilation and recompilation are CPU intensive
operations.
Unnecessary compilation can sometimes be avoided with query
plan reuse. Check for appropriate plan reuse. See section
“Query Plan Reuse”. It is frequently desirable for query plans
to be reused for similar SQL although not always.
Parameterization is important for plan reuse. In addition, some
types of recompilation can be avoided. See the
SQL Server 2005 recompilation paper for more information:
http://www.microsoft.com/technet/prodtechnol/sql/2005/reco
mp.mspx
To obtain initial compilations only, you must subtract
recompilations/sec from compilations/sec.
Compare to batch requests/sec to see extent of compilation.
SQL Re- Only contains recompiles. SQL Profiler can provide information
Compilations/sec about what processors are recompiling, what statement, and
the reason for recompilation. In Profiler, select the stored
procedure event class and SP:recompilation event, and include
the data column eventsubclass. Review the trace searching for
eventsubclass values 1 through 6. The previous statements
caused the recompilation. For more information about
recompilation, see
mp.mspx
Batch Requests/sec Total batch requests should be compared with compilations/sec
Auto-Param Auto-param tries should be compared to failed auto-
params/sec. Appropriate parameterization is important for plan

Attempts/sec reuse. In some cases, Sp_executeSQL could be used with ad

hoc SQL. For more information about recompilation, see
mp.mspx
Failed Auto- Auto-param tries should be compared to failed auto-
Params/sec params/sec. Appropriate parameterization is important for plan
reuse. In some cases, Sp_executeSQL could be used with ad
hoc SQL. For more information about recompilation, see
mp.mspx
Interesting PERFMON Ratios and comparisons

Some counters in PERFMON have to be compared to other counters to draw the
appropriate conclusion Although the following ratios and comparisons are not
exhaustive, they nevertheless point you in the right direction.
1. Batch requests/sec versus SQL Compilations/sec. Plan reuse is desirable for
OLTP workloads because of its tendency toward high volumes of identical
transactions. See section titled "Plan reuse and Performance Counters". The worst
case is when compilations are very high compared with batch requests as this could
be a case of memory pressure where query plans are discarded quickly to make
room for other activity. Another possibility is lack of parameterization, which is
important for ad hoc SQL plan reuse. Parameterization is where variables are used
instead of literal values. sp_executeSQL can be used to parameterize ad hoc SQL.
Perfmon counters are SQLServer:SQL Statistcs:Batch Requests/sec and
SQLServer:SQL Statistics:SQL Compilations/sec
For more information about SQL Server 2005 Optimizer and Statistics see
http://www.microsoft.com/technet/prodtechnol/sql/2005/qrystats.mspx
2. SQL Compilations/sec versus SQL Re-Compilations/sec. SQL Compilations/sec
include all compilations (initial compiles and recompiles) but SQL Re-
Compilations/sec only includes recompiles (excludes initial compiles). See section
titled "Plan reuse and Performance Counters". If initial compiles are low (SQL
Compilations – SQL Re-Compilations) compared to SQL recompilations, there is a
probable recompilation problem. To find the actual statements that are recompiled,
see section titled “Query Plans and DMVs".
Perfmon counters are SQLServer:SQL Statistcs: SQL Compilations/sec and
SQLServer:SQL Statistics:SQL Re-Compilations/sec.
For more information, see the SQL Server 2005 recompilation paper:
http://www.microsoft.com/technet/prodtechnol/sql/2005/recomp.mspx
3. Kernel CPU versus User CPU. If (Kernel CPU/ User CPU) > .25, this can indicate a
network, disk driver, or hardware issue. Network and Disk IO is serviced in kernel
mode. SQL is serviced in user mode. Look at Task Manager.
Perfmon counters are Processor:%Processor Time, Processor:%User time, and
Processor:%Interrupt time.
4. Context switches per second. This value should remain under 20000. Values
greater than this threshold indicate too much context switching. Values greater
than 50000 indicate that the server might be spending more CPU clock cycles

context switching threads instead of actively running given threads. This kind of
scenario was visible on Microsoft Windows 2000 Server. In Windows Server® 2003
it is less likely but can still occur.
5. Disk Queue Length versus Disk sec/Transfer. As disk queue length increases,
so does disk sec/transfer.
Performance monitor counters are PhysicalDisk:Avg Disk Queue Length and
PhysicalDisk:Avg Disk sec/Transfer.
6. Page life expectancy, checkpoint pages/sec, lazy writes/sec comparison.
Memory pressure is indicated with low page life expectancy, and high checkpoint
pages and lazy writes/sec. Memory pressure, which adversely affects performance,
can be lessened one or more of the following
a. Adding more memory to the box
b. Increasing SQL memory
c. Avoiding table and index scans with appropriate indexing
7. SQL buffer cache hit ratio. If this is consistently under 90% it indicates that the
buffer cache is getting flushed.
8. Signal waits and the runnable queue comparison. Basics of execution model
(simplified) is as follows.
a. If a session id is running and must lock a resource that is unavailable at the
time, it moves to the resource wait list (time T0).
b. A signal indicates resource available, SPID moves to runnable queue at time T1.
c. SPID waits running status until T2 as CPU goes through the runnable queue in
order of arrival
d. The resource wait time is the actual time waiting for the resource to be
available, Time T0 to T1.
e. The signal wait time spent in the runnable queue. It starts from the time the
resource is available (T1) to the point in which the process is running again at T2.
Therefore, signal waits are T2-T1.
The larger the value T1 – T0 = means that the particular resource availability is
tending to be more limited. If it is a lock that is waiting then blocking could likely
be occurring. If it is IO that is waiting then the disk subsystem is likely to be
bottlenecking.
T2 – T1 indicates CPU pressure. This indicates that as the value increases the
time that is spent in the runnable queue also increases. Session_ids in the
runnable queue are waiting only for CPU resource. If this is say >25%, there is a
CPU bottleneck.
f. Key questions: Are Resource and Signal time significant?
• Highest waits indicate the bottleneck you need to solve for scalability
• Generally if you have LOW% SIGNAL WAITS, the CPU is handling the
workload. For example, session_ids move through runnable queue quickly
• HIGH % SIGNAL WAITS indicates CPU cannot keep up, significant time for
SPIDs to move up the runnable queue to reach running status
9. Network: Current bandwidth, bytes total/sec, packets/sec. Network
bandwidth issues should be corroborated with bytes total/sec. [Network interface:

bytes total/sec] / [Network interface: Current Bandwidth] > .6, possible network
bottleneck.
10. Page Faults/sec versus Pages/sec. Page faults include both hard faults (those
that require disk access) and soft faults (where the faulted page is found elsewhere
in physical memory). Most processors can handle large numbers of soft faults
without significant consequences. However, hard faults, which require disk access,
can cause significant delays. Pages/sec represents the number of hard page faults
that require physical IO to bring the pages into memory.
Memory Issues
The SQL Server relational database system uses memory for many different purposes
internally. For a complete discussion of the memory uses in SQL Server, see
http://www.winnetmag.com/Article/ArticleID/43419/43419.html.
To summarize, the main uses are as follows:
1. Database page cache, which is used to cache database (table / index) pages
2. Query Workspace memory, which is used by memory intensive query operations
such as Hash and Sort.
3. Plan cache, which is used to cache query plans so that they can be reused
4. Other; such as locks, connection memory, thread stacks, memory for utilities such
as backup/restore and so on.
5. Memory used by other components linked into the SQL Server process such as XPs,
OLE-DB providers and so on. The memory is typically referred to as the
MemToLeave memory area because SQL Server refrains from allocating this
memory so that these other components linked into the process can do so.
Comparison of 32-bit memory architecture vs. 64-

bit flat memory
Of these, only the first use, database page cache, can use AWE memory on 32-bit
systems. The rest of the uses require virtual memory, therefore they are limited to 2GB
(or 3GB with the /3GB switch in boot.ini) on 32-bit systems. If an application stresses
one or more of these other uses of memory in SQL Server to a point beyond what can
be handled by the 32-bit virtual memory limits, you might consider the 64-bit option.
To determine whether this is the case, see the following for some steps that you might
want to consider:
1. Overall Server Memory: Look at counters under SQL Server:Memory Manager. If
Total Server Memory is well below Target Server Memory at steady state, it tells
you that the server is not experiencing memory pressure. In this case, you likely
have no performance-related reason to consider SS64. Otherwise, you need to look
further into the cause of memory pressure by following the following steps. Of
course, if you do have memory pressure, you might already be using additional
memory in SQL Server by enabling AWE. In this case, SQL Server allocates Max
Server Memory at startup and therefore Total Server Memory does not change
dynamically. Additionally in this case, follow these steps to tell if there continues to
be memory pressure.
2. Also, examining the buffer cache hit ratio would be good. Typically one of the first
signs of memory problems.

3. Query Workspace Memory: Look at counters under SQL Server:Memory Manager.

Look at Memory Grants Outstanding and Memory Grants Pending. If you see a long
queue of Pending grants compared to Outstanding grants, there is likely memory
pressure because of query workspace memory. You can confirm this by checking
the Granted Workspace Memory (KB) counter that tells you how much memory has
currently been granted to running queries. If there is memory pressure because of
workspace memory, this value should be at least 25% of the virtual memory
available to SQL Server. If the memory pressure is severe, the server might even
return errors such as 701 or 8645. If this is the case, this might be a good reason to
consider using SS64.
4. Plan Cache: The counter SQL Server:Buffer Manager:Procedure Cache Pages
captures the total number of pages in the plan cache. If this number is a significant
fraction (typically, greater than 25 percent) of the total number of pages in the
buffer pool, the application is plan cache intensive. However, this by itself is
insufficient to consider a move to SS64. If the plan cache is large because it is full of
plans that are rarely reused, moving to SS64 does not yield any benefits (and in
fact might make the situation worse because of the larger size of plan cache as
described previously). To determine what kinds of plan are in the plan cache see the
section titled “Query Plan Reuse” to see whether these plans are being reused,
examine the usecounts column. If the plan cache is full of plans that are being
reused and yet there is memory pressure, this indicates the application would
benefit from more virtual memory and therefore SS64 might be a good option to
consider.
5. Memory pressure on MemToLeave, for example, because of XPs or OLE-DB
providers. Typically, if you have pressure in the MemToLeave area, you might see
errors such as 7399, 17802, or 17803. In these cases, you might have already
considered altering the –g startup parameter for SQL Server to increase the
MemToLeave value. This in turn might translate to some of the other kinds of
memory pressure described here.
6. High CPU cost of AWE memory: In some cases, even if the memory use consists
primarily of database page cache, the CPU cost of mapping and un-mapping
database pages using AWE might become too expensive, as evidenced by high
kernel CPU time. This is especially true when the number of CPUs in the system is 8
or more and/or when the size of physical memory exceeds 32GB on your 32-bit
system. This is another point at which you might consider use of a 64-bit system.
64-bit flat memory vs. higher 32-bit clock speeds

As seen earlier, there are cases where memory pressure is genuine and SS64 might be
an attractive option in those cases. However, even in these cases the choice of a 64-bit
system over 32-bit systems is not straight-forward. Clock speeds on Itanium-based 64-
bit systems are much lower than on Xeon-based 32-bit systems. The Itanium’s ability
to execute multiple instructions at the same time does compensate for this to a degree.
However, if your application is CPU-heavy, you might find you need as many or more
processors on a 64-bit system as on the comparable 32-bit system to handle the same
workload. It is always recommended that the relative performance of the two choices
be verified through a prototype or proof of concept to verify that the 64-bit platform
would be a good investment.

Application Design issues

There are application design considerations resulting from the Waits and Queues
methodology. The following table describes some of the application design implications.
Observation Application issue Possible remedies

High IO waits Database design Bad query plans resulting from inappropriate
Memory pressure indexing.
Add correct indexes to minimize IO.
Add more memory
High CPU use Memory pressure Check for correct plan reuse, parameterization,
Plan reuse recompilation, see
http://www.microsoft.com/technet/prodtechnol/s
Parameterization
ql/2005/recomp.mspx
High blocking Transaction Redo transaction management
/ locking management Use correct transaction isolation levels
Recommendations
The Waits and Queues methodology is recommended as the most effective technique
for identifying and resolving performance issues. It focuses on the best opportunities to
improve performance, the so called “biggest bang for the buck”. These performance
improvements are likely to have a significant return on the performance tuning time
investment.
Conclusion
There are two complimentary sources of performance information for SQL Server. Wait
types are an invaluable clue in analyzing overall system performance from an
application point of view. Wait types provide a view of system performance from a SQL
thread standpoint while Performance Monitor provides a view of system performance
from a resource standpoint.
Wait statistics should be corroborated or associated with resource counters in
Performance Monitor. For example, a high SQL Server wait types signal the need for
additional PERFMON investigation of underlying resources such as processor, IO
subsystem, network and so on. Together, these associations or correlations of wait
types to performance counters, and other related counter ratios provide a broad picture
of application performance.
In come cases, the experienced performance expert must look beyond the symptom to
find the root problem. Although not exhaustive, the correlated performance
information, possible conclusions and actions, and interesting ratios and comparisons
sections shed light on actual root problems, given the symptoms. The waits and queues
methodology presented here, identify system bottlenecks and propose additional
corroboration and conclusions, where appropriate.
In sum, the performance methodology of waits and queues draws on the available
performance information that consists of waitstats, PERFMON counters, and correlated

information, to provide a broad profile of application performance. It is an invaluable

tool in pinpointing bottlenecks and fixing performance problems.
For more information:

http://www.microsoft.com/technet/prodtechnol/sql/bestpractice/default.mspx
Did this paper help you? Please give us your feedback. On a scale of 1 (poor) to 5
(excellent), how would you rate this paper?

Performance Tuning Waits Queues

Uploaded by

Copyright:

Available Formats

Performance Tuning Waits Queues

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Performance Tuning Waits Queues

Uploaded by

Copyright:

Available Formats

SQL Server 2005 Waits and Queues

SQL Server Best Practices Article

Writers: Tom Davidson

Technical Reviewer: Sanjay Mishra

Published: November 2006

 2006 Microsoft Corporation. All rights reserved.

Microsoft Corporation ©2006

Audience: Who should read this paper

Waits and Queues: A Performance Methodology

Associations or correlations of wait types to performance counters, and interesting

Microsoft Corporation ©2006

Execution Model (simplified)

Microsoft Corporation ©2006

The status change sequence of events is as follows:

Figure 3: Execution Model – After status change

Microsoft Corporation ©2006

Waiter List and Wait Types

Dynamic Management Views (DMVs) and

Microsoft Corporation ©2006

The stored procedures track_waitstats_2005 and get_waitstats_2005 can be used to

Track_waitstats_2005 stored procedure

Get_waitstats_2005 stored procedure

Microsoft Corporation ©2006

The script location is:

Performance and Tuning Blue Prints

Common scenarios to avoid in OLTP

Rule Description Value Source Problem

Microsoft Corporation ©2006

1 High >4 Sys.dm_exec_sql_text High

Rule Description Value Source Problem

Microsoft Corporation ©2006

Rule Description Value Source Problem Description

Rule Description Value Source Problem Description

Microsoft Corporation ©2006

sec/read Physical Disk pressure.

4 If Top 2 values Top 2 Sys.dm_os_wait_stats If top 2 wait_stats

Rule Description Value Source Problem Description

Microsoft Corporation ©2006

Rule Description Value Source Problem Description

Microsoft Corporation ©2006

Common Scenarios to avoid with DataWarehousing

Rule Description Value Source Problem Description

Microsoft Corporation ©2006

Resource issue: CPU

Rule Description Value Source Problem Description

Resource issue: Memory

Rule Description Value Source Problem Description

Microsoft Corporation ©2006

Look for possible missing

Rule Description Value Source Problem

4 If Top 2 values for wait Top 2 Sys.dm_os_wait_stats If top 2

Resource issue: Blocking

Rule Description Value Source Problem Description

Microsoft Corporation ©2006

Exactly the opposite of OLTP applications, reporting or relational data warehouse

Microsoft Corporation ©2006

Typical resource bottlenecks

Memory pressure and IO subsystem issues

Microsoft Corporation ©2006

Missing or poorly formed indexes