Chapter 1

CHAPTER-1
Query processing and Optimization

 Query Processing: The process by which the query results
are retrieved from a high-level query such as SQL or OQL.
 Query Optimization: The process of choosing a suitable
execution strategy for retrieving results of query from database
files for processing a query is known as Query Optimization.
 Two main Techniques for Query Optimization
 Heuristic Rules: Rules for ordering the operations in query
optimization.
 Systematical estimation: It estimates cost of different execution
strategies and chooses the execution plan with lowest execution
cost
Prepared by Elisaye B. @WSU-DTC 1
Figure 1 Typical steps when processing a high-level query.
Scanning , Parsing , Validating
 Scanner: The scanner identifies the language
tokens such as SQL Keywords, attribute names,
and relation names in the text of the query.
 Parser: The parser checks the query syntax to
determine whether it is formulated according to
the syntax rules of the query language.
 Validation: The query must be validated by
checking that all attributes and relation names are
valid and semantically meaningful names in the
schema of the particular database being queried .
QUERY DATA STRUCTURE
 Before optimizing the query it is represented in an internal or
intermediate form.
 It is created using two data structures (internal representation of
query)
 Query tree: A tree data structure that corresponds to a relational
algebra expression.
 It represents the input relations of the query as leaf nodes of the
tree, and represents the relational algebra operations as internal
nodes.
 Query graph: A graph data structure that corresponds to a
relational calculus expression. It does not indicate an order on
which operations to perform first.
QUERY PROCESSING
• Query Optimization: The process of choosing a suitable
execution strategy for processing a query. This module
has the task of producing an execution plan.
• Query Code Generator: It generates the code to

execute the plan.
• Runtime Database Processor: It has the task of running

the query code whether in compiled or interpreted mode.
If a runtime error results an error message is generated
by the runtime database processor.

Translating SQL Queries into
Relational Algebra
 Query block: The basic unit that can be translated
into the algebraic operators and optimized.
 A query block contains a single SELECT-FROM-

WHERE expression, as well as GROUP BY and
HAVING clause if these are part of the block.
 Nested queries within a query are identified as

separate query blocks.
 Aggregate operators in SQL must be included in the

extended algebra. Prepared by Elisaye B. @WSU-DTC 6
Translating SQL Queries into
Relational Algebra

Translating SQL Queries into Relational
Algebra
Example:-
SQL query
Select emp_name from Employee where salary>10000
Translated Relational Algebra
• σsalary>10000 (πsalary (Employee)) OR
• πsalary (σsalary>10000 (Employee))

Basic Algorithms for Executing Query
Operations
I. Algorithms for External Sorting
II. Algorithms for SELECT and JOIN

Operations
III.Algorithms for PROJECT and SET

Operations
1. Algorithms for External Sorting
Sorting is one of the primary algorithms used in query
processing.
 For example, whenever an SQL query specifies an ORDER
BY-clause, the query result must be sorted.
 External sorting: Refers to sorting algorithms that are
suitable for large files of records stored on disk that do not fit
entirely in main memory, such as most database files.
 Internal sorting algorithms are suitable for sorting data
structures, such as tables and lists, that can fit
entirely in main memory.
 The typical external sorting algorithm uses a sort-merge
strategy.
 Sort-Merge strategy: Starts by sorting small sub files (runs)
of the main file and then merges the sorted runs, creating larger
sorted sub files that are merged in turn.
 The sort-merge algorithm, like other database algorithms,
requires buffer space in main memory,
where the actual sorting and merging of the runs is performed.
 The buffer space in main memory is part of the DBMS cache
 DBMS cache an area in the computer’s main memory that is
controlled by the DBMS.
 The buffer space is divided into individual buffers, where each
buffer is the same size in bytes as the size of one disk block.
 Thus, one buffer can hold the contents of exactly one disk
block.

Sorting phase: nR = [(b/nB)]
Merging phase: dM = Min (nB-1, nR); nP = [(log
dM(nR))]
 nR: number of initial runs;
 b: number of file blocks;
 nB: available buffer space;
 dM: degree of merging;
 nP: number of passes.

Algorithm:Prepared
Outline of the
by Elisaye sort-merge algorithm for external sorting.
B. @WSU-DTC 13
2. Algorithms for SELECT and JOIN
Operations
 There are many algorithms for executing a SELECT
operation, which is basically a search operation to locate the
records in a disk file that satisfy a certain condition.
 We will use the following operations, specified on the
relational database.
OP1: σ Ssn = ‘123456789’ (EMPLOYEE)
OP2: σ Dnumber > 5 (DEPARTMENT)
OP3: σ Dno = 5 (EMPLOYEE)
OP4: σ Dno = 5 AND Salary > 30000 AND Sex = ‘F’ (EMPLOYEE)
OP5: σ (WORKS_ON)
Search Methods for Simple Selection
 A number of search algorithms are possible for
selecting records from a file. The file search is called
file scans, because they scan the records of a file to
search for and retrieve records that satisfy a
selection condition.
 If the search algorithm involves the use of an index,
the index search is called an index scan.
 The following search methods (S1 through S6) are
examples of some of the simple search algorithms
that can be used to implement a select operation and
(S7 through S9) are examples of Complex search
algorithms: Prepared by Elisaye B. @WSU-DTC 15
 S1: Linear search (brute force Algorithm): Retrieve every
record in the file, and test whether its attribute values satisfy the
selection condition.
 S2: Binary search: If the selection condition involves an
equality comparison on a key attribute on which the file is
ordered, binary search (which is more efficient than linear
search) can be used. (See OP1).
 S3: Using a primary index or hash key to retrieve a single
record: If the selection condition involves an equality
comparison on a key attribute with a primary index (or a hash
key), use the primary index (or the hash key) to retrieve the
record.
 S4: Using a primary index to retrieve multiple records: If
the comparison condition is >, ≥, <, or ≤ on a key field with a
primary index, use the index to find the record satisfying the
corresponding equalityPrepared by Elisaye B. @WSU-DTC
condition, 16
then retrieve all subsequent
S5: Using a clustering index to retrieve multiple
records: If the selection condition involves an equality
comparison on a non key attribute with a clustering index,
use the clustering index to retrieve all the records
satisfying the selection condition.
S6: Using a secondary (B+-tree) index: On an equality
comparison, this search method can be used to retrieve a
single record if the indexing field has unique values (is a
key) or to retrieve multiple records if the indexing field is
not a key.
 In addition, it can be used to retrieve records on conditions
involving >,>=, <, or <=. (FOR RANGE QUERIES)

Search Methods for Complex Selection
 S7: Conjunctive selection: If an attribute involved in any
single simple condition in the conjunctive condition has an
access path that permits the use of one of the methods S2 to
S6, use that condition to retrieve the records and then check
whether each retrieved record satisfies the remaining
simple conditions in the conjunctive condition.
 S8: Conjunctive selection using a composite index: If
two or more attributes are involved in equality conditions
in the conjunctive condition and a composite index (or hash
structure) exists on the combined field, we can use the
index directly.

S9: Conjunctive selection by intersection of
record pointers: This method is possible if secondary
indexes are available on all (or some of) the fields involved
in equality comparison conditions in the conjunctive
condition and if the indexes include record pointers (rather
than block pointers).
 Each index can be used to retrieve the record pointers that
satisfy the individual condition.
 The intersection of these sets of record pointers gives the
record pointers that satisfy the conjunctive condition, which
are then used to retrieve those records directly.
 If only some of the conditions have secondary indexes,
each retrieved record is further tested to determine whether
it satisfies the remaining conditions.
Implementing the JOIN Operation
 The JOIN operation is one of the most time-
consuming operations in query processing.
 Many of the join operations encountered in queries
are of the EQUIJOIN and NATURAL JOIN varieties
 two–way join: a join on two files
e.g. R A=B S
 multi-way joins: joins involving more than two
files.
e.g. R A=B S C=D T
Examples
•(OP6): EMPLOYEE DNO=DNUMBER DEPARTMENT
•(OP7): DEPARTMENT MGRSSN=SSN EMPLOYEE
Methods for implementing joins:
J1 Nested-loop join (brute force):
• For each record t in R (outer loop), retrieve every
record s from S (inner loop) and test whether the
two records satisfy the join condition t[A] = s[B].
J2 Single-loop join (Using an access
structure to retrieve the matching records):
• If an index (or hash key) exists for one of the two
join attributes — say, B of S — retrieve each
record t in R, one at a time, and then use the
access structure to retrieve directly all matching
records s from S that satisfy s[B] = t[A].
J3 Sort-merge join:

• If the records of R and S are physically sorted
(ordered) by value of the join attributes A and B,
respectively, we can implement the join in the most
efficient way possible.
• Both files are scanned in order of the join attributes,
matching the records that have the same values for A
and B.
• In this method, the records of each file are scanned
only once each for matching with the other file—
unless both A and B are non-key attributes, in which
case the method needs to be modified slightly.
J4 Hash-join:
 The records of files R and S are both hashed to the
same hash file, using the same hashing function on
the join attributes A of R and B of S as hash keys.
 A single pass through the file with fewer records
(say, R) hashes its records to the hash file buckets.
 A single pass through the other file (S) then hashes
each of its records to the appropriate bucket,
where the record is combined with all matching
records from R.
Factors affecting JOIN
performance
 Available buffer space
 Join selection factor
 Choice of inner VS outer relation

3. Algorithms for PROJECT and SET
Operations

A PROJECT operation p <attribute list>(R)
1. If <attribute list> has a key of relation R, extract all
tuples from R with only the values for the attributes in
<attribute list>.
2. If <attribute list> does NOT include a key of relation
Reduplicated tuples must be removed from the
results.
Methods to remove duplicate tuples
1. Sorting
2. Hashing Prepared by Elisaye B. @WSU-DTC 25
Algorithm for SET operations
Set operations:
 UNION, INTERSECTION, SET DIFFERENCE and
CARTESIAN PRODUCT
CARTESIAN PRODUCT of relations R and S include
all possible combinations of records from R and S. The
attribute of the result include all attributes of R and S.
 Cost analysis of CARTESIAN PRODUCT: If R has n
records and j attributes and S has m records and k
attributes, the result relation will have n*m records and
j+k attributes.
 CARTESIAN PRODUCT operation is very expensive
and should be avoided if possible.
Algorithm for SET operations
UNION
 Sort the two relations on the same attributes.
 Scan and merge both sorted files concurrently, whenever
the same tuple exists in both relations, only one is kept in
the merged results.
INTERSECTION
 Sort the two relations on the same attributes.
 Scan and merge both sorted files concurrently, keep in the
merged results only those tuples that appear in both
relations.
SET DIFFERENCE R-S
 Keep in the merged results only
Prepared by Elisaye those tuples that appear27
B. @WSU-DTC
Using Heuristics in Query Optimization(1)
Process for heuristics optimization
1. The parser of a high-level query generates an initial
internal representation;
2. Apply heuristics rules to optimize the internal
representation.
3. A query execution plan is generated to execute
groups of operations based on the access paths
available on the files involved in the query.
The main heuristic is to apply first the operations that
reduce the size of intermediate results.

E.g., Apply SELECT and PROJECT operations
before applying thePrepared
JOIN orB. @WSU-DTC
by Elisaye other binary operations.28
 Query tree:
 A tree data structure that corresponds to a relational algebra
expression. It represents the input relations of the query as
leaf nodes of the tree, and represents the relational algebra
operations as internal nodes.
 An execution of the query tree consists of executing an
internal node operation whenever its operands are available
and then replacing that internal node by the relation that
results from executing the operation.
 Query graph:
 A graph data structure that corresponds to a relational calculus
expression.
 It does not indicate an order on which operations to perform
first. There is only a single graph
Prepared by corresponding to each query.
Elisaye B. @WSU-DTC 29
Example:
 For every project located in ‘Stafford’, retrieve the
project number, the controlling department number
and the department manager’s last name, address
and birthdate.
 Relation algebra:
ΠPnumber, Dnum, Lname, Address, Bdate (((σPlocation=‘Stafford’
(PROJECT)) Dnum=Dnumber
(DEPARTMENT)) Mgr_ssn=Ssn
(EMPLOYEE)) Prepared by Elisaye B. @WSU-DTC 30
 This corresponds to the following SQL query:
Q2: SELECT P.Pnumber, P.Dnum, E.Lname,
E.Address, E.Bdate FROM PROJECT AS P,
DEPARTMENT AS D, EMPLOYEE AS E
WHERE P.Dnum=D.Dnumber AND D.Mgr_ssn =
E.Ssn AND P.Plocation= ‘Stafford’;

Heuristic Optimization of Query Trees:
• The same query could correspond to many different
relational algebra expressions — and hence many
different query trees.
• The task of heuristic optimization of query trees is to find
a final query tree that is efficient to execute.
 Example:
• Q: SELECT LNAME
FROM EMPLOYEE, WORKS_ON, PROJECT
WHERE PNAME = ‘AQUARIUS’ AND
PNMUBER=PNO AND ESSN=SSN
AND BDATE > ‘1957-12-31’;
 General Transformation Rules for Relational
Algebra Operations:
1. Cascade of s: A conjunctive selection condition can be broken up
into a cascade (sequence) of individual s operations:
s c1 AND c2 AND ... AND cn (R) = sc1 (sc2 (...(scn(R))...) )
2. Commutativity of s: The s operation is commutative:
 sc1 (sc2(R)) = sc2 (sc1(R))
3. Cascade of p: In a cascade (sequence) of p operations, all but the
last one can be ignored:
 pList1 (pList2 (...(pListn(R))...) ) = pList1(R)
4. Commuting s with p: If the selection condition c involves only the
attributes A1, ..., An in the projection list, the two operations can be
commuted:

5. Commutativity of ( and x ): The operation is

commutative as is the x operation:
R CS = S C R; R x S = S x R
6. Commuting s with (or x ): If all the attributes in the

selection condition c involve only the attributes of one of the
relations being joined—say, R—the two operations can be
commuted as follows:
sc ( R S ) = (sc (R)) S
 Alternatively, if the selection condition c can be written as (c1
and c2), where condition c1 involves only the attributes of R
and condition c2 involves only the attributes of S, the
operations commute as follows:
sc ( R S)= (sc1 (R))
Prepared by Elisaye B. @WSU-DTC
(sc2 (S)) 37
7. Commuting p with (or x): Suppose that the
projection list is L = {A1, ..., An, B1, ..., Bm},
where A1, ..., An are attributes of R and B1, ...,
Bm are attributes of S.
 If the join condition c involves only attributes in L,
the two operations can be commuted as follows:
pL ( R C S ) = (pA1, ..., An (R)) C (p B1, ..., Bm
(S))
 If the join condition C contains additional
attributes not in L, these must be added to the
projection list, and a final p operation is needed.
8. Commutativity of set operations: The set operations
υ and ∩ are commutative but “–” is not.
9. Associativity of , x, υ, and ∩ : These four
operations are individually associative; that is, if q
stands for any one of these four operations
(throughout the expression), we have
(Rq S)q T=Rq (Sq T)
10.Commuting s with set operations: The s operation
commutes with υ , ∩ , and –. If q stands for any one
of these three operations, we have
sc ( R q S ) = (sc (R)) q (sc (S))
Using Selectivity and Cost Estimates in
Query Optimization (1)
 Cost-based query optimization:
• Estimate and compare the costs of executing a
query using different execution strategies and
choose the strategy with the lowest cost estimate.
 Issues
 Cost function
 Number of execution strategies to be considered

Using Selectivity and Cost Estimates in Query
Optimization (2)
Cost Components for Query Execution
1.Access cost to secondary storage
2.Storage cost
3.Computation cost
4.Memory usage cost
5.Communication cost
 Note: Different database systems may focus on
different cost components.
Using Selectivity and Cost Estimates in Query Optimization (3)
Catalog Information Used in Cost Functions

Information about the size of a file
 number of records (tuples) (r),
 record size (R),
 number of blocks (b)
 blocking factor (bfr)
Information about indexes and indexing attributes of a file
 Number of levels (x) of each multilevel index
 Number of first-level index blocks (bI1)
 Number of distinct values (d) of an attribute
 Selectivity (sl) of an attribute
 Selection cardinality (s) of an attribute. (s = sl * r)
Overview of Query Optimization in Oracle
 Oracle DBMS V8
 Rule-based query optimization: the optimizer chooses
execution plans based on heuristically ranked operations.
 (Currently it is being phased out)
 Cost-based query optimization: the optimizer examines
alternative access paths and operator algorithms and
chooses the execution plan with lowest estimate cost.
 The query cost is calculated based on the estimated usage of
resources such as I/O, CPU and memory needed.
 Application developers could specify hints to the
ORACLE query optimizer.
 The idea is that an application developer might know more
information about thePrepared
data. by Elisaye B. @WSU-DTC 43
Semantic Query Optimization
 Semantic Query Optimization:
 Uses constraints specified on the database schema in order to
modify one query into another query that is more efficient to
execute.
 Consider the following SQL query,
SELECT E.LNAME, M.LNAME
FROM EMPLOYEE E M
WHERE E.SUPERSSN=M.SSN AND E.SALARY>M.SALARY
 Explanation:
 Suppose that we had a constraint on the database schema that stated
that no employee can earn more than his or her direct supervisor.
 If the semantic query optimizer checks for the existence of this
constraint, it need not execute the query at all because it knows that
the result of the query will be empty. Techniques known as theorem
proving can be used for this purpose.

Chapter 1

Uploaded by

Copyright:

Available Formats

Chapter 1

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 1

Uploaded by

Copyright:

Available Formats

CHAPTER-1

Query processing and Optimization

• Query Code Generator: It generates the code to

• Runtime Database Processor: It has the task of running

Prepared by Elisaye B. @WSU-DTC 5

 A query block contains a single SELECT-FROM-

 Nested queries within a query are identified as

 Aggregate operators in SQL must be included in the

Prepared by Elisaye B. @WSU-DTC 7

Prepared by Elisaye B. @WSU-DTC 8

I. Algorithms for External Sorting

II. Algorithms for SELECT and JOIN

III.Algorithms for PROJECT and SET

Prepared by Elisaye B. @WSU-DTC 11

Prepared by Elisaye B. @WSU-DTC 12

Prepared by Elisaye B. @WSU-DTC 17

Prepared by Elisaye B. @WSU-DTC 18

J3 Sort-merge join:

Prepared by Elisaye B. @WSU-DTC 24

Prepared by Elisaye B. @WSU-DTC 31

5. Commutativity of ( and x ): The operation is

6. Commuting s with (or x ): If all the attributes in the

Prepared by Elisaye B. @WSU-DTC 40

Catalog Information Used in Cost Functions

You might also like