Unit 1: Course Introduction: Week 1: SAP HANA Query Processing
Unit 1: Course Introduction: Week 1: SAP HANA Query Processing
Unit 1: Course Introduction: Week 1: SAP HANA Query Processing
Target Audience
▪ SAP HANA developers, SAP HANA consultants, and
anyone interested in learning about SAP HANA query
performance
Course Requirements
▪ Basic knowledge of the SQL language
2 4
WEEK WEEK WEEK WEEK
START
1 3
SAP HANA Query Processing Methods for Query
Performance Analysis
[email protected]
Follow all of SAP
www.sap.com/contactsap
SQL Front
End
SQL Optimizer
Calculation Engine
SQL Engine
Optimizer & Executor
(Row Engine)
Row Store OLAP JOIN
Engine Engine
Column Store
SELECT
A.COL1,
B.COL2
FROM TABLE1 A
INNER JOIN TABLE2 B ON A.COL3 = B.COL3;
SQL Front
End
SQL Optimizer
QO
QE
Calculation Engine
SQL Engine
Optimizer & Executor
(Row Engine)
Row Store OLAP JOIN
Engine Engine *QO: Query Optimizer Tree
Column Store
*QE: Query Execution Plan
© 2020 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 4
SQL Query Processing in SAP HANA
Simple SQL statement processing – Cache not found
SQL Front
End
SQL Optimizer
SQL Optimizer:
QO Query Optimizer Tree The SQL Optimizer decides the best way
to call the engines depending on the
involved models and queries.
QE Query Execution Plan
Calculation Engine
SQL Engine
Optimizer & Executor
(Row Engine)
Row Store OLAP JOIN
Engine Engine
Column Store
SQL
Front End
SQL Optimizer
Plan
Calculation Engine
SQL Engine
Optimizer & Executor
(Row Engine)
Row Store OLAP JOIN
Engine Engine
Column Store
Plan
Info columns
▪ EXECUTION_COUNT
▪ PREPARATION_COUNT
Execution
Engines ▪ LAST_EXECUTION_TIMESTAMP
▪ LAST_PREPARATION_TIMESTAMP
LAST_INVALIDATION_REASON
PLAN_ID
USER_NAME LAST_EXECUTION_TIMESTAMP
STATEMENT_STRING
IS_VALID
M_SQL_PLAN_CACHE EXECUTION_COUNT
LAST_PREPARATION_TIMESTAMP
STATEMENT_HASH
PREPARATION_COUNT
PM PM
PM PM
[email protected]
Follow all of SAP
www.sap.com/contactsap
Choose and join tables Aggregate the base data Return the final data Sort the final data
to get base data
Filter the base data Filter the aggregated Remove duplicated values
data marked as DISTINCT
SELECTA.COL1, A.COL2
SELECT /*literal*/
FROM TABLE1 A
INNER JOIN ( SELECT B.COL1, B.COL2
FROM TABLE2 B
LEFT OUTER JOIN (SELECT E.COL1, E.COL2, G.COL2
FROM TABLE3 E
INNER JOIN (SELECT F.COL1, F.COL2
FROM TABLE4 F
GROUP BY F.COL1,F.COL2) G
ON E.COL2=G.COL2) H
ON B.COL1 =H.COL1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WITH HINT (NO_USE_HEX_PLAN);
SELECT
T4
WHERE A.COL3 = 20
SELECT /*literal*/ A.COL1, A.COL2
FROM TABLE1 A
INNER JOIN ( SELECT B.COL1, B.COL2
IJ A.COL2 = D.COL2
Join 1 FROM TABLE2 B
LEFT OUTER JOIN (SELECT E.COL1, E.COL2, G.COL2
Join 2 FROM TABLE3 E
INNER JOIN (SELECT F.COL1, F.COL2
T1 LOJ B.COL1 = H.COL1
Join 3 FROM TABLE4 F
GROUP BY F.COL1,F.COL2) G
ON E.COL2=G.COL2) H T2 IJ E.COL2 = G.COL2
ON B.COL1 =H.COL1) D
ON A.COL2 = D.COL2
WHERE A.COL3 =20 WITH HINT (NO_USE_HEX_PLAN);
T3 G
WHERE
T4
[email protected]
Follow all of SAP
www.sap.com/contactsap
CREATE COLUMN TABLE TABLE1 (COL1 INT, COL2 INT, COL3 INT);
SELECT
STATEMENT_STRING,
STATEMENT_HASH,
EXECUTION_COUNT,
PREPARATION_COUNT,
PARAMETER_COUNT,
LAST_EXECUTION_TIMESTAMP,
LAST_PREPARATION_TIMESTAMP
FROM "M_SQL_PLAN_CACHE"
WHERE SCHEMA_NAME = '<username>'
AND STATEMENT_STRING LIKE '%/*literal*/ A.COL1, A.COL2 … A.COL3=20'
ORDER BY LAST_PREPARATION_TIMESTAMP DESC;
PM PM
Literal query is compiled once, compiled plan is stored in plan cache and is reused
SQL SQL
Front End Front End
Plan cache entry found
Not stored
Plan Plan
in M_SQL_PLAN_CACHE
Execution Execution
Engines Engines
[email protected]
Follow all of SAP
www.sap.com/contactsap
SQL
Front End
Plan cache entry found
SQL Optimizer
Plan
Execution
Engines
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache
SQL Front
End
SQL
Optimizer
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache
SQL Front
End
SQL
Optimizer
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache
Rule-Based
Optimization
SQL Front
End
SQL
Optimizer
Cost-Based
Query
Optimization
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache
Rule-Based
Optimization
SQL Front
End
SQL
Optimizer
Cost-Based
Query
Optimization
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache
Rule-Based
Optimization
SQL Front
End
SQL
Optimizer
Cost-Based
Query
Optimization
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache G T2 T4
Rule-Based
Optimization J13
SQL Front
End
G T3
SQL
Optimizer
T1
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache G T2 T4
Rule-Based
Optimization J13
SQL Front
End
G F Filter Pushdown
SQL
Optimizer
T1 T3
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache Simplify G T2 T4
Rule-Based Group By
Optimization J13
SQL Front
End
G F
SQL
Optimizer
T1 T3
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache JG
13 T2 T4
Rule-Based
Optimization
SQL Front G F
End
SQL T1 T3
Optimizer
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache JG
13
Rule-Based
Optimization
SQL G F
Frontend
SQL T1 T3
Optimizer
REWRITE TREE
Cost-Based
Query
Optimization
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache
Rule-Based
Optimization
SQL Front
End
SQL
Optimizer
Cost-Based
Query
Optimization
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache
Rule-Based
Optimization
SQL Front
End
SQL
Optimizer
Cost-Based
Query
Optimization
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache
Rule-Based
Optimization
SQL Front
End
SQL
*The tree from the rule-based
Optimizer
optimization is sent to cost-based
query optimization
Cost-Based
Query
Optimization
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache
Rule-Based
Optimization
SQL Front
End
SQL
Optimizer
_THRU_
&
PRE_BEFORE_
Cost-Based
Query
Optimization
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache
Rule-Based
Optimization
SQL Front
End
SQL
Optimizer
_THRU_
&
PRE_BEFORE_
Cost-Based
Query
Optimization
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache
Rule-Based
Optimization
SQL Front Plan 3
End
SQL
Optimizer
_THRU_
&
Cost-Based
Query PRE_BEFORE_
Optimization
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache
Rule-Based
Optimization
SQL Front Plan 3
End
SQL
Optimizer
Plan 4
_THRU_
Cost-Based
Query &
Optimization PRE_BEFORE_
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache
Rule-Based
Optimization
SQL Front Plan 3
End
SQL
Optimizer
Plan 4
Cost-Based
Query _THRU_ Plan 5
Optimization
&
PRE_BEFORE_
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache
Rule-Based
A_THRU_B PREA_BEFORE_B
Optimization
SQL Plan 3
Frontend
SQL
Optimizer
Plan 4
▪ AGGR_THRU_JOIN ▪ PREAGGR_BEFORE_JOIN
▪ JOIN_THRU_JOIN ▪ PREAGGR_BEFORE_UNION
Cost-Based
▪ JOIN_THRU_AGGR …
Query Plan 5
Optimization _THRU_
&
PRE_BEFORE_
Plan 6
© 2020 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 22
SQL Optimizer – Architectural Overview
Plan 1
Final plan selection among alternatives
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache
Rule-Based
Optimization
SQL Front Plan 3
End
SQL
Optimizer
Plan 4
Cost-Based
Query Plan 5
Optimization
Plan 6
© 2020 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 23
SQL Optimizer – Architectural Overview
Plan 1
Final plan selection among alternatives
SQL Plan
FROM TABLE3 E
INNE R JOIN ( SELECT F .COL1, F .COL2
FROM TAB LE4 F
GROUP BY F.COL1, F.COL2) G
ON E .COL2=G. COL2) H
ON B.CO L1 =H.CO L1) D ON A.COL2 = D.COL2
WHERE A.COL3 =20 WIT H HINT ( NO_USE_H EX_PLAN) ;
Cache
Rule-Based
Optimization
SQL Front
End
SQL
Optimizer
Plan 4
Cost-Based
Query Plan 5
Optimization
Plan 3 Plan 6
© 2020 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 24
Thank you.
Contact information:
[email protected]
Follow all of SAP
www.sap.com/contactsap
SELECT
Bind variable = ‘A’ COL1, Bind variable = ‘B’
COL2
FROM VIEW1
WHERE COL3 = ?;
SELECT
COL1,
COL2
FROM VIEW1
WHERE COL3 = ?;
SQL Front
End
SQL Front
End
SQL Optimizer
QO
QE
SQL Front
End
SQL Optimizer
QO
QE Precompilation
SQL Optimizer
QO
QE
Calculation Engine
SQL Engine Optimizer &
(Row Engine) Executor
Row Store OLAP JOIN
Engine Engine
Column Store
SQL Optimizer
SQL Optimizer
SELECT
COL1,
COL2 VIEW1
FROM VIEW1
WHERE COL3 = ?;
2020.03.01 PM 2020.03.01 PM
2020.03.01 PM 2020.03.01 PM
2020.03.01 PM
2020.03.01 PM 2020.03.01 PM
2020.03.01 PM 2020.03.01 PM
2020.03.01 PM 2020.03.01 PM
2020.03.01 PM 2020.03.01 PM
AM AM
AM AM
[email protected]
Follow all of SAP
www.sap.com/contactsap
Please note that our recommendation for parameterized query execution is to make the
parameterized query into a single line. This is because different carriage returns across the
interfaces often interfere with the usage of the plan cache.
SELECT /*Parameterized*/ A.COL1, A.COL2
FROM TABLE1 A
INNER JOIN (
SELECT B.COL1, B.COL2
FROM TABLE2 B
LEFT JOIN (SELECT E.COL1, E.COL2, G.COL2
FROM TABLE3 E
INNER JOIN (SELECT F.COL1, F.COL2 FROM TABLE4 F GROUP BY F.COL1, F.COL2) G
ON E.COL2=G.COL2) H
ON B.COL1 =H.COL1) D
ON A.COL2 = D.COL2
WHERE A.COL3 =? WITH HINT (NO_USE_HEX_PLAN);
SELECT /*Parameterized*/ A.COL1, A.COL2 FROM TABLE1 A INNER JOIN (SELECT B.COL1, B.COL2 FROM TABLE2 B LEFT JOIN
(SELECT E.COL1, E.COL2, G.COL2 FROM TABLE3 E INNER JOIN (SELECT F.COL1, F.COL2 FROM TABLE4 F GROUP BY F.COL1,
F.COL2) G ON E.COL2=G.COL2) H ON B.COL1 =H.COL1) D ON A.COL2 = D.COL2 WHERE A.COL3 =? WITH HINT (NO_USE_HEX_PLAN)
SELECT
STATEMENT_STRING,
STATEMENT_HASH,
EXECUTION_COUNT,
PREPARATION_COUNT,
PARAMETER_COUNT,
LAST_EXECUTION_TIMESTAMP,
LAST_PREPARATION_TIMESTAMP
FROM "M_SQL_PLAN_CACHE"
WHERE SCHEMA_NAME = '<username>'
AND STATEMENT_STRING LIKE '%/*parameterized*/ A.COL1, A.COL2 … A.COL3= ?'
ORDER BY LAST_PREPARATION_TIMESTAMP DESC;
PM PM
[email protected]
Follow all of SAP
www.sap.com/contactsap