Oracle CBO Cost Calculation A High level Overview Martin Frauendorfer, SAP Active Global Support November 2007
Agenda 1. Introduction of Cost Based Optimizer 2. Details of Cost Calculation 3. Example for Cost Calculation SAP 2007 / Page 2
Cost Based Optimizer (1) Remark: As of Oracle 10g SAP recommends the creation of system statistics. System statistics provide an additional CPU cost component. Additionally they provide a system dependent cost ratio between single block reads and multi block reads. Experience has shown that system statistics have much less impact on the CBO decisions than the traditional CBO costs. For simplification purposes we will not take into account system statistics in this presentation. SAP 2007 / Page 3
Cost Based Optimizer (2) SELECT * FROM ZVBAK WHERE MANDT = :A0 AND VBELN = :A1 Possible access paths Database Settings OPTIMIZER_FEATURES_ENABLE = 9.2.0 OPTIMIZER_INDEX_COST_ADJ = 10 Parallelism Degree... Estimated Costs............... Full Table Scan Cost Based Optimizer The estimated costs are derived from the blocks that are expected to be read SAP 2007 / Page 4 Index Scan Table Statistics Index Statistics Column Statistics Histograms Optimizer statistics
Oracle Optimizers Cost Evaluation Decision for CBO vs. RBO: Yes Feature used that is not supported by RBO (Parallel Query, Partitioning, IOT ) Other No Hint used? No RULE Cost based optimizer No OPTIMIZER_MODE = RULE OR No statistics AND OPTIMIZER_MODE = CHOOSE Yes Rule based optimizer SAP 2007 / Page 5
Important Statistics Values Most important statistic values: Table Statistics: Number of table blocks (BLOCKS) Number of table rows (NUM_ROWS) Index Statistics: Levels of the index (BLEVEL) Number of leaf blocks (LEAF_BLOCKS) Clustering factor (CLUSTERING_FACTOR) Number of distinct keys (DISTINCT_KEYS) Column Statistics: Number of distinct values (NUM_DISTINCT) Number of NULL values (NUM_NULLS) SAP 2007 / Page 6
Cost Calculation (1) The CBO uses assumptions which part of the whole records fulfills a condition. This relationship is called filter factor (FF) of the condition. = FF = 1 / NUM_DISTINCT(column) LIKE, >, <, >=, <= LIKE on NUMBER field: FF =1 Otherwise: FF = max(0.009, 1 / NUM_DISTINCT(column)) BETWEEN (Oracle <= 9.2.0.5) NUM_DISTINCT(column) = 1 -> FF = 1 NUM_DISTINCT(column) = 2 -> FF = 2 / NUM_ROWS (table) 3 <= NUM_DISTINCT(column) <= 111 -> FF = 0.05 112 <= NUM_DISTINCT(column) -> FF = 0.0045 BETWEEN (Oracle >= 9.2.0.6) FF = max(0.0045, 1 / NUM_DISTINCT(column)) SAP 2007 / Page 7
Cost Calculation (2) 0.1 0.09 0.08 0.07 0.06 0.05 FF (=) FF (LIKE, <, >, <=, >=) FF (BETWEEN, Oracle <= 9.2.0.5) FF (BETWEEN, Oracle >= 9.2.0.6) 0.04 0.03 0.02 0.01 0 1 13 25 37 49 61 73 85 97 109 121 133 145 157 169 181 193 205 217 229 241 253 265 277 289 SAP 2007 / Page 8
Cost Calculation (3) IS NULL NULL constraint exists: FF = 0 Otherwise: NUM_NULLS / NUM_ROWS IN FF = min(1, <length_of_in_list> / NUM_DISTINCT(column)) <cond1> OR <cond2> FF = FF(cond1) + FF(cond2) NOT (<cond>) NOT >, NOT >=, NOT <, NOT <= are transformed to <=, <, >=, > NOT BETWEEN is transformed to < OR > NOT =, NOT IN, NOT LIKE : FF = 1 IS NOT NULL: FF = 1 NUM_NULLS / NUM_ROWS SAP 2007 / Page 9
Cost Calculation (4) If there are several conditions specified with AND, the filter factors of these conditions have to be multiplied. Three different combined filter factors exist: FF1: Product of the filter factors of all conditions that restrict the amount of index leaf blocks to be scanned: The column is part of the index. All preceeding columns in the index are specified with = or IN in the WHERE condition or are skipped with an index skip scan FF2: Product of the filter factors of all conditions that restrict the amount of table blocks to be scanned: The column is part of the index. FF3: Product of the filter factors of all conditions that restrict the number of rows to be returned. All conditions specified in the WHERE clause are relevant. SAP 2007 / Page 10
Cost Calculation (5) Based on the filter factors the estimated number of read blocks can be calculated: Full Table Scan: BLOCKS(table) Index Unique Scan: BLEVEL(index) + 1 Index Unique Scan + Table Access by Index ROWID: BLEVEL(index) + 2 Index Full Scan: BLEVEL(index) + LEAF_BLOCKS(index) Index Fast Full Scan: BLEVEL(index) + LEAF_BLOCKS(index) Index Range Scan: BLEVEL(index) + LEAF_BLOCKS(index) * FF1 Index Range Scan + Table Access by Index ROWID: BLEVEL(index) + LEAF_BLOCKS(index) * FF1 + CLUSTERING_FACTOR(index) * FF2 Index Skip Scan: max(num_distinct(skipped_columns)) * (BLEVEL(index) + LEAF_BLOCKS(index) * FF1) Index Skip Scan + Table Access by Index ROWID: max(num_distinct(skipped_columns)) * (BLEVEL(index) + LEAF_BLOCKS(index) * FF1 + CLUSTERING_FACTOR(index) * FF2) SAP 2007 / Page 11
Cost Calculation (6) The number of blocks read during joins can now be derived from the number of blocks read for individual table accesses: Nested Loop Join: BlocksRead(table1) + NUM_ROWS(table1) * FF3 * BlocksRead(table2) Sort Merge Join: BlocksRead(table1) + BlocksRead(table2) + Sort Overhead Hash Join: BlocksRead(table1) + BlocksRead(table2) + Hash Overhead SAP 2007 / Page 12
Cost Calculation (7) In addition to the estimated blocks that have to be read the following factors influence the cost calculation: Oracle parameter OPTIMIZER_INDEX_COST_ADJ Costs for index accesses (apart from index fast full scans) are changed to the percentage specified. Oracle parameters DB_FILE_MULTIBLOCK_READ_COUNT / _DB_FILE_OPTIMIZER_READ_COUNT Scattered reads like full table scans and index fast full scans can read multiple consecutive blocks (up to the number defined in parameter DB_FILE_MULTIBLOCK_READ_COUNT) from disk with one I/O request. This is taken into account by the CBO: Oracle 9i: Costs for scattered reads are divided by 1.6765 * DB_FILE_MULTIBLOCK_READ_COUNT ^ 0.6581. For a parameter value of 8 this means a ratio of 6.59. For a parameter value of 32 the ratio is 16.41. Oracle 10g: Costs for scattered reads are based on _DB_FILE_OPTIMIZER_READ_COUNT (-> 8) and the system specific system statistics (see note 927295). SAP 2007 / Page 13
Cost Calculation (8) Oracle parameter OPTIMIZER_INDEX_CACHING The parameter contains a percentage that defines how many index blocks are assumed to remain in the Oracle buffer pool during IN list and nested loop join processing. The higher the value is set, the lower are the calculated costs for IN lists and nested loop joins because less physical I/O is assumed. Oracle parameter HASH_AREA_SIZE / PGA_AGGREGATE_TARGET The larger the specified memory for hash operations is defined the less I/O has to be performed and the cheaper are the hash join costs. Oracle parameter SORT_AREA_SIZE / PGA_AGGREGATE_TARGET The larger the specified memory for sort operations is defined the less I/O has to be performed and the cheaper are the sort and sort merge join costs. Parallel Query If parallel query is used, the costs for parallelizable actions like full table scans are divided by the parallelism degree (Oracle <= 9i) or 90 % of the parallelism degree (Oracle >= 10g). SAP 2007 / Page 14
Cost Calculation (9) Formulas for estimated costs: Full Table Scan: BLOCKS(table) / MBRC / parallelism degree + 1 Index Unique Scan: (BLEVEL(index) + 1) * OPTIMIZER_INDEX_COST_ADJ / 100 Index Unique Scan + Table Access by Index ROWID: (BLEVEL(index) + 2) * OPTIMIZER_INDEX_COST_ADJ / 100 Index Full Scan: (BLEVEL(index) + LEAF_BLOCKS(index)) * OPTIMIZER_INDEX_COST_ADJ / 100 Index Fast Full Scan: BLEVEL(index) + LEAF_BLOCKS(index) / MBRC / parallelism degree + 1 Index Range Scan: (BLEVEL(index) + LEAF_BLOCKS(index) * FF1) / OPTIMIZER_INDEX_COST_ADJ * 100 Index Range Scan + Table Access by Index ROWID: (BLEVEL(index) + LEAF_BLOCKS(index) * FF1 + CLUSTERING_FACTOR(index) * FF2) * OPTIMIZER_INDEX_COST_ADJ / 100 SAP 2007 / Page 15
Cost Calculation (10) Index Skip Scan: max(num_distinct(skipped_columns)) * (BLEVEL(index) + LEAF_BLOCKS(index) * FF1) * OPTIMIZER_INDEX_COST_ADJ / 100 Index Skip Scan + Table Access by Index ROWID: max(num_distinct(skipped_columns)) * (BLEVEL(index) + LEAF_BLOCKS(index) * FF1 + CLUSTERING_FACTOR(index) * FF2) * OPTIMIZER_INDEX_COST_ADJ / 100 Nested Loop Join: Costs(table1) + NUM_ROWS (table1) * FF3 * Costs(table2) Sort Merge Join: Costs(table1) + Costs(table2) + Sort Overhead Hash Join: Costs(table1) + Costs(table2) + Hash Overhead SAP 2007 / Page 16
Example for Cost Calculation (1) ALTER SESSION SET OPTIMIZER_INDEX_COST_ADJ = 100; ALTER SESSION SET DB_FILE_MULTIBLOCK_READ_COUNT = 32; CREATE TABLE AAA (WWW VARCHAR2(10), XXX VARCHAR2(10), YYY VARCHAR2(10), ZZZ VARCHAR2(10)); BEGIN FOR I IN 1..100000 LOOP INSERT INTO AAA VALUES ( 100, MOD(I, 10), MOD(I, 100), I); END LOOP; COMMIT; END; / CREATE INDEX AAA_1 on AAA (WWW, XXX, YYY); ANALYZE TABLE AAA COMPUTE STATISTICS; SAP 2007 / Page 17
Example for Cost Calculation (2) The most important statistics values are now: Table Statistics: BLOCKS(AAA) = 275 NUM_ROWS(AAA) = 100.000 Index Statistics: BLEVEL(AAA_1) = 1 LEAF_BLOCKS(AAA_1) = 278 CLUSTERING_FACTOR(AAA_1) = 27300 Column Statistics: NUM_DISTINCT(WWW) = 1 NUM_DISTINCT(XXX) = 10 NUM_DISTINCT(YYY) = 100 NUM_DISTINCT(ZZZ) = 100.000 SAP 2007 / Page 18
Example for Cost Calculation (3) Let s have a closer look at the cost calculation for the following statement: SELECT * FROM AAA WHERE WWW = :A0 and XXX > :A1 AND YYY = :A2 AND ZZZ = :A3 The resulting filter factors are: FF(WWW condition) = 1 / NUM_DISTINCT(WWW) = 1 FF(XXX condition) = max(0.009, 1 / NUM_DISTINCT(XXX)) = 1/10 = 0.1 FF(YYY condition) = 1 / NUM_DISTINCT(YYY) = 1/100 = 0.01 FF(ZZZ condition) = 1 / NUM_DISTINCT(ZZZ) = 1/100.000 = 0.00001 FF1 = FF(WWW condition) * FF(XXX condition) = 1 * 0.1 = 0.1 FF2 = FF(WWW condition) * FF (XXX condition) * FF(YYY condition) = 1 * 0.1 * 0.01 = 0.001 FF3 = FF(WWW condition) * FF (XXX condition) * FF(YYY condition) * FF (ZZZ condition) = 1 * 0.1 * 0.01 * 0.00001 = 0.00000001 SAP 2007 / Page 19
Example for Cost Calculation (4) Based on our formulas the resulting costs for a full table scan or index range scan access are: Costs(Full Table Scan) = BLOCKS(AAA) / DB_FILE_MULTIBLOCK_READ_COUNT / parallelism degree + 1 = 275 / 16.41 / 1 + 1 = 18 Costs(Index Range Scan + Table Access by Index ROWID) = (BLEVEL(index) + LEAF_BLOCKS(index) * FF1 + CLUSTERING_FACTOR(index) * FF2) * OPTIMIZER_INDEX_COST_ADJ / 100 = (1 + 278 * 0.1 + 27300 * 0.001) * 100 / 100 = 1 + 28 + 28 = 57 SAP 2007 / Page 20
Example for Cost Calculation (5) Now let s check which costs are calculated by Oracle: EXPLAIN PLAN FOR SELECT /*+ FULL(AAA) */ * FROM AAA WHERE WWW = :A0 and XXX > :A1 AND YYY = :A2 AND ZZZ = :A3; SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY); -------------------------------------------------------------------- Id Operation Name Rows Bytes Cost -------------------------------------------------------------------- 0 SELECT STATEMENT 1 11 18 * 1 TABLE ACCESS FULL AAA 1 11 18 -------------------------------------------------------------------- EXPLAIN PLAN FOR SELECT /*+ INDEX(AAA AAA_1) */ * FROM AAA WHERE WWW = :A0 and XXX > :A1 AND YYY = :A2 AND ZZZ = :A3; SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY); --------------------------------------------------------------------------- Id Operation Name Rows Bytes Cost --------------------------------------------------------------------------- 0 SELECT STATEMENT 1 11 57 * 1 TABLE ACCESS BY INDEX ROWID AAA 1 11 57 * 2 INDEX RANGE SCAN AAA_1 10001 29 --------------------------------------------------------------------------- SAP 2007 / Page 21
Thank you! SAP 2007 / Page 22
Copyright 2007 SAP AG All rights reserved No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. SAP, R/3, mysap, mysap.com, xapps, xapp, SAP NetWeaver, Duet, Business ByDesign, ByDesign, PartnerEdge and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned and associated logos displayed are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary. The information in this document is proprietary to SAP. This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intended strategies, developments, and functionalities of the SAP product and is not intended to be binding upon SAP to any particular course of business, product strategy, and/or development. SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, or other items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitation shall not apply in cases of intent or gross negligence. The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages Weitergabe und Vervielfältigung dieser Publikation oder von Teilen daraus sind, zu welchem Zweck und in welcher Form auch immer, ohne die ausdrückliche schriftliche Genehmigung durch SAP AG nicht gestattet. In dieser Publikation enthaltene Informationen können ohne vorherige Ankündigung geändert werden. Einige von der SAP AG und deren Vertriebspartnern vertriebene Softwareprodukte können Softwarekomponenten umfassen, die Eigentum anderer Softwarehersteller sind. SAP, R/3, mysap, mysap.com, xapps, xapp, SAP NetWeaver, Duet, Business ByDesign, ByDesign, PartnerEdge und andere in diesem Dokument erwähnte SAP-Produkte und Services sowie die dazugehörigen Logos sind Marken oder eingetragene Marken der SAP AG in Deutschland und in mehreren anderen Ländern weltweit. Alle anderen in diesem Dokument erwähnten Namen von Produkten und Services sowie die damit verbundenen Firmenlogos sind Marken der jeweiligen Unternehmen. Die Angaben im Text sind unverbindlich und dienen lediglich zu Informationszwecken. Produkte können länderspezifische Unterschiede aufweisen. Die in diesem Dokument enthaltenen Informationen sind Eigentum von SAP. Dieses Dokument ist eine Vorabversion und unterliegt nicht Ihrer Lizenzvereinbarung oder einer anderen Vereinbarung mit SAP. Dieses Dokument enthält nur vorgesehene Strategien, Entwicklungen und Funktionen des SAP -Produkts und ist für SAP nicht bindend, einen bestimmten Geschäftsweg, eine Produktstrategie bzw. -entwicklung einzuschlagen. SAP übernimmt keine Verantwortung für Fehler oder Auslassungen in diesen Materialien. SAP garantiert nicht die Richtigkeit oder Vollständigkeit der Informationen, Texte, Grafiken, Links oder anderer in diesen Materialien enthaltenen Elemente. Diese Publikation wird ohne jegliche Gewähr, weder ausdrücklich noch stillschweigend, bereitgestellt. Dies gilt u. a., aber nicht ausschließlich, hinsichtlich der Gewährleistung der Marktgängigkeit und der Eignung für einen bestimmten Zweck sowie für die Gewährleistung der Nichtverletzung geltenden Rechts. SAP übernimmt keine Haftung für Schäden jeglicher Art, einschließlich und ohne Einschränkung für direkte, spezielle, indirekte oder Folgeschäden im Zusammenhang mit der Verwendung dieser Unterlagen. Diese Einschränkung gilt nicht bei Vorsatz oder grober Fahrlässigkeit. Die gesetzliche Haftung bei Personenschäden oder die Produkthaftung bleibt unberührt. Die Informationen, auf die Sie möglicherweise über die in diesem Material enthaltenen Hotlinks zugreifen, unterliegen nicht dem Einfluss von SAP, und SAP unterstützt nicht die Nutzung von Internetseiten Dritter durch Sie und gibt keinerlei Gewährleistungen oder Zusagen über Internetseiten Dritter ab. Alle Rechte vorbehalten. SAP 2007 / Page 23