SQL Mystery I Anatomie eines SQL Befehls Lothar Flatz Senior Principal Consultant

Transkript

1 Software & Informatik SQL Mystery I Anatomie eines SQL Befehls Lothar Flatz Senior Principal Consultant

2 Ich stelle mich vor... Wer bin ich? über 25 Jahre Oracle Database Erfahrung (beginnend mit Version 5) 15 jahre Oracle Mitarbeiter Oak Table Mitglied Ex-Real World Performance Group Oracle ACE Signatur Project: PVSS (CERN) Patent US B2 zusammen mit Björn Engsig

3 Gründung 1994 Rechtsform Standorte Angestellte Aktiengesellschaft in privater Hand Baar ZG und Oberdorf SO 10 Mitarbeitende Zertifizierung SQS Zertifiziertes Managementsystem ISO 9001 Software Beschaffung, Lizenz Management, Software Paketierung, Beratung, Enterprise Reporting

4 Anatomie eines SQL Befehls Arbeit an Performance Problemen

5 Typische Anfrage auf OTN We just migrated to a new dev environment in Linux REDHAT5, and now the query is very slow. I checked toad monitor, it said need to increase db_buffer_cache and shared pool too small.

6 Typical Antwort If you want more help here, let's start with some DATA. Without this, people here can only guess at the actual cause of your problem, although if enough of us guess for a long enough time, somebody will undoubtedly find the right answer eventually. Senior DBA, Calagry

7 Reaktion des IT Management

8 Motivation für Sherlock Holmes In which the hero would treat crime as Dr Bell treated disease and where science would take the place of chance. Dr. Joseph Bell, Das reale Vorbild für Sherlock Holmes (Arthur Conan Doyle)

9 Realer Fall: Hilferuf eines Kunden Das I/O Subsystem ist überlastet Oracle Partner hat das Problem schon untersucht Es wurde kein Fortschritt erzielt

10 Realer Fall: Junior Berater Was normalerweise nicht schaden kann: 1. SGA Speicher erhöhen 2. Statistiken aktualisieren

11 Realer Fall: Senior Berater Ein einziger Befehl verursacht die Misere: SELECT Wir müssen den Applikationshersteller fragen, was sich verändert hat. Die SGA erhöhen und die Statistiken aktuallisieren hat Nichts gebracht.

12 Was soll der erste Schritt sein? Francis Bacon, Philosoph and Scientist Man, being the servant and interpreter of nature, can do and understand so much and so much only as he has observed in fact or in thought of the course of nature.

13 Was soll der erste Schritt sein? It is a capital mistake to theorize before you have all the evidence. It biases the judgment. Sherlock Holmes Quote -A Study in Scarlet

14 Messe! Id Operation Name Starts E-Rows A-Rows A-Time Buffers Reads SELECT STATEMENT :00: K 237K 1 SORT ORDER BY :00: K 237K 2 SORT UNIQUE :00: K 237K 3 UNION-ALL :00: K 237K * 4 HASH JOIN :00: * 5 HASH JOIN :00: * 6 TABLE ACCESS BY INDEX ROWID MEDI_B :00: * 7 INDEX RANGE SCAN I :00: * 8 TABLE ACCESS BY INDEX ROWID MEDI_C :00: * 9 INDEX RANGE SCAN I :00: INDEX FAST FULL SCAN I K 279K 00:00: * 11 HASH JOIN :00: K 237K * 12 HASH JOIN :00: K 237K * 13 TABLE ACCESS BY INDEX ROWID MEDI_C :00: * 14 INDEX RANGE SCAN I :00: * 15 HASH JOIN K 00:00: K 237K * 16 TABLE ACCESS FULL MEDI_A K 00:00: K 237K * 17 TABLE ACCESS FULL MEDI_B 1 598K 598K 00:00: INDEX FAST FULL SCAN I K 279K 00:00:

15 Richtlinien Kann der Befehl optimiert werden? Was ist der Kern des Problems? Was verursacht das Problem? Wie kann das Problem gelöst werden?

16 Kann der Befehl optimiert werden? Martin Berg, Oak Table Member, Tuning Queries by Eliminating Throwaway Internet whitepaper Most badly performing queries usually exhibits a high amount of rows processed and then thrown away because they were not needed in the output.

17 Kann der Befehl optimiert werden? Id Operation Name Starts E-Rows A-Rows A-Time Buffers Reads SELECT STATEMENT :00: K 237K 1 SORT ORDER BY :00: K 237K 2 SORT UNIQUE :00: K 237K 3 UNION-ALL :00: K 237K * 4 HASH JOIN :00: * 5 HASH JOIN :00: * 6 TABLE ACCESS BY INDEX ROWID MEDI_B :00: * 7 INDEX RANGE SCAN I :00: * 8 TABLE ACCESS BY INDEX ROWID MEDI_C :00: * 9 INDEX RANGE SCAN I :00: INDEX FAST FULL SCAN I K 279K 00:00: * 11 HASH JOIN :00: K 237K * 12 HASH JOIN :00: K 237K * 13 TABLE ACCESS BY INDEX ROWID MEDI_C :00: * 14 INDEX RANGE SCAN I :00: * 15 HASH JOIN K 00:00: K 279K 237K * 16 TABLE ACCESS FULL MEDI_A K 00:00: K 237K * 17 TABLE ACCESS FULL MEDI_B 1 598K 598K 00:00: INDEX FAST FULL SCAN I K 279K 00:00: A-Rows K 957K 598K 279K

18 Was ist der Kern des Problems? Dr. Wolfgang Breitling, Oak Table Member, Tuning with Cardinality Feedback, 2006 Hotsos Symposium Whenever the CBO chooses a bad plan it can be traced back to an error in the estimation of the cardinality of one or more row sources.

19 Was ist der Kern des Problems? Id Operation Name Starts E-Rows A-Rows A-Time Buffers Reads SELECT STATEMENT :00: K 237K 1 SORT ORDER BY :00: K 237K 2 SORT UNIQUE :00: K 237K 3 UNION-ALL :00: K 237K * 4 HASH JOIN :00: * 5 HASH JOIN :00: * 6 TABLE ACCESS BY INDEX ROWID MEDI_B :00: * 7 INDEX RANGE SCAN I :00: * 8 TABLE ACCESS BY INDEX ROWID MEDI_C :00: * 9 INDEX RANGE SCAN I :00: INDEX FAST FULL SCAN E-Rows MEDI_C A-Rows 1 279K A-Time 279K 00:00: * 11 HASH JOIN :00: K 237K * 12 HASH JOIN :00: K 237K * 13 TABLE ACCESS BY INDEX ROWID MEDI_C :00: * 14 INDEX RANGE SCAN I :00: * 15 HASH JOIN K 00:00: K 237K * 16 TABLE ACCESS FULL 7161 MEDI_A 957K 00:00: K 237K * 17 TABLE ACCESS FULL MEDI_B 1 598K 598K 00:00: INDEX FAST FULL SCAN I K 279K 00:00:

20 Was ist der Kern des Problems? Focus on the step that takes the most time. Cary Millsap, Oak Table Member, entrepreneur and Oracle software performance specialist

21 Was ist der Kern des Problems? 16 filter (CreationDate>:loaddate OR lastchangedate>:loaddate) AND TO_NUMBER(effective(id))=1

22 Was ist der Kern des Problems? Isoliere deine Variablen. Ein vitaler Gesichtspunkt bei der Durchführung eines Experiments ist es, nur die spezifische Variable, die du testest, anzupassen (und nicht die anderen Variablen).

23 Was ist der Kern des Problems? 16 filter (CreationDate>:loaddate OR lastchangedate>:loaddate) AND TO_NUMBER(effective(id))=1

24 Was ist der Kern des Problems? Condition Estimate Real Error Factor CD & LUD 806,000 4,924,104 6 CD 403,000 4,784, LUD 426,000 4,924,104 12

25 Was ist der Kern des Problems? x + a n = n k=0 n k xk a n k Jonathan Lewis, Oak Table Member, Ordinary Genius and CBO Expert

26 Rows Anatomie eines SQL Befehls Was verursacht das Problem? 7,344,000? Dates

27 Rows Anatomie eines SQL Befehls Was verursacht das Problem? 7,344,000 Annahme: kürzlicher Anstieg der Datenmenge

28 Wie kann das Problem gelöst werden? Christian Antognini, Senior Principal Consultant, Oak Table Member Before you apply tuning techniques make sure the initialization parameters are correctly set and all the necessary system and object statistics are in place.

29 Wie kann das Problem gelöst werden? exec dbms_stats.gather_schema_stats(user, estimate_percent=> 100, method_opt=> 'for all columns size skewonly')

30 Wie kann das Problem gelöst werden? Id Operation Name Starts E-Rows A-Rows A-Time Buffers SELECT STATEMENT :00: SORT AGGREGATE :00: * 2 INDEX FAST FULL SCAN MEDI_A_I K 4924K 00:00: Predicate Information (identified by operation id): filter(("last_update_date">:ladedatumzeit OR "CREATION_DATE">:LADEDATUMZEIT))

31 Was ist der Kern des Problems? Condition Estimate Real Error Factor TO_NUMBER(effective (id))=1 73,440 7,344,

32 Was ist der Kern des Problems? FUNCTION effective (p_id IN VARCHAR2) RETURN VARCHAR2 IS BEGIN -- IF p_id = '0 -- OR p_id = current_id -- OR current_id = '0 -- THEN RETURN('1'); -- ELSE -- RETURN('0'); -- END IF; END;

33 Was verursacht das Problem? The default selectivity for a unknown functions is 1%. Joze Senegacnik, Oak Table Member

34 Wie kann das Problem gelöst werden? ASSOCIATE STATISTICS WITH FUNCTIONS effective DEFAULT SELECTIVITY 100;

35 Wie kann das Problem gelöst werden? The best execution plan usually starts with the best filter condition and minimizes reads to the largest tables. Dr. Daniel Tow, former head of Oracle Applications tuning was lecturing sql tuning at Stanford University, USA. NoCoug Journal, September 2011

36 Wie kann das Problem gelöst werden? Dan Tow intuitiv erklärt: Wie das Wasser sich den kürzesten Weg von einem Berg sucht, indem es die tiefsten Punkte verbindet

37 Wie kann das Problem gelöst werden? The source of biggest throw away will often indicate the point of best selectivity Lothar Flatz, Oak Table Member

38 Wie kann das Problem gelöst werden? Id Operation Name Starts E-Rows A-Rows A-Time Buffers Reads SELECT STATEMENT :00: K 237K 1 SORT ORDER BY :00: K 237K 2 SORT UNIQUE :00: K 237K 3 UNION-ALL :00: K 237K * 4 HASH JOIN :00: * 5 HASH JOIN :00: * 6 TABLE ACCESS BY INDEX ROWID MEDI_B :00: * 7 INDEX RANGE SCAN I :00: * 8 TABLE ACCESS BY INDEX ROWID MEDI_C :00: * 9 INDEX RANGE SCAN I :00: INDEX FAST FULL SCAN I K K 00:00: * 11 HASH JOIN :00: K 237K * 12 HASH JOIN :00: K 237K * 13 TABLE ACCESS BY INDEX ROWID MEDI_C :00: * 14 INDEX RANGE SCAN I :00: * 15 HASH JOIN K 00:00: K 237K * 16 TABLE ACCESS FULL MEDI_A K 957K 00:00: K 237K * 17 TABLE ACCESS FULL MEDI_B 1 598K K 00:00: INDEX FAST FULL SCAN I K 279K 00:00: A-Rows K 957K 598K 279K

39 End Resultat mit Hints Id Operation Name Starts E-Rows A-Rows A-Time Buffers SELECT STATEMENT :00: SORT ORDER BY :00: SORT UNIQUE :00: UNION-ALL :00: * 4 HASH JOIN :00: * 5 HASH JOIN :00: * 6 TABLE ACCESS BY INDEX ROWID MEDI_B :00: * 7 INDEX RANGE SCAN I2_X :00: * 8 TABLE ACCESS BY INDEX ROWID MEDI_C :00: * 9 INDEX RANGE SCAN I :00: INDEX FAST FULL SCAN I K 0 00:00: CONCATENATION :00: NESTED LOOPS :00: NESTED LOOPS :00: NESTED LOOPS :00: * 15 TABLE ACCESS BY INDEX ROWID MEDI_C :00: * 16 INDEX RANGE SCAN I :00: TABLE ACCESS BY INDEX ROWID MEDI_B :00: * 18 INDEX RANGE SCAN I2_X :00: TABLE ACCESS BY INDEX ROWID MEDI_A :00: * 20 INDEX RANGE SCAN i4_x :00: * 21 INDEX UNIQUE SCAN I :00: NESTED LOOPS :00: NESTED LOOPS :00: NESTED LOOPS :00: * 25 TABLE ACCESS BY INDEX ROWID MEDI_C :00: * 26 INDEX RANGE SCAN I :00: TABLE ACCESS BY INDEX ROWID MEDI_B :00: * 28 INDEX RANGE SCAN I2_X :00: * 29 TABLE ACCESS BY INDEX ROWID MEDI_A :00: * 30 INDEX RANGE SCAN I4_X :00: * 31 INDEX UNIQUE SCAN I :00:

40 Ergebnis Database überlädt das I/O sub system => logical reads reduziert um Faktor > 50 => physical reads reduziert auf beinahe Null: Mission abgeschlossen (?)

41 Ruessenstrasse Baar ZG (Switzerland) Phone: +41 (0) Direct: +41 (0) Fax: +41 (0)