Space The Final Frontier

DOAG Konferenz 20. - 22.11.2012 Space The Final Frontier Martin Hoermann info@ordix.de www.ordix.de

Oracle7 Server Concepts Manual 1

Bedienungsanleitung segment.sql FAQ: 340512.1 Timestamp Ohne geht s leider nicht Zusätzliche Unterlagen http://www.interessante.verweise/ Literatur Infos zum Nachschlagen http://www.amazon.de 2

Agenda Tablespaces & Data Files Segments & Extents Blocks & Datentypen Reorganisation Kompression Fazit 3

Unendliche Weiten CREATE TABLESPACE data DATAFILE '/oracle/app/oracle/oradata/startrek/data01.dbf' SIZE 200M AUTOEXTEND ON NEXT 100M MAXSIZE 2G EXTENT MANAGEMENT LOCAL AUTOALLOCATE SEGMENT SPACE MANAGEMENT AUTO; 4

Datafile Fileheader (z. B. 8K) Datafile Extent Map (z. B. 1M) Freies Extent 200 MB aus SIZE Klausel Dateigröße im Betriebssystem 5

Data Dictionary DBA_TABLESPACE_USAGE_METRICS TABLESPACE_SIZE USED_SPACE (in Blöcken) YES USER_BYTES BYTES DBA_DATA_FILES AUTOEXTENSIBLE MAXBYTES 6

Kilo, Mega, Giga, Tera, Peta, Exa Bereich 2er Potenz Entspricht Byte 2 0 1 oder 0 Kilobyte 2 10 1024 Byte Megabyte 2 20 1024 Kilobyte Gigabyte 2 30 1024 Megabyte Terabyte 2 40 1024 Gigabyte Petabyte 2 50 1024 Terabyte Exabyte 2 60 1024 Petabyte 7

Smallfile vs. Bigfile Tablespace 8 KB Block Size Smallfile Bigfile Max. Datafile 32 GB 32 TB Anzahl Datafiles je Tablespace 1.022 1 Summe 31,94 TB 32 TB Physical Database Limit: 65.533 Datendateien pro Datenbank Smallfile: ca. 64 Tablespaces Bigfile: 65.533 Tablespaces 8

Tablespace & Co 10g: BIGFILE Type Tablespaces Versus SMALLFILE Type [ID 262472.1] Master Note for Tablespace Issues [ID 1270839.1] Database Limits [ID 336186.1] What is the Maximum Datafile Size Limit In an Oracle Database? [ID 804733.1] 9

Bigfile Tablespace Database Maximale Größe der Datenbank (vereinfacht ohne Ausnahmen) 8 KB Block Size Smallfile Bigfile Max. Datafile 32 GB 32 TB Anzahl Datafiles 65533 (2 16 3) Summe 2.047 TB 2.097.056 TB ~ 2 PT ~ 2 EB Keine Bigfile Tablespaces bei UNDO TEMP SYSTEM 10

ASM-Limit Noch mehr speicher. Exadata ASM-Disk ASM total Nein 2 TB 20 PB Ja 4 PB 40 EB Vergrößert nicht das maximale Volumen einer einzelnen Datenbank 11

Früher war alles besser SYSTEM DATA INDEX REDO1 REDO2 12

Stripe an Mirror Everything SYSTEM DATA INDEX REDO1 REDO2 SAME 13

Think about it Trennung von Tabellen und Indizes sinnvoll? http://richardfoote.wordpress.com/2008/04/16/separate-indexes-from-tables-some-thoughts-part-i-everything-in-its-right-place/ http://richardfoote.wordpress.com/2008/04/18/separate-indexes-from-tables-some-thoughts-part-ii-there-there/ 14

Wie viele Tablespaces? Interne Tablespaces SYSTEM SYSAUX UNDO TEMP Tablespaces der Anwendungen Starte mit einem Tablespace, Ausnahmen wenn: Sehr viele / große Read Only Objekte (kürzere Backups) Datenbewirtschaftung mit Transportable Tablespaces Sehr große Datendateien wegen langer Restore-Zeiten 15

Was ist ein Segment segments.sql TYP Objekt? Segment? Table Ja Ja Index Ja Ja View Ja Nein Trigger Ja Nein Rollback Nein Ja Lobindex Nein Ja dynamischer Speicherbedarf Speicherbedarf nur Definition z. B. DB Verwaltung 17

Warums Extents Platzverbrauch vs. Anzahl Extents Jedes Extent = 1 Block Jedes Extent ist zu 100% gefüllt (Blockfüllgrad später) Jedes Extent = 64 MB Optimierung von Full Table Scans (db_file_multiblock_read_count) Optimales Pre-Fetch im Storage System Verwaltung Extents sehr sparsam 18

Locally Managed Tablespace Uniform Size Tablespace SIZE_S SIZE_M SIZE_L Size 64K 1M 64M Idee Je Tablespace eine ideale Größe Keine Extent-Fragmentierung Geringe Anzahl Extents Jedes Segment vorgesehene Größe 19

Locally Managed Tablespace Autoallocate Größe Anzahl 64 K 16 1 M 63 8 M 120 64 M 64 MB 1 GB Extent-Größe abhängig von Oracle Version partitioniert oder nicht Platz im Tablespace Parallele Erstellung z. B. 8 MB in 11.2.0.2 Partitionen and we said... Autoallocate is by definition Autoallocate - it is automatic and is not predicable. Well, it is predicable - but only version by version - it changes with releases. 20

Uniform vs. Autoallocate Followup November 30, 2006-8am Central time zone: Actually, over the years since this was published - I've come to recognize that system allocated extents are very nice. I've changed my position on this entirely. Do not use uniform - just use system allocated extent sizes. Segments will start small and grow in extent size as needed. http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::p11_question_id:228413960506 21

DBA_EXTENTS SEGMENT EXTENT_ID BLOCK_ID BLOCKS PETROL 0 128 8 YELLOW 0 136 8 GRAY 0 144 8 YELLOW 1 160 8 PETROL 1 168 8 YELLOW 2 176 8 22

DBA_FREE_SPACE BLOCK_ID BLOCKS 152 8 448 25024 23

Deferred Segment Creation (11.2.0) Initiales Extent wird mit dem ersten Datensatz angelegt CREATE TABLE data ( id NUMBER PRIMARY KEY... ) SEGMENT CREATION DEFERRED ; 24

Deferred Segment Creation Keine Unterstützung Für Objekte der Schemata SYS, SYSTEM, PUBLIC, OUTLN, oder XDB Für index-organized tables, clustered tables, globale und Session-spezifische temporary tables, internal tables, typed tables, AQ tables, external tables Für Bitmap Join und Domain Indizes Für nicht-partitionierte Objekte in Dictionary-managed Tablespaces Exp exportiert keine Segmente ohne Extents Fehler bei Insert innerhalb von serialisierbaren Transaktionen Quota Prüfung erst beim Insert Erste Sequenznummer geht verloren http://tkyte.blogspot.de/2011/02/deferred-segment-creation.html http://www.oracle.com/webfolder/technetwork/de/community/dbadmin/tipps/segments_on_demand/index.html 25

Deferred Segment Creation 11.2 Database New Feature Deferred Segment Creation [Video] [ID 887962.1] Bug 9303326 - OERI [kcbgtcr_1a] from MAX/MIN on table with deferred segment creation Bug 10126822 - ALTER TABLE.. ADD COLUMN causes problems with deferred segment creation Bug 9638090 - ORA-8176 from DML on an INTERVAL Partitioned Table OR table with deferred segment creation Bug 8911160 - ALTER TABLE MOVE creates segment with segment creation deferred table E-ORA: ERROR at line 73: ORA-00922: missing or invalid option / Segment Creation Deferred Bug 14252187 - ORA-600 [qesmagettblseg1] from deferred segment creation in RAC Bug 11775474 - DBMS_REDEFINITION does not use deferred segment creation in 11.2.0.2 Bug 13326736 ORA-604 ORA-959 on object with Deferred Segment Creation Bug 13497523 - Errors from SQLLDR loads into non-partitioned tables with deferred segment creation 26

Truncate Table TRUNCATE TABLE [DROP STORAGE] High Water Mark Extent 0 Extent 1 Extent 2 Extent 0 High Water Mark Annahme: MINEXTENTS = 1 27

Truncate Table TRUNCATE TABLE REUSE STORAGE High Water Mark Extent 0 Extent 1 Extent 2 High Water Mark Extent 0 Extent 1 Extent 2 28

Truncate Table TRUNCATE TABLE DROP ALL STORAGE High Water Mark Extent 0 Extent 1 Extent 2 Extent 0 High Water Mark 29

DBMS_SPACE_ADMIN (11.2.0.2) BEGIN dbms_space_admin.materialize_deferred_segments ( schema_name => user, table_name => NULL -- ALLE ); dbms_space_admin.drop_empty_segments (schema_name => user, table_name => 'TABLE_XYZ' ); END; / High Water Mark High Water Mark Extent 0 Extent 0 30

Unendliche Weiten CREATE TABLE data ( id NUMBER PRIMARY KEY... ) SEGMENT CREATION IMMEDIATE ; 32

Blöcke Extent 0 Class 8 = 1 st Level Bitmap Block Class 9 = 2 nd Level Bitmap Block Class 4 = Segment Header Class 1 = Data SELECT * FROM v$bh; 33

Block Classes Class# Class 1 data block 2 sort block 3 save undo block 4 segment header 5 save undo header 6 free list 7 extent map 8 1st level bmb 9 2nd level bmb 10 3rd level bmb 11 bitmap block 12 bitmap index block 13 file header block 14 unused 15 system undo header 16 system undo block 17 undo header 18 undo block SELECT * FROM v$fixed_view_definition WHERE view_name = 'GV$WAITSTAT' 34

Block Structure (Class 1, data block) FILE#=7 BLOCK#=47 SCN=8889 TYPE=DATA OBJECT#=654 NextFreeBlock=121 1 I 1 4-1 15... 2 I 1 8 1 12... 3 1 8 7 9... I RBS 7 SCN 9711 TransX 12... Block Header Row Directory Interested Transaction List (ITL) Datenbereich 3 Schröder Gerhard Plümeke 7... 2 Adenauer Kirchplatz 3. 1 Kohl An der Eiche 10... SCN=8889 Block Footer 35

Block Structure, Row Directory FILE#=7 BLOCK#=47 SCN=8889 TYPE=DATA OBJECT#=654 NextFreeBlock=121 1 I 1 4 0 15... 2 I 1 8 1 12... 3 1 8 7 9... I RBS 7 SCN 9711 TransX 12... Row Directory Row-Number Referenz auf ITL (Sperre) Länge in Bytes und damit Adresse der Felder Länge 0 = NULL NULL-Felder am Ende zusammengefasst 3 Schröder Gerhard Plümeke 7... 2 Adenauer Kirchplatz 3. 1 Kohl An der Eiche 10... SCN=8889 http://jonathanlewis.wordpress.com/2012/10/01/row-sizes-2/ 36

Block Structure FILE#=7 BLOCK#=47 SCN=8889 TYPE=DATA OBJECT#=654 NextFreeBlock=121 1 I 1 4-1 15... 2 I 1 8 1 12... 3 1 8 7 9... I RBS 7 SCN 9711 TransX 12... Interested Transaction List Referenz auf Transaktion die den Block verändert Bereich dynamisch (MINTRANS MAXTRANS) Verweis vom Row Directory 3 Schröder Gerhard Plümeke 7... 2 Adenauer Kirchplatz 3. 1 Kohl An der Eiche 10... SCN=8889 37

Block Structure FILE#=7 BLOCK#=47 SCN=8889 TYPE=DATA OBJECT#=654 NextFreeBlock=121 1 I 1 4-1 15... 2 I 1 8 1 12... 3 1 8 7 9... I RBS 7 SCN 9711 TransX 12... 3 Schröder Gerhard Plümeke 7... 2 Adenauer Kirchplatz 3. 1 Kohl An der Eiche 10... SCN=8889 Datenbereich Nur Werte (keine Feldtrenner) Wächst dynamisch pctfree Blockfüllgrad: Code Bedeutung 0000 Unformatted block 0001 Block logically full 0010 <25% free space 0011 25% - 50% free 0100 50% - 75% free 0101 >75% free space 38

Datentypen - Date Gruppe Datentyp Bytes Datum DATE 7 TIMESTAMP(6) 11 TIMESTAMP (6) WITH TIMEZONE 13 TIMESTAMP (6) WITH LOCAL TIMEZONE 11 Wähle alle Datentypen die den Anforderungen genügen Wähle von alle potentiellen Kandidaten den mit der geringsten Größe 39

Datentypen - Text Gruppe Datentyp Größe Text CHAR Je nach Definition VARCHAR2 / NVARCHAR2 Abhängig vom Inhalt CLOB / NCLOB / BLOB Abhängig vom Inhalt VARCHAR NLS_LENGTH_SEMANTICS VARCHAR2( 20 BYTE ) VARCHAR2( 20 CHAR ) Character Set Character Set / National Character Set Single Byte vs. Multi Byte 40

Datentypen - Zahl Gruppe Datentyp Größe Zahl NUMBER Abhängig vom Inhalt BINARY_FLOAT 4 BINARY_DOUBLE 4 NUMBER 9.999.999.999 = 6 Byte 10.000.000.000 = 2 Byte Wert = 1/3 NUMBER(10,3) = 3 Byte NUMBER = 21 Byte 41

Datentypen NULL? Kein Platzverbrauch (auch CHAR) im Datenbereich Benötigt 1 Byte im Row-Header Trailing NULL-Columns werden auf 1 Byte komprimiert Analyse dump vsize / sys_op_opnsize http://jonathanlewis.wordpress.com/2012/09/27/row-sizes/ 42

DBMS_SPACE SQL> call show_space('tcom_performance_data'); Unformatted Blocks... 784 FS1 Blocks (0-25)... 1 FS2 Blocks (25-50)... 0 FS3 Blocks (50-75)... 0 FS4 Blocks (75-100)... 225 Full Blocks... 172,410 Total Blocks... 174,080 Unused Blocks... 0 http://asktom.oracle.com/pls/asktom/f?p=100:11:::::p11_question_id:231414051079 43

DBMS_SPACE Extent 0 Extent 1 Extent 2 Extent 3 SPACE_USAGE UNFORMATTED FS1-0% - 25% free FS2-25% - 50% free FS3-50% - 75% free FS4-75% - 100% free UNUSED_SPACE Last Used Block Last Extent File ID Last Extent Block ID Unused Blocks FULL_BLOCKS 44

Reorganisation Warum? Verdichtung der Daten Sortierung der Daten Verfahren Innerhalb des Segments oder außerhalb? Online vs. Offline Optionen wie Sortieren verfügbar? 46

Segment Reorganisation Extents HWM exp / imp Copy ALTER TABLE MOVE dbms_redefinition HWM Segment Header Block leer <25% <50% <75% <100% =100% 47

Segment Shrink Extents Shrink Compact HWM Shrink Cascade HWM HWM Segment Header Block leer <25% <50% <75% <100% =100% 48

Think about it Nachhaltig messbare Vorteile? 49

Kompression Index Compression Table Compression DWH Compression OLTP Compression Secure File Compress Deduplicate Exadata: Hybrid Columnar Compression 51

Indexkompression No Compress Compress Value RowID Meier 0x4711 Meier 0x2311 Meier 0x0815 Meyer 0x3333 Meyer 0x3477 Value RowID [... ] Meier 0x4711 0x2311 0x0815 Meyer 0x3333 0x3477 Meyerei 0x2222 52

Indexkompression analysieren ANALYZE INDEX i_name VALIDATE STRUCTURE; kunde id name vorname SELECT opt_cmpr_count, opt_cmpr_pctsave FROM index_stats; ALTER INDEX i_name REBUILD ONLINE COMPRESS 1; dba_indexes: blevel / leaf_blocks dba_segments: blocks Tuning I/O intensiver Systeme mit 9i DOAG Konferenz 2003

Tabellenkompression Tabelle PODIUM_LE_TOUR Tabelle PODIUM_LE_TOUR_COMP 02 02 02 01 01 01 00 00 00 04 08 05 07 03 06 05 08 0B Andreas KLÖDEN 07 0A 03 06 09 Alexandre VINOKOUROV 04 07 TEAM BIANCHI 03 06 09 0A 0B DISCOVERY CHANNEL TEAM 08 TEAM TELEKOM Daten- Tabelle 2005 2005 2005 2004 2004 2004 2003 2003 2003 Jan ULLRICH Ivan Basso 3 2 T-MOBILE TEAM TEAM CSC Lance ARMSTRONG 1 DISCOVERY CHANNEL TEAM Ivan Basso 3 TEAM CSC Andreas KLÖDEN 2 T-MOBILE TEAM Lance ARMSTRONG 1 US POSTAL-BERRY FLOOR Alexandre VINOKOUROV 3 TEAM TELEKOM Jan ULLRICH 2 TEAM BIANCHI Lance ARMSTRONG 1 US POSTAL-BERRY FLOOR S0B S0A S09 S08 S07 S06 S05 S04 S03 S02 S01 S00 TEAM CSC T-MOBILE TEAM US POSTAL-BERRY FLOOR 3 2 1 Ivan Basso Jan ULLRICH Lance ARMSTRONG 2005 2004 2003 Symbol- Tabelle 54

Secure File - Compress Lustig weihnachtsmann.ppt (7 MB) Compress 55

Secure File - Deduplicate Lustig weihnachtsmann.ppt (7 MB) Deduplicate 56

Think about it Hohe Kompression = schlechtes Datenmodell? 57

Tablespace 1 tablespace1.sql SELECT u.tablespace_name, ROUND( u.used_space*t.block_size / 1048576 ) AS used_space_mb, ROUND( u.tablespace_size*t.block_size / 1048576 ) AS tablespace_size_mb, round( u.used_percent, 2 ) AS used_percent FROM dba_tablespace_usage_metrics u, dba_tablespaces t WHERE u.tablespace_name = t.tablespace_name ORDER BY u.tablespace_name; 59

Tablespace 2 tablespace2.sql SELECT groesse_mb.tablespace_name, round( groesse_mb.a ) gesamt, round( nvl( groesse_belegt.b, 0 )) belegt, round( groesse_mb.a - nvl( groesse_belegt.b, 0 )) frei, round(( groesse_mb.a - nvl( groesse_belegt.b, 0 )) / groesse_mb.a * 100 ) pct_free, round( nvl( groesse_belegt.b, 0 ) / groesse_mb.a * 100 ) pct_used FROM ( SELECT tablespace_name, sum( bytes ) / 1024 / 1024 a FROM dba_data_files GROUP BY tablespace_name UNION SELECT tablespace_name, sum( bytes ) / 1024 / 1024 a FROM dba_temp_files GROUP BY tablespace_name ) groesse_mb, ( SELECT tablespace_name, sum( b ) b FROM ( SELECT tablespace_name, sum( bytes ) / 1024 / 1024 b FROM dba_segments GROUP BY tablespace_name UNION SELECT t.tablespace_name, sum( v.used_blocks * t.block_size ) / 1024 / 1024 b FROM v$sort_segment v, dba_tablespaces t WHERE v.tablespace_name = t.tablespace_name GROUP BY t.tablespace_name ) GROUP BY tablespace_name ) groesse_belegt WHERE groesse_belegt.tablespace_name (+) = groesse_mb.tablespace_name ORDER BY groesse_mb.tablespace_name; 60

Tablespace 2 verfügbarer Platz tablespace2.sql SELECT tablespace_name, sum( bytes ) / 1024 / 1024 a FROM dba_data_files GROUP BY tablespace_name UNION SELECT tablespace_name, sum( bytes ) / 1024 / 1024 a FROM dba_temp_files GROUP BY tablespace_name 61

Tablespace 2 belegter Platz tablespace2.sql SELECT tablespace_name, sum( bytes ) / 1024 / 1024 b FROM dba_segments GROUP BY tablespace_name UNION SELECT t.tablespace_name, sum( v.used_blocks * t.block_size ) / 1024 / 1024 b FROM v$sort_segment v, dba_tablespaces t WHERE v.tablespace_name = t.tablespace_name GROUP BY t.tablespace_name 62

Show Space show_space.sql create or replace procedure show_space ( p_segname in varchar2, p_owner in varchar2 default user, p_type in varchar2 default 'TABLE', p_partition in varchar2 default NULL )... SQL> call show_space('tcom_performance_data'); Unformatted Blocks... 784 FS1 Blocks (0-25)... 1 FS2 Blocks (25-50)... 0 FS3 Blocks (50-75)... 0 FS4 Blocks (75-100)... 225 Full Blocks... 172,410 Total Blocks... 174,080 Unused Blocks... 0 63

Dump dump.sql SELECT dump( a_date ) FROM test; 64

Zentrale Paderborn Westernmauer 12-16 33098 Paderborn Tel.: 05251 1063-0 Seminarzentrum Wiesbaden Kreuzberger Ring 13 65205 Wiesbaden Tel.: 0611 77840-00 Zentrales Fax: 0180 1 67349 0 0180 1 ORDIX 0 Weitere Geschäftsstellen in Köln, Münster und Neu-Ulm E-Mail: info@ordix.de Internet: http://www.ordix.de Vielen Dank für Ihre Aufmerksamkeit!