Technische Implementation von CERA

Transkript

1 Technische Implementation von CERA Hannes Thiemann Max-Planck-Institut für Meteorologie Modelle und Daten zmaw.de Jena, 24. Januar 2007

2 Inhalt Aufgabe und Motivation Umsetzung Datenbanken Anbindung an das HSM Ausblick 2

3 Klimasystem 3

4 Klimamodell: Grid 4

5 Klimamodell: Auflösung T42 (300 km) T106 (120 km) 5

6 Datenmengen Horizontalauflösung des Klimamodells T42: 128 * 64 = 8192 Punkte pro Globalfeld T106: 160 * 320 = Punkte pro Globalfeld Erforderliche Speichereinheiten (GRIB Format) Horizontalfeld (Zugriffseinheit): 17.1 kb (T42) / kb (T106) Unix Filegröße für monatsweise akkumulierte Ergebnisse mit 6 Std. Speicherintervall und 300 2d Variablen (Physikalische Einheit): 616 MB (T42) / 3500 MB (T106) 240 Jahre Modellintegration (Logische Einheit): 1.7 TB (T42) / 10 TB (T106) 6

7 7

8 8

9 9

10 10

11 Umsetzung Datenbanken 11

12 The Winter TopTen Program identifies the world s largest and most heavily used databases... Congratulations on achieving Grand Prize award winner status (1) in Database Size, Other, All and TopTen Winner status Database Size, Other, Linux;Workload, Other, Linux in Winter Corp.'s 2005 TopTen Program!... (1) Grand prizes are awarded for first place winners in the All Environments categories only. WDCC's CERA DB has been identified as the largest Linux DB. 12

13 Wintercorp (2005) - DB Size: Scientific, Archive, and other Company Size (TB) DBMS Platform System Vendor Max-Planck 222 Oracle Federated/SMP NEC USGS/EROS 17 Oracle Centralized/SMP Sun USGS/EROS 17 Oracle Centralized/SMP Sun HP 1 NonStop SQL Centralized/MPP HP T-Systems 1 Oracle RAC Centralized/Cluster Sun See: 13

14 Wintercorp (2005) - DB Size: Data Warehouse Company Size (TB) DBMS Platform System Vendor Yahoo 100 Oracle Centralized/SMP Fujitsu Siemens AT&T 1) 94 Daytona Federated/SMP HP KT IT-Group 50 DB2 Centralized/Cluster IBM LGR 25 Oracle Centralized/SMP HP Amazon 25 Oracle RAC Centralized/Cluster HP 1) 330 GB Norm. Data Volume See: 14

15 CERA: Some Facts Oracle 9.2 single instance running on TX7 Enterprise Edition Partitioning Option Advanced Security 24 Tbyte disk attached to database nodes Database size ~260 Tbyte (logical) Database nodes connected to HSM system Data accessible on the internet 800 named users worldwide Daily access 300 GB/Day (average) New data 250 GB/Day (average) 15

16 16

17 Users GFS Environment SX-6 1.Climate Model writes raw output (GFS I/O) Climate Model raw AP 2.PP reads raw data (GFS I/O) GFS/ Client Post processing System AP AP Post Process Application OCI Application PP writes data (local I/O) GFS/ Server META + Data 3.OCI reads data (Local I/O) Oracle Application Server 5.Data inquiry (OCI) Local disk Oracle AS GFS/Server DXDM AsAmA 16way DXSM DXSN GFS/Server DXDM AsAmA 16way DXSN Oracle Instance AsAmA 4way DXDB... Oracle Instance AsAmA 4way DXDB 4.OCI writes BLOB (via networks) Migration & Staging DiskXtender Disk cache Oracle DB BLOB GE Network Oracle DB BLOB NEC Corporation 17

18 WDCC Data Topology Level 1 - Interface: Metadata entries (XML, ASCII) + Data Files Experiment Description Pointer to Unix-Files Level 2 Interf.: Separate files containing BLOB table data in application adapted structure (time series of single variables) Dataset 1 Description BLOB Data Table Dataset n Description BLOB Data Table BLOB DB Table corresponds to scalable, virtual file at the operating system level. 18

19 Datenbanken: Aufteilung OID Daten Metadaten Enterprise User Security

20 Reference Contact Coverage Status Tabellen Entry 800 GB Parameter Distribution Data Org Local Adm. Data Access Spatial Reference

21 Data matrix of model experiment Model variables Model Run Time T1 T2 T3. Tn. Tend T2M Precip SLP 2D variables.. Temp Water vapour 3D variables.. Raw data file in DKRZ Archive 2 D: small BLOBS (16 KB) 3 D: large BLOBS (3 MB) Raw data file: direct model output ( GB) Each columm is one BLOB Table and one META Table in CERA-DB 21

22 Structure of metadata tables Metadata Table Blob_id Blob_size Start_date Blob_min Blob_max Blob_mean Informationen um Einfache Anfragen ohne Zugriff auf Daten selbst zu beantworten. Konsistenz zu den Daten selbst überprüfen zu können. Qualitätskontrollen durchzuführen. Liegen auf Disk Metadaten erlauben die Abbildung der blob_id auf die wirkliche Modellzeit 22

23 Structure of blob tables Range Partitioning Table Partition 1 Datafile blob_id 1.. n Time t 0.. t n BLOB Data Table blob_id blob_data Table Partition 2 Table Partition n Datafile blob_id n m Datafile blob_id m+1 n.. k Time t n+1.. t m Time t m+1.. t k 23

24 Umsetzung: HSM Anbindung an das HSM 24

25 Migin Migout dxdb TBS - RO Tbl Partition 1 All tablespaces are moved at once to dxdb TBS - RW Tbl Partition 21 25

26 Migout / Migin Migout takes place after files haven t been modified for x minutes Only one migout process per dxdb-filesystem Migin takes place immediately after a file is requested. Only parts accessed are retrieved from the backend storage. One migin process per requested file. 26

27 Purging dxdb HWM LWM 27

28 Criteria for purging Size of datafiles doesn t matter Except: small datafiles can stay on disk Time not modified (easy for read only tablespaces) Time not touched Oracle has the tendency to touch data files quite often Oracle parameter read_only_open_delayed could be an option Prerequisite: 2 copies on tape 28

29 Inside the datafile Header 128k Table Lob Index Primary Key Blob data 29

30 Frontend versus Backend Filesystem Frontend HSM Backend Header 128k Header 128k Part 1 = 512 MB Part 2 = 512 MB 30

31 Retrieving data Header 128k Tape Request 31

32 Usage: Downloads Downloads per year

33 Statistics: Size Database Size TByte Year 33

34 Ausblick: Globalmodell T213 (Atmosphäre) Horizontalauflösung des Klimamodells T213: 640 * 320 = Punkte pro Globalfeld T106: 160 * 320 = Punkte pro Globalfeld Erforderliche Speichereinheiten (GRIB Format) Horizontalfeld (Zugriffseinheit): kb (T213) / kb (T106) Unix Filegröße für monatsweise akkumulierte Ergebnisse mit 6 Std. Speicherintervall und 300 2d Variablen (Physikalische Einheit): 14000MB (T213) / 3500 MB (T106) 240 Jahre Modellintegration (Logische Einheit): 40 TB (T213) / 10 TB (T106) 34

35 Ausblick: Regionalmodell Auflösung und Datenmengen REMO-UBA-Modellgebiet Auslösung: 10x10 km Datenmenge: 5 TB / 100 Jahre (nur Bodenfelder) Orography 35

36 Temperatur Winter Sommer 36

37 Vielen Dank! 37