JSON Daten in Oracle Datenbank 12c Release 2

Transkript

1 JSON Daten in Oracle Datenbank 12c Release 2 Ulrike Schwinn BU Core & Cloud Technologies Oracle Deutschland B.V. & Co KG Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 1

2 Warum überhaupt JSON in der Datenbank? Getrennte Datenhaltung hat Folgen... Umständliche Integration Langsame Interaktion Schwierige Verwaltung und Kontrolle Strukturierte Daten Texte, XML, Semantische Netze... Bilder, Videos... Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 2

3 Warum überhaupt JSON in der Datenbank? Ein einziges zentrales Repository => zentrales Management Nutzung der Mechanismen einer Datenbank wie Backup/Recovery und Security Massendatenverarbeitung und Performance Techniken wie Indizierung, Partitionierung, Parallelisierung, Laden der Daten usw. Einheitliche Werkzeuge Zugriff auf bestehende (relationale) Stammdaten und JSON Daten über SQL Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 3

4 Die Oracle Datenbankfür all your data Alle Arten von Daten mit Advanced Analytics, OLAP, R, PHP,.NET, SQL... select p.id, p.name, sl.json_doc.quantity anzahl from product p, customer c, store s, sales sl where sl.json_value(json_doc,'$.product_id')=p.id and contains(p.notes,fuzzy('lieferkosten'), 1) > 0 and sl.json_value(json_doc,'$.custnumber' returning number) = c.cust_ids and s.store_id = 1234 and sdo_within_distance(c.location, s.location,'distance=20')='true' Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 4

5 JSON in der Datenbank ein Kurzprofil Integraler Bestandteil der Datenbank seit 12c Release 1 ( ) Keine zusätzliche Installation oder Konfiguration erforderlich In allen Datenbank- und Cloud- Editionen Mit allen Standard Funktionen der Datenbank verwendbar Zugriff über SQL und PL/SQL Schnittstelle Funktionen und Operatoren für Zugriff über JSON Pfad Ausdrücke Indizierung - sogar linguistisch Relationale View Generierung JSON Generierung Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 5

6 Anforderung an einem Beispiel 1. Bestellungen (JSON Dokumente) Export Format erzeugt von üblichen NoSQL Datenbanken 2. Suche nach einer bestimmten Bestellung Name des Auftraggebers ist nicht eindeutig Herausforderungen Einfache übersichtliche Codierung Größe des(der) Dokument(e) Große Anzahl an Dokumenten Ungenauigkeit der Anfrage Hohe Performance Anforderungen Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 6

9 JSON Daten und die Datenbank Ausgangssituation: Daten liegen außerhalb der Datenbank im Export Format vor Oracle Beispiel: Vorgehensweise Zugriff über logisches Directory create directory json_dir as '/home/oracle/db-sample-schemas /order_entry'; External Table mit Spalte für die JSON Daten in der Datenbank create table json_contents (json_document CLOB) organization external (type oracle_loader default directory json_dir access parameters (records delimited by 0x'0A' fields (json_document CHAR(5000))) location ('PurchaseOrders.dmp')) reject limit unlimited; Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 9

10 Zugriffe: Dokumente, skalare Werte, Fragmente... SQL> select * from json_contents where rownum = 1; JSON_EXISTS, JSON_VALUE, JSON_QUERY SQL> select min(json_value (json_document, '$.PONumber' Returning number)) min_po, max(json_value (json_document, '$.PONumber' Returning number)) max_po from json_contents; SQL> select json_query(json_document,'$.shippinginstructions') from json_contents where json_value(json_document,'$.ponumber' returning number) = 1000; SQL> select count(*) from json_contents where json_exists(json_document, '$.ShippingInstructions.Address.zipCode'); Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 10

11 Tabelle Laden und Indizieren IS JSON Bedingung prüft auf Gültigkeit create table json_tab (id number generated as identity, json_document clob constraint ensure_json CHECK (json_document IS JSON)); insert into json_tab (json_document) select json_document from json_contents; Commit; Function Based Index für Tuning create index ponumber_idx on json_tab (json_value(json_document, '$.PONumber' returning number error on error)); Demo Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 11

13 JSON in Tabellenform: Relationale Projektionen PURCHASETAB_VIEW ponumber requestor special adresse 1 Martha Sullivan Surface Mail {"street":"200 Sporting Green","city : 2 Trenna Rajs Courier {"street":"2014 Oxford Rd","city": Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 13

14 Relationale Projektion mit JSON_TABLE create or replace view purchasetab_view as select jt. ponumber, jt.requestor, jt.adresse, jt.special from json_tab, json_table (json_document, '$' COLUMNS ( ponumber number PATH '$.PONumber',requestor varchar2(32) PATH '$.Requestor',special varchar2(10) PATH '$."Special Instructions"',adresse varchar2(400) FORMAT JSON PATH '$.ShippingInstructions.Address')) jt where json_value(json_document,'$.ponumber' returning number) < 5000; SQL> select ponumber, adresse from purchasetab_view where special='air Mail' and requestor='karen Partners'; Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 14

15 JSON aus relationalen Daten CUSTOMERS CUST_ID CUST_LAST_NAME CUST_CITY Everett Heilbronn Odenwalld Heilbronn { "custid":25451, "name": Everett", "City": Heilbronn } Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 15

16 JSON Code generieren -ganz einfach mit SQL JSON_OBJECT macht aus Key Value Paaren JSON Objekte select json_object('custid' is CUST_ID, 'name' is CUST_LAST_NAME, 'City' is CUST_CITY) ausgabe from customers JSON_OBJECTAGG ist eine Aggregatsfunktion select cust_city, json_objectagg('name' is cust_last_name) cust_per_city from customers where cust_city like 'Co%' group by cust_city Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 16

17 Ergebnis CUST_CITY CUST_PER_CITY Coburg {"name":"rice","name":"tillman","name":"barnett","name":"zimmer","name":"klesser ","name":"ingersoll","name":"bane","name":"vankirk","name":"lefevre","name":"tri mmer","name":"smeed","name":"lipp","name":"klemm","name":"obrien","name":"kelley ","name":"titus","name":"felt","name":"joseph","name":"colven","name":"snodgrass ","name":"rosenblum","name":"burgess","name":"kraft","name":"krebs","name":"zhao ","name":"carr","name":"kish","name":"dwyer","name":"eastwood","name":"haole","n ame":"lamar","name":"lengel","name":"knox","name":"mannings","name":"curr","name ":"Jeffreys","name":"Lickey","name":"Ingold","name":"Goode","name":"Kelleher","n ame":"ladd","name":"greenley","name":"rush","name":"gilmour","name":"baley"} Cochem {"name":"eden","name":"pack","name":"hale","name":"newsome","name":"calahan","na me":"lucy","name":"newkirk","name":"jewell","name":"parks","name":"rust","name": "Sampler","name":"Stone","name":"Poindexter","name":"Geralt","name":"Wiser","nam e":"valentino","name":"bakker"} Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 17

18 JSON Code generieren -ganz einfach mit SQL JSON_ARRAYAGG ist eine Aggregatsfunktion und gibt ein JSON Array aus select json_object('country_name' is c.country_name, 'ctinfo' is (select json_arrayagg( json_object('name' is cust_last_name, 'status' is cust_marital_status, 'city' is cust_city, 'street' is cust_street_address absent on null) absent on null) from customers co where co.country_id=c.country_id and cust_year_of_birth>1988) absent on null) as ct1 from countries c; Demo Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 18

20 Und jetzt zu unserem Beispiel Suche nach einer bestimmten Bestellung Name des Auftraggebers ist nicht eindeutig Lösung: (Hierarchischer) Data Guide legt Strukturen an um Views zu erzeugen linguistisch zu suchen (Textindex) virtuelle Spalten zu erzeugen oder zu löschen create search index JSON_TAB_GUIDE on JSON_TAB (JSON_DOCUMENT) for json PARAMETERS ('DATAGUIDE ON SYNC (ON COMMIT)') Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 20

22 Data Guide und relationale View Erzeugung Ausführung von DBMS_JSON.CREATE_VIEW_ON_PATH execute dbms_json.create_view_on_path (viewname => 'JSON_PO_VIEW', tablename => 'JSON_TAB', jcolname => 'JSON_DOCUMENT', path => '$') execute dbms_json.create_view_on_path (viewname => 'JSON_PO_VIEW_PART', tablename => 'JSON_TAB', jcolname => 'JSON_DOCUMENT', path => '$.ShippingInstructions.Address') Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 22

24 Data Guide und Virtuelle Spalten Nutzung von DBMS_JSON.ADD_VIRTUAL_COLUMNS Auch automatisch mit CREATE SEARCH Index execute dbms_json.add_virtual_columns (tablename => 'JSON_TAB', jcolname => 'JSON_DOCUMENT', frequency => 100) SQL> desc json_tab Name Null? Type ID NOT NULL NUMBER JSON_DOCUMENT CLOB JSON_DOCUMENT$User VARCHAR2(8) JSON_DOCUMENT$PONumber NUMBER JSON_DOCUMENT$Reference VARCHAR2(32) JSON_DOCUMENT$Requestor VARCHAR2(32) Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 24

25 Die (linguistischen) Abfragen Operatoren wie JSON_TEXTCONTAINS oder JSON_EXISTS verwenden json_textcontains(json_document, '$.CostCenter', 'A40') json_textcontains(json_document,'$','sporting near green') json_textcontains(json_document,'$.shippinginstructions','fuzzy(fransesco)') json_textcontains(json_document, '$','near((united, America), 3)') json_exists(json_document,'$.shippinginstructions.address.zipcode') json_textcontains(json_document, '$.LineItems', '$tie') json_textcontains(json_document, '$.Requestor', 'fuzzy(sarat)') Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 25

28 Performance Definition von virtuellen Spalten Indizierung über Function Based Index mit CREATE INDEX und JSON_VALUE Volltext Index (Data Guide) mit CREATE SEARCH INDEX JSON Optimierungen bei In-Memory Columnstore Verwendung Performancegewinn für Funktionen wie JSON_TABLE, JSON_VALUE oderjson_exists durch spezielles Binary Format Speicherung von virtuellen Spalten Monitoring über Ausführungspläne Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 28

29 Woausprobieren? Oracle Database Cloud Services Exadata Express Database- EC Database - BM Exadata Editions EE with Lockdown SE2, EE, HP, EP SE2, EE, HP, EP EP Implementation 1 PDB in Shared CDB 1 Dedicated CDB 1+ Dedicated DBs 1+ Dedicated DBs Management Oracle Customer Customer Customer Max DB size 5G, 20G, 50GB 50GB 11.2TB 4.2TB 9.6TB 42TB Petabytes CPU range ~ Storage Exadata,Flash:Fixed ZFS Block: $0.50 per GB Local NVMe:Fixed Exadata,Flash:Fixed Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 29

30 Woausprobieren? LiveSQL.oracle.com SQL Scratch Pad in deroracle Cloud Browser und OTN Account nötig Kostenlos Save und Share von SQL Beispielen 12.2 Tutorials Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 30

31 Weitere Informationen Deutschsprachige Blogeinträge: blogs.oracle.com/dbacommunity_deutsch JSON und die Oracle Datenbank: Basisfunktionen JSON in 12.2: JSON Generierung, neues Data Guide Konzept, Objekttypen Textblog: Artikel im Oracle Magazine: JSON in Motion Handbücher wie Database JSON Developer's Guide Dojos: Kontakt: Oracle Technology Monthly BU Core & Cloud Technologies Copyright 2017, Oracle and/or its affiliates. All rights reserved. 31

33