Data Virtualization als Motor für Big Data

Data Virtualization als Motor für Big Data Dr. Christian Kurze Principal Sales Engineer DACH Denodo Technologies info.de@denodo.com

Denodo im Überblick Plattform Standorte Spezialist Kunden Denodo integriert Daten unabhängig von ihrer Quelle und Technologie (DB & DWH bis Hadoop, intern bis Web & Cloud, strukturiert bis unstrukturiert) in einer einheitlichen virtuellen Schicht. Applikationen nutzen die standardisierten Data Services in Real-Time (Right-Time). Headquarter: Palo Alto, CA Leader in Data Virtualization. Cool Vender in Data Integration. Weltweit: USA, United Kingdom, Spanien, Deutschland 15 Jahre Fokus auf Datenvirtualisierung & Data Services. Führendes Produkt. Lösungserfahrung. >250 Kunden aus allen Branchen, davon viele F500 und G2000 Unternehmen: Healthcare & Life Sciences, Technology, Media, Telecommunications, Insurance, Financial Services, Consumer/Retail, Public Sector. Top Product Innovation 2014. Top 10 IT Companies to Watch. Globale Präsenz in Nordamerika, EMEA, APAC, Lateinamerica. Analysten Konstantes Ranking als Leader in Datenvirtualisierung: Innovation, Flexibilität, Ease-of-Use und ROI. Best-of-Breed DV.

ETL Real World of Data

Datenvirtualisierung schließt die Lücke Query, Reference, Browse, Events/Alerts, Search

mit einer Datenmanagement-Strategie JDBC/ODBC Security, WS Security, HTTPS, Encryption SSL, Pass-Through Anonymous Web Browsing Encryption,

und einer ausgereiften Plattform Multiple Protocols, Formats Design Tools Optimizer Cache Scheduler Linked Data Services Query, Search, Browse Request / Reply, Event-Driven Secure Delivery Publish Real-time (Right-time) Data Services Combine Transform, Improve Quality, Integrate Connect Normalized Views of Disparate Sources Library of Wrappers Any Data or Content, Web Automation Read and Write Monitoring Governance Metadata Security

Wie funktioniert Datenvirtualisierung? PUBLISH Real-time (Right-time) Data Services PUBLISH Expose entities as data services Data services support multiple standard protocols including JDBC, ODBC, REST and SOAP/XML WS, etc. COMBINE COMBINE Transform, Improve Quality, Integrate CONNECT Normalized Views of Disparate Data Library of functions for transformation, normalization and data cleansing Graphical drag&drop tool for combining views using relational algebra Extended relational model supports hierarchical data sources CONNECT Normalized view of sources Pre-packaged connectors

Connect

Connect

Connect

Connect

Combine

Publish

Publish SELECT * FROM retailer; SELECT * FROM retailer WHERE retailer_code = 1101; http://denodo:9090/server/globalsales/views/retailer http://denodo:9090/server/globalsales/views/retailer/1101 Message Queue: <userdata> <retailer_code>1101</retailer_code> </userdata>

Wie funktioniert Datenvirtualisierung? SELECT * FROM customer; Kerngedanke: Keine Datenreplikation, Abfrage der Quellen in Echtzeit Anfragen werden an die virtuelle Schicht gesendet, analysiert und automatisch PUBLISH optimiert (z.b. Kostenbasierte Real-time (Right-time) Data Services Abfrageoptimierung, Caching, ). PUBLISH Expose entities as data services Data services support multiple standard protocols including JDBC, ODBC, REST and SOAP/XML WS, etc. Übersetzung der Anfrage in die Sprache COMBINE der Quelle (SQL, API, WebService-Calls, ); Transform, Improve Ausführung innerhalb der Quelle. Quality, Integrate COMBINE Nutzung der gesamten Funktionalität der CONNECT Datenquelle, z.b. Aggregationen innerhalb Normalized Views of einer DWH-Appliance bei analytischen Disparate Data Abfragen. Die Quellen sind und bleiben oftmals die optimale Art, Daten zu halten. Extended relational model supports hierarchical data sources Rückgabe des Ergebnisses an die virtuelle Schicht, ggf. Weiterverarbeitung und Rückgabe an den Konsumenten. Library of functions for transformation, normalization and data cleansing Graphical drag&drop tool for combining views using relational algebra CONNECT Normalized view of sources Pre-packaged connectors

Grafische Analyse der Abfragen

Data Virtualization Einsatzfelder Agile Business Intelligence Real-time Reporting Unified Dashboards Agile Single View Applications Customer Single View - Call Center, Portals Unified Desktops - Underwriting, Loans, Virtual Logical DW / Marts Operational Decision Data Big Data / Predictive Analytics DS for Mobile Apps Self-Service & Mobile BI Big Data for Enterprise Cloud / SaaS Integration Web Data Aggregation Social Media, Feedback Multi-structured Big Data, Cloud Integration Data Services for BPM Data Virtualization Linked Data Services (IaaS) Unified Data Layer Unified Source of Reference Virtual MDM Logical Data Abstraction Migration / Modernization Linked Data Services

Data Virtualization Kunden Agile Business Intelligence Agile Single View Applications Data Virtualization Big Data, Cloud Integration Linked Data Services

Data Virtualization Kundenerfolge Agile Business Intelligence Agile Single View Applications Access new data sources 60% faster with change requests met in just a few days with IT using 40% less analyst time to support. 50% improvement in productivity with 75% reduction in-store waiting times and 87% reduction in errors due to manual data input. Big Data, Cloud Integration Cost savings of $600,000 and 15% ROI in 3 months to delivers secure and timely manner information. Data Virtualization New suite of agile reports to sales & management in 3x the speed with 1/3rd the team size. Linked Data Services

Data Virtualization: Türöffner für Big Data (= ALLE Daten) Data Lakes und Enterprise Data Hubs Alle Quellen sind nutzbar: Hadoop (Cloudera, Hortonworks, Amazon, etc.) NoSQL (MongoDB, Cassandra, Aerospike, etc.) SQLifizierung und Standardisierung des Zugriffs auf Big Data und NoSQL Abstraktion der technischen Komplexität zur einfachen Integration von Big Data mit Unternehmensdaten Analytical Data Integration Sandboxing & Prototyping

Die Challenge für BI

Data Virtualization als Lösung Benefits of Virtual Data Services 1. Unique Business Model by seamless integration of analytical results with enterprise data (30 Trillion of simulation values; 34 TB; unique statistical models) 2. Data Services are directly used by customers and internal departments to create better and unique offers to customers (Sales, Marketing, Risk, Finance, etc.) 3. Centralized Security by offering a companywide virtualization layer.

Telematics & Predictive Maintenance Tableau: Dealer / Customer Dashboard Benefits of Virtual Data Services and Analytics Dealer 1. Improved Service and less Downtime through realtime analytics of sensor data and proactive service to customers Maintenance 2. Predictive Analytics drive Resource Optimization in maintenance, distribution, service by integrating detailed sensor data to enterprise data 3. New Business Model to offer based on the real-time analysis of detailed sensor data Parts Inventory OSI PI Hadoop Cluster

Global Bank: NoSQL for Cold Data Storage Benefits of Virtual Data Services 1. Massive Reduction of Storage space and Cost since the warehouse only contains the most current entries (e.g. last year), historical data is stored in Hadoop. Also works for vertical partitioning, i.e. the most actively used attributes are stored in the DWH, lessoften used attributes are stored in Hadoop Dimension 1 (product) Fact table (sales) Dimension 2 (country) Data Warehouse Data Warehouse Without Data Virtualization 2. Performance Increase by only querying the necessary data set recent vs. historical data Fact table (sales) Fact table (sales > 2 years old) Data Warehouse Hadoop With Data Virtualization 3. Query Simplifaction by transparent union of recent and historical data within the virtual layer, not within the application which requests data

Zusammenfassung Data Virtualization Middleware für virtuelle Datenintegration Standardisiertes logisches Datenmodell für alle Applikationen und Nutzer Universeller Datenzugriff: intern/extern, Schätze aus allen Daten heben Minimierung der Replikation Wiederverwendbare Datenservices Zentrale Security und Governance Web/Cloud/Big Data, strukturiert/unstrukturiert Flexible Integrationsoptionen real-time, cached, scheduled batch Enterprise Class Powerful and Agile Abstrahierte und vereinheitlichte Datenservices, Zugriffskontrolle, SLAs Data Governance, Data Lineage, Data Management, Einfache Integration in Infrastruktur Performance und Skalierbarkeit

Vorteile aus Data Virtualization Hochqualitative Informationen Integration getrennter Datensilos Integration von Web / Cloud, Big Data, unstrukturiert Real-time (Right-time) Data Services Einhaltung von SLAs: IT <-> Fachabteilung <-> Kunde Agilität für die Fachabteilungen Flexibilität für die IT Sicherung von Wettbewerbsvorteilen Geringe Kosten & hohe Agilität Integrationskosten um 80% gesenkt Flexibilität bei Changes Schnelle Realisierung von Lösungen Projekte in 4-6 Wochen, ROI in <6 Monaten Neue Möglichkeiten für IT und Business

The Fastest Way to Data Virtualization. Download Denodo Express now from community.denodo.com

The Fastest Way to Data Virtualization. Your Way to Become a Data Ninja. Download Denodo Express now from community.denodo.com

The Fastest Way to Data Virtualization. Your Way to Become a Data Ninja. Download Denodo Express now from community.denodo.com

www.denodo.com info.de@denodo.com

Copyright Denodo Technologies All rights reserved Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies.