This document is for informational purposes. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in

Transkript

1

2 This document is for informational purposes. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described in this document remains at the sole discretion of Oracle. This document in any form, software or printed matter, contains proprietary information that is the exclusive property of Oracle. This document and information contained herein may not be disclosed, copied, reproduced or distributed to anyone outside Oracle without prior written consent of Oracle. This document is not part of your license agreement nor can it be incorporated into any contractual agreement with Oracle or its subsidiaries or affiliates.

3 FPO In-Database Analytics: Predictive Analytics, Data Mining & R Detlef E. Schröder Leitender Systemberater STCC DB Mitte R Open Source

4 Oracle Data Mining

5 Oracle - Hardware und Software Oracle Advanced Analytics Option Oracle Data Mining 12- in-db Data Mining Algorithmen In-DB Modellbildung In-DB Modelanwendung In-DB Text Mining 50+ in-db Statistische Funktionen Oracle R Enterprise R für alle Daten

6 Was ist Data Mining? Automatische Suche durch die Daten, um Strukturen zu entdecken, Zusammenhänge zu erforschen, und Vorhersagen zu machen Data Mining bietet Ergebnisse für : Vorhersage des Kundenverhaltens (Classification) Vorhersage oder Schätzen des Wertes (Regression) Segmentierung (Clustering) Faktoren entdecken, die zu einer Fragestellung gehören (Attribute Importance) Finde Profile, Zielgruppen oder Zielelemente (Decision Trees) Zusammenhänge entdecken und Warenkorbanalysen (Associations) Datenausreißer (Anomaly Detection)

7 Oracle Data Mining Algorithmen Probleme Algorithmen Anwendung Classification Regression Anomaly Detection Attribute Importance Association Rules Clustering Feature Extraction A1 A2 A3 A4 A5 A6 A7 F1 F2 F3 F4 Logistic Regression (GLM) Decision Trees Naïve Bayes Support Vector Machine Multiple Regression (GLM) Support Vector Machine One Class SVM Minimum Description Length (MDL) Apriori Hierarchical K-Means Hierarchical O-Cluster Nonnegative Matrix Factorization Classical statistical technique Popular / Rules / transparency Embedded app Wide / narrow data / text Classical statistical technique Wide / narrow data / text Lack examples of target field Attribute reduction Identify useful data Reduce data noise Market basket analysis Link analysis Product grouping Text mining Gene and protein analysis Text analysis Feature reduction

8 SQL Developer 3.0/Oracle Data Miner 11g Release 2 GUI GUI für Daten Analysten SQL Developer Extension (OTN download) Daten untersuchen Neue Zusammenhänge entdecken Aufbauen und Anwenden von Modellen Vorhersagen modellieren Aufbau und Verteilen von Workflows und SQL Code

9 Oracle Data Miner Nodes (Partial List) Tabellen und Views Transformationen Data Analyse Modellbildung Text

10 Oracle Data Miner 11g Release 2 GUI Churn Demo Simple Conceptual Workflow Churn models to product and profile likely churners

11 In-Database Data Mining Traditional Analytics Data Import Data Mining Model Scoring Data Preparation and Transformation Data Mining Model Building Data Prep & Transformation Oracle Data Mining Savings Results Faster time for Data to Insights Lower TCO Eliminates Data Movement Data Duplication Maintains Security Model Scoring Data remains in the Database Embedded data preparation Data Extraction Hours, Days or Weeks Source Dataset Analytic Process Target Data s/ Work al Output Area Process ing Model Scoring Embedded Data Prep Model Building Data Preparation Secs, Mins or Hours Cutting edge machine learning algorithms inside the SQL kernel of Database SQL Most powerful language for data preparation and transformation Data remains in the Database

12 InDatabase - Mining

13 11g Statistische & Analytische Fkt. (Free) Statistics Ranking functions rank, dense_rank, cume_dist, percent_rank, ntile Window Aggregate functions (moving and cumulative) Avg, sum, min, max, count, variance, stddev, first_value, last_value LAG/LEAD functions Direct inter-row reference using offsets Reporting Aggregate functions Sum, avg, min, max, variance, stddev, count, ratio_to_report Statistical Aggregates Correlation, linear regression family, covariance Linear regression Fitting of an ordinary-least-squares regression line to a set of number pairs. Frequently combined with the COVAR_POP, COVAR_SAMP, and CORR functions Descriptive Statistics DBMS_STAT_FUNCS: summarizes numerical columns of a table and returns count, min, max, range, mean, median, stats_mode, variance, standard deviation, quantile values, +/- n sigma values, top/bottom 5 values Correlations Pearson s correlation coefficients, Spearman's and Kendall's (both nonparametric). Cross Tabs Enhanced with % statistics: chi squared, phi coefficient, Cramer's V, contingency coefficient, Cohen's kappa Hypothesis Testing Student t-test, F-test, Binomial test, Wilcoxon Signed Ranks test, Chi-square, Mann Whitney test, Kolmogorov-Smirnov test, One-way ANOVA Distribution Fitting Kolmogorov-Smirnov Test, Anderson-Darling Test, Chi- Squared Test, Normal, Uniform, Weibull, Exponential Note: Statistics and SQL Analytics are included in Oracle Database Standard Edition

14 Oracle Data Mining und Unstrukturierte Daten Oracle Data Mining untersucht auch unstrukturierte Daten, wie Texte Inklusive Freitext und Kommentare in ODM Modellen Cluster and Klassifizierung von Dokumenten Oracle Text für die Vorverarbeitung

15 Real-time Klassifizierung für Kundendaten On-the-fly, auf einzelne Datensätze angewendet (z.b. vom Call Center) Select prediction_probability(clas_dt_5_2, 'Yes' USING 7800 as bank_funds, 125 as checking_amount, 20 as credit_balance, 55 as age, 'Married' as marital_status, 250 as MONEY_MONTLY_OVERDRAWN, 1 as house_ownership) from dual; ECM Call Center Social Media Branch BI Web Get Advice Mobile CRM

16 Exadata + Data Mining 11g Release 2 DM Scoring weitergeleitet auf den Storage! schneller In 11g Release 2, SQL Vorhersagen und Oracle Data Mining Modelle werden In die Storage Zellen verlagert z.b: Wechselwillige Kunden in den USA: select cust_id from customers where region = US and prediction_probability(churnmod, Y using *) > 0.8;

17 Oracle Communications Industry Data Model Beispiel Bessere Informationen für OBIEE Dashboards ODM Vorhersagen & Wahrscheinlichkeiten sind verfügbar aus der Datenbank heraus

18 Weitere Beispiele für den Einsatz von ODM Polizei Verbrechensvorhersage Geldwäsche Konzept unter Verwendung von ODM zur Ermittlung Oracle CRM Unterstützung des Kampagnenmanagements Oracle Telekomunikation Datenmodell Chrun und CLTV Prozessananlyse - in Zusammenarbeit mit Robotron...

19 Weitere Beispiele bei OBE

20 Zusammenfassung

21 Advanced Analytics direkt in der DB Vorteile Datentransformation ohne Materialisierung Definition von Views Pipelined Table Functions Unterstützung auch "ausgefallener" Datentypen Ausnutzung des Optimizers Skalierbarkeit auch bei großen Datenmengen Durchgängige Sicherheitskonzepte Virtual Private Database / Row Level Security Schutz vor dem DBA durch Database Vault Greift auch für die Anwendung der Ergebnisse Kontrolle über Ressourcenverbrauch Resource Manager / Enterprise Manager Grid Control

22 Fragen & Antworten

23

24