Fakultät für Informatik & Wirtschaftsinformatik DB & IS II SS NoSQL. Dr. Christian Senger.

Transkript

1 NoSQL NoSQL 1

2 Short History of Databases 1960s - Navigational DBs CODEASYL (COBOL) IMS (IBM) 1980s to 1990s - Object Oriented DBs Object DB's Object-Relational- DBs Object Relational Mapping 1970s - Relational DBs E.F. Codd (IBM) INGRES -> PostgreSQL Oracle 2000s - NoSQL & NewSQL XML Databases NoSQL (e.g. MongoDB) Graph Database (e.g. Neo4j) NewSQL (good scalable RDBMS) NoSQL 2

3 Relational Databases: ACID NoSQL 3

4 Reliability (fault tolerance) Fakultät für Informatik & Wirtschaftsinformatik Distributed Computing Scalability Sharing of Resources Flexibility Performance & Speed Open system NoSQL 4

5 Distributed Computing - Disadvantages Troubleshooting Software Networking Security NoSQL 5

6 Vertical scaling (scale up) Fakultät für Informatik & Wirtschaftsinformatik Scaling Z.B. in einem System (z.b. Server) mehr CPUs, RAM, Festplatten hinzufügen Horizontal scaling (scale out) Mehr Knoten (d.h. Server) zu einem System hinzufügen. NoSQL nutzt und unterstützt in der Regel das outscaling (z.t. nahezu vollautomatisch). NoSQL 6

7 NoSQL Nonrelational database management system Typically not using SQL, but proprietary interterfaces Designed for Distributed Data stores Designed for very big amounts of data (e.g. Google, Facebook) Flexible Schema Avoid JOINS Typically scales horizontally NoSQL 7

8 NoSQL Designed for very large amounts of Data NoSQL 8

9 RDBMS Structured and organized data Structured query language (SQL) Dataand its relationships are stored in separate tables. Data Manipulation Language, Data Definition Language TightConsistency BASE Transaction NoSQL (Stands for Not Only SQL) No declarative query language No predefined schema KeyValue pair storage, Column Store, Document Store, Graph databases Eventual consistency rather ACID property Unstructured and unpredictable data CAP Theorem Prioritizes high performance, high availability and scalability XML und Datenbanken 9 von 24

10 NoSQL Advantages? & Disadvantages? High scalability Distributed Computing Lower cost Schema flexibility, semistructure data No complicated Relationships No standardization Limited query capabilities Eventual consistent is not intuitive to program for NoSQL 10

11 Kritische Anmerkung NoSQL Wenn ich nicht Auto fahren kann, hilft mir auch ein Auto mit Elektromotor nichts! Wenn ich kein SQL kann und nichts von Datenstrukturen verstehe, dann hilft mir auch NoSQL nichts. Ich brauche keinen Lastwagen um einen Koffer zu transportieren Für ein System mit normalem Datenaufkommen, brauche ich kein NoSQL Ich transportiere Schmuck nicht auf der offenen Ladefläche, sondern im Panzerwagen Ist das NoSQL-System meiner Wahl so sicher wie ein Standard-RDBMS? Wenn ich einen Fliwatüüt fahre/fliege, muss ich es auch einparken können Auch NoSQL muss man lernen (und zwar jedes System neu, nicht so einfach wie bei RDBMS) Komplexe Schwerarbeit im Gelände mache ich mit einem Allroundbagger, nicht mit einem Rennwagen. Wenn ich komplexe Datenstrukturen und komplexe Abfragen habe, hilft mir reine Geschwindigkeit erstmal nichts NoSQL 11

12 Verteilte Systeme CAP Theorem Konsistenz (C): Alle Knoten sehen zur selben Zeit dieselben Daten. Diese Konsistenz sollte nicht verwechselt werden mit der Konsistenz aus der ACID-Transaktionen, die die innere Konsistenz eines Datenbestandes betrifft. Verfügbarkeit (A): Alle Anfragen an das System werden stets beantwortet. Partitionstoleranz (P): Das System arbeitet auch bei Verlust von Nachrichten, einzelner Netzknoten oder Partition des Netzes weiter. Nur zwei dieser drei Anforderungen können in verteilten Systemen gleichzeitig vollständig erfüllt sein. Dabei können diese graduell erfüllt sein, z.b. höhere Verfügbarkeit/Geschwindigkeit, niedrigere Konsistzenz bei gleichbleibender Partitionstoleranz NoSQL 12

13 CAP Theorem CA: Single site cluster, therefore all nodes are always in contact. When a partition occurs, the system blocks. CP: Some data may not be accessible, but the rest is still consistent/accurate. AP: System is still available under partitioning, but some of the data returned may be inaccurate. NoSQL 13

14 CAP & BASE The CAP theorem states that a distributed computer system cannot guarantee all of the following three properties at the same time: Consistency Availability Partition tolerance A BASE system gives up on consistency. Basically Available indicates that the system does guarantee availability, in terms of the CAP theorem. Soft state indicates that the state of the system may change over time, even without input. This is because of the eventual consistency model. Eventual consistency indicates that the system will become consistent over time, given that the system doesn't receive input during that time. NoSQL 14

15 ACID vs. BASE ACID Atomicity Consistency Isolation BASE Basically Available Soft state Eventual consistency Durability NoSQL 15

16 Basic NoSQL DB Types Document oriented Keyvalue stores Column oriented Graph NoSQL 16

17 A collection of documents Fakultät für Informatik & Wirtschaftsinformatik Document Oriented Data in this model is stored inside documents. A document is a key value collection where the key allows access to its value. Documents are not typically forced to have a schema and therefore are flexible and easy to change. Documents are stored into collections in order to group different kinds of data. Documents can contain many different keyvalue pairs, or keyarray pairs, or even nested documents. NoSQL 17

18 Relational Model vs. Document Model Relational model Tables Rows Columns Joins Document model Collections Documents Key/value pairs N/A Examples: MongoDB, CouchDB NoSQL 18

19 Document Oriented NoSQL 19

20 Key-Value Stores Most basic types of NoSQL databases. Designed to handle huge amounts of data. Allow developers to store schemaless data. Data is stored as hash table - each key is unique, the value can be string, JSON, BLOB (basic large object) etc. A key may be strings, hashes, lists, sets, sorted sets For example a keyvalue pair might consist of a key like "Name" that is associated with a value like "Robin". Can be used as collections, dictionaries, associative arrays etc. Work well for shopping cart contents, or individual values like color schemes, a landing page URI, or a default account number. Follow the 'Availability' and 'Partition' aspects of CAP theorem. NoSQL 20

21 Key-Value Stores NoSQL 21

22 Key-Value Store NoSQL 22

23 Key-Value Stores Examples: Redis Dynamo Riak NoSQL 23

24 Column-oriented DB Primarily work on columns and every column is treated individually. Values of a single column are stored in close neighbourhood. Stores data in column specific files. All data within each column datafile have the same type which makes it ideal for compression. Can improve the performance of queries as it can access specific column data. High performance on aggregation queries (e.g. COUNT, SUM, AVG, MIN, MAX). Works on data warehouses and business intelligence, customer relationship management (CRM), Library card catalogs etc. Examples: BigTable, Cassandra, SimpleDB NoSQL 24

25 RDBMS vs. Column-oriented DB (Cassandra) NoSQL 25

26 Column-oriented DB T/SCF: Table/ SuperColumnFamily CF: ColumnFamily SC: SuperColumn NoSQL 26

27 Graph DBs Stores data in a graph. (Utilization of Graph Theory/Algorithms!) It is capable of elegantly representing any kind of data in a highly accessible way. A graph database is a collection of nodes and edges Each node represents an entity (such as a student or business) and each edge represents a connection or relationship between two nodes. Every node and edge is defined by a unique identifier. Each node knows its adjacent nodes. As the number of nodes increases, the cost of a local step (or hop) remains the same. Index for lookups. NoSQL 27

28 Graph DBs Related to Triple Stores (RDF/OWL, semantic technologies) Subject, Predicate, Object Schemas/Labels are missing - but under development Relationship have names (meaning) and can have attributes Graphs can be easily modelled with RDBMS, but Graph DBs are optimized for Graph Problems NoSQL 28

29 RDBMS vs. Graph DBs Relational model Tables Rows Columns Joins Graph model Vertices and Edges set Vertices Key/value pairs Edges NoSQL 29

30 Graph DBs NoSQL 30