Data Structures and Algorithm Design

Transkript

1 - University of Applied Sciences - Data Structures and Algorithm Design - CSCI Friedhelm Seutter Institut für Angewandte Informatik Contents 1. Analyzing Algorithms and Problems 2. Data Abstraction 3. Recursion and Induction 4. Sorting 6. Dynamic Sets and Searching 7. Graphs and Graph Traversals 8. Optimization and Greedy Algorithms 10. Dynamic Programming 13. NP-Complete Problems Institut für Angewandte Informatik 2 1

2 6. Dynamic Sets and Searching Dynamic Sets Array Doubling and Amortized Time Analysis Red-Black Trees and Binary Search Hashing Disjoint Sets Institut für Angewandte Informatik 3 Dynamic Sets A dynamic set is a set whose membership varies during computation. The sets initially have a certain number of members, including 0, and then elements are inserted or deleted as the computation progresses. The maximum size to which the set might grow is not known in advance. Institut für Angewandte Informatik 4 2

3 Dynamic and Static Data Structures Static: Size is set at compile time and not changeable at run time, e. g. array. Dynamic: Size is changeable at run time, e. g. linked list, tree. Institut für Angewandte Informatik 5 Dynamic and Static Data Structures To avoid the overhead for managing dynamic data structures an array with space largest as possible is allocated, usually not very satisfactory, although very common. More flexible is to allocate a small array initially and doubling its size whenever it becomes apparent that it is too small (and halving whenever becoming too big). Institut für Angewandte Informatik 6 3

4 Array Doubling The initial array size has to be specified. A counter for the number of entries currently being in the array has to be provided and maintained. The size of the array has to be doubled whenever the counter reaches the actual array size. The previous elements have to be transferred into the new array after doubling. Institut für Angewandte Informatik 7 Complexity of Array Doubling All operations on the array but the transfer operation need constant time, i.e. Θ(1). The transfer operation for inserting the n elements of the previous array into the new array needs linear time, i.e. Θ(n). The worst-case insertion complexity is Θ(n). Institut für Angewandte Informatik 8 4

5 Amortized Time Analysis The complexity for an individual operation of the same type may vary widely, e.g. Θ(1) to Θ(n). But the total time for a sequence of operations may be much less than the worst-case time for one operation multiplied by the length of the sequence. The large costs of one operation is spread out over many less expensive operations, thus in average the cost of an operation is kept small. Institut für Angewandte Informatik 9 Amortized Time Analysis The costs of each individual operation is computed by: amortized cost = actual cost + accounting cost Goals The sum of accounting costs over any legal sequence of operations is nonnegative. Although the actual cost may vary widely from one operation to the next, the amortized cost is fairly regular. Institut für Angewandte Informatik 10 5

6 Amortized Time Analysis The total amortized cost of a sequence of operations starting from the creation of the object is an upper bound on the total actual cost and therefore amenable to complexity analysis. Institut für Angewandte Informatik 11 Analogy Savings Account When times are good, an extra deposit is made for bad times. When the bad time arrives, with an unusually expensive operation, a withdrawal is made. To remain solvent, the account balance cannot go negative. Institut für Angewandte Informatik 12 6

7 Accounting Scheme Main Ideas Normal individual operations should have positive accounting costs. The unusually expensive individual operations should have negative accounting costs. The sum of positive accounting costs should offset the unusual expense and prevent the amortized cost from going negative. Institut für Angewandte Informatik 13 Example A stack with its operations push and pop is implemented with an array and array doubling, whenever the stack overflows. Actual cost of push and pop without resizing: 1 Actual cost of push with resizing from n to 2n: 1+nt, for some constant t Institut für Angewandte Informatik 14 7

8 Accounting Scheme Example Accounting cost of push without resizing: 2t Accounting cost of push with resizing from n to 2n: - nt + 2t Accounting cost of pop: 0 (Factor 2 guarantees the accounting costs not to become negative.) Institut für Angewandte Informatik 15 Accounting Scheme Example push operation Stack size Sum of accounting costs 1 N 2t N N 2Nt N+1 2N Nt+2t 2N 2N 3Nt 2N+1 4N Nt+2t 4N 4N 5Nt 4N+1 8N Nt+2t Institut für Angewandte Informatik 16 8

9 Amortized Cost Example actual accounting amortized push operation cost cost cost 1 1 2t 1+2t N N 2Nt N(1+2t) N+1 Nt+1+N Nt+2t (N+1)(1+2t) 2N Nt+2N 3Nt 2N(1+2t) 2N+1 3Nt+2N+1 Nt+2t (2N+1)(1+2t) 4N 3Nt+4N 5Nt 4N(1+2t) 4N+1 7Nt+4N+1 Nt+2t (4N+1)(1+2t) Institut für Angewandte Informatik 17 Amortized Cost Example The amortized cost of each push operation is 1+2t, independent of array doubling or not. The amortized cost of each pop operation is 1. Thus in the worst case both operations have an amortized cost (complexity) of Θ(1). Institut für Angewandte Informatik 18 9

10 Red-Black Trees Red-black trees are binary trees which satisfy the structural requirement that its height is within a factor of two of the height of the most balanced binary tree with n nodes, i. e. its height cannot exceed 2 lg(n+1). Red-black trees are thus almost balanced. Institut für Angewandte Informatik 19 Binary Search Tree Institut für Angewandte Informatik 20 10

11 Binary Search Tree Institut für Angewandte Informatik 21 Binary Search Tree Institut für Angewandte Informatik 22 11

12 Worst-case Complexity Depending on the height h: W(h) = Θ(h) Depending on the number of nodes n: In any tree: W(n) = Θ(n) In a balanced tree: W(n) = Θ( lg n) Institut für Angewandte Informatik 23 Red-Black Trees and External Nodes In red-black trees empty subtrees are treated as a special kind of node, called external node. An external node doesn t have any children and doesn t contain any data, including a key. All other nodes are called internal nodes. They contain data, including the key, and must have two children. Institut für Angewandte Informatik 24 12

13 Red-Black Trees Institut für Angewandte Informatik 25 Red-Black Tree Institut für Angewandte Informatik 26 13

14 Size and Depth of Red-Black Trees Institut für Angewandte Informatik 27 Height of Red-Black Trees Institut für Angewandte Informatik 28 14

15 Insertion into Red-Black Trees Insertion of the node at a position of an external node considering the binary search tree property. In case of a color constraint violation two nodes must flip colors, and, if necessary, a subtree must be rebalanced. Institut für Angewandte Informatik 29 Insertion Institut für Angewandte Informatik 30 15

16 Insertion Institut für Angewandte Informatik 31 Insertion Institut für Angewandte Informatik 32 16

17 Deletion from Red-Black Trees Locate the node to be logically deleted, call it u. If the right child of u is an external node, identify u as the node to be structurally deleted. If the right child of u is an internal node, find the tree successor of u, copy the key and information from that node to u. Identify the tree successor as the node to be structurally deleted. Carry out the structurally deletion and repair any imbalance of black height. Institut für Angewandte Informatik 33 Deletion Institut für Angewandte Informatik 34 17

18 Deletion Institut für Angewandte Informatik 35 Complexity Given a red-black search tree with n nodes. Worst-case insertion time is in Θ(log n). Worst-case deletion time is in Θ(log n). Institut für Angewandte Informatik 36 18

19 Hashing Hashing is a technique used to map keys of data from a large key space into a smaller index space of an array for storing the keys and the data or a reference to the data in the array. Hashing is used to implement a Dictionary ADT, for example. Institut für Angewandte Informatik 37 Direct Addressing Institut für Angewandte Informatik 38 19

20 Direct Addressing Key k and its data is stored in the array T at position T[k]. Key space must be equal to index space (in number and in values). Complexity of dictionary operations: Θ(1) Institut für Angewandte Informatik 39 Hashing The key space is extremely large but only a few are actually used. The size of the index space has to be adjusted to the actual number of keys. The keys are not restricted to numerical values. Institut für Angewandte Informatik 40 20

21 Hashing Keys and data or its references are stored in an array, called hash table H[0.. h-1]. A hash function maps a key k to an index c(k) of the hash table. The index also is called hash code of the key, 0 c(k) h-1. Several keys may be mapped to the same hash code. That event is called a collision. Institut für Angewandte Informatik 41 Closed Address Hashing Closed address hashing or chained hashing is a technique where collisions are resolved by storing all elements having the same hash code in linked lists. Initially these linked lists are empty. The load factor of the hash table is the average number of elements over all linked lists, α = n/h. (h number of entries in H, n number of currently stored elements.) Institut für Angewandte Informatik 42 21

22 Chained Hashing Institut für Angewandte Informatik 43 Complexity of Search Sum of costs for computing the hash code, accessing the hash table and searching in the linked list. Best and worst case: B(n) = Θ(1) W(n) = Θ(n) Institut für Angewandte Informatik 44 22

23 Complexity of Search Average case: Assume the hash code for each key is equally likely. A(n) = Θ(1+α) If n Θ(h), which can be accomplished by array doubling, then: A(n) = Θ(1) Institut für Angewandte Informatik 45 Open Address Hashing Open address hashing is a technique where collisions are resolved within the hash table by looking for an empty location, called rehashing. Rehashing by linear probing: rehash(j) = (j+1) mod h The load factor always is less than or equal to 1. Institut für Angewandte Informatik 46 23

24 Hash Functions A good hash function spreads the hash codes fairly evenly over the entire range of the hash table entries. Number of collisions should be minimized. Computing the hash code should be in Θ(1). Institut für Angewandte Informatik 47 Hash Functions Example: hash(k) = (a k) mod h (for some constant a) If k is not an integer, e.g. of string type, a function is needed to convert k to an integer. Institut für Angewandte Informatik 48 24

25 Disjoint Sets A disjoint set is a structure for a set (a collection) of sets, S = {S 1, S 2,..., S n }. The elements S i and S j are pairwise disjoint, 1 i < j n. Institut für Angewandte Informatik 49 Operations of Disjoint Sets makeset(x): Creates a set containing the element x. findset(x): union(x,y): Finds the representative of the set which x is a member of. Finds the representatives of the sets which x and y are members of, unites the sets, and destructs the original sets of x and y. Institut für Angewandte Informatik 50 25

26 Implementation (for example) A set of a collection of sets can be implemented as a linked list. The first element is the representative of the set. New elements are inserted at the end of the list (there may be a pointer to the end of the list). Each element has a pointer to the set s representative (first element of the list). Institut für Angewandte Informatik 51 Union-Find Program Create a collection of n disjoint sets, each containing a single element. Any sequence of union or find operations following then is called a Union-Find program. The length of such a program is the number of union and find operations. The makeset operations of the creation process are not counted. Institut für Angewandte Informatik 52 26

27 Union-Find Program (example) Let {{1}, {2}, {3},, {n}} be a collection of n disjoint sets. 1. union(1, 2) 2. union(2, 3)... n-1. union(n-1, n) n. findset(1)... m. findset(1) Institut für Angewandte Informatik 53 Worst-case Complexity (list implementation) The complexity of find is Θ(1). The complexity of union is Θ(k), if k is cardinality of the smaller set of the union. (These elements have to be relinked to the new representative.) A Union-Find program of n sets and length m then has a worst-case complexity of Θ(mn). The amortized worst-case complexity per operation then is Θ(n). Institut für Angewandte Informatik 54 27

28 Implementation (another example) A set of a collection of sets can be implemented as a tree. The root is the representative of the set. Each element has a pointer to its parent, for finding its representative. For a union the tree of smaller cardinality becomes a subtree of the other root. Paths from an element to the root are kept as short as possible. Institut für Angewandte Informatik 55 Worst-case Complexity (tree implementation) A Union-Find program on a set of n elements and length m has a worst-case complexity of Θ((n+m) lg*(n)). Institut für Angewandte Informatik 56 28