Empirische Bewertung von Performance-Analyseverfahren für Software-Architekturen

Transkript

1 FAKULTÄT II - DEPARTMENT FÜR INFORMATIK Abteilung Software Engineering Diplomarbeit Empirische Bewertung von Performance-Analyseverfahren für Software-Architekturen 14. Januar 2005 Bearbeitet von: Heiko Koziolek Schützenweg Oldenburg heiko.koziolek@informatik.uni-oldenburg.de Betreut von: Erstgutachter: Jun.-Prof. Dr. Ralf Reussner Zweitgutachter: Prof. Dr. Wilhelm Hasselbring Dipl.-Wirtsch.-Inform. Steffen Becker Dipl.-Math. Viktoria Firus

2 ii

3 Inhaltsverzeichnis Contents iv 1. Introduction Goals of the Work Research Method Methodology Research Method Questions and Metrics Validity of the Predictions Effort/Costs for the Performance Prediction Methodical in relation to Intuitive Manner Evaluation of Individual Characteristics Summary Concept and Realisation of the Case Study Boundary Conditions Experiment Webserver-Architecture Task Design-alternatives Implementation Performance Analysis of the Implementation Results Validity of the prediction Predictions with SPE Prediction with Capacity Planning Prediction with umlpsi-approach Measurement Results of the Implementation Comparison of Predictions and Measurements Methodical opposite the Intuitive Manner Conclusions and Perspectives Summary iii

4 Inhaltsverzeichnis Literaturverzeichnis 33 A. Vergleichstabelle Performance-Analyseverfahren i B. Folien der Übungen vii C. Übungszettel xli D. Musterlösung Übungen xlv E. Tasks of the experiment li E.1. Task li E.2. Component-diagram lii E.3. Sequence-diagram lii E.4. Further settings lii E.5. Software Resources lii E.6. Processing Overhead liii E.7. Resourcedistribution (just for illustration) liii E.8. Design-alternatives liii F. Lösung des Experiments lxi G. Rohdaten der Ergebnisse lxix H. Diagramme zur Veranschaulichung lxxxiii I. Messungen am Prototypen lxxxix iv

5 1. Introduction 1.1. Goals of the Work A goal of this work is evaluating the applicability of different performance prediction methods from the view of the developer by an empirical study. To attain this goal several research questions are posed: What is the accuracy of the approaches? How much is the effort to build a performance model? What are the benefits of the application of an approach in relation to an intuitive estimation without methodology? How are selected characteristics of the approaches to be assessed (e.g. modeling power, tool support, practice suitability)? For the answer of the questions appropriate metrics are defined Research Method Due to the small number of participants, who can be involved in the context of a thesis (diploma), the research here is performed in form of a comparative case study. The case study takes place in the context of the research group Palladio. As test object a prototypical Web server is examined, which was developed in the Palladio group for test purposes. On the basis of the available design documents input parameters for the approaches can be determined. A group of students uses selected performance prediction techniques in order to evaluate different design alternatives for the Web server. The forecasts of the approaches are afterward compared with measurements on the prototypical implementation of the Web server. 1

6 2

7 2. Methodology 2.1. Research Method Research must proceed purposefully. In the following the Goal, Question, Metric method (GQM) [BCR94] after Basili and Rombach is used. The research questions and criteria for their answer are derived in a top-down approach. First the comprehensive goal of the study is defined and afterward the questions, necessary for attaining this goal, are formulated. Subsequently, metrics are defined, which appear meaningful for the answer of these questions. At the end of the study the measured data can be used on the basis of the metrics for answering the questions and the answered questions can fulfill the goal of the study. Goal Goal Question Question Question Question Question Metric Metric Metric Metric Metric Metric Abbildung 2.1.: Goal, Question, Metric Method [BCR94] A goal is defined within three dimensions (issue, object, point of view) and serves a certain purpose. The goal of this study was formulated already in the introduction and is: Empirical evaluation (=purpose) of the practicability/applicability (=issue) of performance prediction methods for software architectures (=object) from the view of the developer (=point of view) Questions and Metrics Validity of the Predictions 1. Is the accuracy of the prediction with the approaches strict enough? For the answer of the question several metrics must be examined. First the comparison between the results of the forecasts and the measurements of an implementation is the most apparent criterion for the accuracy of the approaches (metric 3). The deviation between forecast and measurement can be indicated as percentage. However the results of the performance analysis depend considerably on the inputs of the approaches. Metric 2 specifies the difference between the input data of the approaches and their 3

8 2. Methodology output data. Due to the heterogeneity of input and output data their difference can only with difficulty be expressed in numbers and is therefore indicated on a rough scale (highly, means, low). Input data for the approaches can be obtained in different way. In an experiment with several participants can be examined, how strongly the input data of different participants differ from each other, if these are determined by them due to same basic informations (metric 1). The authors of the approaches for the early performance analysis often stress that it is not the objective of the approaches to predict exact absolute performance values. Rather the evaluation of different design alternatives is the center of attention. Therefore several design alternatives of the test object must be examined in the study and the participants must be asked for favoring one of the alternatives. The performance of the alternatives must then also be measured on an implementation. Afterward, it can be proportionally indicated how many participants have favored the best design alternative concerning the performance (metric 4). If the forecasts of the participants are so inaccurate that in each case different alternatives are selected then the approaches would be useless for normal developers and performance experts would have to be consulted. The metrics for the answer to first question are summarised in table 2.1. This table is used also later for the evaluation. approach 1 approach 2... approach n Difference in Difference in Difference in percentage of input and input and input data and correct design output data output data measurements decisions of different participant (metric 1) (metric 2) (metric 3) (metric 4) Tabelle 2.1.: Validity of prediction Effort/Costs for the Performance Prediction An argument of practitioners to neglect performance analysis during the design process is still the high investment of time, which the application of these techniques require. The second question is therefore to examine this circumstances more near and is: 2. What is the effort for the performance prediction? Performance prediction methods are deployed in practice only if the expenditure of the costs is justified. A comparison of the costs for the performance analysis with the saved time by additional changes of design is not possible in the context of this case study. Nevertheless at least the learning expenditure for the approaches (metric 5) and the time for the transformation of a design specification into a performance model (metric 6) can be measured. 4

9 2.2. Questions and Metrics Methodical in relation to Intuitive Manner 3. What are the advantages of applying of performance prediction techniques in relation to an intuitive manner? In the study an untrained control group gets the same tasks as the participants, who apply the performance prediction approaches. The recommendations for the best design alternatives by the control group are compared with the recommended alternatives, which were obtained with application of the approaches (metric 7) Evaluation of Individual Characteristics The quality and the degree of mature of the approaches is examined with a further question: 4. How are selected characteristics of the performance analysis methods to be assessed? This general question can be answered by means of a set of concrete criteria. The following criterion pattern was developed (metric 8, partly following [?]): Notation, used software and performance models Power (Which constructs can be specified, which not?) Structuring (Are the representations understandable? Is there a modularity of the performance model?) Specification of performance values (In which form estimations or measurements get into the performance model?) Tool support User interface, ergonomics (How user-friendly are the tools?) Robustness related to incorrect inputs (How robust are the tools in relation to usage errors?) Presentation of the results (Is the comprehensibleness for the developer ensured?) practice suitability (Are the software tools sophisticated for application in the software industry?) Suitability for different domains Evaluation method (which methods are used, which are the assets and drawbacks?) Process integration (Is there a process model, what is the degree of integration into the software development process?) Summary The below located table?? follows [BCR94] and summarises the questions and metrics of the GQM-approach, which is employed in this work. 5

10 2. Methodology Goal Purpose Empirical Evaluation Issue of appliance Object of performance prediction approaches Point of from the view of the developer. view Scale Question F1 What is the accuracy of the prediction with the approaches? Metric M1 Deviation of input data of different participants seconds M2 Deviation of input and output data low, mean, high M3 Deviation of output data and measurements percentage M4 Percentage of correct predicted design alternatives percentage Question F2 What is the effort for the performance prediction? Metric M5 Learning expenditure hours M6 Duration of performance modeling hours Question F3 What advantages have performance evaluation techniques over the intuitive way? Metric M7 Deviation of percentage of correct predicted design decisions Question F4 How are selected characteristics of the performance analysis methods to be assessed? Metric M8 Criterion schema Tabelle 2.2.: Summary GQM percentage 6

11 3. Concept and Realisation of the Case Study The core of this case study is the experimental study of an prototypical Web server with three selected performance analysis approaches (SPE, umlpsi, Capacity Planning) Boundary Conditions The participants of the case study were 24 students of the lecture Component based software development at the University of Oldenburg in the summer term The participants were distributed into 3 groups of 8 persons. The participants had two hours to examine one of the software-architectures of a web server with varying input parameters for the approaches. Additionally they evaluated with the help of the performance prediction approaches five different design alternatives for the Web server in respect of their performance behavior. Details about the experiment can be found in section 3.2. To have a reference group, seven further participants took part in the experiment without knowing the performance-analysis approaches. Their recommendation are given in section 4.2. All design-alternatives have been implemented for the web server. Using this implementations it was possible to evaluate the predictions of the performance-approaches. The results can be found in section Experiment Webserver-Architecture The experiment focused on a experimental web server, which was developed in the palladiogroup. The first version of the implementation was done in a completely object-oriented way, based on the.net-framework under Microsoft Visual Studio. Within a student work the server had been transformed to a component based web server. This includes the introduction of removable components and interfaces to connect to external applications and databases. So the web server could be used as a component based example system. chapter5.tex The server accepts client-request by the HTTP-protocol and delivers the requested web pages (e. g. HTML- or image-files). It is constructed multi-threaded, so that for each request a own program-thread is created to be able to handle multiple parallel requests. HTML-files can be created dynamically as well, e. g. to include user-dependent information. All page-requests are saved into a log file for being able to analyse the user-profile and to create usage-profiles. ======= The server accepts client-request by the HTTP-protocol and deliveres the requested pages (e. g. HTML- or image-files). It is constructed multi-threaded, so that for each request a own programm-thread is created to be able to handle multiple parallel requests. HTML-files can be created dynamically as well, e. g. to include user-dependant information. 7

12 3. Concept and Realisation of the Case Study All page-requests are saved into a logfile for being able to analyse the user-profile and to create usage-profiles HTTP Dispatcher IAcceptRequest Dispatcher Thread IParseRequest Request Analyzer IServeRequest Request Processor IDeliverResponse Static File Provider CGI - Interface IGetResponseStream Exe - Engin HTTP IMonitorWebserver ExtAppInterface ExtApp WebService DB Access DB Abbildung 3.1.: Component diagram of the Web Server The web server consists of about 4000 lines of code in the programming language C# [Lib03]. The web server s architecture was specially developed to be extendable. So additional protocols and components for generating data can integrated easily into the web server. The web server components are shown below: Dispatcher: accepts client-requests on an opened socket DispatcherThread: starts a new thread for each request. RequestAnalyser: verifies the correctness of requests, determines the protocol (HTTP, FTP, ICMP) and parses the request, determines the requested page. RequestProcessor: cares that pages are fetched from the server, sends the answer to the client, contains several classes for error-handling and monitoring the server FileProvider: fetches a file from the local file system CGIInterface: offers an interface for external applications that communicate by CGI chapter5.tex ExeEngine: starts for requests of dynamic sides external applications at the server, that are given as executable files. ======= ExeEngine: starts for requests of dynamic pages external applications at the server, that are given as executeable files Internal-API-Application: offers a non-standard interface for programs that can be started at the server Webservice: offers communication by web services using the SOAP-protocol. DBAccess: offers an interface to access databases 8

13 3.2. Experiment Requests from the RequestProcessor are passed to further components by the IDeliverResponseinterface. This components are organized according to the Chain of Responsibility designpattern [GHJV95]. The results of the performance-analysis of the implementation can be found in chapter Task For the experiment a task had to be found, that could give answers to the research questions from chapter 2. As a typical use-case for performance-analysis-approaches the evaluation of designdecisions should be focused. The absolute performance measuring using the web server would have been possible with the examined approach, but in this experiment the participants did not have the implementation, as the performance-analysis for early stages of the design-process should be examined. The task orientated on the given design-documents of the web server. For each designalternative a UML-component-diagram war given. This diagram had to be prepared to be of use for the experiment. The diagrams should be usable as direct input parameters for the approaches, because the participants of the experiment should do as little routine work as possible. The following artefacts had been created: Abbildung 3.2.: Input parameters SPE SPE: basing on the component diagram and the source code, for each design-alternative a sequence-chart has been created (figure 3.2). These diagrams contained loops and execution alternatives for different choices of the data flow through the web server. So the required 9

14 3. Concept and Realisation of the Case Study execution graphs could be easily created with the SPE-ED-tool. The software-resources were given. The processing-overhead (a table in which the requirements of the softwareresources were depicted to the hardware-resources) was given as well. The preset of these values is usual in practice, as the values are given by performance-experts. The values were determined by measuring different kinds of network-connections and by using typical values for e. g. ISDN-connections to the client or 10 MBit-connections between the web server and an external server. This hardware-settings were simulated for later performance measuring with the web server. Additionally a queuing-model for distributing resources was given, which in fact was simply illustrating the settings, but could not be used with the SPE-ED-tool. Abbildung 3.3.: Input parameters Capacity Planning Capacity Planning: There was no realistical usage-profile available, which might have been used for the capacity-planning. So for this method Microsoft Excel produced an artifical log file, where the CPU-time, the number of hard disk-accesses and the processingtime of the LAN for 1000 accesses to the web server were documentated (figure 3.3). The values had been estimated in a way that the relation of performance and resources had been kept. So the calculations, that are done at this base, should provide enough informations to make proper design-decisions. Anyway it had to be expected that the absolute values would deviate from the later measurings with the implementation. Additionally the load for the resources CPU, HDD, LAN and the transfer-time to the client and the calculation-time of the external server were given. The loads where estimated by executing the prototype. The preset of these values was necessary for the later calculation and refers to the example from [MAD04]. 10

15 3.2. Experiment Abbildung 3.4.: Input parameters umlpsi umlpsi: for this approach a poseidon-project-file has been created, that already contained the required use-case-, activity- and deployment-diagram for the architecture of the web server (figure 3.4). This handicaps were made to reduce the effort of modeling and to get more time for performance-analysis. Such diagrams should exist in practice as well, when the performance-analysis starts. The use-case-diagram contained use-cases for request of static and dynamic HTML-pages. The according activity-charts were created by using the component-diagrams of the finished implementation. They were quite similar to the structure of the sequence-diagrams of SPE. The deployment-diagram used the same resources like the queuing-model, which was used for the SPE-approach and the capacity planning. The stereotypes and tagged-values according to the UML performance profiles, which were required for umlpsi, were set to zero because of the experiences from the exercises. The complete tasks (see appendix E) contained: 1. the instructions (step-by-step-introduction), 2. the software artefacts mentioned above for each approach, 3. supporting material like manuals, and 4. questionnaires to get personal data and comments The exercises followed the extra-subject-draft [Pre01]. The working-time was set to a maximum of 2 hours. The web server and the tasks have been designed to draw the focus on the elements, that are of use for the performance-analysis. Aspects like the monitoring of user-request or the use of different protocols have been ignored by the tasks, as they have just little or no influence on the overall-performance of the system. The connection to a database has been introduced to make 11

16 3. Concept and Realisation of the Case Study the performance-analysis more complex and less obviously. The two-tier architecture of the web server with the database on a separate server is quite easily but usual in practice. For the tasks the number of threads and the size of the used files were given, because this aspect probably would have had a big influence to the performance. The tasks for the experiment had to be designed in a way, that none of the examined approaches could take advantages. In this study none of the three approaches should be disadvantaged, because the input parameters had been customized for each process Design-alternatives Several performance-optimizing design-choices for the architecture of the web server should be given in the experiment, so that the participants just had to customize their models to this alternatives to do some performance-analysis. The design-alternatives should appear to be equal, so that none of the alternatives would be preferred intuitively. To let the participants favor just one alternative they were only allowed to implement one of them. The following design-alternatives where given: 1a) Static cache: As the first component in the Chain of Responsibility a static cache has to implement the IDeliverResponse-interface, which is called by the RequestProcessor. By this construction of the Chain-of-Responsibility requests for static pages first have to pass the main-memory-cache. The size of web pages is quite small in contrast to the usual size of main-memory, because web pages are usually optimized for slow internetconnections. Because of this, the hit-rate of the cache has been given as well: 70% of all request for static HTML-pages should be answered from the cache. 1b) Dynamic cache: this variant of the cache saves dynamic page-requests. The cachehit-rate of this cache is much smaller than the static cache hit-rate. For the experiment it had been set to 20%. Anyway this design-alternative offers a big potential to save expensive accesses to the external database, which costs much more than local harddisk-accesses. The implementation could be done as a further component between the components ExtAppInterface and ExtApp. 2) Singlethreaded server: the original implementation of the web server is multithreaded. For testing purposes a single-threaded version should be tested. This means that all requests are handled sequentially by the same thread. The response-time of the web server should vary, because the time to start threads and the context-change could be saved. On the other side the queue of the web server should get longer, so that the response-time of the server would probably increase subjectively for a user. The implementation could be very easily done by deactivating the start of threads in the DispatcherThread-component. 3) Compressing: means the ability to compress the requested pages before sending them on a slow internet-connection. This might offer a high potential of optimization. The implementation is quite simple, as such algorithms are given in the.net library and all modern internet-browsers support receiving compressed data. For performance-analysis this alternative is interesting as it has to be considered whether the time used for compressing the data is higher than the reduction of time by shorter transmissions. It was part 12

17 3.3. Performance Analysis of the Implementation of the participants choice to estimate the compression-ratio, which directly influences the time to send data. 4) Clustering: for this design-alternative the web server had been distributed on two different computers. A scheduler was distributing the incoming requests constantly to both servers. In an ideal case the performance of the system should be doubled, because all resources are replicated. The reliability should be higher as well. For this concrete scenario it was doubtful, whether the response time would decrease, as only a few users would request small sites. A certain time had to be considered because the scheduler would need time to distribute the requests. The implementation of the scheduler could be done by a new component that was put at the very beginning of the Chain-of-Responsibility Implementation The experiment was done in a pool-room with 8 windows-computers. Each group had 2 hours of time to do the experiment. Most of the participants could model all design-alternative. Some participants (especially capacity planning) could not finish their calculations within time. Anyway they could give some recommendations concerning the design-decision. Evaluation: The main reason of problems in the solutions was the estimated time for the single activities from the use-cases. In sum it was particularly much higher than one second. The ratio of requests was set to exactly one second. This caused resources with a processing-time of more than one second to be completely overloaded. The waiting-queues grew and the simulation could not be solved any more. To handle this the request-rate of each model was individually set to a lower level. This way is was possible to simulate all available solutions, even though the results were not directly comparably to the other methods, where the request-ratio was constantly set to one request per second. The raw data of the experiment can be found in the Appendix G, the results and the evaluation follow in the next chapter Performance Analysis of the Implementation The design-alternatives that were investigated, had been implemented for the web server in the following. This way the performance could be measured and compared with the predictions of the performance-approaches. In addition now some explanation follows to the measurement setup. A Notebook with a 1.5 GHz Intel Pentium-M processor and 512 MB of DDR RAM served as a test computer, on which the different variants of the web server and a Microsoft SQL database were installed. The latter served as an external application to simulate the external application from the experiment. For the simulation of the clients first several tools were tested for the generation of HTTP requests. The results for the reply-time during these tests were to a large extend identical with the different tools. They also matched with the data of the internal monitor component of the web server. It was sufficient to accomplish the complete evaluation of the design-alternatives with only one of these tools which had the best usability. 13

18 3. Concept and Realisation of the Case Study Finally the Web Performance Trainer from the Company Web Performance Inc. [Inc04] was selected. This tool was configured in such a way that the requests could be send to the web server from the simulated clients with a speed of 8 KByte/s. This value was given in the setting of the tasks for the experiment, where the clients should use a simple ISDN connections (64 KBit/s). The calling sequence of the tool consisted of a 5 KByte large HTML file (text), a 5 KByte large JPG-file (picture) and a dynamically generated HTML-page, likewise with a size of 5 KByte. For the dynamic page a retrieval query was performed against the database and a short delay was added, so that the request took exactly 0.5 seconds, how it was indicated in the setting of the tasks for the experiment. The dynamic page was sequentially queried three times, in order to simulate the ratio of 40:60 between static and dynamic requests from the tasks for the experiment. The entire calling sequence of the five request was repeated permanently for a period of 5 minutes, in order to get a large number of measured values and to minimize external influences to the values by averaging. With the tool it was not possible to adjust the arrival rate of requests accurately to one per second as it was given in the tasks. The request rate could only by the adjusted by the number of users, who accessed the web server at the same time. Here the value of 1.5 simultaneous users was selected, as the number of requests with this value was within 5 minutes next to about 300 (1 request per second). The exact request rates are included in the result diagram and were determined for each design-alternative over the number of request within the 5 minutes. Even if the values did not match exactly with the experiments tasks, measurements with much higher request-rates (10 simultaneous users in the system) showed that for the response-time no significant deviation had to be expected. Therefore the measurements should be comparable with the predictions of the approaches. For alternative 1a the cache for static pages was configured to answer 70 percent of the requests from the cache, while the remaining 30 percent were answered from the hard disk. In the case of dynamic caching 20 percent of the requests were answered by the main memory, while for 80 percent the database had to be used. By this the settings from the experiment were exactly simulated. The additional request of a JPG-file was done, because in design-alternative 3 (compression) the type of file has influence on the response-time. HTML-files exist of well compressable ASCII text (compression in average more than about 50%, maximally even up to 90% [Kil02, p.348]). JPG files in contrast are already in a compressed format and can usually not be further compressed. Therefore a differentiated request of strong and weakly compressable files is meaningfully, in order to refine the results for the response-time and to examine the actual effect of design-alternative 3. For alternative 4 (clustering) two web servers and the scheduler were started. The scheduler distributed the requests to both servers even. Parallel to the measurement of the run-time by the Web performance Trainer the percentage utilization of the CPU and the hard disk were monitored with the Windows XP performance monitor during the simulation. These values can be compared with the predictions of load. During the measurements all unnecessary processes on the computer were terminated, in order to avoid uncontrollable affects to the measurements. Nevertheless it can not be excluded that operating-system internal processes distorted the measurements. During the tests no significant 14

19 3.3. Performance Analysis of the Implementation events occured. The measurement took 5 minutes. The average response-times was calculated for each case. The raw data of all measurements can be found in the appendix I, diagrams with the average values follow in section

20 16

21 4. Results In chapter 2 4 questions about performance evaluation approaches were posed. In this chapter the collected data are illustrated and interpreted in respect of this questions Validity of the prediction 1. What is the accuracy of the forecasting with the performance prediction approaches? In order to come up to this question, four metrics were specified: the deviation in the input data of different participants (metric 1), the discrepancy in the input and output data of different approaches (metric 2), the discrepancy of output data of the approaches in relation to measurements (metric 3) and the percentage of proper design decisions (metric 4). As with the approaches equal input data are mapped to identical outputs it is sufficient to consider the output data in order to analyse metric 1. During the experiment response time and throughput(utilisation) were predicted with the three approaches Predictions with SPE Figure 4.1 presents for each participant (german: Teilnehmer, x-axis) with SPE-approach predicted response times for different design alternatives. The arrival rate of client-requests is 1 request/sec, thereby the requests are composed of 40% of static pages and 60% of dynamic ones. Several participants did not succeed to calculate the response time for design alternative 2 (single threaded servers). It was difficult to model the number of threads with the SPE-approach. The mean service time of the non-optimised variant of the Web server (alternative 0) is seconds, the standard deviation is with seconds quite high. Nevertheless significant patterns concerning the performance behavior of the single design alternatives are observable. With seven of eight solutions alternative 3 (compression) has the smallest response time, the second fastest alternative is with six of eight solutions the alternative 1b (dynamic Cache). The other alternatives seem to provide little improvements of the response time, alternative 2 (single threaded servers) causes even a degradation of the performance. For clarity the average utilization over all participants for each resources and each alternative is presented in table 4.1. The individual values of the participants can be seen in the appendix. The load of the Internet and the external application (Delay) is the highest. The utilization of CPU, disk and LAN is so small that it can be neglected. With the dynamic cache the utilization of Delay resources sinks around 10% on 42%. This corresponds to the task formulation, since 17

22 4. Results 3,5 Durchlaufzeit in Sekunden 3,0 2,5 2,0 1,5 1,0 0,5 0, Alternative 0 1,1928 2,9415 0,9562 1,3168 1,5503 1,8476 2,3094 0,6845 Alternative 1a 1,1883 2,9338 0,9499 1,3108 1,5196 1,8204 2,3030 0,6200 Alternative 1b 1,1317 2,8670 0,8981 1,2545 1,4868 1,7826 2,1478 0,5800 Alternative 2 0,9546 1,3482 1,5477 1,8428 2,3944 Alternative 3 0,8188 1,6856 0,7096 0,8766 1,2979 1,1221 1,8822 0,6089 Alternative 4 1,1952 2,9367 0,9588 1,3152 1,5323 1,8421 1,9892 0,6878 Teilnehmer Abbildung 4.1.: Predicted response times with SPE-approach CPU Disk LAN Delay Inet Alternative 0: no optimisation 5% 2% 0% 53% 125% Alternative 1a: statical cache 4% 2% 0% 52% 125% Alternative 1b: dynamical cache 4% 3% 0% 42% 124% Alternative 2: single-threaded Server 8% 4% 0% 102% 178% Alternative 3: compressing 5% 3% 0% 48% 78% Alternative 4: clustering 3% 1% 0% 48% 125% Tabelle 4.1.: Predicted utilization with SPE-approach (mean) 18

23 4.1. Validity of the prediction the data now can be accessed directly from the main storage of the Web server and the external application is addressed more rarely Prediction with Capacity Planning Durchlaufzeit in Sekunden 2,0 1,8 1,6 1,4 1,2 1,0 0,8 0,6 0,4 0,2 0, Alternative 0 1, , , , , ,50522 Alternative 1a 1, , , , , ,50522 Alternative 1b 1, , , , , ,40386 Alternative 2 1, ,49190 Alternative 3 1, , , , , ,10916 Alternative 4 1, , , , , ,44089 Teilnehmer Abbildung 4.2.: Predicted response times with CP-approach (dynamical site requests) For the Capacity Planning separated results for static and dynamic page requests were calculated. The relationship from static to dynamic queries was as with the SPE 40:60 and the user arrival rate was a user per second. The mean response time of the alternative 0 is 1,5053 seconds for dynamical page requests and 0,9449 seconds for statical ones. The standard deviation for both types of requests is 0,0001 seconds. For the alternative 3 the compressing rate has to be estimated by participants. This resulted in deviation of predicted values. With five of the six solutions the response time for alternative 3 was the smallest, for all participants was alternative 1b the second fastest variant of the Web server. Only one participant estimated the CPU time for the compression longer than the time, which could be saved by the shorter occupying of the InterNet connection. Also with this approach most (six of eight) participants did not have an idea for modeling of alternative the 2 (single threaded servers) and provided no results. The utilization of resources CPU, disk and LAN had been given in the task of the experiment, for other resources (INet, Delay) these values could directly be computed. Therefore the values are similar with all participants for alternative 0 (no optimisation). 19

24 4. Results Durchlaufzeit in Sekunden 1,0 0,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0, Alternative 0 0, , , , , ,94464 Alternative 1a 0, , , , , ,92395 Alternative 1b 0, , , , , ,94464 Alternative 2 0, ,94329 Alternative 3 0, , , , , ,52064 Alternative 4 0, , , , , ,90888 Teilnehmer Abbildung 4.3.: Predicted response times with CP-approach (statical site requests) CPU Disk LAN Delay Inet Alternative 0: no optimisation 8,2% 1,2% 0,4% 29,5% 87,5% Alternative 1a: statical cache 8,2% 0,36% 0,4% 29,5% 87,5% Alternative 1b: dynamical cache 8,2% 1,2% 0,3% 23,6% 87,5% Alternative 2: single-threaded server 7,92% 1,2% 0,4% 29,5% 87,5% Alternative 3: compressing 15,63% 1,1% 0,4% 28,8% 48,5% Alternative 4: clustering 4,4% 0,6% 0,4% 29,5% 87,5% Tabelle 4.2.: Predicted throughputs with CP-approach (mean) 20

25 4.1. Validity of the prediction Durchlaufzeit in Sekunden 25,0 0,5 Anfr/sec 0,5 Anfr/sec 0,2 Anfr/sec 0,17 Anfr/sec 0,06 Anfr/sec 0,2 Anfr/sec 0,2 Anfr/sec 0,17 Anfr/sec 20,0 15,0 10,0 5,0 0, Alternative 0 2,3932 5, , , ,1166 6, ,0072 8,7612 Alternative 1a 2,3922 5, , , ,4039 5, ,3145 8,6219 Alternative 1b 2,1750 5,2493 9, , ,7919 5, ,3239 7,1867 Alternative 2 3,2306 6,0873 Alternative 3 1,4886 3,1577 8,3245 5, ,2858 5,7347 7,4177 9,3413 Alternative 4 2,3103 5,0517 9, , ,3719 5, ,0813 8,2451 Teilnehmer Abbildung 4.4.: Predicted Response times with umlpsi-approach (dynamical) Durchlaufzeit in Sekunden 20,0 0,5 Anfr/sec 0,5 Anfr/sec 0,2 Anfr/sec 0,17 Anfr/sec 0,06 Anfr/sec 0,2 Anfr/sec 0,2 Anfr/sec 0,17 Anfr/sec 15,0 10,0 5,0 0, Alternative 0 1,6542 4,8946 8,5177 9, ,5900 3, ,8026 3,7014 Alternative 1a 1,6285 4,8108 7,8333 8, ,2966 3, ,5626 3,6322 Alternative 1b 1,6742 4,8831 7,2854 9, ,0725 3, ,7189 3,6938 Alternative 2 2,5957 3,8117 Alternative 3 0,8839 2,6538 7,3874 5, ,3815 3,4865 5,9705 4,4117 Alternative 4 1,5086 4,3526 7, , ,9589 2, ,8371 3,2515 Teilnehmer Abbildung 4.5.: Predicted Response times with umlpsi-approach (statical) 21

26 4. Results Prediction with umlpsi-approach The umlpsi-approach provides also separate performance-results for statical and dynamical page requests in relation 40:60. For the simulation of the performance models another arrival rate for the incoming requests had to be given than with the other approaches (there it was a query per second). Otherwise the simulator routine due to the high estimations for the input data would be overloaded. The arrival rate for requests was therefore adapted individually in each case for each participant, so that the simulation was not overloaded. Therefore the response times and utilization predicted with this approach are only conditionally comparable with those of the other approaches. For the non-optimised version of the web server (alternative 0) the predicted mean response time is 10,3608 seconds for dynamical page requests and 7,9624 for statical ones. The standard deviation is 6,0659 (dynamic) and 5,4411 (static). All participants were able to model the alternative 2 (single threaded servers). Therefor only the appropriate entry of a parameter in resources Threads was required. During the simulation of the models with umlpsi it turned out however that in most cases no response time for this design alternative could be determined. Reason for it was the strong overloading of resources Threads, which can be observe in table?? for utilization, where this resources are utilised with 100 percent (alternative 2). CPU Disk LAN Delay Inet Threads Alternative 0: no optimisation 19,3% 4,0% 10,0% 9,4% 52,2% 25,8% Alternative 1a: statical Cache 19,6% 1,4% 10,2% 9,4% 52,0% 25,4% Alternative 1b: dynamical Cache 19,0% 4,0% 8,2% 7,5% 51,5% 24,4% Alternative 2: single-threaded Server 17,8% 3,6% 9,5% 8,9% 49,7% 100,0% Alternative 3: compressing 29,9% 3,9% 10,0% 9,4% 34,1% 21,6% Alternative 4: Clustering 12,0% 2,0% 10,0% 9,4% 52,0% 21,2% Tabelle 4.3.: predicted utilisation with umlpsi-approach (mean) Measurement Results of the Implementation The tests setting for performance measuring of the implementation, for the different variants of the web server, have been discussed in section 3.3. Now the results of the measuring will follow. The average values for response time (figure 4.6) and the load (fugure 4.7) of the web server will be presented for each variant. For the non-optimized version of the web server the response times for both types of static pages (HTML-document and the JPG-image from the hard disk, each with a size of 5 KByte) was nearly identically. The request of the dynamic page (HTML-document from the database, 5 KByte of size) took quite exactly 0.5 seconds more of time. This fits to the settings from the task of the experiment and can be used as a initial position. The measuring of the variant with the static cache shows, that within the measuring precision no deviation (at most a little rise of time) of the response time for static pages, compared with the non-optimized variant, could be found. This is surprising, as at least 70 percent of the static requests for this alternative had to be answered from the main memory, which causes no 22

27 4.1. Validity of the prediction statische Abfrage (HTML-Datei, 5 KByte) statische Abfrage (JPG-Datei, 5 Kbyte) dynamische Abfrage (HTML-Datei, 5 Kbyte) 1,80 0,97 Anfr./sec 0,93 Anfr./sec 1,08 Anfr./sec 0,97 Anfr./sec 1,11 Anfr./sec 0,96 Anfr./sec 1,60 1,65 Durchlaufzeit in Sekunden 1,40 1,20 1,00 0,80 0,60 0,40 0,97 1,51 1,53 1,44 1,01 1,02 1,00 0,99 1,00 1,04 1,08 1,56 0,94 0,81 1,10 1,19 0,20 0,00 Alternative 0: keine Optimierung Alternative 1a: Statischer Cache Alternative 1b: Dynamischer Cache Alternative 2: Single- Threaded 0,10 Alternative 3: Komprimierung Alternative 4: Clustering Abbildung 4.6.: Response time of the Web server (measured); y-axis: response time in seconds; x-axis: alternative 0 no optimization, alternative 1a static cache, alternative 1b dynamic cache, alternative 2 single threaded, alternative 3 compression, alternative 4 clustering; blue bar: static request (HTML-file, 5 KB); red bar: static request (JPG-file, 5 KB); white bar: dynamic request (HTML-file, 5 KB) 23

28 4. Results expensive harddisk-access. For the more precisely investigation of this observation the Windows XP Performance Monitor was used to measure the number of reading accesses to the harddisk of the non-optimized version of the server. It became clear, that from the 50 repeating requests only the very first caused a harddisk access at the beginning, though the static cache was deactivated. This phenomenon is caused by the operating system windows itself, which puts the called files to a system cache in the main memory, from which the following request were answered. So the implementation of the static cache was useless, as the operating system did the caching. A manual deactivation of the system cache is not performable, as this option is anchored deep inside of the operating system. For the file opening methods from the.net-classlibrary which were used for the implementation this cache is fully transparent. The suggested design-alternative, that used the static cache was fully irrelevant for practice. This behaviour could not be predicted by the performance-analysis-approaches. For the use of a dynamic cache a reduction of response time of dynamic page requests was determined as expected (from 1.51 seconds to 1.44 seconds). From the raw data in appendix I is can be easily seen, that the dynamic cache was activated after about 80 percent of the requests. The response time for the following dynamic request reduced to about 1 second and corresponds to the response time of static requests that are answered from the main memory. The average value of cached and non-cached requests for this alternative only reveals a little performance-win of 0.07 seconds. This value is tightly coupled to the handicap, that only 20 percent of the requests are cached, so this values is strongly dependent on the used application. If a web-application allows to cache much more dynamic content, the response time would decrease, whereas the lower limit is the response time of static pages. For static pages this variant does not improve the response time. The single threaded server (alternative 2) shows a slightly higher response time in the comparison to the non-optimized multi-threaded version of the web server. The requests retain and the response time increases, as only one request can be processed at the same time by the server. With the multi-threaded version the times for the start of new Threads and the context change do not preponderate that much, and no significant degradation of the performance results. In this scenario the server is loaded still too little, as maximally two request are handled at the same time. For the examined use-case design-alternative 2 thus offers no performance advantage in comparison to the non-optimized version of the web server. Partly very clear Performance-wins result however in the case of alternative 3 (compression). Both the static and the dynamically produced HTML-page could be well compressed, the time for sending the files reduced proportionally to the compression factor. With the JPG file only a slight compression is possible, here the response time only reduced about a few milliseconds. The computations, which are used for the compression, here hardly fall into weight, because with the compression of small files works extremely fast with the used algorithms. With the Clustering minor performance-loss was determined, the service times increases minimal with all queried pages. This can be explained with the time the scheduler uses to distribute the requests to the two web servers. In the given setting of the system it is not that heavily loaded, so the performance could not be improved by clustering. The measured load (figure 4.7) of the system is nearly identical for all variants. The hard disk load is hardly measurable, as, as mentioned above, nearly no hard disk accesses were caused, because the files were already put to the Windows system cache. The CPU-load for all variants 24

29 4.1. Validity of the prediction 10,00% CPU Disk Auslastung in Prozent 9,00% 8,00% 7,00% 6,00% 5,00% 4,00% 3,00% 7,36% 7,86% 8,01% 7,07% 8,20% 6,27% 2,00% 1,00% 0,00% Alternative 0: keine Optimierung 0,28% 0,25% 0,11% Alternative 1a: Statischer Cache 0,41% Alternative 1b: Alternative 2: Single Dynamischer Cache Threaded 0,15% Alternative 3: Komprimierung 0,84% Alternative 4: Clustering Abbildung 4.7.: Load of the web server (measured); y-axis: load in percent; x-axis: alternative 0 no optimization, alternative 1a static cache, alternative 1x dynamic cache, alternative 2 single threaded, alternative 3 compression, alternative 4 clustering; blue bar: CPU; red bar: disk 25

30 4. Results is at the same level (within the measuring precision). For the single-threaded server the CPUload is slightly less in comparison to the non-optimized variant. This could result from the fact that with this alternative in each case only one request is answered at the same time and the CPU is not used for the context change of threads, but the minor load of the CPU causes higher response-times. In the case of clustering the load is reduced as well. This can be explained by the distribution of load to two servers. However the deviation of load is so little (about 1%) with both, the single threaded the server and the clustered server, that simple measurement inaccuracies could be present, which can never be completely excluded with such an experimental setup Comparison of Predictions and Measurements In figures 4.8 and 4.9 the predicted response times are displayed along with measurements of the implementation. The values for each individual prediction approach are the average values over the predictions of all participants, who predicted with this approach. With the SPE there is only one value both for static and dynamic queries, this value appears in both diagrams. 12,00 Statische Anfragen SPE CP umlpsi Messung 10,00 Durchlaufzeit in Sekunden 8,00 6,00 4,00 7,96 7,24 7,34 5,93 6,55 2,00 0,00 1,60 1,58 1,52 1,62 1,56 1,13 0,94 0,99 0,92 1,01 0,94 1,00 0,94 1,06 1,15 0,91 0,67 0,52 Alternative 0 Alternative 1a Alternative 1b Alternative 2 Alternative 3 Alternative 4 Abbildung 4.8.: Comparison of mean response times (static) With the SPE and the CP approaches the response times were predicted in each case with a user arrival rate by a user per second. With the umlpsi approach the user arrival rate had to be individually tuned for each participant because of the problematic simulation. During the measurements of the Web server the queries could not be generated by the test tool accurately in the one-second pulse, however we tried to keep the deviation as small as possible. User requests with higher arrival rates did not show large deviations in the results of measurement, so that the obtained results are sufficient for the comparison with the predictions. The high values for the umlpsi stand out obviously in these diagrams. The proportional deviations of the response times over all variants of the Web server can be seen in table??. 26

31 4.1. Validity of the prediction Dynamische Anfragen SPE CP umlpsi Messung 12,00 10,00 10,36 10,08 Durchlaufzeit in Sekunden 8,00 6,00 4,00 9,36 7,88 9,02 2,00 0,00 1,60 1,51 1,51 1,58 1,51 1,53 1,52 1,62 1,40 1,44 1,49 1,56 1,56 1,65 1,36 1,45 1,13 0,81 Alternative 0 Alternative 1a Alternative 1b Alternative 2 Alternative 3 Alternative 4 Abbildung 4.9.: Comparison of mean response times (dynamic) Deviation SPE CP umlpsi Measurements static sites: 53,59% 9,46% 530,03% 0% dynamic sites: 8,77% 11,93% 494,54% 0% Tabelle 4.4.: mean values of the deviation over all alternatives 27

32 4. Results The authors of the approaches often stress that the approaches are only conditionally suitable for the prediction of absolute performance values. However the approaches should enable the evaluation of different design alternatives. Therefore an analysis of the proportional deviations of the values for the individual design alternatives appears meaningful. For each participant the alternative 0 is set as basis with 100%response time and for the other variants of the Web server the averaged proportional deviation is determined. The results are presented in figures?? and??. Here the predictions with the performance prediction approaches and the measurements reside more closely to each other. In particular with alternative 3 a clear reduction of the response time is observable, while the deviation is small with the other alternatives. 120% Statische Anfragen SPE CP umlpsi Messung 116% Durchlaufzeit in Prozent 100% 80% 60% 40% 102% 100% 100% 100% 100% 101% 99% 100% 101% 100% 98% 95% 91% 92% 107% 74% 70% 70% 53% 97% 96% 82% 20% 0% Alternative 0 Alternative 1a Alternative 1b Alternative 2 Alternative 3 Alternative 4 Abbildung 4.10.: relative response times (static) 4.2. Methodical opposite the Intuitive Manner 3. What are the advantages of applying the prediction techniques opposite to intuitive manner? The tasks for the experiment were additionally presented to 7 further students. These students had no knowledge of the used performance prediction approaches. In this way it should be detected whether the intuitive evaluation of the design alternatives leads to different decisions for the best one. The participants spent on average 40 minutes for the evaluation of the alternatives. Some participants commented that the UML sequence diagrams were of no use for them. The diagrams had been integrated into setting of tasks, so that the inputs for all participating groups were similar and it could not come to any distortion of the results due to different basic informations. 28

33 4.2. Methodical opposite the Intuitive Manner Dynamische Anfragen SPE CP umlpsi Messung 120% Durchlaufzeit in Prozent 100% 80% 60% 40% 100% 100% 100% 100% 99% 100% 101% 97% 95% 95% 93% 90% 101% 99% 103% 90% 76% 70% 54% 109% 97% 96% 87% 20% 0% Alternative 0 Alternative 1a Alternative 1b Alternative 2 Alternative 3 Alternative 4 Abbildung 4.11.: relative response times (dynamic) Four of the intuitive solutions favored design alternative 1b (dynamic Cache), three participants decided for alternative 3 (compression). 29

34 30

35 5. Conclusions and Perspectives The following chapter sums up this work and its results Summary Each participant proposed one alternative out of the five given design-alternatives. The bigger part (72%) favourised alternative 3, the compression of HTTP-requests. This design-alternative had in fact (implementation) the slightest response-time of all alternatives. The predicted absolute response-time gained by the approaches, particularly for some participants, differed several seconds from the measured data. The estimation of some performancevalues of actions and resources was quite difficult for some participants. Anyway they could evaluate the design-alternatives realistically by their relative differences. The predicted percentual performance-profit of the different design-alternatives could be determined with less than 15% deviation to the measurements of the implementation. The independent control-group of seven students, who didn t know the performance-approaches favourised a different design-alternative than the participants of the experiment. The majority preferred alternative 1b, which was the second fast alternative of the measured alternatives. This means that the tasks did not intuitively lead to alternative 3 from the 5 available designalternatives. A lot of properties of the approaches, like mightiness of the notation, the ergonomics of the tools or the evaluation-methods, have already been discussed. Most of the approaches are formally quite elaborated, but their practical usability and the availability of efficient tools is very poor. The developers had to put a lot of manpower into the analyses to get realistical results and predictions. Especially the availability of UML and the UML performance-profiles is a starting-points that might let await a better integration of performance-analysis into the object-oriented software-engineering. The SPE-Approach of Smith and Williams is very usable for early performance-analysis of software-architectures. It is a approach which makes performance-predictions out of little input. The method is well evaluated after 20 years of research. The basical waiting-queue-model is well encapsulated to reduce the complexity. At all this approach is easily usable for developers. Anyway this approach reveals a break between software-modeling and performance-modeling. The used execution-graphs have to be created manually from the software-specification (e. g. UML), a automatic transformation is not offered by SPE-ED. As well a back-transformation of the performance-results to the software-specification is not intended. SPE-ED is not integratable into other (CASE-) tools. Even there is no possibility to model the run-time-environment like in Java or to model the influence of application servers. The results of already measured and implemented parts have be added manually to the performance-model. Not at least the GUI should be improved. 31

36 5. Conclusions and Perspectives The CP-approach is tightly coupled to the classical waiting-queue and is not useful for early performance-prediction, but more for existing systems. After more than 30 years of research in waiting-queues there exist a lot of algorithms and formulas to evaluate a big number of real computer systems. Often only approximated solutions can be created. For early performance prediction it often is not worth creating complex models and employing with the mathematical backgrounds. The input parameters in the most cases are measured data and not estimations. The analysis mostly takes only the hardware-layer into account. A integration of software-models is not intended as well as coupling to software-tools. umlpsi is a interesting approach. It uses UML diagrams out of which an automatically generated simulation of performance-behaviour for a given architecture is done. Therefore the diagramms first have to be annotated with performance-data in the standardised UML Performance Profile. It is the first approach that uses this standard for simulations. After the simulation the results can be transferred back to the diagrams. This approach is the only one which supports a feedback-mechanism. It should not be underrated that the processing of raw data consumes a lot time. umlpsi is the approach out of the three which is the very most like real softwareengineering. Developers might be convinced the easiest way of this performance-approach. They just have to learn UML Performance Profile but no evaluation-methods. Anyway umlpsi is just a prototype which does not work completely stable. Extensions for this tool might be a support for other types of UML-diagrams. The support of further UML Profiles e. g. for reliability, safety and other Quality-of-Service-properties. The profiler allows to measure the run-time for readily implemented code-fragments. This way the estimation in the diagrams can be stated more precisely, the models can create improved overallresults. 32

37 Literaturverzeichnis [BCR94] Victor R. Basili, Gianluigi Caldiera, and H. Dieter Rombach. The goal question metric approach. Encyclopedia of Software Engineering - 2 Volume Set, pages , [GHJV95] E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design Patterns: Elements of Reusable Object-Oriented Systems. Addison-Wesley, [Inc04] Web Performance Inc. Web performance trainer. com, [Kil02] Patrick Killelea. Web Performance Tuning. O Reilly, 2nd edition, [Lib03] Jesse Liberty. Programming #. O Reilly, 3rd edition, June [MAD04] [Mar04] [Pre01] [Smi02] Daniel A. Menasce, Virgilio A.F. Almeida, and Lawrence W. Dowdy. Performance by Design. Prentice Hall, Moreno Marzolla. Simulation-Based Performance Modeling of UML Software Architectures. PhD thesis, Universit a Ca Foscari di Venezia, Lutz Prechelt. Kontrollierte Experimente in der Softwaretechnik. Springer Verlag, Connie U. Smith. Performance Solutions: A Practical Guide To Creating Responsive, Scalable Software. Addison-Wesley,

38 34

39 A. Vergleichstabelle Performance-Analyseverfahren Die folgende Tabelle enthält zusammengefasst die Daten der in Kapitel?? vorgestellten Performance-Analyseverfahren für Software-Architekturen. Eine Abbildung des jeweiligen Vorgehensmodells wurde intergriert, sofern diese vorlag. Folgende Abkürzungen werden verwendet: AQN: Augmented Queueing Network EQN: Extended Queueing Network LTS: Layered Transaction System LQN: Layered Queueing Network MSC: Message Sequence Chart QN: Queueing Network UML: Unified Modelling Language XSLT: Extensible Stylesheet Language Transformations i

40 Name Architektureinschränkung Komponentenbasierte Systeme V1 (SPE) V2 (PRIMA-UML) V3 V4 Allgemeines Autor Smith / Williams Cortellessa / Mirandola / Grassi Cortellessa / D Ambrogio / Iazeolla Gomaa / Menasce Jahr 2002 (bzw. 1990) , 2001 Literatur [Smi90, Smi02] [CM02, GM02] [CDI01] [GM00, GM01] Applikationsdomäne allgemein (Fallstudien für Web- Applikationen und eingebettete Realzeitsysteme) allgemein (spezielle Erweiterung für mobile Software-Architekturen vorhanden) Modellierung Software-Modell MSC, Execution Graphs Use-Case-, Sequenz-, Deploymentdiagramme, Execution Graphs allgemein allgemein Klassen-, Sequenzdiagramme, Execution Graphs Performance-Modell QN EQN LQN EQN Auswertungsmethode Analyse/Simulation Analyse Analyse Analyse Feedback der Ergebnisse in das Software-Modell nein Werkzeug SPE-ED: Erstellung von Software- Execution-Models (Execution Graphs) und System-Execution-Models (QN), automatisiert die Berechnungen Prozessmodell nein nein nein Halbautomatische Ableitung der Performance-Modelle aus den Software-Modellen Manuelle Transformation der Software-Modelle in LQNs, Nutzung von Standard CASE-Tools Klassen-, Kollaborationsdiagramme - Sonstiges Performance Patterns/Antipatterns ausführliche Abhandlung von Vorgehensmodellen (Integration in Unified Process) proprietäre Annotation der UML- Diagramme erster Ansatz für komponentenbasierte Systeme

41 Name V5 V6 (CLISSPE) V7 V8 Allgemeines Autor Petriu / Gu / Wang Menasce / Gomaa Andolfi / Aquilani / Inverardi Woodside / Hrischuk / Selic / Brayarov Jahr 1999, , Literatur [PW99, PS02, GP02] [MG98], [MG00] [AABI00] [WHSB01] Architektureinschränkung Architekturmuster (Client/Server, Pipe-and-Filter, Broker usw.) Applikationsdomäne allgemein allgemein Modellierung Software-Modell UML-Kollaborationen, Deploymentdiagramme Client/Server - - Use-Case-, Klassen-, Kollaborationsdiagramme allgemein allgemein MSC, LTS MSC Performance-Modell LQN QN QN LQN Auswertungsmethode Analyse Analyse Analyse Analyse Feedback der Ergebnisse in das Software-Modell vorgesehen nein? nein Werkzeug XSLT-Transformationen CLISSPE System? ObjecTime Developer, PAMB (Performance Analysis Model Builder) Prozessmodell Sonstiges automatische Transformation des Software-Modells in ein Performance-Modell mit Graphen- Transformation Perfomrance-Annotationen in XML

42 Name V9 V10 V11 V12 Allgemeines Autor Kähkipuro Lindemann et. al. Miguel et. al. Arief / Spears Jahr Literatur [Käh01] [LTK+02] [MLH+00] [AS99] Architektureinschränkung Applikationsdomäne Komponentenbasierte Systeme allgemein Real-Time-Systems allgemein Modellierung Software-Modell UML, Zustands-, Aktivitätsdiagramme UML Klassen-, Sequenzdiagramme Zwischendarstellung Performance Modelling Language (PML) Performance-Modell AQN Generalisierte Semi-Markow OPNet Simulationsmodell Simulation Modelling Language Prozesse (SimML) Auswertungsmethode Analyse Analyse Simulation Simulation Feedback der Ergebnisse in das ja vorgesehen ja nein Software-Modell Werkzeug OAT (Object-oriented performance modelling and Analysis Tool) Prozessmodell DSPNexpress2000 AMG (Analysis Model Generator) SMG (Simulation Model Generator) Parser für die SimML Notation (Perl- Parser erstellt C++SIM Simulationen, Java-Parser erstellt JavaSIM Simulationen) Sonstiges

43 Name V13 (umlpsi) V14 V15 V16 Allgemeines Autor Marzolla / Balsamo Hennig / Revill / Ponitsch King / Pooley Bernardi Jahr Literatur [BM03] [Mar04] [HRP03] [KP00] [BDM02] Architektureinschränkung Applikationsdomäne allgemein allgemein Modellierung Software-Modell Use-Case-, Aktivitäts-, Deploymentdiagramme, UML Performance Profile UML (Sequenz-, Kollaborations-, Deploymentdiagramme) allgemein allgemein UML (Kollaborationsdiagramme mit eingebetteten Zustandsdiagramme) UML (Zustands-, Sequenzdiagramme) Performance-Modell lipcppsim-simulation Simulation Generalized Stochastic Petri-Nets Generalized Stochastic Petri-Nets Auswertungsmethode Simulation Simulation Analyse Analyse Feedback der Ergebnisse in das Software-Modell ja ja nein nein Werkzeug umlpsi (UML Performance Simulator) lipcppsim Prozessmodell OMNET++ mit SPE-Erweiterungen SPNP nein 1. UML-Diagramme 2. manuelle Transformation in GSPN 3. numerische Auswertung der Petri-Netze mit SPNP Sonstiges PhD-Thesis veröffentlicht Tests mit JBoss PhD-Thesis veröffentlicht

44 Name Architektureinschränkung Client-Server Systeme V17 V18 (PERMABASE) V19 V20 (Capacity Planning) Allgemeines Autor Bernardo Waters et. al. Hoeben Menasce / Almeida / Dowdy Jahr Literatur [BCD00a] [WLA+01] [Hoe00] [MA02],[MAD04] Applikationsdomäne allgemein ATM-based Applications and Services allgemein allgemein Modellierung Software-Modell ADL (AEMPA) UML UML Performance-Modell Extended Markovian Process Algebra (EMPA) Zwischendarstellung CMDS (Composite Model Data Structure) (Klassen-, Komponenten-, Sequenz-, Kollaborations-, Deploymentdiagramme) QN QN Auswertungsmethode Analyse Analyse Analyse Analyse Feedback der Ergebnisse in das Software-Modell nein Feedback in CMDS nein nicht möglich Werkzeug TwoTowers 2.0?? Excel-Spreadsheets zur Auswertung von QN. Prozessmodell - Sonstiges PhD-Thesis veröffentlicht keine Modellierung der Software- Architektur, nur System-Modelle

45 B. Folien der Übungen Zum Training der Versuchsteilnehmer wurde für jedes untersuchte Verfahren ein Foliensatz zusammengestellt. Die enthaltenen Abbildungen und Beispiele entstammen jeweils aus der zugehörigen Literatur [Smi02], [MAD04], [Mar04]. Mit den Folien kann nachvollzogen werden, auf welcher Wissenbasis die Versuchsteilnehmer ihre Performance-Analysen durchführten. vii

46 Inhalt Software Performance Engineering (SPE) Smith/Williams Allgemeines Vorgehensmodell SPE und UML erweiterte Sequenzdiagramme Software Execution Models Execution Graphs System Execution Models Queueing Network Performance-orientiertes Design Beispiel Software Performance Engineering 2 Allgemeines 1981: Prägung des Begriffs Software Performance Engineering durch Connie U. Smith 1990: Buch Performance Engineering of Software Systems erste vollständige Methode zur Performance- Analyse von Software-Systemen während des Entwurfs 2002: Buch Performance Solutions Integration der UML Anpassungen für verteilte bzw. eingebettete Systeme Tool SPE-ED Performance-Patterns/Antipatterns SPE-Vorgehensmodell Software Performance Engineering Software Performance Engineering 4 Vorgehensmodell Beispiel: Entwurf eines Geldautomaten Vorgehensmodell Festlegung des Umfangs des SPE- Projekts je nach Signifikanz der Performance Erstellung eines detaillierteren Modells bei Performance-kritischeren Anwendungen Beispiel Geldautomat: geringes Performance-Risiko, einfaches Modell ausreichend Software Performance Engineering Software Performance Engineering 6 1

47 Vorgehensmodell Vorgehensmodell Software Performance Engineering Software Performance Engineering 8 Vorgehensmodell Konkrete Vorgaben für z.b. Antwortszeiten, Durchsatz, Ressourcennutzung usw. Vorgehensmodell Transformation der Sequenzdiagramme in Ausführungsgraphen Beispiel Geldautomat: Die Bedienung darf nicht länger als 30 Sekunden dauern Software Performance Engineering Software Performance Engineering 10 Vorgehensmodell Transformation der Sequenzdiagramme in Ausführungsgraphen Vorgehensmodell Bestimmung der Anforderungen von Software-Elementen Software Performance Engineering Software Performance Engineering 12 2

48 Vorgehensmodell Bestimmung der Anforderungen von Hardware-Elementen Vorgehensmodell Berechnung der Performance-Werte Beispiel Geldautomat: Die Bedienung dauert 29 Sekunden und ist somit knapp innerhalb der Vorgaben bei Nichteinhaltung der Vorgaben: Modifikation der Software Modifikation der Szenarien Überarbeitung der Performance-Ziele Software Performance Engineering Software Performance Engineering 14 SPE und UML SPE und UML UML: Standard-Notation für die Modellierung objektorientierter Systeme Basis für die Erstellung des Performance- Modells Erweiterungen der UML für Performancespezifische Informationen (angelehnt an Message Sequence Charts) hierarchische Strukturierung von Sequenzdiagrammen graphische Repräsentation von Schleifen und Auswahlknoten in Sequenzdiagrammen Nebenläufigkeit in Sequenzdiagrammen Software Performance Engineering Software Performance Engineering 16 Schleife: gebunden an Schleifenklausel Auswahl: gebunden an Bedingungsklausel Referenz: verzweigt in ein weiteres Sequenzdiagramm Beispiel: Erweiterte Sequenzdiagramme Sequenzdiagramme: Kontrollfluss Prozeduraufrufe: Fortsetzung der Ausführung erst nach Beendigung aller verschachtelten Aufrufe Nichtprozeduraler Aufruf: nächster Schritt in der Sequenz Asynchrone Kommunikationen zwischen zwei Objekten return-anweisung eines Prozeduraufrufs Software Performance Engineering Software Performance Engineering 18 3

49 loop alt opt par Erweiterte Features für Sequenzdiagramme: Referenz Details in einem weiteren Sequenzdiagramm Iteration n-malige Wiederholung des eingeschlossenen Abschnitts Alternative Ausführung genau eines Abschnittes gemäß der Bedingungsklausel Optionale Abschnitte Ausführung des eingeschlossenen Abschnitts optional gemäß der Bedingungsklausel Parallele Abschnitte gleichzeitige Ausführung der eingeschlossenen Abschnitte Software Execution Models Software Performance Engineering Software Performance Engineering 20 Vom Sequenzdiagramm zum Ausführungsgraph Execution Graph Modell zur vereinfachten Aufnahme von Performance-Informationen Ähnlichkeit mit UML-Aktivitätsdiagrammen Umwandlung jedes Aufrufes im Sequenzdiagramm in einen Knoten Zusammenfassung verwandter bzw. Performance-unkritischer Aufrufe in einen einzigen Knoten Ergänzung von Ausführungshäufigkeiten / - wahrscheinlichkeiten Beispiel: Sequenzdiagramm -> Ausführungsgraph Software Performance Engineering Software Performance Engineering 22 Beispiel: Sequenzdiagramm -> Ausführungsgraph n Ausführungsgraphen Basisknoten Funktionale Komponente Erweiterter Knoten Verfeinerte Funktion in einem Subgraphen Wiederholungsknoten Schleife: n-malige Wiederholung einer Reihe von Knoten Entscheidungsknoten Ausführung geknüpft an Bedingung versehen mit Wahrscheinlichkeit Parallelknoten parallele Ausführung von Knoten Fortsetzung des Kontrollflusses erst nach Berechnung aller Knoten Spaltungsknoten parallele Ausführung von Knoten Fortsetzung des Kontrollflusses auch wenn nicht alle Knoten berechnet wurden Software Performance Engineering Software Performance Engineering 24 4

50 Ausführungsgraphen: Nicht erlaubte Konstrukte mehrere Anfangsknoten für einen Graphen Aufrufender Prozess: Ausführungsgraphen: Synchronisation Aufgerufener Prozess: Schleifen ohne Wiederholungsknoten Synchroner Aufruf: der Aufrufer wartet auf eine Antwort Verzögerter synchroner Aufruf: Weiterrechnen, danach Warten auf Antwort Asynchroner Aufruf: keine Antwort Antwort keine Antwort Software Performance Engineering Software Performance Engineering 26 Ausführungsgraphen: Synchronisation - Beispiel Eingabedaten für SPE 1. Performance-kritische Szenarios modelliert als Sequenzdiagramme, Ausführungsgraphen 2. Performance-Ziele konkrete Vorgaben, die zu erfüllen sind 3. Systemumgebung Hardware-Konfiguration, Betriebssystem, Middleware 4. Software Resource Requirements Berechnungsanforderung für jeden Rechenschritt in der Übung vorgegeben 5. Computer Resource Requirements Abbildung der Software Resource Requirements auf die Ausführungsumgebung Software Performance Engineering Software Performance Engineering 28 Software Resource Requirements Identifikation der Software Ressourcen Beispiel: Berechnungseinheiten, Datenbankzugriffe, Netzwerknachrichten Schätzung der Werte für die Software-Ressourcen best- und worst-case, später detaillierter Beispiel: Operation x braucht im best-case drei Datenbankzugriffe Angabe der CPU-Belastung Beispiel: WorkUnits über einer Skala von 1 (niedrig) bis 5 (hoch) Beispiel: Autorisierung einer Transaktion mittels einer Datenbank hier: Spezifikation der best-case Werte Software Resource Requirements Software Resource Type CPU usage SQL File I/O Messages Log-Files Einheit Work Units, Anzahl der Instruktionen oder Zeit Anzahl und Typ (read, write, update, ) Anzahl der logischen oder physischen I/Os Größe (z.b. in KByte) und Typ (LAN, Internet) Anzahl der Log-Events Middleware-calls Anzahl und Typ (connectionopen, queueget, requestsend, ) Calls to different processeses, threads or Anzahl, Typ und Ziel des Aufrufs processor Delay for remote processing Zeit eines Requests Software Performance Engineering Software Performance Engineering 30 5

51 Computer Resource Requirements Computer Resource Requirements Abbildung der Software Anforderungen auf Hardware- Elemente Typ, Anzahl und Einheit der relevanten Hardware- Elemente im System Spezifikation, wie hoch die Software Requirements auf den Computer Ressource Requirements sind Dauer einer Einheit für jede Ressource (z.b. durch Messungen) Computer Resource Type CPU Disks, I/O Benutzer Netzwerk Verzögerung (ohne Queue, z.b. Bildschirm, Tastatur ) Box (mit Queue, z.b. Drucker) Einheit Zeit, Anzahl der Instruktionen Anzahl der Operationen Zeit Anzahl der Nachrichten Zeit Zeit Software Performance Engineering Software Performance Engineering 32 Computer Resource Requirements: Berechnung der Antwortzeit Computer Resource Requirements: Berechnung der Antwortzeit Schritt 2: Berechnung der Computer Resource Requirements für alle Knoten eines Graphen Schritt 1: Berechnung der Computer Resource Requirements für einen Knoten (sendresult) Software Performance Engineering 33 Schritt 3: Berechnung der best-case Antwortzeit (ggf. Einbeziehung von Iterationen und Wahrscheinlichkeiten bei Auswahlknoten) (3110 * 0,00001) + (14 * 0,02) + (1 * 0,01) = 0,3211 Sekunden Software Performance Engineering 34 System Execution Models System Execution Models Software Execution Model: zum schnellen Feedback von Performance-Problemen in frühen Entwurfsphasen keine Berücksichtigung von anderen Arbeitslasten oder mehreren Benutzern Vernachlässigung der Verzögerungen, die bei konkurrierender Ressourcennutzung auftreten System Execution Model: Konstruktion erst nach Lösung aller Performance-Probleme im Software Execution Model Modellierung von konkurrierenden Ressourcenzugriffen mittels Queueing Networks zur Identifikation von Flaschenhälsen in der System-Architektur automatisierte Berechnung durch Tools Software Performance Engineering Software Performance Engineering 36 6

52 Queueing Networks Wartezeit Queue Performance Metriken: Verweilzeit Auslastung Durchsatz Queue-Länge Verweilzeit Server Bearbeitungszeit Queueing Networks Annahme: Job Flow Balance Rate der fertiggestellten Jobs (Durchsatz) = Ankunftsrate Beispiel: Vorgabe: Ankunftsrate T = 0,4 jobs/sec Mittlere Bearbeitungszeit S = 2 sec Berechnung: Durchsatz, X = T 0,4 jobs/sec (job flow balance) Auslastung, U = X * S 0,4/sec * 2 sec = 0,8 Verweilzeit, RT = S / (1-U) 2 sec / (1-0,8) = 10 sec Queue-Länge, N = X * RT 0,4/sec * 10 sec = 4 jobs (Little s Formel) Software Performance Engineering Software Performance Engineering 38 Queueing Networks offenes QN: Eingang/Ausgang in das Netzwerk Anzahl der Jobs variiert mit der Zeit geschlossenes QN: keine externen Ankünfte im Netzwerk Zirkulation fester Anzahl von Jobs Offenes Queueing Network: Berechnungen Vorgabe: T Ankunftsrate V i Anzahl der Besuche (Visits) eines Gerätes S i Mittlere Bearbeitungszeit (Service Time) eines Gerätes Berechnung: Systemdurchsatz X 0 : X 0 = T Durchsatz von Gerät i: X i = X 0 * V i Auslastung von Gerät i: U i = X i * S i Verweilzeit von Gerät i: RT i = S i / (1-U i ) Länge der Queue Gerät i: N i = X i * RT i Länge der Systemqueue: N = W N i Systemantwortzeit: RT = N / X Software Performance Engineering Software Performance Engineering 40 Offenes Queueing Network: Beispiel Systemankunftsrate: T = 5 jobs / sec Anzahl der Besuche V: CPU 5 Disk1 3 Disk2 1 Mittlere Bearbeitungszeit S: CPU 0,01 Disk1 0,03 Disk2 0,02 Metrik 1. Durchsatz X i = X 0 * V i 2. Bearbeitungszeit S i (Vorgabe) 3. Auslastung U i = X i * S i 4. Verweilzeit RT i = S i / (1-U i ) 5. Queue Länge N i = X i * RT i CPU 25 0,013 0,325 0,055 0,825 Gesamte Jobs im System = 0, , ,110 = 1,26 Systemantwortzeit: 1,26 / 5 = 0,252 sec 0,022 0, Software Performance Engineering 41 0,01 0,25 Disk1 15 0,03 0,45 Disk2 5 0,02 0,10 System Execution Model Eingaben für das Queueing Network Systemankunftsrate (ermittelt in einem Performance Walkthrough) Anzahl der Besuche an einem Gerät (berechnet mit dem Software Execution Model) Mittlere Bearbeitungszeiten (ebenfalls im Software Execution Model festgelegt) Berechnung der Ausgaben (Systemantwortzeit, Queue- Längen usw.) mit Tool Software Performance Engineering 42 7

53 Performance-Prinzipien Performance-orientiertes Design Software Performance Engineering Software Performance Engineering 44 Performance-Patterns Performance-Antipatterns Software Performance Engineering Software Performance Engineering 46 Beispiel Beispiel Web-Auftritt einer Fluggesellschaft Planung von Flugreisen im Web Ordern von Tickets Software Performance Engineering Software Performance Engineering 48 8

54 1. Eingrenzen des Performance-Risikos 2. Performance-kritische Use- Cases mittleres Performance-Risiko bei schlechter Performance: keine Benutzung der Website -> Umsatzeinbußen Software Performance Engineering Software Performance Engineering Modellierung von Szenarien 3. Modellierung von Szenarien planitinerary login Software Performance Engineering Software Performance Engineering 52 getflightplanningpage 3. Modellierung von Szenarien 4. Festlegung von Performance-Zielen maximale Login-Zeit: 8 Sekunden ansonsten zunächst keine Vorgaben Software Performance Engineering Software Performance Engineering 54 9

55 5. Konstruktion eines Performance-Modells Software Performance Engineering Software Resource Requirements Input: Größe der Eingabenachricht (KByte) DBAccess: Anzahl der Zugriffe auf die Mainframe- Datenbank LocalDB: Anzahl der Zugriffe auf das DotComDBMS Pagesz: Größe der Seite, die dem Benutzer gezeigt wird (KByte) Datasz: Größe der Daten, die aus dem Mainframe empfangen werden (KByte) Software Performance Engineering System Resource Requirements 8. Auswertung des Performance- Modells Software Performance Engineering Software Performance Engineering 58 Änderung des Entwurfs Änderung des Entwurfs Beispiele: kleinere Webseiten verschicken Verwendung von Tabellen anstatt von Frames um mehrfache Requests zu umgehen Möglichkeit zur Festlegung einer Startseite durch den User zur schnelleren Navigation Verwendung von weniger aufwendigen Graphiken Software Performance Engineering Software Performance Engineering 60 10

56 Fazit: Lernziele der Übung Verstehen des SPE-Prozesses Wissen, wie ein Sequenzdiagramm in einen Ausführungsgraphen transformiert wird Wissen, wie Software Resource Requirements festgelegt werden Wissen, wie Computer Resource Requirements festgelegt werden, und wie damit Performance- Werte berechnet werden können Software Performance Engineering 61 11

57 Inhalt Capacity Planning Menasce / Almeida Allgemeines Vorgehensmodell Workload-Modellierung Queueing Networks qualitative Aspekte quantitative Aspekte Beispiel Performance by Design 2 Allgemeines seit Anfang der 70er: Einsatz von Queueing Networks in der Performance-Analyse 1994: Capacity Planning and Performance Modeling: from mainframes to client-server systems 1998: Capacity Planning for Web Performance: metrics, models, and methods 2000: Scaling for E-Business: technologies, models, performance, and capacity planning 2002: Capacity Planning for Web Services 2004: Performance by Design Vorgehensmodell Performance by Design Performance by Design 4 Vorgehensmodell Beispiel: Performance- Modellierung eines einfachen Web-Servers Vorgehensmodell Verstehen der Systemarchitektur Welche Arten von Hardware / Software werden benutzt? Beispiel: ein Web-Server bestehend aus einer CPU und einer Festplatte Performance by Design Performance by Design 6 1

58 Vorgehensmodell Workload-Modellierung Ankunftsrate von Anfragen Einteilung der Systemlast in mehrere Klassen Beispiel: Beobachtung des Web- Server für eine Stunde Anfragen für HTML-Dateien (durchschnittliche Größe: 3000 Bytes) 1034 Anfragen für Image-Dateien (durchschnittliche Größe: Bytes) Vorgehensmodell Benchmarks Netzwerkgeschwindigkeit, Datenbankkapazität usw. Beispiel: CPUDemand = 0, ,002 * Größe der Anfrage konstanter Anteil für das Öffnen einer TCP- Verbindung, die Analyse des HTTP-Requests und das Öffnen der Datei variabler Anteil proportional zur Dateigröße, CPU ist an jeder I/O-Operation beteiligt durchschnittliche Bearbeitungszeit der Festplatte für einen Block von 1000 Bytes: 12 msec Performance by Design Performance by Design 8 Vorgehensmodell Erstellung eines Queueing Networks Beispiel: Modellierung als offenes QN Vorgehensmodell Berechnung aller Eingabeparameter Beispiel: Ankunftsraten: O HTML = / 3600 = 3,9 Anfragen / sec O Image = 1034 / 3600 = 0,29 Anfragen / sec Berechnung der benötigten Rechenzeiten D CPU,HTML = 0, ,002 * 3 = 0,014 sec D CPU,HTML = 0, ,002 * 15 = 0,038 sec D Disk,HTML = 3 * 0,012 = 0,036 sec D Disk,HTML = 15 * 0,012 = 0,18 sec Performance by Design Performance by Design 10 Vorgehensmodell Eingabe der Parameter in ein Spreadsheet zur automatischen Berechnung des QN Arrival Rate (req / sec) Service Demands (sec) CPU Disk Utilizations (%) CPU Disk Residence Times (sec) CPU Disk Response Times (sec) HTML 0,014 0,036 0,015 0,045 0,060 1,1 5,2 0,041 0,223 0, Performance by Design 11 3,90 5,5 14 Images 0,29 0,038 0,180 Vorgehensmodell Wurde das richtige Modell für das System erstellt? Verhält sich das Modell wirklich genauso wie das System? Performance by Design 12 2

59 Vorgehensmodell Statistische Methoden zur Ermittlung zukünftiger Workloads Beispiel: Erhöhung der Systemlast um das 5-fache Ankunftsraten: O HTML = 5 * 3,9 = 19,5 Anfragen / sec O Image = 5 * 0,29 = 1,45 Anfragen / sec Performance by Design 13 Vorgehensmodell erneute Berechnung des QN Response Time: HTML: 0,93 sec (das 15,5 fache) Images: 4,61 sec (das 17,5 fache) Utilization (disk): 96 % (vorher: 19,2 %) Erreichung der Maximalkapazität bei 5-facher Systemlast Performance by Design 14 Vorgehensmodell Vorgehensmodell Testen verschiedener Entwurfsalternativen jeweils Modifikation des QN Beispiel: Wie wirkt sich eine doppelt so schnelle CPU auf die Performance des Systems aus? in der Übung vorgegeben Performance by Design Performance by Design 16 Workload-Modellierung Workload-Modellierung Typen von Workloads Natürliche Modelle (Traces, Logs von Webservern) Künstliche Modelle Instruction Mixes Kernels Synthetic Programs Artificial Benchmarks Drivers ausführbare / nicht-ausführbare Modelle Performance by Design Performance by Design 18 3

60 Workload-Modellierung Systemlast gekennzeichnet durch Intensität Ankunftsrate (z.b. 5 Anfragen / sec) Anzahl der Clients und Think Times Anzahl der gleichzeitig ausgeführten Prozesse und Threads Service-Anforderungen (D i1, D i2,, D in ), wobei D ij die Anforderung der Last i an Ressource j ist (z.b. Anfrage i braucht 2 sec auf Ressource j) Workload-Modellierung Anfragen an ein System unterschiedliche Anforderungen bzgl. CPU- Zeit, I/O, Speicherverbrauch, Netzwerkbelastung z.b. Abfrage statischer HTML-Seite gegenüber Datenbanktransaktion Mittelung der Anforderungen über alle Anfragen -> ungenaue Modellbildung Performance by Design Performance by Design 20 Workload-Modellierung Einteilung aller Anfragen in Klassen z.b. einfache, mittlere, komplexe Anfragen Kriterien Ressourcenverbrauch (hoch, niedrig) Anwendungen (ftp, http) Dokumenttyp (html, jpg, avi) Herkunft (.de,.uk,.fr) bei ähnlichen Objekten: Mittelwertbildung bei unterschiedlichen Objekten: Cluster-Analyse Workload-Modellierung: Cluster-Analyse Darstellung der Anfragen eines Beispieldurchlaufs als Punkte in einem mehrdimensionalen Raum (z.b. bzgl. Anzahl der I/O Operationen und CPU-Zeit) ID CPU- Time I/O- Count Performance by Design Performance by Design 22 Workload-Modellierung: Cluster-Analyse Berechnung Euklidischer Abstands der Punkte Koordinaten der Mittelpunkte von Clustern k-means-algorithmus 1. Setze die Anzahl der Cluster auf k. 2. Wähle k Startpunkte als initiale Mittelpunkte. 3. Ordne jedem weiteren Punkt den jeweils nahsten Mittelpunkt zu und berechne dabei den Mittelpunkt neu. 4. Wiederhole 3. bis sich kein Mittelpunkt mehr ändert oder eine maximale Anzahl an Durchläufen erfolgt ist. Queueing-Networks Performance by Design Performance by Design 24 4

61 Queueing Networks: qualitative Aspekte offene/geschlossene Klassen Ressourcentypen Blocking Software Contention Gleichzeitige Ressourcennutzung Scheduling-Strategien Queueing Networks Sicht auf ein Computersystem als eine Ansammlung von zusammenhängenden Warteschlangen einfaches Modell, das die Berechnung von Performance-Werten (wie Antwortzeiten, Durchsatz, Auslastung ) zulässt nicht unbedingt genaue absolute Werte, sondern relativer Vergleich verschiedener Architekturalternativen Performance by Design Performance by Design 26 QN: offene/geschlossene Klassen Aufbau des Queueing Networks abhängig von der Klassifizierung der Systemlast mehrere Lastklassen offene, geschlossene, gemischte Systemlast Performance by Design 27 QN: offene Klassen Lastintensität abhängig von Ankunftsrate (z.b. 0,5 Transaktionen / Sekunde) Lastintensität hängt nicht von Anzahl der Benutzer im System ab unbegrenzte Anzahl von Benutzern Durchsatz = Ankunftsrate Performance by Design 28 QN: geschlossene Klassen QN: gemischte Klassen Lastintensität anhängig von der Benutzeranzahl im System (z.b. 7 Benutzer im System) begrenzte Anzahl von Benutzern Durchsatz ist ein Ausgabeparameter beinhaltet offene und geschlossene Klassen z.b. Online-Transaktionen (offen) und gleichzeitig Batch-Generierung von Managementberichten (geschlossen) Performance by Design Performance by Design 30 5

62 QN: Ressourcentypen Load independent Servicerate hängt nicht von der Systemlast ab Load dependent Servicerate variiert mit der Systemlast (z.b. ist eine Ressource bei größerer Systemlast stärker ausgelastet) Delay keine Warteschlange, sofortige Bearbeitung konstante Bearbeitungszeit Performance by Design 31 QN: Ressourcentypen Beispiel Client (Delay): entweder Warten auf Serverantwort oder Eingabe neuer Befehle keine Warteschlange, einfache Verzögerung LAN (Load dependent): Abnahme der effektiven Bandbreite bei Zunahme der Clientanfragen aufgrund von Paketkollisionen (Servicerate hängt von der Auslastung ab) CPU/Disk (Load independent): konstante Servicerate Performance by Design 32 QN: Blocking QN: Software Contention zur Garantierung von Antwortzeiten: Begrenzung der maximalen Anzahl von Transaktionen Durchsatz = Ankunftsrate * (1 Wahrscheinlichkeit der Ablehnung) Performance by Design 33 Software Contention: Konkurrenz um begrenzte Anzahl an Threads Physical Contention: Konkurrenz um begrenzte Anzahl an Ressourcen Performance by Design 34 QN: gleichzeitige Ressourcennutzung Datenbank-Lock führt zu gleichzeitiger Ressourcennutzung (hier: CPU und Disk) Wahrscheinlichkeit, dass ein Lock durch eine andere Transaktion gehalten wird, erhöht sich mit der Systemlast Performance by Design 35 QN: Scheduling Strategien First Come First Serve Priority Queue Round Robin Processor Sharing Shortest Job First Random Performance by Design 36 6

63 QN: formale Definition K: Anzahl der Queues R: Anzahl der Systemlastklassen Eingabeparameter: Lastintensität offene Klassen: O = (O 1,, O R ) (Ankunftsrate) geschlossene Klassen: N = (N 1, N R ) (Anfragen im System Rechenzeit D i,r (Queue i, Klasse r) Queueing Networks: quantitative Aspekte Grundlegende Performance Ergebnisse Utilization Law Service Demand Law Forced Flow Law Little s Law Interactive Response Time Law Performance-Grenzen Performance by Design Performance by Design 38 QN: Grundlagen QN: Grundlagen Beispiel: Eine CPU wird für eine Minute beobachtet. Während dieser Zeit finden 1800 Transaktionen statt und die CPU rechnet für 36 Sekunden. Alle Transaktionen werden innerhalb einer Minute fertig gestellt. Was ist die Performance des Systems? Mittlere Bearbeitungszeiten pro Transaktion Auslastung der Ressource Durchsatz des Systems T K B i A 0 C 0 Beschreibung Länge der Beobachtungzeit Anzahl der Ressourcen im System gesamte Rechenzeit der Ressource i in der Beobachtungszeit gesamte Anzahl der Anfragen an das System in der Beobachtungszeit gesamte Anzahl der fertiggestellten Anfragen an das System in der Beobachtungszeit Beispiel 60 sec 1 (die CPU) 36 sec 1800 Transaktionen 1800 Transaktionen (job-flow-balance) Performance by Design Performance by Design 40 QN: Grundlagen S i (= Service Time) mittlere Bearbeitungszeit pro Anfrage an Ressource i S i = B i / C i Beispiel: S 1 = 36 / 1800 = 1 / 50 sec pro Transaktion U i (= Utilization) Auslastung der Ressource i U i = B i / T Beispiel: U 1 = 36 / 60 = 60 % Performance by Design 41 QN: Grundlagen O i (= Arrival Rate) Ankunftsrate pro Zeiteinheit an Ressource i O i = A i / T Beispiel: O 1 = 1800 / 60 = 30 tps X 0 (= Throughput) Systemdurchsatz X 0 =C 0 / T Beispiel: X 0 = 1800 / 60 = 30 tps Performance by Design 42 7

64 QN: Utilization Law U i = S i * X i = S i * O i Beispiel: Die Bandbreite eines Modems beträgt bps, es werden pro Sekunde 3 Pakete (1500 Byte) übertragen. Wie hoch ist die Auslastung? Mittlere Übertragungszeit S i = (1500 * 8) / = 0,214 sec / Paket Auslastung U i = 0,214 * 3 = 0,642 = 64,2 % QN: Service Demand Law D i = U i / X 0 = V i * S i Beispiel: Ein Web-Server wird für 10 Minuten beobachtet und ist zu 90% ausgelastet. Laut Web-Server-Log wurden in dieser Zeit Anfragen verarbeitet. Wie hoch ist der CPU- Bedarf einer Anfrage? Durchsatz des Servers X 0 = / 600 = 50 Anfragen / sec CPU-Bedarf D i = 0,9 / 50 = 0,018 sec / Anfrage Performance by Design Performance by Design 44 QN: Forced Flow Law X i = V i * X 0 V i : Anzahl der Besuche einer Anfrage auf einer Ressource Beispiel: Während einer einstündigen Beobachtung eines Datenbank-Servers werden 7200 Transaktionen durchgeführt, die jeweils 4,5 I/O Operationen brauchen. Wie hoch ist der Durchsatz der Festplatte? Durchsatz des Servers X 0 = 7200 / 3600 = 2 tps Durchsatz der Festplatte X i = 4,5 * 2 = 9 tps Performance by Design 45 QN: Little s Law N = R * X N: Länge der Warteschlange Beispiel: Ein NFS-Server wurde 30 Minuten lang beobachtet und es wurden I/O Anfragen durchgeführt. Bei durchschnittlich 9 offenen Anfragen (N req ), wie hoch war die Antwortzeit für eine Anfrage? X Server = / 1800 = 18 Anfragen / sec R req = N req / X Server = 9 / 18 = 0,5 sec Performance by Design 46 QN: Interactive Response Time Law R = (M / X 0 ) -Z M: Anzahl der Clients, die eine Ressource gleichzeitig bearbeiten Z: think time der Clients Beispiel: Wenn 7200 Anfragen von 40 Clients während einer Stunde bearbeitet werden und die durchschnittliche think time 15 Sekunden beträgt, wie hoch ist die Antwortzeit? R = (40 / (7200/3600)) 15 = 5 sec Beispiel: Datenbank-Server Performance by Design Performance by Design 48 8

65 Beispiel Analyse der Systemlast Analyse der Systemlast Cluster-Einteilung der Systemlast Erstellen eines Performance-Modells Berechnung der Performance-Daten Testen von Entwurfsalternativen Hinzufügen einer dritten Festplatte Nutzung eines Mehrprozessorsystems Nutzung schnellerer Festplatten CPU 116,824 64,383 35, , ,793 47,956 72,453 33,800 70,900 86,329 39,710 76,018 59,795 58,789 76,889 Disk Disk TR ID Log vom DBMS Performance Monitor 200 Transaktionen innerhalb von 150 Sekunden Durchsatz X 0 = 200 / 150 = 1,33 tps Performance by Design Performance by Design 50 Analyse der Systemlast Analyse der Systemlast Messung der Auslastung der Ressourcen mit OS Performance Monitor Ressource CPU Disk 1 Disk 2 Auslastung (%) Mean Standard Deviation Sample Variance Coeff. of Variation Minimum First Quartile (Q1) Median (Q2) Third Quartile (Q3) Maximum Range Largest Smallest Sum CPU Time (msec) 238,2 165, ,4 0,696 23,6 104,4 151,6 418,1 507,5 483,9 507,5 23, ,8 No. I/Os Disk 1 51,38 27,0 728,7 0, No. I/Os Disk 2 44,85 26,4 698,1 0, Berechnungen z.b. mit Excel Coefficient of Variation CV = Standard Deviation / Mean relatives Maß der Variabilität Faustregel: Wenn CV < 0,25 gehören die Anfragen in eine Lastklasse Performance by Design Performance by Design 52 Analyse der Systemlast Erstellung eines Performance Modells Number of IOs CPU Time (msec) Cluster I/Os on I/Os on No. of Number CPU Time disk 1 disk 2 points 1 67,5 8,0 11, ,2 62,4 73, ,1 69,8 36, Performance by Design Performance by Design 54 9

66 Erstellen eines Performance Modells Gegeben: Ankunftsrate: 1,33 tps (vom DBMS Performance Monitor) Auslastung (vom OS Performance Monitor) Durchsatz (= Ankunftsrate) Einteilung der Systemlast in drei Klassen Gesucht für das QN: Rechenzeiten für jede Klasse (Service Demands) Erstellen eines Performance Modells Service Demand Law D i,r = U i,r / X 0 (i ist die Ressource, r die Lastklasse) Berechnung der Auslastung für die einzelnen Lastklassen: U i,r = U i * f i,r f i,r ist ein Faktor, der von Ressourcentyp und Lastklasse abhängt Performance by Design Performance by Design 56 Erstellen eines Performance Modells Faktoren für die CPU f CPU,r = Gesamte CPU-Zeit für Klasse r / Gesamte CPU Zeit für alle Klassen f CPU,1 = (67,5 * 50) / 47640,8 = 0,071 f CPU,2 = (434,2 * 80) / 47640,8 = 0,729 f CPU,3 = (136,1 * 70) / 47640,8 = 0,200 Erstellen eines Performance Modells Faktoren für Disk1 f disk i,r = Gesamte Anzahl an I/Os für Klasse r / Gesamte Anzahl an I/Os für alle Klassen f disk 1,1 = (8,0 * 50) / = 0,039 f disk 1,2 = (62,4 * 80) / = 0,486 f disk 1,3 = (69,8 * 70) / = 0, Performance by Design Performance by Design 58 Erstellen eines Performance Modells Faktoren für Disk2 f disk i,r = Gesamte Anzahl an I/Os für Klasse r / Gesamte Anzahl an I/Os für alle Klassen f disk 2,1 = (11,0 * 50) / 8969 = 0,061 f disk 2,2 = (73,1 * 80) / 8969 = 0,652 f disk 2,3 = (36,7 * 70) / 8969 = 0,287 Erstellen eines Performance Modells Auslastung einer Lastklasse: U i,r = U i * f i,r U CPU,1 = U CPU * f CPU,1 = 0,45 * 0,071 = 0,032 U CPU,2 = U CPU * f CPU,2 = 0,45 * 0,729 = 0,328 U CPU,3 = U CPU * f CPU,3 = 0,45 * 0,200 = 0,090 analog für Disk1 und Disk Performance by Design Performance by Design 60 10

67 Erstellen eines Performance Modells Erstellen eines Performance Modells Auslastungen für alle Ressourcen und Klassen (U i,r ) Rechenzeiten (in Sekunden) D i,r = U i,r / X 0 CPU Disk 1 Disk 2 CPU Disk 1 Disk 2 Class 1 Class 2 Class 3 Total 0,032 0,328 0,090 0,45 0,029 0,365 0,356 0,75 0,040 0,424 0,187 0,65 Class 1 Class 2 Class 3 0,096 0,615 0,193 0,088 0,683 0,763 0,119 0,795 0,400 alle Eingabeparameter vorhanden Eingabe in Spreadsheet Berechnung der Antwortzeiten für das System CPU Disk 1 Disk 2 Response Time Class 1 0, , , ,86513 Disk1 ist Bottleneck Class 2 1, , , ,12246 Class 3 0, , , , Performance by Design Performance by Design 62 Testen von Entwurfsalternativen Vorhersage der Systemlast-Entwicklung: Arrival Month Rate (tps) 1 January 1,33 2 February 1,45 3 March 1,68 4 April 2,26 5 May 2,68 6 June 3,25 7 July 3,98 8 August 4,78 9 September 5,74 10 October 6,76 Testen von Entwurfsalternativen Hinzufügen einer dritten Festplatte Load Balancing D neu i,r = (D alt 1,r + D alt 2,r ) / 3 für i = 1,2,3 Service Demand CPU Disk 1 Disk 2 Disk 3 Class 1 0,096 0,069 0,069 0,069 Class 2 Class 3 0,615 0,193 0,493 0,388 0,493 0,388 0,493 0, Performance by Design Performance by Design 64 Testen von Entwurfsalternativen Weitere Alternativen neue Antwortzeiten: Arrival Rate (tps) R1 R2 R3 1,33 0,56 3,89 2,53 1,45 0,61 4,21 2,75 1,68 0,73 5,02 3,28 2,26 1,39 9,65 6,37 2,68 4,34 30,28 20, Performance by Design Performance by Design 66 11

68 Fazit: Lernziele der Übung Verstehen wie das Vorgehensmodell aufgebaut ist Wissen, wie man Workloads in Klassen einteilt Wissen, wie man die Eingabeparameter für ein Queueing Network berechnet Performance by Design 67 12

69 Inhalt Simulation von UML-Modellen Balsamo/Marzolla Allgemeines Vorgehensmodell Attributierung von UML-Modellen Simulationen von UML-Modellen mit umlpsi Ergebnisse der Simulation Beispiel umlpsi 2 Allgemeines Ziel: nahe Integration von Performance- Analysen in den Entwicklungsprozess Verwendung von Standard-Notationen zur Modellierung UML (1997) UML Performance Profile (2002) umlpsi-tool Thema einer Dissertation Februar 2004 Attributierung von UML-Diagrammen mit anschließender automatischer Simulation Vorgehensmodell umlpsi umlpsi 4 Vorgehensmodell Vorgehensmodell 1. Spezifikation der Architektur mit UML 2. Parametrisierung mit UML Performance Profile Beispiel: Modellierung eines Videostreams im Internet 3. Konfiguration der Simulation 4. Automatische Simulation mit umlpsi 5. Feedback der Ergebnisse in CASE-Tool 6. Überarbeitung der Architektur umlpsi umlpsi 6 1

70 Vorgehensmodell Vorgehensmodell umlpsi umlpsi 8 Vorgehensmodell Vorgehensmodell Festlegung der Simulationszeit genauere Ergebnisse bei längerer Simulation z.b ms Festlegung von Konfidenzintervallen Einschränkung für die Ergebnisse der Simulation z.b. 10% umlpsi umlpsi 10 Vorgehensmodell Vorgehensmodell XMI-Datei XMI-Datei mit Ergebnissen umlpsi umlpsi 12 2

71 Vorgehensmodell Vorgehensmodell Ergebnisse: Auslastungen von Ressourcen Durchsatz von Ressourcen Mittlere Ausführungszeiten für Aktionen und Use-Cases spezielle Attribute zur Speicherung der Ergebnisse im UML-Modell umlpsi umlpsi 14 Vorgehensmodell Vorgehensmodell Ergebnisse: Internet-Ressource am stärksten belastet (~76%) Testen verschiedener Entwurfsalternativen umlpsi umlpsi 16 Attributierung von UML- Diagrammen UML Performance Profile umlpsi 17 Erweiterungsmechanismen für UML Stereotypes Definition von Subklassen für existierenden Metamodell- Elemente Gleiche Struktur wie Original, Erweiterungen graphische Repräsentierung: <<stereotype>> Tagged-Values Attribute für Modell-Elemente Beispiel: PAhost = Workstation Constraints Einschränkungen für ein Attribut Beispiel: PArespTime < 1.5 Profile Paket mit speziell für eine Domäne angepassten Modell- Elementen Definition von Stereotypes, Tagged Values und Constraints umlpsi 18 3

72 UML Performance Profile UML Profile for Schedulability, Performance and Time specification Standardisierung März 2002 durch die OMG Einheitliche Schnittstelle zwischen Software-Modell (in UML) und Performance-Modell Möglichkeit zur Verwendung unterschiedlichster Performance Modelle wie Queueing Networks, Petri- Netze, Simulationsmodelle Vorbereitung von UML-Modellen zur toolgestützten Auswertung UML Performance Profile Möglichkeit zur: Aufnahme von Performance-Anforderungen in den Entwurfskontext Assoziation von QoS-Charakteristiken mit ausgewählten Elemente eines UML-Modells Spezifikation von Ausführungsparametern zur Nutzung durch Modellierungstools für die Berechnung von Performance-Charakteristiken Präsentation von Performance-Ergebnissen, die durch Tools ermittelt wurden umlpsi umlpsi 20 UML Performance Profile Domain Viewpoint umlpsi-ansatz Modifikationen des UML Performance Profiles zur leichteren Erstellung der Simulation Erweiterungen: Composite Pattern für Actions Zuweisungen von Wahrscheinlichkeiten an Transitionen Join/Fork-Actions zur Modellierung von nebenläufigen Threads Modellierung von Workloads mit Use-Case Diagrammen umlpsi umlpsi 22 Performance-Modell Abbildung auf UML- Erweiterungen Performance-Modell UML Extensions Klasse Stereotype Attribut Tagged Values umlpsi umlpsi 24 4

73 Modellierung von Workloads: offene Systemlastklassen Modellierung von Workloads: geschlossene Systemlastklassen PAoccurrence PAprob Ankunftsrate für Anfragen an das System Wahrscheinlichkeit, dass eine Transition genommen wird PApopulation PAextDelay PAprob Anzahl der Anfragen im System externe Verzögerung zwischen zwei Anfragen (think time) Wahrscheinlichkeit, dass eine Transition genommen wird umlpsi umlpsi 26 Modellierung von Szenarien: einfache Zustände Modellierung von Szenarien: Akquirierung von passiven Ressourcen Aquire/Release <<GRMaquire>> / <<GRMrelease>> PAresource = Memory PAquantity = [ assm, dist, [ constant, 2]] PArep PAinterval PAdemand PAhost PAdelay PArespTime Anzahl der Wiederholungen Intervallzeit zwischen zwei Wiederholungen Service Demand: die benötigte Rechenzeit für diesen Schritt Ressource, auf der dieser Schritt ausgeführt wird (diese muss im Deployment-Diagramm enthalten sein) Verzögerung (z.b. für eine Benutzerinteraktion) Antwortzeit für diesen Schritt (berechnet umlpsi) PAresource PAquantity Ressource, die besetzt oder freigegeben wird Anzahl der Ressourcen, die besetzt oder freigegeben werden umlpsi umlpsi 28 Modellierung von Ressourcen: Aktive Ressourcen Modellierung von Ressourcen: Passive Ressourcen <<PAhost>> Workstation PAschedPolicy = FIFO PActxSwT = [ constant, 0,1] PArate = 2.0 PAutilization PAthroughput PAschedPolicy PActxSwT PArate PAutilization PAthroughput Scheduling-Strategie: (FIFO, LIFO, PS(=Processor Sharing)) Zeit für eine Kontextwechsel (nur falls Scheduling-Strategie PS) Rechengeschwindigkeit im Verhältnis zum Referenz-Prozessor (benötigt 1 sec Rechenzeit für 1 sec Service Demand) Auslastung der Ressource (berechnet umlpsi) Durchsatz der Ressource (berechnet umlpsi) PAcapacity PAaxTime PAutilization PAthroughput maximale Anzahl von Instanzen physikalische Zugriffszeit, Zeit zwischen Besetzung der Ressource und ihrer Einsatzfähigkeit (Freigabe verbraucht keine Zeit) Auslastung der Ressource (berechnet umlpsi) Durchsatz der Ressource (berechnet umlpsi) umlpsi umlpsi 30 5

74 Performance Context Tagged Values (1/3) Konfiguration der Simulation im Root- Knoten des Modells Tag paramfilename simduration confrelwidth Type String Real Real Multiplicity [0..1] [0..1] [0..1] Domain Attribute Name PerformanceContext:parameters PerformanceContext:simulation duration PerformanceContext:confidence width simduration paramfilename confrelwidth Maximale Länge der Simulation in ms Name einer externen Parameter-Datei (z.b. parameter.pl ) relative width of confidence intervals (default: 90% confidence level) PAprob PAprob PAoccurrence PApopulation Real Real RTarrivalPattern Integer 1 1 [0..1] [0..1] Association:probability Transition:probability OpenWorkload:occurrence ClosedWorkload:population PAextDelay PAperfValue [0..1] ClosedWorkload:extDelay umlpsi umlpsi 32 Tagged Values (2/3) Tagged Values (3/3) Tag PAutilization Type mean [0..1] Domain Attribute Name Resource::utilization Tag PArep Type Integer Multiplicity Multiplicity [0..1] Domain Attribute Name ActionBase::repetitions PAthroughput mean [0..1] Resource::throughput PAinterval PAperfValue [0..1] ActionBase::interval PAschedPolicy PActxSwT PArate PAcapacity PAaxTime Enumeration { LIFO, FIFO, PS } PAperfValue Real Real PAperfValue [0..1] [0..1] [0..1] [0..1] [0..1] ActiveResource::schedPolicy ActiveResource:ctxSwTime ActiveResource::rate PassiveResource::capacity PassiveResource::accessTime PAdemand PAhost PAdelay PArespTime PAresource PAquantity PAperfValue String PAperfValue mean String PAperfValue [0..1] [0..1] [0..1] [0..1] 1 [0..1] SimpleAction::demand SimpleAction::host SimpleAction::delay ActionBase::responseTime ResActionBase::resource ResActionBase::quantity umlpsi umlpsi 34 PAperfValue: Tagged Value Types < PAperfValue > := [ < SourceModifier >, dist, < PDFstring > ] < SourceModifier > := assm pred msrd < PDFstring > := [ < constantpdf > < uniformpdf > < exponentialpdf > < normalpdf >] < constantpdf > := < real > constant, < real > < uniformpdf > := uniform, < real >, < real > < exponentialpdf >:= exponential, < real > < normalpdf > := normal, < real >, < real > Tagged Value Types PAperfValue: konstanter Wert: < constantpdf >:=< real > constant, < real > Exponentialverteilung mit Mittelwert: < exponentialpdf >:= exponential, < mean > Gleichverteilung über Intervall [a,b]: < uniformpdf >:= uniform, < a >, < b > Normalverteilung mit Mittelwert und Standardabweichung: < normalpdf >:= normal, < mean >, < stddev > umlpsi umlpsi 36 6

75 Tagged Value Types PAperfValue Beispiele: PAdemand = [ assm, dist, [ exponential, 2.5]] (angenommene Exponentialverteilung um den Mittelwert 2,5 Sekunden) PActxSwT = [ msrd, dist, [ constant, 0.1]] (gemessener konstanter Wert von 0.1 Sekunden) PAdelay = [ pred, dist, [ uniform, 0.2, 0.5]] (vorhergesagter, über dem Intervall 0,2-0,5 Sekunden gleichverteilter Wert) Tagged Value Types RTarrivalPattern: < RTarrivalPattern > := [< bounded > < unbounded > < bursty >] < bounded > := bounded, < real >, < real > < bursty > := bursty, < PDFstring >, < integer > < unbounded > := unbounded, < PDFstring > Bounded: Zeit zwischen zwei aufeinanderfolgenden Anfragen ist gleichverteilt über einem Intervall 1. Parameter: obere Grenze 2. Parameter: untere Grenze umlpsi umlpsi 38 Tagged Value Types RTarrivalPattern: < RTarrivalPattern > := [< bounded > < unbounded > < bursty >] < bounded > := bounded, < real >, < real > < bursty > := bursty, < PDFstring >, < integer > < unbounded > := unbounded, < PDFstring > Bursty: Anfragen kommen in Schüben an. 1. Parameter: mittlere Zeit zwischen zwei Schüben 2. Parameter: maximale Größe eines Schubs Berechnung der eigentlich Schubgröße als eine gleichverteilte Zufallsvariable über dem Intervall [0 maximale Schubgröße] Unbounded: Zeit zwischen zwei aufeinanderfolgenden Anfragen mit entsprechender Verteilungsfunktion 1. Parameter: Zeit zwischen zwei Anfragen umlpsi 39 Tagged Value Types RTarrivalPattern Beispiele: PAoccurrence = [ unbounded, [ exponential, 0.15]] Intervall zwischen zwei Anfragen 0,15 Sekunden umlpsi 40 Simulation von UML-Modellen umlpsi Vergleich von Performance- Modellen analytisch Menge von Formeln oder Algorithmen Berechnung von Performance-Werten als eine Funktion der Eingabeparameter begrenzter Detaillierungsgrad aufgrund der mathematischen Komplexität effiziente Ausführung einfache, billige Konstruktion simulationsbasiert Emulation von dynamischen Aspekten eines Systems durch Computerprogramme beliebiger Detaillierungsgrad meist hoher Rechenaufwand für die Ausführung Ausgabe: statistische Werte (mehrere Ausführungen notwendig) umlpsi umlpsi 42 7

76 umlpsi Eingabe: XMI-Datei (momentan nur Parser für von Poseidon 1.4 bzw. ArgoUML generierten XMI-Dateien) Parsen der UML-Modelle in eine Zwischendarstellung automatische Transformation in eine prozessorientierte Simulation in C++ erlaubt z.b. fork/join-operationen, gleichzeitige Ressourcennutzung, beliebige Scheduling-Strategien Ausführung der Simulation mit libcppsim Ausgabe: Simulationsergebnisse (Durchsatz, Auslastungen, Antwortzeiten) XMI-Datei mit den in der Simulation ermittelten Performance- Werten umlpsi 43 Transformation des UML- Modells mit umlpsi Für jedes Use-Case Diagramm U erstelle einen Simulationsprozess des Typs Open_Workload oder Closed_Workload, abhängig vom Stereotype von U Für jede Action A erstelle einen Simulationsprozess des Typs Simple_Action, Aquire_Action, Release_Action, abhängig vom Stereotyp von A verbinde die Actions mit den gleichen Vorgänger-Nachfolger Beziehungen wie im UML-Modell Für jeden Knoten N im Deployment Diagramm erstelle einen Simulationsprozess vom Typ Active_Resource, abhängig vom Stereotype von N umlpsi 44 Ergebnisse der Simulation Tags für die Aufnahme von Performance-Ergebnissen Integration von Tags für die Ergebnisse in das Ausgangsmodell PArespTime für Actions PAthroughput, PAutilization für Node umlpsi überschreibt die Werte der Tags mit den Simulationsergebnissen umlpsi umlpsi 46 Ergebnisse Ergebnisse umlpsi umlpsi 48 8

77 NICE Beispiel Naval Integrated Communication Environment (NICE) Sprach-, Daten- und Videokommunikation umlpsi umlpsi 50 Use-Cases Szenarien neue Parameter- Einstellung für eine oder mehrere Equipments Performance-Vorgabe: mittlere Ausführungszeit < 5 sec ± 1sec Systemwiederherstellung nach dem Ausfall eines Equipments Performance-Vorgabe: mittlere Ausführungszeit < 5 sec ± 1sec Configuration Scenario Recovery Scenario umlpsi umlpsi 52 Mittlere Ausführungszeiten Aktivitätsdiagramme Configuration Scenario: Recovery Scenario: Configuration Scenario Recovery Scenario umlpsi umlpsi 54 9

78 Ergebnisse der Simulation N: Anzahl der Equipments Fazit: Lernziele Verstehen, wie das Vorgehensmodell aufgebaut ist Wissen, wie man UML-Diagramme mit Performance-Attributen versieht Verstehen des Unterschiedes zwischen einem analytischen und simulationsbasierten Verfahren umlpsi umlpsi 56 10

79 C. Übungszettel Die folgenden Übungszettel wurden jeweils nach der Vorstellung eines Verfahrens ausgegeben, damit die Teilnehmer die Verfahren auch praktisch anwendeten und erste Erfahrungen mit den Werkzeugen sammelten. Die Studenten hatten jeweils eine Woche Zeit für die Bearbeitung der Aufgaben. xli

80 Übungsblatt 1 Software Performance Engineering Tragen Sie folgende Werte ein: Hinweise: Für diese Übung wird ein Windows-Rechner benötigt. Das SPE-ED-Tool ist für andere Betriebssysteme leider nicht verfügbar. Abgabe der Lösung per Mail (s.u.) bis zum Die Übung soll als Einzelperson und nicht in Gruppen bearbeitet werden. Aufgabe 1 (Einarbeitung) Installieren Sie das SPE-ED-Tool (kann von Stud-IP heruntergeladen werden). Arbeiten Sie die Quick-Start-Anleitung gründlich durch. Aufgabe 2 (Beispielarchitektur) Das folgende Sequenzdiagramm modelliert die Recherche in einem Bibliotheksverzeichnis. Der Benutzer kann Suchanfragen stellen, die an das lokale DBMS weitergeleitet werden. Alternativ kann über das Internet die Zentraldatenbank eines Verbundes mehrerer anderer Bibliotheken abgefragt werden. Der Web-Server und das lokale DBMS werden gemeinsam auf einem Rechner mit einer CPU und einer Festplatten ausgeführt. Legen Sie in SPE-ED ein neues Projekt an, das Sie nach ihrem eigenen Namen benennen ( Markus Mustermann ). Erstellen Sie darin ein Szenario booksearch. Erstellen Sie für das Sequenzdiagramm einen Execution-Graphen (Synchronisation kann zunächst vernachlässigt werden). Beachten Sie, dass in der Demo-Version von SPE-ED nur maximal 4 Subgraphen erstellt werden können. Gehen Sie davon aus, dass die Suche nach einem Buchtitel 10 mal wiederholt wird (Schleife im Sequenzdiagramm), bevor der Benutzer viewdetails aufruft. Bestimmen Sie die Software Resource Requirements. Wählen Sie dazu den Menüpunkt (Software Model / Assign Template Spec) aus und weisen Sie aus den Beispiel-Projekten (Button Show All ) die Vorlage Distr sys spec zu. Tragen Sie nun für jeden Schritt im Execution Graph die jeweiligen geschätzten Werte ein (Software Model / Specify values). Versuchen Sie möglichst realistische Werte anzugeben und modifizieren Sie dazu diese Werte ggf. nachträglich. Weisen Sie die Computer Resource Requirements zu. Wählen Sie dazu den Menüpunkt (Overhead / Facility template) aus und weisen Sie aus den Beispiel-Projekten (Button Show All ) die Facility WebServer zu. Berechnen Sie die Antwortzeiten des Szenarios (No Contention-Solution). Spezifizieren Sie den Service-Level der Architektur (Software Execution Model / Service level ). Geben Sie als Performance-Ziel 120 Sekunden an und probieren Sie verschiedene Ankunftsraten zwischen 0-1 aus. Stellen Sie fest, bis zu welcher Ankunftsrate das Performance-Ziel eingehalten werden kann (Contention-Solution). In den Web-Server werden eine Dual-CPU und eine zweite (identische) Festplatte eingebaut. Wie verändert sich die Antwortzeit des Szenarios, wenn Sie die in der vorherigen Aufgabe ermitteltete maximale Ankunftsrate beibehalten? Machen Sie Vorschläge zur Überarbeitung der Architektur. Zur Abgabe: Schreiben Sie eine mit dem Betreff SPE-Übung an heiko.koziolek@informatik.uni-oldenburg.de Geben Sie im Body der an: die Antwortzeit des Szenarios (No Contention-Solution), die ermittelte maximale Ankunftsrate bei einem Performance-Ziel von 120 Sekunden, die Antwortzeit des aufgerüsteten Web-Servers bei der vorherigen maximalen Ankunftsrate und ihre Verbesserungsvorschläge für die Architektur. Fügen Sie als Anhang die vier Dateien (spedb0, spedb1, spedb2, spedb3) hinzu, nachdem Sie Szenario und Projekt gespeichert haben. (In der Demo-Version von SPE-ED ist die Exportfunktionalität deaktiviert, so dass die ganze SPE-ED- Datenbank geschickt werden muss.)