Modellbasiertes Performance-Testen modernisierter Software

Transkript

1 Modellbasiertes Performance-Testen modernisierter Software Wilhelm Hasselbring (Christian-Albrechts-Universität zu Kiel), Andre van Hoorn (Universität Stuttgart), Eike-Christian Schulz (Christian-Albrechts-Universität zu Kiel) , KoSSE-Tag, Lübeck W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

2 Agenda Project Context Project Context 2 Dynamic Analysis 3 Model-Based Testing 4 Zusammenfassung und Ausblick W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

3 KoSSE Project DynaMod [vhfg+ ] Project Context Dataport, Altenholz Software Engineering Group University of Kiel, Kiel b+m Informatik AG, Melsdorf HSH Nordbank AG, Kiel W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

4 Work Packages in DynaMod Project Context W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

5 Kieker DynamicMonitoring Analysis Framework: Modular, flexible, and extensible architecture (Probes, records, readers, writers, filters etc.) Pipes-and-filters framework for analysis configuration Distributed tracing (logging, reconstruction, visualization) Low overhead (designed for continuous operation) Evaluated in lab and industrial case studies Kieker.Monitoring Periodic Sampling JMX Interface Monitoring Monitoring Monitoring Probe Probe Probe Logging Monitoring Controller Logging Monitoring Writer Time Source Monitoring Probes/Samplers Time Sources Monitoring Writers Control-flow tracing Resource monitoring System time <your time source> Asynchr. writers Syncr. Manual instrumentation AspectJ Servlet Database (SQL) Named pipe Spring CXF/SOAP <your interception technology> Servlet Sigar CPU utilization Memory usage <your technology> <your monitoring probe> File system Java Messaging Service (JMS) Java Management Ext. (JMX) Database (SQL) File system <your monitoring writer> Output stream Kieker is open-source software (Apache License, V. 2.0) [vhwh2, vhrh + 09, RvHM + 08] Recommended Tool of the SPEC Research Group Kieker is distributed as part of SPEC RG s repository of peer-reviewed tools for quantitative system evaluation and analysis, Monitoring Log/Stream Monitoring Record Serialization Deserialization Monitoring Monitoring Monitoring Reader Reader Reader Analysis Controller Analysis / Analysis // Visualization Visualization Plugin Plugin Kieker.Analysis Monitoring Records Monitoring Readers Analysis/Visualization Plugins Operation execution Control-flow events CPU utilization Resource utilization Current time <your monitoring record type> File system Java Messaging Service (JMS) Java Management Ext. (JMX) Database (SQL) Named pipe <your monitoring reader> Pipe-and-filter framework Architecture reconstr. Trace analysis Visualization Memory/swap usage Real-time replayer Dependency graphs Sequence diagrams Call graphs <your visualization> <your trace analysis> <your reconstruction plugin> <your analysis plugin/tool> W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

6 Reverse Engineering of AIDA-SH [vhkgh] Dynamic Analysis frmlogin.cmdok_click() frmlogin.anwendung() frmmain.mdiform_load() Module.Main() frmlogin.form_load() frmakte.tool_menu_begin() frmlogin.txtpassword_gotfocus() frmakte.ob_frei() $ frmmain.mnudatafrmakte_click() frmakte.form_resize() frmakte.form_load() frmakte.ob_sperren() frmakte.felder_sperren() frmakte.option_click(integer) clsklasse.class_initialize() clsklasse.pruef_user() clsklasse.pruef_abt() clsklasse.pruef_rechte() BENUTZER.select() ABTEILUNG.select() BENABT.select() frmakte.felder_frei() frmakte.setbuttons(boolean) AKTE.select() clsklasse.adoprimaryrs_movecomplete(adodb.eventreasonenum, ADODB.Error, ADODB.EventStatusEnum, ADODB.Recordset) frmakte.cmdedit_click() frmakte.suchen() frmakte.felder_sichern() frmakte.indexposlesen() 8 clsklasse.class_getdatamember(string, Object) IPOS.select() frmstat frmakte.pruefung_ob() frmakte.fuehrende_nullen(long) frmakte.txtfields_change(integer) ms frmakte.daten_binden() 6 Statistik 6.7 ms ms frmakte.tool_menu_edit() 276 Ausgabe cmd_start_click frmakte.abteilung() frmakte.rtxtboxtitel_change() ms ms frmakte.rtxtboxenthaelt_change() Form_Load txt_abt_gotfocus MouseWheel mnufilestart_click Zeile 2 mnuhelpabout_click MSHFlexGrid_RowColChange mnuhelpsearchforhelpon_click 3.29 ms ms Form_KeyDown cmdclose_click mnuhelpcontents_click mnufileexit_click cmd_druck_click W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

7 Agenda Model-Based Testing Project Context 2 Dynamic Analysis 3 Model-Based Testing 4 Zusammenfassung und Ausblick W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

8 Dimensions of Testing [UL07] Model-Based Testing Workload generation is one of the main challenges in performance testing [AATP2]. W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

9 Markov4JMeter [vhrh08] Model-Based Testing Markov4JMeter Add-on to the performance testing tool JMeter ( Employs an analytical-based approach to workload generation Open Source: Enhancements to the so-called ON/OFF model [BC98] Definition of probabilistic user behavior behavior models (customer behavior model graphs [MAFM99]) Transition guards (extended finite state machines [KRM06]) Relative frequencies for different user types behavior mix Varying workload intensity W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

10 Customer Behavior Model Graphs [MAFM99] Model-Based Testing Example customer behavior model graph W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

11 User Behavior Mix Model-Based Testing User Behavior Mix for application A Probability of occurrence for each user behavior model B A,i {(B A,0, p 0 ),..., (B A,n, p n )} with relative frequencies p i Let A be an application with 2 user behavior models: Behavior 0 Behavior Example for a behavior mix: {(Behavior0, 0.3), (Behavior, 0.7)} W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

12 Markov4JMeter Test Plan Model-Based Testing W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

13 Markov4JMeter Test Plan Model-Based Testing W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

14 Model-Based Testing in DynaMod 0.0 Login.0 Verzeichnisgut aufnehmen.02 Verzeichnisgut aendern.03 Verzeichnisgut loeschen Verzeichnisgut aendern Login Verzeichnisgut aufnehmen $ Verzeichnisgut loeschen Verzeichnisgut aendern 0.0 Login.0 Verzeichnisgut aufnehmen.03 Verzeichnisgut loeschen Model-Based Testing Manual Specification Manual Specification Use cases (domain) Login Aktenbearbeitung Dynamic Analysis Aktenbearbeitung Login Aktenbearbeitung $ Screen flows (sessions) Automatic Extraction Behavior model(s) (markov chains) Session layer Application model Protocol layer Test/workload model Workload Generation Legacy system System under test W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

15 Automatic Extraction of Behavior Models Model-Based Testing. Login Aktenbearbeitung 0.0 Login 2.0 Verzeichnisgut aufnehmen 2. Individual sessions with absolute transition frequencies among use cases Aktenbearbeitung $ Login 0.0 Login Aktenbearbeitung $ Individual sessions derived from screen flow extraction (dyn.) 3. Behavior model(s).0 Verzeichnisgut aufnehmen $ W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

16 Behavior Model Markov model of user sessions (states = use cases) Model-Based Testing Example Behavior Model: Users access Verzeichnisgut 0.0 Login Verzeichnisgut aufnehmen Verzeichnisgut aendern Verzeichnisgut loeschen $ W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

17 Agenda Zusammenfassung und Ausblick Project Context 2 Dynamic Analysis 3 Model-Based Testing 4 Zusammenfassung und Ausblick W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

18 AIDA-SH Neuentwicklung Zusammenfassung und Ausblick Einige weitere Ergebnisse aus DynaMod, die die Neuentwicklung von AIDA-SH unterstützt haben: Statische Analyse: Entdeckung redundanter SQL-Statements Dynamische Analyse: Entdeckung von Zeitfressern bei SQL-Statements Hybride Analyse: Entdeckung von nicht mehr genutzten Funktionen / Forms W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

19 Jubiläumsfeier 25-Jahre AIDA-SH AIDA-SH 3.0 Zusammenfassung und Ausblick Präsentation des neuen AIDA-SH vor den Nutzern Ende 202: Dr. Johann Bizer, Vorsitzender des Dataport-Vorstands. W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

20 Zusammenfassung und Ausblick Zusammenfassung und Ausblick Statische/dynamische Analyse VB 6 Werkzeuge für statische Analyse (Parser, Sprachmodell etc.) Instrumentierte Version von AIDA-SH im Landesarchiv produktiv Hybride Analyse Modellbasiertes Testen Modellbasierte Lastgenerierung für b+m Gear (am Beispiel AIDA-SH) Automatische Extraktion von Benutzungsmodellen aus Monitoringdaten Ausblick Vollständige Automatisierung der Konfiguration von Markov4JMeter und modellgetriebene Generieung der Last-Skripte. Validierung durch Vergleich der Messungen aus dem Produktivbetrieb mit den Messungen aus Lastexperimenten. W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

21 Literature Literature [AATP2] [BC98] [KRM06] [MAFM99] [RvHM + 08] [UL07] [vhfg + ] Fredrik Abbors, Tanwir Ahmad, Dragoş Truşcan, and Ivan Porres. MBPeT: A Model-Based Performance Testing Tool. VALID 202, The Fourth International Conference on Advances in System Testing and Validation Lifecycle, 202. Paul Barford and Mark Crovella. Generating Representative Web Workloads for Network and Server Performance Evaluation. Proceedings of the 998 ACM SIGMETRICS Joint International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS 98/PERFORMANCE 98), pages 5-60, 998. D Krishnamurthy, J.A. Rolia, and S. Majumdar. A Synthetic Workload Generation Technique for Stress Testing Session-Based Systems. IEEE Transactions on Software Engineering, Volume 32, Issue, pages , Daniel A. Menascé, Virgilio A. F. Almeida, Rodrigo Fonseca, and Marco A. Mendes. A Methodology for Workload Characterization of E-commerce Sites. Proceedings of the st ACM conference on Electronic commerce (EC 99), pages 9-28, 999. Matthias Rohr, André van Hoorn, Jasminka Matevska, Nils Sommer, Lena Stoever, Simon Giesecke, and Wilhelm Hasselbring. Kieker: Continuous monitoring and on demand visualization of Java software behavior. In Claus Pahl, editor, Proceedings of the IASTED International Conference on Software Engineering 2008 (SE 08), pages 80 85, February Mark Utting and Bruno Legeard. Practical Model-Based Testing. A Tools Approach. Morgan Kaufmann Publishers, Burlington, Massachusetts, USA, André van Hoorn, Sören Frey, Wolfgang Goerigk, Wilhelm Hasselbring, Holger Knoche, Sönke Köster, Harald Krause, Marcus Porembski, Thomas Stahl, Marcus Steinkamp, and Norman Wittmüss. DynaMod project: Dynamic analysis for model-driven software modernization. In Andreas Fuhr, Wilhelm Hasselbring, Volker Riediger, Magiel Bruntink, and Kostas Kontogiannis, editors, Joint Proceedings of the st International Workshop on Model-Driven Software Migration (MDSM 20) and the 5th International Workshop on Software Quality and Maintainability (SQM 20), volume 708 of CEUR Workshop Proceedings, pages 2 3, March 20. Invited paper. W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22

22 Literature (cont d) Literature [vhkgh] [vhrh08] [vhrh + 09] [vhwh2] André van Hoorn, Holger Knoche, Wolfgang Goerigk, and Wilhelm Hasselbring. Model-driven instrumentation for dynamic analysis of legacy software systems. In Proceedings of the 3th Workshop Software-Reengineering (WSR 20), pages 26 27, May 20. (Softwaretechnik-Trends 3(2) (May 20) 8 9). André van Hoorn, Matthias Rohr, and Wilhelm Hasselbring. Generating probabilistic and intensity-varying workload for web-based software systems. In Samuel Kounev, Ian Gorton, and Kai Sachs, editors, Performance Evaluation Metrics, Models and Benchmarks: Proceedings of the SPEC International Performance Evaluation Workshop (SIPEW 08), volume 59 of Lecture Notes in Computer Science (LNCS), pages 24 43, Heidelberg, June Springer. André van Hoorn, Matthias Rohr, Wilhelm Hasselbring, Jan Waller, Jens Ehlers, Sören Frey, and Dennis Kieselhorst. Continuous monitoring of software services: Design and application of the Kieker framework. Technical Report TR-092, Department of Computer Science, University of Kiel, Germany, November André van Hoorn, Jan Waller, and Wilhelm Hasselbring. Kieker: A framework for application performance monitoring and dynamic software analysis. In Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering (ICPE 202), pages ACM, April 202. W. Hasselbring, A. van Hoorn, E. Schulz Modellbasiertes Performance-Testen / 22