Inspection Tutorial Fraunhofer IESE Slide 1

Transkript

1 Slide 1

2

3 Small Bug, Great Disaster - Ariane 5 Flight 501 (1996) Cause Malfunction in control software reused from Ariane 4 (arithmetic overflow during number conversion) Effect Loss of spacecraft Cost >> $1 billion In safety-critical domains, software defects may lead to great disasters Slide 3

4 Typical problems Actual project costs exceed planned costs Actual project schedule exceeds planned schedule 40-50% of the effort is avoidable rework Results: reduced product quality and low customer satisfaction Slide 4

5 Chaos Report % 20% 40% 60% 80% 100% Software projects completed on time Projects cancelled before completion Late and over budget Slide 5

6 Which Quality Assurance (QA) Techniques Exist? Contract Analysis Design Implementation Software System Test Integration Test Module Test Dynamic QA techniques Evaluation of a product/system with respect to fulfillment of requirements (specification) Testing Static QA techniques Evaluation of documents that are created within the development process, in order to find defects Inspections/Reviews Slide 6

7 Typical Quality Assurance Strategy - Testing Testing is often used as the only quality assurance activity Executable code is needed Testing is very expensive (up to 50% of the total costs) Not all defects can be addressed by testing Late detection can have destructive impact (erosion of software) Defects in the early phases of development are hard to find and therefore remain in the product throughout later development phases High concentration of defects at the end of product development Defect injection Requirements Design Code Test Maintenance Defect detection Requirements Design Code Test Maintenance Chaos Zone Slide 7

8 Costs for Late Defect Detection Increase Dramatically Costs for correcting one defect (in ) Defects detected (in %) Defects introduced (in %) Analysis Design Implementation Testing Maintenance Source: SQS AG, analysis of 5000 projects Slide 8

9 Solution: Use of Reviews / Inspections Customized reviews / inspections in the early phases of software development in combination with testing Goal: to find and remove defects in the phase in which they are injected Defect injection Requirements Design Code Test Maintenance Defect detection Requirements Design Code Test Maintenance Focused use of inspections in order to guarantee the desired/required quality of the documents created within the development process Slide 9

10 Comparison of Inspections and Testing Inspections Can be used in early development stages Can address different aspects of quality Usually manual performance Identification of potential defects Testing Is usually used in late development stages Requires an executable system/model Identification of wrong behavior debugging is necessary Very useful for dynamic aspects For effective QA, joint use of inspections and testing is required Slide 10

11 Definition of Formal Inspections Definition Formal inspections are methods of static analysis, used for evaluating quality attributes of software documents, with the goal of identifying defects Characteristics Follow a structured and defined process Team is composed of professionally qualified people Use of specific reading techniques for the detection of defects Results are documented Formal Inspections are the most formal static quality assurance method; most of the other methods can be derived from this one. Slide 11

12 The Basic Process of Inspections Software Artifact Overview Organizer Planning Problemlist Inspectionspackage Inspector Preparation Third-hour Data Collection Moderator Inspector Author Meeting Defectlist Role Activity Optional Document Software Inspection Author Correction Follow-Up Correction list Corrected Software Artifact Slide 12

13 Overview: Static Quality Assurance Methods Different approaches and definitions exist Depending on the goals, the context of development, and the quality requirements, different methods can be used Most formal Least formal Formal Inspection Team Review Walkthrough Peer Deskcheck, Passaround Ad-hoc Empirical studies show that formal inspections are the most effective method Inspections typically allow for defect detection ratios of between 30% and 90% Slide 13

14 Ideen der verschiedenen Ansätze Ad-hoc-Review: Ziele: Fehlerreduzierung, Einholen einer zweiten Meinung Ablauf: Eine weitere Person (z.b. Kollege) wird bei einem konkreten (häufig lokalen ) Problem spontan eingebunden Vorteile Nachteil einfach und überall einsetzbar geringe Kosten Ergebnis ist abhängig von der Erfahrung der eingebundenen Person Slide 14

15 Ideen der verschiedenen Ansätze Peer Deskcheck: Ziele: Fehlerfindung, konstruktive Hinweise, Mentoring / Coaching Ablauf: Eine Person begutachtet ein Dokument und kommentiert entsprechend Hinweise Für nicht-kritische Systemteile zu empfehlen Weder Prozess noch weitere Hilfsmittel sind vorgegeben, können aber eingesetzt werden Vorteile Geringer Aufwand, einfach einzusetzen Verminderter Druck für den Autor, da nur ein Inspektor das Dokument inspiziert Nachteile Ergebnis abhängig von Wissen, Erfahrung und Disziplin der eingebundenen Person Gruppeneffekte werden nicht genutzt Autor nicht anwesend Anmerkung: Ein Deskcheck als solcher entspricht einem Selbst-Review Slide 15

16 Ideen der verschiedenen Ansätze Passaround: Ziele: Fehlerfindung, konstruktive Hinweise Ablauf: Mehrere Personen begutachten ein Dokument und fügen Kommentare ein Hinweis: Für nicht kritische Systemteile zu empfehlen Vorteile Geringer Aufwand, einfach einzusetzen Schwächen des Deskcheck reduziert (Gefahr von keinem Feedback sowie geringwertigem Feedback) Überbrückung geographischer Differenzen Nachteile Widersprüchliche oder identische Vorschläge Kein oder verspätetes Feedback möglich Keine Team-Inspiration Autor nicht anwesend Slide 16

17 Ideen der verschiedenen Ansätze Walkthrough: Ziele: Präsentation einer Lösungsidee, gemeinsames Verständnis, Diskussionen zu Lösungsideen, Fehlerfindung Ablauf: Der Autor eines Dokuments präsentiert es weiteren Person, regt eine entsprechende Diskussion an und sammelt Kommentare der Teilnehmer Vorteile Teilnehmer erlangen ein besseres Verständnis des Dokumentes Teamaktivität (z.b. Diskussion, Feedback der Teilnehmer) Autor ist aktive Rolle Nachteile Kein formaler Prozess, keine Rollenzuweisung, keine Datensammlung keine Hilfestellung zur Bewertung der Qualität Risiko: Autor stellt Arbeit extrem positiv dar Risiko: Verständnislücken bei Teilnehmern werden nicht ausgeräumt und dadurch Fehler maskiert Risiko: Kommentare der Teilnehmer werden nicht umgesetzt Slide 17

18 Ideen der verschiedenen Ansätze Team-Review: Ziel: Fehlerfindung, Diskussionen zu Lösungsideen Ablauf: Verschiedene Kollegen begutachten ein Dokument, um potentielle Fehler zu finden Hinweise Sehr ähnlich zur Inspektion Autor darf Moderatorrolle einnehmen Mehr Spielraum für inhaltliche / konstruktive Diskussionen Vorteile Verständnis zum Dokument wird verbessert Ergebnisse werden in einem Meeting mit dem Autor konsolidiert Nachteil Keine definierte Unterstützung bei der Fehlersuche Metriken spielen eine untergeordnete Rolle Slide 18

19 Reading Techniques Overview (1/2) Ad-hoc No support, experience-based Checklist-based reading Checklist-based reading (CBR): one checklist for all Focused checklist: Inspectors receive different checklists containing questions that focus on the needs of different perspectives Guided checklist: Inspectors receive different checklists containing questions that focus on different quality aspects Slide 19

20 Reading Techniques Overview (2/2) Scenario-based reading Perspective-based reading (PBR): The Inspector performs defect detection from one perspective. The Inspector receives a scenario that describes the activities that have to be performed in order to identify defects and which focuses on defects that are important for the respective perspective. Defect-based reading (DBR): The Inspector receives a scenario that describes which defect classes the Inspector has to focus on during inspection as well as how to perform defect detection. Usage-based reading (UBR): The Inspector receives a set of Use Cases. These will be executed manually during the inspection, in order to check if the design/the code implements the Use Cases. Others Stepwise abstraction Slide 20

21 Reading Techniques Coverage and overlap Better coverage of potential defect sources higher effectiveness Reduction of detection overlapping higher efficiency Reduction of the required experience and impact of experience Artifact with defects Single Checklist PBR scenarios Focused checklist Slide 21

22 Reading Techniques PBR Example Example of a tester scenario for requirements inspection: quality focus Introduction: Imagine you are a tester of the requirements. One of your main tasks is to derive acceptance test cases for the various requirements specified in the functional specification and the related user scenarios. To successfully perform your task, it is most important for you that the test cases are easy to derive from the requirements artifact unit perspective activity Instructions: For each assigned requirement of the functional specification in the document and for each related functional user scenario generate a test case or set of test cases that allow you to ensure that an implementation of the system satisfies the functional specification (set of acceptance cases)... Document all issues you identify while performing the task in the issue list. Questions: While following these instructions answer the following questions: Which parts of the requirements are difficult to understand and hard to derive the test cases from, i.e. where is it unclear what the correct behaviour should be? Which necessary information is missing to identify the item being tested and the test criteria (e.g. where are valid inputs or outputs missing, are acceptance criteria missing)? task questions Slide 22

23 Literature Peer Reviews in Software, K. Wiegers, 2002 Software Inspection, T. Gilb, D. Graham, Bibliographie Slide 23