Visualisierung I. Bildgebende Verfahren in der Medizin und medizinische Bildverarbeitung Visualisierung I 1 / 85

Transkript

1 1 / 85

2 Usage of Medical Volume Data Diagnosis Simulation & training Treatment planning e.g., radiation therapy (oncology) e.g., surgical interventions (orthopedics, oncology) Intraoperative navigation Monitoring e.g., chemotherapy (oncology) 2 / 85

3 Motivation Medical Imaging is a fundamental diagnostic and therapeutic tool Single Imaging X-rays, Ultrasound, Angiography Image Sequences: CT, MRI/MRA, PET/SPECT, Gamma Camera 3 / 85

4 The Future of Medicine is Technology 4 / 85

5 Interventional Closed Loop Manual Robotic Irradiation 5 / 85

6 Trends Not diagnosis only : impact on therapy necessary From Diagnosis to Intervention: intra-operative imaging, navigation, image guided therapy From 2D to 3D: Objects vs. images Holistic approaches & closed loops 6 / 85

7 Überblick Rendering-Techniken I Visualisierung des 3D-Bildstapels als geschlossenes 3D-Objekt Oberflächen-basierte Visualisierung: Generierung von Oberflächenrepräsentationen der zu visualisierenden Objekte Rendering mit Standardmethoden der Computergraphik intermediate surface surface extraction representation (polygons) image volume surface rendering 7 / 85

8 Überblick Rendering-Techniken II Volumen-basierte Visualisierung: Generierung der 3D-Ansicht direkt aus den Volumendaten Verwendung der gesamten Grauwertinformation zum Rendering von Oberflächen, Schnittebenen und (semi-)transparenten Volumina Transformations-basierte Visualisierung: Rendering-Prozedur findet nicht im Ortsraum statt Transformation der Bilddaten in eine äquivalente Darstellung segmentation image volume direct volume rendering. attribute volume direct surface rendering 8 / 85

9 Visualization Pipeline image acquisition preprocessing (filtering, interpolation) segmentation/ interpretation rendering 9 / 85

10 Volume Visualization Methods Surface Representation Triangulation using Surface Contours Iso-value Surface Polygonalization: Marching cubes Polygonlization Simplification and Optimization: Surface Fitting or Matching Volume Representation Image Space Ray Casting Ray Tracing Speedup Ray Traversal Improve Ray Sampling Object Space Splatting, Shear-warp Frequency Domain 10 / 85

11 Medical Volume Data CT lung MRI brain thorax NMI fetus US 11 / 85

12 Series of Slices of Images E.g. CT Scans 12 / 85

13 3D Imaging CT Scans 13 / 85

14 Medical Volume Data - CT spatial distribution of X-ray attenuation coefficient normalized by Hounsfield Units (HU) high contrast between bone/tissue resolution: 512x512 per slice HU material air 0 water fat tissue muscle > 200 bone > 100 blood vessels (contrast media filled) 14 / 85

15 Medical Volume Data - MRI spatial density distribution (e.g., hydrogen) allows different imaging parameters high contrast between different tissue types resolution: up to 256x256 per slice 15 / 85

16 Medical Volume Data - 3D-US sound waves (1-15 MHz) are reflected at interfaces (e.g., muscle/fat tissue) noisy, unsharp low dynamic range shadowing reflection depends on angle fast, non-invasive 16 / 85

17 Medical Volume Data - NMI radio active source is injected, radiation is recorded & spatially reconstructed allows monitoring of physiological processes low resolution 17 / 85

18 Volume Data - Terms assigns scalar value to every voxel (volume element) of a given grid cell (space between 8 voxel centers) 18 / 85

19 Voxel Models binary gray level (intensity, e.g., 12 bits/voxel) generalized (+ attributes, e.g., segment, colour, material) intelligent (+ properties of objects and their relationship on a symbolic level) 19 / 85

20 Grid Types in Medicine Cartesian Regular Cylindrical interpolation can transform any grid into a Cartesian grid 20 / 85

21 Volume Data interpolations in cell space nearest neighbor Tri-linear interpolation Tri-cubic interpolation V 3 V 1 V 2 V 4 V I(x,y,z) 5 V 6 V 7 V 8 21 / 85

22 Visualization Pipeline image acquisition preprocessing (filtering, interpolation) segmentation/ interpretation rendering 22 / 85

23 Preprocessing application of a histogram transfer function e.g. thresholding e.g. classification of different tissue types e.g. color mapping filtering extraction of (fuzzy) surfaces 23 / 85

24 Color Mapping MRA brain image 24 / 85

25 Preprocessing Gaussian pyramid (simulates nerve cells of visual cortex) increasing levels contain decreasing degree of detail increasing levels contain voxels of increasing size interpolation between levels constant time filtering 25 / 85

26 Gaussian Pyramid successive filtering using a Gauss filter 26 / 85

27 Extracting Surfaces from US Successive filtering steps on Gaussian pyramid [Sakas, Walter 1995] 27 / 85

28 Extracting Surfaces from US result for a liver data set (Gauss kernel: 5 3, increasing binarization threshold) 28 / 85

29 Filtering Gauss filtered mask size: 3 3, 5 3, median filtered (3 3 ) 29 / 85

30 Visualization Pipeline image acquisition preprocessing (filtering, interpolation) segmentation/ interpretation rendering 30 / 85

31 Volume Visualization Methods Surface Representation Triangulation using Surface Contours Iso-value Surface Polygonalization: Marching cubes Polygonlization Simplification and Optimization: Surface Fitting or Matching Volume Representation Image Space Ray Casting Ray Tracing Speedup Ray Traversal Improve Ray Sampling Object Space Splatting, Shear-warp Frequency Domain 31 / 85

32 Triangulation using Surface Contours Extracting Contour on each slice Constructing triangular Mesh between each two slices Rendering using graphics pipeline 32 / 85

33 Iso-value Surface Polygonalization: Marching Cubes separation of object and background voxels Classify each Voxel Cube Vertex status as IN (less than) or OUT (larger than) iso-value Construct triangles among iso-points modification: triangulation of distinct parts starting from an interactive given point 33 / 85

34 Marching cubes Algorithm 3-d / 85

35 Marching Cubes - result wireframe shaded 35 / 85

36 Triangulation 36 / 85

37 Video 37 / 85

38 Automatic 3D-Scanning Hardware 3D-Scanner Roboter Turntable Benefits Fully automated scanning Hole-free Intuitive, fast Applications 3D-Productcataloges Rapid Prototyping Reverse Engineering QA, 3D-measurements 3D-Documentation Culture, Museums / 85

39 Beispiele 39 / 85

40 Beispiele 40 / 85

41 Animation Drahtgitter 41 / 85

42 Mesh simplification 42 / 85

43 Adaptation: Merging & Subdivision of triangles 43 / 85

44 Example 1. Marching Cube 2. Decimation Full Resolution 569K Triangles 75% decimated (142K Triangles) 90% decimated (57K Triangles) 44 / 85

45 GDMs - Operations Operation Deletion of redundant edges Deletion of short edges Subdivision Smoothing Reproject Description Mesh vertices are deleted, if its normal just differs slightly from the normals of its neighbours Short edges are collapsed to one point Triangles are subdivided according to their distance to the implicitely defined surface Every vertex is replaced by the center of gravity of its surrounding polygon Every vertex is replaced by the intersection of its normal with the surface 45 / 85

46 3D Camera In-Situ 46 / 85

47 3D_camera Registration of Volunteer 47 / 85

48 Deformable Models deformation based on elasticity theory deformation forces (external energy) deform model to fit data model elasticity (internal energy) preserve smoothness 48 / 85

49 Surface Fitting or Matching: Geometrical Deformable Models Define an object approximation (contour) Define a set of operations and an energy field for each point on object and contour Move contour points until they approach the minimum energy position i = 1 I = n 49 / 85

50 Generierung der Organmodelle Organ per Thresholding selektieren Auflösung des Datensatzes verringern Oberfläche glätten (Morphologie) Netzpunkt-Oberfläche des Modells erstellen (Marching Cubes) 50 / 85 50

51 Segmentierung: Ablauf 1. Platzierung des Modells 2. Suche nach Strukturpunken (Kanten, HU) 3. Automatische Adaption 51 / 85

52 Kantenfindung 3D Sobel-Filter 52 / 85

53 Anpassung und Formerhaltung Iterative Minimierung der Summe aus interner (Formerhaltung) und externer (Anpassung an den Datensatz) Energie. Organ model E p s ext E i P i i i 2 i ( pi n j ) T( mi m j 2 int ) i P j N Suche nach Strukturpunkten Balance der Energien 53 / 85 53

54 Segmentation Active Contours 54 / 85

55 Results initial contour 10 iterations 300 iterations 55 / 85

56 Results - EBCT 56 / 85

57 Results - CT initial contour 10 iterations 300 iterations 57 / 85

58 Snake Results 58 / 85

59 Anpassung und Formerhaltung Organ model Suche nach Strukturpunkten Balance der Energien 59 / 85 59

60 Nicht uniform deformierbare Modelle 60 / 85

61 Lokale Anpassungkriterien Beispiel Lunge: Rot = lokal höhere Elastizität Grün = globale Elastizität 61 / 85

62 Beispiel Modelladaption Bildgebende Verfahren in der Medizin und 62 / 85

63 Dental CAD Einführung Ausgangssituation: 3D-Scan der Situation im Patientenmund 63 / 85

64 Solution: Automated CAD/CAM System Work Flow Bildgebende Verfahren in der Medizin und medizinische Bildverarbeitung

65 Fraunhofer s Model Based Approach Individual adaptation: Model adapts automatic to patient s tooth anatomy» Lib tooth model Scan preparation Adaptation Inlay generation Bildgebende Verfahren in der Medizin und medizinische Bildverarbeitu 65

66 Dental CAD Our Approach (3) Smart library teeth can by generated by the user + Mathematical model w/dental knowledge User-selected example tooth Smart library tooth w/dental knowledge 66 / 85

67 T-RecS - Adaptation Cusp tips Cusp line Fissures Equatorial line Bildgebende Verfahren in der Medizin und medizinische Bildverarbeitu 67

68 Results range image tooth model reconstruction 68 / 85

69 Results 69 / 85

70 Artikulator Software-Artikulator Artikulatormodell nach Kundenwunsch 2008 Fraunhofer IGD 70 / 85

71 Advantages of our approach Reconstructions with functional and attractive occlusal surfaces 71 / 85

72 Results Large bridge, ready for CAM 72 / 85

73 Dental CAD - Verfahren Automatische Generierung anatomischer Rekonstruktionen 73 / 85

74 Dental CAD - Schrumpfen Rekonstruktion wird geschrumpft Computergenerierter Teil: Wird gefräst oder gesintert aus Titan, Zirkon, etc Anschließend: Manuelle optisch ansprechende Verblendung mit Keramik 74 / 85

75 IGD Dental-CAD-Complete Solution Intuitive GUI, fast results Fraunhofer IGD

76 Vom CAD-Modell zum Zahn CAD-Modell Aus Zirkon gefertigtes Brückengerüst Manuelle Verkleidung mit Schmelzmasse Nach dem Brand 76 / 85

77 Problem: Kein realistisches Rendering Oberkieferrekonstruktion in der Software und in echt 77 / 85

78 Realistisches Zähnerendering 78 / 85

79 Triangular Surfaces can be rendered by common rendering systems (evtl. HW) reduces data size (fast rendering) prone to false segmentation re-triangulation on parameter change loss of information removal/hidden information 79 / 85

80 Simulators 80 / 85

81 Interventional Closed Loop Manual Robotic Irradiation 81 / 85

82 Simulator System Components U/S Mockup Realistic keypad Database with Pathologies 82 / 85

83 Mannequins Male & female mannequin Tissue mimicking material Oesophagus, rectal, vaginal 83 / 85

84 Different transducers 84 / 85

85 SonoSim-II 85 / 85

86 Fetal Scans Videos Fetal ZNS Study NT- measurement 86

87 Biopsy Simulation 87

88 Abdominal & Ovarian Scans 88

89 Internal Medicine Pleura Erguss Gallenkolik 89

90 Cardiology: TEE & Transthoracic 90

91 Whole Torso 91

92 Atlas matching 92

93 Training & Congresses 93 / 85

94 Virtual Bronchoscopy 94 / 85

95 Bronchoskopie Interaktionsschnittstelle Eigenentwicklung haptisches Rendering Kollisionsdetektion Simulation von Gleiten Simulation flexibler Optik 95 / 85

96 Haptisches Rendering 96 / 85

97 Virtual Bronchoscopy (Video) 97 / 85

98 Rhinoskopie XiTact Force- Feedback Kollisionsdetektion Haptischer Renderer Fraunhofer IGD 98 / 85

99 Rhinoskopie Vorbereitung anstehender Therapien Aufbau einer Trainingsdatenbank Anbindung Patienten-Phantom Elastizitätssimulation (geplant) 99 / 85