Image and Sensor Fusion for Airborne Hyperspectral Sensors M. Bachmann a, S. Holzwarth a, R. Richter a, M. Habermeyer a, E. Borg b and A. Müller a (martin.bachmann@dlr.de) a DLR-DFD DFD,, D-82234 D Wessling b DLR-DFD DFD,, D-17235 D Neustrelitz DIN-Workshop FUSION, Berlin, 20.11.2006
Contents Part 1: Introduction Airborne Optical Sensors at DLR C-AF, Oberpfaffenhofen Pre-Processing Chain for Hyperspectral Data Part 2: Data Fusion Fusion for Hyperspectral Data Selected Methods Image Fusion Example
Spectral Information Sensor Calibration and Atmospheric Correction Spectral information of carbonate, clay minerals, water, cellulose Reflectance Spectral information of vegetation condition Spectral information of Fe 2+,3+ content VNIR SWIR- I SWIR- II 0,4 0,6 0,8 1,0 1,2 1,4 1,6 1,8 2,0 2,2 2,4 Wavelength [µm] Mixture of Green Vegetation & Kaolinite
Spectral Information Multi-spectral Hyper-spectral chlorite calcite dolomite alunite gypsum Few fixed bands minimum identification field knowledge and labanalysis required low confidence Contiguous bands maximum identification increased classification accuracy high confidence kaolinite
Übersicht über Facility Airborne Optical Remote Sensing am C-AF Oberpfaffenhofen DFD-US, Arbeitsgruppe Abbildende Spektroskopie IMF-EV, Experimentelle Verfahren
ARES (Airborne Reflective Emissive Spectrometer) RES
Basic Spectrometer Design e.g. ARES, HyMap e.g. ROSIS
Flugzeugsensoren (hyperspektral) am C-AF Oberpfaffenhofen Abbildendes Spektrometer ROSIS 03 Abbildendes Spektrometer ARES (ISPL) Multispektral-Scanner AADS 1268 (Daedalus) Abbildendes Spektrometer AISA (Specim) Aufnahmemodus Flächenarray, Pushbroom Whiskbroom Optomech. Spiegelabtaster Pushbroom Spektralbereich 0,43-0,86 µm 0,47 12,1 µm 0,4-14 µm 0,42 0,9 µm Bandbreite 4,0 nm 16 150 nm 0,02-5 µm 1,5 1,7 nm FOV ±8 ±31 ±43 / ±22 47,6 / 19,2 IFOV 0,56 mrad 2,0 mrad 2,5 / 1.25 mrad 2,39 / 0,92 mrad Px / Zeile 512 Kanäle 115 Abtastrate Digitalisierung 62 Hz 14 Bit Kalibrier. im Flug Interne Spek.lampe Betrieb ab 1992/1999 Anwendung: Spektrale Feinstrukturen in Küstenzonen und Binnengewässern, Böden, Vegetation 813 155 35 Hz 16 Bit 3 Flächenstrahler, Spek.lampen 2007 Thematische Erkundung, Geologie, Hydrologie, Vegetation, Exploration, Thermale Aufnahmen 716 11 100 Hz 8 Bit 2 Schwarzkörper 1986 Thematische Kartierung, Landnutzung, Stadtklima, Thermal Aufnahmen 392 286 50 Hz 12 Bit ----- 2005 Feldspektrometer zur Unterstützung von Flugmesskampagnen, Vegetation, Hydrologie
ROSIS ARES AADS 115 Spectral Layout 28 29 29 29 30 8 1 1 1 115 channels 155 channels 11 channels UV VIS NIR SWIR I II MIR 100 TIR FIR Transmission [%] 0 0.3 0.5 1.0 1.5 2.0 3.0 5.0 10.0 15.0 20.0 Wavelength (log) [µm]
Flugzeugsensoren (Kameras) am C-AF Oberpfaffenhofen Digitale Stereokamera ADS40 (LGGI) Digitalkamera EOS1Ds Mark II (Canon) Reihenmesskammer RMK A (Zeiss) Aufnahmemodus Spektralbereich FOV ±32 IFOV Pixel Kanäle Abtastrate Digitalisierung 3-Zeilen-Stereokamera RGB, Pan, NIR 0,1 mrad 2x 12000 (staggered) 3x PAN (Stereo), RGB, NIR 800 Hz Betrieb ab 2007 Anwendung: 12 Bit (8 Bit radiometrisch) Topografische Kartierung, Stereoaufnahmen, DGM 3-fach-Digitalkamera, Objektiv 50 mm RGB ±55 x ±13,5 (jeweils ±20 x ±13,5 ) 0,15 mrad 15000 x 3300 (jeweils 5000 x 3300) RGB 0,5 Hz kont., 3.. 8 Hz diskont. 2006 Verkehrsbeobachtung Zentralverschluss Filmtypen: Luftbildfilme in Schwarzweiß, Farbumkehr und Falschfarben eff. Bildformat 23 x 23 cm Wechselobjektive: ±20,5, ±37,0 und ±53,5 Diverse Filtervorsätze 1975 Topografische Kartierung
HyMap, Munich 2004 Example for a typical data acquisition scenario Altitude above ground ~2400m Swath width 2.6 km GSD ARES: 5m GSD ADS40: 0.24m
EnMAP Spaceborne Imaging Spectrometer Spectral range 420 nm 2450 nm Spectral bandwidth 5nm -10nm Ground sampling distance 30 m at nadir Swath width 30 km Target revisit time 4 days, 30 pointing Current Status Phase B Planned Launch 2010
Generic Processing System Automated pre-processing chain for airborne hyperspectral sensors, incl. System correction Parametric geocoding using ORTHO Atmospheric correction using ATCOR, incl. BRDF and terrain correction Archiving in DIMS
Generic Processing System Level 0 Product Laboratory Calibration System Correction Vicarious Calibration Attitude Data, Position Data, DEM Level 1 Product Radiative Transfer Model, Meteorologic Data Parametric Geocoding Atmospheric Correction Level 2a Product Level 2b Product Atmospheric Correction Parametric Geocoding Level 2 Product
Geometric Correction Process Level 1 Product Components External Orientation Additional Data Internal Orientation BSQ Quicklook Sync Processor DEM Boresight Angles Sync Nav Data BSQ ORTHO Scan Angle File
Geo-Correction Raw Sensor Data Georectified
Accuracy of Geo-Correction Highly critical for image fusion Typical accuracies in flat terrain: better than 1.4 pixels RMS in x and y For image fusion, additional image-to-image matching might be required HyMap, 6m GSD ROSIS, 2m GSD
Atmospheric Correction Process Level 1 Product Components longitude, latitude, time, flight altitude asl, heading, average ground elevation asl or DEM Internal Orientation BSQ Quicklook Additional Data External Orientation from geocoding or calculated slope, aspect, skyview Scan Angle File Sensor Characteristics Digital Elevation Model Reflectance/Temperature Cube ATCOR Value Added Products visibility map, water vapor map, surface emissivity, illumination, surface cover map
Illumination Correction Ground Reflectance Ground Reflectance after Terrain Correction
De-Shadowing Example: Hyperspectral Airb. Imagery HyMap, Chinchon, Spain, 12 July 2003 RGB=878/646/462 nm
Example: Multispectral Satellite Data, Ikonos Example of building shadows Ikonos Munich, 17 Sept. 2003, RGB=bands 4/3/2 Courtesy of European Space Imaging / European Space Imaging GmbH
Definition of Fusion Data Fusion is a formal framework in which are expresed means and tools for the alliance of data originating from different sources. It aims at obtaining information of greater quality; the exact definition of greater quality will depend upon the application. Wald, L. (1999): Some terms of reference in data fusion. IEEE Transactions on Geosciences and Remote Sensing, 37(3), pp. 1190-1193
Conceptual Levels of Fusion Relevant applications for fusion of hyperspectral data: Signal level Fusion of different detector arrays of sensor VNIR - TIR Image level High spectral res. ARES data with high spatial res. ADS40 High VNIR res. ROSIS with wider spec. range ARES Object level ADS40-derived Digital Surface Model (DSM) for object extraction, ARES for spectral material identification
http://www.eecs.lehigh.edu/spcrl/if/image_fusion.htm
Requirements Requirements on methods when fusing hyperspectral data: Spectral information must be preserved to a high degree DN of fused image must be related to physical properties (i.e. at-sensor-radiance or reflectance values) Large number of bands (>100) must be fused Handle low correlation between input images Higher spatial resolution including textural properties should be introduced in the fused image
Requirements Consequence: Standard approaches often unsuitable Brovey & color-related (e.g. IHS) fusion: => 3 bands only, correlation between images required Component substitution techniques: (PCT, multiresolution representation by high-pass filter / wavelets) => sometimes loss of information, no spectral fidelity
MMT nach Zhukov et al. Prinzip (stark vereinfacht): Klassifikation des geometrisch hochaufgelösten Bilds Ermittlung der durchschnittlichen Spektralsignatur im spektral hochaufgelösten Bild für jede Klasse Zuweisung dieser Spektralsignatur zu jeder Klasse des geometrisch hochaufgelösten Bilds Vorteil: Benötigt keine Korrelation zwischen Input-Images Fusion unterschiedlicher Wellenlängenbereiche möglich (VIS-TIR) Physikalischer Zusammenhang bleibt erhalten Sensor-Charakteristik (PSF) wird berücksichtigt Nachteil: Spektralsignatur wird über jede Klasse im Bild / Moving Window gemittelt ZHUKOV, B., OERTEL, D., LANZL, F. and REINHAECKEL, G. (1999): Unmixing-based multisensor multiresolution image fusion. IEEE Transactions on Geoscience and Remote Sensing, 37, pp. 1212 1225.
RESOLUTION - Prozessoraufbau Modularer, itertiver Ansatz Physikalisch - leistungsbasiert, d.h. Berücksichtigung der spektralen und räumlichen Energiebilanzierung Nachteil: spektrale Überlappung erforderlich Multispektral- und panchromatische Daten Vorverarbeitung VV Multispektrale Leistungsbilanz ML Datenfusion F Qualitätskontrolle Q Höher aufgelöste Multispektraldaten Räumliche Leistungsbilanz RL Iterative Verbesserung Schematische Darstellung des Resolution-Prozessors
RESOLUTION First Results Landsat7 ETM+ Fusion of full scene Pan-Merge, 1 Iteration Calculation time: <6 min PAN Band 4-3-2 Fused Image (4-3-2)
Synthetic Scenes
Without Fusion MMT WI MMT MW Fusion - Results Synthetic Scene Reference PCA Fusion LowRes 126 bands VNIR/SWIR HighRes 4 bands VNIR Spatial improvement by factor 5
Fusion Scene Simulation HyMap scene Barrax, La Mancha (126 fused bands, CIR band combination) Left half: HyMap-Data, geometric resolution decreased by a factor of 5 using PSF simulation Right half: Reconstructed to original resolution using MMT-WI fusion
Fusion - Results Reference PCT Fusion MMT WI MMT WI with improved parameters Without Fusion MMT MW Classified input scene for MMT HyMap-Scene Barrax, Spain (Subset) Red: Band 25 Green: Band 40 Blue: Band 105
Fusion - Results When fusing hyperspectra data, spectral fidelity of fused image must be guaranteed!
Measuring Fusion Performance Visual assessment Scene statistics (bandwise mean, stdev, dynamic range) Difference to reference scene (correlation, RMSE) Spectral properties of fused image (spectral errors) Accuracy of image classification Methodology by Wald et al. (ERGAS) Universal Image Quality Index by Wang & Bovik
Conclusions Fusion of hyperspectral data requires Adequate pre-processing Image-to-image co-registration Adequate fusion algorithms Perspective from 2007 onward: State-of-the-art VIS, SWIR & TIR ARES data & very high spatial res. ADS40, incl. DSM available operationally on the same platform Full potential and the synergy of both sensors can be exploited by data fusion
References Bachmann, M.; Habermeyer, M. (2003): Evaluation of Image Fusion Techniques for Large-Scale Mapping of Non-Green Vegetation. In: Proc. 3rd EARSeL Workshop on Imaging Spectroscopy, Herrsching, 13-16 May 2003 Colditz, R.; Wehrmann, T.; Bachmann, M.; Steinnocher, K.; Schmidt, M.; Strunz, G.; Dech, S. (2006): Influence of Image Fusion Approaches on Classification Accuracy A Case Study. International Journal of Remote Sensing, 27 (15)