IDRT: Unlocking Research Data Sources with ETL for use in a Structured Research Database

Transkript

1 First European i2b2 Academic User Meeting IDRT: Unlocking Research Data Sources with ETL for use in a Structured Research Database The IDRT Team (in alphabetical order): Christian Bauer (presenter), Benjamin Baum, JanChristoph, Igor Engel, Thomas Ganslandt, Matthias Löbe, Sebastian Mate, Daniel Plog, Hans-Ulrich Prokosch, Matthias Quade, Ulrich Sax, Sebastian Stäubert, Lars Voitel, Alfred Winter

2 IDRT Architecture Loading Concept Dimension Table Observation Fact Table Patient Dimension Table i2b2 Transformation Target Metadata Mapping Talend Open Studio Oracle Database Modelling Staging Standard Metadata Source Metadata Source Fact Data Oracle Database PIDgen Extraction Individual extractors for each terminology CSV OWL... Configurable extractors for each type of datasource ODM SQL CSV Talend Open Studio Java Datasources ICD LOINC... Standard Terminologies CDMS EHR 21 Operative Systems Source Systems 1. European i2b2 Academic User Meeting 25/03/13 - Erlangen page 2

3 IDRT - Goal create tools for a simple and easy import of medical data into the i2b2 database challenges How do we get the data into the database? How do we get i2b2 ontologies for the data? 1. European i2b2 Academic User Meeting 25/03/13 - Erlangen page 3

4 ETL Loading Concept Dimension Table Observation Fact Table Patient Dimension Table i2b2 Transformation Target Metadata Mapping Talend Open Studio Oracle Database Modelling Staging Standard Metadata Source Metadata Source Fact Data Oracle Database PIDgen Extraction Individual extractors for each terminology CSV OWL... Configurable extractors for each type of datasource ODM SQL CSV Talend Open Studio Java Datasources ICD LOINC... Standard Terminologies CDMS EHR 21 Source Systems 1. European i2b2 Academic User Meeting 25/03/13 - Erlangen page 4

5 ETL Which data formats do we need? CSV SQL CDISC-ODM How can we import the data into the i2b2 database? create generic ETL jobs for the data formats How can we get i2b2 ontologies for the patient data? use configuration files to get some user input How will we create the ETL? use Talend Open Studio 1. European i2b2 Academic User Meeting 25/03/13 - Erlangen page 5

6 ETL / Talend Open Studio open source data integration program used for the creation of ETL (extract transform load) graphical code generator (Java) 1. European i2b2 Academic User Meeting 25/03/13 - Erlangen page 6

7 CSV/SQL-ETL user creates configuration file nice names, data types, patient ids job loads files/database patient data job creates i2b2 ontology and patients based on configuration and patient data job loads i2b2 specific data and patient data into the i2b2 database 1. European i2b2 Academic User Meeting 25/03/13 - Erlangen page 7

8 CSV/SQL-ETL user creates configuration file nice names, data types, patient ids job loads files/database patient data job creates i2b2 ontology and patients based on configuration and patient data job loads i2b2 specific data and patient data into the i2b2 database 1. European i2b2 Academic User Meeting 25/03/13 - Erlangen page 8

9 CSV/SQL-ETL ETL jobs can be run inside Talend Open Studio, as java code or in a gui gui has automatic creation of configurations gui has easy editing and automatic saving of configurations gui has server browser 1. European i2b2 Academic User Meeting 25/03/13 - Erlangen page 9

10 CSV/SQL-ETL 1. European i2b2 Academic User Meeting 25/03/13 - Erlangen page 10

11 ODM-ETL CDISC xml standard represents a paper based trial study ( study -> events -> forms -> item groups -> items) no configuration needed! 1. European i2b2 Academic User Meeting 25/03/13 - Erlangen page 11

12 ETL all ELT jobs have two sections writing to i2b2 database is the same job for all the IDRT imports Datasources Extraction Loading Werkzeug Datei TOS Datenbank Teilprozess Extraction Loading Arbeitsschritt file/db Extraction Transformation Transformation Loading i2b2 Datenformat ODM TOS-Intern IDRT-Datenstruktur TOS-Intern i2b2-oracle 1. European i2b2 Academic User Meeting 25/03/13 - Erlangen page 12

13 Security: integration of a patient pseudonymization service patient data with personal information Talend Open Studio sub job patient data with psyeudoym answer from the patend identifaction Success:ZLTGHE3D:Es wurde ein passender Fall gefunden. 1. European i2b2 Academic User Meeting 25/03/13 - Erlangen page 13

14 Discussion & Outlook IDRT ETL provides an easy to use package for importing patient data into i2b2. But CSV/SQL i2b2 ontologies are unstructured and often not nice to look at no complexe i2b2 ontologies we need sub data elements ( 1 patient -> n biomaterial specimen ) IDRT 2 solutions: expand the ETL to incorporate sub data elements create a editor to manipulate and create ontologies 1. European i2b2 Academic User Meeting 25/03/13 - Erlangen page 14

15 IDRT2 - Editor Source-Ontologie Source-Ontology Source-Ontologie Target-Ontology > termionolgies > ICD10 > patient data > diagnose > date > diagnose > medication takings > Menge > Einheit > Trinkgewohnheit > 0,2l Rotwein pro Tag > 200ml Wein pro Tag > 2 kleine Glas Rotwein tgl mapping ot termionology mapping of a datet 1 3 mapping of more than one item > Ontology > diagnose > A00-B99 > C00-D48 > > medication takings > Menge > Einheit > Trinkgewohnheit > 200 ml Wein/Tag bundle c oncepts 2 1. European i2b2 Academic User Meeting 25/03/13 - Erlangen page 15

16 Queries i2b2 Ontology Tables Concepts Modifier Observation Fact Table Patient Tables i2b2 Oracle Database Transformation Target Metadata Mapping Talend Open Studio Oracle Database Modelling i2b2-ontology-editor Java/RCP Talend Open Studio Oracle Database Queries Staging i2b2 Ontology Standard Tables Metadata Concepts Observation Source Fact Metadata Table Source Patient Tables Fact Data i2b2 Oracle Database PIDgen Extraction Individual extractors for eachterminology ClaML CSV Configurable extractors for each type of datasource ODM SQL CSV Talend Open Studio Java Datasources ICD 21 Standard Terminologies CDMS EHR 21 Operative Systems Antrag IDRT2 / TMF ITQM / Christian Bauer / Seite 16

17 Discussion & Outlook IDRT ETL provides an easy to use package for importing patient data into i2b2. But CSV/SQL i2b2 ontologies are unstructured and often not nice to look at no complexe i2b2 ontologies we need sub data elements ( 1 patient -> n biomaterial specimen ) IDRT 2 solutions: expand the ETL to incorporate sub data elements create a editor to manipulate and create ontologies 1. European i2b2 Academic User Meeting 25/03/13 - Erlangen page 17