Parallelizing derivative calculations using OpenMP and ADOL-C

Transkript

1 Parallelizing derivative calculations using OpenMP and ADOL-C Andreas Kowarz Technische Universität Dresden 4th European orkshop on Automatic Differentiation December 8, 2006 Granted by DFG (A 1607/2-1) 1 / 19

2 Outline 1 Parallelization in general 2 Automatic Differentiation exploiting parallel hardware 3 ADOL-C and OpenMP 4 Numerical example Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

3 Parallelization in general Parallelization in general Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

4 Parallelization in general Currently: increasing availability of multi-processor computers even for small scale systems Offering: higher overall peak performance lower power consumption Necessary: parallelized software Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

5 Parallelization in general Consequences: steady (fast?) increase of parallel hardware for ordinary office computers easier access to parallel test environments no proposals for HPC-systems, known working environment, parallelization strategies much easier testable and probably earlier/more often considered, therefore Parallelization compatibility will be a crucial aspect of the acceptance of software. Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

6 Automatic Differentiation exploiting parallel hardware Automatic Differentiation exploiting parallel hardware Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

7 Automatic Differentiation exploiting parallel hardware Parallelization models Multiplexed thread-safe environment simple model each thread uses a standalone AD-environment function values and derivatives computed within the same thread remember: most static variables break thread safety Example: begin parallel for compute function values compute derivatives end parallel for Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

8 Automatic Differentiation exploiting parallel hardware Parallelization models (2) parallel AD-approach advanced model parallel regions within the function are recognized derivative calculations are parallelized accordingly Example: begin parallel for compute function values end parallel for compute derivatives Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

9 Automatic Differentiation exploiting parallel hardware Question: Does a correct parallelization of the function induces the possibility to parallelize the derivative computations as well? Example: vector x, y; scalar s FO i = 1,, n y i = x i s vector x, y; scalar s FO i = n,, 1 x i += ȳ i s s += ȳ i x i ȳ i = 0 Data race in s if loops get parallelized!!! Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

10 ADOL-C and OpenMP Combining ADOL-C and OpenMP Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

11 ADOL-C and OpenMP Active data type - internals adouble 1 - loc. 1 adouble 2 - loc. 4 adouble 3 - loc. 9 store (values) Creating an adouble means: 1. allocate space for the adouble object 2. acquire the next free location 3. increase the size of the store if necessary Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

12 ADOL-C and OpenMP OpenMP and shared adoubles default usage, once adouble creation is done but: location acquiring and store management needs to be protected by usage of a critical section esult: limited degree of parallelization due to compiler generated temporaries Processor 1 Processor 2 Processor 3 Processor 4 Processor 5 operation operation operation operation operation Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

13 ADOL-C and OpenMP Parallelization technique used in ADOL-C internal environment multiplexed at the begin of a parallel program part by use of a initialization directive advantage: no critical section needed disadvantage: copy overhead variables are firstprivate different tapes for serial and parallel part execution order Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

14 ADOL-C and OpenMP Parallel loops OpenMP directives are not recognized by ADOL-C loop structure is not contained within the internal representation no direct handling of parallel loops possible Possible workaround: encapsulate the loop within an external differentiated function loop iterations represented as subtapes manage parallelization by hand for function and derivative part Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

15 ADOL-C and OpenMP Open questions: handling of the remaining OpenMP directives, e.g., barriers, reductions, data exchange between serial and parallel parts To do: bug fixing optimization runtime comparisons Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

16 Numerical example Numerical example Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

17 Numerical example Time propagation of a 1D quantum plasma N Gaussian wavefunctions set of N coupled equations for T time steps U k+1 i Ψ k+1 i = U k i Ψ k i i = 1,, N; k = 0,, T where U i depends on V i V i = f (Ψ 1,, Ψ N, dt, dz) target function: <ρ>= N dz Ψi 2 z i=1 derivatives needed for design optimization and interpretation of physical dependencies 3N + 2 input parameters, one output parameter reverse mode of AD applicable Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

18 Numerical example Core program structure FO each time step FO each wavefunction outer loop enforces checkpointing - about 100MB tape size per timestep order of steps needed inner loop can be parallelized each iteration represents the evaluation of one equation work in progress Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19

19 Numerical example THANKS TO THE AUDENCE Andreas Kowarz (, TU Dresden) Parallelizing derivative calculations Aachen, December 8, / 19