Experiments in Term Translation

Transkript

1 Experiments in Term Translation Mihael Arcan DERI, NUI Galway Supervised by Dr. Paul Buitelaar Monnet is supported by the European Union under Grant No

2 Generation of multilingual ontologies Motivation most of the ontologies are in English language terms need to be translated

3 Overview Monnet Project Research building domain-specific resources architecture domain-specific resources results and evaluation main findings term disambiguation building a contextual-semantic resource general parallel resource ontology future work

4 Monnet Business Information in EN, DE, NL, ES etc.

5 Research Objectives & Use Cases Research Objectives Development and use of multilingual ontologies ontologies with rich multilingual descriptors Exploit domain semantics to improve Machine Translation use of ontological, terminological, linguistic knowledge Use Cases Financial Use Case Cross-lingual Business Intelligence Public Services Use Case Multilingual Access to Government Information

6 Financial Use Case Harmonizing Business Registration across Europe XBRL (extensible Business Reporting Language) Europe Working Group works with Monnet on the xebr taxonomy xebr (XBRL European Business Register) taxonomy defines common concepts with mappings to country/language specific taxonomies National Bank of Belgium (Belgium) Eogs / DCCA (Denmark) Registrite ja infosüsteemide Keskus erik (Estonia) Bilans Service - Infogreffe (France) Bundesanzeiger (Germany) Infocamere (Italy) RSCL (Luxembourg) Kamer van Koophandel (Netherlands) Informa DB Colegio de Registradores (Spain) Bolagsverket (Sweden) Companies House (United Kingdom) EBR (Europe) GBR (Global) IASCF Bank of Spain Software Audit Consulting

7 Public Services Use Case Translation of Dutch regulation (legal ontology) into several EU languages: Immigration law Tax law Student benefit law Health care benefit law Social security law Law on higher education

8 Basic Ideas of my research Term translation in isolation (no document or sentence context) Experiment 1: domain-specific resources generation addressing out-of-vocabulary issue Experiment 2: contextual-semantic resource generation term disambiguation

9 Experiment 1 Building and exploiting domain-specific resources 1 2 [1] [2]

10 Architecture of Experiment 1 Cross-Lingual Lexicon generation xebr Taxonomy Extraction of financial labels Wikipedia Title extraction Generation of financial lexicon Decoding process Extraction of a parallel resource Phrase Table generation Domain-specific parallel corpus generation

11 Domain-specific parallel corpus generation xebr Taxonomy Extraction of financial labels Querying financial labels Parsing HTML files Phrase Table generation Decoding process

12 Linguee

13 Cross-Lingual Lexicon Generation xebr Taxonomy Wikipedia Title extraction Generation of financial lexicon Extraction of financial labels Decoding process

14 Cross-Lingual Lexicon Generation

22 Domain-Specific resource Generation Overview Linguee parallel corpus on xebr EN terms English: 24K sentences (1M tokens) German: 24K sentences (0.85M tokens) EuroParl (version 6): 1.7M sentences, 43M English, 40M German tokens JRC Acquis (version 3.0): 1.2M sentences, 32M English, 29M German tokens Wikipedia lexicon generation: 7334 translations (translation pairs) from English to German the other way around Current assets <-> Umlaufvermögen Balance sheet <-> Bilanz Unpaid calls on subscribed capital capital -> Kapital Social security, post-employment and other employee benefit costs employee benefit -> Sachbezug

23 Automatic Evaluation Translation source Acquis Europarl Linguee Linguee + Domainspecific lexicon substitution Google Translate Translation Exact Direction Translation BLEU Meteor English to German German to English English to German German to English English to German German to English English to German German to English English to German German to English Evaluation on 63 English-German financial labels

27 Mono-lingual Human Evaluation

28 Manual Evaluation of Translation Quality Evaluation on 63 English-German financial labels Translation into German Acceptable Can easily be fixed None of both Translation into English A C N Linguee+Wikipedia 58% 27% 15% 56% 32% 12% Google Translate 55% 31% 14% 56% 31% 13% Linguee 51% 37% 12% 39% 40% 21% JRC-Acquis 32% 28% 40% 39% 31% 30% Europarl 5% 25% 70% 15% 30% 55%

29 Annotator agreement scores Evaluation on 63 English-German financial labels Agreement Metric Translation into German Translation into English S π κ α S π κ α Linguee+Wikipedia Google Translate Linguee JRC-Acquis Europarl

30 Annotator agreement scores Evaluation on 63 English-German financial labels Agreement Metric Translation into German Translation into English S π κ α S π κ α Linguee+Wikipedia Google Translate Linguee JRC-Acquis Europarl substantial agreement

31 Cross-Lingual Human Evaluation

32 Manual Evaluation of Translation Quality Evaluation on 142 English financial labels Translation into German Acceptable Can easily be fixed None of both Linguee+Wikipedia 59.15% 29.34% 11.50% Agreement Metric Translation into German S π κ α Linguee+Wikipedia

33 Manual Evaluation of Translation Quality Evaluation on 142 English financial labels Translation into German Acceptable Can easily be fixed None of both Linguee+Wikipedia 59.15% 29.34% 11.50% Agreement Metric Translation into German S π κ α Linguee+Wikipedia fair agreement

34 Discussion Reference reduces semantics Source: Long-term financial assets Reference: Finanzanlagen Translation: Langfristige finanzielle Vermögenswerte Reference adds semantics Source: Financial result Reference: Finanz- und Beteiligungsergebnis Translation: Finanzergebnis Domain training needed Source : equity Reference: Eigenkapital Translation (Google Translate): Gerechtigkeit

35 Main Findings of Experiment 1 Domain-specific resource gives better results than a bigger, but more general one Linguee parallel corpus + domain-specific multilingual Wikipedia outperform Google Translate for translating German terms into English

36 Experiment 2: Term Disambiguation

37 Experiment 2: Term Disambiguation Experiment 1: using the ontology vocabulary to generate new domain-specific resources domain-specific resource (Linguee / Wikipedia) Experiment 2: using the ontology vocabulary for term disambiguation general resource (Europarl)

38 Experiment 2: Term Disambiguation equity Eigenkapital , , , ,... equity Gerechtigkeit , , , ,... p(equity Gerechtigkeit) > p(equity Eigenkapital)

39 Experiment 2: Term Disambiguation... to decrease significant the negative equity of the Group short-term and so zu einer Verringerung erhebliche negative Eigenkapital der Fraktion kurzfristige und...

40 Experiment 2: Term Disambiguation... to decrease significant the negative equity of the Group short-term and so zu einer Verringerung erhebliche negative Eigenkapital der Fraktion kurzfristige und... Generation of a contextual-semantic resource concentrating only on unigrams

41 Europarl Phrase Table for Equity equity Equity... equity Beteiligungskapital... equity Eigen-... equity Eigenkapital... equity Eigenkapitalquote... equity Eigenkapitals... equity Eigenmittel... equity Eigenmitteln... equity Equity... equity Fairness... equity Fairneß... equity Gerechtigkeit,... equity Gerechtigkeit... equity Geschlechtergleichstellung... equity Gleichbehandlung... equity Gleichheit... equity Gleichheitsprinzips... equity Kapitalanlagegesellschaften... equity Kapitalbeteiligung... equity Kapitalbeteiligungen... equity Stammkapital... equity der Gerechtigkeit... equity der Gleichheit... equity die Gerechtigkeit... equity die Gleichbehandlung... equity gerechten... equity gerechtes... equity gleichberechtigten......

42 Generating a contextual-semantic resource... please tell me which agreement or conclusion of equity or redistribution sagen Sie mir bitte, in welchem Übereinkommen oder welcher Schlussfolgerung die Rede von Gerechtigkeit oder Umverteilung ist... Source Label Target Label Context (frequency + 1) equity Gerechtigkeit please, tell, me, agreement, conclusion, redistribution

43 Generating a contextual-semantic resource... Minority interests relate to the percentage of equity of subsidiaries owned by third-party shareholders Minderheitsanteile betreffen die von Drittaktionären gehaltenen Anteile am Eigenkapital von Tochtergesellschaften... Source Label Target Label Context (frequency + 1) equity Eigenkapital minority, subsidiaries, interests, owned, relate, third-party, percentage, shareholders

44 Source label Target label Context (frequency) equity Eigenkapital capital(43), SMEs(8), risk(7), enterprises(7), tax(6), banks(6), finance(5), European(5)... first(1), problematic(1), candidate(1), industries(1) equity Generating a contextual-semantic resource Gerechtigkeit social(24), efficiency(21), education(17), European(15), justice(15), training(11), principle(11), quality(10), system(9)... migration(1), information(1), directive(1), inspection(1)...

45 Financial vocabulary of the UK GAAP ontology Source UK GAAP Ontology vocabulary other non financial creditors after more than one year(1), social security costs(1), other value adjustments and provisions(1), net financial income(1)... assets(24), year(24), charges(20), financial(20), income(18), payable(14), amounts(12), fixed(11), operating(11), taxes(9)...

46 Cosine similarity

47 Cosine similarity

48 Cosine similarity Ontology Context Equity-Gerechtigkeit extracted Context from Europarl Equity-Eigenkapital extracted Context from Europarl

49 Translation Probability vs Cosine similarity Translation probability (Equity) p(e f) Gerechtigkeit -10,6227 Gleichheit -11,5476 Eigenkapital -12,7612 Gleichbehandlung -13,0936 Fairness -13,6301 Equity -13,6523 Beteiligungskapital -14,2502 Fairneß -14,2729 Eigenmitteln -14,5058 Eigenkapitals -14,8938 Kapitalanlagegesellschaften -15,2789 Kapitalbeteiligungen -15,6793 Kapitalbeteiligung -15,6971 Cosine similarity (Equity) Cosine Eigenkapital 0, Eigenkapitals 0, Kapitalbeteiligungen 0, Kapitalbeteiligung 0, Equity 0, Gerechtigkeit 0, Beteiligungskapital 0, Gleichheit 0, Gleichbehandlung 0, Kapitalanlagegesellschaften 0, Fairness 0, Fairneß 0, Eigenmitteln 0,005969

50 Translation Probability vs Cosine similarity Translation probability (Equity) p(e f) Gerechtigkeit (justice, equity) -10,6227 Gleichheit (equality, equity) -11,5476 Eigenkapital (equity) -12,7612 Gleichbehandlung -13,0936 Fairness -13,6301 Equity -13,6523 Beteiligungskapital -14,2502 Fairneß -14,2729 Eigenmitteln -14,5058 Eigenkapitals -14,8938 Kapitalanlagegesellschaften -15,2789 Kapitalbeteiligungen -15,6793 Kapitalbeteiligung -15,6971 Cosine similarity (Equity) Cosine Eigenkapital 0, Eigenkapitals 0, Kapitalbeteiligungen 0, Kapitalbeteiligung 0, Equity 0, Gerechtigkeit 0, Beteiligungskapital 0, Gleichheit 0, Gleichbehandlung 0, Kapitalanlagegesellschaften 0, Fairness 0, Fairneß 0, Eigenmitteln 0,005969

51 Translation Probability vs Cosine similarity Translation probability (Equity) p(e f) Gerechtigkeit -10,6227 Gleichheit -11,5476 Eigenkapital -12,7612 Gleichbehandlung -13,0936 Fairness -13,6301 Equity -13,6523 Beteiligungskapital -14,2502 Fairneß -14,2729 Eigenmitteln -14,5058 Eigenkapitals -14,8938 Kapitalanlagegesellschaften -15,2789 Kapitalbeteiligungen -15,6793 Kapitalbeteiligung -15,6971 Cosine similarity (Equity) Cosine Eigenkapital 0, Eigenkapitals 0, Kapitalbeteiligungen 0, Kapitalbeteiligung 0, Equity 0, Gerechtigkeit 0, Beteiligungskapital 0, Gleichheit 0, Gleichbehandlung 0, Kapitalanlagegesellschaften 0, Fairness 0, Fairneß 0, Eigenmitteln 0,005969

52 Future Work Experiment 1: domain-specific resource generation cross-lingual manual evaluation with financial experts extraction of information in non-parallel resources Experiment 2: contextual-semantic resource generation extending the model to all n-grams examine the structure of the ontology part-of relation parent-child relation

53 Thank You