Preference Grammars and Soft Syntactic Constraints for GHKM Syntax-based SMT

Transkript

1 Preference Grammars and oft yntactic Constraints for GHKM yntax-based MT Matthias Huck, Hieu Hoang, Philipp Koehn University of Edinburgh 25 Ocber 2014

2 Introduction Feature-based integration of syntactic information in GHKM string--tree translation Preference grammars: soft target-side syntax Target syntax as a feature rar than via labeled non-terminals in CFG oft syntactic constraints: non-restrictive source-syntactic enhancement No hard source-syntactic constraints (as in standard tree--tree translation) imposed in extraction or decoding Our empirical evaluation: Englh German WMT task

3 Related Work GHKM string--tree translation: Galley et al (2004) Open-source Moses implementation for GHKM translation GHKM rule extraction: Williams and Koehn (2012) Decoding with CYK+ parsing and cube pruning: Hoang et al (2009) Competitive results for European language pairs: Nadejde et al (2013); Williams et al (2014) Preference grammars: beneficial as a syntactic extension of hierarchical systems (Venugopal et al, 2009; tein et al, 2010) oft syntactic constraints: related source-syntactic techniques improved hierarchical (Marn and Resnik, 2008; Vilar et al, 2008; Hoang and Koehn, 2010) and or syntax-based systems (Zhang et al, 2011; Huang et al, 2013) on Chinese Englh and Arabic Englh tasks

4 Preference Grammars Target-side non-terminals not decorated with syntactic labels, but with a single generic non-terminal symbol baseline X,ADJD,anwesend X,ADV,anwesend X, AP-PD, anwesend preference grammar system X,X,anwesend Dtribution of implicit target label vecrs sred as additional information with each translation rule X, X, anwesend # (ADJD) 098 (ADV ) 0001 (AP-PD) 001 Computation of a tree-wellformedness feature during decoding

5 oft ource yntactic Constraints Parse source-side data as well GHKM extracr sres an additional rule property: source syntactic label vecrs Provide parsed input data decoder During decoding, score matches and mmatches of source label vecrs and input labels oft features, no hard constraints rule source label vecrs X, -OC, präsentieren (VB),() X,ADJD,anwesend (),(AD), X, X 0 X 1,jetzt t -B 0 -OC 1 (,,), X, X 0 X 1,t ADV 0 ADJD 1 (,AD,),

6 flowers

7 t jetzt präsentieren flowers

8 -OA t jetzt präsentieren flowers

9 -OA t jetzt präsentieren flowers

10 -OA t jetzt präsentieren jetzt t präsentieren flowers

11 -B -OC -OA t jetzt präsentieren jetzt t präsentieren flowers

12 -B -OC -OA t jetzt präsentieren jetzt t präsentieren flowers

13 -B -OC -OA t jetzt präsentieren jetzt t präsentieren flowers VB TO IN AD

14 -B -OC -OA t jetzt präsentieren jetzt t präsentieren flowers VB TO IN AD

15 präsentieren -OA t jetzt flowers VB TO IN AD -OC präsentieren -B jetzt t JJ AD

16 präsentieren -OA t jetzt flowers VB TO IN AD -OC präsentieren -B jetzt t JJ AD

17 präsentieren -OA t jetzt flowers VB TO IN AD anwesend heute t JJ AD

18 präsentieren -OA t jetzt flowers VB TO IN AD ADJD anwesend ADV heute t JJ AD

19 präsentieren -OA t jetzt flowers VB TO IN AD ADJD anwesend ADV heute t JJ AD

20 Preference Grammars and oft yntactic Constraints for GHKM yntax-based MT Matthias Huck Hieu Hoang Philipp Koehn Motivation oft ource yntactic Constraints Experimental etup Feature-based integration of syntactic information in GHKM string--tree stattical machine translation The hard target-side syntactic constraints that are imposed by target non-terminal labels might be o restrictive hould we soften m? Preference grammars promote syntactic wellformedness on target language side while also allowing for derivations that are not lingutically motivated (as in hierarchical translation) Tree--tree translation often underperforms How can we effectively enhance a strong string--tree baseline with source-side syntactic information? Training: oft syntactic constraints augment system with additional source-side syntax features while not modifying set of string--tree translation rules or baseline feature scores Preference Grammars Target-side non-terminals not decorated with syntactic jetzt t labels, but with a single generic non-terminal symbol Extracted rules which differ only with respect ir non-terminal labels are collapsed a single entry in rule table, and ir rule counts are pooled baseline X, ADJD, anwesend X, ADV, anwesend X, AP-PD, anwesend preference grammar system X, X, anwesend Dtribution of implicit target label vecrs sred as additional information with each translation rule X, X, anwesend # (ADJD) 098 (ADV ) 0001 (AP-PD) 001 Decoding: Computation of a tree-wellformedness feature Training: Provide syntactic parses of source side of data GHKM extracr collects source syntactic labels that cover source-side span of non-terminals ets of source syntactic label vecrs are memorized with rules as an additional property Decoding: Input data parsed in a preprocessing step Computation of three dense features which score matches and mmatches of input labels and source label vecrs that are associated with translation rules rule source label vecrs X, -OC, präsentieren (VB), () X, ADJD, anwesend (), (AD), X, X 0 X 1, jetzt t -B 0 -OC 1 (,, ), X, X 0 X 1, t ADV 0 ADJD 1 (, AD, ), -B Dense features: -OC präsentieren AD JJ t ADV heute AD ADJD anwesend JJ a source syntactic label vecr fully matches input labels left-hand side non-terminal label mmatch number of right-hand side non-terminals label mmatches Englh German WMT task (45 M sentence pairs) yntactic annotation: BitPar for German, Berkeley Parser for Englh Right binarization of target parse trees AMT-style composite labels on source side inglen hierarchical rules are dcarded No more than 50 most frequent label vecrs per rule sred Decoding with CYK+ and cube pruning Tuning with batch MIRA Development set: 2000 selected sentences from newstest Experimental Results (Englh German) system dev newstest2013 newstest2014 Bleu Ter Bleu Ter Bleu Ter GHKM string--tree baseline + hard source syntactic constraints soft source syntactic constraints string--string (GHKM syntax-directed extraction) preference grammar soft source syntactic constraints parse Features for oft yntactic Constraints Large number of binary features which depend on label identity eparate weight tuned for each of m Optionally: Restrict number of sparse features by specifying a core set of labels core=non-composite plain constituent labels as given by syntactic parser (no AMT-style composite labels) core=dev-min-occ100 labels in input data on development set with minimum occurrence count threshold 100 system (tuned on newstest2012) newstest2012 newstest2013 newstest2014 Bleu Ter Bleu Ter Bleu Ter GHKM string--tree baseline + soft source syntactic constraints sparse features sparse features (core = non-composite) sparse features (core = dev-min-occ100)

21 References I Galley, M, Hopkins, M, Knight, K, and Marcu, D (2004) What s in a translation rule? In Proc of Human Language Technology Conf / North American Chapter of Assoc for Computational Lingutics (HLT-NAACL), pages , Bosn, MA, UA Hoang, H and Koehn, P (2010) Improved Translation with ource yntax Labels In Proc of Workshop on tattical Machine Translation (WMT), pages , Uppsala, weden Hoang, H, Koehn, P, and Lopez, A (2009) A Unified Framework for Phrase-Based, Hierarchical, and yntax-based tattical Machine Translation In Proc of Int Workshop on poken Language Translation (IWLT), pages , Tokyo, Japan Huang, Z, Devlin, J, and Zbib, R (2013) Facred oft ource yntactic Constraints for Hierarchical Machine Translation In Proc of Conf on Empirical Methods for Natural Language Processing (EMNLP), pages , eattle, WA, UA

22 References II Marn, Y and Resnik, P (2008) oft yntactic Constraints for Hierarchical Phrased-Based Translation In Proc of Annual Meeting of Assoc for Computational Lingutics (ACL), pages , Columbus, OH, UA Nadejde, M, Williams, P, and Koehn, P (2013) Edinburgh s yntax-based Machine Translation ystems In Proc of Workshop on tattical Machine Translation (WMT), pages , ofia, Bulgaria tein, D, Peitz,, Vilar, D, and Ney, H (2010) A Cocktail of Deep yntactic Features for Hierarchical Machine Translation In Proc of Conf of Assoc for Machine Translation in Americas (AMTA), Denver, CO, UA Venugopal, A, Zollmann, A, mith, N A, and Vogel, (2009) Preference Grammars: oftening yntactic Constraints Improve tattical Machine Translation In Proc of Human Language Technology Conf / North American Chapter of Assoc for Computational Lingutics (HLT-NAACL), pages , Boulder, CO, UA

23 References III Vilar, D, tein, D, and Ney, H (2008) Analysing oft yntax Features and Heurtics for Hierarchical Phrase Based Machine Translation In Proc of Int Workshop on poken Language Translation (IWLT), pages , Waikiki, HI, UA Williams, P and Koehn, P (2012) GHKM Rule Extraction and cope-3 Parsing in Moses In Proc of Workshop on tattical Machine Translation (WMT), pages , Montréal, Canada Williams, P, ennrich, R, Nadejde, M, Huck, M, Hasler, E, and Koehn, P (2014) Edinburgh s yntax-based ystems at WMT 2014 In Proc of Workshop on tattical Machine Translation (WMT), pages , Baltimore, MD, UA Zhang, J, Zhai, F, and Zong, C (2011) Augmenting tring--tree Translation Models with Fuzzy Use of ource-side yntax In Proc of Conf on Empirical Methods for Natural Language Processing (EMNLP), pages , Edinburgh, cotland, UK