Natürlichsprachliche Systeme I

Transkript

1 Natürlichsprachliche Systeme I WordNet in NLTK D. Rösner Institut für Wissens- und Sprachverarbeitung Fakultät für Informatik Otto-von-Guericke Universität Magdeburg WS 2013/14, 17. Dezember 2013, c D.Rösner D. Rösner NSS I 2013/

2 Gliederung 1 WordNet in NLTK D. Rösner NSS I 2013/

3 WordNet WordNet in NLTK ursprünglich: ein semantisch-orientiertes Wörterbuch für Englisch auch für andere Sprachen GermaNet EuroWordNet u.v.a.m. NLTK includes the English WordNet, with 155,287 words and 117,659 synonym sets. " D. Rösner NSS I 2013/

4 WordNet WordNet in NLTK Synonymie 1 Benz is credited with the invention of the motorcar. 2 Benz is credited with the invention of the automobile. wechselseitige Substitution von motorcar und automobile ändert Bedeutung des Satzes nicht Folgerung: motorcar und automobile sind synonym D. Rösner NSS I 2013/

5 WordNet WordNet in NLTK unterschiedliche Wortbedeutungen organisiert durch (für Synonym set ) >>> wn.synsets( motorcar ) [Synset( car.n.01 )] zu einem Synset gehören (meist) unterschiedliche Wörter (Lemmata) >>> wn.synset( car.n.01 ).lemma_names [ car, auto, automobile, machine, motorcar ] D. Rösner NSS I 2013/

6 WordNet: senses of bank Beispiel: bank hat in WordNet 18 Bedeutungen ( senses ) >>> len(wn.synsets( bank )) 18 die Bedeutungen von bank mit mehrelementigen >>> [(synset, synset.lemma_names) for synset in wn.synsets( bank ) if len(synset.lemma_names) > 1] [(Synset( depository_financial_institution.n.01 ), [ depository_financial_institution, bank, banking_concern, banking_company ]), (Synset( bank.n.07 ), [ bank, cant, camber ]), (Synset( savings_bank.n.02 ), [ savings_bank, coin_bank, money_box, bank ]), (Synset( bank.n.09 ), [ bank, bank_building ]), (Synset( deposit.v.02 ), [ deposit, bank ]), (Synset( trust.v.01 ), [ trust, swear, rely, bank ])] D. Rösner NSS I 2013/

7 WordNet: senses of bank haben meist auch Definitionen >>> wn.synsets( bank )[1].definition a financial institution that accepts deposits and channels the money into lending activities... und illustrierende Beispiele >>> wn.synsets( bank )[1].examples [ he cashed a check at the bank, that bank holds the mortgage on my home ] D. Rösner NSS I 2013/

8 WordNet: senses of bank Definition, Beispiele zu den ersten sechs Bedeutungen >>> for sense in wn.synsets( bank )[:6]: print sense.definition print sense.examples sloping land (especially the slope beside a body of water) [ they pulled the canoe up on the bank, he sat on the bank of the river and watched the currents ] a financial institution that accepts deposits and channels the money into lending activities [ he cashed a check at the bank, that bank holds the mortgage on my home ] a long ridge or pile [ a huge bank of earth ] an arrangement of similar objects in a row or in tiers [ he operated a bank of switches ] a supply or stock held in reserve for future use (especially in emergencies) [] the funds held by a gambling house or the dealer in some gambling games [ he tried to break the bank at Monte Carlo ] D. Rösner NSS I 2013/

9 WordNet: senses of bank Definition, Beispiele zu den zweiten sechs Bedeutungen >>> for sense in wn.synsets( bank )[6:12]: print sense.definition print sense.examples a slope in the turn of a road or track; the outside is higher than the inside in order to reduce the effects of centrifugal force [] a container (usually with a slot in the top) for keeping money at home [ the coin bank was empty ] a building in which the business of banking transacted [ the bank is on the corner of Nassau and Witherspoon ] a flight maneuver; aircraft tips laterally about its longitudinal axis (especially in turning) [ the plane went into a steep bank ] tip laterally [ the pilot had to bank the aircraft ] enclose with a bank [ bank roads ] D. Rösner NSS I 2013/

10 WordNet: senses of bank Definition, Beispiele zu den dritten sechs Bedeutungen >>> for sense in wn.synsets( bank )[12:]: print sense.definition print sense.examples do business with a bank or keep an account at a bank [ Where do you bank in this town? ] act as the banker in a game or in gambling [] be in the banking business [] put into a bank account [ She deposits her paycheck every month ] cover with ashes so to control the rate of burning [ bank a fire ] have confidence or faith in [ We can trust in God, Rely on your friends, bank on your good education, "I swear by my grandmother s recipes"] D. Rösner NSS I 2013/

11 in WordNet sind in WordNet in einer Spezialisierungshierarchie organisiert speziellere werden als Hyponyme bezeichnet abstraktere werden als Hypernyme bezeichnet entsprechende Methoden liefern zu einem die unmittelbaren Nachkommen (Spezialisierungen): hyponyms() bzw. Vorfahren (Generalisierungen): hypernyms() D. Rösner NSS I 2013/

12 in WordNet Beispiel: unmittelbare Nachfahren (Hyponyme) Spezialisierungen von car.n.01 >>> motorcar = wn.synset( car.n.01 ) >>> motorcar Synset( car.n.01 ) >>> motorcar.lemma_names [ car, auto, automobile, machine, motorcar ] >>> types_of_motorcar = motorcar.hyponyms() >>> len(types_of_motorcar) 31 D. Rösner NSS I 2013/

13 in WordNet >>> [synset.lemma_names for synset in types_of_motorcar] [[ Stanley_Steamer ], [ hardtop ], [ loaner ], [ cruiser, police_cruiser, patrol_car, police_car, prowl_car, squad_car ], [ convertible ], [ minicar ], [ minivan ], [ hot_rod, hot-rod ], [ pace_car ], [ sports_car, sport_car ], [ Model_T ], [ bus, jalopy, heap ], [ jeep, landrover ], [ hatchback ], [ coupe ], [ beach_wagon, station_wagon, wagon, estate_car, beach_waggon, station_waggon, waggon ],... D. Rösner NSS I 2013/

14 in WordNet... [ electric, electric_automobile, electric_car ], [ used-car, secondhand_car ], [ limousine, limo ], [ sedan, saloon ], [ gas_guzzler ], [ stock_car ], [ sport_utility, sport_utility_vehicle, S.U.V., SUV ], [ racer, race_car, racing_car ], [ compact, compact_car ], [ horseless_carriage ], [ ambulance ], [ roadster, runabout, two-seater ], [ touring_car, phaeton, tourer ], [ cab, hack, taxi, taxicab ], [ subcompact, subcompact_car ]] D. Rösner NSS I 2013/

15 in WordNet Beispiel: Hypernyme und Hypernym-Pfade motorcar hat nur eine Generalisierung >>> motorcar.hypernyms() [Synset( motor_vehicle.n.01 )] im allgemeinen Fall können aber mehrere unmittelbare Vorfahren haben >>> wn.synset( wheeled_vehicle.n.01 ).hypernyms() [Synset( container.n.01 ), Synset( vehicle.n.01 )] D. Rösner NSS I 2013/

16 in WordNet: Hypernym-Pfade daher kann es auch mehrere Pfade zu einem allgemeinsten Hypernym (oder Wurzelhypernym) geben >>> paths = motorcar.hypernym_paths() >>> len(paths) 2 >>> [synset.name for synset in paths[0]] [ entity.n.01, physical_entity.n.01, object.n.01, whole.n.02, artifact.n.01, instrumentality.n.03, container.n.01, wheeled_vehicle.n.01, self-propelled_vehicle.n.01, motor_vehicle.n.01, car.n.01 ] >>> [synset.name for synset in paths[1]] [ entity.n.01, physical_entity.n.01, object.n.01, whole.n.02, artifact.n.01, instrumentality.n.03, conveyance.n.03, vehicle.n.01, wheeled_vehicle.n.01, self-propelled_vehicle.n.01, motor_vehicle.n.01, car.n.01 ] D. Rösner NSS I 2013/

17 Lexikalische Relationen in WordNet zu den weiteren lexikalischen Relationen in WordNet gehören: Meronymie (Ganzes-zu-Teil-Beziehung) Holonymie (Teil-zu-Ganzes-Beziehung) weitere Verfeinerungen: Teil-Meronyme: the parts of a tree are its trunk, crown, and so on" Substanz-Meronyme: the substance a tree is made of includes heartwood and sapwood" Member-Holonyme: a collection of trees forms a forest" für Verben: Entailment-Beziehung Antonymie bei Gegensatzpaaren D. Rösner NSS I 2013/

18 Literatur: I WordNet in NLTK Steven Bird, Ewan Klein, and Edward Loper. Natural Language Processing with Python Analyzing Text with the Natural Language Toolkit. O Reilly Media, Daniel Jurafsky and James H. Martin. Speech and Language Processing: An introduction to natural language processing, computational linguistics, and speech recognition. Prentice Hall, 2nd (May 26, 2008) edition, Christopher D. Manning and Hinrich Schütze. Foundations of Statistical Natural Language Processing. MIT Press, fifth printing edition, D. Rösner NSS I 2013/