Version classiqueVersion mobile

Formaliser les langues avec l’ordinateur : de INTEX à Nooj

 | 
Svetla Koeva
, 
Denis Maurel
, 
Max Silberztein

Deuxieme partie. Analyse de corpus

17. Extraction of named entities in Serbian using INTEX

Duško Vitas et Gordana Pavlovic-Lazetic

Texte intégral

Introduction

1The task of named entities recognition is defined at the MUC 1997, according to (N. Chinchor, 1997), as follows :

2On the level of entity extraction, Named Entities (NE) were defined as proper names and quantities of interest. Person, organization, and location names were marked as well as dates, times, percentages, and monetary amounts.

3Identification of the class of named entities such as dates, time periods, percentages and currencies can be successfully approximated in the INTEX environment by local grammars. One such approach is illustrated by the FST that recognizes and tags the dates and time periods in Serbian given in Example 1 in section 5.

4Proper names (abbr. NP) represent significant part of the large portion of texts being processed nowadays. The problem of their recognition is, however, more complex, and it may be approached by different methods, including recognition based on statistical models or using simple heuristics based on orthographic tradition of a specific language (e.g., capitalizing). Such methods, although usually provide for the extraction of significant amount of NP from a sample text neglect their language nature. Even when they are highly successful, explicit information about language function of a proper name is missing. There are thorough lists of such language units given in gazetteers, encyclopedias, phonebooks, etc. However, traditional grammars do not give much attention to personal names and named entities in general : in this way the problem of their lexicographic processing is neglected. In that respect, if only approximate methods are applied to NP recognition, their systematic processing is avoided from lexicographic point of view.

5On the other side, writing conventions for NP may differ in various languages. Methods giving good results for English are just rough approximation for the languages with rich morphology. The differences will be illustrated by the toponym New York, written basically in the same way in West European languages. However, in Serbian as a Slavic language with rich morphology (and unstable orthography), it may be recorded in a written text in at least a dozens of different ways. We will point some of these variations :

  1. New York may be written in different alphabets (Cyrillic or Latin) : Њујорк or Njujork. For Cyrillic alphabet different coding schemes are in use as well as for Latin alphabet (ISO 646 IRV, ISO 8859-2 or – 5, Win CP 1250/1, Unicode, etc). Moreover, in Latin writing digraph Nj has two interpretations, either as one grapheme corresponding to Cyrillic Њ, or a consonant group n+j. In Unicode this ambiguity is preserved as letter Nj can be recorded as a digraph using two codes or as a ligature using one code. Moreover, capital Nj can be written using both capital letters NJ or using only first capital Nj. The conclusion is that in Unicode there are four possibilities to record the Latin capital letter Nj (as well as letters Lj and ).

  2. New York is usually written transcribed (in accordance with the phonetic principle of Serbian orthography), but the original form is also in use. This phenomenon also represents one of the basic differences between Serbian and Croatian variants of Serbo-Croatian – practice of writing Croatian is opposite : original form is usually written, but it is possible to write in a transcribed way, too.

  3. At the inflectional level, the noun Njujork has its Inflectional paradigm including even the vocative form : Njujorče.

  4. At the derivational level, this noun has its relational adjective njujorški (with 14 different forms, not counting the possible graphemic variations, and 7*2*3 = 42 combinations of the basic grammatical categories). Relational adjective is written in lowercase so that among absolute synonyms dokovi Njujorka “docks of New York” and njujorški dokovi “New York docks” a complex derivational relation is established that can be resolved using local grammars. The names for the inhabitants of Njujork are derived, too : Njujorčanin “man inhabitant of New York”, Njujorčanka “woman inhabitant of New York” having their own Inflectional and derivational features (diminutive, augmentative, possessive and relational adjectives, etc).

6The mentioned variations are illustrated in the corpus of contemporary Serbian in which the influence of the used coding schemas has been neutralized by the usage of the internal coding scheme. In this corpus of approximately 23 million words, 68 different word forms related to the toponym New York occurred with the frequency 2 455, or 0.01 % of the total number of simple words in corpus. Out of this number, noun Njujork appears in 14 different inflectional and graphemic variants, relational adjective njujorški in 36, the inhabitant of New York in 9 (see Table 1).

Table 1 : The examples of the various inflectional and graphemic variants of Njujork

Table 1 : The examples of the various inflectional and graphemic variants of Njujork
  • 2 See the text in the Appendix 1.

7On the other side, translation of NP represents a specific problem in multilingual applications such as alignment or machine translation. In statistical methods for alignment at sentence level, proper names represent significant source of cognates. But in the previous example, New York is a compound proper name, while in Serbian it is a simple word which makes it difficult to identify them as cognates (C. Krstev, D. Vitas, 2005). An experiment of aligning Serbian and Croatian translation with a French original of the Jules Verne’s novel2 Le tour du monde en quatre-vingts jours showed that in the very first sentence the number of named entitiesoccur. Table 2 illustrates the variations of proper names when they are being transferred to the foreign language and to the closely related language. In row 1, noun godina “year” has been omitted in Croatian, in row 2 word order has changed, in row 3 proper name has been transcribed in Serbian but not in Croatian (still it has the genitive ending, etc.). A few strings that remained identical in all three texts are given in bold.

Table 2. The named entities in the first sentence of Le tour du monde en quatre-vingts jours

Table 2. The named entities in the first sentence of Le tour du monde en quatre-vingts jours

8Variations exhibited by proper names in Serbian also affect information retrieval. Namely, it is difficult to effectively match documents and query patterns due to morphological variations. A log file of one Belgrade search engine shows that a naïve user poses a query in nominative singular form forgetting that the relevant documents may contain a key word in a morphologically modified form. Among the most frequent key terms in the mentioned log file proper names (in nominative singular form) prevail, such as :

toponyms : Slovenija, Hrvatska, londonski klub, Apatinska pivara firms : Apatinska pivara, Telekom, Beopetrol, Sartid, NIS

9Among other queries in the same log file are plate u republikama bivše Jugoslavije “salaries in former Yugoslav republics” or Vojo Koštunica. In the later example the query contains a nickname of the current prime minister of Serbia instead of his first name Vojislav. The former query is more complex : the user expectes to obtain information about salaries in the region of former Yugoslavia, i.e., in Serbia, Croatia, Macedonia, Slovenia, Bosnia, etc, and not the document containing the literal noun phrase plate u republikama bivše Jugoslavije. Thus, it is not surprising that not even Google records any document matching this query (Figure 1). Still, meaning of the query is completely clear.

Figure 1 : Recall from Google in December 2004 for the query "Plate u republikama bivše Jugoslavije"

Figure 1 : Recall from Google in December 2004 for the query "Plate u republikama bivše Jugoslavije"

I. Named entities as unknown words

10Lexical recognition in Serbian is based on a system of morphological electronic dictionaries, following the model described in (B. Courtois, M. Silberztein 1989). These dictionaries have been described in (D. Vitas, C. Krstev et al, 2001), and processing is based on the system INTEX (M. Silberztein, 1993 and 2004). In the lexical recognition using the dictionary corresponding to a one-volume dictionary of Serbian, except for spelling errors, two significant classes of unrecognized words are distinguished (file err). One of the classes represents results of regular derivational processes described in (D. Vitas, 2004), (C. Krstev, D. Vitas, 2005) such as possessive and relational adjectives derived from nouns, verbal nouns etc. Another significant and richer class consists of proper names. Analysis of this phenomenon has been performed on a sample of fifteen issues of the web-edition of a Belgrade daily newspaper Politika of September 2000. This sample contains 741 573 tokens in total, 68 968 being different. There are 582 072 simple words in total, 68 925 of which are different simple words. Distribution of different words is as follows :

11• If only the basic DELAF dictionaries are included (without proper names), INTEX recognizes 55 581 simple words, while 13 344 tokens remain unrecognized, i.e., 19.36 % of the overall amount of different words. The lines at the beginning of the err file contain various types of proper names, including their regular derivation :

Aabebe Abazi Abazija ABC Abdel Abdela Abdelu Abdicxa Abdulnaser Abdurahama Abdurahman Abebe abebe Abej Abel ABELA Aberdarevoj ABM Aboridyinka Abraham Abramovicx Abrams Abrasxevicx abs Aca Ace ACEH Aceh Aceha Acehu Aco Acxim Acximovicx Acximovicxa Acximovicxeva Acximovicxu AD ADAC Adam Adamos Adamovicx Adamovicxa Adamovicxem Adamsa Adamu Adancyicx Adancyicxa adaptibilnih Adema Ademija Ademir adenoma adenomu Adevarul Adija Adila Adolfa Adrija Adyemovicx Adyicx AEK aerotransporta AFA AFL aformaciji AFP AFRICyKE africyke Africyki africyki africykih africykim africyko africykog africykom Afrikancima afrimisxe Afroamerikanac aganlijama Agim Agima agitator Aglike AGN agolu Agota AgresijaNATO agresijeNATO agresoraNATO agresorskih agresorskim agresorsko agroekosistemima Agroindustrijska agroindustrijskog Agronet agronomskih AGROS Agrouniverzal Agrouniverzala ahbab Ahena Ahicxima ahmarskom Ahmed Ahmicxe Ahmicxima ahmicxke ahmicxku Aho AIK Ajaks Ajaksove Ajdaha aje Ajerts Ajetovicx Ajetovicxa Ajetovicxev Ajetovicxu Ajgnerom Ajhel Ajks Ajksa Ajlen Ajlendu Ajnsxtajn Ajovi Ajres Ajresa Ajresu Ajvarusxa Akcident AKCIDENTA akcidenti akdemiji...

12In this example, unknown words that are not proper names – those given in bold face and italic – have different sources such as spelling errors (aformaciji instead of afirmaciji, prem agolu instead of prema golu), regular derivation (agresorski is a relational adjective of the noun agresor), lemmas missing from DELAS (adenom, agitator, ajvaruša) or foreign words (abebe from Amharic).

13• If the graph NomsPropres+.fst which approximats a proper name by a alphabetic string beginning with a capital letter is included in recognition, then 5 008 tokens (or 7.3 %) remain unknown. Lines at the beginning of the err file show are now :

ABC abebe ABELA ABM abs ACEH AD ADAC adaptibilnih adenoma adenomu AEK aerotransporta AFA AFL aformaciji AFP AFRICyKE africyke africyki africykih africykim africyko africykog africykom afrimisxe aganlijama agitator AGN agolu AgresijaNATO agresijeNATO agresoraNATO agresorskih agresorskim agresorsko agroekosistemima agroindustrijskog agronomskih AGROS ahbab ahmarskom ahmicxke ahmicxku AIK aje AKCIDENTA akcidenti akdemiji AKM...

14Such an approximation has several disadvantages. First of all, the category N+NProp is assigned to each and every unrecognized word beginning in a capital letter, which is not correct as such strings may also be possessive adjectives. Moreover, the recognized words lack any grammatical information such as case, number, and gender. Second disadvantage is that those proper names written in lower case (here, e.g., afrički “related to Africa”) have not been recognized, nor have the acronyms (ABC, AFP, etc). The third and the most significant disadvantage is that attributes determining “the meaning” of a proper name (toponym, hydronym, etc.) have not been assigned.

15• If lexical resources describing proper names are included, and then the graphs ABB+.fst and NomsPropres+.fst are applied (the first one recognizes capital letter strings, i.e., regular expression of the form <U><U>+ and appends to them the marker ABB), only 4 048 tokens (or about 5.8 %) remain unrecognized. Lines at the beginning of the err file now include mostly spelling errors or missing words from dictionary (in bold face) :

abebe abs adaptibilnih adenoma adenomu aerotransporta aformaciji afrimisxe aganlijama agitator agolu AgresijaNATO agresijeNATO agresoraNATO agresorskih agresorskim agresorsko agroekosistemima agroindustrijskog agronomskih ahbab ahmarskom ahmicxke ahmicxku aje akcidenti akdemiji...

16The only unrecognized named entities now are the forms of the relational adjective ahmićki that is derived from the toponym Ahmići, small town in Bosnia. On the other hand, the relational adjective afrički is now recognized since Afrika and all its derived forms are in the dictionary of proper names. The applied procedure has now associated the proper PoS, lemma, grammatical category, and semantic markers (such as +Top, +NProp, etc.) to the significant number of word forms that were before roughly recognized as named entities on the basis of the loose condition that they begin with an upper-case letter. Further approximations of unknown words can be successfully processed using derivational graphs (e.g., prefixes aero- or agro-, relational adjective agresorski “in the way of the aggressor”, etc.) as has been shown in (G. Pavlovic-Lazetic et al, 2004), (C. Krstev, D. Vitas, 2005). These results are in accordance with the analysis performed for French (D. Morel, 2004).

II. Lexical resources for proper names recognition

17The above analysis shows the significance of developing lexical resources encompassing different categories of proper names. At present, lexical resources aimed at NP recognition are grouped into several separate dictionaries of the DELAF type according to the model suggested for Prolinntex in [O. Piton, D. Morel, 2004) :

  • Geographic names in DELA-Top.dic

  • First (last) names of inhabitants of Serbia in DELA-First.dic (DELA-Last.dic)

  • Transcribed English first and last names in DELA-EN-First.dic and DELA-EN-Last.dic

18Besides that, the recognition of abbraviations is distributed between the usage of the special dictionaty Abbrev.dic and the approximation FST ABB+.fst.

1. The dictionary of toponyms

19Sources used for this version of the dictionary DELA-Top were the Geographic atlas used in the education in Serbia and Official census register of inhabited places in former Yugoslavia. Specifically, material covers, with various level of detail, several regions. A choice of proper names to be included in the dictionary has been made for each region of geographic atlas. The following geographic entities have been chosen : names of countries, official languages, capital cities, administrative divisions of common importance (e.g., US states), cities with more than 10 000 inhabitants in Serbia and Montenegro, or 50 000 in case of former Yugoslav republics, or 100 000 in other regions ; hydronyms such as lakes, swamps, rivers, which have been associated with a mouth-country in case they spread through more than one country, oronyms such as mountains, volcanoes, etc, if of importance for a casual person (for example, for a newspaper reader). Besides proper names collected in this way, the dictionary of geographic names contains also the names of inhabitants (together with the pejoratives, if exist), and the relational and possessive adjectives that are derived from them.

20With each entry in our DELA-Top dictionary morphologic, syntactic and semantic attributes are supplied. Morphologic attributes have the form of the Inflectional classes used for the simple words in DELAS for Serbian. Based on it, a part of DELAF-Top dictionary has been generated. Semantic attributes are in accordance with the corresponding codes in the Prolintex system. So, in this version of the DELA-Top dictionary, we use, among the others, the following general semantic markers :

  • Top (toponym, place) : Beograd, . N+Top : ms

  • Hum- human beings : Beogradxanin, . N+Hum : ms

  • IsoXX – ISO-country-code for toponyms : Beograd,.N+ IsoYU : ms

  • NProp – proper names : Beograd,.N+Top+NProp :ms

21and the following specific semantic markers as, for example :

  • Inh (the inhabitant) : Beogradxanka,.N+Hum+Inh :fs

  • Cen (regional center) : Niš,.N+Cen :ms

  • Drz (country) : Francuska,.N+Drz :fs

  • Gr1-Gr4, Ggr (cities of different size) : Sofija,.N+Top+Ggr+Gr4+IsoBG :fs

  • Kon (continent) : Evropa,.N+Kon :fs

  • Pla (mountain) : Rila,.N+PR+Oro+Pla+IsoBG :fs

  • Reg (region) : Sxumadija,.N+Top+Reg+IsoYU :fs

22An excerpt from the DELAF-Top is :

Beograd,Beograd.N1001+NProp+Top+Gr+IsoYU :ms1q :ms4q
Beograda,Beograd.N1001+NProp+Top+Gr+IsoYU :ms2q
Beograde,Beograd.N1001+NProp+Top+Gr+IsoYU :ms5q
Beogradom,Beograd.N1001+NProp+Top+Gr+IsoYU :ms6q
beogradska,beogradski.A2+PosQ+Top+Ggr+IsoYU :akms2g :akms4 v : (...) aenp5g
beogradske,beogradski.A2+PosQ+Top+Ggr+IsoYU :aemp4g :aefs2g : (...) aefp5g
beogradski,beogradski.A2+PosQ+Top+Ggr+IsoYU :adms1g :aems4 q : (...) aemp5g
beogradskih,beogradski.A2+PosQ+Top+Ggr+IsoYU :aemp2g :aefp2 g : aenp2g
(..................................................................)
Beogradu,Beograd.N1001+NProp+Top+Gr+IsoYU :ms3q :ms7q
Beogradxana,Beogradxanin.N60+Hum+NProp+Inh+IsoYU :mp2v
Beogradxane,Beogradxanin.N60+Hum+NProp+Inh+IsoYU :mp4v
Beogradxani,Beogradxanin.N60+Hum+NProp+Inh+IsoYU :mp1v : mp5v
(..................................................................)
Beogradxaninov,Beogradxaninov.A1+Pos+Hum+Top+PGgr+IsoY U :akms1g :aems4q
Beogradxaninova,Beogradxaninov.A1+Pos+Hum+Top+PGgr+IsoY U :akms2g (...) aen

23In writing proper names, several different approaches have been adopted in Serbian orthography. For the domestic geography, local official names of toponyms, hydronyms and oronyms are retained and strictly reproduced in writing. To the contrary, for foreign geography, official local names are rarely represented, and exonyms, basically traditional names, are predominantly used. In Serbian, there are quite a few local names for foreign names, that vary significantly from there originals, such as Beč (Wien), Rim (Roma), Solun (Thessaloniki), Prag (Praha) ; this phenomena is also present in writing proper names from former Yugoslav republics, such as Bitolj (Bitola), Skoplje (Skopje), etc.

2. Structural properties of DELA-Top entries

24In respect to inflectional properties, the most regular of all proper names are the names for woman inhabitants and the derived adjectives. Feminine inhabitants may be grouped into three groups :

  • Feminine inhabitants ending in -ka (94 %) belong to the Inflectional class N661 (distinguished by the use of the ending -i in genitive plural). Examples are : Beograđanka, Bugarka, Parižanka, Kanađanka (woman inhabitants of Belgrade, Bulgaria, Pariz, and Kanada, respectively) ;

  • Feminine inhabitants ending in -ica (2 %) belong to the Inflectional classe N651 (distinguished by the use of the ending -e in vocative singular). Examples are : Švabica, Sremica (woman inhabitant of Germany and Srem, region in Vojvodina) ;

  • Feminine inhabitants ending in -inja (4 %) belong to the Inflectional class N601 (the unmarked inflection) ; Examples are : Grkinja, Poljakinja, Turkinja (woman inhabitans of Greece, Poland, and Turkey respectively).

25There are two groups of adjectives derived from toponyms and the corresponding inhabitants : relational and possessive :

  • Relational adjectives, corresponding to toponyms, hydronyms and oronyms, make 60 % of the overall number of adjectives derived. They end in -ski, -ški, -čki, e.g., beogradski “related to Belgrade”, praški “related to Prague”, bečki “related to Vienna” and all belong to the Inflectional class A2. They are written in lower case.

  • Possessive, corresponding to inhabitants, make 40 % of the number of adjectives derived. They end in -in (for both genders), e.g., Beograđankin (f.) “belonging to the woman inhabitant of Belgrade”, Bečlijin (m.) “belonging to the man inhabitant of Vienna”, or in -ov, -ev (for masculine gender), e.g., Beograđaninov (m) “belonging to the man inhabitant of Belgrade”, Prisxtincyev (m) “belonging to the man inhabitant of Pristine”. This adjectives belong to the inflectional class A1.

26Nouns denoting the masculine inhabitants are much more versatile and belong to the various inflectional classes : N2u, e.g. Kosovar “the inhabitant of Kosovo”, N10, e.g. Poljak (plural Poljaci) “the inhabitant of Poland”, N14, e.g. Čeh (plural Česi) “the inhabitant Czech Republic”, N42 Afrikanac (plural Afrikanci) “the inhabitant of Africa”, N60, e.g. Beograđanin (plural Beograđani) “the inhabitant of Belgrade”, N741 Sarajlija (plural Sarajlije f.) “the inhabitant of Sarajevo”.

27Toponyms themselves have various morphological structures. They can be masculine Istanbul, neuter Skoplje “Skopje” or feminine Sahara. Many toponyms have the structure of relational adjectives, they are homonymous with them and this inflect according to that structure, e.g. Engleska “England”, has the feminine gender form of the relational adjective engleski “related to England”. There are pluralia tantum names, both for domestic names Dobrinci m. “small place in Serbia” and Pljevlja f. “town in Montenegro”, and foreign names Alpi “Alps” and Filipini “Philippines”.

3. The dictionary of personal names

28Dictionaries of personal names of inhabitants of Serbia have been extracted from the census data on inhabitants of Belgrade in 1993, and contain first and last names that occurred with the frequency grater than 10 in a population of about 1.700.000 persons. The names that occurred less then 10 times were ignored because of many errors. An excerpt from the dictionary of first names :

Aci, Aca. N1741+NProp+Hum+First+Nick+SR : ms3v : ms7v
Acika, Acika.N1685+NProp+Hum+First+Nick+SR :ms1v :ms5v
Acike, Acika. N1685+NProp+Hum+First+Nick+SR : ms2v
Aciki, Acika. N1685+NProp+Hum+First+Nick+SR : ms3v : ms7v
Acikom, Acika. N1685+NProp+Hum+First+Nick+SR : ms6v
Aciku, Acika. N1685+NProp+Hum+First+Nick+SR : ms4v
Acka, Acko. N1068+NProp+Hum+First+Nick+SR : ms2v : ms4v
Acko,Acko.N1068+NProp+Hum+First+Nick+SR :ms1v :ms5v
Ackom, Acko. N1068+NProp+Hum+First+Nick+SR : ms6v
Acku, Acko. N1068+NProp+Hum+First+Nick+SR : ms3v : ms7v
Aco, Aca. N1741+NProp+Hum+First+Nick+SR : ms5v

29Surnames in Serbian are rather uniform. Most of them (87 %) end in -ić and belong to the same morphological class N28, with the same inflectional and derivational properties. Excerpt from the dictionary of last names :

Abadicxem, Abadicx. N28+NProp+Last+SR : ms6v
Abadicxi,Abadicx.N28+NProp+Last+SR :mp1v :mp5v
Abadyicxa,Abadyicx.N28+NProp+Last+SR :ms2v :ms4v :mp2v
Abadyicxe,Abadyicx.N28+NProp+Last+SR :ms5v :mp4v
(..........................................................................................)
Kosxtunica,Kosxtunica.N679+Hum+NProp+Last+SR :ms1v :ms5v :fp2v
Kosxtunicama,Kosxtunica.N679+Hum+NProp+Last+SR :fp3v :fp6 v:fp7v
Kosxtunice,Kosxtunica.N679+Hum+NProp+Last+SR :ms2v :fp1v :fp4v:fp5v

30When recognizing the Serbian personal names, the additional syntactic conditions have to be taken into consideration. For example, while surnames in front of the first names are never inflected, surnames of female persons do not inflect after the first names either. So :

fem> Marjanovicx GordanI, Marjanovicx Gordana. NPROP : Ndsf
fem> GordanI Marjanovicx, Gordana Marjanovicx NPROP : Ndsf
masc> Marjanovicx MilosxU, Marjanovicx Milosx. NPROP : Ndsm
masc> MilosxU MarjanovicxU, Milosx Marjanivicx NPROP : Ndsm

31The additional agreement conditions are necessary when one last name is connected to two or more persons of different sex :

fem+masc>... Gordani i Milošu Marjanoviću (agree with Milošu)
masc+fem>... Milošu i Gordani Marjanović (agree with Gordani)

32This agreement conditions can be tested by using grammatical information on gender and semantic markers of names +First and +Last.

4. The dictionary of foreign personal names

33Dictionaries of toponyms and Serbian personal names are supplemented with a dictionary of English first and last names according to a transcriptional dictionary (T. Prčić, 1992). The marker +Val has been added to each transcribed name that connects it to the original form. The marker +Norm has been added to the names that are in use though not transcribed according to the rule. It is used to connect such names to their normative form. An excerpt from the dictionary of first names with English transcription :

Abi,Abi.N1064+NProp+Hum+First+EN+Val=Abbey+Val=Abbie :ms1v : ms4v
Abigejl,Abigejl.N1002+NProp+Hum+First+EN+Val=Abigail :ms1v
Abigejla, Abigejl. N1002+NProp+Hum+First+EN+Val=Abigail : ms2v : ms 4v
Abigejle, Abigejl. N1002+NProp+Hum+First+EN+Val=Abigail : ms5v
Abigejlom, Abigejl. N1002+NProp+Hum+First+EN+Val=Abigail : ms6v
Abigejlu, Abigejl. N1002+NProp+Hum+First+EN+Val=Abigail : ms3v : ms 7v
Abija, Abi. N1064+NProp+Hum+First+EN+Val=Abbey+Val=Abbie : ms2v
Adrianu, Adrian. N1002+NProp+Hum+First+EN+Val=Adrian+Norm=A drijan : ms3v : ms7v
Adrien, Adrien. N801+Const+NProp+Hum+First+EN+Val=Adrienne+No rm=Adrijen : fv
Adrijan, Adrijan. N1002+NProp+Hum+First+EN+Val=Adrian :ms1v

34The dictionary of transcribed English surnames is being developed according to the same principles.

III. Examples of application

35Various types of graphs that enable the recognition of named entities have been designed and applied to the corpus of daily newspaper Politika. Some of them are represented in the following examples.

36Example 1. Recognition and marking of dates in text, a task often encountered in information retrieval, is accomplished by using the graph DATUM. GRF that relies on subgraphs and lexical resources (Figure 2).

37An excerpt from the produced concordances is :

cyavajucxi sankcije 26. aprila 1999, EU je uvela ZABRANU VIZA za u manastiru Hopovu, 7. aprila 1757. godine, kada je carski komesar dolazio,
Luj Paster (1822-1895), francuski biolog i fiziolosxki hemicyar, cy
ublike br. 310/î od 27. 9. 2000. godine. Drugim, takodxe osporenim resxenxe
eventivne medicine. Godine 1990, ispred centralnog ulaznog prostora, otkriv
olzxenxicin napisao izmedxu 1950. i 1966, a medxu nxima su i najpoznatija "
em kojim je krenula januara 1904. godine, kada je pocyela da izlazi i kada
cykih ratova Srbije od 1912. do 1920. godine i potomaka sa generalnim sekre
h funkcionera SPO-a 3. oktobra prosxle godine. Prema stavu poslanika grupe
sednicu zakazanu za ponedelxak, 9. oktobar. Kako saznajemo od dr Milovana
: Blagodat tisxine (24. septembar) i Zvuk cyarsxije (prvi oktobar), pa je u
Sledecxeg dana, u sredu, 4. oktobra, sala Medija centra, ispunxena je do

Figure 2 : Graph extracting dates (subgraph “trenutak”) as well as time intervals*

Figure 2 : Graph extracting dates (subgraph “trenutak”) as well as time intervals*

* (subgraphs “OdDo”, “OdDoMesec” and “OdDoDan” – from-to)

38Example 2. The graph on the Figure 3 retrieves from the text the answers to the query Plate u republikama bivše Jugoslavije “salaries in the republics of former Yugoslavia”. It uses sub-graph Synset-Plata to retrieve all the synonyms of the noun plata and the subgraph ExYU to retrieve all the possible ways to refer to the former Yugoslavia as well as some of its constituents. The subgraph Synset-Plata is automatically produced from the set of synonyms for “salary” in Serbian Wordnet (Krstev et al., 2004). This graph allows the free order of this noun phrases as well as rather free insertions. As a result, a good recall and precisions were obtained, although the precision is not 100 %. In the following excerpt of the obtained results it can be seen that besides plata, synonyms nadnica and zarada occurred, while sub-graph ExYU found the numerous references to the former Yugoslav space : Srbija “Serbia”, Crna Gora “Montenegro”, Hrvatska “Croatia”, Slovenija “Slovenia”, Republika “Republic (meaning Serbia)”, SRJ “FRY”, SFRJ.

Figure 3 : The FST for the extraction of the relevant answers to the query “salaries in the republics of former Yugoslavia”

Figure 3 : The FST for the extraction of the relevant answers to the query “salaries in the republics of former Yugoslavia”

39An excerpt from the resulting concordances is :

bi ekonomsku cenu kroz pad plata i izvoza u onim sektorima gde bi Srbija bila
Prosjecyna novembarska plata u Crnoj Gori iznosila je 218 maraka.
je istog meseca prosecyna plata u Republici iznosila nesxto visxe od 11.500
Prosecyna oktobarska neto plata u Srbiji bila je 10.044 dinara, sxto je xuje U odnosu na prosek plata u Srbiji od 11.043 dinara – natprosecynu
procenili da je prosecyna plata u SRJ povecxana u septembru za oko 5,6 odsto i xte prakse uporedili su sa platama svojih kolega u zemlxama bivsxe SFRJ : u
Martovske prosecyne neto plate u Republici iznose 8.204 dinara, sxto je 3,5
put u istoriji, prosecyne plate u Srbiji postale vecxe od onih u Sloveniji
ika sxest preduzecxa RTB-a u Beograd bio postignut dogovor
poslovodstva borskog ga u zemlxama bivsxe SFRJ : u Hrvatskoj je plata lekara do 900 evra
verovali ili ne". Danas je u Srbiji nadnica tekstilnih radnika nizxa nego u
nxegovim recyima, opsxtine u Srbiji u kojima su prosecyne neto zarade, bez
i rang u VJ, obucyavaju se u SRJ i primaju platu iz Beograda. Izgleda da je i
podacima Zavoda, prosecyna zarada bez poreza i doprinosa koja je u aprilu ispla
a oktobar ove godine, neto zarada u Crnoj Gori porasla je za 0,5 odsto. Buducxi z posla, isplatu zaostalih zarada i nxihovo uskladxivanxe s platama zaposlenih
196 evra Prosecyna bruto zarada isplacxena u martu u Srbiji iznosi 18.681
govori prosecyna mesecyna zarada po zaposlenom koja je u nasxoj republici u
cxera. Prosecyna mesecyna zarada po zaposlenom u Srbiji u decembru 2003.
Prosecyna majska neto zarada zaposlenih u Srbiji realno je smanxena u
ominalno. Isplacxene bruto zarade zaposlenim u Srbiji u prvih sxest meseci ove

40Example 3. The graph in Figure 4 recognizes many cases of person’s official position followed by recognizing the following structures :

  • human noun preceded by optional adjectives, followed by optional preposition, toponym, and personal name(s) ; or

  • human noun followed by noun(s) in genitive, followed by optional preposition, toponym, and personal name(s).

41Concordances contain many examples of personal names completely identified by their roles (e.g., novi predsednik Jugoslavije Vojislav Kosxtunica “the new president of Jugoslavia Vojislav Kosxtunica” :

jem se cyelu nalazi bivsxi americyki ambasador u Hrvatskoj Vilijem Montgomeri.
zxe Jovanovicx. Cylan Predsednisxtva Sindikata Srbije Srbijanka Rafailovicx
Apelujem na novog, demokratski izabranog predsednika Jugoslavije Vojislava Kos
ork, 4. oktobra Guverner Teksasa Dyordy Busx Mladxi nedavno je izjavio kako
yestitao je danas u ime Saveta Evrope Vojislavu Kosxtunici na izboru za
razxila audijenciju kod predsednika Jugoslavije Vojislava Kosxtunice oktobra (Beta) Komandir policije u Mladenovcu Miroslav Anticx cyestitao je
rmat" prenosi da je ministar finansija Austrije Karl-Hajnc Graser izdao nalog
tilo je jucye da se nekadasxnxi predsednik Srbije Ivan Stambolicx "ne nalazi, nxegovim recyima, i novi predsednik Jugoslavije Vojislav Kosxtunica i, sada
Evropu Otpravnik poslova Italije u Jugoslaviji Dxovani Cxiracxolo u SRJ Predsednik Jugoslavije Vojislav Kosxtunica primio je minist je, jednim delom, i predsednik Republike Srbije Milan Milutinovicx. Medxu
ku SRJ Predsednik Rusije Vladimir Putin uputio je jucye poruku ornu volxu Predsednik vlade Italije Dxulijano Amato garantuje punu
uvecye telefonom sa predsednikom Jugoslavije Vojislavom Kosxtunicom pekulacije potvrdio premijer Vlade Srbije Mirko Marjanovicx u holu Skupsxtine,
pismo predsedniku Republike Belorusije Aleksandru Grigorijevicyu Lukasxenku,
tobra Generalni sekretar Saveta Evrope Valter Sxvimer cyestitao je danas u
ti ukinut tek nakon smene ministra policije Srbije Vlajka Stojilxkovicxa.
RJ Muftija Islamske zajednice Republike Srbije Hadyi Hamdija efendija

42In the concordances the recognized names for the official positions are in bold face, toponyms are in italic, and personal names are in small capitals.

Figure 4. The FST for extracting from text the various ways to denote the official positions

Figure 4. The FST for extracting from text the various ways to denote the official positions

Conclusion

43Named entities and especially their subclass consisting of proper names, form the typical queries submitted to Web search machines. Thus improvement of retrieval performance directly depends on available lexical resources describing this lexical fund.

44Recognition of named entities in a corpus is achieved using the INTEX system and lexical resources developed, such as local grammars and specialized dictionaries of geographic and personal names. Various problems in developing these lexical resources for Serbian are outlined, e.g., graphical variations, morphological and derivational complexity, unstable orthography. Then structure and contents of dictionaries of toponyms and personal (first and last) names, as well as foreign personal names, are represented, along with general and specific semantic tags sets.

45Examples of application of the lexical resources developed demonstrate that named entities recognition may be used as a powerfull mechanism for the various types of applications, e.g., information extraction.

46Both basic and proper names dictionaries will be further enhanced, and new dictionaries will be developed covering the complete proper names ontology (M. Tran et al, 2004) (e.g., dictionaries of celebrities, artworks, acronyms, organizations, etc.) Special attention will be paid to the development of dictionaries of compounds. Local grammars will be developed describing semantic-like relations, such as synonymy. Finally, a database structure will be defined for storing DELA dictionaries with possibilities of converting them into different formats.

Bibliographie

Bibliography

COURTOIS B., SILBERZTEIN M. (1990), Dictionnaires électroniques du français, Langues française, n° 87, p. 11-22.

CHINCHOR N. (1997), Muc-7 Named Entity Task Definition, http://www.itl.nist.gov.

KRSTEV Cvetana, VITAS Duško, STANKOVIĆ Ranka, OBRADOVIĆ Ivan and PAVLOVIĆ-LAŽETIĆ Gordana, (2004), “Combining Heterogeneous Lexical Resources”, in LINO M.T. & al. (eds.), IV International Conference on Language Resources and Evaluation LREC 2004, ELRA, Lisboa, p. 1103-1108.

KRSTEV C., VITAS D. (2005), “Extending Serbian E-dictionary by the Use of the Lexical Transducers”, [in this volume].

MAUREL D. (2004), “Les mots inconnus sont-ils des noms propres ?”, Septièmes Journées internationales d’Analyse statistique des Données Textuelles (JADT 2004), Louvain-la-Neuve, Belgique.

PAVLOVIC-LAZETIC G., VITAS D., KRSTEV C. (2004), "Towards Full Lexical Recognition", in P. SOJKA, I. KOPECEK, K. PALA (eds.), Text, Speech and Dialogue, TSD 2004, Lecture Notes in Computer Science 3206, Springer, p. 179-186.

PITON O., MAUREL D. (2004), “Les Noms Propres Géographiques et le Dictionnaire Prolintex", in MULLER Claude, Jean ROYAUTE, Max SILBERZTEIN (eds.), INTEX pour la linguistique et le traitement automatique des langues, Presses Universitaires de Franche-Comté, p. 53-76.

PRCIC T. (1992), Transkripcioni rečnik engleskih ličnih imena, Beograd, Nolit.

SILBERZTEIN Max D. (1993), Le dictionnaire électronique et analyse automatique de textes : Le systeme INTEX, Paris, Masson.

SILBERZTEIN Max D. (2004), INTEX Manual, v. 4.33. (http://intex.univ-fcomte.fr/downloads/Manual.pdf).

TRAN M., GRASS T., MAUREL D. (2004), An Ontology for Multilingual Treatment of Proper Names, LREC, OntoLex 2004, Lisbonne.

VITAS D. (2004), Morphologie dérivationnelle et mots simples : Le cas du serbo-croate. Lexique, Syntaxe et Lexique-Grammaire, Papers in honour of Maurice Gross, Lingvisticæ Investigationes Supplementa 24, John Benjamins, p. 629-640.

VITAS Duško, KRSTEV Cvetana, and Pavlović-Lažetić Gordana (2001), “The Flexible Entry”, in ZYBATOW G. et al. (eds.), Current Issues in Formal Slavic Linguistics, Leipzig, University of Leipzig, p. 461-468.

Annexes

Appendix

An excerpt from the Vern’s “Around the world in eighty days”

French :

<p><seg>En l’année 1872, la maison portant le numéro 7 de Saville-row, Burlington Gardens – maison dans laquelle Sheridan mourut en 1814 --, était habitée par Phileas Fogg, esq., l’un des membres les plus singuliers et les plus remarqués du Reform-Club de Londres, bien qu’il semblât prendre à tâche de ne rien faire qui pût attirer l’attention. </seg></p>

Serbian :

<p><seg>Godine 1872, u kucxi broj 7 u Ulici Sevil-rou Barlington Gardenz, u kojoj je 1816. godine umro Sxeridan, stanovao je gospodin Fileas Fog, jedan od najcyudnovatijih i najzapazxenijih cylanova londonskog Reform-kluba, iako je izgledalo da se on trudi da ne ucyini nisxta sxto bi moglo na njega privucxi pazxnju.</seg></p>

Croatian :

<p><seg>U ulici Saville row kod Burlington-Gardena, u kucxi broj 7, u kojoj je 1816. umro Sheridan, stanovao je 1872. Phileas Fogg esq. najcyudnovatiji i najistaknutiji cylan kluba Reform u Londonu, iako se cyinilo, da je nastojao da ne ucyini nisxta cyime bi mogao svratiti na sebe pozornost. </seg></p>75

Notes

2 See the text in the Appendix 1.

Table des illustrations

Titre Table 1 : The examples of the various inflectional and graphemic variants of Njujork
URL http://books.openedition.org/pufc/docannexe/image/27169/img-1.jpg
Fichier image/jpeg, 26k
Titre Table 2. The named entities in the first sentence of Le tour du monde en quatre-vingts jours
URL http://books.openedition.org/pufc/docannexe/image/27169/img-2.jpg
Fichier image/jpeg, 35k
Titre Figure 1 : Recall from Google in December 2004 for the query "Plate u republikama bivše Jugoslavije"
URL http://books.openedition.org/pufc/docannexe/image/27169/img-3.jpg
Fichier image/jpeg, 36k
Titre Figure 2 : Graph extracting dates (subgraph “trenutak”) as well as time intervals*
Légende * (subgraphs “OdDo”, “OdDoMesec” and “OdDoDan” – from-to)
URL http://books.openedition.org/pufc/docannexe/image/27169/img-4.jpg
Fichier image/jpeg, 11k
Titre Figure 3 : The FST for the extraction of the relevant answers to the query “salaries in the republics of former Yugoslavia”
URL http://books.openedition.org/pufc/docannexe/image/27169/img-5.jpg
Fichier image/jpeg, 11k
Titre Figure 4. The FST for extracting from text the various ways to denote the official positions
URL http://books.openedition.org/pufc/docannexe/image/27169/img-6.jpg
Fichier image/jpeg, 10k

© Presses universitaires de Franche-Comté, 2007

Conditions d’utilisation : http://www.openedition.org/6540

Rechercher dans OpenEdition Search

Vous allez être redirigé vers OpenEdition Search