Introduction

1Besides lexical and morphological analysis, INTEX is capable of performing higher level linguistic analysis. The final goal of my work is to develop a complete tool chain under INTEX for analyzing Hungarian corpora. The analysis would include tokenization, sentence splitting, lexical / morphological analysis, recognition of multiword expressions and named entities (as a part of the tokenization), disambiguation and shallow syntactic analysis.

2As INTEX functions are available not only via the interface but also as command-line functions, I decided to build up a batch file from these functions which would perform all the above mentioned tasks, and store the output of each step in a separate file. The input of the processing is raw Hungarian text, the output contains all the syntax-related annotation of the sentences, while lexical and morphological information are coded in the dictionaries. The tool would enable users to annotate their texts and run queries on its elements according to their syntactic function.

3My paper focuses on two subtasks : syntactic analysis and named entity recognition. Both rule systems have been developed and tested on a corpus of short business news consisting of 251.000 words.

I. Named entity recognition

1. The role of named entity recognition

4Named entity recognition and classification (NERC) is widely considered as a major subtask of information extraction [Evans, 2003]. The task of extracting information from natural language texts consists in recognizing and identifying different types of entities, events and relations between them. Entities in question are frequently expressed by NPs that contain named entities. However, any language processing application that requires higher level linguistic analysis needs named entity recognition, as these elements belong to an open class of tokens, which means that they are not part of the finite vocabulary of a language. Thus, they have to be treated differently at a lower level of analysis. Moreover, they constitute a problem not only for morphology but also for syntactic analysis, since structure of noun phrases with a named entity tend not to obey the same rules that operate on “normal” NPs. For example, Hungarian grammar does not contain a productive rule of the form NP ➔ N N N, but the string “Neumann János Szakközépiskola” (Neumann János Technical College) is still a well-formed NP. Foreign proper names are even more difficult to process on both morphological and syntactic level, still they can function as NPs in Hungarian sentences. Accordingly, syntactic parsing needs to be completed by a set of rules aiming at recognizing the boundaries of named entities. Another task that needs to be performed is categorization of named entities by providing them with semantic features (such as +animate, +place, +institution. These features are relevant for nominal elements because they are used in verbal argument structure descriptions.

2. Structure of the NERC rules

5As a first step of the NERC task, I handled the following types of named entities : people’s names, places, and institutions2. According to my basic assumption, these multiword expressions usually consist of one or more words belonging to an open class, and the so-called anchors that can be listed for any NE type. Such anchors are for example prefixes like Dr or Ifj (Jr) for people’s names, or utca (street) for places. The method I adopted for the recognition and classification of named entities builds upon the distinction between these two types of NE constituents. The process uses INTEX functions, namely dictionary lookup and transducers of both types (syntactic and word-level transducers). INTEX makes use of two kinds of dictionaries : delaf dictionaries for simple word units and delacf dictionaries for compound words. Lexical entries may also be formalized by transducers, which method allows of describing open classes of tokens. Apart from dictionaries and transducers, I also made use of a manually precompiled list of named entities, stored in a compound word dictionary, which is being expanded continually.

6Closed-class anchors have been listed for every NE type to be dealt with, and stored in special delaf and delacf dictionaries. Apart from their word form and lemma, dictionary entries contain semantic features that identify corresponding NE class(es) and some information about the syntactic properties of the anchors. In most cases, the anchor unequivocally identifies the NE type of the expression containing it, but it is not always the case : for example, people’s names may occur in institution’s names, but in such cases the institution itself is named by another anchor. Single word NEs that belong to an open class are treated in parallel, by word-level transducers. They describe names derived from other named entities, for example women’s names which are formed by adding a suffix to the husband’s name.

7When the grammar is applied to a text, first dicos.exe searches for the anchors and provides them with the features which are associated to them in the delaf dictionary. Morphological transducers are applied in parallel to this procedure. When single word NEs and anchors are already recognized, the next task to perform is to find the edges of the NE in the text, i.e. to identify the words around the anchors that belong to the expression. This task is performed by regular grammars compressed to finite state transducers. I use the dicoe function which applies finite state transducers to the text the way that their output is not inserted to the text but stored in a special kind of dictionary, the so called dle file. This solution is useful for the further steps of text processing, since recognized NEs can be treated later as one lexical unit. Another advantage of this method is that strings marked as NEs by the grammars are listed in one single file, hence they can be manually checked, and correct entries can be added to the multiword named entity dictionary.

8The last part of the procedure is the annotation of NEs in the text. This is completed by the etiqc.exe program, which takes as input the dle file with all the named entities, and marks them up in the text, indicating their NE type and the subgroup they belong to. Alternatively, we might insert a path to the noun phrase graph, which would be supposed to annotate NEs as NPs, so that they could be treated similarly. The choice between the two solutions depends on further applications.

Figure 1 : Grammar for people’s names

II. Syntactic analysis

1. Structure of the grammars

9Syntactic analysis is performed by a set of cascaded local grammars which do not aim at building a complete parse tree for each input sentence, but to capture dependency relations between main constituents of the sentence. The grammar is able to determine basic syntactic relations even if some part of the input string is not recognized by it.

10Verbal predicates are considered to be the central element of the clause. Dependency relations to be covered are predicate-argument and predicate-free modifier relations. The assumption behind this method is that predicates enter in dependency relations with (heads of) fully built phrases. Hence, the above-mentioned analysis can be decomposed to the following subtasks :

1. finding and labeling phrases,

2. finding clause boundaries,

3. identifying the elements that belong to the verbal argument structure,

4. identifying free modifiers, negation words, and sentence adverbs.

11Tasks 1-4) are performed by regular grammars organized in a pipeline process. Grammars are represented by INTEX graphs whose output is linguistic annotation (labeled parenthesis completed by attributes that describe either properties specific to the given phrase or inter-phrase dependencies).

12All of the grammars refer to the annotation which was inserted to the text by the preceding grammars, thus all information of lower level grammatical analysis have to be accessible for syntactic graphs. Although the grammar may lose of its flexibility by rendering it more difficult to insert new rules, this disadvantage is highly compensated by the fact that any of the rules may refer to any kind of earlier obtained information. This also reduces the risk of destroying the full analysis of the sentence by leaving a string unanalyzed at a lower linguistic level, because such unanalyzed strings may still serve as a valid input for subsequently applied local grammars. Another useful feature of this method is that rules can easily modify the output of the lower level grammars if necessary.

2. The steps of the analysis

a. Prase grammars

13The starting point of the syntactic analysis is the cascaded application of six local grammars responsible for the recognition and labeling of phrases that may be potential verbal arguments in the sentence : Noun Phrases, Adjectival Phrases, Postpositional Phrases and Infinitival Phrases. They are all characterized by bound word order, and their head occupies the rightmost position within the phrase. First, the grammars responsible for finding and marking the edges of the phrases are applied to the text. As NPs show the biggest complexity among the potential argument phrases, their treatment requires the most complex grammars. The NP grammar is divided into two subgraphs : one of them describes ‘simple’ NPs (including possessive structures), while the other one – which, in turn, contains the first one as a subgraph – is designed for recognizing NPs that contain a participial structure. The complexity of this graph is due to the fact that participles may be extended by almost any sort of arguments or modifiers that the base verb of the participle can have. However, as they occur inside an NP their word order is bound, and their structure can be described by local regular grammars.

14Recognition and annotation of noun phrases is performed in three steps : first, the two grammars find left and right edges of the phrase. In a second stage, the relevant features of the head are percolated up to the phrase. These features are number, person (agreement features), case and semantic features. They are all used in verbal argument specifications, hence they are needed so that verbal argument frames could be matched later. As the head is the rightmost constituent of the phrase, and all of these features’ values are inherited directly from the head3, the grammar simply copies the head’s feature values to that of the phrase. The third operation of the NP grammar serves the recognition of coordinated NPs. The difficulty of this task is induced by the fact that conjunctions which occur between coordinated NPs may also connect clauses. Therefore, the grammar has to check the output of the NP annotation, more precisely the value of the case feature : only NPs with the same case suffix are marked as coordinated NPs.

Figure 2 : A subgraph of the NP grammar : possessive structure

b. Clause boundary grammars

15As a sentence may consist of more than one clause, each of them having its own predicate with its arguments, it is a necessary step to find and mark clause boundaries.

16I consider clauses as the domain within which verbal argument frames have to be matched, although clauses themselves may also play the role of a complement.

17Clause boundary detection is a problematic task for several reasons. Since in Hungarian not all clauses contain a finite verb, and the order of phrases within the clause is free, it did not seem to be feasible to describe the structure of the Hungarian clause as such. Therefore I decided to try to describe the local context of the boundary itself. As a working hypothesis, I accepted that there is a finite set of clause boundary markers (i.e. punctuation marks, conjunctions, certain adverbs and pronouns) whose occurrences (or co-occurrences) reveal the exact location of the boundary. In [Gábor, Héja, Mészáros 2004], we accomplished a corpus-based analysis of the boundary markers that resulted in a classification of the conjunctions, adverbs and pronouns according to the position they occupy in the clause. Clause boundary detection rules use this information. They consist of cascaded local grammars referring not only to the boundary marker elements but to the syntactic annotation produced by the earlier steps of the process. They are all organized around the types of probable boundary markers. The following grammars are included :

18Relative clauses, introduced by a relative pronoun (optionally preceded by a comma)

19Clauses introduced by one conjunction. If the conjunction is preceded by a comma and it is not inside an already marked phrase, the clause boundary annotation is inserted before the conjunction. However, some conjunctions do not occupy the first position in the clause but may be preceded by one (top-level) constituent – in these cases, the annotation is inserted between the comma and the phrase.

20Citations. They are followed by a dash, which is directly followed by a finite verb (e.g. declare, announce, report etc.)

21More than one adjacent conjunctions, or one conjunction and one question word : they are labeled as boundary markers if the conjunction has not been recognized by earlier grammars.

22Question words, if they co-occur with a comma – adjacently, or with at most one top level constituent placed between them – are also exact boundary markers.

23Sentence boundaries are naturally considered as clause boundaries. They get the same annotation as further syntactic rules do not differentiate between them.

24Past and present participles are not considered as predicates in my grammar, thus they do not form a clause-like domain with their arguments. On the other hand, adverbial participles and their arguments may produce a structure which is similar to clauses with a finite predicate in that they can be expanded by other clauses. These clauses could not be linked to the finite predicate of the main clause, as they are in predicate-argument relation with the adverbial participle. For example :

Nemzetközileg egységes könyvviteli normák általános bevezetésére tett javaslatot a PriceWaterhouseCoopers, [clauseboundary g)]4 azt remélve, [clauseboundary b)] hogy ily módon visszaszerezhető a befektetők bizalma.

Price WaterhouseCoopers made a proposal for introducing international accountancy standards, [clauseboundary] hoping [clauseboundary] that this will help them regain the investors’ confidence.

25Finally, the grammar is supplemented by a “guesser rule”. As I supposed that there is always a clause boundary between two finite verbs in a sentence, but some of them are still uncovered by ag), it is necessary to examine all the points in the domain which are susceptible to be boundaries. Thus, this rule operates on sequences between finite verbs which do not contain any clause boundary annotation yet, and it marks every possible boundary (i.e. punctuation marks and conjunctions which are not inside phrases).

c. Verbal argument structure grammars

26I intended to use the lexicon-grammar function of INTEX for the recognition of the predicate’s argument structure. This task is based on a valence structure database, developed at our department between 2001 and 2003, containing over 17.400 different argument structures for 9.500 verbs. Its vocabulary is based on a frequency list from the Hungarian National Corpus [Váradi 2002]. The bottom line of this task is to find and mark the arguments of the predicate of the clauses with a finite verb. In other words, the goal is to create a grammar that identifies the clauses of the input text with one record of the argument structure database.

27Description of valence frames is founded on a dependency grammar approach. Three types of relations were defined between the verb and its arguments : there can be one subject, one direct object, and at most two other arguments that are simply labeled as complements. The database encodes morphosyntactic and semantic information which mostly relate to the head of the complement phrase. The selection criteria for differentiating between complements and adjuncts were not transparent syntax and / or semantics. Complements can be of one of the following categories : NP / AP / INFP (infinitival phrase) / ADV / POSTP (postpositional phrase) / CLAUSE. Apart from infinitives, all of the complements is inflected for case, but only nominal complements may have semantic features, which are encoded in an other database.

28As Hungarian language presents some peculiarities which have a relevant impact on syntactic analysis, I will briefly present its most important features.

29Hungarian is a highly Inflectional language with 18 cases and a (roughly) free word order : this means that almost any ordering of the verb and its expansions is acceptable. As described in [É.Kiss 2002], in the neutral sentence verbal complements and adjuncts follow the predicate within the VP. However, in most sentences, we find at least one verbal complement in a position preceding the verb – this is the topicalized constituent, which is raised to the leftmost position in the sentence. The function of the topic is to foreground an entity as subject of the predication in the sentence.

30Another syntactic movement that changes neutral word order is focusing : the focus position is the one immediately preceding the finite verb. Focus serves to exhaustively identify the entities for which the predicate holds. Any type of complements or adjuncts can be topicalized and focused, hence moved outside the VP. Furthermore, there is a third distinguished position before the verb : that of the verb modifier. This position is most frequently filled by a verbal prefix or an adverb, but bare NPs with no determiner may occupy it as well. In this case they function as modifiers – they do not refer to an individual, but specify a circumstance and precise the meaning of the verb.

31Because of the above mentioned movement opportunities, we can state that most verbal complements and adjuncts are free to appear before or after the predicate. When parsing Hungarian texts, one has to face the difficulty of being unable to determine dependency relations and grammatical functions on the basis of constituent order. Moreover, we can say that any ordering of the verb and its constituents is acceptable, although different orders serve to express different structuring of the same information.

32Another problematic issue for shallow parsing is that not only the respective order of verbs and complements is free, but there may also be optional adjuncts between them, which are left unanalyzed by the earlier grammars5.

33The first experiments with the argument-pattern matching were carried out using the lexicon-grammar function of INTEX. This function is designed for handling phenomena at the intersection of lexicon and syntax. A lexicon-grammar consists of a table whose cells are filled up with the same kind of grammatical information as syntactic graphs, and a master graph whose nodes – or at least some of them – refer to the cells of the table. During compilation, each record of the table is confronted to the master graph, and it results in a transducer which contains all the data from the table with all the paths from the graph [Siblerztein, 2004].

34In order to be able to use this function, the database had to be converted to INTEX format, and a master graph had to be designed. The graph’s paths had to cover the following structures (a path corresponds to one possible order of the constituents) :

• a verb with a subject and no other complements (2 paths),

• a verb with a subject and an object (6 paths),

• a verb with subject, object and one more complement (24 paths),

• a verb with subject, object and two more complements (120 paths),

35and, in every case, there may be any number of words (but no clause boundaries) between them. This yields a master graph with 152 paths. Although compilation of the lexicon grammar worked with a small test set taken from the valence database, it failed when I tried to compile it with all the 17.400 argument structures.

3. Future work

36A possible solution to the problem of argument structure recognition would be to decompose this step of the analysis into more subtasks. As a first step, a filter grammar could select possible complements, and only those elements would be handed over to the next level. Then a separate graph for each verb’s each argument structure would be generated, for every possible word order. These graphs do not need to be compiled into a single master graph, they can be applied one after another from a command line script. As they are lexically restricted patterns they probably would be matched faster. If it is not the case, the further step of the decomposition would be to chop up the graphs into as many subgraphs as the verb has arguments, where each subgraph would belong to one of the arguments. Thus, there would be one subgraph for the subject, another for the object, and two more for other complements, if any. These graphs would contain exactly two paths, because there are two possible orderings for two constituents. The problem with this method is that sometimes a given verb can only have one of its arguments if another one is present, i.e. they are not independent of each other. Although this problem concerns a smaller set of verbs, it is still not marginal, because the most frequent verbs have a lot of different argument structures. To avoid their mishandling, these verbs can be taken out and treated separately in a small lexicon-grammar.