1 Introduction

1The task of automated text simplification has been investigated within the NLP community for several years with a number of different approaches, from rule-based ones (Siddharthan, 2010; Barlacchi and Tonelli, 2013; Scarton et al., 2017) to supervised (Bingel and Søgaard, 2016; Alva-Manchego et al., 2017) and unsupervised ones (Paetzold and Specia, 2016), including recent studies using deep learning (Zhang and Lapata, 2017; Nisioi et al., 2017). Nevertheless, only recently researchers have started to build simplification systems that can adapt to users, based on the observation that the preceived simplicity of a document depends a lot on the user profile, including not only specific disabilities but also language proficiency, age, profession, etc. Therefore in the last few months the first approaches to personalised text simplification have been proposed at major conferences, with the goal of simplifying a document for different language proficiency levels (Scarton and Specia, 2018; Bingel et al., 2018; Lee and Yeung, 2018).

2Along this research line, we present in this paper an approach to perform automated lexical simplification for L2 learners, able to adapt to the user mother tongue. To our knowledge, this is the first work taking into account this aspect and presenting a solution that, given an Italian document and the user’s mother tongue as input, selects only the words that the user may find difficult given his/her knowledge of another language. Specifically, we detect and simplify automatically the terms that may be misleading for the user because they are false friends, while we do not simplify those that have an orthographically and semantically similar translation in the user native language (so-called cognates). In multilingual settings, for instance while teaching, learning or translating a foreign language, these two phenomena have proven to be very relevant (Ringbom, 1986), because the lexical similarities between the two languages in contact have proven to create interferences, favouring or hindering the course of learning.

3We compare our approach to the selection of words to be simplified with a standard frequency-based one, in which only the terms that are not listed in De Mauro’s Dictionary of Basic Italian¹ are simplified, regardless of the user native language. Our experiments are evaluated on the Italian-French pair, but the approach is generic.

2 Approach description

4Given a document D_i to be simplified, and a native language L₁ spoken by the user, our approach consists of the following steps:

1. Candidate selection: for each content word² w_i in D_i, we automatically generate a list of words which are orthographically similar to w_i. In this phase, several orthographical similarity metrics are evaluated. We keep the 5 most-similar terms to w_i.

2. False friend and cognate detection: for each of the 5 most similar words in W₁, we classify whether it is a false friend of w_i or not.

3. Simplification choice: Based on the output of the previous steps, the system marks w_i as difficult to understand for the user if there are corresponding false friends in L₁. Otherwise, w_i is left in its original form. When a word is marked as difficult, a subsequent simplification module (not included in this work) should try to find an alternative form (such as a synonym, or a description) to make the term more understandable to the user.

2.1 Candidate Selection

5A number of similarity metrics have been presented in the past to identify candidate cognates and false friends, see for example the evaluation in Inkpen and Frunza (2005). We choose three of them, motivated by the fact that we want to have at least one ngram-based metric (XXDICE) and one non ngram-based (Jaro/Winkler). To that, we add a more standard metric, Normalized Edit Distance (NED). The three metrics are explained below:

• XXDICE (Brew et al., 1996). It takes in consideration the shared number of extended bigrams³ and their position relative to two strings S₁ and S₂. The formula is:

where B is the set of pairs of shared extended bigrams (x,y), x in S₁ and y in S₂. The functions pos(x) and xb(S) return the position of extended bigram x and the number of extended bigrams in string S respectively.

• NED, Normalized Edit Distance (Wagner and Fischer, 1974). A regular Edit Distance calculates the orthographic difference between two strings assigning a cost to any minimum number of edit operations (deletion, substitution and insertion, all with cost of 1) needed to make them equal. NED is obtained by dividing the edit cost by the length of the longest string.

• Jaro/Winkler (Winkler, 1990). The Jaro similarity metric for two strings S₁ and S₂ is computed as follows:

where m is the number of characters in common, provided that they occur in the same (not interrupted) sequence, and T is the number of transpositions of character in S₁ to obtain S₂. The Winkler variation of the metric adds a bias if the two strings share a prefix.

where l is the number of characters of the common prefix of the two strings, up to four, and p is a scaling factor, usually set to 0.1.

6Each of these three measures has some disadvantages. For example, we found that Jaro/Winkler metric boosts the similarity of words with the same root. On the other hand, applying NED leads to several pairs of words having the same similarity score. As a result, two words that are close according to a metric can be far using another metric. To overcome this limitation, we balance the three metrics by computing a weighted average of the three scores tuned on a training set. For details, see Section 3.

2.2 False Friend and Cognate Detection

7As for false friend and cognate detection, we rely on a SVM-based classifier and train it on a single feature obtained from a multilingual embedding space (Mikolov et al., 2013), where the user language L₁ and the language of the document to be simplified L₂ are aligned. In particular, the feature is the cosine distance between the embeddings of a given content word w_i in the language L₂ and the embedding of its candidate false friends or cognates L₁. The intuition behind this approach is that two cognates have a shared semantics and therefore a high cosine similarity, as opposed to false friends, whose meanings are generally unrelated. While past approaches to false friend and cognate detection have already exploited monolingual word embeddings (St Arnaud et al., 2017), we employ for our experiments a multilingual setting, so that the semantic distance between the candidate pairs can be measured in their original language without a preliminary translation.

3 Experimental Setup

8In our experiments, we consider a setting in which French speakers would like to make Italian documents easier for them to read. Nevertheless, the approach can be applied to any language pair, given that it requires minimal adaptation.

9In order to tune the best similarity metrics combination and to train the SVM classifier, a linguist has manually created an Italian-French gold standard, containing pairs of words marked as either cognates or false friends. These terms were collected from several lists available on the web. Overall, the Ita-Fr dataset contains a training set of 1,531 pairs (940 cognates and 591 false friends) and a test set of 108 pairs (51 cognates and 57 false friends).

10For the candidate selection step, the goal is to obtain for each term w_i in Italian, the 5 French terms with the highest orthographic similarity. Therefore, given w_i, we compute its similarity with each term in a French online dictionary⁴ (New, 2006) using the three scores described in the previous section. The lemmas were normalized for accents and diacritics, in order to avoid poor results of the metrics in cases like général and generale, where the accented é character would be considered different with respect to e.⁵

11In order to identify the best way to combine the three similarity metrics detailed in Section 2.1., we compute all the possibile combinations of weights on 10 groups of 200 word pairs randomly extracted from the 1,531 pairs in the training set, and then keep the combination that scores the highest average similarity.

12In Table 1 we report the percentage of times in which the cognate or false friend of w_i in the training set would appear among the 5 most-similar terms extracted from the French online dictionary according to the three different scores in isolation: XX for XXDICE, JW for Jaro/Winkler and NED for Normalized Edit Distance. We also report the best configuration of the three metrics with the corresponding weight to maximise the presence of a cognate or false friend among the 5 most similar terms. We observe that, while the three metrics in isolation yield a similar result, combining them effectively increases the presence of cognates and false friends among the top candidates. This confirms that the metrics capture three different types of similarity, and that it is recommended to take them all into account when performing candidate selection: an approach where evey metric contributes to detecting false friend / cognate candidates outperforms the single metrics.

13For false friends and cognates detection, we proceed as follows. Given a word w_i in Italian, we identify the 5 most similar words in French using the 0.2-0.4-0.4 score introduced before. In case of ties in the 5th positon, we extend the selection to all the candidates sharing the same similarity value.

14Each word pair including w_i and one of the 5 most similar words is then classified as false friend or cognate with a SVM using a radial kernel trained on the 1,531 word pairs in the training set. For the multilingual embeddings used to compute the semantic similarity between the Italian words and their candidates, we use the vectors from Bojanowski et al. (2016)⁶ trained on Wikipedia data with fastText (Joulin et al., 2016). We chose these resources since they are available both for Italian and French (and several other languages). For the alignement of the semantic spaces of the two languages we use 22,767 Italian-French word pairs collected from an online dictionary.⁷

4 Evaluation

15We perform two types of evaluation. In the first one, the goal is to assess whether the system can correctly identify false friends and cognates in a text. In the second one, we want to check what is the difference between the terms simplified by a system with our approach compared with a standard frequency-based simplification system.

16For the first evaluation, we manually create a set of 108 Italian sentences containing one false friend or cognate for French speakers taken from the test set. On each term, we run our algorithm and we consider a term a false friend according to two strategies: a) if all 5 most similar words in French are classified as false friends, or b) if the majority of them are classified as false friends. Results are reported in Table 2.

17The evaluation shows that the two settings lead to two different outcomes. In general terms, the first strategy is more conservative and favours Precision, while the second boosts Recall and F1.

18As for the second evaluation, on the same set of sentences, we run our algorithm again, this time trying to classify any content word as being a false friend for French speakers or not. We evaluate this component as being part of a simplification system that simplifies only false friends, and we compare this choice with a more standard approach, in which only ‘unusual’ or ‘unfrequent’ terms are simplified. This second choice is taken by comparing each content word with De Mauro’s Dictionary of Basic Italian and simplifying only those that are not listed among the 7,000 entries of the basic vocabulary.

19This evaluation shows that out of 1,035 content words in the test sentences, our simplification approach based on a) would simplify 367 words, and 823 if we adopt the strategy b). Based on De Mauro’s dictionary, instead, 240 terms would be simplified. Furthermore, there would be only 76 terms simplified using both strategy a) and De Mauro’s list, and 154 overlaps for strategy b). This shows that the two approaches are rather complementary and based on different principles. This is evident also looking at the evaluated sentences: while considering frequency lists like De Mauro’s, terms such as accademico and speleologo should be simplified because they are not frequently used in Italian, our approach would not simplify them because they have very similar French translations (académique and spéléologue respectively), and are not classified as false friends by the system. On the other hand, vedere would not be simplified in a standard frequency-based system because it is listed among the 2,000 fundamental words in Italian. However, our approach would identify it as a false friend to be simplified because vider in French (transl. svuotare) is orthographically very similar to vedere but has a completely different meaning.

5 Conclusions

20In this work, we have presented an approach supporting personalized simplification in that it enables to adapt the selection of difficult words for lexical simplification to the native language of L2 learners. To our knowledge, this is the first attempt to deal with this kind of adaptation. The approach is relatively easy to apply to new languages provided that they have a similar alphabet, since multilingual embeddings are already available and lists of cognates and false friends, although of limited size, can be easily retrieved online.⁸

21The work will be extended along different research directions: first, we will evaluate the approach on other language pairs. Then, we will add a lexical simplification module selecting only the words identified as complex by our approach. For this, we can rely on existing simplification tools (Paetzold and Specia, 2015), which could be tuned to adapt also the simplification choices to the user native language, for example by changing the candidate ranking algorithm. Finally, it would be interesting to involve L2 learners in the evaluation, with the goal to measure the effectiveness of different simplification strategies in a real setting.

Acknowledgments

22This work has been supported by the European Commission project SIMPATICO (H2020-EURO-6-2015, grant number 692819). We would like to thank Francesca Fedrizzi for her help in creating the gold standard.