• Contenu principal
  • Menu
OpenEdition Books
  • Accueil
  • Catalogue de 16075 livres
  • Éditeurs
  • Auteurs
  • Facebook
  • X
  • Partager
    • Facebook

    • X

    • Accueil
    • Catalogue de 16075 livres
    • Éditeurs
    • Auteurs
  • Ressources numériques en sciences humaines et sociales

    • OpenEdition
  • Nos plateformes

    • OpenEdition Books
    • OpenEdition Journals
    • Hypothèses
    • Calenda
  • Bibliothèques

    • OpenEdition Freemium
  • Suivez-nous

  • Lettre d’information
OpenEdition Search

Redirection vers OpenEdition Search.

À quel endroit ?
  • Accademia University Press
  • ›
  • Collana dell'Associazione Italiana di Li...
  • ›
  • Proceedings of the Seventh Italian Confe...
  • ›
  • Contributed Papers
  • ›
  • How good are humans at Native Language I...
  • Accademia University Press
  • Accademia University Press
    Accademia University Press
    Informations sur la couverture
    Table des matières
    Liens vers le livre
    Informations sur la couverture
    Table des matières
    Formats de lecture

    Plan

    Plan détaillé Texte intégral 1. Introduction 2. Related work 3. Test Description 4. Conclusion Bibliographie Notes de bas de page Auteurs

    Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020

    Ce livre est recensé par

    Précédent Suivant
    Table des matières

    How good are humans at Native Language Identification? A case study on Italian L2 writings

    Elisa Di Nuovo, Elisa Corino et Cristina Bosco

    p. 164-170

    Résumé

    In this paper we present a pilot study on human performance for the Native Language Identification task. We performed two tests aimed at exploring the human baseline for the task in which test takers had to identify the writers’ L1 relying only on scripts written in Italian by English, French, German and Spanish native speakers. Then, we conducted an error analysis considering the language background of both test takers and text writers.

    Note de l’éditeur

    We would like to thank the anonymous reviewers as well as our TTs, who have dedicated their time to us by carrying out these long tests which require a lot of concentration: E. L. Baratono, C. Borge, C. Borgia, C. Bosco, V. Calabria, S. Cicillini, A. T. Cignarella, C. Conti, V. De Iacovo, G. Esposito, K. Florio, G. Giaccone, A. Giacosa, L. Giannelli, L. Inserra, I. Iubini, A. Marra, M. Pellegrini, S. Peyronel, F. Poletto, S. Racca, A. Rotolo, M. Sanguinetti, E. Truc, and M. C. Zaccone.

    Texte intégral Bibliographie Notes de bas de page Auteurs

    Texte intégral

    1. Introduction

    1Native Language Identification (NLI) is a task usually performed by machines consisting in identifying the mother tongue (henceforth L1) of a person based only on their writings in another language (e.g. L2 or L31). To date, the majority of the existing studies have focused on English as L2, that is English used by people who are acquiring English as a second or foreign language (Tomokiyo and Jones 2001; Koppel, Schler, and Zigdon 2005; Malmasi, Tetreault, and Dras 2015; Kulmizev et al. 2017; Markov et al. 2017; Cimino and Dell’Orletta 2017 among others). Three editions of the NLI shared task had been organized (Tetreault, Blanchard, and Cahill 2013; Schuller et al. 2016; Malmasi et al. 2017) in which systems had to correctly identify the L1 among 11 L1s.

    2The basic assumption of this task is that when we learn a new language (henceforth Target Language, TL), our L1 interferes within the learning process introducing in the TL productions clues that can be automatically detected. Nevertheless, another issue to be investigated within this task is the interference in the TL learner’s productions also of other languages previously learned as L2 or L3. In fact, L1 may not be the only language playing a role in the acquisition of a TL, since “bi- or multilingualism is as frequent in the population of the world as pure monolingualism, perhaps even more frequent” (Hammarberg 2001, 21). This issue is especially relevant performing the NLI task in languages other than English. For instance, when someone learns Italian, it is likely their L3, since English is the language taught worldwide as L2 (with more that 100 million learners, as stated in the British Council annual report 2018-19).

    3In this paper, we investigate the human performance for NLI applied on productions of learners Italian, thus focusing not only on the issues related to second language acquisition (Ellis 2015; Slabakova 2016), but also to third language acquisition (Cenoz, Hufeisen, and Jessner 2001; Picoral 2020). We asked human Test Takers (TTs) to perform a simplified NLI task on learner Italian scripts extracted from VALICO (Corino and Marello 2017), selecting only four L1s2—i.e. English (EN), French (FR), German (DE) and Spanish (ES). This simplified task will be challenging since all the selected languages are Indo-European languages sharing typological similarities. Moreover, we performed an error analysis of the test results considering learners’ L1 and L2(s) and learning stage (i.e. year of study), in addition to text features and TTs’ language background. Test results could be useful for the improvement of the error annotation scheme introduced in Di Nuovo et al. (2019).

    4Our research questions therefore are:

    1. How good are humans in performing the NLI task?

    2. What features share the most problematic texts?

    5We try to answer to these questions in this paper organized as follows: in Section 2 we briefly describe previous work on the subject; in Section 3 we describe the tests performed and discuss the results; in Section 3.2 we conduct the error analysis; and in Section 4 we conclude the paper.

    2. Related work

    6To the best of our knowledge, the first study assessing the human ability in identifying members of the same native language group is that of Ioup (1984). She demonstrated that native speakers were able to identify these groups only when relying on phonological cues, supporting the assumption that “syntactic errors in L2 acquisition cannot be accounted for as a result of the transfer of L1 patterns” (Broselow and Newmeyer 1988, 197).

    7Odlin (1996) proved instead that readers who know the observed L1s (Korean and Spanish) can distinguish certain syntactic patterns in L2 English texts. Nowadays, his hypothesis is supported by the improved accuracy of machine learning systems using syntactic information (Malmasi, Wong, and Dras 2013; Markov et al. 2017 among others), such as PoS tags, Context-free Grammar Production Rules, Tree Substitution Grammar fragments, Grammatical Dependencies (see Malmasi (2016) for more details).

    8A more recent study simplified the NLI task to be performed by humans (Malmasi, Tetreault, and Dras 2015). The authors selected from TOEFL 11 (Blanchard et al. 2013)3 30 texts (6 per each of the 5 languages included: Arabic, Chinese, German, Hindi, and Spanish), 15 considered easy and 15 hard, according to the ease with which most systems predicted the L1. They chose ten raters, all professors or researchers who might have had exposure to the 5 selected L1s. The average accuracy achieved by raters (37.3%, top rater 53.3% and lowest 30%) shows how difficult the task can be for humans. Their approach does not give attention to text features and writer’s language background that, in addition to experts’ knowledge of the 5 L1s involved, could have had an impact on the task performance.

    9Two NLI experiments on English L2 performed by humans are also reported by Jarvis et al. (2019). In both cases, humans were asked to identify in L2 English texts writers of their same L1. In the first experiment, six Finnish speakers were asked to identify the author of the L2 English text as being a Finnish or Swedish native speaker. In the second experiment, in addition to the six Finnish raters, participated ten Spanish-speaker raters and all had to identify if the writers’ L1 shared their same L1 (Spanish or Finnish). The features that lead the raters to their decision were used to discuss the results achieved (in the second experiment, over 80% accuracy for the top Finnish raters and over 90% for the top Spanish raters). It is important to note that the accuracy achieved in Malmasi et al. (2015) and Jarvis et al. (2019) is not comparable since the experiment settings are completely different.

    10In this paper we describe experiments which are more similar to that carried out by Malmasi et al. (2015). However, differently from the papers described, we focus on data extracted from the VALICO corpus and in Italian, a language for which there are no previous studies about human NLI (see Malmasi and Dras for a multilingual approach to NLI in which they developed also a system for L2 Italian, using precisely texts collected from VALICO).

    3. Test Description

    11We asked our TTs to perform two tests. The first (Test 1) is a simplified 4-class NLI test and the second (Test 2) a sort of guided Logistic Regression.

    12To Test 1 participated 25 TTs, 11 of them to Test 2.4 They are all Italian native speakers and were selected according to their knowledge of Italian as L2, foreign languages or linguistics. Our average TT has a master’s degree in a field related to Linguistics or Foreign Languages and speak on average two additional languages among the selected L1s.

    13The selected L1s—FR, DE, ES and EN—are the most commonly taught in Italy, so theoretically it would have been easier to find human experts knowing them5. In addition, these four languages represent two different families: while FR and ES belong to Romance languages like Italian, EN and DE belong to Germanic languages. For this reason we believe the task, although simplified (because constrained to only four languages), challenging enough. Furthermore, we expect different transfer patterns from the speakers of the two families.

    14Test 1 is a simplified NLI task, namely a multiple-choice test made of 48 questions. Each question contains a short text written in Italian by a non-native speaker and the TT had to identify the writer’s L1 choosing it between EN, FR, DE and ES. Texts were randomly selected from VALICO-UD (Di Nuovo et al. 2019) with an almost balanced distribution with respect to the L1: 11 EN, 14 FR, 10 DE and 13 ES—TTs were not aware about this distribution. Their length ranged from 58 to 484 words (mean length 136.19 words, standard deviation (SD) of 64.73 words). There were no statistically significant differences in length between the L1 groups as determined by one-way ANOVA (F = 2.24, p = .09).

    15Test 2 consisted of 24 texts drawn from Test 1 according to the difficulty human TTs had in identifying the correct L1. The TTs were asked to assign a percentage to each of the four L1s involved, performing a sort of guided Logistic Regression: the higher the percentage assigned to a language, the higher chance of being the writer’s L1.

    16Table 1 resumes the information about the number of texts written by EN, FR, DE, ES native speakers (# Text/L1), and the percentage of TTs knowing that L1 (TT/L1) for both Test 1 and 2.

    Table 1: Test 1 (25 TTs) and 2 (11 TTs) in figures

    —

    L1

    # Text/L1

    TT/L1

    *Test 1

    EN

    11

    100%

    FR

    14

    72%

    DE

    10

    28%

    ES

    13

    56%

    *Test 2

    EN

    5

    100%

    FR

    10

    82%

    DE

    7

    36%

    ES

    2

    73%

    3.1 Test Results and Discussion

    17The best result on Test 1 was achieved by TT1, correctly identifying the L1 of 26 out of 48 texts (54.2% accuracy), the lowest by TT25, correctly predicting the L1 of only 10 texts (20.8% accuracy). TT1 and TT25 are both PhD students in Digital Humanities: TT1 speaks EN and ES, while TT25 EN and DE. Table 1 shows score, accuracy (Acc.) and TTs’ teaching experience (TTE)6.

    18These results suggest that, in our sample, teaching experience as well as knowing all the four L1s involved (as TT2, TT15 and TT23 did) were not discriminant factors. In addition, we checked if TTs speaking EN, FR, DE or ES could improve the identification of that L1(s) they know, but we did not find significant difference. Conversely to what stated by Jarvis et al. , TT18—knowing FR and Italian as L1 and EN as L2—did not identify more FR texts than other TTs knowing FR as L2.

    Table 2: Test 1 results - TT, score and accuracy achieved and TTE

    TT

    Score

    Acc.

    TTE

    TT1

    26

    54.2%

    yes

    TT2, TT3, TT4, TT5

    25

    52.1%

    3 yes, 1 no

    TT6, TT7, TT8

    24

    50.0%

    2 yes, 1 no

    TT9

    23

    47.9%

    no

    TT10, TT11, TT12

    22

    45.8%

    3 yes

    TT13, TT14, TT15

    21

    43.7%

    1 yes, 2 no

    TT16

    20

    41.7%

    yes

    TT17

    19

    39.6%

    yes

    TT18

    18

    37.5%

    yes

    TT19

    17

    35.4%

    yes

    TT20

    15

    31.2%

    yes

    TT21, TT22, TT23, TT24

    13

    27.1%

    1 yes, 3 no

    TT25

    10

    20.8%

    yes

    Mean

    20

    SD (σ)

    4.7

    19We classified the texts into 2 major categories, easy and hard, according to the percentage above or below 50% of correct answers assigned by the TTs, respectively. In total we have 17 easy texts (10 ES, 3 FR, 3 EN, 1 DE) and 31 hard texts (3 ES, 11 FR, 8 EN, 9 DE). Almost the totality of ES texts were identified by more than 50% of TTs, but we will comment on this in the next sections.

    20We further divided these two categories into subcategories. Easy texts are divided into Clear-cut texts, in which authors’ L1 has been easily identified by almost all the TTs, and Confusing texts, identified by the majority of the TTs but a number leaned towards the same wrong class. Hard texts are further divided as Scattered, Wrong Scattered, and Wrong Clear-cut. In Scattered texts, votes are spread across two or three L1s, but the L1 receiving the majority of votes is the correct one. In Wrong Scattered, votes are spread across two or three L1s, but this time, the L1 receiving the majority of votes is a wrong one. Finally, in Wrong Clear-cut texts the L1 receiving more than 50% of the votes is an incorrect one.

    21Figure 1 shows the texts divided into these five categories. In the x axis we have the four possible L1s and in y axis the text identification number. It is interesting to notice the similarity of Clear-cut and Wrong Clear-cut categories. In both categories, more than 50% of TTs opted for the same L1. The only difference relies in the fact that in the former it is the correct prediction, in the latter a wrong prediction.

    22Our hypothesis is that texts in the same subcategory share similar features. All Clear-cut texts contain spelling errors, literal translations or calques that clearly and explicitly lead to one of the four L1s (e.g. cuando for ES, bizarre for FR, piance for DE). Confusing texts had some ambiguous clues (e.g. qualquosa which can be written by ES or FR speakers, but for different reasons) which may cast doubt on at least 2 L1s. The main clues in Scattered texts were at a grammatical level (e.g. -ing form used instead of relative clauses, wrong agreement in gender and number), so TTs had to pay more attention. In Wrong Scattered and Wrong Clear-cut texts there were shallow clues, such as misspellings and loanwords—as in Clear-cut texts—(for example basura, ES word for ‘garbage’) which might be indicative of negative transfer, but—differently from Clear-cut texts—these clues were not due to the L1 (in our example EN) but to other known L2s (in this case ES). However, in most of the cases, L2 transfers were in conjunction with L1 clues (such as the use of ritornare instead of restituire in “rotornare la borsa a Maria”, literal for ‘to return the bag to Maria’, or also nel piano instead of per terra, probably a translation error due to the polysemy of ‘floor’) that our TTs did not notice or considered less relevant clues. In order to clarify TTs recognition of the clues, we created another test, featuring the same 48 texts, but this time we told the TTs to highlight the clues that they used to make their prediction. We cannot provide information about this because we are still collecting the answers.

    Figure 1: Confusion matrix with Test 1 aggregated data

    Image 10000000000003B0000002D0BA7C5A1E0643B79F.jpg

    23However, besides the clues, we wanted to capture also TTs’ uncertainty. Thus, in Test 2, we asked our TTs to assign a percentage to each L1 of the 24 most challenging texts. Figure 1 shows the average results for each L1 per text (correct answer in bold) divided into the subcategories of hard texts: Scattered (S), Wrong Scattered (WS), Wrong Clear-Cut (WCC).

    Figure 2: Test 2 results aggregated for L1 and text

    Image 10000000000003B000000130E0DF440202CEDF3E.jpg

    24Broadly speaking, there was high uncertainty among the TTs, not always in line with our hypotheses. Even when TTs were particularly sure about one of the four L1s (e.g. assigning 99% to a L1 and 0% to the others), it was not always the correct L1, nor was explainable by writer’s L2 knowledge. For example in T14, T21, T48, most of TTs thought that ES was the correct L1. Although these texts showed numerous ES-like transfers (also syntactic ones, e.g. ES personal ‘a’ in aiutare a la donna, meaning ‘to help the woman’), the writers were FR native speakers, and only one of them claimed to know ES as L2. Still, in the majority of texts, the correct L1 received one of the two highest percentages, suggesting that L1 cues are present and that the TTs correctly interpreted them. It is also interesting to notice that also the analysis conducted by Malmasi about the Oracle classifier suggests that the correct L1 is often in the top two predictions.

    3.2 Error Analysis

    25We calculated recall, precision, accuracy and F1 aggregating all the TTs’ Test 1 answers per class; results—in terms of precision (Pre.), recall (Rec.), accuracy (Acc.) and F1—are shown in Table 3. As known, accuracy is influenced by the class distribution, hence F1 is a better metric in this case.

    26Overall, it was a challenging task as expected. On the one hand, ES proved to be easier to identify than the other three L1s (F1 score 56% compared to FR 38%). However, since all texts belonged to different proficiency levels not balanced across L1s, we cannot say if it is due to easily recognizable L1 patterns or to different interlanguage stages. On the other hand, DE proved to be the hardest L1 to identify. This could be motivated by the fact that only 28% of the TTs that participated to Test 1 have studied DE as L2. It is interesting to notice that DE is the most confused L1 to identify also in the experiment carried out by Malmasi et al. as clearly stated in Malmasi’s PhD thesis .

    Table 3: Test 1 aggregated results per class

    Class

    Pre.

    Rec.

    Acc.

    F1

    EN

    0.34

    0.38

    0.69

    0.36

    FR

    0.41

    0.36

    0.66

    0.38

    DE

    0.35

    0.30

    0.74

    0.32

    ES

    0.52

    0.60

    0.74

    0.56

    27Linking these results with TTs’ language knowledge and writers’ L2 background, we can speculate that TTs’ language knowledge in itself is not enough to improve L1 identification. In fact, all the TTs know EN as L2, but ES was the language identified correctly most of the time. This might be due to the fact that also all of the writers (except EN native speakers) know EN as L2, making the identification of EN native speakers harder.

    28We then plotted a confusion matrix (reported in Figure 3) to see the trends per class. Surprisingly, EN, correctly identified 38% of the time, was almost equally confused with the other three L1s (slightly more with FR and ES, 22% and 20% respectively, than with DE, 19%). FR, correctly identified 36% of the time, was confused more with ES (25%) and EN (24%) than with DE (15%). DE, correctly identified 30% of the time, was rarely confused with ES (13%), but frequently mistaken for FR (29%) and EN (28%). ES, when incorrectly identified (40% of the time), was confused slightly more with EN and FR (15% both) than with DE (10%). This might suggest that there is not a clear distinction between the two language families. In addition, it is interesting to notice that the directionality of the confusion is not always bidirectional for the four L1s (e.g. DE is confused with FR and ES but not vice versa).

    Figure 3: Confusion matrix with Test 1 aggregated data

    Image 100000000000026B00000237A8B51220E6AC49C5.jpg

    4. Conclusion

    29In this paper we described two human NLI tests for Italian. Although it was a simplified NLI task, tailored bearing in mind human skills, it proved to be a difficult task even for experts.

    30The error analysis showed that ES was the easiest L1 to identify—correctly identified 60% of the time—while DE the hardest. L2 transfer was misleading, even when L1 clues were present. TTs knowledge of the involved L1s proved not to be a discriminant factor.

    31It would have been interesting to ask the TTs to point out the clues that supported their answers to be less hypothetical in the discussion, especially when dealing with texts featured by L1 and L2 transfer. For this reason, we asked our TTs to take part in another test based on the same texts in which they have to highlight the clues. At the moment, we are collecting the answers.

    32In the future, we will test a machine learning system on the same texts to compare its results with those of our TTs.

    Bibliographie

    Des DOI sont automatiquement ajoutés aux références bibliographiques par Bilbo, l’outil d’annotation bibliographique d’OpenEdition. Ces références bibliographiques peuvent être téléchargées dans les formats APA, Chicago et MLA.

    Format

    • APA
    • Chicago
    • MLA
    Newmeyer, F. J. (Ed.). (1988). Linguistics: The Cambridge Survey. Cambridge University Press. https://doi.org/10.1017/cbo9780511621062
    Intelligence and Security Informatics. (2005). In P. Kantor, G. Muresan, F. Roberts, D. D. Zeng, F.-Y. Wang, H. Chen, & R. C. Merkle (Eds.), Lecture Notes in Computer Science. Springer Berlin Heidelberg. https://doi.org/10.1007/b136511
    Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications. (2017). Presented at the Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications. https://doi.org/10.18653/v1/w17-50
    Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications. (2017). Presented at the Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications. https://doi.org/10.18653/v1/w17-50
    Schuller, B., Steidl, S., Batliner, A., Hirschberg, J., Burgoon, J. K., Baird, A., … Evanini, K. (2016). The INTERSPEECH 2016 Computational Paralinguistics Challenge: Deception, Sincerity & Native Language. Presented at the Interspeech 2016. https://doi.org/10.21437/interspeech.2016-129
    Allott, N., Lohndal, T., & Rey, G. (Eds.). (2021). A Companion to Chomsky. Wiley. https://doi.org/10.1002/9781119598732
    Newmeyer, Frederick J., ed. Linguistics: The Cambridge Survey. []. Cambridge University Press, 1988. doi:10.1017/cbo9780511621062.
    Kantor, Paul, Gheorghe Muresan, Fred Roberts, Daniel D. Zeng, Fei-Yue Wang, Hsinchun Chen, and Ralph C. Merkle, eds. Intelligence and Security Informatics. Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2005. doi:10.1007/b136511.
    “Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications”. []. Association for Computational Linguistics, 2017. doi:10.18653/v1/w17-50.
    “Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications”. []. Association for Computational Linguistics, 2017. doi:10.18653/v1/w17-50.
    Schuller, Björn, Stefan Steidl, Anton Batliner, Julia Hirschberg, Judee K. Burgoon, Alice Baird, Aaron Elkins, Yue Zhang, Eduardo Coutinho, and Keelan Evanini. “The INTERSPEECH 2016 Computational Paralinguistics Challenge: Deception, Sincerity &Amp; Native Language”. Interspeech 2016. ISCA, September 8, 2016. doi:10.21437/interspeech.2016-129.
    Allott, Nicholas, Terje Lohndal, and Georges Rey, eds. “A Companion to Chomsky”. []. Wiley, April 27, 2021. doi:10.1002/9781119598732.
    Newmeyer, Frederick J., editor. Linguistics: The Cambridge Survey. [], Cambridge University Press, 1988. Crossref, https://doi.org/10.1017/cbo9780511621062.
    Kantor, Paul, et al., editors. Intelligence and Security Informatics. Lecture Notes in Computer Science, Springer Berlin Heidelberg, 2005. Crossref, https://doi.org/10.1007/b136511.
    Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications. [], Association for Computational Linguistics, 2017. Crossref, https://doi.org/10.18653/v1/w17-50.
    Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications. [], Association for Computational Linguistics, 2017. Crossref, https://doi.org/10.18653/v1/w17-50.
    Schuller, Björn, et al. “The INTERSPEECH 2016 Computational Paralinguistics Challenge: Deception, Sincerity &Amp; Native Language”. Interspeech 2016, ISCA, 8 Sept. 2016. Crossref, https://doi.org/10.21437/interspeech.2016-129.
    Allott, Nicholas, et al., editors. A Companion to Chomsky. [], Wiley, 27 Apr. 2021. Crossref, https://doi.org/10.1002/9781119598732.

    Cette bibliographie a été enrichie de toutes les références bibliographiques automatiquement générées par Bilbo en utilisant Crossref.

    Blanchard, Daniel, Joel Tetreault, Derrick Higgins, Aoife Cahill, and Martin Chodorow. 2013. “TOEFL11: A corpus of non-native English.” ETS Research Report Series 2013 (2): i–15.

    Broselow, Ellen, and F Newmeyer. 1988. “Second Language Acquisition.” Linguistics: The Cambridge Survey: Volume 3, Language: Psychological and Biological Aspects, 194–209.

    10.1017/CBO9780511621062 :

    Cenoz, Jasone, Britta Hufeisen, and Ulrike Jessner. 2001. Cross-Linguistic Influence in Third Language Acquisition: Psycholinguistic Perspectives. Multilingual Matters.

    Cimino, Andrea, and Felice Dell’Orletta. 2017. “Stacked Sentence-Document Classifier Approach for Improving Native Language Identification.” In Proceedings of the 12th Workshop on Innovative Use of Nlp for Building Educational Applications, 430–37.

    Corino, Elisa, and Carla Marello. 2017. Italiano Di Stranieri. I Corpora VALICO E VINCA. Guerra.

    Di Nuovo, Elisa, Cristina Bosco, Alessandro Mazzei, and Manuela Sanguinetti. 2019. “Towards an Italian Learner Treebank in Universal Dependencies.” In 6th Italian Conference on Computational Linguistics, Clic-It 2019, 2481:1–8. CEUR-WS.

    Ellis, Rod. 2015. Understanding Second Language Acquisition 2nd Edition-Oxford Applied Linguistics. Oxford university press.

    Hammarberg, Björn. 2001. “Roles of L1 and L2 in L3 production and acquisition.” In Cross-linguistic influence in third language acquisition: Psycholinguistic perspectives, edited by Jasone Cenoz, Britta Hufeisen, and Ulrike Jessner, 31:21–41. Multilingual Matters.

    Koppel, Moshe, Jonathan Schler, and Kfir Zigdon. 2005. “Automatically Determining an Anonymous Author’s Native Language.” In International Conference on Intelligence and Security Informatics, 209–17. Springer.

    10.1007/b136511 :

    Kulmizev, Artur, Bo Blankers, Johannes Bjerva, Malvina Nissim, Gertjan van Noord, Barbara Plank, and Martijn Wieling. 2017. “The Power of Character N-Grams in Native Language Identification.” In Proceedings of the 12th Workshop on Innovative Use of Nlp for Building Educational Applications, 382–89.

    10.18653/v1/W17-50 :

    Malmasi, Shervin, Keelan Evanini, Aoife Cahill, Joel Tetreault, Robert Pugh, Christopher Hamill, Diane Napolitano, and Yao Qian. 2017. “A Report on the 2017 Native Language Identification Shared Task.” In Proceedings of the 12th Workshop on Innovative Use of Nlp for Building Educational Applications, 62–75.

    Malmasi, Shervin, Joel Tetreault, and Mark Dras. 2015. “Oracle and Buman Baselines for Native Language Identification.” In Proceedings of the Tenth Workshop on Innovative Use of Nlp for Building Educational Applications, 172–78.

    Malmasi, Shervin, Sze-Meng Jojo Wong, and Mark Dras. 2013. “NLI Shared Task 2013: MQ Submission.” In Proceedings of the Eighth Workshop on Innovative Use of Nlp for Building Educational Applications, 124–33.

    Markov, Ilia, Lingzhen Chen, Carlo Strapparava, and Grigori Sidorov. 2017. “CIC-FBK approach to native language identification.” In Proceedings of the 12th Workshop on Innovative Use of Nlp for Building Educational Applications, 374–81.

    10.18653/v1/W17-50 :

    Picoral, Adriana. 2020. “L3 Portuguese by Spanish-English Bilinguals: Copula Construction Use and Acquisition in Corpus Data.” PhD thesis, The University of Arizona.

    Schuller, Björn, Stefan Steidl, Anton Batliner, Julia Hirschberg, Judee K. Burgoon, Alice Baird, Aaron Elkins, Yue Zhang, Eduardo Coutinho, and Keelan Evanini. 2016. “The INTERSPEECH 2016 Computational Paralinguistics Challenge: Deception, Sincerity & Native Language.” In Interspeech 2016, 2001–5. https://doi.org/10.21437/Interspeech.2016-129.

    10.21437/Interspeech.2016-129 :

    Slabakova, Roumyana. 2016. Second Language Acquisition. Oxford University Press.

    10.1002/9781119598732 :

    Tetreault, Joel, Daniel Blanchard, and Aoife Cahill. 2013. “A Report on the First Native Language Identification Shared Task.” In Proceedings of the 8th Workshop on Innovative Use of Nlp for Building Educational Applications, 48–57.

    Tomokiyo, Laura M., and Rosie Jones. 2001. “You’re Not From ’Round Here, Are You? Naive Bayes Detection of Non-Native Utterances.” In NAACL.

    Notes de bas de page

    1 L3 is any language acquired in addition to someone’s L1 and another or more L2s.

    2 In evaluation campaigns, NLI systems are trained and tested on 11 L1s.

    3 TOEFL 11 is the dataset used in NLI shared tasks.

    4 We would like to stress the difficulty we faced in finding the TTs, not only due to the skills required, but also to the time and concentration required to perform these not at all short tests. We want also to point out that Test 1 is the experiment, to date, featuring the highest number of TTs, which are 10 in Malmasi et al. (2015) and 16 in Jarvis et al. (2019).

    5 However, we had difficulties in finding human experts knowing all the four languages. Only three out of twenty five know all the four considered languages.

    6 As teachers of Italian as L2, EN, ES, FR or DE

    Auteurs

    • Elisa Di Nuovo

      Dipartimento di Lingue e Letterature Straniere e Culture Moderne, University of Turin – elisa.dinuovo@unito.it

    • Elisa Corino

      Dipartimento di Lingue e Letterature Straniere e Culture Moderne, University of Turin – elisa.corino@unito.it

    • Cristina Bosco

      Dipartimento di Informatica, University of Turin – bosco@di.unito.it

    Précédent Suivant
    Table des matières

    Le texte seul est utilisable sous licence Licence OpenEdition Books. Les autres éléments (illustrations, fichiers annexes importés) sont « Tous droits réservés », sauf mention contraire.

    Voir plus de livres
    Proceedings of the Second Italian Conference on Computational Linguistics CLiC-it 2015

    Proceedings of the Second Italian Conference on Computational Linguistics CLiC-it 2015

    3-4 December 2015, Trento

    Cristina Bosco, Sara Tonelli et Fabio Massimo Zanzotto (dir.)

    2015

    Proceedings of the Third Italian Conference on Computational Linguistics CLiC-it 2016

    Proceedings of the Third Italian Conference on Computational Linguistics CLiC-it 2016

    5-6 December 2016, Napoli

    Anna Corazza, Simonetta Montemagni et Giovanni Semeraro (dir.)

    2016

    EVALITA. Evaluation of NLP and Speech Tools for Italian

    EVALITA. Evaluation of NLP and Speech Tools for Italian

    Proceedings of the Final Workshop 7 December 2016, Naples

    Pierpaolo Basile, Franco Cutugno, Malvina Nissim et al. (dir.)

    2016

    Proceedings of the Fourth Italian Conference on Computational Linguistics CLiC-it 2017

    Proceedings of the Fourth Italian Conference on Computational Linguistics CLiC-it 2017

    11-12 December 2017, Rome

    Roberto Basili, Malvina Nissim et Giorgio Satta (dir.)

    2017

    Proceedings of the Fifth Italian Conference on Computational Linguistics CLiC-it 2018

    Proceedings of the Fifth Italian Conference on Computational Linguistics CLiC-it 2018

    10-12 December 2018, Torino

    Elena Cabrio, Alessandro Mazzei et Fabio Tamburini (dir.)

    2018

    EVALITA Evaluation of NLP and Speech Tools for Italian

    EVALITA Evaluation of NLP and Speech Tools for Italian

    Proceedings of the Final Workshop 12-13 December 2018, Naples

    Tommaso Caselli, Nicole Novielli, Viviana Patti et al. (dir.)

    2018

    EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

    EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

    Proceedings of the Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for Italian Final Workshop

    Valerio Basile, Danilo Croce, Maria Maro et al. (dir.)

    2020

    Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020

    Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020

    Bologna, Italy, March 1-3, 2021

    Felice Dell'Orletta, Johanna Monti et Fabio Tamburini (dir.)

    2020

    Proceedings of the Eighth Italian Conference on Computational Linguistics CliC-it 2021

    Proceedings of the Eighth Italian Conference on Computational Linguistics CliC-it 2021

    Milan, Italy, 26-28 January, 2022

    Elisabetta Fersini, Marco Passarotti et Viviana Patti (dir.)

    2022

    Voir plus de livres
    1 / 9
    Proceedings of the Second Italian Conference on Computational Linguistics CLiC-it 2015

    Proceedings of the Second Italian Conference on Computational Linguistics CLiC-it 2015

    3-4 December 2015, Trento

    Cristina Bosco, Sara Tonelli et Fabio Massimo Zanzotto (dir.)

    2015

    Proceedings of the Third Italian Conference on Computational Linguistics CLiC-it 2016

    Proceedings of the Third Italian Conference on Computational Linguistics CLiC-it 2016

    5-6 December 2016, Napoli

    Anna Corazza, Simonetta Montemagni et Giovanni Semeraro (dir.)

    2016

    EVALITA. Evaluation of NLP and Speech Tools for Italian

    EVALITA. Evaluation of NLP and Speech Tools for Italian

    Proceedings of the Final Workshop 7 December 2016, Naples

    Pierpaolo Basile, Franco Cutugno, Malvina Nissim et al. (dir.)

    2016

    Proceedings of the Fourth Italian Conference on Computational Linguistics CLiC-it 2017

    Proceedings of the Fourth Italian Conference on Computational Linguistics CLiC-it 2017

    11-12 December 2017, Rome

    Roberto Basili, Malvina Nissim et Giorgio Satta (dir.)

    2017

    Proceedings of the Fifth Italian Conference on Computational Linguistics CLiC-it 2018

    Proceedings of the Fifth Italian Conference on Computational Linguistics CLiC-it 2018

    10-12 December 2018, Torino

    Elena Cabrio, Alessandro Mazzei et Fabio Tamburini (dir.)

    2018

    EVALITA Evaluation of NLP and Speech Tools for Italian

    EVALITA Evaluation of NLP and Speech Tools for Italian

    Proceedings of the Final Workshop 12-13 December 2018, Naples

    Tommaso Caselli, Nicole Novielli, Viviana Patti et al. (dir.)

    2018

    EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

    EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

    Proceedings of the Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for Italian Final Workshop

    Valerio Basile, Danilo Croce, Maria Maro et al. (dir.)

    2020

    Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020

    Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020

    Bologna, Italy, March 1-3, 2021

    Felice Dell'Orletta, Johanna Monti et Fabio Tamburini (dir.)

    2020

    Proceedings of the Eighth Italian Conference on Computational Linguistics CliC-it 2021

    Proceedings of the Eighth Italian Conference on Computational Linguistics CliC-it 2021

    Milan, Italy, 26-28 January, 2022

    Elisabetta Fersini, Marco Passarotti et Viviana Patti (dir.)

    2022

    Voir plus de chapitres

    Editoriale

    Cristina Bosco, Sara Tonelli et Fabio Massimo Zanzotto

    KIPoS @ EVALITA2020: Overview of the Task on KIParla Part of Speech Tagging

    Cristina Bosco, Silvia Ballarè, Massimo Cerruti et al.

    Overview of the EVALITA 2018 Hate Speech Detection Task

    Cristina Bosco, Felice Dell’Orletta, Fabio Poletto et al.

    Parsing di corpora di apprendenti di italiano: un primo studio su VALICO

    Elisa Corino et Claudio Russo

    Overview of the EVALITA 2016

    Part Of Speech on TWitter for ITAlian Task

    Cristina Bosco, Fabio Tamburini, Andrea Bolioli et al.

    Subjective Well-Being and Social Media: A Semantically Annotated Twitter Corpus on Fertility and Parenthood

    Emilio Sulis, Cristina Bosco, Viviana Patti et al.

    Building a Treebank in Universal Dependencies for Italian Sign Language

    Gaia Caligiore, Cristina Bosco et Alessandro Mazzei

    Auxiliary selection in Italian intransitive verbs: a computational investigation based on annotated corpora

    Ilaria Ghezzi, Cristina Bosco et Alessandro Mazzei

    Analyzing and annotating for sentiment analysis the socio-political debate on #labuonascuola

    Marco Stranisci, Cristina Bosco, Viviana Patti et al.

    Dealing with Italian Adjectives in Noun Phrase: a study oriented to Natural Language Generation

    Giorgia Conte, Cristina Bosco et Alessandro Mazzei

    TWITTIRÒ: a Social Media Corpus with a Multi-layered Annotation for Irony

    Alessandra Teresa Cignarella, Cristina Bosco et Viviana Patti

    Building a Corpus on a Debate on Political Reform in Twitter

    Mirko Lai, Daniela Virone, Cristina Bosco et al.

    Voir plus de chapitres
    1 / 12

    Editoriale

    Cristina Bosco, Sara Tonelli et Fabio Massimo Zanzotto

    KIPoS @ EVALITA2020: Overview of the Task on KIParla Part of Speech Tagging

    Cristina Bosco, Silvia Ballarè, Massimo Cerruti et al.

    Overview of the EVALITA 2018 Hate Speech Detection Task

    Cristina Bosco, Felice Dell’Orletta, Fabio Poletto et al.

    Parsing di corpora di apprendenti di italiano: un primo studio su VALICO

    Elisa Corino et Claudio Russo

    Overview of the EVALITA 2016

    Part Of Speech on TWitter for ITAlian Task

    Cristina Bosco, Fabio Tamburini, Andrea Bolioli et al.

    Subjective Well-Being and Social Media: A Semantically Annotated Twitter Corpus on Fertility and Parenthood

    Emilio Sulis, Cristina Bosco, Viviana Patti et al.

    Building a Treebank in Universal Dependencies for Italian Sign Language

    Gaia Caligiore, Cristina Bosco et Alessandro Mazzei

    Auxiliary selection in Italian intransitive verbs: a computational investigation based on annotated corpora

    Ilaria Ghezzi, Cristina Bosco et Alessandro Mazzei

    Analyzing and annotating for sentiment analysis the socio-political debate on #labuonascuola

    Marco Stranisci, Cristina Bosco, Viviana Patti et al.

    Dealing with Italian Adjectives in Noun Phrase: a study oriented to Natural Language Generation

    Giorgia Conte, Cristina Bosco et Alessandro Mazzei

    TWITTIRÒ: a Social Media Corpus with a Multi-layered Annotation for Irony

    Alessandra Teresa Cignarella, Cristina Bosco et Viviana Patti

    Building a Corpus on a Debate on Political Reform in Twitter

    Mirko Lai, Daniela Virone, Cristina Bosco et al.

    Accès ouvert

    Accès ouvert

    ePub

    PDF

    PDF du chapitre

    1 L3 is any language acquired in addition to someone’s L1 and another or more L2s.

    2 In evaluation campaigns, NLI systems are trained and tested on 11 L1s.

    3 TOEFL 11 is the dataset used in NLI shared tasks.

    4 We would like to stress the difficulty we faced in finding the TTs, not only due to the skills required, but also to the time and concentration required to perform these not at all short tests. We want also to point out that Test 1 is the experiment, to date, featuring the highest number of TTs, which are 10 in Malmasi et al. (2015) and 16 in Jarvis et al. (2019).

    5 However, we had difficulties in finding human experts knowing all the four languages. Only three out of twenty five know all the four considered languages.

    6 As teachers of Italian as L2, EN, ES, FR or DE

    Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020

    X Facebook Email

    Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020

    Vous pouvez vous connecter à votre bibliothèque à l’adresse suivante : https://freemium.openedition.org/oebooks

    Suggérer l’acquisition à votre bibliothèque

    Si vous avez des questions, vous pouvez nous écrire à access[at]openedition.org

    Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020

    Vérifiez si votre bibliothèque a déjà acquis ce livre : authentifiez-vous à OpenEdition Freemium for Books.

    Vous pouvez suggérer à votre bibliothèque d’acquérir un ou plusieurs livres publiés sur OpenEdition Books. N’hésitez pas à lui indiquer nos coordonnées : access[at]openedition.org

    Vous pouvez également nous indiquer, à l’aide du formulaire suivant, les coordonnées de votre bibliothèque afin que nous la contactions pour lui suggérer l’achat de ce livre. Les champs suivis de (*) sont obligatoires.

    Veuillez, s’il vous plaît, remplir tous les champs.

    La syntaxe de l’email est incorrecte.

    Référence numérique du chapitre

    Format

    Nuovo, E. D., Corino, E., & Bosco, C. (2020). How good are humans at Native Language Identification? A case study on Italian L2 writings. In F. Dell’Orletta, J. Monti, & F. Tamburini (éds.), Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020. Torino: Accademia University Press. https://doi.org/10.4000/books.aaccademia.8475
    Nuovo, Elisa Di, Elisa Corino, et Cristina Bosco. « How Good Are Humans at Native Language Identification? A Case Study on Italian L2 Writings ». In Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-It 2020, édité par Felice Dell’Orletta, Johanna Monti, et Fabio Tamburini. Torino: Accademia University Press, 2020. doi:10.4000/books.aaccademia.8475.
    Nuovo, Elisa Di, et al. « How Good Are Humans at Native Language Identification? A Case Study on Italian L2 Writings ». Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-It 2020, édité par Felice Dell’Orletta et al., Accademia University Press, 2020, https://doi.org/10.4000/books.aaccademia.8475.

    Référence numérique du livre

    Format

    Dell’Orletta, F., Monti, J., & Tamburini, F. (éds.). (2020). Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020. Torino: Accademia University Press. https://doi.org/10.4000/books.aaccademia.8203
    Dell’Orletta, Felice, Johanna Monti, et Fabio Tamburini, éd. Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-It 2020. Torino: Accademia University Press, 2020. doi:10.4000/books.aaccademia.8203.
    Dell’Orletta, Felice, et al., éditeurs. Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-It 2020. Accademia University Press, 2020, https://doi.org/10.4000/books.aaccademia.8203.
    Compatible avec Zotero Zotero

    1 / 3

    Accademia University Press

    Accademia University Press

    • Plan du site
    • Se connecter

    Suivez-nous

    • Facebook
    • Flux RSS

    URL : http://www.aaccademia.it/

    Email : info@aaccademia.it

    Adresse :

    Accademia University Press

    Via Carlo Alberto 55

    I‐10123

    Torino

    Italia

    OpenEdition
    • Candidater à OpenEdition Books
    • Connaître le programme OpenEdition Freemium
    • Commander des livres
    • S’abonner à la lettre d’OpenEdition
    • CGU d’OpenEdition Books
    • Accessibilité : partiellement conforme
    • Données personnelles
    • Gestion des cookies
    • Système de signalement