• Contenu principal
  • Menu
OpenEdition Books
  • Accueil
  • Catalogue de 16080 livres
  • Éditeurs
  • Auteurs
  • Facebook
  • X
  • Partager
    • Facebook

    • X

    • Accueil
    • Catalogue de 16080 livres
    • Éditeurs
    • Auteurs
  • Ressources numériques en sciences humaines et sociales

    • OpenEdition
  • Nos plateformes

    • OpenEdition Books
    • OpenEdition Journals
    • Hypothèses
    • Calenda
  • Bibliothèques

    • OpenEdition Freemium
  • Suivez-nous

  • Lettre d’information
OpenEdition Search

Redirection vers OpenEdition Search.

À quel endroit ?
  • Accademia University Press
  • ›
  • Collana dell'Associazione Italiana di Li...
  • ›
  • EVALITA Evaluation of NLP and Speech Too...
  • ›
  • EVALITA 2020: Overview of the 7th Evalua...
  • Accademia University Press
  • Accademia University Press
    Accademia University Press
    Informations sur la couverture
    Table des matières
    Liens vers le livre
    Informations sur la couverture
    Table des matières
    Formats de lecture

    Plan

    Plan détaillé Texte intégral 1. Introduction 2. EVALITA 2020 Tracks and tasks 3. Participation 4. Award: Best System Across Tasks 5. Final Remarks Bibliographie Notes de bas de page Notes de fin Auteurs

    EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

    Ce livre est recensé par

    Précédent Suivant
    Table des matières

    EVALITA 2020: Overview of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian

    Valerio Basile, Danilo Croce, Maria Di Maro et Lucia C. Passaro

    p. 5-11

    Remerciements

    We would like to thank our sponsors Amazon Science7, Bnova8, CELI9, European Language Resources Association (ELRA)10 and Google Research11 for their support to the virtual event and to the best-system across task award.
    Moreover, we gratefully acknowledge the members of the AILC board for their trust and support, our EVALITA advisor Nicole Novielli and all the chairs of the 2018 edition, who helped us during the organization process of EVALITA 2020. In addition, we sincerely thank the Best System across Tasks committee for providing their expertise and experience.
    Finally, we know that EVALITA 2020 would not have been possible without the tireless effort, enthusiasm, and originality of the task organizers, the colleagues and friends involved in running them, and all the participants who contributed to make the workshop a success.

    Texte intégral 1. Introduction 2. EVALITA 2020 Tracks and tasks Affect, Hate, and Stance Creativity and Style Semantics and Multimodality Time and Diachrony New Challenges in Long-standing Tasks 3. Participation 4. Award: Best System Across Tasks 5. Final Remarks Bibliographie Notes de bas de page Notes de fin Auteurs

    Texte intégral

    1. Introduction

    1The Evaluation Campaign of Natural Language Processing and Speech Tools for Italian (EVALITA) is the biennial initiative aimed at promoting the development of language and speech technologies for the Italian language. EVALITA is promoted by the Italian Association of Computational Linguistics (AILC)1 and it is endorsed by the Italian Association for Artificial Intelligence (AIxIA)2 and the Italian Association for Speech Sciences (AISV)3.

    2EVALITA provides a shared framework where different systems and approaches can be scientifically evaluated and compared with each other with respect to a large variety of tasks, suggested and organized by the Italian research community. The proposed tasks represent scientific challenges where methods, resources, and systems can be tested against shared benchmarks representing linguistic open issues or real world applications, possibly in a multilingual and/or multi-modal perspective. The collected data sets provide big opportunities for scientists to explore old and new problems concerning NLP in Italian as well as to develop solutions and to discuss the NLP-related issues within the community. Some tasks are traditionally present in the evaluation campaign, while others are completely new.

    3This paper introduces the tasks proposed at EVALITA 2020 and provides an overview to the participants and systems whose descriptions and obtained results are reported in these Proceedings4. The EVALITA 2020 edition, held online on December 17th due to the COVID-19 pandemic, counts 14 different tasks. In particular, the selected tasks are grouped in five research areas (tracks) according to their objective and characteristics, namely (i) Affect, Hate, and Stance, (ii) Creativity and Style, (iii) New Challenges in Long-standing Tasks, (iv) Semantics and Multimodality, (v) Time and Diachrony.

    4This edition was highly participated, with 51 groups whose participants have affiliation in 14 countries. Although EVALITA is generally promoted and targeted to the Italian research community, this edition saw an international participation, also thanks to the fact that several Italian researchers working in different countries contributed to the organization of the tasks or participated in them as authors.

    5This overview is organized as follows: in Section 2 a brief description of the tasks belonging to the various areas is reported. Section 3 discusses the participation to the workshop referred to several aspects, from the research area, to the affiliation of authors. Section 4 describes the criteria used to assign the best system across tasks award, made by an ad-hoc committee starting from the suggestions of task organizers and reviewers. Finally, section 5 points out on both the obtained results and on the future of the workshop.

    2. EVALITA 2020 Tracks and tasks

    6In the 2020 edition of EVALITA, 14 different tasks were proposed, peer-reviewed and accepted. Data were produced by the task organizers and made available to the participants. For the future availability of this data we are going to release them on GitHub5, in accordance to the terms and conditions of the respective data sources. Such a repository will also reference alternative repositories managed by the task organizers. The tasks of EVALITA 2020 are grouped according to the following tracks:

    Affect, Hate, and Stance

    7AMI - Automatic Misogyny Identification (Fersini, Nozza, and Rosso 2020). This shared task is aimed at automatically identifying misogynous content in Twitter for the Italian language. In particular, the AMI challenge is focused on: (1) recognizing misogynous and aggressive messages and (2) discriminating misogynistic contents from the non-misogynistic ones, while guaranteeing the fairness of the model.

    8ATE_ABSITA - Aspect Term Extraction and Aspect-Based Sentiment Analysis (De Mattei, De Martino, et al. 2020). A task on Aspect Term Extraction (ATE) and Aspect-Based Sentiment Analysis (ABSA). The task is approached as a cascade of three subtasks: Aspect Term Extraction (ATE), Aspect-based Sentiment Analysis (ABSA) and Sentiment Analysis (SA).

    9HaSpeeDe - Hate Speech Detection (Sanguinetti et al. 2020). A rerun of the shared task on hate speech detection at the message level on Italian social media texts proposed for the first time in 2018 for the EVALITA evaluation campaign. The main task is a binary hate speech detection task, one in-domain and one out-of-domain. On the same data provided for the main task, the topics of stereotypes in communication and nominal utterances are investigated by of two pilot tasks.

    10SardiStance - Stance Detection (Cignarella et al. 2020). The goal of the task is to detect the stance of the author towards the target “Sardines movement” in Italian tweets. Two subtasks model (A) Textual Stance Detection and (B) Contextual Stance Detection. Both the subtasks consist on a three-class (in favour, against, neutral) classification problem based on textual information only (A) or on the text provided with additional information about the author and the post of the tweet.

    Creativity and Style

    11CHANGE-IT - Style Transfer (De Mattei, Cafagna, et al. 2020). The first natural language generation task for Italian. Change-IT focuses on style transfer performed on the headlines of two Italian newspapers at opposite ends of the political spectrum. Specifically, the goal is to “translate” the headlines from a style to another.

    12TAG-it - Topic, Age and Gender Prediction (Cimino, Dell’Orletta, and Nissim 2020). TAG-IT is a profiling task for Italian. It is a follow-up of the GxG task organised in the context of EVALITA 2018. The task is aimed at profiling along with three dimensions (Gender, Age, and Topic). Authors propose several subtasks where participants are asked to predict one or more classes starting from the others.

    Semantics and Multimodality

    13CONcreTEXT - Concreteness in Context (Gregori et al. 2020). The task focuses on automatic assignment of concreteness values to words in context for the Italian and English language. Participants are required to develop systems able to rate the concreteness of a target word in a sentence on a scale from 1 (for fully abstract) to 5 (for maximally concrete).

    14DANKMEMES - Multimodal Artefacts Recognition (Miliani et al. 2020). The first multimodal task for Italian. The goal of the task is to deal with Italian memes considering textual and visual cues together. Providing a corpus of memes on the 2019 Italian Government Crisis, DANKMEMES features three subtasks: A) Meme Detection, B) Hate Speech Identification, and C) Event Clustering.

    15Ghigliottin-AI - Evaluating Artificial Players for the Language Game “La Ghigliottina” (Basile, Lovetere, et al. 2020). The task challenges researchers to develop a system able to defeat human players at the language game “La Ghigliottina”, which represents one of the most followed and appreciated quiz games in Italy.

    16PRELEARN - Prerequisite Relation Learning (Alzetta et al. 2020). The task is devoted to automatically inferring prerequisite relations from educational texts. The task consists in classifying prerequisite relations between pairs of concepts distinguishing between prerequisite pairs and non-prerequisite pairs.

    Time and Diachrony

    17DaDoEval - Dating Documents (Menini et al. 2020). The task focuses on assigning a temporal span to a document, by recognising when a document was issued. A first one coarse-grained classification subtask, participants are asked to provide a document with a class encoding the historical period (5 classes). The second Fine-grained classification task requires to attribute documents with a temporal slice of 5 years.

    18DIACR-Ita - Diachronic Lexical Semantics (Basile, Caputo, et al. 2020). The first task on automatic detection of lexical and semantic shift for Italian. The task challenges participants to develop systems that can automatically detect if a given word has changed its meaning over time, given contextual information from corpora.

    New Challenges in Long-standing Tasks

    19AcCompl-it- Acceptability & Complexity evaluation (Brunato et al. 2020). The task is aimed at developing and evaluating methods to classify Italian sentences according to Acceptability and Complexity. Given a set of sentences, two independent subtasks focus on predicting their acceptability and complexity rate.

    20KIPoS - Part-of-speech Tagging on Spoken Language (Bosco et al. 2020). The first task on Part of Speech tagging of spoken language held at EVALITA. Benefiting from the KIParla corpus, a resource of transcribed spoken Italian, the task provides three evaluation exercises focused on formal versus informal spoken texts.

    3. Participation

    21EVALITA 2020 attracted the interest of a large number of researchers from academia and industry, for a total of 51 teams composed of about 130 individuals participating in one or more of the 14 proposed tasks. After the evaluation period, 58 system descriptions were submitted (reported in these proceedings), i.e., a 70% percentage increase with respect to the previous EVALITA edition (Caselli et al. 2018).

    22Moreover, task organizers allowed participants to submit more than one system result (called runs), for a total of 240 submitted runs. Table 1 shows the different tracks and tasks along with the number of participating teams and submitted runs. The data reported in the table is based on information provided by the task organizers at the end of the evaluation process. Such data represents an overestimation with respect to the systems described in the proceedings. The trends are similar, but there are differences due to groups participating in more than a task, and groups that have not produced a system report.

    Table 1

    Track

    Task

    Teams

    Runs

    Affect, Hate, and Stance

    AMI

    8

    31

    ATE_ABSITA

    3

    8

    HaSpeeDe

    14

    27

    SardiStance

    12

    36

    Creativity and Style

    CHANGE-IT

    0

    0

    TAG-it

    3

    20

    New Challenges in Long-standing Tasks

    AcCompl-it

    2

    6

    KIPoS

    3

    14

    Semantics and Multimodality

    CONcreTEXT

    4

    15

    DANKMEMES

    5

    15

    Ghigliottin-AI

    2

    2

    PRELEARN

    3

    14

    Time and Diachrony

    DaDoEval

    2

    16

    DIACR-Ita

    9

    36

    Number of participating teams and number of runs organized by track and task. The data reported is an overestimation with respect to the systems described in the proceedings (e.g. teams participating in more than a task are counted according to the number of tasks they participated in)

    23Differently from the previous EVALITA editions, the organizers were discouraged from distinguishing the submissions between unconstrained and constrained runs6. The rationale for this decision is that the recent spread and extensive use of pre-trained word embedding representations, especially as a strategy to initialize Neural Network architectures, challenges this distinction at its very heart. Participation was quite imbalanced across different tracks and tasks, as reported in Figure 1: each rectangle represents a task whose size reflects the number of participants, while the color indicated the corresponding track.

    Figure 1

    Image 100000000000057500000372746FC60FF9F9EF6C.jpg

    Number of participating teams organized by track (color) and task. The red color is adopted for the track “Affect, Hate, and Stance”, the yellow color for “Creativity and Style”, green for ”New Challenges in Long-standing Tasks”, blue for “Semantics and Multimodality” and purple for “Time and Diachrony”.

    24In line with the previous editions of EVALITA, the track “Affect, Creativity and Style” covers about half of the total in terms of participating teams. On the one hand, this demonstrates the well-known interest of the NLP community for Social Media platforms and user-generated content. On the other hand, we report a better balance with respect to the 2018 edition, where about 80% of the teams participated in similar tracks (“Affect, Creativity and style” and “Hate Speech”, which have been merged in this edition). Another significant number of teams participated to the “Semantics and Multimodality” and “Time and Diachrony” tracks, while the other tracks where less participated. Unfortunately, no team participated to the CHANGE-IT task, mainly due to the complexity of the task.

    25In addition to being widely participated, the over 180 proceedings authors, including both participants and task organizers, have affiliation in 18 countries, with the 64% from Italy and the 36% of participants from Institutions and companies abroad. The group of the 59 task organizers have affiliations in 6 countries (90% from Italy while 10% from Institutions and companies abroad). The gender distribution is highly balanced, with 30 females and males.

    4. Award: Best System Across Tasks

    26In line with the previous edition, we confirmed the award to the best system across-task. The award was introduced with the aim of fostering student participation to the evaluation campaign and to the workshop. EVALITA received sponsorship funding from from Amazon Science, Bnova s.r.l., CELI s.r.l., the European Language Resources Association (ELRA) and Google Research.

    27A committee of 5 members was asked to choose the best system across tasks. Four of the five members come from academia while the last one is from industry. The composition of the committee is balanced with respect to the level of seniority as well as for their academic background (computer science-oriented vs. humanities-oriented). In order to select a short list of candidates, the task organizers were invited to propose up to two candidate systems participating to their tasks (not necessarily top ranking). The committee was provided with the list of candidate systems and the criteria for eligibility, based on:

    • novelty with respect to the state of the art;

    • originality, in terms of identification of new linguistic resources, identification of linguistically motivated features, and implementation of a theoretical framework grounded in linguistics;

    • critical insight, paving the way to future challenges (deep error analysis, discussion on the limits of the proposed system, discussion of the inherent challenges of the task);

    • technical soundness and methodological rigor.

    28We collected 10 system nominations from the organizers of 7 tasks from across all tracks. The candidate systems are authored by 20 authors, among whom 12 are students, either at the master’s or PhD level. The award recipient(s) will be announced during the final EVALITA workshop, during the plenary session, held online.

    5. Final Remarks

    29A record number of 14 tasks were organized at EVALITA 2020: some of them were revivals of tasks in the past editions (such as Hate Speech Detection or Part-of-Speech Tagging), while others were completely new (such as the ones involving Meme Processing or Stance Detection), and were greeted with great enthusiasm by the NLP community.

    30In this edition, the topics of Affect and Semantics were confirmed as two of the most interesting and thriving ones, both in the number of organized tasks and actual participants. In any case, almost all tasks involved the analysis of written texts. In fact, although the KIPoS task considered transcriptions of spoken Italian utterances, no speech related tasks was proposed.

    31Anyways, tasks concerning new problems and modalities have been proposed, such as the analysis of memes, and two tasks oriented to the problem of time and diachrony. Moreover, this edition saw an increase in tasks related to creativity and style, despite the fact that one one such tasks, namely CHANGE-it, had no participation, probably due to its complexity and the lack of specific resources for the task in the Italian community. Another task that received a rather low number of submission due to its complexity is GhigliottinAI. Despite being a rather simple word-correlation problem by itself, it required complex modelling of language and semantics to beat the challenge. A very interesting innovation for this task was the evaluation framework, based on APIs, via a Remote Evaluation Server (RES). In general, the most participated tasks have been those by which the linguistic problem could be modelled as a direct classification or regression task.

    32The competition attracted a record number of participating teams from academia and industry, for a total of 51 teams and more than 180 authors with affiliations in 18 countries. Hopefully, this means that EVALITA is becoming more and more popular also with foreign contributors, and it is becoming an international workshop. First of all, this success confirms the beneficial impact of the organization of the evaluation period based on non-overlapping windows (adopted from EVALITA 2018) in order to help those who want to participate in more than one task. Moreover, we speculate that the technological advancements and ease of use of existing open-source libraries for machine learning and natural language processing improved the accessibility to the tasks, even for master students. In fact, we noticed an increase in the participation of students, that contributed with state-of-the-art solutions to the tasks. We can argue that the spread of frameworks such as PyTorch and Keras, together with pre-trained, off-the-shelf language models, lowered the set-up costs to deal with complex NLP tasks. In general, we noticed that most of the best systems are based on neural approaches. Among them, BERT or similar Transformer-based architectures achieved the best results: more specifically, at least in 11 out of 14 tasks best results (in at least one sub-task) were obtained by neural architectures based on or combined with Transformers.

    33We are confident that the positive trends observed in this edition, concerning the participation and the proliferation of tasks, has not yet reached a plateau. It would be auspicabile, among other aspects, to see more tasks involving challenging settings such as, for example, multi-modal or multi-lingual analysis involving Italian, in future EVALITA 2020 editions. Several areas represent fertile ground to organize future tasks, such as domain adaptation (which was considered in previous editions of EVALITA), or few-shot learning to support the definition of robust systems in challenging low-resource settings. Finally, we believe in the importance of defining more structured tasks involving real applications to challenge the Italian community, e.g., Question Answering or Dialogue Agents.

    Bibliographie

    Chiara Alzetta, Alessio Miaschi, Felice Dell’Orletta, Frosina Koceva, and Ilaria Torre. 2020. “PRELEARN@EVALITA2020: Overview of the Prerequisite Relation Learning Task for Italian.” In Proceedings of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian (Evalita 2020), edited by Valerio Basile, Danilo Croce, Maria Di Maro, and Lucia C. Passaro. Online: CEUR.org.

    Pierpaolo Basile, Annalina Caputo, Tommaso Caselli, Pierluigi Cassotti, and Rossella Varvara. 2020. “DIACR-Ita@EVALITA2020: Overview of the Evalita2020 Diachronic Lexical Semantics (Diacr-Ita) Task.” In Proceedings of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian (Evalita 2020), edited by Valerio Basile, Danilo Croce, Maria Di Maro, and Lucia C. Passaro. Online: CEUR.org.

    Pierpaolo Basile, Marco Lovetere, Johanna Monti, Antonioa Pascucci, Federico Sangati, and Lucia Siciliani. 2020. “Ghigliottin-AI@EVALITA2020: Evaluating Artificial Players for the Language Game ‘La Ghigliottina’.” In Proceedings of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian (Evalita 2020), edited by Valerio Basile, Danilo Croce, Maria Di Maro, and Lucia C. Passaro. Online: CEUR.org.

    Cristina Bosco, Silvia Ballarè, Massimo Cerruti, Eugenio Goria, and Caterina Mauri. 2020. “KIPoS@EVALITA2020:Overview of the Task on Kiparla Part of Speech Tagging.” In Proceedings of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian (Evalita 2020), edited by Valerio Basile, Danilo Croce, Maria Di Maro, and Lucia C. Passaro. Online: CEUR.org.

    Dominique Brunato, Cristiano Chesi, Felice Dell’Orletta, Simonetta Montemagni, Giulia Venturi, and Roberto Zamparelli. 2020. “AcCompl-it@EVALITA2020: Overview of the Acceptability & Complexity Evaluation Task for Italian.” In Proceedings of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian (Evalita 2020), edited by Valerio Basile, Danilo Croce, Maria Di Maro, and Lucia C. Passaro. Online: CEUR.org.

    T. Caselli, N. Novielli, V. Patti, and P. Rosso. 2018. “EVALITA 2018: Overview of the 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian.” In Proceedings of Sixth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2018), edited by Tommaso Caselli, Nicole Novielli, Viviana Patti, and Paolo Rosso. Turin, Italy: CEUR.org.

    Alessandra Teresa Cignarella, Mirko Lai, Cristina Bosco, Viviana Patti, and Paolo Rosso. 2020. “SardiStance@EVALITA2020: Overview of the Task on Stance Detection in Italian Tweets.” In Proceedings of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian (Evalita 2020), edited by Valerio Basile, Danilo Croce, Maria Di Maro, and Lucia C. Passaro. Online: CEUR.org.

    Andrea Cimino, Felice Dell’Orletta, and Malvina Nissim. 2020. “TAG-it@EVALITA2020: Overview of the Topic, Age, and Gender Prediction Task for Italian.” In Proceedings of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian (Evalita 2020), edited by Valerio Basile, Danilo Croce, Maria Di Maro, and Lucia C. Passaro. Online: CEUR.org.

    Lorenzo De Mattei, Michele Cafagna, Felice Dell’Orletta, Malvina Nissim, and Albert Gatt. 2020. “CHANGE-IT@EVALITA2020: Change Headlines, Adapt News, Generate.” In Proceedings of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian (Evalita 2020), edited by Valerio Basile, Danilo Croce, Maria Di Maro, and Lucia C. Passaro. Online: CEUR.org.

    Lorenzo De Mattei, Graziella De Martino, Andrea Iovine, Alessio Miaschi, Marco Polignano, and Giulia Rambelli. 2020. “ATE_ABSITA@EVALITA2020: Overview of the Aspect Term Extraction and Aspect-Based Sentiment Analysis Task.” In Proceedings of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian (Evalita 2020), edited by Valerio Basile, Danilo Croce, Maria Di Maro, and Lucia C. Passaro. Online: CEUR.org.

    Elisabetta Fersini, Debora Nozza, and Paolo Rosso. 2020. “AMI@EVALITA2020: Automatic Misogyny Identification.” In Proceedings of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian (Evalita 2020), edited by Valerio Basile, Danilo Croce, Maria Di Maro, and Lucia C. Passaro. Online: CEUR.org.

    Lorenzo Gregori, Maria Montefinese, Daniele P. Radicioni, Andrea Amelio Ravelli, and Rossella Varvara. 2020. “CONCRETEXT@EVALITA2020: The Concreteness in Context Task.” In Proceedings of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian (Evalita 2020), edited by Valerio Basile, Danilo Croce, Maria Di Maro, and Lucia C. Passaro. Online: CEUR.org.

    Stefano Menini, Giovanni Moretti, Rachele Sprugnoli, and Sara Tonelli. 2020. “DaDoEval@EVALITA2020: Same-Genre and Cross-Genre Dating of Historical Documents.” In Proceedings of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian (Evalita 2020), edited by Valerio Basile, Danilo Croce, Maria Di Maro, and Lucia C. Passaro. Online: CEUR.org.

    Martina Miliani, Giulia Giorgi, Ilir Rama, Guido Anselmi, and Gianluca E. Lebani. 2020. “DANKMEMES@EVALITA2020: The Memeing of Life: Memes, Multimodality and Politics.” In Proceedings of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian (Evalita 2020), edited by Valerio Basile, Danilo Croce, Maria Di Maro, and Lucia C. Passaro. Online: CEUR.org.

    Manuela Sanguinetti, Gloria Comandini, Elisa Di Nuovo, Simona Frenda, Marco Stranisci, Cristina Bosco, Tommaso Caselli, Viviana Patti, and Irene Russo. 2020. “HaSpeeDe 2@EVALITA2020: Overview of the Evalita 2020 Hate Speech Detection Task.” In Proceedings of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian (Evalita 2020), edited by Valerio Basile, Danilo Croce, Maria Di Maro, and Lucia C. Passaro. Online: CEUR.org.

    Notes de bas de page

    1 http://www.ai-lc.it

    2 http://www.aixia.it

    3 http://www.aisv.it

    4 The presentations of these works are publicly available at https://vimeo.com/showcase/evalita2020 . All videos are also grouped according to different tasks at https://vimeo.com/user125537954/albums

    5 https://github.com/evalita2020

    6 A system is considered constrained when using the provided training data only; on the contrary, it is considered unconstrained when using additional material to augment the training dataset or to acquire additional resources.

    Notes de fin

    7 https://www.amazon.science/

    8 https://www.bnova.it

    9 https://www.celi.it/

    10 http://elra.info/en/

    11 https://research.google/

    Auteurs

    • Valerio Basile

      University of Turin – valerio.basile@unito.it

    • Danilo Croce

      University of Rome “Tor Vergata” – croce@info.uniroma2.it

    • Maria Di Maro

      University of Naples “Federico II” – maria.dimaro2@unina.it

    • Lucia C. Passaro

      University of Pisa – lucia.passaro@fileli.unipi.it

    Précédent Suivant
    Table des matières

    Creative Commons - Attribution - Pas d'Utilisation Commerciale - Pas de Modification 4.0 International - CC BY-NC-ND 4.0

    Le texte seul est utilisable sous licence Creative Commons - Attribution - Pas d'Utilisation Commerciale - Pas de Modification 4.0 International - CC BY-NC-ND 4.0. Les autres éléments (illustrations, fichiers annexes importés) sont « Tous droits réservés », sauf mention contraire.

    Voir plus de livres
    Proceedings of the Second Italian Conference on Computational Linguistics CLiC-it 2015

    Proceedings of the Second Italian Conference on Computational Linguistics CLiC-it 2015

    3-4 December 2015, Trento

    Cristina Bosco, Sara Tonelli et Fabio Massimo Zanzotto (dir.)

    2015

    Proceedings of the Third Italian Conference on Computational Linguistics CLiC-it 2016

    Proceedings of the Third Italian Conference on Computational Linguistics CLiC-it 2016

    5-6 December 2016, Napoli

    Anna Corazza, Simonetta Montemagni et Giovanni Semeraro (dir.)

    2016

    EVALITA. Evaluation of NLP and Speech Tools for Italian

    EVALITA. Evaluation of NLP and Speech Tools for Italian

    Proceedings of the Final Workshop 7 December 2016, Naples

    Pierpaolo Basile, Franco Cutugno, Malvina Nissim et al. (dir.)

    2016

    Proceedings of the Fourth Italian Conference on Computational Linguistics CLiC-it 2017

    Proceedings of the Fourth Italian Conference on Computational Linguistics CLiC-it 2017

    11-12 December 2017, Rome

    Roberto Basili, Malvina Nissim et Giorgio Satta (dir.)

    2017

    Proceedings of the Fifth Italian Conference on Computational Linguistics CLiC-it 2018

    Proceedings of the Fifth Italian Conference on Computational Linguistics CLiC-it 2018

    10-12 December 2018, Torino

    Elena Cabrio, Alessandro Mazzei et Fabio Tamburini (dir.)

    2018

    EVALITA Evaluation of NLP and Speech Tools for Italian

    EVALITA Evaluation of NLP and Speech Tools for Italian

    Proceedings of the Final Workshop 12-13 December 2018, Naples

    Tommaso Caselli, Nicole Novielli, Viviana Patti et al. (dir.)

    2018

    EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

    EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

    Proceedings of the Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for Italian Final Workshop

    Valerio Basile, Danilo Croce, Maria Maro et al. (dir.)

    2020

    Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020

    Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020

    Bologna, Italy, March 1-3, 2021

    Felice Dell'Orletta, Johanna Monti et Fabio Tamburini (dir.)

    2020

    Proceedings of the Eighth Italian Conference on Computational Linguistics CliC-it 2021

    Proceedings of the Eighth Italian Conference on Computational Linguistics CliC-it 2021

    Milan, Italy, 26-28 January, 2022

    Elisabetta Fersini, Marco Passarotti et Viviana Patti (dir.)

    2022

    Voir plus de livres
    1 / 9
    Proceedings of the Second Italian Conference on Computational Linguistics CLiC-it 2015

    Proceedings of the Second Italian Conference on Computational Linguistics CLiC-it 2015

    3-4 December 2015, Trento

    Cristina Bosco, Sara Tonelli et Fabio Massimo Zanzotto (dir.)

    2015

    Proceedings of the Third Italian Conference on Computational Linguistics CLiC-it 2016

    Proceedings of the Third Italian Conference on Computational Linguistics CLiC-it 2016

    5-6 December 2016, Napoli

    Anna Corazza, Simonetta Montemagni et Giovanni Semeraro (dir.)

    2016

    EVALITA. Evaluation of NLP and Speech Tools for Italian

    EVALITA. Evaluation of NLP and Speech Tools for Italian

    Proceedings of the Final Workshop 7 December 2016, Naples

    Pierpaolo Basile, Franco Cutugno, Malvina Nissim et al. (dir.)

    2016

    Proceedings of the Fourth Italian Conference on Computational Linguistics CLiC-it 2017

    Proceedings of the Fourth Italian Conference on Computational Linguistics CLiC-it 2017

    11-12 December 2017, Rome

    Roberto Basili, Malvina Nissim et Giorgio Satta (dir.)

    2017

    Proceedings of the Fifth Italian Conference on Computational Linguistics CLiC-it 2018

    Proceedings of the Fifth Italian Conference on Computational Linguistics CLiC-it 2018

    10-12 December 2018, Torino

    Elena Cabrio, Alessandro Mazzei et Fabio Tamburini (dir.)

    2018

    EVALITA Evaluation of NLP and Speech Tools for Italian

    EVALITA Evaluation of NLP and Speech Tools for Italian

    Proceedings of the Final Workshop 12-13 December 2018, Naples

    Tommaso Caselli, Nicole Novielli, Viviana Patti et al. (dir.)

    2018

    EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

    EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

    Proceedings of the Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for Italian Final Workshop

    Valerio Basile, Danilo Croce, Maria Maro et al. (dir.)

    2020

    Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020

    Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020

    Bologna, Italy, March 1-3, 2021

    Felice Dell'Orletta, Johanna Monti et Fabio Tamburini (dir.)

    2020

    Proceedings of the Eighth Italian Conference on Computational Linguistics CliC-it 2021

    Proceedings of the Eighth Italian Conference on Computational Linguistics CliC-it 2021

    Milan, Italy, 26-28 January, 2022

    Elisabetta Fersini, Marco Passarotti et Viviana Patti (dir.)

    2022

    Voir plus de chapitres

    Domain Adaptation for Text Classification with Weird Embeddings

    Valerio Basile

    On the Readability of Deep Learning Models: the role of Kernel-based Deep Architectures

    Danilo Croce, Daniele Rossini et Roberto Basili

    Nyström Methods for Efficient Kernel-Based Methods for Community Question Answering

    Danilo Croce, Simone Filice et Roberto Basili

    GQA-it: Italian Question Answering on Image Scene Graphs

    Danilo Croce, Lucia C. Passaro, Alessandro Lenci et al.

    ItEM: A Vector Space Model to Bootstrap an Italian Emotive Lexicon

    Lucia C. Passaro, Laura Pollacci et Alessandro Lenci

    Exploiting Emotive Features for the Sentiment Polarity Classification of tweets

    Lucia C. Passaro, Alessandro Bondielli et Alessandro Lenci

    Preface to the EVALITA 2020 Proceedings

    Valerio Basile, Danilo Croce, Maria Di Maro et al.

    Overview of the EVALITA 2018 Spoken Utterances Guiding Chef’s Assistant Robots (SUGAR) Task

    Maria Di Maro, Antonio Origlia et Francesco Cutugno

    A Markovian Kernel-based Approach for itaLIan Speech acT labEliNg

    Danilo Croce et Roberto Basili

    Neural Surface Realization for Italian

    Valerio Basile et Alessandro Mazzei

    Long-term Social Media Data Collection at the University of Turin

    Valerio Basile, Mirko Lai et Manuela Sanguinetti

    Dialogue Analysis with Graph Databases: Characterising Domain Items Usage for Movie Recommendations

    Martina Di Bratto, Maria Di Maro, Antonio Origlia et al.

    Voir plus de chapitres
    1 / 12

    Domain Adaptation for Text Classification with Weird Embeddings

    Valerio Basile

    On the Readability of Deep Learning Models: the role of Kernel-based Deep Architectures

    Danilo Croce, Daniele Rossini et Roberto Basili

    Nyström Methods for Efficient Kernel-Based Methods for Community Question Answering

    Danilo Croce, Simone Filice et Roberto Basili

    GQA-it: Italian Question Answering on Image Scene Graphs

    Danilo Croce, Lucia C. Passaro, Alessandro Lenci et al.

    ItEM: A Vector Space Model to Bootstrap an Italian Emotive Lexicon

    Lucia C. Passaro, Laura Pollacci et Alessandro Lenci

    Exploiting Emotive Features for the Sentiment Polarity Classification of tweets

    Lucia C. Passaro, Alessandro Bondielli et Alessandro Lenci

    Preface to the EVALITA 2020 Proceedings

    Valerio Basile, Danilo Croce, Maria Di Maro et al.

    Overview of the EVALITA 2018 Spoken Utterances Guiding Chef’s Assistant Robots (SUGAR) Task

    Maria Di Maro, Antonio Origlia et Francesco Cutugno

    A Markovian Kernel-based Approach for itaLIan Speech acT labEliNg

    Danilo Croce et Roberto Basili

    Neural Surface Realization for Italian

    Valerio Basile et Alessandro Mazzei

    Long-term Social Media Data Collection at the University of Turin

    Valerio Basile, Mirko Lai et Manuela Sanguinetti

    Dialogue Analysis with Graph Databases: Characterising Domain Items Usage for Movie Recommendations

    Martina Di Bratto, Maria Di Maro, Antonio Origlia et al.

    Accès ouvert

    Accès ouvert

    ePub

    PDF

    PDF du chapitre

    1 http://www.ai-lc.it

    2 http://www.aixia.it

    3 http://www.aisv.it

    4 The presentations of these works are publicly available at https://vimeo.com/showcase/evalita2020 . All videos are also grouped according to different tasks at https://vimeo.com/user125537954/albums

    5 https://github.com/evalita2020

    6 A system is considered constrained when using the provided training data only; on the contrary, it is considered unconstrained when using additional material to augment the training dataset or to acquire additional resources.

    7 https://www.amazon.science/

    8 https://www.bnova.it

    9 https://www.celi.it/

    10 http://elra.info/en/

    11 https://research.google/

    EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

    X Facebook Email

    EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

    Vous pouvez vous connecter à votre bibliothèque à l’adresse suivante : https://freemium.openedition.org/oebooks

    Suggérer l’acquisition à votre bibliothèque

    Si vous avez des questions, vous pouvez nous écrire à access[at]openedition.org

    EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

    Vérifiez si votre bibliothèque a déjà acquis ce livre : authentifiez-vous à OpenEdition Freemium for Books.

    Vous pouvez suggérer à votre bibliothèque d’acquérir un ou plusieurs livres publiés sur OpenEdition Books. N’hésitez pas à lui indiquer nos coordonnées : access[at]openedition.org

    Vous pouvez également nous indiquer, à l’aide du formulaire suivant, les coordonnées de votre bibliothèque afin que nous la contactions pour lui suggérer l’achat de ce livre. Les champs suivis de (*) sont obligatoires.

    Veuillez, s’il vous plaît, remplir tous les champs.

    La syntaxe de l’email est incorrecte.

    Référence numérique du chapitre

    Format

    Basile, V., Croce, D., Di Maro, M., & Passaro, L. C. (2020). EVALITA 2020: Overview of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian (V. Basile, D. Croce, M. Maro, & L. C. Passaro, éds.). Torino: Accademia University Press. https://doi.org/10.4000/books.aaccademia.6747
    Basile, Valerio, Danilo Croce, Maria Di Maro, et Lucia C. Passaro. « EVALITA 2020: Overview of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian ». édité par Valerio Basile, Danilo Croce, Maria Maro, et Lucia C. Passaro. EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020. Torino: Accademia University Press, 2020. doi:10.4000/books.aaccademia.6747.
    Basile, Valerio, et al. « EVALITA 2020: Overview of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian ». EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020, édité par Valerio Basile et al., Accademia University Press, 2020, https://doi.org/10.4000/books.aaccademia.6747.

    Référence numérique du livre

    Format

    Basile, V., Croce, D., Maro, M., & Passaro, L. C. (éds.). (2020). EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020. Torino: Accademia University Press. https://doi.org/10.4000/books.aaccademia.6732
    Basile, Valerio, Danilo Croce, Maria Maro, et Lucia C. Passaro, éd. EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020. Torino: Accademia University Press, 2020. doi:10.4000/books.aaccademia.6732.
    Basile, Valerio, et al., éditeurs. EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020. Accademia University Press, 2020, https://doi.org/10.4000/books.aaccademia.6732.
    Compatible avec Zotero Zotero

    1 / 3

    Accademia University Press

    Accademia University Press

    • Plan du site
    • Se connecter

    Suivez-nous

    • Facebook
    • Flux RSS

    URL : http://www.aaccademia.it/

    Email : info@aaccademia.it

    Adresse :

    Accademia University Press

    Via Carlo Alberto 55

    I‐10123

    Torino

    Italia

    OpenEdition
    • Candidater à OpenEdition Books
    • Connaître le programme OpenEdition Freemium
    • Commander des livres
    • S’abonner à la lettre d’OpenEdition
    • CGU d’OpenEdition Books
    • Accessibilité : partiellement conforme
    • Données personnelles
    • Gestion des cookies
    • Système de signalement