Version classiqueVersion mobile

EVALITA Evaluation of NLP and Speech Tools for Italian

 | 
Tommaso Caselli
, 
Nicole Novielli
, 
Viviana Patti
, 
et al.

Table des matières

Tommaso Caselli, Nicole Novielli, Viviana Patti et al.

Preface to the Evalita 2018 Proceedings

Part I. Introduction to Evalita 2018 and task overviews

Tommaso Caselli, Viviana Patti, Nicole Novielli et al.

Evalita 2018: Overview on the 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian

1 Introduction
2 Tasks and Challenge
Affect, Creativity and Style
Dialogue Systems
Hate Speech
Semantics4AI
3 Fostering Reproducibility and Cross-community Engagement
4 Award: Best System Across-tasks
5 Participation
6 Final Remarks
Acknowledgments
Pierpaolo Basile, Danilo Croce, Valerio Basile et al.

Overview of the EVALITA 2018 Aspect-based Sentiment Analysis task (ABSITA)

1 Introduction
2 Definition of the task
3 Dataset
4 Evaluation measures and baselines
5 Results
6 Discussion
7 Conclusion
Francesco Ronzano, Francesco Barbieri, Endang Wahyu Pamungkas et al.

Overview of the EVALITA 2018 Italian Emoji Prediction (ITAMoji) Task

1 Introduction
2 Emojis and Italian
3 Task Description
4 Task Data
5 Evaluation
5.1 Metrics
5.2 Baseline
5.3 Participating Systems and Results
5.4 Analysis
5.5 Emoji prediction by humans
5.5.1 Figure 8 human annotation
5.5.2 Twitter human annotation
5.5.3 Comparing human and automated emoji predictions
6 Conclusion
Alessandra Teresa Cignarella, Simona Frenda, Valerio Basile et al.

Overview of the EVALITA 2018 Task on Irony Detection in Italian Tweets (IronITA)

1 Introduction
2 Task Description
3 Training and Test Data
3.1 Composition of the datasets
3.2 Annotation of the datasets
3.3 Data Release
4 Evaluation Measures
5 Participants and Results
5.1 Task A: Irony Detection
5.2 Task B: Different types of irony
6 Discussion
7 Concluding remarks
Acknowledgments
Felice Dell’Orletta et Malvina Nissim

Overview of the EVALITA 2018 Cross-Genre Gender Prediction (GxG) Task

1 Introduction
2 Task
3 Data
3.1 Genres
3.2 Train and test sets
3.3 Format
4 Participants and Results
5 Analysis and Discussion
5.1 Approaches
5.2 Results
6 Conclusions
Acknowledgments
Pierpaolo Basile et Nicole Novielli

Overview of the Evalita 2018 – itaLIan Speech acT labEliNg (iLISTEN) Task

1 Introduction
2 Task Description
3 Development and Test Data
3.1 A Dataset of Dialogues
3.2 Annotation Schema
3.3 Data Format and Distribution
4 Evaluation
5 Participants and Results
6 Final Remarks and Conclusions
Francesco Cutugno, Maria Di Maro, Sara Falcone et al.

Overview of the EVALITA 2018 Evaluation of Italian DIALogue systems (IDIAL) Task

1 Task Motivations
2 IDIAL Evaluation Protocol
2.1 Evaluation Method
2.2 Evaluation Procedure
2.3 Submission Requirements
3 Participant Systems
3.1 CH1 Conversational System for Diabetics
3.2 Interactive Media Call-Steering System
3.3 Cedat 85: Speech Technologies in Action
4 Application of the IDIAL Evaluation protocol
4.1 User Experience
4.2 Linguistic Stress Tests
5 Qualitative Analysis and Discussion
6 Post-Evaluation Questionnaire
7 Conclusion
Elisabetta Fersini, Debora Nozza et Paolo Rosso

Overview of the Evalita 2018 Task on Automatic Misogyny Identification (AMI)

1 Introduction
2 Task Description
3 Training and Testing Data
4 Evaluation Measures and Baseline
5 Participants and Results
5.1 Subtask A: Misogyny Identification
5.2 Subtask B: Misogynistic Behaviour and Target Classification
6 Discussion
7 Conclusions and Future Work
Acknowledgements
Cristina Bosco, Felice Dell’Orletta, Fabio Poletto et al.

Overview of the EVALITA 2018 Hate Speech Detection Task

1 Introduction and Motivations
2 Task Organization
3 Datasets and Format
3.1 Facebook Dataset
3.2 Twitter Dataset
3.3 Format and Data in HaSpeeDe
4 Evaluation
5 Overview of the Task: Participation and Results
5.1 Task Participants and Submissions
5.2 Systems
5.3 Results and Discussion
6 Closing Remarks
Acknowledgments
Pierpaolo Basile, Marco de Gemmis, Lucia Siciliani et al.

Overview of the EVALITA 2018 Solving language games (NLP4FUN) Task

1 Motivation
2 Task Description: Dataset, Evaluation Protocol and Measures
2.1 Data Format
2.2 Evaluation
3 Systems
4 Results
5 Conclusions
Maria Di Maro, Antonio Origlia et Francesco Cutugno

Overview of the EVALITA 2018 Spoken Utterances Guiding Chef’s Assistant Robots (SUGAR) Task

1 Introduction
2 Corpus Collection and Description
3 Evaluation Protocol
4 Participating Systems
4.1 FBK-HLT-NLP
4.2 Deep neural network for SUGAR
5 Conclusions
Acknowledgments

Part II. Participant reports

Andrea Cimino, Lorenzo De Mattei et Felice Dell’Orletta

Multi-task Learning in Deep Neural Networks at EVALITA 2018

1 Description of the System
1.1 Lexical Resources
1.1.1 Automatically Generated Sentiment Polarity Lexicons for Social Media
1.1.2 Automatically translated Sentiment Polarity Lexicons
1.1.3 Word Embedding Lexicons
1.2 The Classifier
1.2.1 The SVM classifier
1.2.2 The Deep Neural Network classifier
2 Results and Discussion
2.1 ABSITA
2.2 GxG
2.3 HaSpeeDe
2.4 IronITA
3 Conclusions
Acknowledgments
Rodolfo Delmonte

ItVENSES - A Symbolic System for Aspect-Based Sentiment Analysis

1 Introduction
2 The System and the Modules
3 Sifting Aspect and Polarity with Semantic Sieves
4 Defining Preferences from Persistence
5 Results and Discussion
Emanuele Di Rosa et Alberto Durante

Aspect-based Sentiment Analysis: X2Check at ABSITA 2018

1 Introduction
2 Systems description
2.1 X2C-A
2.2 X2C-B
3 Results on the ABSITA testset
3.1 Aspect Category Detection
3.2 Aspect Category Polarity
4 Conclusions
Giancarlo Nicola

Bidirectional Attentional LSTM for Aspect Based Sentiment Analysis on Italian

1 Introduction
2 Description of the system
3 Training and results
4 Discussion
Mauro Bennici et Xileny Seijas Portocarrero

Ensemble of LSTMs for EVALITA 2018 Aspect-based Sentiment Analysis task (ABSITA)

1 Introduction
2 Description of the system
3 Results
4 Conclusion
Jacob Anderson

Fully Convolutional Networks for Text Classification

1 Introduction
2 Model Description
2.1 Hardware Limitations
2.2 Residual Block
2.3 Dilated Convolution
2.4 Residual Dilated Convolution
2.5 Skip Connections
2.6 Attention and Self-Attention
2.7 Simplified and Local Attention
2.8 Multi-Head Attention
2.9 Model Modifications for Attention
2.10 Global Max Pooling
3 Experiment and Results
3.1 ITAmoji Task
3.2 Results
4 Discussion
4.1 Loss of Information While Pooling
4.2 Potential Uses of Simplified and Local Attention
5 Conclusion
Daniele Di Sarli, Claudio Gallicchio et Alessio Micheli

ITAmoji 2018: Emoji Prediction via Tree Echo State Networks

1 Introduction
2 Task and Dataset
3 Preprocessing
4 Description of the system
4.1 TreeESN models
4.2 CharLSTM model
4.3 Ensemble
5 Training
6 Results
7 Discussion and conclusions
Lucia Siciliani et Daniela Girardi

The UNIBA System at the EVALITA 2018 Italian Emoji Prediction Task

1 Introduction
2 System Description
2.1 Features
3 Evaluation
4 Discussion
5 Conclusion
Andrei Catalin Coman, Yaroslav Nechaev et Giacomo Zara

Predicting Emoji Exploiting Multimodal Data: FBK Participation in ITAmoji Task

1 Introduction
2 Description of the System
2.1 Textual features
2.2 User-based features
2.3 RNN exploiting textual features
2.4 Overall implementation
3 Evaluation setting
4 Results
5 Discussion and Conclusions
Mauro Bennici et Xileny Seijas Portocarrero

The validity of word vectors over the time for the EVALITA 2018 Emoji prediction task (ITAmoji)

1 Introduction
2 Description of the system
3 Results
4 Discussion
5 Conclusion
Andrea Santilli, Danilo Croce et Roberto Basili

A Kernel-based Approach for Irony and Sarcasm Detection in Italian

1 Introduction
2 System Description
2.1 Tweet processing and resources
2.2 Modeling irony and sarcasm in kernel-based learning
2.2.1 Irony-specific Features
2.2.2 Features based on Distribution Polarity Lexicons
2.2.3 Irony Corpus Features
3 Experimental evaluation and results
4 Conclusions
Pierpaolo Basile et Giovanni Semeraro

UNIBA - Integrating distributional semantics features in a supervised approach for detecting irony in Italian tweets

1 Introduction
2 System Description
2.1 Distributional Semantics Features
3 Evaluation
4 Conclusions
Emanuele Di Rosa et Alberto Durante

Irony detection in tweets: X2Check at Ironita 2018

1 Introduction
2 Systems description
2.1 X2C-A
2.2 X2C-B
3 Results and discussion
4 Conclusions
Valentino Giudice

Aspie96 at IronITA (EVALITA 2018): Irony Detection in Italian Tweets with Character-Level Convolutional RNN

1 Introduction
2 Description of the System
3 Results
4 Related work
5 Discussion
Reynier Ortega-Bueno et José E. Medina Pagola

UO_IRO: Linguistic informed deep-learning model for irony detection

1 Introduction
2 UO_IRO system for irony detection
2.1 Preprocessing
2.2 Attention Based LSTM
2.3 Convolutional Neural Network
2.4 Linguistic Features
Stylistic Features
Structural and Content Features
Polarity Contrast Features
3 Experiments and Results
4 Conclusions
Angelo Basile, Gareth Dwyer et Chiara Rubagotti

CapetownMilanoTirana for GxG at Evalita2018. Simple n-gram based models perform well for gender prediction. Sometimes.

1 Introduction
2 Data
3 Experiments
3.1 Pre-processing
3.2 Model and Features
4 Evaluation and Results
4.1 Development Results
4.2 Test Results
5 Conclusions
Danilo Croce et Roberto Basili

A Markovian Kernel-based Approach for itaLIan Speech acT labEliNg

1 Introduction
2 A Markovian Kernel-based Approach
2.1 A Markovian Support Vector Machine
2.2 Structured Kernel Methods for Speech Act Labeling
3 Experimental Results
4 Conclusions
Elena Shushkevich et John Cardiff

Misogyny Detection and Classification in English Tweets: The Experience of the ITT Team

1 Introduction
2 Related work
3 System
3.1 Preprocessing
3.2 Models
4 Results
5 Conclusion
Simona Frenda, Bilal Ghanem, Estefanía Guzmán-Falcón et al.

Automatic Expansion of Lexicons for Multilingual Misogyny Detection

1 Introduction
2 Related Work
3 Proposed Approaches
3.1 Approach 1: using manually-modeled lexica (MML)
3.2 Approach 2: using automatically-enriched lexica (AEL)
3.3 Experiments and Results
4 Discussion and Conclusions
Acknowledgments
Resham Ahluwalia, Himani Soni, Edward Callow et al.

Detecting Hate Speech Against Women in English Tweets

1 Introduction
2 Description of the System
2.1 Task A: Misogyny
2.2 Task B: Category And Target
3 Results and Discussion
3.1 Results for Task A
3.2 Results for Task B
4 Conclusion
Endang Wahyu Pamungkas, Alessandra Teresa Cignarella, Valerio Basile et al.

Automatic Identification of Misogyny in English and Italian Tweets at EVALITA 2018 with a Multilingual Hate Lexicon

1 Introduction
2 Task Description
3 Description of the System
3.1 Features
4 Experimental Setup
5 Results
6 Discussion
7 Conclusions
Acknowledgments
Angelo Basile et Chiara Rubagotti

CrotoneMilano for AMI at Evalita2018. A performant, cross-lingual misogyny detection system.

1 Introduction
2 Data
3 Experiments
3.1 Pre-processing
3.2 Model and Features
4 Evaluation and Results
4.1 Development Results
4.1.1 Cross-lingual Results
4.2 Test Results
5 Discussion and Conclusions
Acknowledgements
Amir Bakarov

Vector Space Models for Automatic Misogyny Identification

1 Introduction
2 Related Work
3 Experimental Setup
4 Results and Discussion
5 Conclusions
Davide Buscaldi

Tweetaneuse @ AMI EVALITA2018: Character-based Models for the Automatic Misogyny Identification Task

1 Introduction
2 Methods
2.1 Locally-weighted Bag-of-Ngrams
2.2 Char and Word-level bi-LSTM
3 Results
4 Conclusions
Acknowledgments
Paula Fortuna, Ilaria Bonavita et Sérgio Nunes

Merging datasets for hate speech classification in Italian

1 Introduction
2 Related Work
3 Methodology
3.1 Data
3.2 Text pre-processing
3.3 Feature extraction and classification
3.3.1 Word embedding
3.3.2 Deep Learning
4 Tasks and runs description
5 Results and Discussion
5.1 Results for Tuning and Validation
5.2 Contest Results
6 Conclusion
Acknowledgments
Marco Polignano et Pierpaolo Basile

HanSEL: Italian Hate Speech detection through Ensemble Learning and Deep Neural Networks

1 Introduction and background
2 Task and datasets description
3 Description of the system
Sentence processing.
Classification strategy.
4 Results and discussion
5 Conclusion
Michele Corazza, Stefano Menini, Pinar Arslan et al.

Comparing Different Supervised Approaches to Hate Speech Detection

1 Introduction
2 System Description
2.1 Preprocessing
2.2 Feature Description
2.3 Recurrent Neural Network Model
2.4 Ngram-based Neural Networks
2.5 Linear SVC System
3 Submitted Runs and Results
3.1 Task 1: HaSpeeDe-FB
3.2 Task 2: HaSpeeDe-TW
3.3 Task 3.1: Cross-HaSpeeDe_FB
3.4 Task 3.2: Cross-HaSpeeDe_TW
4 Error Analysis and Discussion
5 Conclusions
Acknowledgments
Gretel Liz De la Peña Sarracén, Reynaldo Gil Pons, Carlos Enrique Muñiz Cuza et al.

Hate Speech Detection using Attention-based LSTM

1 Introduction
2 System
2.1 Preprocessing
2.2 Method
2.3 Bidirectional LSTM
2.4 Attention Layer
2.5 Post-Attention LSTM
3 Results
4 Conclusion
Acknowledgments
Valentino Santucci, Stefania Spina, Alfredo Milani et al.

Detecting Hate Speech for Italian Language in Social Media

1 Introduction
2 Architecture of the Hate Speech Detector
3 The Tokenizer
4 The Word Embedding Model
5 The Features Generator
6 The Classifier
7 Experiments
7.1 Experimental Setting
7.2 Experimental Results
8 Conclusion and Future Work
Xiaoyu Bai, Flavio Merenda, Claudia Zaghi et al.

RuG @ EVALITA 2018: Hate Speech Detection In Italian Social Media

1 Introduction
2 Task
3 Data and Resources
3.1 Resources Provided by the Organisers
3.2 Additional Resources: Source-Driven Embeddings
4 Systems and Runs
4.1 Run 1: Binary SVM
4.2 Run 2: Binary Ensemble Model
5 Results and Ranking
6 Discussion
7 Conclusion and Future Work
Giulio Bianchini, Lorenzo Ferri et Tommaso Giorni

Text analysis for hate speech detection in Italian messages on Twitter and Facebook

1 Introduction
2 System overview
3 Tools Used
4 Preprocessing
5 Word Embeddings with Word2Vec
6 Model training
7 Experimental Results
8 Conclusion and Future Work
Acknowledgments
Federico Sangati, Antonio Pascucci et Johanna Monti

Exploiting Multiword Expressions to solve “La Ghigliottina”

1 Introduction
2 Related work
3 Solving the Ghigliottina game
3.1 Analysis of real game instances
3.2 Pattern Analysis
3.3 Linguistic Resources
4 System description
5 Results
5.1 Data-set analysis
5.2 System error analysis
6 Conclusions and future work
Acknowledgments
Luca Squadrone

Computer challenges guillotine: how an artificial player can solve a complex language TV game with web data analysis

1 System description
2 The memory of the system: Morph-it!
3 The algorithm
4 Test and Results
4.1 Test the algorithm
4.2 Test the knowledge base: does it always contain the solution?
5 Analysis of the results and future work
Simone Magnolini, Vevake Balaraman, Marco Guerini et al.

The Perfect Recipe: Add SUGAR, Add Data

1 Introduction
2 Task and Data description
3 Data Pre-processing
4 System 1: Memory + Pointer Networks
4.1 Encoder
4.2 Decoder
5 System 2: Fairseq
6 Data augmentation
7 Results
8 Conclusion and Future Work
Acknowledgments
 

Acheter

Rechercher dans OpenEdition Search

Vous allez être redirigé vers OpenEdition Search