Table des matières
Valerio Basile, Danilo Croce, Maria Di Maro et al.
Preface to the EVALITA 2020 ProceedingsValerio Basile, Danilo Croce, Maria Di Maro et al.
EVALITA 2020: Overview of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for ItalianTrack “Affect, Hate, and Stance”
AMI: Automatic Misogyny Identification
Elisabetta Fersini, Debora Nozza et Paolo Rosso
AMI @ EVALITA2020: Automatic Misogyny Identification- 1. Introduction
- 2. Task Description
- Subtask A - Misogyny & Aggressive Behaviour Identification
- Subtask B - Unbiased Misogyny Identification
- 3. Training and Testing Data
- 3.1 Raw dataset
- 3.2 Synthetic dataset
- 4. Evaluation Measures and Baseline
- 5. Participants and Results
- 5.1 Subtask A: Misogyny & Aggressive Behaviour Identification
- 5.2 Subtask B: Unbiased Misogyny Identification
- 6. Discussion
- Textual Feature Representation
- Machine Learning Models
- Error Analysis
- Bias Mitigation strategies.
- 7. Conclusions and Future Work
Arianna Muti et Alberto Barrón-Cedeño
UniBO @ AMI: A Multi-Class Approach to Misogyny and Aggressiveness Identification on Twitter Posts Using AlBERToSamuel Fabrizi
fabsam @ AMI: A Convolutional Neural Network ApproachAlyssa Lees, Jeffrey Sorensen et Ian Kivlichan
Jigsaw @ AMI and HaSpeeDe2: Fine-Tuning a Pre-Trained Comment-Domain BERT ModelGiuseppe Attanasio et Eliana Pastor
PoliTeam @ AMI: Improving Sentence Embedding Similaritywith Misogyny Lexicons for Automatic Misogyny Identification
in Italian Tweets
- 1. Introduction
- 2. Description of the system
- 2.1 Sentence embedding
- 2.2 TF-IDF and Misogyny Lexicons
- Pre-processing
- Input features
- 2.3 Multi-agent prediction
- 2.4 Approach to subtask A
- 2.5 Approach to subtask B
- 3. Results
- 3.1 Experimental setting
- 3.2 Subtask A
- 3.3 Subtask B
- 4. Discussion and Conclusions
- 4.1 Analysis on subtask A
- 4.2 Analysis on subtask B
- Conclusion
Samer El Abassi et Sergiu Nisioi
MDD @ AMI: Vanilla Classifiers for Misogyny IdentificationAdriano dos S.R. da Silva et Norton Trevisan Roman
No Place For Hate Speech @ AMI: Convolutional Neural Network and Word Embedding for the Identification of Misogyny in ItalianATE_ABSITA: Aspect Term Extraction and Aspect-Based Sentiment Analysis
Lorenzo De Mattei, Graziella De Martino, Andrea Iovine et al.
ATE_ABSITA @ EVALITA2020: Overview of the Aspect Term Extraction and Aspect-based Sentiment Analysis TaskFrancesco Mele, Antonio Sorgente et Giuseppe Vettigli
SentNA @ ATE_ABSITA: Sentiment Analysis of Customer Reviews Using Boosted Trees with Lexical and Lexicon-based FeaturesMauro Bennici
ghostwriter19 @ ATE_ABSITA: Zero-Shot and ONNX to speed up BERT on sentiment analysis tasks at EVALITA 2020Emanuele Di Rosa et Alberto Durante
App2Check @ ATE_ABSITA 2020: Aspect Term Extraction and Aspect-based Sentiment AnalysisHaSpeeDe: Hate Speech Detection
Manuela Sanguinetti, Gloria Comandini, Elisa di Nuovo et al.
HaSpeeDe 2 @ EVALITA2020: Overview of the EVALITA 2020 Hate Speech Detection Task- 1. Introduction and Motivations
- 2. Task Description
- 3. Datasets and Formats
- 3.1 Twitter Dataset
- 3.2 News Dataset
- 3.3 Formats
- 4. Evaluation
- 5. Task Overview: Participation and Results
- 5.1 Participants
- 5.2 Systems Overview
- Approaches
- Features and Lexical Resources
- Additional data
- Interaction between Task A and B
- 5.3 Results
- 6. Discussion
- 7. Conclusions
Xiaozhi Ou et Hongling Li
YNU_OXZ @ HaSpeeDe 2 and AMI : XLM-RoBERTa with Ordered Neurons LSTM for classification task at EVALITA 2020- 1. Introduction and Background
- 1.1 Hate Speech (HaSpeeDe 2)
- 1.2 Misogyny (AMI)
- 2. Task and Data description
- 2.1 Task description
- 2.2 Data description
- Description of the system
- 3.1 XLM-RoBERTa and hidden layer state
- 3.2 CNN and K-max Pooling
- 3.3 Ordered Neurons LSTM
- 3.4 Capsule Network
- 3.5 Experiment setting
- 4. Results and Discussion
- 4.1 HaSpeeDe 2 Task A
- AMI subtask A
- 5. Conclusion
Elisa Leonardelli, Stefano Menini et Sara Tonelli
DH-FBK @ HaSpeeDe2: Italian Hate Speech Detection via Self-Training and OversamplingTao Deng, Yang Bai et Hongbing Dai
By1510 @ HaSpeeDe 2: Identification of Hate Speech for Italian Language in Social Media DataRodolfo Delmonte
Venses @ HaSpeeDe2 & SardiStance: Multilevel Deep Linguistically Based Supervised Approach to Classification- 1. Introduction
- 2. The Module for Supervised Automatic Classification
- First approach
- The second approach
- Third Approach
- 3. The Semantically-Based Module And The N-gram Models
- 4. The Experiment and the Evaluation Module of ItVenses
- 4.1 Creating and Accessing N-grams models
- Phase 1. Building fully supervised n-gram models
- Phase 2. Choice constraints on n-grams
- Phase 3. Evaluating n-gram candidates
- 5. Results and Discussion
- 5.1 The Improvements in the Statistical Module
- 6. Conclusion
Julia Hoffmann et Udo Kruschwitz
UR_NLP @ HaSpeeDe 2 at EVALITA 2020: Towards Robust Hate Speech Detection with Contextual EmbeddingsMichele Fontana et Giuseppe Attardi
Fontana-Unipi @ HaSpeeDe2: Ensemble of Transformers for the Hate Speech Task at EvalitaEric Lavergne, Rajkumar Saini, György Kovács et al.
TheNorth @ HaSpeeDe 2: BERT-based Language Model Fine-tuning for Italian Hate Speech Detection- 1. Introduction
- 2. System Description
- 2.1 Fine-tuning process
- 2.2 Layer-wise learning rate
- 2.3 Monolingual and multilingual language models
- 2.4 Random search hyper-parameters tuning
- 2.5 Multitask Learning
- 2.6 Cross-validation ensembling and submitted models
- 3. Data Description
- 4. Results
- 4.1 Macro F1-score
- 4.2 Baselines
- 4.3 Validation Results
- 4.4 Test Results
- 5. Discussion
- 5.1 Multilingual and monolingual models
- 5.2 Out-of-domain data and in-domain data
- 5.3 Multitasking Benefits
- 6. Conclusion
Mariano Jason Rodriguez Cisnero et Reynier Ortega Bueno
UO @ HaSpeeDe2: Ensemble Model for Italian Hate Speech DetectionAdriano dos S.R. da Silva et Norton T. Roman
No Place For Hate Speech @ HaSpeeDe 2: Ensemble to Identify Hate Speech in ItalianSvea Klaus, Anna-Sophie Bartle et Daniela Rossmann
Svandiela @ HaSpeeDe: Detecting Hate Speech in Italian Twitter Data with BERTGiuseppe Gambino et Roberto Pirrone
CHILab @ HaSpeeDe 2: Enhancing Hate Speech Detection with Part-of-Speech Tagging- 1. Introduction
- 2. Description of the Systems
- 2.1 Pre-processing
- 2.2 Part-of-Speech Tagging
- 2.3 Word Embedding
- 2.4 System 1: The Transformer
- 2.5 System 2: Depth-wise Separable Convolutional Neural Network
- 3. Results
- 3.1 HaSpeeDe 2 Task A - Hate Speech Detection
- 3.2 HaSpeeDe 2 Task B - Stereotype Detection
- 3.3 Multilingual Detection of Hate Speech
- 4. Discussion
- 5. Conclusion
Elia Bisconti et Matteo Montagnani
Montanti @ HaSpeeDe2 EVALITA 2020: Hate Speech Detection inOnline Contents
SardiStance: Stance Detection
Alessandra Teresa Cignarella, Mirko Lai, Cristina Bosco et al.


- 1. Introduction/Motivation
- 2. Definition of the Task
- Task A - Textual Stance Detection
- Task B - Contextual Stance Detection
- 3. Data
- 3.1 Collection and annotation of the data
- 3.2 Analysis of the annotation
- 3.3 Composition of the dataset
- 3.4 Data Release
- Task A
- Task B
- 4. Evaluation Measures
- 4.1 Baselines
- 5. Participants and results
- 5.1 Task A: Textual Stance Detection
- Task B: Contextual Stance Detection
- 6. Discussion
- 6.1 Systems participating to Task A
- 6.2 Systems participating to Task B
- 7. Conclusions
Simone Giorgioni, Marcello Politi, Samir Salman et al.
UNITOR @ Sardistance2020: Combining Transformer-based Architectures and Transfer Learning for Robust Stance Detection- 1. Introduction
- 2. Transformer-based architectures and Transfer Learning for Stance Detection
- 2.1 UNITOR as a Transformer-based Architecture
- 2.2 Supporting UNITOR through Auxiliary tasks
- 2.3 Transferring auxiliary tasks in the Transformer-based learning
- 2.4 Distant Supervision for Stance Detection
- 3. Results and Discussion
- 4. Conclusion
Mauro Bennici
ghostwriter19 @ SardiStance: Generating new tweets to classify SardiStance EVALITA 2020 political tweetsRabab Alkhalifa et Arkaitz Zubiaga
QMUL-SDS @ SardiStance: Leveraging Network Interactions to Boost Performance on Stance Detection using Knowledge GraphsMaría S. Espinosa, Rodrigo Agerri, Alvaro Rodrigo et al.
DeepReading @ SardiStance: Combining Textual, Social and Emotional FeaturesFederico Ferraccioli, Andrea Sciandra, Mattia Da Pont et al.
TextWiller @ SardiStance, HaSpeede2: Text or Con-text? A Smart Use of Social Network Data in Predicting PolarizationMaurizio Moraca, Gianluca Sabella et Simone Morra
UninaStudents @ SardiStance: Stance Detection in Italian Tweets - Task AS. Kayalvizhi, D. Thenmozhi et Aravindan Chandrabose
SSN_NLP@SardiStance : Stance Detection from Italian Tweets using RNN and TransformersB. Bharathi, J. Bhuvana et Nitin Nikamanth Appiah Balaji
SSNCSE-NLP @ EVALITA2020: Textual and Contextual Stance Detection from Tweets Using Machine Learning ApproachTrack “Creativity and Style”
CHANGE-IT: Style Transfer
Lorenzo De Mattei, Michele Cafagna, Felice Dell’Orletta et al.
CHANGE-IT @ EVALITA 2020: Change Headlines, Adapt News, GEnerateTAG-it: Topic, Age and Gender Prediction
Andrea Cimino, Felice Dell’Orletta et Malvina Nissim
TAG-it @ EVALITA2020: Overview of the Topic, Age, and Gender Prediction Task for ItalianMaria Fernanda Artigas Herold et Daniel Castro Castro
UO_4to@TAG-it 2020: Ensemble of Machine Learning MethodsRoberto Labadie Tamayo, Daniel Castro Castro et Reynier Ortega Bueno
UOBIT @ TAG-it: Exploring a Multi-faceted Representation for Profiling Age, Topic and Gender in Italian TextsDaniela Occhipinti, Andrea Tesei, Maria Iacono et al.
ItaliaNLP @ TAG-IT: UmBERTo for Author Profiling at TAG-it 2020Track “Semantics and Multimodality”
DANKMEMES: Multimodal Artefacts Recognition
Martina Miliani, Giulia Giorgi, Ilir Rama et al.
DANKMEMES @ EVALITA 2020: The Memeing of Life: Memes, Multimodality and Politics- 1. Introduction
- 2. Task Description
- Task A: Meme Detection
- Task B: Hate Speech Identification
- Task C: Event Clustering
- 3. Dataset
- 3.1 Composition of the dataset
- 3.2 Annotation of the dataset
- 3.3 Training and Test Data
- Dataset for Meme Detection (Task A)
- Dataset for Hate Speech Identification (Task B)
- Dataset for Event Clustering (Task C)
- 3.4 Data release
- 4. Evaluation Measures
- Task A: Meme Detection
- Task B: Hate Speech Identification
- Task C: Event Clustering
- 5. Participants and Results
- Task A: Meme Detection.
- Task B: Hate Speech Identification
- Task C: Event Clustering
- 6. Discussion
- System architecture
- External resources
- Multimodality
- Data Augmentation
- Exploited features
- Event Clustering
- 7. Final Remarks
Stefano Fiorucci
SNK @ DANKMEMES: Leveraging Pretrained Embeddings for Multimodal Meme Detection- 1. System description
- 1.1 General approach and tools
- 1.2 Features
- 1.2.1 DANKMEMES dataset
- 1.2.2 Feature selection and preprocessing
- 1.2.3 Text representation
- 1.2.4 Image representation
- 1.3 System architecture
- 2. Experiments and results
- 2.1 Experimental settings
- 2.2 Results
- 3. Discussion and conclusion
George-Alexandru Vlad, George-Eduard Zaharia, Dumitru-Clementin Cercel et al.
UPB @ DANKMEMES: Italian Memes Analysis - Employing Visual Models and Graph Convolutional Networks for Meme Identification and Hate Speech DetectionJinen Setpal et Gabriele Sarti
ArchiMeDe @ DANKMEMES: A New Model Architecture for Meme Detection- 1. Introduction
- 2. System Description
- 2.1 Metadata
- Engagement
- Date
- Manipulation
- Visual Actors
- 2.2 Textual input
- SentenceTransformers
- 2.3 Visual input
- ResNet
- AlexNet
- DenseNet
- 3. Results
- 4. Other Embedding Approaches
- CNN without Metadata
- Single Pre-trained Image Encoder
- Multimodal Ensemble
- 4.1 Data Augmentation
- 5. Discussion and Conclusion
Claudia Breazzano, Edoardo Rubino, Danilo Croce et al.
UNITOR @ DANKMEMES: Combining Convolutional Models and Transformer-based architectures for accurate MEME managementCONcreTEXT: Concreteness in Context
Lorenzo Gregori, Maria Montefinese, Daniele P. Radicioni et al.
CONcreTEXT @ EVALITA2020: The Concreteness in Context TaskArmand Stefan Rotaru
ANDI @ CONcreTEXT: Predicting concreteness in context for English and Italian using distributional models and behavioural normsAlessandro Bondielli, Gianluca E. Lebani, Lucia C. Passaro et al.
CAPISCO @ CONcreTEXT 2020: (Un)supervised Systems to Contextualize Concreteness with Norming DataYulia Badryzlova
KonKretiKa @ CONcreTEXT: Computing concreteness indexes with sigmoid transformation and adjustment for context- 1. Introduction
- 2. Description of the system
- 2.1 Computation of raw indexes with paradigm words and distributional semantic similarity
- 2.2 Sigmoid transformation of raw indexes
- 2.3 Contextual adjustment
- 3. Results and discussion
- 3.1 Analysis of contextual adjustment
- 3.2 Comparison of computational and psycholinguistic data
- 4. Conclusions
Ghigliottin-AI: Evaluating Artificial Players for the Language Game “La Ghigliottina”
Pierpaolo Basile, Marco Lovetere, Johanna Monti et al.
Ghigliottin-AI @ EVALITA2020: Evaluating Artificial Players for the Language Game “La Ghigliottina”Federico Sangati, Antonio Pascucci et Johanna Monti
“Il Mago della Ghigliottina” @ Ghigliottin-AI: When Linguistics meets Artificial Intelligence- 1. Introduction
- 2. Related work
- 3. The Ghigliottin-AI task
- 4. System description
- 5. Results
- 6. Error analysis
- 6.1 High correlation between clue(s) and our solution
- 6.2 Right kind of reasoning, wrong solution
- 6.3 Clue(s) and solution are synonyms
- 6.4 Unclear solutions
- 6.5 Part-of-speech errors
- 7. Conclusions and future work
Nazareno De Francesco
GUL.LE.VER @ GhigliottinAI: A Glove based Artificial Player to Solve the Language Game “La Ghigliottina”PRELEARN: Prerequisite Relation Learning
Chiara Alzetta, Alessio Miaschi, Felice Dell’Orletta et al.
PRELEARN @ EVALITA 2020: Overview of the Prerequisite Relation Learning Task for Italian- 1. Introduction
- 2. Task Description
- 2.1 Problem Formulation
- 2.2 Task Settings
- 2.3 Evaluation
- Metrics
- Baseline
- 3. Data
- 3.1 Format
- 3.2 Train and Test Sets
- 4. Participants
- 4.1 Submitted Systems
- NLP-CIC
- B4DS
- UNIGE_SE
- 5. Results
- 5.1 Comparing Scenarios
- In–Domain Scenario
- Cross–Domain Scenario
- 5.2 Domains Impact
- 6. Conclusion
Giovanni Puccetti, Luis Bolanos, Filippo Chiarello et al.
B4DS @ PRELEARN: Ensemble Method for Prerequisite LearningAlessio Moggio et Andrea Parizzi
UNIGE_SE @ PRELEARN: Utility for Automatic Prerequisite Learning from Italian WikipediaJason Angel, Segun Taofeek Aroyehun et Alexander Gelbukh
NLP-CIC @ PRELEARN: Mastering Prerequisites Relations, from Handcrafted Features to EmbeddingsTrack “Time and Diachrony”
DaDoEval: Dating Documents
Stefano Menini, Giovanni Moretti, Rachele Sprugnoli et al.
DaDoEval @ EVALITA 2020: Same-Genre and Cross-Genre Dating of Historical DocumentsDIACR-Ita: Diachronic Lexical Semantics
Pierpaolo Basile, Annalina Caputo, Tommaso Caselli et al.
DIACR-Ita @ EVALITA2020: Overview of the EVALITA2020 Diachronic Lexical Semantics (DIACR-Ita) TaskBenyou Wang, Emanuele Di Buccio et Massimo Melucci
University of Padova @ DIACR-Ita- 1. Introduction
- 2. Problem definition
- 3. Methodology
- 3.1 Contextualized Word Embedding
- 3.2 Measuring Semantic Change Degree
- 3.2.1 Distance-based methods
- Average Geometric Distance
- Hausdorff distance
- 3.2.2 Clustering-based Methods
- 3.3 Threshold and Ensemble
- 4. Experiments
- 4.1 Dataset and Evaluation Methodology
- 4.2 Experimental Results
- 4.3 Post-hoc Analysis
- 5. Limitations
- 5.1 Embedding-based Distance Metrics
- Can distance metrics distinguish semantic shift patterns?
- Normalization
- Outliers
- Model Fine-tuning
- 5.2 Clustering-based Distance Metrics
- 6. Conclusions
- A Appendix
Ondřej Pražák, Pavel Přibáň et Stephen Taylor
UWB @ DIACR-Ita: Lexical Semantic Change Detection with CCA and Orthogonal Transformation- 1. Introduction
- 1.1 Related Work
- 1.2 The DIACR-Ita task
- 2. Data
- 3. System Description
- 3.1 Overview
- 3.2 Semantic Space Transformation
- 3.3 Canonical Correlation Analysis
- 3.4 Orthogonal Transformation
- 3.5 Binary Strategy
- 3.6 Ranking Strategy
- 4. Experimental Setup
- 4.1 Corpus variants
- 5. Results
- 6. Conclusion
Rabab Alkhalifa, Adam Tsakalidis, Arkaitz Zubiaga et al.
QMUL-SDS @ DIACR-Ita: Evaluating Unsupervised Diachronic Lexical Semantics Classification in ItalianSeverin Laicher, Gioia Baldissin, Enrique Castañeda et al.
CL-IMS @ DIACR-Ita: Volente o Nolente: BERT does not Outperform SGNS on Semantic Change DetectionJens Kaiser, Dominik Schlechtweg et Sabine Schulte im Walde
OP-IMS @ DIACR-Ita: Back to the Roots: SGNS+OP+CD still Rocks Semantic Change DetectionFederico Belotti, Federico Bianchi et Matteo Palmonari
UNIMIB @ DIACR-Ita: Aligning Distributional Embeddings with a Compass for Semantic Change Detection in the Italian LanguageJason Angel, Carlos A. Rodriguez-Diaz, Alexander Gelbukh et al.
NLP-CIC @ DIACR-Ita: POS and Neighbor Based Distributional Models for Lexical Semantic Change in Diachronic Italian CorporaTrack “New Challenges in Long-standing Tasks”
AcCompl-it: Acceptability & Complexity evaluation
Dominique Brunato, Cristiano Chesi, Felice Dell’Orletta et al.
AcCompl-it @ EVALITA2020: Overview of the Acceptability & Complexity Evaluation Task for ItalianGabriele Sarti
UmBERTo-MTSA @ AcCompl-It: Improving Complexity and Acceptability Prediction with Multi-task Learning on Self-Supervised AnnotationsRodolfo Delmonte
Venses @ AcCompl-It: Computing Complexity vs Acceptability with a Constituent Trigram Model and SemanticsKIPoS: Part-of-speech Tagging on Spoken Language
Cristina Bosco, Silvia Ballarè, Massimo Cerruti et al.
KIPoS @ EVALITA2020: Overview of the Task on KIParla Part of Speech TaggingGiovanni Luca Izzi et Stefano Ferilli
UniBA @ KIPoS: A Hybrid Approach for Part-of-Speech TaggingThomas Proisl et Gabriella Lapesa
KLUMSy@KIPoS: Experiments on Part-of-Speech Tagging of Spoken Italian