Version classiqueVersion mobile
OpenEdition Books

Proceedings of the Second Italian Conference on Computational Linguistics CLiC-it 2015

Cristina Bosco
Sara Tonelli
Fabio Massimo Zanzotto

SentIta and Doxa: Italian Databases and Tools for Sentiment Analysis Purposes

Serena Pelosi


This reserach presents SentIta, a Sentiment lexicon for the Italian language, and Doxa, a prototype that, interacting with the lexical database, applies a set of linguistic rules for the Documentlevel Opinionated teXt Analysis. Details about the dictionary population, the semantic analysis of texts written in natural language and the evaluation of the tools will be provided in the paper.

Questa ricerca presenta SentIta, un lessico dei Sentimenti per l’Italiano, e Doxa, un prototipo che, interagendo con il lessico, applica una serie di regole linguistiche per l’analisi di testi contenenti opinioni. Dettagli in merito al popolamento del dizionario, all’analisi semantica di testi scritti in linguaggio naturale e alla valutazione degli strumenti utilizzati saranno forniti nell’articolo.

Texte intégral

1. Introduction

1Through online customer review systems, Internet forums, discussion groups and blogs, consumers are allowed to share positive or negative information than can influence in different ways the purchase decisions and model the buyer expectations, above all with regard to experience goods (Nakayama et al., 2010); such as hotels (Ye et al., 2011), restaurants (Zhang et al., 2010), movies (Duan et al., 2008), books (Chevalier and Mayzlin, 2006) or videogames (Zhu and Zhang, 2006).

2The consumers, as Internet users, can freely share their thoughts with huge and geographically dispersed groups of people, competing, this way, with the traditional power of marketing and advertising channels.

3Differently from the traditional word-of-mouth, which is usually limited to private conversations, the user generated contents on Internet can be directly observed and described by the researchers.

4The present paper will provide in Section 2 a concise overview on the most popular techniques for both sentiment analysis and polarity lexicon propagation. Afterward, it will describe in Section 3 the method used to semi-automatically populate SentIta, our Italian sentiment lexicon and in Section 4 the rules exploited to put the words’ polarity in context . In the end, Sections 5 will describe our opinion analyzer Doxa, that performs documentlevel sentiment analysis, sentiment role labeling and feature-based opinion summarization.

2. State of the Art

5The most used approaches in sentiment analysis include, among others, the lexicon-based methods, which are grounded on the idea that the text semantic orientation can be inferred by the orientation of words and phrases it contains.

6In literature, polarity indicators are usually adjectives or adjective phrases (Hatzivassiloglou and McKeown, 1997; Hu and Liu, 2004; Taboada et al., 2006); but recently also the use of adverbs (Benamara et al., 2007), nouns (Vermeij, 2005; Riloff et al., 2003) and verbs (Neviarouskaya et al., 2009) became really common.

7Among the most popular lexicons for the sentiment analysis, WordNet-Affect (Strapparava et al., 2004), SentiWordNet (Esuli and Sebastiani, 2006) and SentiFul (Neviarouskaya et al., 2011) deserve to be cited. Because the largest part of the state of the art lexicons focuses on the English language, Italian lexical databases are mostly created by translating and adapting the English ones (Steinberger et al., 2012; Baldoni et al., 2012; Basile and Nissim, 2013; Hernandez-Farias et al., 2014).

8As regards the works on lexicon propagation, we mention thesauri-based works (Kim and Hovy, 2004; Esuli and Sebastiani, 2006; Hassan and Radev, 2010; Maks and Vossen, 2011), corpusbased approaches (Turney, 2002; Baroni and Vegnaduzzo, 2004; Qiu et al., 2009; Wawer, 2012) and morphological strategies (Moilanen and Pulman, 2008; Ku et al., 2009; Neviarouskaya, 2010; Wang et al., 2011).

9Due to the strong impact of the syntactic structures in which the lemmas occur on the resulting polarity of the sentences, we can find in literature many studies on contextual polarity shifters, e.g. Choi and Cardie (2008), Benamara et al. (Benamara et al., 2012) on negation; Kennedy and Inkpen (2006) and Polanyi and Zaenen (2006) on intensification; Taboada et al. (2011) and Narayanan et al. (2009) on irrealis markers and conditional tenses.

3. Prior Polarity: Building and Propagating Italian Lexical Resources

10In this work we created SentIta, a Sentiment lexicon for the Italian language, that has been semiautomatically generated on the base of the richness of the Italian lexical databases of Nooj1 (Silberztein, 2003; Vietri, 2014) and the Italian Lexicon-grammar (LG) resources2 (Elia et al., 1981; Elia, 1984).

11The tagset used for the Prior Polarity annotation (Osgood, 1952) of the lexical resources is composed of four tags (POS positive; NEG negative; FORTE intense and DEB weak), that combined together generate an evaluation scale that goes from -3 (+NEG+FORTE) to +3 (+POS+FORTE) and a strength scale that ranges from -1 (+DEB) to +1 (+FORTE). Neutral words have been excluded from the lexicon.

12Details about the lexical asset available for the Italian language is summarized in Table 1.

13Because hand-built lexicons are definitely more accurate than the automatically-built ones, especially in cross-domain sentiment analysis tasks (Taboada et al., 2011; Bloom, 2011), we started the creation of the SentIta database with the manual tagging of part of the lemmas contained in the Nooj Italian dictionaries. The adjectives and the bad words have been manually extracted and evaluated starting from the Nooj databases, preserving their inflectional (FLX) and derivational (DRV) properties.

Table 1: Composition of SentIta










Compound Adv


a gonfie vele



essere in difetto




Psych Verbs



LG Verbs



Bad words






  • 3 LG classes of compound adverbs: PC, PDETC, PAC, PCA, PCDC, PCPC, PCONG, CAC, CPC, PV, PF, PCPN, PEW (...)
  • 4 LG classes of frozen sentences with essere as support verb: CEAC, EAA, EAPC, ECA, PECO.
  • 5 LG classes of psych verb: 41, 42, 43, 43B.
  • 6 Other LG classes of verbs: 2, 2A, 4, 9, 10, 11, 18, 20I, 20NR, 20UM, 21, 21A, 22, 23R, 24, 27, 28ST (...)

14Compound adverbs3 (Elia, 1990), idioms4 (Vietri, 1990; Vietri, 2011), psych verbs5 and other LG verbs6 (Elia et al., 1981; Elia, 1984) have been manually weighted starting from the Italian LG tables, in order to maintain the syntactic, semantic and transformational properties connected to each one of them.

15However, to manually draw up a complete dictionary is a strong time-consuming activity, therefore we automatically derived the remaining parts of the lexicon exploiting a set of morphological rules.

  • 7 The suffixes for the quality noun formation (Rainer, 2004) are -edine, -edine, -età, -izie, -ela, - (...)

16To be more precise, the Prior Polarity has been assigned to adverbs ending in -mente, “-ly” (Ricca, 2004) on the base of the morpho-phonological relations with their known adjectival bases (e.g. allegro and allegramente), with 0.99 Precision and 0.88 Recall. In a similar way, with a Precision of 0.93 an a Recall of 0.72, we made a set of quality nouns inherit the semantic tags of the qualifier adjectives with which they were related (e.g. eccellente and eccellenza) using a set of 21 suffixes for the noun formation from qualifier adjectives7. In the end, we also collected a list of 37 prefixes (Iacobini, 2004), that, as morphological Contextual Valence Shifters (CVS), are able to negate (e.g. anti-, contra-, non-, ect) or to intensify/downtone (e.g. arci-, semi-, ect) the orientation of the words in which they appear. They directly work on opinionated documents, in order to make the machine understand the actual orientation of the words occurring in real texts, also when their morphological context shifts the polarity of the words listed in the dictionaries.

4. Contextual Polarity: Rules for the Sentence-level Sentiment Analysis

17Grounding a sentiment analysis tool only on the simple lexical valence of negative or positive words can become misleading in all the cases in which the sentence or the discourse context shifts the valence of individual terms.

18In order to annotate real texts with the proper semantic orientation, we took advantage of the finite-state technology, thanks to which we could systematically recall and modify the prior polarities expressed in the dictionaries on the base of the syntactic contexts in which the words occur.

19Among the most used CVS, we took into account linguistic phenomena like Intensification, Negation, Modality and Comparison.

  • 8 As negation indicators we took into account negative operators (e.g. non, “not”, mica), negative qu (...)

20In this work the sentence annotations is not performed through mathematical computations; instead, it is grounded on the semantic labels attributed to each one of the embedded nodes of the finite-state automata (FSA), which contain, just as boxes, all the syntactic structures that should obtain the same score. This choice is due to the fact that the effects of the interactions between word polarities and CVS on the semantics of phrases and sentences are various, but regular. As an example, negation8 does not always switch the polarity of the modified words: as it can be observed in Table 2, there are cases in which it is only shifted.

21Regarding intensification, we considered the cases in which intensifiers and downtoners modify the opinionated words; superlative, and repetition of positive or negative words. Basically, adjectives can only modify the intensity of nouns, while adverbs intensify or attenuate adjectives, verbs and other adverbs.

22Because intensification and negation can also appear together in the same sentence, we took into account this eventuality by weighting firstly the intensification and then the negation.

Table 2: An example of intensification and negation rules on a positive adjective








bello, bello



molto bello






poco bello



non bello


Adj+2+Neg + Adv+

non molto bello



non bellissimo



non poco bello


23As concerns modality (Benamara et al., 2012), we contemplated modal verbs (e.g. dovere, potere), conditional tenses (Narayanan et al., 2009) and doubt or necessity adverbs (e.g. più o meno, di sicuro).

24In order to detect similarities and differences between two or more objects, we listed a set of comparative opinion indicators; namely, comparative frozen sentences of the type N0 Agg come C1 (Vietri, 1990); some simple comparative sentences that involve the expressions meglio di, inferiore a; and the comparative superlative (e.g. il più, il peggiore).

5. Doxa: a Document-level Opinionated teXt Analyzer

25Using the command-line program noojapply.exe, we built a Java prototype by which users can automatically apply the resources formalized in Nooj to every kind of text.

26Doxa, in its standard version, is a Documentlevel Opinionated teXt Analyzer that evaluates customer reviews, by summing up the values corresponding to every sentiment expression and, then, normalizes the result for the total number of sentiment expressions contained in the text. In the end, it compares the resulting values with the stars that the opinion holders gave to their reviews and provides statistics about the opinions expressed in each case (Maisto and Pelosi, 2014a).

27The test dataset used to evaluate the performances of Doxa contains Italian opinionated texts in the form of users reviews and comments from e-commerce and opinion websites. It lists 600 texts units (300 positive and 300 negative for each product class) and refers to experience and search goods, for all of which different websites have been exploited9.

28It the sentence-level sentiment analysis Doxa achieved an average Precision of 0.75 and a Recall of 0.73, and in the document-level classification an average Precision of 0.74 and a Recall of 0.97.

29Although the document-level analysis provides important information regarding the consumers needs and expectations, for companies it is crucial to discern the aspects of the products that must be improved, or whether the opinions extracted online by the sentiment analysis applications are relevant to the products or not. That is why we designed new sentiment analysis modules for Doxa: a sentiment role labeling module (Pelosi et al., 2015 in press) and a feature-based sentiment analyzer (Maisto and Pelosi, 2014b).

Figure 1: Sentiment Role Labeling

Figure 1: Sentiment Role Labeling

30In the first task the achieved F-score was 0.71 in a Twitter dataset and 0.76 in a free web news headings dataset provided by DataMediaHub. As shown in Figure 1, the tool is based on the matching between the definitional syntactic structures, attributed to the each one of the 28 LG class of Italian verbs, and the semantic information attached in the sentiment database to every lexical entry. The semantic roles evoked by this selection of verbs pertain to three frames: Sentiment (experiencer, causer), Opinion (holder, target) and Physical act (agent, patient).

Figure 2: Feature-based module of Doxa

Figure 2: Feature-based module of Doxa

31In the feature based sentiment analysis (Maisto and Pelosi, 2014b), which allows the comparison between more than one object on the base of different kind of aspects that characterize them (Figure 2), we achieved 0.81 F-score on a corpus of hotels reviews.

32Because sometimes a lexicon is not enough to correctly extract and classify the product features, domain-specific FSA have been formalized in order to make the analyses adequate to the corpus (e.g. L’hotel vicinissimo alla metro., BENEFIT TYPE=“LOCATION” SCORE=“3”).

6. Conclusion

33In the present paper we underlined that the social and economic impact of the online customer opinions and the huge volume of raw data available on the web, concerning users point of views, offer new opportunities both to marketers and researchers.

34Indeed, sentiment analysis applications, able to go deep in the semantics of sentences and texts, can play a crucial role in tasks like web reputation monitoring, in social network analysis, in viral tracking campaigns, etc...

35Therefore, we presented SentIta, a semiautomatic Italian Lexicon for Sentiment Analysis, and Doxa, a Document-level Opinionated teXt Analyzer that exploits finite-state technologies to go through the subjective dimension of user generated contents.


Matteo Baldoni, Cristina Baroglio, Viviana Patti, and Paolo Rena. 2012. From tags to emotions: Ontology-driven sentiment analysis in the social semantic web. Intelligenza Artificiale, 6(1):41–54.

Marco Baroni and Stefano Vegnaduzzo. 2004. Identifying subjective adjectives through web-based mutual information. 4:17–24.

Valerio Basile and Malvina Nissim. 2013. Sentiment analysis on italian tweets. In Proceedings of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pages 100–107.

Farah Benamara, Carmine Cesarano, Antonio Picariello, Diego Reforgiato Recupero, and Venkatramana S Subrahmanian. 2007. Sentiment analysis: Adjectives and adverbs are better than adjectives alone. In ICWSM.

Farah Benamara, Baptiste Chardon, Yannick Mathieu, Vladimir Popescu, and Nicholas Asher. 2012. How do negation and modality impact on opinions? pages 10–18.

Kenneth Bloom. 2011. Sentiment analysis based on appraisal theory and functional local grammars.

Judith A Chevalier and Dina Mayzlin. 2006. The effect of word of mouth on sales: Online book reviews. Journal of marketing research, 43(3):345– 354.

Yejin Choi and Claire Cardie. 2008. Learning with compositional semantics as structural inference for subsentential sentiment analysis. pages 793–801.

Wenjing Duan, Bin Gu, and Andrew B Whinston. 2008. The dynamics of online word-of-mouth and product salesan empirical investigation of the movie industry. Journal of retailing, 84(2):233–242.

Annibale Elia, Maurizio Martinelli, and Emilio D’Agostino. 1981. Lessico e Strutture sintattiche. Introduzione alla sintassi del verbo italiano. Napoli: Liguori.

Annibale Elia. 1984. Le verbe italien. Les complétives dans les phrases à un complément.

Annibale Elia. 1990. Chiaro e tondo: LessicoGrammatica degli avverbi composti in italiano. Segno Associati.

Andrea Esuli and Fabrizio Sebastiani. 2006. Determining term subjectivity and term orientation for opinion mining. 6:2006.

Livio Gaeta. 2004. Nomi d’azione. La formazione d elle parole in italiano. Tübingen: Max Niemeyer Verlag, pages 314–51.

Ahmed Hassan and Dragomir Radev. 2010. Identifying text polarity using random walks. pages 395– 403.

Vasileios Hatzivassiloglou and Kathleen R McKeown. 1997. Predicting the semantic orientation of adjectives. In Proceedings of the 35th annual meeting of the association for computational linguistics and eighth conference of the european chapter of the association for computational linguistics, pages 174– 181. Association for Computational Linguistics.

Irazú Hernandez-Farias, Davide Buscaldi, and Belém Priego-Sánchez. 2014. Iradabe: Adapting english lexicons to the italian sentiment polarity classification task. In First Italian Conference on Computational Linguistics (CLiC-it 2014) and the fourth International Workshop EVALITA2014, pages 75–81.

Minqing Hu and Bing Liu. 2004. Mining opinion features in customer reviews. In AAAI, volume 4, pages 755–760.

Claudio Iacobini. 2004. Prefissazione. La formazione delle parole in italiano. Tübingen: Max Niemeyer Verlag, pages 97–161.

Alistair Kennedy and Diana Inkpen. 2006. Sentiment classification of movie reviews using contextual valence shifters. Computational intelligence, 22(2):110–125.

Soo-Min Kim and Eduard Hovy. 2004. Determining the sentiment of opinions. page 1367.

Lun-Wei Ku, Ting-Hao Huang, and Hsin-Hsi Chen. 2009. Using morphological and syntactic structures for chinese opinion analysis. pages 1260–1269.

Alessandro Maisto and Serena Pelosi. 2014a. Featurebased customer review summarization. In On the Move to Meaningful Internet Systems: OTM 2014 Workshops, pages 299–308. Springer.

Alessandro Maisto and Serena Pelosi. 2014b. A lexicon-based approach to sentiment analysis. the italian module for nooj. In Proceedings of the International Nooj 2014 Conference, University of Sassari, Italy. Cambridge Scholar Publishing.

Isa Maks and Piek Vossen. 2011. Different approaches to automatic polarity annotation at synset level. pages 62–69.

Karo Moilanen and Stephen Pulman. 2008. The good, the bad, and the unknown: morphosyllabic sentiment tagging of unseen words. pages 109–112.

Makoto Nakayama, Norma Sutcliffe, and Yun Wan. 2010. Has the web transformed experience goods into search goods? Electronic Markets, 20(34):251–262.

Ramanathan Narayanan, Bing Liu, and Alok Choudhary. 2009. Sentiment analysis of conditional sentences. pages 180–189.

Alena Neviarouskaya, Helmut Prendinger, and Mitsuru Ishizuka. 2009. Compositionality principle in recognition of fine-grained emotions from text. In ICWSM.

Alena Neviarouskaya, Helmut Prendinger, and Mitsuru Ishizuka. 2011. Sentiful: A lexicon for sentiment analysis. Affective Computing, IEEE Transactions on, 2(1):22–36.

Alena Neviarouskaya. 2010. Compositional approach for automatic recognition of fine-grained affect, judgment, and appreciation in text.

Charles E Osgood. 1952. The nature and measurement of meaning. Psychological bulletin, 49(3):197.

Serena Pelosi, Annibale Elia, and Alessandro Maisto. 2015 (in press). Towards a lexicon-grammar based framework for nlp: an opinion mining application. In Recent Advances in Natural Language Processing 2015 - RANLP 2015 Proceedins.

Livia Polanyi and Annie Zaenen. 2006. Contextual valence shifters. pages 1–10.

Guang Qiu, Bing Liu, Jiajun Bu, and Chun Chen. 2009. Expanding domain sentiment lexicon through double propagation. 9:1199–1204.

Franz Rainer. 2004. Derivazione nominale deaggettivale. La formazione delle parole in italiano, pages 293–314.

Davide Ricca. 2004. Derivazione avverbiale. La formazione delle parole in italiano, pages 472–489.

Ellen Riloff, Janyce Wiebe, and Theresa Wilson. 2003. Learning subjective nouns using extraction pattern bootstrapping. In Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003-Volume 4, pages 25–32. Association for Computational Linguistics.

Max Silberztein. 2003. Nooj manual. Available for download at:

Josef Steinberger, Mohamed Ebrahim, Maud Ehrmann, Ali Hurriyetoglu, Mijail Kabadjov, Polina Lenkova, Ralf Steinberger, Hristo Tanev, Silvia Vázquez, and Vanni Zavarella. 2012. Creating sentiment dictionaries via triangulation. Decision Support Systems, 53(4):689–694.

Carlo Strapparava, Alessandro Valitutti, et al. 2004. Wordnet affect: an affective extension of wordnet. In LREC, volume 4, pages 1083–1086.

Maite Taboada, Caroline Anthony, and Kimberly Voll. 2006. Methods for creating semantic orientation dictionaries. In Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC), Genova, Italy, pages 427–432.

Maite Taboada, Julian Brooke, Milan Tofiloski, Kimberly Voll, and Manfred Stede. 2011. Lexiconbased methods for sentiment analysis. Computational linguistics, 37(2):267–307.

Peter D Turney. 2002. Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews. pages 417–424.

MJM Vermeij. 2005. The orientation of user opinions through adverbs, verbs and nouns. In 3rd Twente Student Conference on IT, Enschede June. Citeseer.

Simonetta Vietri. 1990. On some comparative frozen sentences in italian. Lingvisticæ Investigationes, 14(1):149–174.

Simonetta Vietri. 2011. On a class of italian frozen sentences. Lingvisticæ Investigationes, 34(2):228– 267.

Simona Vietri. 2014. The italian module for nooj. In In Proceedings of the First Italian Conference on Computational Linguistics, CLiC-it 2014. Pisa University Press.

Xin Wang, Yanqing Zhao, and Guohong Fu. 2011. A morpheme-based method to chinese sentence-level sentiment classification. Int. J. of Asian Lang. Proc., 21(3):95–106.

Aleksander Wawer. 2012. Extracting emotive patterns for languages with rich morphology. International Journal of Computational Linguistics and Applications, 3(1):11–24.

Qiang Ye, Rob Law, Bin Gu, and Wei Chen. 2011. The influence of user-generated content on traveler behavior: An empirical investigation on the effects of e-word-of-mouth to hotel online bookings. Computers in Human Behavior, 27(2):634–639.

Lei Zhang, Bing Liu, Suk Hwan Lim, and Eamonn O’Brien-Strain. 2010. Extracting and ranking product features in opinion documents. In Proceedings of the 23rd international conference on computational linguistics: Posters, pages 1462–1470. Association for Computational Linguistics.

Feng Zhu and Xiaoquan Zhang. 2006. The influence of online consumer reviews on the demand for experience goods: The case of video games. ICIS 2006 Proceedings, page 25.


1 The Nooj software and the Italian module of Nooj are available for download at

2 LG tables anavailable for consultation at

3 LG classes of compound adverbs: PC, PDETC, PAC, PCA, PCDC, PCPC, PCONG, CAC, CPC, PV, PF, PCPN, PEW. This list comprises 70 discourse operators, able to summarize (e.g. in parole povere), invert (e.g. nonostante ciò), confirm (e.g. in altri termini), compare (e.g. allo stesso modo) and negate (e.g. neanche per sogno) the opinion expressed in the previous sentences of the text.

4 LG classes of frozen sentences with essere as support verb: CEAC, EAA, EAPC, ECA, PECO.

5 LG classes of psych verb: 41, 42, 43, 43B.

6 Other LG classes of verbs: 2, 2A, 4, 9, 10, 11, 18, 20I, 20NR, 20UM, 21, 21A, 22, 23R, 24, 27, 28ST, 44, 44B, 45, 45B, 47, 47B, 49, 50, 51, 53, 56.

7 The suffixes for the quality noun formation (Rainer, 2004) are -edine, -edine, -età, -izie, -ela, -udine, -ore, (z)ione, -anza, -itudine, -ura, -mento, -izia, -enza, -eria, -ietà, -aggine, -ia, -ità, -ezza, -igia, -(z)a. They generally make the new words simply inherit the orientation of the derived adjectives. Exceptions are -edine and -eria that almost always shift the polarity of the quality nouns into the weakly negative one (-1), e.g. faciloneria “slapdash attitude”. Also the suffix -mento differs from the others, in so far it belongs to the derivational phenomenon of the deverbal nouns of action (Gaeta, 2004). It has been possible to use it in this work noun derivation by using the past participles of the verbs listed in the adjective dictionary of sentiment.

8 As negation indicators we took into account negative operators (e.g. non, “not”, mica), negative quantifiers (e.g. nessuno, “nobody” niente, nulla, “nothing”) and lexical negation (e.g. senza, “without”, mancanza di, assenza di, carenza di, “lack of”) (Benamara et al., 2012).

9 Books:,, Movies:,,,, Hotels:,,,, and Videogames:, Cars:, Smartphones:,,,

Table des illustrations

Titre Figure 1: Sentiment Role Labeling
Fichier image/jpeg, 44k
Titre Figure 2: Feature-based module of Doxa
Fichier image/jpeg, 84k


Department of Political, Social and Communication Science - University of Salerno -


Open access