1. Introduction

1In recent years, public administrations (PA) in the Italian government have been forced to publish a huge amount of documents, to make them available to citizens, organisations, and authorities. This is the result of the recent legislation about the transparency. For instance, municipalities have to share their documents in a virtual place called Albo Pretorio. In most cases, the online publication of these acts is a necessary condition for their purposes to become effective.2

2On the other side, the General Data Protection Regulation (GDPR), approved in 2016 by the European Union, enhances individuals’ control and rights over their personal data, limiting its diffusion over any medium (especially including online platforms such as websites and social networks).

3In this context, it is important for the public servants within the PA to amend some documents by hiding the data that cannot be publicly published. Nowadays, most of this work is done manually, hiding the sensitive information document by document. This procedure is clearly time-consuming, non-scalable, and error-prone.

4Natural Language Processing (NLP) techniques can be seen as a watershed between a manual management of the PA documents and a new generation of instruments that will finally speed up the process, leaving manual effort as the sole final check just before the publication of the data.

5This is not the first time this problem is tackled using NLP, but past works are mainly focused on English and limited to the entity extraction task (Guo et al. 2021).

6Our approach to the extraction of personal data from documents focuses on a combination of three NLP instruments:

• Named-entity Recognition (NER). This task consists in seeking texts in natural language to locate and classify named entities (NE) mentioned in them. This search is usually limited to a few needed categories: the most common are persons, locations, and organisations. Several approaches have been used in literature, between completely rule-based (Appelt et al. 1993; Budi and Bressan 2003) and machine learning-based (Chiu and Nichols 2016; Strubell et al. 2017; Devlin et al. 2019), including some hybrid approaches, for example using gazettes of known entities belonging to a particular category (Finkel, Grenager, and Manning 2005). In this paper, we use the last approach, mixing a Conditional Random Fields (CRF) algorithm (Lafferty, McCallum, and Pereira 2001) with the addition of a list of entities, extracted from various knowledge bases, that describe persons, companies, and locations. We describe this process in detail in Section 5.

• Structured-entity Identification. A parallel rule-based task is used to extract entities that can easily be recognised without the need of training data. Among them: dates and times, numbers, email addresses, Italian “codice fiscale”, that are based on textual patterns; roles and document types, that are based on prepacked lists.

• Relation extraction (RE). It is the task of extracting semantic relationships from text. Extracted relationships usually occur between two or more entities of a certain type (for example persons, locations, etc., see previous points), and fall into a number of semantic categories (such as birth location, role in a company, etc.). Relation extraction is widely used also in specific domains such as medicine (Giuliano et al. 2007) and finance (Vela and Declerck 2009). Successful experiments made use of Conditional Random Fields (Surdeanu et al. 2011), Dependency-based Neural Networks (Liu et al. 2015), and transformers like BERT (Baldini Soares et al. 2019).

7In this paper we present REDIT (Relation and Entities Dataset for Italian with Tint), a complete framework that aims to solve the personal data identification in textual documents. The software is mainly based on Tint (Palmero Aprosio and Moretti 2018), an NLP pipeline specifically designed for Italian and based on Stanford CoreNLP (Manning et al. 2014). REDIT includes part of the annotated dataset (Section 4), the compiled model, and the supporting Java code. It is available for free on Github (see Section 6).

8The content is structured as follows. Section 2 presents in detail how we collected the documents that are annotated and how we used fictitious data to make the resource available for download. In Section 3, we describe the process used to annotate the data. Section 4 illustrates the dataset, giving some statistics on the entities and relations included in it. In Section 5 we give some results on the performance of the resulting entity extraction and relation extraction system. The downloadable package (that contains the dataset, the model and the Java code) is finally described in Section 6.

2. Data Collection

9The corpus is composed of documents taken from different institutions of the public administration. The documents with which we have worked are different types of forms, varying from license for parking to adoption forms, school enrollments, marriage licenses and so on.

10Starting from this set, we create two datasets. One is composed of documents compiled with real data and one with documents compiled by us with fictitious data, using lists of all the Italian streets and surnames in order to guarantee the diversification of the data in the compiled forms, and to not exclusively rely on the annotators’ fantasy. The fictitious compilation aims to avoid using sensitive data in terms of privacy issues, leading to the possibility of publicly releasing the dataset. The documents which contain real data are indeed not included in the public dataset. For instance, a sentence such as Il sottoscritto Gianluca Freschi, nato a Pesaro il 12/12/1990 e residente in Pesaro, Via Virgilio n.76 presents data whose association was invented by the annotator. It could be possible that a person called Gianluca Freschi exists in real world but it is almost impossible that he would fit with the rest of the data since they all derive by annotator’s fantasy. However, as we can see from the example, while the data are fictitious the structure of the document is identical to that of real ones.

3. The Annotation

11Each document in the set is annotated both with entities and relations between them.

12For the annotation of entities we adopt the guidelines already used for KIND (Paccosi and Palmero Aprosio 2021), a corpus containing NE on documents taken from Wikinews. The named entities included in KIND belong to the standard NE classes and are of three types: LOC, PER, and ORG. As already noticed by (Passaro et al., 2017), these categories are quite unsatisfactory to deal with the information contained in the PA documents, since the model is not designed at capturing information such as laws or protocols. In REDIT we then distinguish different types of ORGs, differentiating public offices and municipality and companies: the former is annotated as ENTE, while the latter as usual (ORG). Finally, we add a label to mark laws and protocols, LEX, so that in the present work there are five types of annotated entities: LOC, PER, ORG, LEX, and ENTE. The original guidelines used in KIND have therefore been slightly modified to meet our needs (see Section 5.1 for more details).

13In addition to the NE annotation, we are interested in annotating the relations among them. In particular, we need to develop a system of relations which links the person with its personal data or with its role in terms of responsibility of the company/public administration or in terms of relative/family relationships.

14Since for the annotation task a relation must connect two entities, some additional entity types are annotated only when involved in a relation (see below). The list of additional entities includes ROLE for personal and organisation roles (for example, words such as “responsabile”, “titolare”, “genitore”, and so on, representing the role of a person in a company, in the PA domain, or in a family), DOCTYPE for document types (such as “passaporto”, “patente”), EMAIL for e-mail addresses, DATE for dates, NUMBER for generic numbers (such as VAT), CF for the Italian “codice fiscale” sequence of chars.

15Regarding relations, address is used for instance to link a LOC entity representing an address to the person or company to which the address belongs, while birthDate, birthLoc link respectively the date and location of birth.

16Table 1 shows the complete list of the relations included in the dataset.

17The annotation is performed by a domain expert using INCEpTION (Klie et al. 2018), a web-based text-annotation environment which allows users to: (i) select a group of tokens and assign a label to it (entities); (ii) connect two entities among them and assign a label to the link (relations).

18This is an example of NER annotation:

Al [Comune di Alessandria]_ENTE.
[Casale Monferrato]_LOC, 20 settembre 2021.
Il sottoscritto [Davide Aiello]_PER, nato a [Milano]_LOC il [31/07/1985]_DATE, [titolare]_ROLE della ditta [Aiello Ceramiche S.r.l.]_ORG, ai sensi dell’ [art. 76 del D.P.R. n. 445/2000]_LEX, dichiara di voler partecipare all’evento “Il mercante in Fiera”.

19These are the corresponding relations:

• birthLoc (Davide Aiello, Milano)

• birthDate (Davide Aiello, 31/07/1985)

• companyRole (Davide Aiello, titolare)

• personInOrg (Davide Aiello, Aiello Ceramiche S.r.l.)

20In the example, “31/07/1985” is tagged as DATE, since it is involved in the birthDate relation. On the contrary, since no relations include “20 settembre 2021”, it’s not mandatory, for the annotator, to mark it as DATE.

21The system uses two different approaches to identify entities. Entities such as DATE or ROLE are annotated only when involved in a relation because they are labels identified through a rule-based approach which can be easily recognised without the need of training data. For what concerns instead PER, LOC, ORG, ENTE and LEX the identification occurs using a machine-learning technique and they need to be always annotated.

4. The Dataset

22As we have seen in Section 2, the complete dataset consists of two parts: the first one presents the documents fictitiously compiled and it is publicly released; the latter, on the contrary, comprehends instances compiled with real data and is not released. Nevertheless, we consider also the unreleased dataset in training the model, so that the amount of annotated relations in the final dataset is 7,821, while that of annotated entities is 21,307. The released one presents 1,439 annotated entities and 1,476 annotated relations.

23Looking at the data in Table 1, it is possible to notice that the amount of annotations referring to some relations (marked with *) are considerably fewer than others. Despite the small amount, we have already annotated them in the view of future works on these relations but we do not consider them in the experiments.

5. The Pipeline

24To work properly, REDIT relies on a complex pipeline that includes various steps, very different in structure and management (see Figure 1). Most of the steps are performed using well-known tools and algorithms (sometimes not reaching state-of-the-art accuracy), so that the whole program does not need particular hardware (such as the GPUs needed in environments using deep learning and transformers) and is easy to run on almost every common software environment.

Figure 1: A chart depicting the REDIS architecture and its interaction with Tint

1. First, the input text is parsed with Tint (Palmero Aprosio and Moretti 2018) using these annotators: tokenizer, sentence splitter, truecaser, part-of-speech tagger, lemmatizer, dependency parser.

2. Named-entities are extracted using the CRF implementation included in Stanford NER (Finkel, Grenager, and Manning 2005) and the model trained on the annotated dataset (see Subsection 5.1).

3. A second run on named-entities, with the rule-based Stanford TokensRegex software (Chang and Manning 2014), is performed (see Subsection 5.2)

4. ORG and LOC entities are passed into a Support Vector Machines classifier (Cortes and Vapnik 1995) to extract ENTE entities (see Subsection 5.3).

5. Finally, the Stanford Relation Extractor (Surdeanu et al. 2011) is used to find relations between entities in the text (see Subsection 5.4).

5.1 The CRF Named-Entities Tagger

25Since the sole REDIT dataset is not sufficient to train a robust NER tagger, we use it in combination with KIND (see Section 3). Guidelines for the two datasets are, of necessity, slightly different, therefore we need to use some precautions in merging them.

26Sometimes, the entities annotated as ORG in KIND (such as “Unione Europea”) should have been annotated as ENTE in REDIT. We then decided, in the training phase, to merge all ENTE entities into ORG. We then trained a classifier dedicated to the ENTE tag (see Subsection 5.3), trained on REDIT dataset only, that performs the sole disambiguation between ORG and ENTE.

27To enhance the classification, Stanford NER also accepts gazettes of names labelled with the corresponding tag. We collect a list of persons, organizations and locations from the Italian Wikipedia using some classes in DBpedia (Auer et al. 2007): Person, Organisation, and Place, respectively. In addition to this, we collect the list of streets from OpenStreetMap (OpenStreetMap contributors 2017), limiting the extraction to Italian names. Table 3 shows statistics about the gazettes.

28The evaluation is performed by randomly splitting the dataset into train/dev/test using 80/10/10 ratio. During training phase, we tried some sets of features choosing among the ones available in Stanford NER. We obtained the best results (considering also a good balance between training/testing time and performances) with word shapes, n-grams with length 6, previous, current, and next token/lemma/class. Table 4 displays the results of the NER module.

5.2 The Rule-Based Named-Entities Tagger

29As said in Section 3, there is the need for more entity types, because in the training phase we need to have both arguments of a relation annotates as an entity (of any type). For this reason, we use a rule-based approach to annotate DATE, ROLE, DOCTYPE, EMAIL, NUMBER, and CF.

• Tint TIMEX annotator is used to tag DATE entities.

• ROLE and DOCTYPE entities are extracted given a list of roles taken from the annotated training set.

• Numbers, e-mail addresses and Italian codice fiscale are tagged using regular expressions.

5.3 The SVM Classifier for ENTE Entities

30After the previous steps, the entities that should be marked with ENTE now falls into the ORG or LOC entity sets. We then use a simple SVM classifiers (using shallow features, such as words, bigrams, previous and following content words, etc.) that, given an entity tagged as LOC or ORG, return whether it should be annotated as ENTE. The training set used by the classifier consists in entities taken from REDIT and annotated as ORG, LOC, and ENTE. The first two categories represent the zero class, while entities tagged with ENTE represent the other class. It is therefore a binary classifier. In a 10-fold cross-validation environment, results shows a F-score equals to 0.978 (precision 0.981, recall 0.974).

5.4 Relation Extractor Module

31The last module in REDIT is Stanford Relation Extractor (Surdeanu et al. 2011), used to train and extract relations in the text.

32Similarly to the NER training, we test approaches with different sets of features, obtaining the best results with unigrams/bigrams, adjacent words, argument words, argument class, dependency path between the arguments, entities and concatenation of POS tags between arguments.

33Table 5 shows the results on the relation extractor (the evaluation is performed using gold-labeled entities).

6. The Release

34All parts of REDIT (except part of the annotated dataset, see Section 4) are released for free under the CC BY 4.0 license,3 and can be downloaded on Github.4 These include the annotations, in WebAnno format (Yimam et al. 2013), the gazettes, both the NER and the RE models (created using the whole corpus), and the source code, written in Java, used to parse the files and run the classifiers.

35A working demo of the tool is available online (See Figure 2).5 Its web interface is written with VueJS/Boostrap and it is available for download in the Github project page.

Figure 2: A screenshot of the demo interface.

7. Conclusion and Future Work

36In this paper we present a completely automatic approach to extract personal data (view as entities) and relations between them from documents of the public administration written in Italian texts. The pipeline relies on a mix of rule-based and machine learning-base modules. The latter are trained using a manually annotated dataset, which is in part available for download. All the source code, instead, is released and available for download.

37In the future, we plan to enhance the coverage of our system by adding more examples on relations that are less represented (see Table 1).