1. Prosopographical research and the method of Record Linkage

1The main concern of my paper is to present some ideas about the support that can be gamed for prosopographical research by using the software system kleio1. Due to the very nature of prosopographical research2, the emphasis of my presentation has been put onto the record linkage features3; even in the definition of the School of prosopographical research from Freiburg and Munster prosopography has to be understood as the description of persons with regard to their functions and to their relations to other persons4. With the emphasis being put onto a description of individuals as comprehensive as possible this denotation fits well the concept of historical research that has been presented by Ian Winchester at a conference on nominal record linkage in 1973 In defining the methodology of record linkage, he stated that "... historical reference, i.e. referring to individuals in the past and trying to identify them, cannot be understood without consideration of the notion of record linkage"5. Therefore, he argued that "without record linkage the increase of historical knowledge would be strictly impossible"6.

2Restricting our reasoning to medieval prosopography and to the social historical approach of micro analysis this notion can be proved easily. In both fields we are dealing with numerous references to persons in a great number of sources. And in both fields the first challenge to the historian is to merge this information at the level of the individual person in order to create a prosopographical catalogue which will contain all of our knowledge about every person in question7.

3The creation of a prosopographical catalogue is based upon the concept of "historical individuation". According to the definition of Ian Winchester, it "consists in bringing together pairs of records pertaining to histoneal individuals, and comparing certain items on such records as to their agreement or disagreement, when the items are present on both records In the event that all, or at any rate a sufficient number of the identifying items on a particular pair of records are in agreement, the histonan may then judge that a histoneal individuation has been successfully brought about"8. After every feasible individuation has been done successfully, all records belonging to the same individual are merged to make up the prosopographical catalogue. Usually this comparing and merging of references is referred to as nominal record linkage. Thus, working in the field of prosopography necessitates not only the consideration of record linkage as a methodology but also the application of nominal record linkage as the associated research strategy. It is defined by E. A. Wrigley as "the process by which items of information about a particular named individual are associated with each other into a coherent whole in accordance with certain rules"9 The final aim of this strategy is to create a "meta-source" that supplies information about those aspects of the social reality in question that can not be learned from the analysis of an unlinked source. Going beyond the limits of the existing knowledge has to be regarded thus as the main impulse for using nominal record linkage. From this angle, E. A. Wrigley makes a quite interesting comparison between the historians motivation to employ this research strategy and the instigation of technical and methodological innovations within the sciences during the 18th and the 19th century. In both disciplines there was an attempt to establish some evidence about realities that could not be seen with the "naked eye"10. Only the application of new techniques and new methods promised to allow the access to new worlds. Concerning the method of nominal record linkage this access is certainly not the only result of a simple reference to an otherwise hidden reality. Following the "conjectural or semiotic paradigm"11 the historian uses his ingenious ability, when he takes all accessible clues into account to make the individuation as reliable as possible and to generate the prosopographical catalogue. Thus, the outcome of the nominal record linkage is almost the creation of a new reality. But the historians ingenuity is not the only influence on its composition. There are at least two further aspects to be regarded:

1. We have to consider the certainty of the individuation, that is the certainty of the identification and re-identification of individuals. From this point of view, the reliability of the identifying items and their discriminating power are of main importance.
2. We must not forget the great influence of the amount and the variance of information in our sources on the limitations of our final analysis.

4Principally, all this theoretical reflection about record linkage is useful either at the very beginning of a research project or during the presentation of the results. Usually it is only of minor relevance during the carrying out of the nominal record linkage. Nevertheless, we learned from this theoretical reasoning the prominent impact of the historians ingenuity, which has to be considered, therefore, when we put our research strategy into practice Especially, in deciding whether to use a computer for performing the vast amount of sorting procedures and of decision making more efficiently, we are confronted with the menace of loosing flexibility and sensitivity to the local context; in short: of loosing our own ingenuity during the decision making All projects dealing with numerous references to many individuals, however, have to use the support of computers as the only means to solve the task within the available time12. As the embarrassing effects of this support result mainly from the formal definition of decision making, there is no inevitable coherence between the use of computers and the shortcomings mentioned above. For it is finally our own duty to choose:

• either a very flexible software system that allows for a great variance due to the local context of the sources;
• or a software system, that makes all the sorting procedures in order to prepare the decision making, but keeps the decision itself back to the experts imagination

5Also from a more practical point of view we are confronted with the importance of flexibility. That is due to, above all, the peculiarity of the registration until the end of the 19th century. At the turn of the centuries, the police departments introduced a new and unambiguous way of referring to "ordinary persons": with finger-prints and anthropometrical statistics they started to use a biological identity of individuals for the first time. But until this time, the reference to a person was anything but unambiguous13. With considering only the name and the social identity the person in question was referred to by connecting him/her to the associated contexts of his/her social position, of the function held, of the place of origin resp. of residence or of his/her kinship relations. As there was no standardized, pro forma registration for listing the social identity at least until the end of the 18th century, the specific shape of the reference and thus the amount of identifying items was susceptible to great variance14. When we use this information for the purposes of nominal record linkage, we have to consider all clues to a persons identity which can be derived among other things from the context of the reference, too. It stands to reason that it depends largely on the point of view of the scribe and vanes to a great extent. In short: the histonan is forced to use non-standardized references for reconstructing those aspects of social life which are of main interest to prosopographical analysis. As there is usually no predefined strategy for an appropriate way of linkage – except the methodology of family reconstitution15 – we have to treat these non-standardized references in a non-standardized manner.

6The resulting demands on the flexibility of the software system range from the definition of proper data models to the adaption of record linkage strategies to specific needs At the very beginning of the project we do not really know all the information that will be of use during the identification process. As "... historical identification of an historical individual, is a matter of discrimination based upon a finite number of identifiers"16, it is recommended to consider as much of the clues to the identity of a person as possible. They can be either mentioned as attributes in the reference (i.e. the place of residence) or derived from the context of the source (i.e. the authenticity of a document) or from the context of the reference itself (i.e. the relative position within a group of witnesses). Considering all these clues implies to tailor both the data model and the record linkage strategy to our specific needs. This peculiar strategy has to consider not only the problem of the finite number of identifiers but also the ambiguity of the items which results from a lacking certainty of single attributes as well as from more or less severe spelling variances.

7Concerning the names as the most important identifying items the lacking certainty can be attributed to three main reasons17:

81. There are varying or alternate names, esp surnames and nicknames That is due to an alternating use of surnames and nicknames, of surnames and references to the place of residence and of surnames and references to the occupation. The whole concept of a "surname" being fairly modern, it is quite often all but certain, into which of these categories an identifying item belongs, as, e g., in:

9John Kogler / John Mayer, farmer at the Koglergut

10In this example the surname and the name of the holding are used

11alternately.

122. There are double names, esp. double first names:

13i.e. Anna / Anna-Maria / Anna, Magdalene

14People who are mentioned with double names can be referred to in another file with just one of them. A person who is called Anna-Maria in the first record can be called only Anna in another one. To make a successful individuation becomes quite difficult in this case, when no further information is available.

153. Above all, the occupation and the place of residence is susceptible to variations due to his/her position in the life-cycle:

16i.e. servant (until marriage) /farmer (until retirement)

17The variability of these attributes causes great difficulties in building up a prosopographical catalogue. The attempt to describe a person as comprehensively as possible requires obviously the reconstruction of the social career, too. Considering the variability of this information, that implies actually to match records, the items of which seem not to match on first look.

18On the one hand, it is impossible to present a definitive solution for overcoming these problems. Because the experts background knowledge, the careful consideration of all available clues together with some ingenuity has to be regarded as the final strategy of deciding also the more difficult individuation on a case by case basis. On the other hand, there are some ways to support this final decision-making both in employing a proper linkage strategy and in preparing the data for record linkage

191. We can record all double names and/or double first names as multiple entries to enable a comparison within all single entries that refer to a name or first name.

202. We can record all attributes which are available in addition to the first name also as multiple entries for being able to compare all single entries with each other in order to find out all potential agreements between given attributes, even if they are alternating.

213. When there is enough certainty about the very nature of single attributes we can split them up into different categories. But in such cases we should define an additional substitution list that forces the software to use i.e. the place of residence instead of a lacking surname during the comparison of two records.

22Independent of the way in which the categorization of the attributes will be organized finally, we have to decide first whether to record the attributes in a standardized manner or in the original spelling of the sources. Due to the very nature of registration the spelling of names and of other attributes varies obviously. As a standardization of spelling variations at the very beginning of a project causes a loss of information, it is recommended to use the original spelling. Following this path, however, results in a demand for normalizing the spelling variations subsequently. Handling the spelling variations is anything, but a secondary matter as the statement of E. A. Wrigley indicates: "Names were so frequently abbreviated, modified, changed or dropped, and their spelling when they were written out was so often wayward, that there is a prior problem of great complexity to be solved in standardizing names before the stage of linkage proper is reached"18. In the same way spelling variances are taken by M. Gutmann "as an important subject, because resolving spelling variation is one of the most complex tasks involved in historical record linkage"19. As there is no unique method to standardize names due to the specific requirements of the local and/or temporal conditions it is necessary to adapt existing strategies to specific needs. Concerning the software this claim refers again to the demand for a highly flexible system which allows for all these local and/or temporal variations.

23After this repeated reference to the demand for flexibility from a more practical point of view, I want to start the discussion of advantages and shortcomings of the software system kleio for the needs which were pointed out so far and which can be summed up as following:

1. The need for allowing for the experts ingenuity during the decision-making.
2. The need for allowing for the local context during the decision-making.
3. The possibility of handling non-standardized references in a non-standardized manner. That is, to be provided with great flexibility as to the data input and to the definition of the data model That is moreover, to be provided with enough flexibility to implement a specific linkage strategy.
4. It is necessary to dispose of highly flexible features for standardizing the spelling variations.

2. The software system kleio

24In fact, most of these demands are mentioned by Manfred Thaller as the characteristics of a genuine historical software system20. The treatment of non-standardized information, the preparation of statistical analysis and the support of record linkage approaches are all at the core of History and Computing. Thus, kleio as a software system that is dedicated to historians, seems to meet almost all of these requirements. That can be seen already with regard to the flexibility of data input and of data models, kleio administers all the data as collections of pieces of text, without any assumptions about its meaning. All those definitions that give the information their actual meaning are open to the users modifications. As they are administered in tables which are completely independent from the data as such, they can be adapted easily to one’s improving knowledge about the social reality in question.

25The data input is organized in variables. However, these vanables do not really represent the classical concept of variables, which implies that "every vanable has a value". They have to be seen rather as "structures of interrelated items of information"21, in short: as multiple entries that can be assigned to more than one variable at the same time, when its meaning is not unambiguous Besides, we can set up a substitution list, that uses the value of a second variable when there is no entry made to the first vanable; i e. it would use the place of origin as a substitute for a lacking surname. Principally, all these vanables belong to larger entities that are either found in the sources or conceived within our analytical concepts. Together with relational and hierarchical relationships they describe the basic unit of the database, i.e. one family, one biography, etc.22.

26It is not enough, to relate the record data to variables. For defining their meaning, we need a further tool, the data types. In kleio we dispose of several data types for an appropriate handling of information: numerical, textual, temporal and graphical information are supported as well as spatial information which relates an object to its position within a coordinate grid system. In principle, these components of variables, entities, relationships and data types allow the proper treatment of all available sources.

27The flexibility of the input system is based on the idea of supplying the historian with a framework consisting of several elements which can be assembled in nearly any order for meeting the specific needs. The same idea can be found with the record linkage features, too. They consist of the following elements:

1. the definition of a pocket variable, which organizes the comparison between two records efficiently;
2. the definition of the rules which are used during the search for matching entries;
3. the definition of the information that illustrates the quality of the proposal on a split screen (available from autumn 1992) for supporting the historians decision making;
4. the definition of how to merge two matching records

28The definition of rules that manifest the prerequisites for matching records and the final decision making have to be considered as the main steps during the record linkage. Therefore, I want to concentrate on these aspects during the final part of my paper The definition of rules for ascertaining a match between two records consists at least of two parts:

1. We have to specify the identifying items that should be taken into account; i.e. first name, surname
2. We have to specify all "excuse procedures" (Winchester) that are necessary to overcome the "fuzziness" (Thaller) of our data, i.e the standardization of spelling variations. For normalizing the spelling of character strings there are at least two methods available:
   
   1. For names which are built strictly according to philological rules, complete dictionaries of all possible variations of the segments of names can be given as well as rules for their combination into valid "normal names". A generalized re-implementation of the lemmatization system for Germanic names from the early medieval penod, which has been developed by Geuenich et al., has been completed.
      
   2. Making a lemmatization allows for the best results during the search for identities as it assigns an unambiguous lemma to each spelling variation. But it is also the most time-consuming way to overcoming this problem. Therefore, other options for carrying out the standardization are available, too. They permit the definition of rules for reducing the spelling of a name to the skeleton of significant characters. One of the best known code systems to bring about the skeleton is named Soundex23 kleio provides us with the basic rules of this and two further code systems as a framework that is open to our own definition24. So, there is no predefined standardization of spelling variations. On the one hand, that allows for a great flexibility and for the adaption of the codes to specific conditions. On the other hand, it requires a great amount of work to be done by ourselves in order to prepare the record linkage in a proper way. For all this inevitable work, it is the only way to make a general framework available that is not restricted to an application within a limited area and/or time-span. Neither the lemmatized string nor the skeleton replaces the original string in the database. They are rather administrated in tables which are completely independent from the data as such. Therefore, the standardization rules can be changed easily when they turn out to be wrong and they can be exchanged easily when they turn out to be right. This can provide the scientific community with a whole set of rules, that are nearly built in standardization.

29After having specified all identifying items together with the excuse procedures the search for matching records is carried out. As a result, kleio presents his proposals for matches (from autumn 1992) with the help of a split screen We will be able to inspect the potential matches and to make the decisions interactively. With the standard display commands of the retrieval system we can select any combination of information that is available in the database to illustrate the quality of the proposal. Due to the great flexibility of kleio, it is possible to use really all accessible clues to support the final decision-making. They can be taken from the "variables" belonging to the entity which is compared actually as well as from the context of the entities, i.e. from their proximity to other entities. Concerning the witness lists of medieval charters and the libri memoriales of the Middle Ages this proximity has to be seen as the only available "context", as the only indicator for identifying the person. In the words of Dieter Geuemch: "Many persons can only be identified, because they occur in several, more or less identical contexts. Behind this is the well-known fact, that persons of the middle ages are represented in the written sources almost exclusively within their social surroundings"25.

30In short: kleio supplies the historians ingenuity with all available clues and restricts the application of this ingenuity to promising matches The required flexibility, however, is bought at the cost of lacking predefined strategies and thus it is always our own job, to adopt the available framework to our needs.

31Desisting from this potential shortcomings there is one further advantage of kleio’s flexibility. As kleio does not really merge the files but rather sets up pointers between them, there is always the whole amount of information available during the final analysis of the prosopographical catalogue. Both during the data retrieval and the preparation of a statistical analysis, all those contexts that were used as clues to the social identity of the person in question are available to the further analysis where the social identity is used again to improve our knowledge about new social worlds