Précédent Suivant

An Expertise Recommender System based on Data from an Institutional Repository (DiVA)

p. 135-149

Résumé

Finding experts in academics is an important practical problem, e.g., recruiting reviewers for reviewing conference, journal or project submissions, partner matching for research proposals, finding relevant MSc or PhD supervisors, etc. In this work, we discuss an expertise recommender system that is built on data extracted from the Blekinge Institute of Technology (BTH) instance of the institutional repository system DiVA. The developed prototype system is evaluated and validated on information extracted from the BTH DiVA installation, concerning thesis supervision of researchers affiliated with BTH. The extracted DiVA classification terms are used to build an ontology that conceptualizes the thesis domain supported by the university. The supervisor profiles of the tutors affiliated with the BTH are constructed based on the extracted DiVA data. These profiles can further be used to identify and recommend relevant subject thesis supervisors.

Entrées d’index

Keywords : data mining, DiVA, expertise retrieval, knowledge management, natural language processing

Remerciements

This work is part of the research project "Scalable resource-efficient systems for big data analytics" funded by the Knowledge Foundation (grant: 20140032) in Sweden.


Texte intégral

Introduction

1Finding experts in academics is an important practical problem, e.g., recruiting reviewers for reviewing conference, journal or project submissions, partner matching for research proposals, finding relevant MSc or PhD supervisors, etc. In this work, we discuss an expertise recommender system that is built on data extracted from the Blekinge Institute of Technology (BTH) instance of the institutional repository system DiVA (Digital Scientific Archive). DiVA is a publication and archiving platform for research publications and student essays used by 46 publicly funded universities and authorities in Sweden and the rest of the Nordic countries.1 The DiVA classification system is based on the Swedish Higher Education Authority (UKÄ) and the Statistic Sweden’s (SCB) three levels classification system. Using the classification terms associated with student M. Sc. and B. Sc. theses published in the DiVA platform, we have developed a prototype system which can be used to identify and recommend subject thesis supervisors in academy.

Related Work

2The discovery of expertise is crucial in supporting a number of tasks. For example, finding an appropriate expert when one needs guidance on a subject matter, or needs to fill a vacancy based on relevant expertise, or needs to find research collaborators working in similar areas, etc. The methods used for developing systems that facilitate mining of expertise can be classified into two main categories: methods based on mining unstructured information and methods based on social networking sites.

3Unstructured information includes emails, Web pages, wiki, reports, etc. Text mining tools are used to index technical terms from unstructured documents, which can be queried to identify subject experts. Some of the important tools using this information include email expertise extraction (e3) system (Krulwich and Burkey), ContactFinder (Campbell et al.), MIT Expert Finder (Vivacqua), etc. A major limitation of these methods is the authentication of information.

4Today there are many social networking sites, some specifically for the scientific community, such as Research Gate and Nature Network. Experts have to feed in information about their subject expertise, domains, publications, credentials, etc. A major limitation of these methods is in the adding and updating of information.

5A common drawback to both approaches is that they are based on non-peer-reviewed information provided by the user, i.e., they can be biased.

6In the recent years research on identifying experts from online data sources, such as the DBLP library, Microsoft Academic Search, Google Scholar Citation, LinkedIn, PubMed, etc., has been gradually gaining interest (Abramowicz et al.; Balog and de Rijke; Bozzon et al.; Boeva et al., “Semantic-Aware”; Boeva et al., “Identifying a Group”; Hristoskova et al.; Jung et al.; Singh et al.; Stankovic et al.; Tsiporkova and Tourwé; Zhang et al.). An example is a Web-based biomedical expert finding system, proposed by Singh et al., which can be applied to identify subject experts and subjects associated with an expert. The system builds and maintains a big repository of biomedical experts by extracting the information about experts’ peer-reviewed articles that are published and indexed in PubMed. Two other interesting approaches using semantic matching and developing trust algorithms are those of Charlin and Zemel, as well as Osman et al. Charlin and Zemel have developed an automated reviewer assignment system that can be used to find suitable reviewers for conference papers. The assessment of reviewers’ expertise is based on their previously published papers and self-assessed expertise about the submissions. Osman et al. propose a trust model that calculates the expectation about an agent’s future performance in a given context by assessing both the agent’s willingness and capability through the semantic comparison of the current context in question with the agent’s performance in past similar experiences. А comprehensive survey on the state-of-the-art methods in expert finding and summary of these methods into different categories based on their underlying algorithms and models is presented by Lin et al.

7Institutional repositories have been around for at least 20 years, but it is only since recently we find some discussions on how to create new services using text analysis or data mining in institutional repositories or library catalogues (Kerdprasop et al.; Lawrence; Okamoto; Tonon and Fusco) and even more rare are any evidence on actual resources based on these ideas.

8The use of university repositories is not so common, in spite of their wealth of bibliographic metadata of both local scientific records and student theses records. There are papers describing mining the contents of scientific papers for more effective information discovery and selection (Saggion and Ronzano; Tonon and Fusco; Okamoto). Other areas that are reported in the literature is personalized services for library user using data mining technology or book recommendation systems for library user based on user profiling (Ding; Jomsri; Cui et al.). These are the main uses of data mining in university institutional repositories that we can find being reported in the literature so far. One of the reasons for not exploiting institutional repositories on a greater scale could well be that they are often created, administered and run by librarians who generally are not research oriented and especially not towards computer science. Another reason might be that the data in the repositories often is entered by researchers and students themselves and therefore, in many cases, are not quality controlled. An elaborated discussion of this issue can be found in the section “Data”.

Identifying Experts from Institutional Data Sources

9Many scientists who work on the expertise retrieval problem distinguish two information retrieval tasks: expert finding and expert profiling, where expert finding is the task of finding experts given a topic describing the required expertise (Craswell et al.), and expert profiling is the task of returning a list of topics that a person is knowledgeable about (Balog et al.). In the considered context we need to deal with both expertise retrieval tasks.

Profiling of Expertise

10An expert profile may be quite complex and can, for example, be associated with information that includes: email address, affiliation, a list of publications, co-authors, but it may also include or be associated with: educational and (or) employment history. This information can be separated into two parts: expert’s personal data and information that describes the expert area of competence. Personal data is used to resolve the problem with ambiguity. This problem refers to the fact that multiple profiles may represent one and the same person and therefore must be merged into a single generalized expert profile. Namely, the process of merging expert profiles is driven by the calculation of the similarity scores between different entities composing the profile, e.g., expert name, affiliation, email address (Boeva et al., “Measuring Expertise”).

11The data needed for constructing the expert profiles could be extracted from various Web sources, e.g., LinkedIn, the DBLP library, Microsoft Academic Search, Google Scholar Citation, PubMed. In our work, we have used data extracted from the DiVA repository to construct the tutor profiles and ontology model. Initially, the extracted data was divided into personal data and a list of subject terms. Personal data has entities, e.g., first and last name of each thesis supervisor, affiliation containing information about university, faculty and department, and also the academic role of the supervisor. Each BSc or MSc thesis presented in DiVA is associated with a list of keywords that are selected and entered by the author in order to describe their thesis subject. We have used these keywords and the built ontology to describe the supervisors’ area of competence. The built ontology is a conceptual model of the domain of interest and it is used to attain accurate and topic relevant expert profiles. When a conceptual model is missing then, e.g., the Stanford part-of-speech tagger (Toutanova and Manning) can be used to annotate the different words in the text collected for each expert (supervisor) with their specific part of speech. The tagger also defines whether a noun is a plural, whether a verb is conjugated. The annotated text can further be reduced to a set of keywords (tags) by removing all the words tagged as articles, prepositions, verbs, and adverbs.

12In view of the above, an expert profile is defined as a list of keywords (domain-specific topics), extracted from the available information, describing their expert area. The problem with ambiguity has been resolved using the extracted personal data. The expert profiles with the same names are candidates to be merged into a single expert profile. For instance, if the supervisors’ names and affiliations are identical then their expert profiles are merged into a single one. Most supervisors who are affiliated with BTH have not entered their email addresses in the DiVA repository. Therefore, we have not used the email address as an entity to resolve the problem with ambiguity.

Expertise Similarity

13An important issue in an expertise retrieval context is to establish a way to quantify how well the area of expertise of an individual expert conforms to a certain subject or to estimate the expertise similarity between two experts. The calculation of expertise similarity is a complicated task, since the supervisor expertise profiles usually consist of domain-specific keywords that describe their area of competence without any information for the best correspondence between the different keywords of two compared profiles. One possibility to measure the expertise similarity between two expert profiles (or between a supervisor profile and a subject profile) is by taking into account the semantic similarities between any pair of keywords that are contained in the two profiles. Several semantic distance and similarity algorithms can be applied within structured terminological resources (Leacock and Chodorow; Rada et al.; Wu and Palmer). In this case, the algorithms count the number of edges (links) between the two keywords (classification terms) in order to compute the relatedness of these terms.

14In the current study, we use a modified Wu and Palmer measure, proposed in Manjula Shenoy et al., to calculate similarity between two ontological elements. In the considered context all expert profiles are described by a list of domain specific terms, where they are an expert. Assume that each expert profile i is described by a list of pi keywords. In our experiments, we use the modified Wu and Palmer measure to calculate the semantic similarity between two keywords. Manjula Shenoy et al. propose an improved version of the standard Wu and Palmer measure. The modified Wu and Palmer semantic similarity measure s is defined as follows:

Image

where L is the shortest distance between two given concepts, D is the depth of the ontology, N is the distance from the least common ancestor to the root, N1 and N2 are respectively, the distances from the two considered concepts to the root, and λ is a coefficient that is 0 when the concepts are in the same hierarchy and 1 when they are neighborhood concepts. Namely, the authors consider how far the two considered concepts are semantically and where they are located in the ontology. The modified Wu and Palmer measure (noted by s herein), finds the shortest path between two concepts in the ontology tree and the depth of the whole ontology. Then the expertise similarity Sij between two expert profiles i and j (i j), can be defined by the arithmetic mean of semantic similarities between the corresponding keywords, i.e.

Image

Where s(kil, kjm) is the semantic similarity between keywords kil and kjm calculated accordingly to the definition given above (Manjula Shenoy et al.).

Expert Finding

15The experts finding task can be viewed as a list completion task, i.e. the user is supposed to provide a small number of example experts who have been used to work on similar problems in the past, and the system has to return experts with similar areas of competence. In our context, the user can be a university student, a director of a study program or other administrative staff and the returned experts are recommended subject thesis supervisors.

16Another possibility is to present the domain of interest by several preliminary specified subject categories and then the available experts can be grouped with respect to these categories into a number of disjoint expert areas (clusters) by using some clustering algorithm, e.g. ones developed in “Semantic-Aware” and “Identifying a Group” (Boeva et al.). In the considered context each cluster of experts can itself be thought as the expertise area of any expert (supervisor) assigned to the cluster. In this case, in order to select the right individuals (thesis supervisors) for а specified task the user may restrict their considerations only to those experts who are within the cluster that is identical with (or at least most similar to) the task’s subject. The specified subject and the expert area can themselves be described by lists of keywords, i.e. they can be compared by way of similarity measurement.

Description of the Prototype System and Results

17The developed prototype system has been evaluated and validated on information extracted from the BTH DiVA installation, concerning thesis supervision of researchers affiliated with BTH. The extracted DiVA classification terms are used to build an ontology that conceptualizes the thesis domain supported by the university. The supervisor profiles of the tutors affiliated with the BTH are constructed based on the extracted DiVA data. Each supervisor profile is defined by a list of keywords (classification terms) used in the DiVA theses that have been supervised by the researcher in question to describe their area of expertise.

Data

18The data set consists of 2,216 records of student theses published between 2010 and 2017. One issue with data entered during a long interval like this is that the quality of the records varies. In the case of our sample data this certainly is true. In 2015 the BTH repository joined the national Swedish consortia DiVA-system. At around the same time the library started supplying records to a national portal “SwePuB” intended to become a national tool and a resource for the collection of publication data for bibliometric analyzes and data processing (SwePub). The DiVA migration and having SwePub harvesting our records meant that the library needed to focus much more attention on the quality of the records. The amount of metadata attached to every record increased.

19Records entered by students or staff themselves usually have quality issues. In our case we have quality issues with records entered before the second half of 2015. Beginning in the fall of 2015 records are always quality checked by staff from the library or the university departments.

20Another data issue is knowing what expert in the used data set still is affiliated with BTH. In a data set covering eight years many supervisors have come and gone. In our case we needed a big data set to create and validate a prototype system so the issues discussed above are not really a problem. But if you want to put an Expertise Recommender System into everyday use issues like these must be paid close attention to.

Ontology Description

21We have defined our ontology as a formalization of concepts and relations between them. The domain concepts are described by classes and the relations between them is defined as “isSubclassOf”. Main focus in our ontology are classes and their sub-classes. The model in this study is created using National Subject Categories in DiVA (Linköping University Library) that is standard for the Swedish listing of research subjects 2011 and classification terms from extracted data. National Subject Categories in DiVA are separated into three levels. Namely, the main categories (the highest level) are Agricultural Sciences, Engineering and Technology, Humanities, Medical and Health Sciences, Natural Sciences, and Social Science. These main categories are located at the first level of the ontology. The others two levels are described by the sub-subject areas that belong to each of the main categories. The second level has 26 subject categories and the third level consists of 257 categories, respectively. From the extracted data we have taken the subject terms (specific domain keywords) and added them to the structure of ontology model as classes. In that way, the created ontology tree has a four level depth.

Metrics

22Silhouette Index (Rousseeuw) is a cluster validity index that is used to judge the quality of any clustering solution C = {C1, C2, …, Ck}. Suppose that the considered data set contains the attribute vectors of m objects. Then the Silhouette Index is defined as

Image

where ai represents the average distance of object i to the other objects of the cluster to which the object is assigned, and bi represents the minimum of the average distances of object i to object of the other clusters. The values of Silhouette Index vary from -1 to 1.

Results

23As it was mentioned above, the test data is downloaded from BTH DiVA installation, concerning thesis supervision of researchers affiliated with BTH. The extracted data has been used to construct the supervisor profiles and ontology model. Each supervisor profile is defined by a list of keywords (domain-specific terms). After resolving the problem with ambiguity, the set of supervisors has been reduced to 375 expert profiles.

24We have studied two different experiment scenarios. In the first scenario, the user is supposed to provide an example supervisor and the developed recommender system will return a list of experts (supervisors) with close (similar) expertise. In this scenario, the system initially calculates the expertise similarity scores between the given supervisor and all other supervisors presented in the system by using the expertise similarity formula described in section “Expertise Similarity”. Then the system returns a ranked list of similar experts who are recommended subject thesis supervisors. For instance, we have tested the system by using an example supervisor whose expertise can be described by the following subject terms: database, performance, usability, web server, cloud computing and Amazon web services. The system has returned a list of the supervisors ranked with respect to the similarity of their expert profiles to the profile of the example supervisor. In table 1, we list the ten top ranked supervisors who have been recommended by the system. The calculated expertise similarity scores between the example supervisor and the all other supervisors presented in the system are in the range between 0.046 and 0.77. The expertise of each top recommended supervisor is described by the keywords found in their expert profile (the second column in table 1). As one can notice the all top ranked supervisors have expertise that is overlapped to different extent with the competence of the given example supervisor. For example, the expertise of supervisor 1 covers usability subject from the example expert profile while supervisor 8 has competence overlap in two subjects: database and performance.

Table 1.

ExpertKeywordsExpertise similarity score
1user experience, usability0.770
2privacy, security, cloud computing0.763
3cloud computing, security metrics, security threats, security measurement frameworks0.760
4procedural city generation, perlin noise, performance, game content0.760
5machine learning, parallel computing, multiprocessor, performance0.760
6mobile, power, consumption, android, native, web, enterprise service bus, performance, framework0.754
7mongodb, couchdb, python, pymongo, couchdb-python, nosql, document database, json, dbms, database0.754
8compression, sms, arithmetic, lambda, huffman, lzw, lz77, lz78, fristående kurs, voltdb, mysql, databases, main-memory database, primary memory database, performance0.751
9non-functional search-based software testing, non-functional system properties, search-based software testing, meta-heuristic techniques, performance testing, load testing, load patterns0.75
10digital multimedia broadcasting, mpeg-2 standard, mpeg-4 standard, video transport stream0.75

25In the second scenario, the set of supervisors is clustered into groups of experts with similar expertise by applying k-means partitioning algorithm. The optimal number of clusters is determined by clustering the set of supervisors applying k-means for different k and evaluating the obtained solutions by the Silhouette Index (SI). We have applied k-means for k between 2 and 257 (the maximum domain-specific classes in ontology). The selected optimal value for k is 16 and the corresponding average SI score given on the clustering solution generated for k = 16 is 0.38. In this scenario, in order to select the right individuals for а specified thesis subject the user may restrict their considerations only to those supervisors who are within the cluster that is identical with (or at least most similar to) the given subject. The experts in the selected cluster can be ranked with respect to the similarity of their expert profiles to the specified subject. For example, if we need a supervisor with expertise in database and parallel computing we will use cluster 0 (see table 2), in which all experts have competence either in one or in both fields.

Table 2.

ExpertsClusterDescription of clusters
3, 4, 7, 80usability; tessellation; android; security threats; main-memory database; database; distributed databases; parallel computing; security; data mining, etc.
2, 5, 6, 103usability; data mining; performance monitoring; systematic review; video streaming; parallel computing; mpeg-2 standard; mpeg-2 standard; nosql database; machine learning; cloud computing, etc.
1, 915usability; quality of experience; urban design; systematic literature review, etc.

26We have checked how the 10 top ranked supervisors returned by the system in the first experiment scenario has been distributed between the generated clusters. The obtained grouping is presented in table 2. As we can see the supervisors who have been grouped together have a similar area of competence. For instance, supervisors 3, 4, 7 and 8 are distributed in one and the same cluster (cluster 0). The main expertise of supervisors 3 and 4 is in cloud computing while supervisors 7 and 8 both have competence in database. The supervisors 2, 5, 6 and 10 are grouped in the cluster 3. For example, the supervisors 5 and 6 have expertise in performance and the others are in cloud computing and video streaming. However, supervisors 1 and 9 are in a different cluster (cluster 15 in table 2), because as we can see in table 1 they have very unique expertise in comparison with the others.

27It is interesting to notice that the generated clustering solution partitions the supervisors into 16 disjoint clusters (subject categories), i.e. each supervisor belongs to only one cluster. However, as it can be seen in table 2 there are subject terms that participate in the description of more than one subject category, e.g., parallel computing is used for clusters 0 and 3 (the third column in table 2), i.e. the obtained subject categories can have an overlap on some topics.

Conclusion and Future Work

28The main contributions of this work are:

  • a prototype system for ranking potential thesis supervisor candidates based on their previous supervision experience and expertise retrieval approaches;
  • an evaluation of the prototype with real-world data extracted from the BTH instance of the institutional repository system DiVA.

The proposed system is based on online information, can be updated regularly, and uses standardized subject vocabulary, i.e., DiVA classification terms associated with each thesis. It can be implemented by any university that uses an institutional repository with a controlled vocabulary including keyword fields in the records, i.e., it is general.

29For future work, we aim to pursue further evaluation and validation of the developed expert recommended system on bigger datasets and also on data extracted from other DiVA university repositories. Our future plans also involve using additional information about the supervisors’ expertise, e.g., extracting information about the supervisors’ research publications, in order to build richer expert profiles and ontology model.

Bibliographie

Des DOI sont automatiquement ajoutés aux références bibliographiques par Bilbo, l’outil d’annotation bibliographique d’OpenEdition. Ces références bibliographiques peuvent être téléchargées dans les formats APA, Chicago et MLA.

Abramowicz, Witold, Elżbieta Bukowska, Jakub Dzikowski, Agata Filipowska and Monika Kaczmarek. “Semantically Enabled Experts Finding System – Ontologies, Reasoning Approach and Web Interface Design.” Proceedings II of the 15th East-European Conference on Advances in Databases and Information Systems, 2011, pp. 157-166.

Balog, Krisztian, Toine Bogers, Leif Azzopardi, Maarten de Rijke and Antal van den Bosch. “Broad Expertise Retrieval in Sparse Data Environments.” Proceedings of 30th Annual International ACM SIGIR Conference on R&D in Information Retrieval, ACM Press, 2007.

Balog, Krisztian and Maarten de Rijke. “Finding Similar Experts.” Proceedings of 30th Annual International ACM SIGIR Conference on R&D in Information Retrieval, ACM Press, 2007.

10.1145/1277741 :

Boeva, Veselka, Maria Krusheva and Elena Tsiporkova. “Measuring Expertise Similarity in Expert Networks.” Proceedings of 6th IEEE International Conference on Intelligent Systems, IEEE Sofia Bulgaria, 2012, pp. 53-57.

10.1109/IS.2012.6335190 :

Boeva, Veselka, Liliana Boneva and Elena Tsiporkova. “Semantic-Aware Expert Partitioning.” Artificial Intelligence: Methodology, Systems, and Applications, Springer, 2014, pp. 13-24.

10.1007/978-3-319-10554-3 :

Boeva, Veselka, Milena Angelova, Liliana Boneva and Elena Tsiporkova. “Identifying a Group of Subject Experts Using Formal Concept Analysis.” Proceedings of 8th IEEE International Conference on Intelligent Systems, IEEE Sofia Bulgaria, 2016, pp. 464-469.

10.1109/IS.2016.7737462 :

Bozzon, Alessandro, Marco Brambilla, Stefano Ceri, Matteo Silvestri and Giuliano Vesci. “Choosing the Right Crowd: Expert Finding in Social Networks.” Proceedings of EDBT/ICDT’13, ACM, 2013, pp. 637-648.

10.1145/2452376 :

Campbell, Christopher S., Paul P. Maglio, Alex Cozzi and Byron Dom. “Expertise identification using email communications.” Proceedings of the 12th International Conference on Information and Knowledge Management, ACM, 2003, pp. 528-531.

Charlin, Laurent and Richard S. Zemel. “The Toronto Paper Matching System: An Automated Paper-Reviewer Assignment System.” Proceedings of the 30th International Conference on Machine Learning, Atlanta, JMLR, 2013.

Craswell, Nick, Arjen P. de Vries and Ian Soboroff. “Overview of the TREC-2005 Enterprise Track.” Proceedings of 14th Text REtrieval Conference, 2005.

Cui, Lin, Hong Li, Caiying Wang and Baosheng Yan. “Personalized Book Recommendation Based on Ontology and Collaborative Filtering Algorithm.” The Open Cybernetics & Systemics Journal, vol. 8, 2014, pp. 632-637.

10.2174/1874110X01408010632 :

Ding, E. “Research on Personalized Service of Library in Colleges and Universities Based on Data Mining Technology. Agro Food Industry Hi-Tech, vol. 28, no. 3, 2017, pp. 789-793.

Hristoskova, Anna, Elena Tsiprokova, et al. “A Graph-based Disambiguation Approach for Construction of an Expert Repository from Public Online Sources.” Proceedings of 5th IEEE International Conference on Agents and Artificial Intelligence, 2013.

Jomsri, Pijitra. “Book Recommendation System for Digital Library Based on User Profiles by Using Association Rule.” In Proceedings of INTECH 2014, 2014, pp. 130-134.

10.1109/INTECH.2014.6927766 :

Jung, Hanmin, Mikyoung Lee, In-Su Kang, Seung-Woo Lee and Won-Kyung Sung. “Finding Topic-Centric Identified Experts based on Full Text Analysis.” Proceedings of FEWS’07, 2007, pp. 56-63.

Kerdprasop, Nittaya, Ekkachai Naenudorn et al. “Knowledge Mining in Higher Education.” International Journal of Mathematical Models and Methods in Applied Sciences, vol. 6, no. 7, 2012, pp. 861-872.

Krulwich, Bruce and Chad Burkey. “The ContactFinder Agent: Answering Bulletin Board Questions with Referrals.” Proceedings of 13th National Conference on AI, 1996.

Lawrence, David. “A Challenge to Research Libraries – Don´t Just Display Your Data, Use It!” Blog post, 2016. http://librisbloggen.kb.se/wp-content/uploads/2016/10/Text-analysis-and-other-uses-of-SwePub-data_20161017.pdf

Leacock, Claudia and Martin Chodorow. “Combining Local Context and WordNet Similarity for Word Sense Identification.” WordNet: An Electronic Lexical Database, vol. 49, no. 2, 1998, pp. 265-283.

Lin, Shuyi, Wenxing Hong, Dingding Wang and Tao Li. “A Survey on Expert Finding Techniques.” Journal of Intelligent Information Systems, vol. 49, no. 2, 2017, pp. 255-275.

Linköping University Library. “National Subject Categories in Diva.” 2011, https://www.ep.liu.se/authorinf/pdf/national_subject_categories.pdf

Manjula Shenoy, Karthish Shet and Dinesh U. Acharya. “A New Similarity Measure for Taxonomy Based on Edge Counting.” International Journal of Web & Semantic Technology, vol. 3, no. 4, 2012, pp. 23-30.

Okamoto, Kazushi. “Text Analysis of Academic Papers Archived in Institutional Repositories.” In Proceedings of ICIS 2016, 2016.

10.1109/ICIS.2016.7550876 :

Osman, Nardine, Carles Sierra, Fiona Mcneill, Juan Pane and John Debenham. “Trust and Matching Algorithms for Selecting Suitable Agents.” ACM Transactions on Intelligent Systems and Technology, Special Section on Intelligent Mobile Knowledge Discovery and Management Systems and Special Issue on Social Web Mining, vol. 5, no. 1, 2013.

10.1145/2542182.2542198 :

Rada, Roy, Hafedh Mili, Ellen Bicknell and Maria Blettner. “Development and Application of a Metric on Semantic Nets.” IEEE Transactions on Systems, Man, and Cybernetics, vol. 19, no. 1, 1989, pp. 17-30.

Rousseeuw, Peter J. “Silhouettes: A Graphical Aid to the Interpretation and Validation of Cluster Analysis.” Journal of Computational and Applied Mathematics, vol. 20, 1987, pp. 53-65.

10.1016/0377-0427(87)90125-7 :

Saggion, Horacio and Francesco Ronzano. “Scholarly Data Mining: Making Sense of Scientific Literature.” Proceedings of the ACM/IEEE Joint Conference on Digital Libraries, 2017.

10.1109/JCDL.2017.7991622 :

Singh, Harpreet, Reema Singh, Arjun Malhotra and Manjit Kaur. “Developing a Biomedical Expert Finding System Using Medical Subject Headings.” Healthcare Informatics Research, vol. 19, no. 4, 2013, pp. 243-249.

10.4258/hir.2013.19.4.243 :

Stankovic Milan, Jelena Jovanovic and Philippe Laublet. “Linked Data Metrics for Flexible Expert Search on the Open Web.” Proceedings of ESWC 2011, Springer, 2011, pp. 108-123.

10.1007/978-3-642-21034-1 :

SwePub för analys och bibliometri. 2018, http://bibliometri.swepub.kb.se/bibliometri

Toutanova, Kristina and Christopher D. Manning. “Enriching the Knowledge Sources Used in a Maximum Entropy Part-of-Speech Tagger.” Proceeding of the Joint SIGDAT Conference on Empirical Methods in NLP and Very Large Corpora, Association for Computational Linguistics, 2000, pp. 63-70.

Tonon, Leticia and Elvis Fusco. “Data Mining as a Tool for Information Retrieval in Digital Institutional Repositories.” Proceedings of CSSS 2014, Bangkok, 2014, pp. 180-183.

Tsiporkova, Elena and Tom Tourwé. “Tool Support for Technology Scouting Using Online Sources.” Proceedings of ER Workshops 2011, Springer, 2011, pp. 371-376.

10.1007/978-3-642-24574-9 :

Vivacqua, Adriana S. “Agents for Expertise Location.” Proceedings of AAAI Spring Symposium Workshop on Intelligent Agents in Cyberspace, Palo Alto, CA, 1999, pp. 9-13.

Wu, Zhibiao and Martha Palmer. “Verb Semantics and Lexical Selection.” Proceedings of 32nd Annual Meeting on Associations for Computational Linguistics, 1994, pp. 133-138.

Zhang, Jing, Jie Tang and Juanzi Li. “Expert Finding in a Social Network.” Proceedings of DASFAA 2007, Springer, 2007, pp. 1066-1069.

10.1007/978-3-540-71703-4 :

Notes de bas de page

Précédent Suivant

Le texte seul est utilisable sous licence Creative Commons - Attribution - Pas d'Utilisation Commerciale - Pas de Modification 4.0 International - CC BY-NC-ND 4.0. Les autres éléments (illustrations, fichiers annexes importés) sont « Tous droits réservés », sauf mention contraire.