A proposal of an indicator to evaluate research activities based on Scimago Institutions Ranking (SIR) data: an application to Italian High Education Institutions

In this paper we propose a synthetic indicator for evaluating Italian universities’ scientific research using Scimago Institutions Ranking (SIR) 2014 data. It is an annual report published by Scimago Lab., a Spanish company which produces some bibliometric indicators using Scopus databank, an Elsevier product, for several kinds of research Institutions. Differently from other world university research ranking, such as University Ranking by Academic Performance (URAP) or Performance Ranking of Scientific Papers for World Universities (NTUR), SIR doesn’t supply a league table of Institutions, preferring not to carry on an indicators aggregation to obtain a global ranking. Starting from some indicators computed by SIR, after having analysed the distribution shape, fitted parameters of some statistical models, calculated an appropriate standardization, we aggregate the indicators to get a Synthetic Indicator (from now, SI) of research evaluation. The obtained synthetic indicator has been used to rank Italian Higher Education Institutions (from now, HEIs). This ranking has been compared first with the ranking of the National Agency for the Evaluation of Universities and Research Institutes (ANVUR), based on the Evaluation of Research Quality (VRQ 2004-2010) results, and then with the ranking based on the assignments of the competitive allocation model (research share of FFO) yearly attributed to the Italian HEIs by the Ministry of University and Research (MIUR). The results of the analysis show a moderate positive correlation between SI and the VQR 2004-2010 indicators, the standardized mark (r=0,543) and FFO per capita (r=0,487). The original contributions of the paper are i) the creation of a Synthetic Indicator, with a Gaussian distribution, summarizing the SIR variables; ii) the highlighting of a convergence between ANVUR evaluation, based on peer to peer and bibliometric analysis but using only few publications, and the analysis proposed in this paper, which uses bibliometric data from Scopus, but related to all the publications in the same period.