Economics & Sociology

ISSN: 2071-789X eISSN: 2306-3459 DOI: 10.14254/2071-789X
Index PUBMS: f5512f57-a601-11e7-8f0e-080027f4daa0
Article information
Title: Measuring and ranking R&D performance at the country level
Issue: Vol. 12, No 1, 2019
Published date: 03-2019 (print) / 03-2019 (online)
Journal: Economics & Sociology
ISSN: 2071-789X, eISSN: 2306-3459
Authors: Marianela Carrillo
Universidad de La Laguna
Keywords: D performance, Data Envelopment Analysis, Cross-efficiency, Ranking, OECD countries
DOI: 10.14254/2071-789X.2019/12-1/5
Index PUBMS: dfb6c9a6-5d47-11e9-8b68-fa163e6feac6
Language: English
Pages: 100-114 (15)
JEL classification: C44, H50, O30, O57

In the context of generally growing interest in R&D (Research and Development) efficiency, the main objective of this work is evaluation and ranking of the R&D performance on a set of selected countries with the highest worldwide level of engagement in R&D activities. To that aim, R&D efficiency of the sample countries is assessed with Data Envelopment Analysis, then the overall performance score is obtained with the cross-efficiency method and the considered countries are listed in the order according to their R&D performance. The findings of this study point at Switzerland, the United Kingdom and the Netherlands as the three leading countries as far as R&D performance is concerned, while the countries that make important investment efforts in terms of their GDP, such as Japan or Israel, do not seem to obtain the desired results and need to implement targeted policy actions to encourage R&D outputs.


1. Abramo,G., Cicero, T., & D’Angelo, C:A. (2011). A field-standardized application of DEA to national-scale research assessment of universities. Journal of Informetrics, 5(4), 618-628. doi: 10.1016/j.joi.2011.06.001

2. Adler, N., Friedman, L., & Sinuany-Stern, Z. (2002). Review of ranking methods in the data envelopment analysis context. European Journal of Operational Research, 140(2), 249-265. doi: 10.1016/S0377-2217(02)00068-1

3. Akcali, B.Y., & Sismanoglu, E. (2015). Innovation and the effect of Research and Development (R&D) expenditure in growth in some developing and developed countries. Procedia – Social and Behavioral Sciences, 195(3), 768-775. doi: 10.1016/j.sbspro.2015.06.

4. Aristovnik, A. (2012). The relative efficiency of education and R&D expenditures in the new EU member states. Journal of Business Economics and Management, 13(5), 832-848. doi:10.3846/16111699.2011.620167

5. Banker, R. D., Charnes, A., & Cooper, W. W. (1984). Some models for estimating technical and scale inefficiencies in data envelopment analysis. Management Science, 30(9), 1078–1092.

6. Barro, R.J., & Sala-i-Martín, X. (1995). Economic growth. McGraw-Hill, New York.

7. Bonaccorsi, A. and Daraio, C. (2004). Econometric approaches to the analysis of productivity of R&D systems. In Moed, H.F., Glänzel, W. & Schmoch, U. (Eds), Handbook of quantitative Science and Technology Research. (pp. 51-74). Kluwer Academic Publishers,

8. Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision making units. European Journal of Operational Research, 2(6), 429-444. doi: 10.1016/0377-2217(78)90138-8

9. Cooper, W.W., Ruiz, J.L., & Sirvent, I. (2011). Choices and uses of DEA weights. In Cooper, W.W, Seiford, L.M. & Zhu, J. (Eds), Handbook of Data Envelopment Analysis, International Series in Operations Research and Management Science, 164, (pp. 93-126). S

10. Cullman, A.,& Zloczysti, P. (2014). R&D efficiency and heterogeneity-a latent class application for the OECD. Applied Economics, 46(30), 3750-3762. doi: 10.1080/00036846.2014.939410

11. Daraio, C., & Simar, L. (2007). Advanced Robust and Nonparametric Methods in Efficiency Analysis. New York, NY: Springer.

12. Doyle, J.,& Green, R (1994). Efficiency and Cross-Efficiency in DEA: Derivations, meaning and uses. Journal of the Operational Research Society, 45(5), 567-578.

13. European Commission (2010). Europe 2020. A strategy for smart, sustainable and inclusive growth. Retreived June 19, 2018, from COM:2010:2020:FIN:EN:PDF.

14. Fu, X.,& Yang, Q.G. (2009). Exploring the cross-country gap in patenting: A Stochastic Frontier Approach. Research Policy, 38(7), 1203-1213. doi: 10.1016/j.respol.2009.05.005

15. Goto, A., & Suzuki, K. (1989). R&D Capital, Rate of Return on R&D Investment and Spillover of R&D in Japanese Manufacturing Industries. The Review of Economic and Statistics, 71(4), 555-564.

16. Grossman, G.M., & and Helpman, E. (1994). Endogenous innovation in the theory of growth. Journal of Economic Perspectives, 8(1), 23-44.Guloglu, B., & Tekin, R.B. (2012). A Panel Causality Analysis of the Relationship among Research and Development, Innova

17. Han, C., Thomas, S.R., Yang, M., Ieromonachou, P.,& Zhang, H. (2017). Evaluating R&D investment efficiency in China’s high-tech industry. Journal of High Technology Management Research, 28(1), 93-109. doi: 10.1016/j.hitech.2017.04.007

18. Horvath, R. (2011). Research & development and growth: a bayesian model averaging analysis. Economic Modelling, 28(6), 2669-2673. doi: 10.1016/j.econmod.2011.08.007

19. Inekwe, J.N. (2015). The contribution of R&D expenditure to economic growth in developing economies. Social Indicators Research, 124(3), 727-745. doi: 10.1007/s11205-014-0807-3

20. Kaur, M., & Singh, L. (2016). R&D expenditure and economic growth: An empirical analysis. International Journal of Technology Management & Sustainable Development, 15(3), 195-213. doi: 10.1386/tmsd.15.3.195_1

21. Kocher, M. G., Luptacik, M., & Sutter, M. (2006). Measuring productivity of research in economics: A cross-country study using DEA. Socio-Economic Planning Sciences, 40(4), 314-332. doi: 10.1016/j.seps.2005.04.001

22. Lee, S., & Lee, H. (2015). Measuring and comparing the R&D performance of government research institutes: A bottom-up data envelopment analysis approach. Journal of Informetrics, 9(4), 942-953. doi: 10.1016/j.joi.2015.10.001

23. Lee, H., & Park, Y. (2005). An international comparison of R&D efficiency: DEA approach. Asian Journal of Technology Innovation, 13(2), 207-222. doi: 10.1080/19761597.2005.9668614

24. Lee, H., Park, Y., & Choi, H. (2009). Comparative evaluation of performance of national R&D programs with heterogeneous objectives: A DEA approach. European Journal of Operational Research, 196(3), 847-855. doi: 10.1016/j.ejor.2008.06.016

25. Lee, H.,& Shin, J. (2014). Measuring journal performance for multidisciplinary research: An efficiency perspective. Journal of Informetrics, 8(1), 77-88. doi: 10.1016/j.joi.2013.10.004

26. Lim, S. (2012). Minimax and maximin formulations of cross-efficiency in DEA. Computers & Industrial Engineering, 62(3), 726–731. doi:10.1016/j.cie.2011.11.010

27. Lim, S., & Zhu, J. (2015). DEA Cross Efficiency under variable returns to scale. In Zhu, J. (Ed), Data Envelopment Analysis. A handbook of models and methods, International Series in Operations Research and Management Science, 221, (pp.45- 66). Springer,

28. Liu, J.S.,& Lu, W.M. (2010). DEA and ranking with the network-based approach: a case of R&D performance. Omega, 38(6), 453-464. doi:10.1016/

29. Liu, J.S., Lu, L.Y.Y., Lu, W.M., & Lin, B.J.Y. (2013). A survey of DEA applications. Omega, 41(5), 893-902. doi: 10.1016/

30. Lo Storto, C., & Goncharuk, A.G. (2017). Efficiency vs Effectiveness: a Benchmarking Study on European Healthcare Systems. Economics and Sociology, 10(3), 102-115. doi:10.14254/2071-789X.2017/10-3/8

31. Martić, M., &Savić, G. (2001). An application of DEA for comparative analysis and ranking of regions in Serbia with regards to social-economic development. European Journal of the Operational Research, 132(2), 343-356. doi: 10.1016/S0377-2217(00)00156-9

32. Matei, M. M., & Aldea, A. (2012). Ranking National Innovation Systems according to their technical efficiency. Procedia-Social and Behavioral Sciences, 62, 968-974. doi: 10.1016/j.sbspro.2012.09.165

33. Meng, W., Zhang, D., Qi, L., & Liu, W. (2008). Two-level DEA approaches in research evaluation. Omega, 36(6), 950-957. doi: 10.1016/

34. Pessoa, A. (2010). R&D and economic growth: how strong is the link? Economics letters, 107(2), 152-154. doi:10.1016/j.econlet.2010.01.010

35. Roman, M. (2010). Regional efficiency of knowledge economy in the new EU countries: The Romanian and Bulgarian case. MPRA Paper No. 23083. Retrieved June, 7, 2018, from

36. Rousseau, S., & Rousseau, R. (1998). The scientific wealth of European nations: Taking effectiveness into account. Scientometrics, 42(1), 75-87.

37. Ruiz J.L., & Sirvent, I. (2016). Ranking Decision Making Units: The Cross-Efficiency Evaluation. In Hwang, S.N., Lee, H.S. and Zhu, J. (Eds), Handbook of Operations Analytics Using Data Envelopment Analysis, International Series in Operations Research & M

38. Sexton, T.R., Silkman, R.H., & Hogan, A.J. (1986). Data Envelopment Analysis: Critique and Extensions. New Directions for Program Evaluation, Volume 1986, issue 32, 73-105. doi:10.1002/ev.1441

39. Sharma, S., & Thomas, V.J. (2008). Inter-country R&D efficiency analysis: an application of data envelopment analysis. Scientometrics, 76(3) 483-501. doi: 10.1007/s11192-007-1896-4

40. Sokolov-Mladenović, S., Cvetanović, S., & Mladenović, I. (2016). R&D expenditure and economic growth: EU 28 evidence for the period 2002-2012. Economic Research-Ekonomska Istrazivanja, 29(1), 1005-1020. doi: 10.1080/1331677X.2016.1211948

41. Wang, E.C. (2007). R&D efficiency and economic performance: A cross-country analysis using the stochastic frontier approach. Journal of Policy Modeling, 29(2), 345-360. doi: 10.1016/j.jpolmod.2006.12.005

42. Wang, E.C., & Huang, W. (2007). Relative efficiency of R&D activities: A cross-country study accounting for environmental factors in the DEA approach. Research Policy, 36(2), 260-273. doi: 10.1016/j.respol.2006.11.004

43. Zabala-Iturriagagoitia J.M., Voigt, P., Gutiérrez-Gracia, A., & Jiménez-Sáez, F. (2007). Regional Innovation Systems: How to Assess Performance, Regional Studies, 41(5), 661-672. doi: 10.1080/00343400601120270