Richard Holmes
The QS World University Rankings and THE World University Rankings by Subject Group and Subject
The QS World University Rankings
History
The QS World University Rankings began life in 2004 as the Times Higher Education Supplement (THES) – Quacquarelli Symonds (QS) World University Rankings. THES was then a weekly newspaper which was not a part of the Times newspaper although at that time it had the same owner. QS Quacquarelli Symonds Ltd (QS) was a company founded in 1990 that specialised in the recruitment and admission of international students.
The arrangement was that QS collected and processed the data while THES decided on the weighting assigned to the various indicators. In the first edition the rankings gave a weighting of 50% to a survey of academic opinion, 20 % to faculty student ratio, 20 % to citations per faculty, 5 % to international students, and 5% to international faculty.
In 2005 QS reduced the weighting for the academic survey to 40% and added an employer survey with a 10% weighting.
In 2007 QS introduced several changes. They switched from Thomson Reuters to Scopus as a source of bibliometric data. They also introduced Z-scores and switched from head counts to full time equivalents for faculty and student data. After that there was no formal announcement of methodological changes, although there have been some modifications.
At the end of 2009, THE announced that it had ended its partnership with QS and would team up with data and media giant Thomson Reuters to publish the Times Higher Education World University Rankings.
QS inherited the data and methodology of the rankings and has continued to publish them as the QS World University Rankings
Publisher
QS Quacquarelli Symonds, a private company with HQ in the UK and branches in New York,Boston, Washington DC, Paris,Singapore, Stuttgart,Sydney, Shanghai,JohannesburgandAlicante, is a “global provider of specialist higher education and careers information and solutions.”
Frequency
Once a year in September
Publication Mode
Online: Supplementary material is available with registration. Currently, some material is not available. Analysis and more data are available at
Ranking Products
World University Rankings,World University Rankings by Subject, Faculty (subject group) Rankings, BRICS, Asian, Latin America, Best Student Cities, Under 50 universities. Arab region pilot rankings.
Target Audience
Emphasises international students, especially graduate students, and sponsors. “Mum and Dad in Mainland China”.
Scope
890 institutions listed in 2014.
Other Activities
Consulting, QS Stars rating system, Grad School Tours.
Accreditation
Accredited by IREG.
Volatility
Between 2013 and 2014 the average top 100 university moved up or down 3.94 places.
Bias
A paper by Christopher Claassen shows a strong bias to the UK, Australia and the Netherlands and against the US.
“UK universities are rated more highly, on average, by the QS and THE ranking systems than they are by the latent trait model,9 and theWebometrics ranking also shows a positive bias toward Spanish universities. These threesystems favour their home country universities, on average, by at least .25 points on thelatent variable scale: enough for a university outside the top 200 to increase its rankingby 20 to 50 places.”
(Claassen, 2015)
Highest Ranked Russian Institution in World University Rankings 2014
Lomonsov Moscow State University 114
Other Russian institutions ranked (5top100 universities highlighted)
St Petersburg State University 233
Bauman Moscow State Technical University 322
Novosibersk State University 328
Moscow State Institute of International Relations 399
Moscow Institute of Physics and Technology State University 411-420
People’s Friendship University of Russia 471-480
National Research Nuclear University MEPhI (Moscow Engineering Physics Institute) 481-490
St Petersburg Polytechnic State University 481-490
Tomsk State University 491-500
Higher School of Economics 501-550
National Research Tomsk Polytechnic University 501-550
Kazan Federal University 551-600
Ural Federal University 551-600
National Research University Saratov State University 601-650
Southern Federal University 601-650
Far Eastern Federal University 701+
Lobachevsky State University of Nizhniy Novgorod 701+
National University of Science and Technology MISIS 701+
Plekhanov Russian University of Economics 701+
Voronezh State University 701+
Indicators 2014
See here for details of methodology.
Academic Survey
QS claim a total of 63,676 responses in 2014, including those carried over from the previous 2 years. Responses varied by faculty area: from Life Sciences and Medicine with 17.9% to Social Sciences and Management with 30.9%. Respondents are asked to nominate the best universities for research in their subject area.
Respondents are reached through several channels with response rates ranging from 2 to 8%:
- Previous respondents 1,724 in 2014
- The mailing lists of World Scientific publishing company were used until 2011 and may be used in future
- Lists from Mardev, an online publishing company provided 200,000 names
- An Academic survey sign up facility yielded 25,000 names between 2010 and 2014
- Institutional lists of up to 400 names per institution. In 2014 400 institutions supplied lists with a total of 190,000 names.
QS state that respondents are screened and that no respondent can vote for their declared affiliation. For the subject areas there is a weighting for regions indicated by respondents based on completed responses. International and domestic responses are counted separately and then combined with an 85% weighting for international responses. The count of domestic respondents is adjusted against the number of institutions available for selection in that country and the total response from that country.
Each subject area has a maximum score of 100 and they are combined with an equal weighting.
In 2013, 10.0% of responses were from the US, 6.5% from the UK, 6.3% from Brazil and 1.7% from Russia.
Employer survey
Data comes from several sources: respondents, databases, partners and an employer survey sign up facility. There were 28,759 responses in 2014:10.3% were from the USA, 6.6% UK, France 5.4% and 2.4% from Russia. Unchanged responses are carried over for 3 years and there is a regional weighting based on completed responses. There is a 70% weighting for international responses followed by square rooting.
Citations per faculty
The citations are from a five year period and are derived from Scopus. Faculty data are Full Time Equivalent and include research and teaching staff. Self-citations are excluded.
Faculty Student Ratio
Faculty and Student data are full time equivalent and are derived from a variety of sources. QS prefer to use data submitted from institutions but will use other sources if necessary.
International students and faculty
Data is derived from a variety of sources. International refers to citizenship.
QS Ranking by Faculty (Subject group)
QS now provides rankings of universities in five faculty or subject areas and in 30 subjects. They use two indicators from the world rankings and two additional indicators with different weightings, as indicated in Table 1 and Table 2. The top 400 universities in each subject area are listed.
Table1. Weighting of indicators in QS Faculty/Subject area rankings.
Faculty area / Academic survey / Employer survey / Citations per paper / H-indexArts and humanities / 60 / 20 / 10 / 10
Engineering and technology / 40 / 30 / 15 / 15
Life sciences and medicine / 40 / 10 / 25 / 25
Natural sciences / 40 / 20 / 20 / 20
Social sciences and management / 50 / 30 / 10 / 10
Russian Universities in the QS Faculty/Subject Are Rankings
Arts and Humanities
92 Lomonosov Moscow State University
263 St Petersburg State University
Engineering and Technology
132 Lomonosov Moscow State University
313 St Petersburg State University
Life Sciences and Medicine
374 Lomonosov Moscow State University
Natural Sciences
34 Lomonosov Moscow State University
209 St Petersburg State University
210 Novosibersk State University
248 National Research Nuclear University MEPhI
367 Moscow Institute of Physics and Technology State University
Social Science and Management
122 Lomonosov Moscow State University
199 St Petersburg State University
232 Higher School of Economics
QS Rankings by Subject
Thirty subjects are ranked according to varied weightings of the four indicators used in the Subject area rankings.
Table 2. Indicator weightings QS Subject Rankings
Subject / Academic / Employer / Citations / HBiology / 40% / 10% / 25% / 25%
Earth Sciences / 40% / 10% / 25% / 25%
Environmental Sci / 40% / 10% / 25% / 25%
Medicine / 40% / 10% / 25% / 25%
Materials / 40% / 10% / 25% / 25%
Pharmacy / 40% / 10% / 25% / 25%
Chemistry / 40% / 20% / 20% / 20%
Mathematics / 40% / 20% / 20% / 20%
Physics / 40% / 20% / 20% / 20%
Computer Science / 40% / 30% / 15% / 15%
Chemical Eng / 40% / 30% / 15% / 15%
Electrical Eng / 40% / 30% / 15% / 15%
Mechanical Eng / 40% / 30% / 15% / 15%
Agriculture / 50% / 10% / 20% / 20%
Economics / 50% / 30% / 10% / 10%
Civil Eng / 50% / 30% / 10% / 10%
Psychology / 50% / 30% / 10% / 10%
Law / 50% / 30% / 5% / 15%
Acc & Finance / 50% / 40% / 5% / 5%
Politics / 50% / 40% / 5% / 5%
Communication / 50% / 20% / 10% / 20%
Education / 60% / 10% / 15% / 15%
Geography / 60% / 10% / 15% / 15%
Statistics / 60% / 10% / 15% / 15%
Languages / 70% / 30% / 0% / 0%
History / 70% / 10% / 5% / 15%
Sociology / 70% / 10% / 5% / 15%
Linguistics / 80% / 10% / 5% / 5%
Philosophy / 80% / 10% / 5% / 5%
English / 90% / 10% / 0% / 0%
Performance of Russian universities in QS subject rankings
Modern languages
51-100 Lomonosov Moscow State University
201-250 St Petersburg State University
Philosophy
51-100 Lomonosov Moscow State University
Computer Science and Information Systems
101-150 Lomonosov Moscow State University
Electrical and Electronic Engineering
301-350 St Petersburg State University
Mechanical Aeronautical and Mechanical Engineering
151-200 Lomonosov Moscow State University
Biological Sciences
251-300 Lomonosov Moscow State University
Chemistry
101-150 Lomonosov Moscow State University
301-350 St Petersburg State University
Earth and Marine Sciences
101-150 Lomonosov Moscow State University
Environmental Sciences
201-250 Lomonosov Moscow State University
Materials Science
151-200 Lomonosov Moscow State University
Mathematics
49 Lomonosov Moscow State University
151-200 St Petersburg State University
201-250 Novosibersk State University
Physics and Astronomy
49 Lomonosov Moscow State University
201-250 St Petersburg State University
201-250 Novosibersk State University
211-300 National Research Nuclear University MEPhI
301-350 St Petersburg Polytechnical University
Statistics and Operational Research
151-200 Lomonosov Moscow State University
The Times Higher Education Subject Rankings
History
In 2009 Times Higher Education (THE) announced that it was ending its partnership with QS and that it would use data supplied and analysed by Thomson Reuters. The THE-QS were “unfit for purpose”, especially the academic survey. The first edition of the new THE rankings came out in 2010 and was subject to much criticism partly because of the elevation of Alexandria University in Egypt to 4th place in the world for research impact.
There was a good deal of tweaking for the 2011 rankings and the status of the rankings appears to have recovered, especially among leading research-intensive universities and their supporters in Europe and the United Kingdom.
In November 2014, THE suddenly announced that it was ending its relationship with Thomson Reuters and would use data from Scopus. THE indicated that it was entering a new period of “transparency and accountability” and spoke of “reforms” to the ranking.
So far, based on a “master class” earlier this year in Moscow, it appears that THE will keep the structure and weighting of the current rankings. However, there has already been one possibly significant change. THE have taken over the reputation survey themselves and have apparently succeeded in ensuring a broader participation.
It is not possible to compare the current reputation survey with the previous editions since Thomson Reuters apparently did not allow the release of data on the nationality of respondents. However, THE seem to have got more respondent s from China (13.8% of total respondents), Japan (7.2%) , Russia (5.5%) and Germany (4.8%). The corresponding scores for the QS academic survey are China 1.1%, Japan 2.9%, Russia 1.7% and Germany 3.8%. It also seems to have got more respondents from the arts and humanities.
It is not clear exactly what the effect of this. Tsinghua University has risen from 36th to 26th place in the THE reputation rankings between 2014 and 2015 and Peking University from 41st to 32nd. Oddly enough, considering the small number of respondents from China, Peking still does better on the QS reputation survey where it is 19th and Tsinghua 27th.
Tokyo has fallen slightly to 12th place and does worse than it does in the QS survey where it is 7th.
Moscow State University does well in this year’s reputation survey where it is 25th better than last year and much better than QS, who put it in 83rd place for academic reputation.
There are also signs that THE are considering some changes to the citations indicator. Phil Baty has suggested that something will be done about the problem of papers, many of them in particle physics, with thousands of citations and thousands of contributors . One solution is to just delate such papers, which is what Thomson Reuters did when compiling its list of influential scientific minds. Another is to use fractional counting of contributors or authors, which is an option in the Leiden Ranking.
Summary of the THE World Rankings
“Underpinning theWorld University Rankings 2014-2015is a sophisticated exercise in information-gathering and analysis: here we detail the criteria used to assess the global academy's greatest universities
TheTimes Higher EducationWorld University Rankings are the only global university performance tables to judge research-led universities across all their core missions - teaching, research, knowledge transfer and international outlook.
We employ 13 carefully calibrated performance indicators to provide the most comprehensive and balanced comparisons, which are trusted by students, academics, university leaders, industry and governments.
The methodology for the 2014-2015 World University Rankings is identical to that used since 2011-2012, offering a year-on-year comparison based on true performance rather than methodological change.”
[my highlights]
(
Publisher
Times Higher Education, a weekly magazine published in the UK by TES Global. Formerly Times Higher Education Supplement (newspaper format) but never a section of the Times.
Frequency
Once a year in October
Publication Mode
Print, iPhone/iPad and online.
Ranking products
World University Rankings, Reputation rankings, Under 50 rankings, Asian rankings, BRICS plus emerging economies rankings, MENA rankings “snapshot.
Target Audience
Leading European research universities, administrators and policy makers in Europe, especially the UK. Currently interested in Asia and emerging economies.
In December 2009 Phil baty wrote:
“We want academics and university administrators to have confidence in a more rigorous and transparent ranking.”
Scope
Over 800 universities assessed. Data provided for 400 universities. Now talking about expansion to include about 1000 institutions and institutions outside a narrow elite.
Other activities
THE World Universities Summit, Africa Universities Summit, MENA Universities Summit, conferences, seminars, masterclasses. It seems that theyare talking about integrating rankings, advertising and benchmarking.
Volatility
Between 2013 and 2014 the average place among top 100 universities was 4.34, more than the QS world rankings and more than the Shanghai ARWU in most years. The volatility could be due to the large weighting for the reputation survey, where universities get a handful of votes outside the top 50 or one hundred, the ranking of more universities which changes the averages against which raw scores are benchmarked, the scaling of many indicators, and/or the citation indicator where a single paper can produce a large number of citations.
Bias
The Claassen paper shows a strong bias to the UK and against Brazil, China and continental Europe and is neutral with regard to USA. Possible explanations include reliance until last year on the Thomson Reuters database which is biased towards English language publications, an emphasis on income which is measured by three indicators, a heavy weighting for reputation and international orientation citations.
Top Ranked Russian Institution in 2014
196 Lomonosov Moscow State University (well below QS)
Others
None in the top 400.
Indicators with current weighting
To date it appears that THE are keeping the structure and weighting of their rankings. Note that scores are standardised and converted into Z-scores.
Teaching: The learning environment (30%)
Teaching Reputation Survey 15%. Distributed according to UNESCO count of the number of researchers in each country.
Faculty student Ratio 4.5%. Full time equivalent numbers for students and faculty.
Doctorate to bachelor degrees awarded 2.25%
Doctoral degrees awarded per academic staff 6%
Institutional income per academic staff 2.25%.
Research: Volume Income Reputation (30%)
Research reputation 18%. Distributed according to UNESCO count of the number of researchers in each country.
Research income per staff and normalised for purchasing power parity 6%
Papers per academic and research staff 6%.
Industry income: Innovation (2.5%)
Research income from industry per academic staff.
International outlook (7.5%)
International students as a percentage of total students 2.5%
International faculty as a percentage of total faculty 2.5%
Percentage of publications with International Collaborators 2.5%
Citations: Research Impact (30%)
Citations over a six year period to papers published over a five year period. Normalised by field (250 according to Simon Pratt of Thomson Reuters, may be 300 this year) and year. The citations in a specific year in a specific field are compared with the world average. Until now the Web of Science database has been used. This year it will be the Scopus (Reed-Elsevier) database.
Regional modification (the score for the university is divided by the square root of the score for the whole country). In other words universities benefit from being in a country with low citations. The effect of this was seen in 2011 when Hong Kong universities suffered after being counted separately rather than as a part of China.
Comments on Methodology
THE originally hoped to reduce the reliance on reputation data but found that it was difficult to get reliable data from institutions.
THE expects universities to submit data on an annual basis and will not include those who fail to provide information in contrast to QS who will make up the gaps with old data or from third party sources.
Several universities do not supply data about research income from industry and there is formula for making up the gap.
Eleven of the indicators are grouped into clusters of five, three and three. A combined score is given to the public but detailed information is provided only to the institutions themselves. There is a serious reduction in transparency since it can be difficult to work out exactly what has contributed to a rising or falling score for teaching or research.
THE Rankings by Subject-Group
So far, THE has only published subject-group rankings, not subject rankings. It would not be a surprise if THE did start adding subject rankings to its menu in the near future.
THE have six subject groups compared to five for QS with medical sciences classified separately.
Methodology
See here for details. 100 ranked in each subject area, compared to 400 for the QS faculty/subject group rankings.