Research Messenger
Volume 1 Issue 4
January 29, 2007
SDSU'S Joint Doctoral Programs
I'd like to highlight the article that appeared in the January 12 issue of the Chronicle of Higher Education, introducing a new metric for ranking doctoral programs.
Evaluations by the National Research Council are the gold standard of graduate rankings, but are conducted only every dozen years, such that data points are too sparse to track progress. U. S. News and World Report offers annual rankings, but these are widely dismissed as being subjective.
Into this breach of either currency or objectivity has stepped Academic Analytics (AA), a for-profit company affiliated with SUNY Stony Brook. They replaced the tweedy peculiarities of professorial opinion with brute force data mining, scanning the public domains of libraries, journals, funding and honorary agencies to provide analyses that are both objective and current.
Academic Analytics reviews and weighs three factors: publications and their associated citations (60%), federal grants (30%), and honors and awards (10%).
Publications include books and journal articles. The former are derived from Amazon.com's database, which matches that of the Library of Congress; the latter from Scopus, a database that covers more than 15,000 journals. A book is weighed as five articles. For its 2005 assessments, AA reviewed book titles from the preceding five years and journal articles from three. Added to this index was the number of citations a faculty member's publications received during the same period.
Grant funding was evaluated by scanning the awards made by the NIH, NSF, DOEd, USDA, and DOE. Awards from the NEH were also captured, but treated as honors because of their small numbers and size.
Honors were counted from the web sites of 55 organizations that make the most widely recognized awards, and were weighed according to the level of prestige associated with each.
The result is an index that is objective, transparent, and annual, promising to generate an accumulating record of a doctoral faculty's scholarly productivity. By measuring achievement, and ignoring reputations that are freighted with inertia, it leads to surprising outcomes.
None more so than SDSU's rankings. Of the 10 doctoral programs that were extant during the evaluation period (we have since added six(, three received special mention. Language and Communicative Disorders (joint with UCSD) ranked fourth nationally, Teacher Education (with Claremont) ranked third, and Clinical Psychology (with UCSD), in one of the nation's largest and most competitive academic arenas, was rated first.
As an institution, SDSU's faculty productivity was ranked 46th nationally, as measured by the z-scores reported. We were immediately below UC Irvine and U. Illinois Chicago, and immediately above Indiana U., Boston U., Purdue, RPI, UC Riverside, and UT Austin.
Yes, we profited from the productivity of our partners – the only instance I can think of where the strictures of the Master Plan worked to our advantage in research – but in each of the three cases cited above, ours was a contribution of strength to the collaboration. Those whose productivity would have been evaluated as part of clinical psychology, for example, published a mean of nearly seven articles and won some &500,000 in funding annually. Rather than our partnerships, valuable as they are, I believe we were helped by the fact that, to entertain a doctorate, our programs must have proven themselves highly worthy in scholarly productivity and funding (in disciplines where that is a factor(. Thus, success in the measures that AA uses is a prerequisite for each of our programs. None comes into being casually.
Assessment is upon us in higher education. An objective measure of the common currency of scholarship – the impact of the printed word on its discipline – is a welcome tool. All the more so when it reaffirms our belief that we have outstripped our reputation, and reveals that fact publicly.
Tom Scott