New Project About Ranking Researchers.
March 13, 2011 Leave a comment
I have just thought an idea of ranking researchers, which I think is a good topic for my course project of EC 724.
There are millions of researchers all around the world. Despite the fact that science has been broken down into many specific fields , there are still thousands of researchers in each field.
Who is the best researcher in a field? A researcher can usually point out several famous researchers in the field he is familiar with. However, we still lack a reasonable quantitative metric to describe “how famous” a researcher is.
We realize that in real life the harm of a ranking system usually surplus the benefit it brings. A poor designed ranking system which relies obviously on a certain set of metrics will draw people’s attention to those metrics and make them ignore the true meaning of research.
For example, the existing university ranking system in China relies heavily on the number of published papers. That’s the reason why the number of paper published by Chinese researchers has increased so rapidly. However, a large portion of those papers don’t worth reading.
In our research, we want to design a ranking system which is independent of specific metric. The score we assign to each professor just reflect other researchers’ opinions towards him.
That’s a score that
This is indeed a optimization problem. Suppose there are N researchers. vector x is the score assigned to each researchers. We need to find an optimal x* that satisfies each professors as much as possible.
So it is a multiobjective optimization problem with N objectives. We can find all the pareto optimal solutions first.
What’s the constraint. There are some training set. Which should be satisfied. For example, if we know that professor A is surely better than B, the score of A should be no less that of B.
The problem is that we cannot send questionnaire to every researchers. Instead we deduce a researcher’s opinion from his publications. We assume that the publication of professor can represent all his academic opinions.
We need overcome the problem of frequent mutual reference. We know that researcher in the same group will refer each other’s literature frequently. We need prove that our algorithm is insensitive to this factor. That is to say, people in a group can not gain advantage by deliberately mutually refers each others’ paper.
We need to show that the result is insensitive to the number of publications. A researcher cannot gain advantage by publishing more low quality papers.