Next time you are asked your h-index ...
Next time you are asked your h-index, I suggest replying by pointing to related work by Ike Antkare [1].
| [1] | Ike Antkare. Ike
Antkare one of the great stars in the scientific
firmament. April, 2010. http://membres-lig.imag.fr/labbe/Publi/IkeAntkareSub.pdf, retrieved November 25th, 2010. |
Although some people still don't get it, the bottom line is always the same: you can't rely on a fully automatic method to evaluate intelligent entities which are aware of the evaluation method, as they are very likely to adapt to it. This is even more so when the automatic method relies on a data source which is not under the control of the evaluator(s).
Thanks to Lucas for sharing the bibliography entry.
Well, IMHO the point here isn't really about free (as in free beer) vs payware.
Rather, the point is what is the "authoritative" database of scientific publications that is used to compute h-index, or any other bibliometric measure. The payware services you mention happen to have databases with an entry barrier which is not based on automated document retrieval over the net as Google Scholar is. This is what is saving them from this specific "attack vector".
Obviously, establishing a privileged database as "authoritative" is hardly a solution and, when there is a for-profit company behind it, also induces serious ethical problems.
Publish or perish?
The Web of Science (ISI) indexes a small part of all the scientific publication, partly because of thematic choices, partly because of language restrictions (for example, the french reference in Law, which is the Dalloz journal, seem not to appear at all in their indexes), and a wealth of other reasons (for example, conferences in Computer Science are quite underrepresented). So the fact that ISI did not pick up this particular poison pill does not prove anything in the closed versus free contest: it is more a matter of human intelligence versus automated tools; the same technique can work quite well on ISI too, if one takes the time to poison it properly.
Zack, I was not trying to make the point that the payware is somehow "better", but ISI has to be mentioned for the sake of completeness: a good paper needs a good bibliography and forgetting this particular data source would not be correct in this context. The fact that the source which resisted the spoofing happens to be the "for pay" one adds a layer of "wrong", which I wanted to underline.
I think we agree these numbers are mostly useless and often downright stupid. But the spoofing work you link to is not very convincing since it unfortunately fails to spoof the de facto authoritative source (unless I am mistaken about the "de facto authoritative" status of ISI in computer science).
Roberto, I agree this proves nothing in a "closed vs free" contest. But your statement that the same technique can work well on ISI too is not proven yet (unfortunately, as I wrote in my initial comment). The only proof we have is that google scholar is not reliable for h estimations (independently of the lack of significance of h).
Actually I fear this work reinforces ISI's "authoritative" status in some minds since its resilience to at least one form of attack is now proven. This is unfortunate for the reasons pointed out by Zack, and adds to the already unfortunate use of h.
Credits to Francesco Lovergine aka frankie, who linked me to these two articles on ISI web:
Title: DENSE-FLUID LYAPUNOV SPECTRA VIA CONSTRAINED MOLECULAR-DYNAMICS Author(s): HOOVER, WG; POSCH, HA; BESTIALE, S Source: JOURNAL OF CHEMICAL PHYSICS Volume: 87 Issue: 11 Pages: 6665-6670 Published: DEC 1 1987 Times Cited: 25
Title: DIFFUSION IN A PERIODIC LORENTZ GAS Author(s): MORAN, B; HOOVER, WG; BESTIALE, S Source: JOURNAL OF STATISTICAL PHYSICS Volume: 48 Issue: 3-4 Pages: 709-726 Published: AUG 1987 Times Cited: 108
http://tinyurl.com/6yetlfz
this is not as resounding as Ike Antkare, but noticeable. I hardly imagine that Stronzo Bestiale can be the name of an actual scientist - or even a person.