Sunday, September 25, 2011

Altmetrics and the "Impact report for Institutet för miljömedicin (IMM)"

As you know (if not, and you are a scientist, then you must be living in a cave), science is discussing heavily the role of impact analysis in research assessment. What is the impact of a particular study? Is this young scientist worthy of becoming a professor? Etc, etc. There is no ultimate, one-fits-all solution, but if your institute is still using the Journal Impact Factor... (see below).

As possible solutions alternative metrics are being proposed, and even manifestos like that of #altmetrics. Some of the proposals can at least be used as a filter. Many of my readers are already using the social webs for filtering out interesting papers. You have too, because of the tsunami of publications. Solutions have been abundant, from many perspectives. I will not review all those here, and suffice with mentioned (deep links to where literature is discussed and/or proposed) Chemical blogspace, Mendeley, CiteULike, ResearchBlogging, the new Altmetrics Explorer (see below), etc, etc, etc.

These services point you to papers that others found interesting. And the more people find it interesting, to more likely it is that you will too. This is why alternative metrics do focus on the number of HTML reads, bookmarks, etc. They reflect how interesting the paper is. I doesn't say much about to future impact of the paper, but more about the potential.

The real impact of a paper is much harder to assess. Really, if you still believe the Journal Impact Factor does that, think again. The CDK project is suffering from focus on wrong metrics: it is not published in Nature or Science, but the impact is very significant: it is being used in many tools that are extensively used. So, to measure the impact of the CDK, you should take into account also the impact of research standing on the shoulders of the CDK. In a weighted manner, of course.

And mind you, even the number of citations of the paper itself already gives a much more accurate view on the impact than the JIF of the J. Chem. Inf. Mod. which is insignificant compared to that of Nature or Science. But, the CDK papers are cited well above the JIF of Nature; In fact, the first CDK is in the top 5%  of most cited literature in the world (according to the KI bibliometrics project), and the second CDK paper close to it. That is impact, I would say. And I am now just waiting to have that impact reflect on my future. While, according to this KI project, I am cited three times the world average, 2.5 times the average KI scientist, but still a mere post-doc (one, in fact, returning to the Netherlands; for family reasons, and I am going to miss my current colleagues very, very much!).

Now, various projects are popping up to visualize various aspects of impact (short and long term), such as CitedIn, Google Scholar, and a new tool called Total Impact which I ran on our IMM group at Mendeley (see this link):

I will not deny if is rewarding to see people care enough about your publications that they bookmarked them, but there is another important aspect as to why these services are important. Business intelligence is probably the proper term here. The long term impact indicators are, of course, much easier to interpret here. If you work is being cited, it is easy to check if and how your work is being used. This is harder with people bookmarking your work, but it does point you to new application areas, if you take the time every other month or so, to see what fields people are working in that bookmark your paper. That way, you get to see what research fields your work may have impact in, and thus, perhaps, be a topic of your next paper.

No comments:

Post a Comment