"But look at how few citations she got" - The problematic nature of citations

Google Scholar citation counts say
nothing about the quality or
importance of research.

The use of the number of citations to reflect quality is tempting but quite problematic. The citation count is nevertheless used abundantly and feeds our addiction to become better. It has been a while, but I didn't forget the first time I was cited. Neither will I easily forget getting cited within 10 days of the publication of an article. I also will not forget the many times my work was mentioned in article but without the matching journal article. Those citations "do not count".

I cannot link this to an article (plz let me know if you know a good study about this), but it's a public secret that scholars prefer citing prestigious journals when they have the choice to cite the equal or sometimes even more informative article in a less prestigious journal. I suspect some "by association" mechanism to be in action here.

I do know two sources that study or describe this phenomenon were published this spring. And I hope this is the start of many to follow, because there are many other problems. We all experienced that "but didn't you read my paper?!?" moment, and that moment of unbelieve when you hear that there are journals that restrict the number of citations you can make.

Citational justice

The first is a write up by Maastricht University colleagues Aurélie Carlier, Hang Nguyen, Nicole Basaraba, Sally Wyatt, and Sharon Anyango: UM Citation Guide - A Guide by FEM. Besides this link (for future readers: the Internet Archive copy), I could not find a DOI for this article, nor a copy in our library, but I am happy it is getting some international attention, e.g. in this post (where I first saw the term citational justice). It is a good, tho disturbing read with a good number of citations of further reading. It writes: "Several recent studies have highlighted the systematic under-citation of works by women and other marginalized groups in various research fields". 

Given my aforementioned experiences, I am hardly surprised. Quite disappointed, but not surprised.

A second source is this paper: Leading countries in global science increasingly receive more citations than other countries doing similar research (doi:10.1038/s41562-022-01351-5), by Charles Gomez, Andrew Herman, and Paolo Parigi. The study shows fascinating data. I haven't read all the details of the methods, and believe the authors studied a really hard topic. Their results confirm what we see, but backed with data, we can start talking seriously about how to get out of this.

Harassment and inequality

The link to Recognition&Rewarding is very obvious, but I cannot help but wonder too how this relates to Harassment as a consequence and cause of inequality in academia: A narrative review (doi:10.1016/j.eclinm.2022.101486), by Susanne Täuber, Kim Loyens, Sabine Oertelt-Prigione, and Ina Kubbe. Indeed, we all see at least weekly how citation counts are used to establish hierarchy. Staring blind on citation numbers is, umm, misguided (euphemism). It may be hard, it is expensive, but we can and must do better than we do now. Yes, exactly, this is why I struggle with the Dutch narrative of "we must be able to use the Journal Impact Factor".

BTW, citational injustice follows, at least in a significant part, from how we select and read literature. I see this all the time: researchers chose to ignore literature and therefore will not know about the literature and therefore not cite it. So, literature reading injustice should be part of our university curricula too. This is hinted at in the UM Citation Guide, where the Who to cite? section asks for awareness. But what you did not read in the first place, you cannot consider citing. Yes, exactly, this is why Open Access matters so much to me! Here, I also strongly encourage you to read up on Whose knowledge when you limit your literature searching to PubMed, Scopus, or Google Scholar.

If I didn't convince you yet, let me pose this situation. Would you have given my message in the blog some amount of, more, or less attention if I had it appeared in the Observant or in the NRC? It would not. I hope you see the same pattern as I do.

Reflection

First, the quote in the title makes me feel uneasy. It is sarcastic but, as they say, very close to home. Sadly, I hear and see such statements regularly. It is very depressing and I feel powerless. And nervous: how well am I doing myself? Do I get enough time to double check I am not making these mistakes? Surely, I am not reading enough. Surely, I am not always citing all the articles I should be citing. How bad are my practices? Am I bringing up these issues enough (I already head some colleagues reply "It cannot be that bad?". Am I confronting fellow researchers enough with bad practices? Actually, no, I am not doing this enough yet (I already hear the comments "oh, there is that guy on Twitter again").

But I am not powerless and feel I have to stand up too (like FEM did very well!). and that is why I wrote this blog post. We can and must do better, particularly when it makes us feel uneasy.