Worth a read: A simple proposal for the publication of journal citation distributions

This paper in BioRXiv is definitely worth checking out. Abstract is below:

Although the Journal Impact Factor (JIF) is widely acknowledged to be a poor indicator of the quality of individual papers, it is used routinely to evaluate research and researchers. Here, we present a simple method for generating the citation distributions that underlie JIFs. Application of this straightforward protocol reveals the full extent of the skew of distributions and variation in citations received by published papers that is characteristic of all scientific journals. Although there are differences among journals across the spectrum of JIFs, the citation distributions overlap extensively, demonstrating that the citation performance of individual papers cannot be inferred from the JIF. We propose that this methodology be adopted by all journals as a move to greater transparency, one that should help to refocus attention on individual pieces of work and counter the inappropriate usage of JIFs during the process of research assessment.

Source: A simple proposal for the publication of journal citation distributions | bioRxiv

(crossposted at the ICIS Blog)

Draft Outline of Workshop "Publish or perish? The future of academic publishing and careers" #UCDavis 2/13-2/14

Thanks to EVERYONE on Twitter and elsewhere who gave useful feedback on my request for ideas about a workshop we are planning to have at UC Davis.  For background see:

Over the last few days I have discussed the meeting with many many people and we have come up with a more detailed / revised draft of the whole meeting.  I thought I would share that here — a formal announcement will be coming soon with details on registering and submitting abstracts for short talks and such.

Publish or Perish? The Future of Academic Publishing and Careers (tentative title …)

February 13-14, 2013
University of California, Davis

Hosted by the UC Davis IFHA Innovating Communication in Scholarship (ICIS) Project
Day 1: Innovations in Scholarly Publishing
  • The Changing Nature of the Journal 
  • Beyond Journals and New Forms of Digital Publishing
  • Peer Review: Assessment and Evolution
  • Keynote talk by Yochai Benkler (which will also be part of the UC Davis Provost’s Forum)

Day 2: Assessment 

  • Tracking and Measuring Impact
  • Assessment by Institutions: Current Practices 
  • Assessment by Institutions: How to Change 

Potential Topics for the Sessions – and I note for every session we hope to cover how the topic area differs between fields and regions and will have a discussion panel / discussion session at the end.
Session 1: The Changing Nature of the Journal 
  • Role of societies in publishing
  • Financial side of journal publishing
  • Institutional open access policies
  • The spread of open access publishing
  • Preprints and repositories
Session 2: Beyond Journals and New Forms of Digital Publishing
  • Data publishing
  • Code and workflows
  • Books vs. journals
  • Social media
  • The living paper
  • Micro and nano publications
Session 3: Peer Review – Assessment and Evolution
  • New models of peer review: technical merit, open review
  • Post publication review
  • Reproducibility
  • Retraction 
Session 4: Tracking and Measuring Impact
  • Article level vs journal level metrics
  • Metrics in the humanities vs. sciences
  • Institutional level metrics
  • Integration of metrics beyond articles / books 
  • Predicting future impact
  • Unique identifiers and tracking individuals
Session 5: Assessment by Institutions: Current Practices 
  • How do hiring, promotion and tenure cases work?
  • Differences between fields
  • Tenure track vs. other academic positions
  • Assessment for grants / funding
Session 6: How to Change Models of Assessment
  • Search committee training
  • Bias: implicit and otherwise
  • Methods to limit implicit and explicit bias
  • Administration perspective