We are pleased to invite you to a conference on:
GAMING METRICS: INNOVATION & SURVEILLANCE IN ACADEMIC MISCONDUCT
UC Davis, February 4-5, 2016
Organized by the Innovating Communication in Scholarship Project (ICIS) with support from the Center for Science and Innovation Studies (CSIS)
DESCRIPTION:
Misconduct has traditionally been tied to the pressures of "publish or perish" and, more recently, to the new opportunities offered by electronic publishing. The conference takes the next step to asks whether the modalities of misconduct are now evolving to adapt themselves to modern metrics-based regimes of academic evaluation. Have we moved from "publish or perish" to "impact or perish"? If so, are metrics of evaluation now creating new incentives for misconduct? And can we still reliably draw a clear separation between gaming the metrics game and engaging in misconduct? Traditional discourses and policies of misconduct were rooted in oppositions between truth and falsehood, right and wrong, honest mistake and fabrication, but new metrics-based misconduct seems to be defined by the extent of the gaming involved. In sum, are new metrics-based forms of misconduct asking us to rethink and redefine misconduct?
PROGRAM:
* DAY 1 (MU II Room, Memorial Union)
9:00-9:15 Welcoming Remarks (Ralph Hexter, Provost, UC Davis)
9:15-9:30 “FROM PUBLISH OR PERISH TO IMPACT OR PERISH”
A brief discussion of the conference themes and working hypothesis concerning the relation between academic metrics and misconduct. Current scenarios exemplify a vast increase of kinds of misconduct compared to traditional definitions (fabrication, falsification, plagiarism), but also point to a shift in the very goals of misconduct. Initially driven by “publish or perish,” misconduct has become geared toward maximizing more complex metrics of academic credit encapsulated in a new imperative: “have impact or perish.”
Mario Biagioli (UC Davis, Science and Technology Studies, Law, & History)
9:30-10:30 “GAMING THE GAME ACROSS THE BOARD”
This session introduces and contextualizes the conference’s subsequent discussions by casting a wider net on metrics-gaming well beyond the specific field academic publishing, looking at how different communities and professions construe the line between acceptable and unacceptable gaming. Mapping a wide range of gaming scenarios will then allow us to contextualize the specific forms of academic misconduct involving metrics gaming concerning academic credit.
Timothy Lenoir (UC Davis, Cinema and Digital Media & Science and Technology Studies) (Chair)
Sally Engle Merry (NYU, Anthropology)
Alex Csiszar (Harvard University, History of Science)
Paul Wouters (Leiden University, Science and Technology Studies)
Karen Levy (NYU, Media, Culture, and Communication)
11:00-12:00 “UNIVERSITY RANKINGS: GAMING OR COOKING?”
As university rankings are gaining increasing importance across the globe, they have been praised as agents of democratization against traditional academic “brands” living off reputational rent, but also criticized for the substantial ranking distortions that their easy gaming allows for. When can these practices be treated as ranking gaming, and when do they cross over into institutional misconduct?
Martin Kenney (UC Davis, Human Ecology) (Chair)
Barbara Kehm (University of Glasgow, School of Education)
Lior Pachter (UC Berkeley, Mathematics)
1:30-3:00 “PERSONAL V. INDUSTRIAL CHEATING”
One conspicuous difference from the days of “traditional” misconduct is the shift between misconduct as the work of individual scientists and scholars to scenarios in which misconduct is a more “collaborative” endeavor, as in the case of citation rings among journals to maximize their impact factors. (The production of fake alternative impact factors may be another example). In addition to these novel conspiracies (which typically involve editors and publishers rather than traditional individual cheats like scientists and scholars), modern misconduct also involves businesses and organizations providing tools, platforms, and opportunities to academics interested in misconducting themselves. These include so-called “predatory” journals, fake conferences, fake prizes, etc., that is, tools that enable and entice academics to meet the demands of their institutions’ evaluation metrics by gaming/cheating them. Also, while these activities concern publications, they are not limited to the production of a fraudulent text (as “traditional” misconduct typically was), but aim at facilitating its publication. They may be perhaps termed “postproduction” misconduct.
MacKenzie Smith (UC Davis, University Librarian) (Chair)
Finn Brunton (NYU, Media, Culture, and Communication)
Sarah de Rijcke (Leiden University, Science and Technology Studies)
Jeffrey Beall (University of Colorado, Denver, Information Science)
Dan Morgan (University of California Press, Collabra Project)
3:15-5:00 “META GAMING, META CHEATING”
This session has a double goal. First, to analyze the kind of gaming that involve not the manipulation of a metric but the construction or adoption of a metrics – not gaming an established game, but the gaming that goes into defining the game itself. Is the competitive market of academic metrics (from faculty performance to university rankings) a form of gaming the game itself? And where/when/how can it become misconduct? Second, this session aims at engaging with Goodhart’s law, which is taken to show not only that the introduction of any kind of metric creates a market for gaming it, but that by so doing it invalidates the significance of that metrics. If so, one could argue that any metrics will create the possibility of misconduct, but that the articulation of forms of misconduct specific to that metric will eventually “crowd” that market, thus creating an incentive to change the metrics, which in turn will usher in the next generation of innovative misconduct. Or can we argue, against Goodhart, that it is possible to find a metrics of academic evaluation that can break the nexus with gaming/misconduct?
Anupam Chander (UC Davis, Law) (Chair)
Johan Bollen (Indiana University, Informatics and Computing)
Carl T. Bergstrom (University of Washington, Biology)
Jennifer Lin (Crossref)
Michael Power (London School of Economics, Accounting)
James Griesemer (UC Davis, Philosophy)
* DAY 2 (Kalmanovitz Appellate Courtroom, King Hall)
9:15-9:30 Welcoming Remarks (Kevin Johnson, Dean, UC Davis School of Law)
9:30-10:30 “MISCONDUCT WATCHDOGS (I)”
The emergence and pervasiveness of new forms of misconduct exceed the reach, resources, and conceptual framework of traditional governmental watchdog organizations typically connected to funding agencies like, in the US, the ORI. This has spawned a new generation and new figures of misconduct surveillance, detection, and prosecution. Among these is a new breed of “watchdogs” — new actors who are often institutionally unaffiliated. These “watchdogs” have assumed an important role and a credible voice, often by creating new “ecologies of support” for themselves — websites, blogs, wikis, social media, etc. Does their somewhat unique role indicate something about the specific nature of modern academic misconduct? Does it suggest that the “battlefield” of misconduct is moving away from governmental agencies (acting according to traditional and possibly outdated definitions of misconduct) and toward journals and the watchdogs who monitor their publications?
Jonathan Eisen (UC Davis, Genome Center) (Chair)
Ivan Oransky (Retraction Watch & NYU)
John Bohannon (Science Magazine)
Elizabeth Wager (Sideview)
10:45-12:00 “MISCONDUCT WATCHDOGS (II)”
Jonathan Eisen (UC Davis, Genome Center) (Chair)
Darren Taichman (Editor, Annals of Internal Medicine)
Debora Weber-Wulff (University of Applied Sciences Berlin, HTW, Media and Computing)
Brandon Stell (The PubPeer Foundation & CNRS)
1:30-2:30 “COUNTERFEITING BRANDS V. FAKING PRODUCTS”
This session looks at a specific form of fakery rooted in “brand appropriation.” While the preceding session considers generally fake journals, conferences, etc., here we want to look more specifically at imaginary journals whose titles (as well as the look and feel of their websites) are made to resemble those of well-known and respectable journals. One could perhaps add to this list certain “academic” conferences that take place in prestigious locations (say, Oxford) but are not actually affiliated with the university, or the appropriation of the names of respected academics that are then listed (without authorization) on editorial boards of fake journals or organizing committees of fake conferences. Similarly, fake universities who sell degrees without any attempt at educating their students (not even online) tend to assume names with an Ivy League ring to them. The common denominator here is an attempt at the mimicry of a “brand” rather than just the copying/pirating of a product.
Madhavi Sunder (UC Davis, Law) (Chair)
Marie-Andree Jacob (Keele University, Law)
Alessandro Delfanti (University of Toronto, Communication, Culture, Information and Technology)
Sergio Sismondo (Queen’s University, Philosophy)
2:30-3:30 CARNIVALESQUE RESPONSES
While the misconduct “watchdogs” (discussed in a previous session) expose through public communication and denunciation, this session focuses on other actors who reveal misconduct and poor oversight through a carnivalesque approach. Humor and absurdity– submitting profane papers and computer-generated gibberish articles that “sound” academic, or whistleblowers using clever anagrams as aliases — become a mode of critique and unmasking. Neither clearly “predatory” journals, “fake” conferences nor “legitimate” journals are immune to being the subject of a joke — a joke that, in some cases, may be more powerful than punishment. In a way, carnivalesque responses to misconduct continue the logic of an older history of art forgery-as-prank in which the forgery reveals through a kind of satire. Are these cases telling us, perhaps, that satire is the best approach to both metrics and the gaming they elicit?
Alexandra Lippman (UC Davis, Innovating Communication in Scholarship Project) (Chair)
Cyril Labbé (Joseph Fourier University – Grenoble I)
Burkhard Morgenstern (Universität Göttingen, Bioinformatics)
Paul Brookes (University of Rochester, Medicine)
Co-organizers: Mario Biagioli and Alexandra Lippman
REGISTRATION:
The conference is open to the public. Please register here. Although attendance will be on first-come first-serve basis, we will reserve seats for out-of-town participants.
LOCATION:
The conference will be held at two different locations on the UC Davis campus. On Thursday, February 4 we will convene at the MUII Hall at the Memorial Union (map here). On Friday, February 5 our proceedings will take place in the Kalmanovitz Appellate Courtroom at King Hall (UC Davis Law School) (located here).
CAMPUS MAP: Can be found at http://campusmap.ucdavis.edu/
ADDITIONAL INFORMATION: Please see our alippman)
Jonathan Eisen's Lab 