Tag: Open Access

Well this certainly is interesting: The Open Science Peer Review Oath – F1000Research.  This emerged apparently from the AllBio: Open Science & Reproducibility Best Practice Workshop.  The “Oath” is summarized in the following text from a box in their paper:

Box 1. While reviewing this manuscript:

1. I will sign my review in order to be able to have an open dialogue with you
2. I will be honest at all times
3. I will state my limits
4. I will turn down reviews I am not qualified to provide
5. I will not unduly delay the review process
6. I will not scoop research that I had not planned to do before reading the manuscript
7. I will be constructive in my criticism
8. I will treat reviews as scientific discourses
9. I will encourage discussion, and respond to your and/or editors’ questions
10. I will try to assist in every way I ethically can to provide criticism and praise that is valid, relevant and cognisant of community norms
11. I will encourage the application of any other open science best practices relevant to my field that would support transparency, reproducibility, re-use and integrity of your research
12. If your results contradict earlier findings, I will allow them to stand, provided the methodology is sound and you have discussed them in context
13. I will check that the data, software code and digital object identifiers are correct, and the models presented are archived, referenced, and accessible
14. I will comment on how well you have achieved transparency, in terms of materials and methodology, data and code access, versioning, algorithms, software parameters and standards, such that your experiments can be repeated independently
15. I will encourage deposition with long-term unrestricted access to the data that underpin the published concept, towards transparency and re-use
16. I will encourage central long-term unrestricted access to any software code and support documentation that underpin the published concept, both for reproducibility of results and software availability
17. I will remind myself to adhere to this oath by providing a clear statement and link to it in each review I write, hence helping to perpetuate good practice to the authors whose work I review.

I note – I reformatted the presentation a tiny bit here.   The Roman numerals in the paper annoyed me.  Regardless of the formatting, this is a pretty long oath.  I think it is probably too long.  Some of this could be reduced.  I am reposting the Oath below with some comments:

1. I will sign my review in order to be able to have an open dialogue with you.  I think this is OK to have in the oath. 
2. I will be honest at all times. Seems unnecessary.
3. I will state my limits. Not sure what this means or how it differs from #4.  I would suggest deleting or merging with #4.
4. I will turn down reviews I am not qualified to provide.  This is good though not sure how it differs from #3. 
5. I will not unduly delay the review process. Good. 
6. I will not scoop research that I had not planned to do before reading the manuscript. Good. 
7. I will be constructive in my criticism. Good. 
8. I will treat reviews as scientific discourses.  Not sure what this means or how it is diffeent from #9. 
9. I will encourage discussion, and respond to your and/or editors’ questions.  Good though not sure how it differs from #8. 
10. I will try to assist in every way I ethically can to provide criticism and praise that is valid, relevant and cognisant of community norms. OK though this seems to cancel the need for #7. 
11. I will encourage the application of any other open science best practices relevant to my field that would support transparency, reproducibility, re-use and integrity of your research.  Good.  Seems to cancel the need for #13, #14, #15, #16. 
12. If your results contradict earlier findings, I will allow them to stand, provided the methodology is sound and you have discussed them in context. OK though I am not sure why this raises to the level of a part of the oath over other things that should be part of a review. 
13. I will check that the data, software code and digital object identifiers are correct, and the models presented are archived, referenced, and accessible.  Seems to be covered in #11. 
14. I will comment on how well you have achieved transparency, in terms of materials and methodology, data and code access, versioning, algorithms, software parameters and standards, such that your experiments can be repeated independently. Seems to be covered in #11. 
15. I will encourage deposition with long-term unrestricted access to the data that underpin the published concept, towards transparency and re-use. Seems to be covered in #11. 
16. I will encourage central long-term unrestricted access to any software code and support documentation that underpin the published concept, both for reproducibility of results and software availability. Seems to be covered in #11. 
17. I will remind myself to adhere to this oath by providing a clear statement and link to it in each review I write, hence helping to perpetuate good practice to the authors whose work I review.  Not sure this is needed.

The paper then goes on to provide what they call a manifesto.  I very much prefer the items in the manifesto over those in the oath:

• Principle 1: I will sign my name to my review – I will write under my own name
• Principle 2: I will review with integrity
• Principle 3: I will treat the review as a discourse with you; in particular, I will provide constructive criticism
• Principle 4: I will be an ambassador for good science practice
• Principle 5: Support other reviewers

In fact I propose here that the authors considering reversing the Oath and the Manifesto.  What they call the Manifesto shoud be the Oath.  It is short.  And works as an Oath.  The longer, somewhat repetitive list of specific details would work better as the basis for a Manifesto.

Anyway – the paper is worth taking a look at.  I support the push for more consideration of Open Science in review though I am not sure if this Oath is done right at this point.