What does peer review mean to you?
To some, it means research that is probably funded by industry or government. Many of these people believe the funder influences the quality of the work.
While this clearly happens in some fields, such as the study of particular drugs (most pharmacological research is at a more basic level, to find what and how, not to confirm the latest product), how big a problem is this otherwise? It’s pretty clear that the President isn’t able to influence the outcome of studies, rather he and Congress have most influence on which questions are asked. They can do this through earmarks, more than $2 billion/year just in science, or through the questions asked of National Academy of Sciences or President’s Committee of Advisers on Science and Technology and others. For example, a decade ago, PCAST was asked to report on Federal Energy Research and Development for the Challenges of the 21st Century. The authors recommended that the report be updated every 5 years, but a decade later the update still hasn’t been requested.
We all know stories of company doctors who never found problems with toxins (did they submit articles with that claim to journals?). On the other hand, an article I read long ago (where?) described the US system of government, businesses, and universities working together as the best system for science in the world, as the questions examined at the university level are questions people want answers to. Computer business folks approached universities in the Boston area about collaboration, and were rejected, before settling in Silicon Valley — now the two best colleges for computer science are UC, Berkeley and Stanford.
The Wikipedia description reflects how people in science are likely to see peer review:
Peer review (known as refereeing in some academic fields) is a process of subjecting an author’s scholarly work, research or ideas to the scrutiny of others who are experts in the same field. It is used primarily by editors to select and to screen submitted manuscripts, and by funding agencies, to decide the awarding of grants. The peer review process aims to make authors meet the standards of their discipline, and of science in general. Publications and awards that have not undergone peer review are likely to be regarded with suspicion by scholars and professionals in many fields. Even refereed journals, however, can contain errors….
Reviewers are typically anonymous and independent, to help foster unvarnished criticism, and to discourage cronyism in funding and publication decisions. However, as discussed below under the next section, US government guidelines governing peer review for federal regulatory agencies require that reviewer’s identity be disclosed under some circumstances.
In addition, since reviewers are normally selected from experts in the fields discussed in the article, the process of peer review is considered critical to establishing a reliable body of research and knowledge. Scholars reading the published articles can only be expert in a limited area; they rely, to some degree, on the peer-review process to provide reliable and credible research that they can build upon for subsequent or related research.
Fraud is hard to catch by this method; often it is caught when others do not produce the same results. This is the method of catching mistakes as well; peer review does not confirm accuracy, but over time, mistakes tend to be caught.
For some, the term peer review means a real study, one that is financed by some means, and there could be a problem if the financing interferes with the result. People triggered by this term are put in an awful position, since every study either falls into the category serious and peer-reviewed, or not serious.
What are your feelings about the term and why?
I feel that peer review typically means a study that has been through an evaluation process by those people most able to do exactly that. It is part of the system that science and research has built up to make its results as accurate and focussed as possible.
My feelings about the term are positive. I think that the proper use of peer review acts as a check on the influence of funding – not as a mechanism for bias, but as a mechanism to reduce bias.
I have a generally positive attitude toward peer review also. It is a mechanism to have the science and results evaluated by someone in your field who is competent to judge it, and is essential to maintain quality and standards in scientific fields. However, it is not the reviewer’s job to inquire about the sources of funding, which might or might not be disclosed – so if a paper on a certain source of energy was funded by an oil company, this could have an important bearing on the approach and results of the paper, but might not be obvious. I must add that I do not know if this kind of funding and bias is common, and it may be quite rare – but I suspect not. For instance, it has been disclosed recently that 60% of the heads of medical schools are affiliated with some pharmaceutical company or other.