What is peer review? 5 things you should know before you start researching (2023)

As academics and other expertsrush to launch new researchTo better understand the coronavirus pandemic, newsrooms need to be more careful than ever when vetting the biomedical studies they choose to cover. One of the first steps journalists should take in assessing the quality of any type of investigation is to answer this important question: Has the article been peer reviewed?

Peer review is a formal process through which researchers evaluate and provide feedback on each other's work, ideally filtering out poor-quality and flawed studies while strengthening others. Academic journals generally do not publish articles that have not survived the process. Researchers often share studies that have not been peer-reviewed, often called working papers orpreprints— publish them on servers and online repositories.

The world's largest preprint servers for life sciences are worth a look:bioRxiv— and health sciences —medRxiv— Review of documents for plagiarism and content that is offensive, non-scientific, or that may represent a risk to health or biosafety. But there are prepress servers in other fields that don't apply the same level of scrutiny.

While peer review is intended for quality control, it is imperfect. For example, reviewers, who are typically college professors with experience in the same field of work they are reviewing, sometimes miss fraud, data discrepancies, and other issues. Even some of the most prestigious journals with the most rigorous peer review processes have had to withdraw articles. However, retractions are rare.

“Only about four out of 10,000 newspapers now retract. And while the rate has nearly doubled from 2003 to 2009, it has held steady since 2012."Sciencesmagazinereported in 2018.

At the beginning of May 2021,a total of 108 articles on COVID-19, most of which appeared in newspapers, was withdrawn, according toretraction clock, which maintains an online database of retractions of investigations going back decades.

Despite its flaws, researchers, in general, seem to trust peer review. Duringa 2019 surveyOf more than 3,000 researchers from multiple disciplines in various countries, 85% agreed or strongly agreed that without peer review there is no control in scholarly communication. The survey - conducted byElsevier, one of the largest magazine publishers in the world, andsense of science, a London-based non-profit organization that promotes public interest in science and evidence, also found that 90% of participating researchers agreed or strongly agreed that peer review improves the quality of research. the investigation.

Several published studies present similar findings.A 2017 newspaperinsidepost learnedindicates that early-career researchers "generally support peer review," but complain that the process is time-consuming and that reviewers, who usually work as volunteers, should be rewarded with some form of professional recognition or payment.

Regardless of the type of investigation journalists cover, they should have at least a basic understanding of the peer review process and its advantages and disadvantages.

Here we explain some of the most important aspects with the help of various experts, includingdiane sullenberger, executive editor ofAnnals of the National Academy of Sciences;Miriam Lewis Sabin, senior editor atthe lancet; miJuan Inglis, CEO of Cold Spring Harbor Laboratory Press and co-founder of bioRxiv and medRxiv.

1. Peer reviewers are not fraud detectors. They also do not verify the accuracy of a research study.

The peer review process is meant to validate the research, not verify it. Reviewers generally do not authenticate study data or guarantee that study authors actually followed the procedures they claim to have followed to reach their conclusions. Reviewers, sometimes called arbitrators, also do not determine whether findings are correct given the data and other evidence used to reach them.

Reviewers review scholarly papers to answer a variety of relevant questions. They look at whether the research questions are clear, for example, and whether the study design, sampling methods, and analysis are appropriate for answering those questions. They also assess whether the article answers questions such as:

  • Is the study explained clearly and in enough detail so that another researcher can replicate it?
  • How does the study challenge or add to the body of knowledge on this topic?
  • Does it meet the standards and scope of the journal to which it was submitted?
  • If the study involves humans or animals, did the authors obtain the necessary approvals and meet ethical standards?
  • Do you give due attribution to prior research?

When the German theologianHenry Oldenburgcreated the first magazine dedicated to science in 1665, consideredThe main functions of a research journal.being registration, certification, diffusion and archiving, he writesRoberto Campbell, senior editor at Wiley-Blackwell Publishing, on the bookAcademic and professional publications..

Peer review is considered the gold standard for evaluating research content, Sullenberger explained in an email interview. But journalists need to understand that it is not foolproof, he added.

“Science is self-correcting through replication and reproducibility, and research fraud can be difficult to detect in peer review,” he wrote.

2. Journalists can help the public recognize the value of peer review by looking at whether the studies they cover have been peer reviewed.

Academics, research organizations, and others regularly criticize the media for failing to explain whether the new research they report or the older studies they incorporate into their stories have been peer-reviewed. It's important for journalists to differentiate between peer-reviewed research and preprint articles, which often present preliminary findings.

sullenberger thesejr: “More clarity is needed when journalists cover unreviewed preprints; they should not be reported as having the same validity and authority as peer-reviewed research articles. 🇧🇷

A recent studyin the newspaperHealth Communicationfinds that many of the news articles written about COVID-related preprints during the first four months of 2020 did not indicate the scientific uncertainty of this research. About 43% of the stories analyzed did not mention that the investigation was preprint, unreviewed, preliminary, or in need of verification.

However, at the time of this study, many of the journalists drawn to reporting on the pandemic story frenzy were unfamiliar with preprints, Inglis says. Today, she adds, journalists covering the coronavirus are much more likely to include phrases like "not yet peer-reviewed" to describe preprints.

Sense About Science urges the public to pay attention to whether a study being discussed at a government meeting or in the media has been peer reviewed. "The more we ask, 'Is it peer reviewed?', the more journalists will be forced to include this information," the organization said.states in a brochureCreated to help the public examine the scientific information presented in the news.

Knowing whether research has been peer reviewed helps a person judge how much weight to give the claims made by its authors.Tracey Brown, the managing director of Sense About Science, explained duringan interviewcomthe academic kitchenBlog.

“We have to establish an understanding that the state of the research results is just as important as the results themselves,” Brown says ina prepared statement🇧🇷 "That understanding has the ability to improve the decisions we make throughout society."

3. Reviewers help decide the fate of a study.

Journal editors typically assign two or more reviewers to each research article. Some also employ a statistician.

While the selection process differs, journals select reviewers based on factors such as the journal's experience, reputation, and prior experience with the reviewer. While it can be difficult to recruit scientists willing to review manuscripts because of the time required for proper scrutiny, many do so out of "a sense of duty to help advance their disciplines, as well as a need for reciprocity, knowing that other researchers care." volunteer." . for peer review of your manuscript submissions,”Sciencesmagazinereported earlier this year.

Reviewers can make recommendations as to whether a journal should accept, reject, or return an article for minor or major revision. Reviewers often submit reports that provide their overall impressions of an article and suggestions for improving it. However, more often than not, the final decision rests with one or more of the journal's editors or their editorial board.

Inglis, former deputy editor ofthe lancetwho is now editor of five peer-reviewed journals, says a common criticism of the peer-review process is its long timeframe, which can last anywhere from weeks to a year or more. Another complaint: Journals sometimes return a study and notify the authors that they would be willing to accept or reconsider the article for publication if the authors do more research.

“Sometimes the demands that are made are completely unrealistic,” adds Inglis. “The criticism from the authors is that the editors don't know that when they say 'do this extra experiment,' it's another year [added to the timeline]. In the meantime, the work is perfectly worth it."

Inglis says bioRxiv (pronounced "bio-archive") and medRxiv (pronounced "med-archive") were created so that researchers could publish draft versions of their papers, allowing the scientific community to immediately use and begin developing these findings and data. .

4. The peer review process varies significantly among academic journals.

There are several types of peer review, and journals often indicate on their websites which one they use. The most common are single-blind peer review, which allows the reviewers to know the identities of the authors while the identities of the reviewers remain anonymous, and double-blind peer review, in which the authors and reviewers are blinded to the identity of the reviewers. identities of others.

Both have advantages. The lawyers argue that anonymity protects reviewers from retaliation. It also helps protect authors from bias based on factors such as gender, nationality, language, and affiliations with less prestigious institutions.Tony Ross Hellauer, postdoctoral researcher at theknowledge centerin Austria, he writes in “What is open peer review? A systematic review”, published on the European open access platformF1000Pesquisaem 2017.

However, keeping identities secret can create problems.

“At the editorial level, a lack of transparency means that editors can unilaterally reject submissions or shape review results by selecting reviewers based on their known preference or dislike for certain theories and methods,” Ross-Hellauer writes. . She adds that reviewers, "protected by anonymity, may act unethically in their own best interest, hiding conflicts of interest."

A new type of peer review, calledopen peer reviewIt's not that frequent. But the scientific community has ongoing discussions about whether its increased transparency can help improve the quality of research.

While there is no universally accepted definition of open peer review, also known as open identity peer review, the identities of the authors and reviewers are generally known to each other. Ross-Hellauer notes that disclosing reviewer names may force reviewers to "think more carefully about scientific issues and write more thoughtful reviews."

More and more journals publish not only the articles they accept, but also the comments that reviewers have given to the authors of the articles.

5. Peer review continues to evolve.

Some journals have begun to initiate peer review after an article is published rather than before, although this is not yet common.MedEdPublish, an online scholarly journal, is one of those that usepost-publication peer review🇧🇷 Your articles are peer-reviewed on the website by members of the medical education community, which may include the journal's editor, members of its editorial board, or a review panel.

According to the MedEdPublish template, an article has undergone formal peer review after it has been assessed by at least two members of the journal's review panel. The article can be critiqued and improved over time as a living document on the journal's website.

“Post-publication peer review follows an open and transparent process, the goal of which is to avoid editorial bias and increase publication speed,” the website says. “We use an 'open identities' principle whereby all reviewers submit their comments publicly, under their own name, and everyone who visits an article's page can see all peer review reports, referee names and comments, and can participate in the discussion if they wish. . 🇧🇷

Another notable change: Some journals are working to diversify their review groups, making sure that women, racial and ethnic minorities, and scientists from other countries help evaluate and select studies for publication.

Research indicates that the vast majority of experts chosen as reviewers are men. Astudy published earlier this yearinsideadvances in scienceexamines internal data from 145 academic journals across all fields and finds that women made up 21% of their reviewers between 2010 and 2016. In journals dedicated to biomedical and health research, 24.6% of reviewers were women.

the lancetmedical journalset goalsfor increasing the number of women and scientists from low- and middle-income countries, Sabin, one of its senior editors, wrote in an email interview withjr. em 2019,the lancetnewspaper family announced itsDiversity Commitment.

“We track, monitor, and report the representation of authors, reviewers, and editorial consultants by genre and geography,” Sabin said.jrin an email.

He added that the journal formed a task force late last year to, among other things, examine its policies and processes to find ways to increase the representation of experts who are racial and ethnic minorities.

oCoalition for Diversity and Inclusion in Scholarly Communicationsfocused on the global issue.More than 90 organizationsadopted the coalitionJoint Statement of Principles, whose goal is to “promote engagement, innovation, and greater access to leadership opportunities that maximize engagement across identity groups and professional levels.”

Identity groups include racial and sexual minorities, military veterans, pregnant women, parents, and people from lower social classes and socioeconomic levels.

The journalist's resourceI would like to thank Rick Weiss, SciLine Director, and Meredith Drosback, SciLine Associate Director of Science, for their help in creating this cheat sheet.

Top Articles
Latest Posts
Article information

Author: Eusebia Nader

Last Updated: 01/30/2023

Views: 5785

Rating: 5 / 5 (80 voted)

Reviews: 87% of readers found this page helpful

Author information

Name: Eusebia Nader

Birthday: 1994-11-11

Address: Apt. 721 977 Ebert Meadows, Jereville, GA 73618-6603

Phone: +2316203969400

Job: International Farming Consultant

Hobby: Reading, Photography, Shooting, Singing, Magic, Kayaking, Mushroom hunting

Introduction: My name is Eusebia Nader, I am a encouraging, brainy, lively, nice, famous, healthy, clever person who loves writing and wants to share my knowledge and understanding with you.