Minimum Reviews Required for a Submission

[Devika008]

Specs Update as of Thursday 27th February, 2025

Workflows affected by this change

• In Dashboard > Editorial Activity, the "Add Reviewer" button should be displayed only when no reviewers are assigned to the submission. However, if at least one reviewer is assigned, but the Minimum confirmed reviews required have not been met, the button will be removed, and the submission will still appear under the "Needs Reviewers" view.
• In Dashboard > Editorial Activity, a new status should be added when the Minimum confirmed reviews required have been confirmed by the editor, indicating: "Minimum required number of reviews have been confirmed. A decision is needed."
• - [ ] In Dashboard > View > Submission Workflow page, a new status should be displayed ** when the minimum number of reviews are confirmed by the editor", indicating: "Minimum number of reviews have been confirmed. A decision is needed."
• In Settings > Workflow > Review Setup, a new setting should be added, allowing the Admin, Journal Manager, and Journal Editor to define the Minimum number of confirmed reviews required for a submission.
• In Dashboard > View > Submission Workflow page, a new status should be displayed at all times, indicating: "Minimum number of confirmed reviews required: {number}"
• If the minimum number of confirmed reviews has not been met and the editor attempts to take an editorial decision—"Create a New Round," "Request Revisions," or "Accept Submission"—an alert should be displayed stating: "The minimum number of confirmed reviews has not been met. Do you still want to proceed with this editorial decision?"
  If the editor selects "Yes, Continue," the user journey for these editorial decisions will follow the existing process from OJS 3.5 without changes.

Results from the user research

Minimum Reviews vs. Minimum Reviewers Required

Question: Which option makes the most sense?

• Option 1: Minimum Number of Reviewers Only → 27 respondents
• Option 2: Minimum Required Reviews Only → 59 respondents ✅ (Most preferred)
• Option 3: Hybrid Approach (Combination of the two) → 47 respondents
• No response → 14 respondents

📌 Key Insight: The majority of respondents (40%) preferred Option 2, as it provides flexibility in setting required reviews per submission.

Key Takeaways:

✅ Editors prefer tracking completed reviews rather than just the number of assigned reviewers.
✅ Flexibility is important – Several respondents mentioned that editors should be able to adjust the required number of reviews at the submission level.
✅ The "Assign Reviewers" button should remain visible until the required reviews are completed, rather than disappearing when a single reviewer is assigned. (I believe this will make the dashboard visually cluttered when applied to a single submission and could lead to information overload when displayed alongside all other review statuses.)
✅ Concerns about complexity – A few respondents suggested avoiding excessive configuration, as most journals use a fixed number of reviewers per submission.

Description

On the new submission dashboard, there's a navigation option called "Needs reviewers," which currently sorts submissions with no reviewers assigned. However, once a single reviewer is assigned, the submission moves out of this category, and no further indication appears on the dashboard to assign additional reviewers. Editors can still go into the workflow to add more reviewers if needed.

What if we introduced a "minimum number of reviewers" setting?
With this setting, editors could define a minimum number of reviewers required for all submissions in the journal. The "assign reviewers" indication would remain visible on the dashboard until the set number of reviewers is assigned, even if some reviews are already completed and confirmed.

The key question to consider:

1. If the minimum required reviewers haven't been met but some reviews are completed, should the editor be able to move the submission forward to the next round?

This scenario introduces some complexities that we might need to address.

Tagging @asmecher @Vitaliy-1 @jardakotesovec on this issue. Please continue tagging any stakeholders that you think can provide more insight on this situation

[alexxxmendonca]

I fully support this!

ScholarOne has the exact same feature and it works really well with our journals.

If the minimum required reviewers haven't been met but some reviews are completed, should the editor be able to move the submission forward to the next round?

Yes. In ScholarOne, there is a minimum required reviews (and not reviewers) setting that is set in the journal configuration, but this can be changed on a submission basis. So the editor can change to less or more than the minimum required reviews. In some cases they can change it to "0" and the submission would move to the next stage without any review needed (usually used for revisions when there is no need to send the paper back to reviewers, or in case of book reviews or editorials).

[asmecher]

@alexxxmendonca, I think there are potentially two different requirements in discussion:

• The number of reviewers an editor should assign before the submission moves out of the "needs reviewers" category, which is what @Devika008 is discussing, and
• What you're proposing, which is setting a number of completed reviews that a submission should receive before it comes back to an Editor for next steps, which is what you're describing. OJS doesn't currently support a number (e.g. "3 completed reviews and we'll let the editor know"), it expects all reviews to be completed and will show that status in the submission list:
  

[alexxxmendonca]

@asmecher I am not opposed to allowing editors defining the number of required reviewers and/or reviews, it's just that based on our experience with ScholarOne, the latter is enough.

We know that assigning 10 reviewers does not mean that 10 reviews will be received, so what we see is editors assigning as many reviewers as they want and the system would notify them when the minimum number of reviews received is reached. The editor will decide what to do next (making a decision or increasing the number of required reviews and thus making the review round longer).

I understand OJS curently does not support that, but it would allow notifications and reminders concerning reviews.