Journals can have a publishing status in addition to being hidden from the press

docs/md/journal-visibility-and-publishing-status.md

@@ -0,0 +1,58 @@
+# Journal visibility and publishing status
+
+Janeway has several mechanisms for allowing editors and press managers to control content visibility and filter data streams for each journal.
+
+The following Journal fields provide the configuration options:
+
+- `Journal.hide_from_press`
+- `Journal.status`
+
+## What are some example use cases for these configuration options?
+
+Users may find it tricky to distinguish `hide_from_press` and the "Test" publishing status. But they have to remain distinct for a few use cases. Here are some.
+
+### Not hidden from press
+
+* Active: publishing normally
+* Archived: no longer publishing but have a backlist
+* Coming soon: planning to start publishing soon
+* Test: not applicable, as users should always hide test journals from the press
+
+### Hidden from press
+
+* Active: publishing normally but do not wish to be listed on the publisher site
+* Archived: no longer published by the press but back content is still available at the journal site
+* Coming soon: planning to start publishing soon but want to avoid appearing on the press site
+* Test: testing options and training editors
+
+## What areas are affected by these configuration options?
+
+The `hide_from_press` field does exactly what it says--it puts up a wall between the journal and press, preventing records like `Journal`, like `Issue`, `Section`, `Article`, and `NewsItem` from showing up on press level websites and APIs.
+
+The "Test" publishing status prevents users from accidentally sending data to places it is difficult to remove, like DOI registration. It does not interfere with anything else, including sitemaps, APIs, or RSS feeds, because these too are features that users would want to test at the journal level. This is why it is important for test journals to have `hide_from_press` turned on.
+
+### User interfaces
+
+| Area                                      | Hide from press   | Test status |
+|-------------------------------------------|-------------------|-------------|
+| Lists of journals on press website        | Does what it says | No effect   |
+| Journal submission in repository system   | No effect         | Prevented   |
+| Publications list on public user profiles | Does what it says | Not listed  |
+| Back-office menus that list journals      | No effect         | No effect   |
+| Django admin menus that list journals     | No effect         | No effect   |
+| Reporting (plugin)                        | Does what it says | No effect   |
+
+### Data feeds and alternate user interfaces
+
+| Area                       | Hide from press   | Test status                       |
+|----------------------------|-------------------|-----------------------------------|
+| sitemaps                   | Does what it says | No effect                         |
+| APIs                       | Does what it says | No effect                         |
+| RSS/Atom feed              | Does what it says | No effect                         |
+| reader notifications       | Not applicable    | No effect                         |
+| Crossref deposits          | Not applicable    | Deposits use Crossref test server |
+| Datacite deposits (plugin) | Not applicable    | Deposits use Datacite test server |
+| Galley healthcheck command | Not applicable    | Articles ignored                  |
+| DOI check command          | Not applicable    | Articles ignored                  |
+| Store ithenticate command  | Not applicable    | Articles ignored                  |
+| Metrics core app           | Does what it says | Articles excluded                 |

src/api/oai/views.py

@@ -50,7 +50,6 @@ class OAIListRecords(OAIPagedModelView):
 
     def get_queryset(self):
         queryset = super().get_queryset()
-
         if self.request.journal:
             queryset = queryset.filter(journal=self.request.journal)
         else:
@@ -217,6 +216,10 @@ def get_context_data(self, *args, **kwargs):
                 journal=self.request.journal,
                 stage=submission_models.STAGE_PUBLISHED,
             )
+        else:
+            articles = articles.filter(
+                journal__hide_from_press=False,
+            )
 
         context["earliest_article"] = articles.earliest("date_published")
         context["verb"] = self.request.GET.get("verb")
@@ -247,6 +250,16 @@ def get_context_data(self, *args, **kwargs):
             sections = sections.filter(
                 journal=self.request.journal,
             )
+        else:
+            journals = journals.filter(
+                hide_from_press=False,
+            )
+            all_issues = all_issues.filter(
+                journal__hide_from_press=False,
+            )
+            sections = sections.filter(
+                journal__hide_from_press=False,
+            )
 
         context["journals"] = journals
         context["all_issues"] = all_issues

src/api/tests/test_api.py

@@ -1,17 +1,21 @@
 from django.test import TestCase, override_settings
 from django.urls import reverse
+from django.utils import timezone
 
 from rest_framework.test import APIClient
 
 from utils.testing import helpers
 from core import models as core_models
+from submission import models as submission_models
 
 
 class TestAPI(TestCase):
     @classmethod
     def setUpTestData(cls):
         cls.press = helpers.create_press()
-        cls.journal, _ = helpers.create_journals()
+        cls.journal, cls.hidden_journal = helpers.create_journals()
+        cls.hidden_journal.hide_from_press = True
+        cls.hidden_journal.save()
         cls.staff_member = helpers.create_user(
             username="[email protected]",
             roles=["author"],
@@ -34,6 +38,14 @@ def setUpTestData(cls):
         helpers.create_roles(
             ["journal-manager"],
         )
+        cls.hidden_article = helpers.create_article(
+            cls.hidden_journal,
+            with_author=True,
+            title="Article in journal hidden from press",
+        )
+        cls.hidden_article.stage = submission_models.STAGE_PUBLISHED
+        cls.hidden_article.date_published = timezone.now()
+        cls.hidden_article.save()
         cls.api_client = APIClient()
 
     @override_settings(URL_CONFIG="domain")
@@ -89,3 +101,39 @@ def test_editor_cannot_assign_journal_manager_role(self):
                 journal_manager_role,
             )
         )
+
+    @override_settings(URL_CONFIG="domain")
+    def test_press_api_excludes_journal_hidden_from_press(self):
+        url = self.press.site_url(reverse("journal-list"))
+        response = self.api_client.get(
+            url,
+            SERVER_NAME=self.press.domain,
+        )
+        self.assertNotIn(
+            self.hidden_journal.pk,
+            [journal["pk"] for journal in response.json().get("results", [])],
+        )
+
+    @override_settings(URL_CONFIG="domain")
+    def test_press_api_excludes_article_in_journal_hidden_from_press(self):
+        url = self.press.site_url(reverse("article-list"))
+        response = self.api_client.get(
+            url,
+            SERVER_NAME=self.press.domain,
+        )
+        self.assertNotIn(
+            self.hidden_article.title,
+            [article["title"] for article in response.json().get("results", [])],
+        )
+
+    @override_settings(URL_CONFIG="domain")
+    def test_api_works_at_journal_level_even_if_hidden_from_press(self):
+        url = self.hidden_journal.site_url(reverse("article-list"))
+        response = self.api_client.get(
+            url,
+            SERVER_NAME=self.hidden_journal.domain,
+        )
+        self.assertIn(
+            self.hidden_article.title,
+            [article["title"] for article in response.json().get("results", [])],
+        )

src/api/tests/test_oai.py

@@ -60,7 +60,9 @@ def assertXMLEqual(self, a, b):
     @classmethod
     def setUpTestData(cls):
         cls.press = helpers.create_press()
-        cls.journal, _ = helpers.create_journals()
+        cls.journal, cls.hidden_journal = helpers.create_journals()
+        cls.hidden_journal.hide_from_press = True
+        cls.hidden_journal.save()
         cls.author = helpers.create_author(cls.journal)
         cls.article = helpers.create_submission(
             journal_id=cls.journal.pk,
@@ -86,6 +88,13 @@ def setUpTestData(cls):
         )
         cls.article.primary_issue = cls.issue
         cls.article.save()
+        cls.hidden_article = helpers.create_article(
+            cls.hidden_journal,
+            with_author=True,
+            stage=sm_models.STAGE_PUBLISHED,
+            title="Article in journal hidden from press",
+            date_published="1986-07-12T17:00:00.000+0200",
+        )
 
     @classmethod
     def validate_oai_schema(cls, xml):
@@ -96,7 +105,9 @@ def validate_oai_schema(cls, xml):
     @override_settings(URL_CONFIG="domain")
     @freeze_time(FROZEN_DATETIME_2012)
     def test_list_records_dc(self):
-        expected = LIST_RECORDS_DATA_DC
+        expected = LIST_RECORDS_DATA_DC.format(
+            article_id=self.article.pk,
+        )
         response = self.client.get(
             reverse("OAI_list_records"), SERVER_NAME="testserver"
         )
@@ -105,7 +116,9 @@ def test_list_records_dc(self):
     @override_settings(URL_CONFIG="domain")
     @freeze_time(FROZEN_DATETIME_2012)
     def test_list_records_jats(self):
-        expected = LIST_RECORDS_DATA_JATS
+        expected = LIST_RECORDS_DATA_JATS.format(
+            article_id=self.article.pk,
+        )
         path = reverse("OAI_list_records")
         query_params = dict(
             verb="ListRecords",
@@ -118,13 +131,15 @@ def test_list_records_jats(self):
     @override_settings(URL_CONFIG="domain")
     @freeze_time(FROZEN_DATETIME_2012)
     def test_get_record_dc(self):
-        expected = GET_RECORD_DATA_DC
+        expected = GET_RECORD_DATA_DC.format(
+            article_id=self.article.pk,
+        )
 
         path = reverse("OAI_list_records")
         query_params = dict(
             verb="GetRecord",
             metadataPrefix="oai_dc",
-            identifier="oai:TST:id:1",
+            identifier=f"oai:TST:id:{self.article.pk}",
         )
         query_string = urlencode(query_params)
 
@@ -134,8 +149,6 @@ def test_get_record_dc(self):
     @override_settings(URL_CONFIG="domain")
     @freeze_time(FROZEN_DATETIME_1976)
     def test_get_records_until(self):
-        expected = GET_RECORD_DATA_UNTIL
-
         path = reverse("OAI_list_records")
         query_params = dict(
             verb="ListRecords",
@@ -145,13 +158,18 @@ def test_get_records_until(self):
         query_string = urlencode(query_params)
 
         # Create article that will be returned
-        helpers.create_submission(
+        returned_article = helpers.create_submission(
+            title="Returned article",
             journal_id=self.journal.pk,
             stage=sm_models.STAGE_PUBLISHED,
             date_published="1975-01-01T17:00:00.000+0200",
             authors=[self.author],
         )
 
+        expected = GET_RECORD_DATA_UNTIL.format(
+            article_id=returned_article.pk,
+        )
+
         # Create article that will not be returned
         helpers.create_submission(
             journal_id=self.journal.pk,
@@ -165,7 +183,9 @@ def test_get_records_until(self):
     @override_settings(URL_CONFIG="domain")
     @freeze_time(FROZEN_DATETIME_2012)
     def test_get_record_jats(self):
-        expected = GET_RECORD_DATA_JATS
+        expected = GET_RECORD_DATA_JATS.format(
+            article_id=self.article.pk,
+        )
         # Add a non correspondence author
         author_2 = helpers.create_author(self.journal, email="[email protected]")
         author_2.snapshot_as_author(self.article)
@@ -180,7 +200,7 @@ def test_get_record_jats(self):
         query_params = dict(
             verb="GetRecord",
             metadataPrefix="jats",
-            identifier="oai:TST:id:1",
+            identifier=f"oai:TST:id:{self.article.pk}",
         )
         query_string = urlencode(query_params)
 
@@ -190,7 +210,9 @@ def test_get_record_jats(self):
     @override_settings(URL_CONFIG="domain")
     @freeze_time(FROZEN_DATETIME_2012)
     def test_list_identifiers_jats(self):
-        expected = LIST_IDENTIFIERS_JATS
+        expected = LIST_IDENTIFIERS_JATS.format(
+            article_id=self.article.pk,
+        )
 
         path = reverse("OAI_list_records")
         query_params = dict(
@@ -297,3 +319,33 @@ def test_oai_resumption_token_encode(self):
             expected_encoded in unquote_plus(response.context["resumption_token"]),
             "Query parameter has not been encoded into resumption_token",
         )
+
+    @override_settings(URL_CONFIG="domain")
+    @freeze_time(FROZEN_DATETIME_2012)
+    def test_list_records_jats_excludes_hidden_journal(self):
+        path = self.press.site_url(reverse("OAI_list_records"))
+        query_params = dict(
+            verb="ListRecords",
+            metadataPrefix="jats",
+        )
+        query_string = urlencode(query_params)
+        response = self.client.get(
+            f"{path}?{query_string}",
+            SERVER_NAME=self.press.domain,
+        )
+        self.assertNotIn(self.hidden_article, response.context["object_list"])
+
+    @override_settings(URL_CONFIG="domain")
+    @freeze_time(FROZEN_DATETIME_2012)
+    def test_list_records_jats_works_at_journal_level_even_if_hidden_from_press(self):
+        path = self.hidden_journal.site_url(reverse("OAI_list_records"))
+        query_params = dict(
+            verb="ListRecords",
+            metadataPrefix="jats",
+        )
+        query_string = urlencode(query_params)
+        response = self.client.get(
+            f"{path}?{query_string}",
+            SERVER_NAME=self.hidden_journal.domain,
+        )
+        self.assertIn(self.hidden_article, response.context["object_list"])

src/api/tests/test_oai_data/get_record_data_dc.xml

@@ -10,7 +10,7 @@
   <GetRecord>
     <record>
       <header>
-        <identifier>oai:TST:id:1</identifier>
+        <identifier>oai:TST:id:{article_id}</identifier>
         <datestamp>1986-07-12T15:00:00Z</datestamp>
       </header>
       <metadata>
@@ -30,8 +30,8 @@
           <dc:issue>1</dc:issue>
           <dc:publisher>Press</dc:publisher>
           <dc:journalTitle>Journal One</dc:journalTitle>
-          <dc:identifier>http://testserver/article/id/1/</dc:identifier>
-          <dc:fullTextUrl>http://testserver/article/id/1/</dc:fullTextUrl>
+          <dc:identifier>http://testserver/article/id/{article_id}/</dc:identifier>
+          <dc:fullTextUrl>http://testserver/article/id/{article_id}/</dc:fullTextUrl>
           <dc:source>0000-0000</dc:source>
           <dc:format.extent>1</dc:format.extent>
         </oai_dc:dc>

src/api/tests/test_oai_data/get_record_data_jats.xml

@@ -9,7 +9,7 @@
   <GetRecord>
     <record>
       <header>
-        <identifier>oai:TST:id:1</identifier>
+        <identifier>oai:TST:id:{article_id}</identifier>
         <datestamp>1986-07-12T15:00:00Z</datestamp>
       </header>
       <metadata>
@@ -34,7 +34,7 @@
               </publisher>
             </journal-meta>
             <article-meta>
-              <article-id pub-id-type="publisher-id">1</article-id>
+              <article-id pub-id-type="publisher-id">{article_id}</article-id>
               <article-categories>
                 <subj-group>
                   <subject>Article</subject>
@@ -96,14 +96,14 @@
               <issue>1</issue>
               <issue-id>1</issue-id>
               <issue-title xml:lang="en">Test Issue from Utils Testing Helpers</issue-title>
-              <elocation-id>1</elocation-id>
+              <elocation-id>{article_id}</elocation-id>
               <permissions>
                 <copyright-statement>Copyright: © 1986 The Author(s)</copyright-statement>
                 <copyright-year>1986</copyright-year>
               </permissions>
               <self-uri
                 content-type="text/html"
-                xlink:href="http://testserver/article/id/1/"
+                xlink:href="http://testserver/article/id/{article_id}/"
               />
               <abstract>
                 <p>A Test article abstract</p>