Clarify text alignment and break behaviour (#2427)

Author: vkbo

docs/source/usage/advanced_formatting.rst

@@ -56,6 +56,20 @@ activated by clicking the left-most icon button in the editor header.
 .. versionadded:: 2.2
 
 
+.. _docs_usage_formatting_shortcodes_break:
+
+Forced Line Break
+-----------------
+
+Inserting ``[br]`` in the text will ensure a line break is always inserted in that place, even if
+you turn off **Preserve Hard Line Breaks** in your manuscript build settings.
+
+You can add a manual line break after it too, for a better visual representation in the editor, but
+keep in mind that this line break is removed before the text is processed, so the text on either
+side of the ``[br]`` shortcode will be considered as belonging to the same line. This can affect
+how alignment is treated. See :ref:`docs_usage_align_indent_forced` for more details.
+
+
 .. _docs_usage_formatting_breaks:
 
 Vertical Space and Page Breaks

docs/source/usage/alignment_and_indent.rst

@@ -53,18 +53,33 @@ the entire paragraph. For the following text, all lines will be centred:
 
 .. code-block:: md
 
-   >> I am the very model of a modern Major-General
+   >> I am the very model of a modern Major-General <<
    I've information vegetable, animal, and mineral
    I know the kings of England, and I quote the fights historical
-   From Marathon to Waterloo, in order categorical <<
+   From Marathon to Waterloo, in order categorical
+
+If you have multiple conflicting alignments on a paragraph, only one is applied. The order of
+precedence is:
+
+#. Left alignment
+#. Right alignment
+#. Centred text
+#. Justified text
+
+.. note::
+
+   It is strongly recommended that you keep the **Preserve Hard Line Breaks** setting enabled in
+   your manuscript build settings. This setting assumes all single line breaks in your text are
+   intended. Turning this off makes adding line breaks more complicated, but it is still possible.
+   See :ref:`docs_usage_align_indent_forced`.
 
 
 Alignment with First Line Indent
 ================================
 
 If you have first line indent enabled in your manuscript build settings, you probably want to
-disable it for text in verses. Adding any alignment tags will cause the first line indent to be
-switched off for that paragraph.
+disable it for text in verses. Adding any alignment tags on a paragraph will cause the first
+line indent to be switched off for that paragraph.
 
 :bdg-info:`Example`
 
@@ -76,3 +91,35 @@ The following text will always be aligned against the left margin:
    I've information vegetable, animal, and mineral
    I know the kings of England, and I quote the fights historical
    From Marathon to Waterloo, in order categorical
+
+
+.. _docs_usage_align_indent_forced:
+
+Alignment with Forced Line Breaks
+=================================
+
+If you turn off **Preserve Hard Line Breaks** in your manuscript build settings, you can still
+force line breaks in paragraphs using the ``[br]`` shortcode. For clarity in the text, you can add
+a line break after it as well. It doesn't result in two line breaks.
+
+Keep in mind that when the text is processed, the lines on either side of a ``[br]`` shortcode are
+combined, and any trailing hard line break is *ignored*. This means that when such a paragraph is
+processed, these line breaks count as the same line. This affects how alignment tags are handled.
+For instance, this text becomes centred instead of left aligned.
+
+.. code-block:: md
+
+   >> I am the very model of a modern Major-General[br]
+   I've information vegetable, animal, and mineral[br]
+   I know the kings of England, and I quote the fights historical[br]
+   From Marathon to Waterloo, in order categorical <<
+
+Since this is understood as one line, this is the only way you can actually centre this paragraph.
+
+.. caution::
+
+   Due to this difference in how text with ``[br]`` tags are processed, it is generally better to
+   stick with the **Preserve Hard Line Breaks** setting enabled. It ensures a better correspondence
+   between what you see in the editor and what output you get.
+
+See also :ref:`docs_usage_formatting_shortcodes_break`.

novelwriter/formats/tokenizer.py

@@ -880,38 +880,40 @@ def tokenizeText(self) -> None:
                 if nBlock[0] != BlockTyp.TEXT:
                     # Next block is not text, so we add the buffer to blocks
                     nLines = len(pLines)
-                    cStyle = pLines[0][4]
-                    if firstIndent and not (self._noIndent or cStyle & BlockFmt.ALIGNED):
-                        # If paragraph indentation is enabled, not temporarily
-                        # turned off, and the block is not aligned, we add the
-                        # text indentation flag
-                        cStyle |= BlockFmt.IND_T
+                    tFmt: T_Formats = []
+                    pTxt = ""
+                    cStyle = BlockFmt.NONE
 
                     if nLines == 1:
-                        # The paragraph contains a single line, so we just save
-                        # that directly to the blocks list. If justify is
-                        # enabled, and there is no alignment, we apply it.
-                        if doJustify and not cStyle & BlockFmt.ALIGNED:
-                            cStyle |= BlockFmt.JUSTIFY
-
+                        # The paragraph contains a single line
+                        tFmt = pLines[0][3]
                         pTxt = pLines[0][2].translate(transMapB)
-                        sBlocks.append((
-                            BlockTyp.TEXT, pLines[0][1], pTxt, pLines[0][3], cStyle
-                        ))
+                        cStyle = pLines[0][4]
 
                     elif nLines > 1:
                         # The paragraph contains multiple lines, so we need to
                         # join them according to the line break policy, and
                         # recompute all the formatting markers
                         tTxt = ""
-                        tFmt: T_Formats = []
                         for aBlock in pLines:
                             tLen = len(tTxt)
                             tTxt += f"{aBlock[2]}{lineSep}"
                             tFmt.extend((p+tLen, fmt, key) for p, fmt, key in aBlock[3])
                             cStyle |= aBlock[4]
 
                         pTxt = tTxt[:-1].translate(transMapB)
+
+                    if nLines:
+                        isAligned = cStyle & BlockFmt.ALIGNED
+                        if firstIndent and not (self._noIndent or isAligned):
+                            # If paragraph indentation is enabled, not temporarily
+                            # turned off, and the block is not aligned, we add the
+                            # text indentation flag
+                            cStyle |= BlockFmt.IND_T
+
+                        if doJustify and not isAligned:
+                            cStyle |= BlockFmt.JUSTIFY
+
                         sBlocks.append((
                             BlockTyp.TEXT, pLines[0][1], pTxt, tFmt, cStyle
                         ))

tests/test_formats/test_fmt_tohtml.py

@@ -147,9 +147,6 @@ def testFmtToHtml_ConvertParagraphs(mockGUI):
     html._isNovel = True
     html._isFirst = True
 
-    # Paragraphs
-    # ==========
-
     # Text
     html._text = "Some **nested bold and _italic_ and ~~strikethrough~~ text** here\n"
     html.tokenizeText()
@@ -280,6 +277,54 @@ def testFmtToHtml_ConvertParagraphs(mockGUI):
     )
 
 
+@pytest.mark.core
+def testFmtToHtml_Alignment(mockGUI):
+    """Test paragraph alignment in the ToHtml class."""
+    project = NWProject()
+    html = ToHtml(project)
+    html.initDocument()
+
+    # Left
+    html._text = "This is text <<\nspanning multiple\nlines"
+    html.tokenizeText()
+    html.doConvert()
+    assert html._pages[-1] == (
+        "<p style='text-align: left;'>This is text<br>spanning multiple<br>lines</p>\n"
+    )
+
+    # Right
+    html._text = ">> This is text\nspanning multiple\nlines"
+    html.tokenizeText()
+    html.doConvert()
+    assert html._pages[-1] == (
+        "<p style='text-align: right;'>This is text<br>spanning multiple<br>lines</p>\n"
+    )
+
+    # Centre
+    html._text = ">> This is text <<\nspanning multiple\nlines"
+    html.tokenizeText()
+    html.doConvert()
+    assert html._pages[-1] == (
+        "<p style='text-align: center;'>This is text<br>spanning multiple<br>lines</p>\n"
+    )
+
+    # Left before Right
+    html._text = ">> This is text\nspanning multiple <<\nlines"
+    html.tokenizeText()
+    html.doConvert()
+    assert html._pages[-1] == (
+        "<p style='text-align: left;'>This is text<br>spanning multiple<br>lines</p>\n"
+    )
+
+    # Right before Centre
+    html._text = ">> This is text <<\n>> spanning multiple\nlines"
+    html.tokenizeText()
+    html.doConvert()
+    assert html._pages[-1] == (
+        "<p style='text-align: right;'>This is text<br>spanning multiple<br>lines</p>\n"
+    )
+
+
 @pytest.mark.core
 def testFmtToHtml_Dialog(mockGUI):
     """Test paragraph formats in the ToHtml class."""

tests/test_formats/test_fmt_tokenizer.py

@@ -1074,6 +1074,80 @@ def testFmtToken_Paragraphs(mockGUI):
     ]
 
 
+@pytest.mark.core
+def testFmtToken_BreakAlignIndent(mockGUI):
+    """Test the splitting of paragraphs with alignment."""
+    project = NWProject()
+    tokens = BareTokenizer(project)
+    tokens._handle = TMH
+
+    for text in [
+        "This is text <<\nspanning multiple\nlines",
+        "This is text\nspanning multiple <<\nlines",
+        "This is text\nspanning multiple\nlines <<",
+    ]:
+        # Preserve Breaks
+        tokens.setKeepLineBreaks(True)
+        tokens._text = text
+        tokens.tokenizeText()
+        assert tokens._blocks == [
+            (BlockTyp.TEXT, "", "This is text\nspanning multiple\nlines", [], BlockFmt.LEFT),
+        ]
+
+        # Don't Preserve Breaks
+        tokens.setKeepLineBreaks(False)
+        tokens._text = text
+        tokens.tokenizeText()
+        assert tokens._blocks == [
+            (BlockTyp.TEXT, "", "This is text spanning multiple lines", [], BlockFmt.LEFT),
+        ]
+
+        # With Justify
+        # This should disable justify
+        tokens.setKeepLineBreaks(True)
+        tokens.setJustify(True)
+        tokens._text = text
+        tokens.tokenizeText()
+        assert tokens._blocks == [
+            (BlockTyp.TEXT, "", "This is text\nspanning multiple\nlines", [], BlockFmt.LEFT),
+        ]
+
+        # With Indent
+        # This should disable indent
+        tokens.setKeepLineBreaks(True)
+        tokens.setFirstLineIndent(True, 1.0, False)
+        tokens._text = text
+        tokens.tokenizeText()
+        assert tokens._blocks == [
+            (BlockTyp.TEXT, "", "This is text\nspanning multiple\nlines", [], BlockFmt.LEFT),
+        ]
+
+
+@pytest.mark.core
+def testFmtToken_BreakJustify(mockGUI):
+    """Test the of processing of justify with breaks."""
+    project = NWProject()
+    tokens = BareTokenizer(project)
+    tokens._handle = TMH
+    tokens.setJustify(True)
+
+    # Applied to all lines when breaks are preserved
+    tokens._text = "This is text\nspanning multiple\nlines"
+    tokens.setKeepLineBreaks(True)
+    tokens.tokenizeText()
+    assert tokens._blocks == [
+        (BlockTyp.TEXT, "", "This is text\nspanning multiple\nlines", [], BlockFmt.JUSTIFY),
+    ]
+
+    # Turning off breaks should make no difference (see issue #2426)
+    tokens._text = "This is text\nspanning multiple\nlines"
+    tokens.setKeepLineBreaks(False)
+    tokens.tokenizeText()
+    assert tokens._blocks == [
+        (BlockTyp.TEXT, "", "This is text spanning multiple lines", [], BlockFmt.JUSTIFY),
+    ]
+
+
 @pytest.mark.core
 def testFmtToken_TextFormat(mockGUI):
     """Test the tokenization of text formats in the Tokenizer class."""

tests/test_formats/test_fmt_toodt.py

@@ -733,7 +733,7 @@ def getStyle(styleName):
         '<office:text>'
         '<text:h text:style-name="Heading_20_2" text:outline-level="2">Scene</text:h>'
         '<text:p text:style-name="P7">Regular paragraph</text:p>'
-        '<text:p text:style-name="Text_20_body">with<text:line-break />break</text:p>'
+        '<text:p text:style-name="P7">with<text:line-break />break</text:p>'
         '<text:p text:style-name="Text_20_body">Left Align</text:p>'
         '</office:text>'
     )