Improvements in regular expression doc #114357

adorilson · 2024-01-20T21:21:40Z

📚 Documentation preview 📚: https://cpython-previews--114357.org.readthedocs.build/

The check about the f argument type was removed in this commit: python@2c94aa5 Thanks for Pedro Arthur Duarte (pedroarthur.jedi at gmail.com) by the help with this bug.

…#106335) Remove private _PyThreadState and _PyInterpreterState C API functions: move them to the internal C API (pycore_pystate.h and pycore_interp.h). Don't export most of these functions anymore, but still export functions used by tests. Remove _PyThreadState_Prealloc() and _PyThreadState_Init() from the C API, but keep it in the stable API.

…paragraph

terryjreedy

This PR does 3 things.

Add headers. I have thought to propose the same. Please add 1 more at 320, something like

.. _re_extension_notation

Extension notation
^^^^^^^^^^^^^^^^^^

CHANGE

Add double backticks, either new or extending single backticks. The existing text always put backticks on REs and sometimes on text matched. PR makes that (nearly, 2 expections noted) always on matches. Defensible since this seems the majority of existing cases. CHANGE
Add 'only' in several places. I am not sure these are needed, but I see existing similar uses.

@serhiy-storchaka I want to finish this RE doc change. Any additional comments from you?

terryjreedy · 2024-02-25T04:17:23Z

Doc/library/re.rst

   only 'foo'.  More interestingly, searching for ``foo.$`` in ``'foo1\nfoo2\n'``
-   matches 'foo2' normally, but 'foo1' in :const:`MULTILINE` mode; searching for
+   matches 'foo2' normally, but ``'foo1'`` in :const:`MULTILINE` mode; searching


To be consistent with other additions, 'foo' above and 'foo2' here should be backticked. But see review summary.

bedevere-app · 2024-02-25T04:23:16Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

serhiy-storchaka · 2024-02-25T12:31:34Z

I am not sure that there is a need in these changes.

New headers and anchors. It is perhaps harmless, but there are some problems in the text (re: documentation claim that special characters lose their special meaning inside […] seems wrong #106482) which requires more serious rewriting, so some parts of the text can be moved and headers and anchors can change.
I used double backquotes to highlight regular expressions. Single quotes are used for strings, they are not fragments of the Python code, they are just strings in quotes. If use the same style in both cases, it will be more difficult to distinguish REs from strings. Maybe you have better solution?
I have no opinion about "only", I left the decision on the native English users.

adorilson · 2024-09-26T08:54:13Z

Hi, @terryjreedy. Thank you for your review and comments.

The items 1 and 2 are done.

Concern 3: the idea is to make the re.ASCII use more explicit.

Without re.ASCII, [^0-9] is matched, but something more can be matched too, i.e.:

>>> import re
>>> re.findall(r'\d+', '567abc123٠١٢٣٤٥٦٧٨٩')
['567', '123٠١٢٣٤٥٦٧٨٩']

However, with re.ASCII only (and just only) [^0-9] is matched, i.e.:

>>> import re
>>> re.findall(r'\d+', '567abc123٠١٢٣٤٥٦٧٨٩', re.ASCII)
['567', '123']

adorilson · 2024-09-26T08:57:18Z

requires more serious rewriting

This can start adding more in-line examples, like in progress with strings (#119445).

bedevere-app · 2024-09-26T08:58:02Z

Thanks for making the requested changes!

@terryjreedy: please review the changes made to this pull request.

…labels

vadmium

I appreciate the subheadings to divide up the syntax section.

I agree with adding only to clarify when ASCII mode matches less than previously described. But the cases with complemented sets don’t have this problem, and I think adding only to them only hurts.

\D:
Matches any character which is not a decimal digit. This is the opposite of \d.

Matches only [^0–9] if the ASCII flag is used.

reads as “Matches only the universe except zero to nine”?

vadmium · 2024-10-16T08:27:07Z

Doc/library/re.rst

@@ -514,6 +529,9 @@ The special characters are:

 .. _re-special-sequences:

+Special sequences
+^^^^^^^^^^^^^^^^^


Can we call them all escape sequences? Differentiates better from the multi-character “special character” sequences above.

Are there other subtypes of special characters? What if about subsections?

I suggest change the next heading to something like String literal escapes, and change this heading from Special sequences to Escape sequences.

These are the types of the special characters I can think of for REs:

The single-character metacharacters: $, *, [, ], \, etc, as listed in the how-to https://cpython-previews--114357.org.readthedocs.build/en/114357/howto/regex.html#matching-characters

Multicharacter syntax built with the metacharacters, like *?, {m,n} and the bracketed extension notation (?. . .)

“Special sequences” a.k.a. escape sequences, which begin with a backslash. These could be subdivided into

Non-alphanumeric, for escaping metacharacters and other syntax: \$, \*, \\, \', \", etc

Group references \1–\99

Alphanumeric sequences that specify locations to match, or categories of characters: \A, \b, \d, etc

String literal escapes: \n, \\, \N{. . .}, \0–\777, etc. Excludes \b and \<newline>.

Characters only special in “verbose” expressions: whitespace and #

Additional backslash sequence for re.sub templates: \g<. . .>

Special characters inside square-bracketed classes/sets [. . .], especially -, ^, ], \b, and reserved [, &&, etc

Doc/library/re.rst

vadmium · 2024-10-16T08:52:29Z

Doc/library/re.rst

+   matches both ``'foo'`` and ``'foobar'``, while the regular expression ``foo$``
+   matches
+   only ``'foo'``.  More interestingly, searching for ``foo.$`` in ``'foo1\nfoo2\n'``
+   matches ``'foo2'`` normally, but also ``'foo1'`` in :const:`MULTILINE` mode; searching


I thought the original was easier to read, with the full string being searched given in a different font from the substrings that are found

Firstly, it was inconsistent with the "(In the rest of this section, we’ll write RE’s in this special style, usually without quotes, and strings to be matched 'in single quotes'.)"

However, you highlighted that 'strings to be matched' is different from 'the matches'. On the other hand, both are literal strings, and this is a common pattern around all docs.

I would like some more opinions here.

vadmium · 2024-10-16T08:59:45Z

Doc/library/re.rst

-   many repetitions as are possible.  ``ab*`` will match 'a', 'ab', or 'a' followed
-   by any number of 'b's.
+   many repetitions as are possible.  ``ab*`` will match ``'a'``, ``'ab'``, or
+   ``'a'`` followed by any number of ``'b'`` s.


The -s signifying plural has become disconnected from the b

Waiting decision about: #114357 (comment)

Not that I think it is great formatting, but looking at the markup under *+ you might join the s on with

Suggested change

``'a'`` followed by any number of ``'b'`` s.

``'a'`` followed by any number of ``'b'``\ s.

vadmium · 2024-10-16T09:00:18Z

Doc/library/re.rst


 .. index:: single: + (plus); in regular expressions

 ``+``
   Causes the resulting RE to match 1 or more repetitions of the preceding RE.
-   ``ab+`` will match 'a' followed by any non-zero number of 'b's; it will not
-   match just 'a'.
+   ``ab+`` will match ``'a'`` followed by any non-zero number of ``'b'`` s; it


-s disconnected again

This reverts commit 94f765f.

adorilson · 2024-10-20T21:34:05Z

I agree with adding only to clarify when ASCII mode matches less than previously described. But the cases with complemented sets don’t have this problem, and I think adding only to them only hurts.

I've reverted this change.

\D:
Matches any character which is not a decimal digit. This is the opposite of \d.
Matches only [^0–9] if the ASCII flag is used.

reads as “Matches only the universe except zero to nine”?

HUmm... It is curious. Even without the only, this can sound weird, mainly if you don't read the first paragraph.

Maybe this would be "Matches all the ASCII universe except zero to nine."

adorilson · 2024-10-20T22:45:28Z

I have made the requested changes; please review again

bedevere-app · 2024-10-20T22:45:31Z

Thanks for making the requested changes!

@terryjreedy: please review the changes made to this pull request.

adorilson · 2024-10-20T22:46:31Z

I have made the requested changes; please review again

bedevere-app · 2024-10-20T22:46:34Z

Thanks for making the requested changes!

@terryjreedy: please review the changes made to this pull request.

vadmium · 2024-10-25T06:30:19Z

\D:
Matches any character which is not a decimal digit. This is the opposite of \d.

Matches [^0–9] if the ASCII flag is used.

HUmm... It is curious. Even without the only, this can sound weird, mainly if you don't read the first paragraph.

Maybe this would be "Matches all the ASCII universe except zero to nine."

I don’t understand what is weird. It matches all Unicode characters, not just ASCII, except for ASCII zero to nine. The only suggestion I can think of is saying “Equivalent to [^0–9]” rather than “Matches”. Maybe that is clearer to you? (Although the equivalency doesn’t quite work when \D is already inside a square-bracket character class/set.)

adorilson · 2024-10-28T11:58:50Z

\D: Matches any character which is not a decimal digit. This is the opposite of \d.

Matches [^0–9] if the ASCII flag is used.

HUmm... It is curious. Even without the only, this can sound weird, mainly if you don't read the first paragraph.
Maybe this would be "Matches all the ASCII universe except zero to nine."

I don’t understand what is weird. It matches all Unicode characters, not just ASCII, except for ASCII zero to nine. The only suggestion I can think of is saying “Equivalent to [^0–9]” rather than “Matches”. Maybe that is clearer to you? (Although the equivalency doesn’t quite work when \D is already inside a square-bracket character class/set.)

Oh, my goodness.

I had a misconception about how re.ASCII works. I thought it was like a filter: "filter all ASCII characters and after matching against the re". So, necessarily, with the ASCII flag, the matches only had ASCII characters. But it is not true, especially with the negative set of characters ([^ re ]).

In this case, we might need to improve the re.ASCII definition to avoid this misconception.

adorilson · 2024-10-28T13:32:59Z

The only suggestion I can think of is saying “Equivalent to [^0–9]” rather than “Matches”. Maybe that is clearer to you?

Yeah. With the correct understanding of how ASCII works, it sounds better.

Can I change this in all occurrences like that?

willingc · 2024-11-01T23:16:48Z

Thank you @adorilson for your patience and effort on this PR.

@terryjreedy I'm going through and triaging a bunch of docs PRs. If you have time, please review this one again. Thanks.

adorilson and others added 6 commits January 20, 2024 20:23

Doc: Fix the array.fromfile method doc

817b3f3

The check about the f argument type was removed in this commit: python@2c94aa5 Thanks for Pedro Arthur Duarte (pedroarthur.jedi at gmail.com) by the help with this bug.

[Doc] Divide RE Syntax in subsections

1b4d152

[DOC] Add crasis surrounding some RE-matched words

6ad009c

[DOC] Make clearer what will be matched with a RE

94f765f

Doc: minor change

292672b

bedevere-app bot added awaiting review docs Documentation in the Doc dir skip news labels Jan 20, 2024

adorilson added 6 commits February 3, 2024 22:43

Merge branch 'python:main' into re_improvements

65b4278

Merge branch 'python:main' into re_improvements

fe7389a

Merge branch 'python:main' into re_improvements

8394cd3

Doc: Put PatternError's attributes inside a table instead of regular …

e2023e0

…paragraph

Doc: Fix PatternError's attributes

cdaa9ae

Doc: fix lint issue

bb98dad

terryjreedy requested changes Feb 25, 2024

View reviewed changes

bedevere-app bot removed the awaiting review label Feb 25, 2024

bedevere-app bot added the awaiting changes label Feb 25, 2024

adorilson added 2 commits February 25, 2024 18:32

Merge branch 'main' into re_improvements

22ffed7

Merge branch 'python:main' into re_improvements

6a1e74e

adorilson marked this pull request as draft September 25, 2024 09:45

bedevere-app bot removed the awaiting changes label Sep 25, 2024

adorilson added 4 commits September 25, 2024 17:43

Doc: Add extension notation header

6b357af

Doc: Add some more backticks

8f7356d

Merge branch 'python:main' into re_improvements

6ed5109

Doc: Fix malformed hyperlink target

9c17aa8

bedevere-app bot requested a review from terryjreedy September 26, 2024 08:58

adorilson added 5 commits September 26, 2024 09:58

Merge branch 'main' into re_improvements

acb2e38

Merge branch 'python:main' into re_improvements

4d3b8dd

Merge branch 'main' into re_improvements

643070c

Docs: add a 'also' for $ special character and RE examples reference …

17baf98

…labels

Docs: add some RE raw string notation references

4e12f7c

vadmium reviewed Oct 16, 2024

View reviewed changes

adorilson added 2 commits October 20, 2024 21:20

Merge branch 'python:main' into re_improvements

a09a187

Revert "[DOC] Make clearer what will be matched with a RE"

625a5cf

This reverts commit 94f765f.

adorilson added 2 commits October 20, 2024 23:08

Doc: Put some subheadings at Special Character section

12ecb3a

Doc: Fix raw string notation reference

f576282

adorilson requested a review from vadmium October 20, 2024 22:46

adorilson added 2 commits October 28, 2024 09:26

Merge branch 'python:main' into re_improvements

337e4b4

Doc: Include "Python's" to a link text in RE module

0e0e082

Doc: Add some backticks in re.IGNORECASE section

f094a90

Merge branch 'main' into re_improvements

fd24e0f

willingc added the skip issue label Nov 2, 2024

adorilson added 3 commits November 21, 2024 22:50

Merge branch 'main' into re_improvements

a8c44e1

Doc: rename some heading in RE

f970235

Doc: Connect some s in RE

8d52469

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improvements in regular expression doc #114357

Improvements in regular expression doc #114357

adorilson commented Jan 20, 2024 •

edited by github-actions bot

Loading

terryjreedy left a comment •

edited

Loading

terryjreedy Feb 25, 2024

adorilson Sep 25, 2024

bedevere-app bot commented Feb 25, 2024

serhiy-storchaka commented Feb 25, 2024

adorilson commented Sep 26, 2024

adorilson commented Sep 26, 2024

bedevere-app bot commented Sep 26, 2024

vadmium left a comment •

edited

Loading

vadmium Oct 16, 2024

adorilson Oct 20, 2024

vadmium Oct 25, 2024

vadmium Oct 16, 2024

adorilson Oct 20, 2024

vadmium Oct 16, 2024

adorilson Oct 20, 2024

vadmium Oct 25, 2024

adorilson Mar 15, 2025

vadmium Oct 16, 2024

adorilson Mar 15, 2025

adorilson commented Oct 20, 2024

adorilson commented Oct 20, 2024

bedevere-app bot commented Oct 20, 2024

adorilson commented Oct 20, 2024

bedevere-app bot commented Oct 20, 2024

vadmium commented Oct 25, 2024

adorilson commented Oct 28, 2024

adorilson commented Oct 28, 2024

willingc commented Nov 1, 2024

	``'a'`` followed by any number of ``'b'`` s.
	``'a'`` followed by any number of ``'b'``\ s.

Improvements in regular expression doc #114357

Are you sure you want to change the base?

Improvements in regular expression doc #114357

Conversation

adorilson commented Jan 20, 2024 • edited by github-actions bot Loading

terryjreedy left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bedevere-app bot commented Feb 25, 2024

serhiy-storchaka commented Feb 25, 2024

adorilson commented Sep 26, 2024

adorilson commented Sep 26, 2024

bedevere-app bot commented Sep 26, 2024

vadmium left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adorilson commented Oct 20, 2024

adorilson commented Oct 20, 2024

bedevere-app bot commented Oct 20, 2024

adorilson commented Oct 20, 2024

bedevere-app bot commented Oct 20, 2024

vadmium commented Oct 25, 2024

adorilson commented Oct 28, 2024

adorilson commented Oct 28, 2024

willingc commented Nov 1, 2024

adorilson commented Jan 20, 2024 •

edited by github-actions bot

Loading

terryjreedy left a comment •

edited

Loading

vadmium left a comment •

edited

Loading