Skip to content

Commit ee261ee

Browse files
author
Kirk Hansen
committed
Use pathlib.full_match for pattern matching
Allows full directory matches and recursive patterns to succeed. Copies in some python 3.13 code for earlier python compatibility
1 parent 561aa04 commit ee261ee

File tree

11 files changed

+243
-26
lines changed

11 files changed

+243
-26
lines changed

README.rst

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ ignoring all directory events:
6262
.. code-block:: bash
6363
6464
watchmedo log \
65-
--patterns='*.py;*.txt' \
65+
--patterns='**/*.py;**/*.txt' \
6666
--ignore-directories \
6767
--recursive \
6868
--verbose \
@@ -74,7 +74,7 @@ response to events:
7474
.. code-block:: bash
7575
7676
watchmedo shell-command \
77-
--patterns='*.py;*.txt' \
77+
--patterns='**/*.py;**/*.txt' \
7878
--recursive \
7979
--command='echo "${watch_src_path}"' \
8080
.
@@ -101,9 +101,9 @@ An example ``tricks.yaml`` file:
101101
102102
tricks:
103103
- watchdog.tricks.LoggerTrick:
104-
patterns: ["*.py", "*.js"]
104+
patterns: ["**/*.py", "**/*.js"]
105105
- watchmedo_webtricks.GoogleClosureTrick:
106-
patterns: ['*.js']
106+
patterns: ['**/*.js']
107107
hash_names: true
108108
mappings_format: json # json|yaml|python
109109
mappings_module: app/javascript_mappings

THIRD_PARTY_LICENSES.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Third Party Licenses
2+
3+
## Python Standard Library Compatibility Code
4+
5+
This project includes the following unmodified functions from the Python 3.13 standard library:
6+
7+
- `glob.translate` (from `Lib/glob.py`)
8+
- `fnmatch._translate` (from `Lib/fnmatch.py`)
9+
10+
These are included in `backwards_compat.py` to provide backwards compatibility with older Python versions.
11+
12+
**License**: Python Software Foundation License Version 2
13+
**Copyright**: © 2001–2024 Python Software Foundation; All Rights Reserved
14+
**Source**: https://github.com/python/cpython
15+
16+
---
17+
18+
### Python Software Foundation License Version 2
19+
20+
1. This LICENSE AGREEMENT is between the Python Software Foundation ("PSF"), and the Individual or Organization ("Licensee") accessing and otherwise using this software ("Python") in source or binary form and its associated documentation.
21+
22+
2. Subject to the terms and conditions of this License Agreement, PSF hereby grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee.
23+
24+
3. In the event Licensee prepares a derivative work that is based on or incorporates Python or any part thereof, and wants to make the derivative work available to others as provided herein, then Licensee hereby agrees to include in any such work a brief summary of the changes made to Python.
25+
26+
4. PSF is making Python available to Licensee on an "AS IS" basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT INFRINGE ANY THIRD PARTY RIGHTS.
27+
28+
5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON, OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
29+
30+
6. This License Agreement will automatically terminate upon a material breach of its terms and conditions.
31+
32+
7. Nothing in this License Agreement shall be deemed to create any relationship of agency, partnership, or joint venture between PSF and Licensee. This License Agreement does not grant permission to use PSF trademarks or trade name in a trademark sense to endorse or promote products or services of Licensee, or any third party.
33+
34+
8. By copying, installing or otherwise using Python, Licensee agrees to be bound by the terms and conditions of this License Agreement.

changelog.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,10 @@ Changelog
1111
- Adjust ``Observer.schedule()`` ``path`` type annotation to reflect the ``pathlib.Path`` support. (`#1096 <https://github.com/gorakhargosh/watchdog/pull/1096>`__)
1212
- Thanks to our beloved contributors: @BoboTiG, @tybug
1313

14+
**Breaking Changes**
15+
16+
- Fix #798 by changing pattern matching from using `path.match` to `path.full_match` Users must update patterns to glob like syntax. E.g., `*.py` to `**/*.py`.
17+
1418
6.0.0
1519
~~~~~
1620

docs/source/examples/patterns.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,9 @@ def on_any_event(self, event: FileSystemEvent) -> None:
1313
logging.debug(event)
1414

1515

16-
event_handler = MyEventHandler(patterns=["*.py", "*.pyc"], ignore_patterns=["version.py"], ignore_directories=True)
16+
event_handler = MyEventHandler(
17+
patterns=["**/*.py", "**/*.pyc"], ignore_patterns=["version.py"], ignore_directories=True
18+
)
1719
observer = Observer()
1820
observer.schedule(event_handler, sys.argv[1], recursive=True)
1921
observer.start()

docs/source/examples/tricks.json

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@
22
{
33
"watchdog.tricks.LoggerTrick": {
44
"patterns": [
5-
"*.py",
6-
"*.js"
5+
"**/*.py",
6+
"**/*.js"
77
]
88
}
99
},
@@ -22,7 +22,7 @@
2222
"suffix": ".min.js",
2323
"source_directory": "app/static/js/",
2424
"hash_names": true,
25-
"patterns": ["*.js"],
25+
"patterns": ["**/*.js"],
2626
"destination_directory": "app/public/js/",
2727
"compilation_level": "advanced",
2828
"mappings_module": "app/javascript_mappings.json"
Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
# ruff: noqa
2+
# fmt: off
3+
"""
4+
This file includes unmodified functions copied from Python 3.13's standard library
5+
for use on older Python versions.
6+
7+
Functions copied:
8+
- glob.translate (from Lib/glob.py)
9+
- fnmatch._translate (from Lib/fnmatch.py)
10+
11+
Source: https://github.com/python/cpython
12+
License: Python Software Foundation License Version 2
13+
Copyright (c) 2001-2024 Python Software Foundation; All Rights Reserved
14+
15+
Please delete me if/when this project releases forcing python >= 3.13
16+
"""
17+
18+
import os
19+
import re
20+
21+
22+
# Copied from python 3.13 fnmatch._translate
23+
def _translate(pat, STAR, QUESTION_MARK):
24+
res = []
25+
add = res.append
26+
i, n = 0, len(pat)
27+
while i < n:
28+
c = pat[i]
29+
i = i+1
30+
if c == '*':
31+
# compress consecutive `*` into one
32+
if (not res) or res[-1] is not STAR:
33+
add(STAR)
34+
elif c == '?':
35+
add(QUESTION_MARK)
36+
elif c == '[':
37+
j = i
38+
if j < n and pat[j] == '!':
39+
j = j+1
40+
if j < n and pat[j] == ']':
41+
j = j+1
42+
while j < n and pat[j] != ']':
43+
j = j+1
44+
if j >= n:
45+
add('\\[')
46+
else:
47+
stuff = pat[i:j]
48+
if '-' not in stuff:
49+
stuff = stuff.replace('\\', r'\\')
50+
else:
51+
chunks = []
52+
k = i+2 if pat[i] == '!' else i+1
53+
while True:
54+
k = pat.find('-', k, j)
55+
if k < 0:
56+
break
57+
chunks.append(pat[i:k])
58+
i = k+1
59+
k = k+3
60+
chunk = pat[i:j]
61+
if chunk:
62+
chunks.append(chunk)
63+
else:
64+
chunks[-1] += '-'
65+
# Remove empty ranges -- invalid in RE.
66+
for k in range(len(chunks)-1, 0, -1):
67+
if chunks[k-1][-1] > chunks[k][0]:
68+
chunks[k-1] = chunks[k-1][:-1] + chunks[k][1:]
69+
del chunks[k]
70+
# Escape backslashes and hyphens for set difference (--).
71+
# Hyphens that create ranges shouldn't be escaped.
72+
stuff = '-'.join(s.replace('\\', r'\\').replace('-', r'\-')
73+
for s in chunks)
74+
# Escape set operations (&&, ~~ and ||).
75+
stuff = re.sub(r'([&~|])', r'\\\1', stuff)
76+
i = j+1
77+
if not stuff:
78+
# Empty range: never match.
79+
add('(?!)')
80+
elif stuff == '!':
81+
# Negated empty range: match any character.
82+
add('.')
83+
else:
84+
if stuff[0] == '!':
85+
stuff = '^' + stuff[1:]
86+
elif stuff[0] in ('^', '['):
87+
stuff = '\\' + stuff
88+
add(f'[{stuff}]')
89+
else:
90+
add(re.escape(c))
91+
assert i == n
92+
return res
93+
94+
95+
def translate(pat, *, recursive=False, include_hidden=False, seps=None):
96+
"""Translate a pathname with shell wildcards to a regular expression.
97+
98+
If `recursive` is true, the pattern segment '**' will match any number of
99+
path segments.
100+
101+
If `include_hidden` is true, wildcards can match path segments beginning
102+
with a dot ('.').
103+
104+
If a sequence of separator characters is given to `seps`, they will be
105+
used to split the pattern into segments and match path separators. If not
106+
given, os.path.sep and os.path.altsep (where available) are used.
107+
"""
108+
if not seps:
109+
if os.path.altsep:
110+
seps = (os.path.sep, os.path.altsep)
111+
else:
112+
seps = os.path.sep
113+
escaped_seps = ''.join(map(re.escape, seps))
114+
any_sep = f'[{escaped_seps}]' if len(seps) > 1 else escaped_seps
115+
not_sep = f'[^{escaped_seps}]'
116+
if include_hidden:
117+
one_last_segment = f'{not_sep}+'
118+
one_segment = f'{one_last_segment}{any_sep}'
119+
any_segments = f'(?:.+{any_sep})?'
120+
any_last_segments = '.*'
121+
else:
122+
one_last_segment = f'[^{escaped_seps}.]{not_sep}*'
123+
one_segment = f'{one_last_segment}{any_sep}'
124+
any_segments = f'(?:{one_segment})*'
125+
any_last_segments = f'{any_segments}(?:{one_last_segment})?'
126+
127+
results = []
128+
parts = re.split(any_sep, pat)
129+
last_part_idx = len(parts) - 1
130+
for idx, part in enumerate(parts):
131+
if part == '*':
132+
results.append(one_segment if idx < last_part_idx else one_last_segment)
133+
elif recursive and part == '**':
134+
if idx < last_part_idx:
135+
if parts[idx + 1] != '**':
136+
results.append(any_segments)
137+
else:
138+
results.append(any_last_segments)
139+
else:
140+
if part:
141+
if not include_hidden and part[0] in '*?':
142+
results.append(r'(?!\.)')
143+
results.extend(_translate(part, f'{not_sep}*', not_sep))
144+
if idx < last_part_idx:
145+
results.append(any_sep)
146+
res = ''.join(results)
147+
return fr'(?s:{res})\Z'

src/watchdog/utils/patterns.py

Lines changed: 33 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,13 +14,40 @@
1414
# - `PureWindowsPath` is always case-insensitive.
1515
# - `PurePosixPath` is always case-sensitive.
1616
# Reference: https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.match
17-
from pathlib import PurePosixPath, PureWindowsPath
17+
import re
18+
from pathlib import PurePath, PurePosixPath, PureWindowsPath
1819
from typing import TYPE_CHECKING
1920

21+
from watchdog.utils.backwards_compat import translate
22+
2023
if TYPE_CHECKING:
2124
from collections.abc import Iterator
2225

2326

27+
def _get_sep(path: PurePath) -> str:
28+
"""
29+
Python < 3.13 doesn't have a clean way to expose the path separator
30+
It's either this, or make use of `path._flavour.sep`
31+
"""
32+
if isinstance(path, PureWindowsPath):
33+
return "\\"
34+
if isinstance(path, PurePosixPath):
35+
return "/"
36+
raise TypeError("Unsupported")
37+
38+
39+
def _full_match(path: PurePath, pattern):
40+
try:
41+
return path.full_match(pattern)
42+
except AttributeError:
43+
# Replicate for python <3.13
44+
# Please remove this, backwards_compat.py, and python license attributions
45+
# if/when we can pin a release to python >= 3.13
46+
regex = translate(pattern, recursive=True, include_hidden=True, seps=_get_sep(path))
47+
reobj = re.compile(regex)
48+
return reobj.match(str(path))
49+
50+
2451
def _match_path(
2552
raw_path: str,
2653
included_patterns: set[str],
@@ -42,7 +69,9 @@ def _match_path(
4269
error = f"conflicting patterns `{common_patterns}` included and excluded"
4370
raise ValueError(error)
4471

45-
return any(path.match(p) for p in included_patterns) and not any(path.match(p) for p in excluded_patterns)
72+
return any(_full_match(path, p) for p in included_patterns) and not any(
73+
_full_match(path, p) for p in excluded_patterns
74+
)
4675

4776

4877
def filter_paths(
@@ -59,7 +88,7 @@ def filter_paths(
5988
ignored patterns.
6089
:param included_patterns:
6190
Allow filenames matching wildcard patterns specified in this list.
62-
If no pattern list is specified, ["*"] is used as the default pattern,
91+
If no pattern list is specified, ["**"] is used as the default pattern,
6392
which matches all files.
6493
:param excluded_patterns:
6594
Ignores filenames matching wildcard patterns specified in this list.
@@ -70,7 +99,7 @@ def filter_paths(
7099
A list of pathnames that matched the allowable patterns and passed
71100
through the ignored patterns.
72101
"""
73-
included = set(["*"] if included_patterns is None else included_patterns)
102+
included = set(["**"] if included_patterns is None else included_patterns)
74103
excluded = set([] if excluded_patterns is None else excluded_patterns)
75104

76105
for path in paths:

tests/test_0_watchmedo.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -324,7 +324,7 @@ def test_tricks_from_file(command, tmp_path):
324324
"""
325325
tricks:
326326
- watchdog.tricks.LoggerTrick:
327-
patterns: ["*.py", "*.js"]
327+
patterns: ["**/*.py", "**/*.js"]
328328
"""
329329
)
330330
args = watchmedo.cli.parse_args([command, str(tricks_file)])

tests/test_fsevents.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -273,7 +273,7 @@ def done(self):
273273

274274
cwd = os.getcwd()
275275
os.chdir(p())
276-
event_handler = TestEventHandler(patterns=["*.json"], ignore_patterns=[], ignore_directories=True)
276+
event_handler = TestEventHandler(patterns=["**/*.json"], ignore_patterns=[], ignore_directories=True)
277277
observer = Observer()
278278
observer.schedule(event_handler, ".")
279279
observer.start()

tests/test_pattern_matching_event_handler.py

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -19,25 +19,25 @@
1919

2020
path_1 = "/path/xyz"
2121
path_2 = "/path/abc"
22-
g_allowed_patterns = ["*.py", "*.txt"]
23-
g_ignore_patterns = ["*.foo"]
22+
g_allowed_patterns = ["**/*.py", "**/*.txt"]
23+
g_ignore_patterns = ["**/*.foo"]
2424

2525

2626
def assert_patterns(event):
2727
paths = [event.src_path, event.dest_path] if hasattr(event, "dest_path") else [event.src_path]
2828
filtered_paths = filter_paths(
2929
paths,
30-
included_patterns=["*.py", "*.txt"],
31-
excluded_patterns=["*.pyc"],
30+
included_patterns=["**/*.py", "**/*.txt"],
31+
excluded_patterns=["**/*.pyc"],
3232
case_sensitive=False,
3333
)
3434
assert filtered_paths
3535

3636

3737
def test_dispatch():
3838
# Utilities.
39-
patterns = ["*.py", "*.txt"]
40-
ignore_patterns = ["*.pyc"]
39+
patterns = ["**/*.py", "**/*.txt"]
40+
ignore_patterns = ["**/*.pyc"]
4141

4242
dir_del_event_match = DirDeletedEvent("/path/blah.py")
4343
dir_del_event_not_match = DirDeletedEvent("/path/foobar")

0 commit comments

Comments
 (0)