Skip to content

Commit d85be50

Browse files
author
Kirk Hansen
committed
Use pathlib.full_match for pattern matching
Allows full directory matches and recursive patterns to succeed. Copies in some python 3.13 code for earlier python compatibility
1 parent 561aa04 commit d85be50

File tree

12 files changed

+249
-27
lines changed

12 files changed

+249
-27
lines changed

README.rst

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ ignoring all directory events:
6262
.. code-block:: bash
6363
6464
watchmedo log \
65-
--patterns='*.py;*.txt' \
65+
--patterns='**/*.py;**/*.txt' \
6666
--ignore-directories \
6767
--recursive \
6868
--verbose \
@@ -74,7 +74,7 @@ response to events:
7474
.. code-block:: bash
7575
7676
watchmedo shell-command \
77-
--patterns='*.py;*.txt' \
77+
--patterns='**/*.py;**/*.txt' \
7878
--recursive \
7979
--command='echo "${watch_src_path}"' \
8080
.
@@ -101,9 +101,9 @@ An example ``tricks.yaml`` file:
101101
102102
tricks:
103103
- watchdog.tricks.LoggerTrick:
104-
patterns: ["*.py", "*.js"]
104+
patterns: ["**/*.py", "**/*.js"]
105105
- watchmedo_webtricks.GoogleClosureTrick:
106-
patterns: ['*.js']
106+
patterns: ['**/*.js']
107107
hash_names: true
108108
mappings_format: json # json|yaml|python
109109
mappings_module: app/javascript_mappings

THIRD_PARTY_LICENSES.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Third Party Licenses
2+
3+
## Python Standard Library Compatibility Code
4+
5+
This project includes the following unmodified functions from the Python 3.13 standard library:
6+
7+
- `glob.translate` (from `Lib/glob.py`)
8+
- `fnmatch._translate` (from `Lib/fnmatch.py`)
9+
10+
These are included in `backwards_compat.py` to provide backwards compatibility with older Python versions.
11+
12+
**License**: Python Software Foundation License Version 2
13+
**Copyright**: © 2001–2024 Python Software Foundation; All Rights Reserved
14+
**Source**: https://github.com/python/cpython
15+
16+
---
17+
18+
### Python Software Foundation License Version 2
19+
20+
1. This LICENSE AGREEMENT is between the Python Software Foundation ("PSF"), and the Individual or Organization ("Licensee") accessing and otherwise using this software ("Python") in source or binary form and its associated documentation.
21+
22+
2. Subject to the terms and conditions of this License Agreement, PSF hereby grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee.
23+
24+
3. In the event Licensee prepares a derivative work that is based on or incorporates Python or any part thereof, and wants to make the derivative work available to others as provided herein, then Licensee hereby agrees to include in any such work a brief summary of the changes made to Python.
25+
26+
4. PSF is making Python available to Licensee on an "AS IS" basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT INFRINGE ANY THIRD PARTY RIGHTS.
27+
28+
5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON, OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
29+
30+
6. This License Agreement will automatically terminate upon a material breach of its terms and conditions.
31+
32+
7. Nothing in this License Agreement shall be deemed to create any relationship of agency, partnership, or joint venture between PSF and Licensee. This License Agreement does not grant permission to use PSF trademarks or trade name in a trademark sense to endorse or promote products or services of Licensee, or any third party.
33+
34+
8. By copying, installing or otherwise using Python, Licensee agrees to be bound by the terms and conditions of this License Agreement.

changelog.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,10 @@ Changelog
1111
- Adjust ``Observer.schedule()`` ``path`` type annotation to reflect the ``pathlib.Path`` support. (`#1096 <https://github.com/gorakhargosh/watchdog/pull/1096>`__)
1212
- Thanks to our beloved contributors: @BoboTiG, @tybug
1313

14+
**Breaking Changes**
15+
16+
- Fix #798 by changing pattern matching from using `path.match` to `path.full_match` Users must update patterns to glob like syntax. E.g., `*.py` to `**/*.py`.
17+
1418
6.0.0
1519
~~~~~
1620

docs/source/examples/patterns.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,9 @@ def on_any_event(self, event: FileSystemEvent) -> None:
1313
logging.debug(event)
1414

1515

16-
event_handler = MyEventHandler(patterns=["*.py", "*.pyc"], ignore_patterns=["version.py"], ignore_directories=True)
16+
event_handler = MyEventHandler(
17+
patterns=["**/*.py", "**/*.pyc"], ignore_patterns=["version.py"], ignore_directories=True
18+
)
1719
observer = Observer()
1820
observer.schedule(event_handler, sys.argv[1], recursive=True)
1921
observer.start()

docs/source/examples/tricks.json

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@
22
{
33
"watchdog.tricks.LoggerTrick": {
44
"patterns": [
5-
"*.py",
6-
"*.js"
5+
"**/*.py",
6+
"**/*.js"
77
]
88
}
99
},
@@ -22,7 +22,7 @@
2222
"suffix": ".min.js",
2323
"source_directory": "app/static/js/",
2424
"hash_names": true,
25-
"patterns": ["*.js"],
25+
"patterns": ["**/*.js"],
2626
"destination_directory": "app/public/js/",
2727
"compilation_level": "advanced",
2828
"mappings_module": "app/javascript_mappings.json"

src/watchdog/observers/fsevents.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,6 @@
1212
import threading
1313
import time
1414
import unicodedata
15-
from pathlib import Path
1615
from typing import TYPE_CHECKING
1716

1817
import _watchdog_fsevents as _fsevents
@@ -33,6 +32,8 @@
3332
from watchdog.utils.dirsnapshot import DirectorySnapshot
3433

3534
if TYPE_CHECKING:
35+
from pathlib import Path
36+
3637
from watchdog.events import FileSystemEvent, FileSystemEventHandler
3738
from watchdog.observers.api import EventQueue, ObservedWatch
3839

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
# ruff: noqa
2+
# fmt: off
3+
# type: ignore
4+
"""
5+
This file includes unmodified functions copied from Python 3.13's standard library
6+
for use on older Python versions.
7+
8+
Functions copied:
9+
- glob.translate (from Lib/glob.py)
10+
- fnmatch._translate (from Lib/fnmatch.py)
11+
12+
Source: https://github.com/python/cpython
13+
License: Python Software Foundation License Version 2
14+
Copyright (c) 2001-2024 Python Software Foundation; All Rights Reserved
15+
16+
Please delete me if/when this project releases forcing python >= 3.13
17+
"""
18+
19+
import os
20+
import re
21+
22+
23+
# Copied from python 3.13 fnmatch._translate
24+
def _translate(pat, STAR, QUESTION_MARK):
25+
res = []
26+
add = res.append
27+
i, n = 0, len(pat)
28+
while i < n:
29+
c = pat[i]
30+
i = i+1
31+
if c == '*':
32+
# compress consecutive `*` into one
33+
if (not res) or res[-1] is not STAR:
34+
add(STAR)
35+
elif c == '?':
36+
add(QUESTION_MARK)
37+
elif c == '[':
38+
j = i
39+
if j < n and pat[j] == '!':
40+
j = j+1
41+
if j < n and pat[j] == ']':
42+
j = j+1
43+
while j < n and pat[j] != ']':
44+
j = j+1
45+
if j >= n:
46+
add('\\[')
47+
else:
48+
stuff = pat[i:j]
49+
if '-' not in stuff:
50+
stuff = stuff.replace('\\', r'\\')
51+
else:
52+
chunks = []
53+
k = i+2 if pat[i] == '!' else i+1
54+
while True:
55+
k = pat.find('-', k, j)
56+
if k < 0:
57+
break
58+
chunks.append(pat[i:k])
59+
i = k+1
60+
k = k+3
61+
chunk = pat[i:j]
62+
if chunk:
63+
chunks.append(chunk)
64+
else:
65+
chunks[-1] += '-'
66+
# Remove empty ranges -- invalid in RE.
67+
for k in range(len(chunks)-1, 0, -1):
68+
if chunks[k-1][-1] > chunks[k][0]:
69+
chunks[k-1] = chunks[k-1][:-1] + chunks[k][1:]
70+
del chunks[k]
71+
# Escape backslashes and hyphens for set difference (--).
72+
# Hyphens that create ranges shouldn't be escaped.
73+
stuff = '-'.join(s.replace('\\', r'\\').replace('-', r'\-')
74+
for s in chunks)
75+
# Escape set operations (&&, ~~ and ||).
76+
stuff = re.sub(r'([&~|])', r'\\\1', stuff)
77+
i = j+1
78+
if not stuff:
79+
# Empty range: never match.
80+
add('(?!)')
81+
elif stuff == '!':
82+
# Negated empty range: match any character.
83+
add('.')
84+
else:
85+
if stuff[0] == '!':
86+
stuff = '^' + stuff[1:]
87+
elif stuff[0] in ('^', '['):
88+
stuff = '\\' + stuff
89+
add(f'[{stuff}]')
90+
else:
91+
add(re.escape(c))
92+
assert i == n
93+
return res
94+
95+
96+
def translate(pat, *, recursive=False, include_hidden=False, seps=None):
97+
"""Translate a pathname with shell wildcards to a regular expression.
98+
99+
If `recursive` is true, the pattern segment '**' will match any number of
100+
path segments.
101+
102+
If `include_hidden` is true, wildcards can match path segments beginning
103+
with a dot ('.').
104+
105+
If a sequence of separator characters is given to `seps`, they will be
106+
used to split the pattern into segments and match path separators. If not
107+
given, os.path.sep and os.path.altsep (where available) are used.
108+
"""
109+
if not seps:
110+
if os.path.altsep:
111+
seps = (os.path.sep, os.path.altsep)
112+
else:
113+
seps = os.path.sep
114+
escaped_seps = ''.join(map(re.escape, seps))
115+
any_sep = f'[{escaped_seps}]' if len(seps) > 1 else escaped_seps
116+
not_sep = f'[^{escaped_seps}]'
117+
if include_hidden:
118+
one_last_segment = f'{not_sep}+'
119+
one_segment = f'{one_last_segment}{any_sep}'
120+
any_segments = f'(?:.+{any_sep})?'
121+
any_last_segments = '.*'
122+
else:
123+
one_last_segment = f'[^{escaped_seps}.]{not_sep}*'
124+
one_segment = f'{one_last_segment}{any_sep}'
125+
any_segments = f'(?:{one_segment})*'
126+
any_last_segments = f'{any_segments}(?:{one_last_segment})?'
127+
128+
results = []
129+
parts = re.split(any_sep, pat)
130+
last_part_idx = len(parts) - 1
131+
for idx, part in enumerate(parts):
132+
if part == '*':
133+
results.append(one_segment if idx < last_part_idx else one_last_segment)
134+
elif recursive and part == '**':
135+
if idx < last_part_idx:
136+
if parts[idx + 1] != '**':
137+
results.append(any_segments)
138+
else:
139+
results.append(any_last_segments)
140+
else:
141+
if part:
142+
if not include_hidden and part[0] in '*?':
143+
results.append(r'(?!\.)')
144+
results.extend(_translate(part, f'{not_sep}*', not_sep))
145+
if idx < last_part_idx:
146+
results.append(any_sep)
147+
res = ''.join(results)
148+
return fr'(?s:{res})\Z'

src/watchdog/utils/patterns.py

Lines changed: 36 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,13 +14,43 @@
1414
# - `PureWindowsPath` is always case-insensitive.
1515
# - `PurePosixPath` is always case-sensitive.
1616
# Reference: https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.match
17-
from pathlib import PurePosixPath, PureWindowsPath
17+
import re
18+
from pathlib import PurePath, PurePosixPath, PureWindowsPath
1819
from typing import TYPE_CHECKING
1920

21+
from watchdog.utils.backwards_compat import translate # type: ignore[attr-defined]
22+
2023
if TYPE_CHECKING:
2124
from collections.abc import Iterator
2225

2326

27+
def _get_sep(path: PurePath) -> str:
28+
"""
29+
Python < 3.13 doesn't have a clean way to expose the path separator
30+
It's either this, or make use of `path._flavour.sep`
31+
"""
32+
if isinstance(path, PureWindowsPath):
33+
return "\\"
34+
if isinstance(path, PurePosixPath):
35+
return "/"
36+
raise TypeError("Unsupported")
37+
38+
39+
def _full_match(path: PurePath, pattern: str) -> bool:
40+
try:
41+
return path.full_match(pattern)
42+
except AttributeError:
43+
# Replicate for python <3.13
44+
# Please remove this, backwards_compat.py, and python license attributions
45+
# if/when we can pin a release to python >= 3.13
46+
# Construct a pathlib object using the same class as the path to get the
47+
# same pattern path separater when constructing the regex
48+
normalized_pattern = str(type(path)(pattern))
49+
regex = translate(normalized_pattern, recursive=True, include_hidden=True, seps=_get_sep(path))
50+
reobj = re.compile(regex)
51+
return bool(reobj.match(str(path)))
52+
53+
2454
def _match_path(
2555
raw_path: str,
2656
included_patterns: set[str],
@@ -42,7 +72,9 @@ def _match_path(
4272
error = f"conflicting patterns `{common_patterns}` included and excluded"
4373
raise ValueError(error)
4474

45-
return any(path.match(p) for p in included_patterns) and not any(path.match(p) for p in excluded_patterns)
75+
return any(_full_match(path, p) for p in included_patterns) and not any(
76+
_full_match(path, p) for p in excluded_patterns
77+
)
4678

4779

4880
def filter_paths(
@@ -59,7 +91,7 @@ def filter_paths(
5991
ignored patterns.
6092
:param included_patterns:
6193
Allow filenames matching wildcard patterns specified in this list.
62-
If no pattern list is specified, ["*"] is used as the default pattern,
94+
If no pattern list is specified, ["**"] is used as the default pattern,
6395
which matches all files.
6496
:param excluded_patterns:
6597
Ignores filenames matching wildcard patterns specified in this list.
@@ -70,7 +102,7 @@ def filter_paths(
70102
A list of pathnames that matched the allowable patterns and passed
71103
through the ignored patterns.
72104
"""
73-
included = set(["*"] if included_patterns is None else included_patterns)
105+
included = set(["**"] if included_patterns is None else included_patterns)
74106
excluded = set([] if excluded_patterns is None else excluded_patterns)
75107

76108
for path in paths:

tests/test_0_watchmedo.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -324,7 +324,7 @@ def test_tricks_from_file(command, tmp_path):
324324
"""
325325
tricks:
326326
- watchdog.tricks.LoggerTrick:
327-
patterns: ["*.py", "*.js"]
327+
patterns: ["**/*.py", "**/*.js"]
328328
"""
329329
)
330330
args = watchmedo.cli.parse_args([command, str(tricks_file)])

tests/test_fsevents.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -273,7 +273,7 @@ def done(self):
273273

274274
cwd = os.getcwd()
275275
os.chdir(p())
276-
event_handler = TestEventHandler(patterns=["*.json"], ignore_patterns=[], ignore_directories=True)
276+
event_handler = TestEventHandler(patterns=["**/*.json"], ignore_patterns=[], ignore_directories=True)
277277
observer = Observer()
278278
observer.schedule(event_handler, ".")
279279
observer.start()

0 commit comments

Comments
 (0)