Skip to content

Commit a01d6fb

Browse files
fguillotfguillot
committed
docs(rules): reorganize and clarify filtering rules section
1 parent 082042e commit a01d6fb

1 file changed

Lines changed: 59 additions & 57 deletions

File tree

‎content/docs/rules.md‎

Lines changed: 59 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -1,50 +1,58 @@
11
---
22
title: Filter, Rewrite, and Scraper Rules
3-
description: How to write custom scraper and rewrite rules
3+
description: Documentation for Miniflux's filtering, rewriting, and scraper rules.
44
url: /docs/rules.html
55
---
66

77
- [Feed Filtering Rules](#feed-filtering-rules)
8-
- [Global Filtering Rules](#global-filtering-rules)
9-
- [Rewrite Rules](#rewrite-rules)
10-
- [Scraper Rules](#scraper-rules)
8+
- [Entry Filtering Rules](#filtering-rules)
9+
- [Content Rewrite Rules](#rewrite-rules)
1110
- [URL Rewrite Rules](#rewriteurl-rules)
11+
- [Scraper Rules](#scraper-rules)
1212

1313
<h2 id="feed-filtering-rules">Feed Filtering Rules <a class="anchor" href="#feed-filtering-rules" title="Permalink">¶</a></h2>
1414

15-
Miniflux has a basic filtering system that allows you to ignore or keep articles.
15+
Miniflux has a regex-based filtering system that allows you to ignore or keep articles.
16+
For more advanced filtering, you can use the [Entry Filtering Rules](#filtering-rules) feature.
1617

17-
### Block Rules
18+
### Regex-Based Blocking Filters
1819

1920
Block rules ignore articles with a title, an entry URL, a tag, or an author that matches the regex ([RE2 syntax](https://golang.org/s/re2syntax)).
2021

2122
For example, the regex `(?i)miniflux` will ignore all articles with a title that contains the word Miniflux (case insensitive).
2223

2324
Ignored articles won't be saved into the database.
2425

25-
### Keep Rules
26+
### Regex-Based Keep Filters
2627

2728
Keep rules retain only articles that match the regex ([RE2 syntax](https://golang.org/s/re2syntax)).
2829

2930
For example, the regex `(?i)miniflux` will keep only the articles with a title that contains the word Miniflux (case insensitive).
3031

31-
<h2 id="global-filtering-rules">Global Filtering Rules <a class="anchor" href="#global-filtering-rules" title="Permalink">¶</a></h2>
32+
<h2 id="filtering-rules">Entry Filtering Rules <a class="anchor" href="#entry-filtering-rules" title="Permalink">¶</a></h2>
3233

33-
Global filters are defined on the Settings page and are automatically applied to all articles from all feeds.
34+
Since Miniflux 2.2.10, filtering rules can be defined for each feed and globally on the Settings page.
3435

35-
- Each rule must be on a separate line.
36-
- Duplicate rules are allowed. For example, having multiple `EntryTitle` rules is possible.
37-
- The provided regex should use the [RE2 syntax](https://golang.org/s/re2syntax).
38-
- The order of the rules matters as the processor stops on the first match for both Block and Keep rules.
36+
There are two types of rules:
37+
- **Block Rules**: Ignore articles that match the regex.
38+
- **Keep Rules**: Retain only articles that match the regex.
3939

40-
Rule Format:
40+
### Rules Format and Syntax
41+
42+
Example:
4143

4244
```
4345
FieldName=RegEx
4446
FieldName=RegEx
4547
FieldName=RegEx
4648
```
4749

50+
- Each rule must be on a separate line.
51+
- Duplicate rules are allowed. For example, having multiple `EntryTitle` rules is possible.
52+
- The provided regex should use the [RE2 syntax](https://golang.org/s/re2syntax).
53+
- The order of the rules matters as the processor stops on the first match for both Block and Keep rules.
54+
- Invalid rules are ignored.
55+
4856
Available Fields:
4957

5058
- `EntryTitle`
@@ -63,35 +71,29 @@ The `EntryDate` field supports the following date patterns:
6371
- `before:YYYY-MM-DD` - Match entries published before a specific date.
6472
- `after:YYYY-MM-DD` - Match entries published after a specific date.
6573
- `between:YYYY-MM-DD,YYYY-MM-DD` - Match entries published between two dates.
74+
- `max-age:duration` - Match entries that are not older than a specific duration (e.g., `max-age:7d` for 7 days). Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h", "d".
6675

67-
Date format must be YYYY-MM-DD, for example: 2024-01-01.
76+
Date format must be `YYYY-MM-DD`, for example: `2024-01-01`.
6877

69-
### Block Rules
78+
Block rules examples:
7079

71-
Block rules ignore articles that match a single rule.
72-
73-
For example, the rule `EntryTitle=(?i)miniflux` will ignore all articles with a title that contains the word Miniflux (case insensitive).
74-
75-
Here are additional examples you can consider:
7680
```regex
77-
EntryDate=future # Ignore articles with future publication dates.
78-
EntryDate=before:2024-01-01 # Ignore articles published before January 1st, 2024.
79-
EntryTitle=(?i)\b(save|take|get)\s+\$\d{2,5}\b # Ignore articles with “Save $50”, “Get $100…” in the title
80-
EntryTitle=(?i)\$\d{2,5}\s+(off|discount)\b # Ignore articles with “$50 off”
81-
EntryTitle=(?i)\bbest\s+.*\bdeals?\b # Ignore articles with “Best Foobar Deals …”
82-
EntryTitle=(?i)\bgift\s+(guide|ideas|list)\b # Ignore articles that look like listicles
81+
EntryDate=future # Ignore articles with future publication dates
82+
EntryDate=before:2024-01-01 # Ignore articles published before January 1st, 2024
83+
EntryDate=max-age:30d # Ignore articles older than 30 days
84+
EntryTitle=(?i)miniflux # Ignore articles with "Miniflux" in the title
85+
EntryTitle=(?i)\b(save|take|get)\s+\$\d{2,5}\b # Ignore articles with "Save $50", "Get $100…" in the title
86+
EntryTitle=(?i)\$\d{2,5}\s+(off|discount)\b # Ignore articles with "$50 off"
87+
EntryTitle=(?i)\bbest\s+.*\bdeals?\b # Ignore articles with "Best Foobar Deals…"
88+
EntryTitle=(?i)\bgift\s+(guide|ideas|list)\b # Ignore articles that look like listicles
8389
```
8490

85-
### Keep Rules
86-
87-
Keep rules retain articles that match a single rule.
91+
Keep rules examples:
8892

89-
For example, the rule `EntryTitle=(?i)miniflux` will keep only the articles with a title that contains the word Miniflux (case insensitive).
90-
91-
Examples:
92-
93-
- `EntryDate=between:2024-01-01,2024-12-31` will keep only articles published in 2024.
94-
- `EntryDate=after:2024-03-01` will keep only articles published after March 1st, 2024.
93+
```regex
94+
EntryDate=between:2024-01-01,2024-12-31 # Keep only articles published in 2024
95+
EntryDate=after:2024-03-01 # Keep only articles published after March 1st, 2024
96+
```
9597

9698
### Global Rules & Feed Rules Ordering
9799

@@ -102,7 +104,7 @@ Rules are processed in this order:
102104
3. Global Keep Rules
103105
4. Feed Keep Rules
104106

105-
<h2 id="rewrite-rules">Rewrite Rules <a class="anchor" href="#rewrite-rules" title="Permalink">¶</a></h2>
107+
<h2 id="rewrite-rules">Content Rewrite Rules <a class="anchor" href="#rewrite-rules" title="Permalink">¶</a></h2>
106108

107109
To improve the reading experience, it's possible to alter the content of feed items.
108110

@@ -175,7 +177,7 @@ especially on mobile devices where there is no `hover` event.
175177
<dd>
176178
Removes DOM elements.
177179
</dd>
178-
<dt><code>parse_markdown</code> (Removed in v2.2.4)</dt>
180+
<dt><code>parse_markdown</code></dt>
179181
<dd>
180182
Converts Markdown to HTML. <strong>This rule has been removed in version 2.2.4.</strong>
181183
</dd>
@@ -215,25 +217,6 @@ rule1,rule2
215217

216218
Separate each rule with a comma.
217219

218-
<h2 id="scraper-rules">Scraper Rules <a class="anchor" href="#scraper-rules" title="Permalink">¶</a></h2>
219-
220-
When an article contains only an extract of the content, you can fetch
221-
the original web page and apply a set of rules to get relevant content.
222-
223-
Miniflux uses CSS selectors for custom rules. These custom rules can be
224-
saved in the feed properties (select a feed and click on edit).
225-
226-
| CSS Selector | Description |
227-
|---|---|
228-
| `div#articleBody` | Fetch a `div` element with the ID `articleBody`. |
229-
| `div.content` | Fetch all `div` elements with the class `content`. |
230-
| `article, div.article` | Use a comma to define multiple rules. |
231-
232-
Miniflux includes a list of [predefined rules](https://github.com/miniflux/v2/blob/main/internal/reader/scraper/rules.go) for popular websites.
233-
You can contribute to the project to keep them up to date.
234-
235-
Under the hood, Miniflux uses the library [Goquery](https://github.com/PuerkitoBio/goquery).
236-
237220
<h2 id="rewriteurl-rules">URL Rewrite Rules <a class="anchor" href="#rewriteurl-rules" title="Permalink">¶</a></h2>
238221

239222
Sometimes it might be required to rewrite a URL in a feed to fetch better-suited content.
@@ -263,3 +246,22 @@ The URL rewrite rule for that would be:
263246
```
264247
rewrite("(.*?\.html)"|"$1?seite=all")
265248
```
249+
250+
<h2 id="scraper-rules">Scraper Rules <a class="anchor" href="#scraper-rules" title="Permalink">¶</a></h2>
251+
252+
When an article contains only an extract of the content, you can fetch
253+
the original web page and apply a set of rules to get relevant content.
254+
255+
Miniflux uses CSS selectors for custom rules. These custom rules can be
256+
saved in the feed properties (select a feed and click on edit).
257+
258+
| CSS Selector | Description |
259+
|---|---|
260+
| `div#articleBody` | Fetch a `div` element with the ID `articleBody`. |
261+
| `div.content` | Fetch all `div` elements with the class `content`. |
262+
| `article, div.article` | Use a comma to define multiple rules. |
263+
264+
Miniflux includes a list of [predefined rules](https://github.com/miniflux/v2/blob/main/internal/reader/scraper/rules.go) for popular websites.
265+
You can contribute to the project to keep them up to date.
266+
267+
Under the hood, Miniflux uses the library [Goquery](https://github.com/PuerkitoBio/goquery).

0 commit comments

Comments
 (0)