Skip to content

Commit 1f9167f

Browse files
authored
docs(ruby-gem): sync docs for gem release v0.25.0 (#1229)
- Document --limit flag under html2rss auto CLI reference - Document channel-level author and image discovery in auto-source reference - Document article limit option (limit: 20) in auto-source reference - Add 0.25.0 migration notes for legacy :link selector removal in backward-compatibility guide
1 parent a0d620a commit 1f9167f

3 files changed

Lines changed: 29 additions & 8 deletions

File tree

src/content/docs/ruby-gem/guides/backward-compatibility.mdx

Lines changed: 24 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,15 @@ import { Code } from "@astrojs/starlight/components";
77

88
This page outlines recent breaking changes, purged legacy aliases, and migration steps for older feed configurations.
99

10+
## 0.25.0
11+
12+
Upgrade notes for configs and integrators targeting gem **0.25.0**:
13+
14+
1. **Removed `link` selector alias** — Custom selector configurations must use the canonical `url` key. The legacy `link` selector alias and exclusivity checks have been removed.
15+
2. **`auto` article limit (`--limit N`)** — Discovery now supports a configurable article limit (`--limit N` CLI flag, default `20`) and `Html2rss.auto_source(url, limit: 20)` in Ruby.
16+
3. **Channel metadata in `auto`**`auto_source` automatically extracts channel-level `author` and `image` (favicon, touch-icon, or OpenGraph images) from the host page.
17+
4. **Actionable `NoFeedItemsExtracted` error**`auto` mode fails loud with diagnostic surface classifications (blocked surface, app-shell, unsupported extraction surface) when zero items are found.
18+
1019
## 0.24.0
1120

1221
Upgrade notes for configs and integrators targeting gem **0.24.0**:
@@ -21,21 +30,28 @@ In previous versions, `html2rss` accepted legacy selector names with a deprecati
2130

2231
| Removed Name | Required Name | Description |
2332
| :----------- | :------------- | :-------------------- |
33+
| `link` | `url` | Item destination URL |
2434
| `pubDate` | `published_at` | Item publication date |
2535
| `updated` | `published_at` | Item publication date |
2636

2737
### Migration Example
2838

29-
Update any occurrences of `pubDate` or `updated` in your `selectors` block to `published_at`:
39+
Update any occurrences of `link`, `pubDate`, or `updated` in your `selectors` block to `url` or `published_at`:
3040

3141
<Code
3242
code={`
3343
# Deprecated / Removed
3444
selectors:
45+
link:
46+
selector: "a.title"
47+
extractor: "href"
3548
updated:
3649
selector: ".date"
3750
# Current & Required
3851
selectors:
52+
url:
53+
selector: "a.title"
54+
extractor: "href"
3955
published_at:
4056
selector: ".date"
4157
`}
@@ -68,9 +84,10 @@ Update any occurrences of `pubDate` or `updated` in your `selectors` block to `p
6884

6985
When upgrading to modern `html2rss` releases:
7086

71-
1. **Rename date selectors**: Ensure date selectors use `published_at` rather than `updated` or `pubDate`.
72-
2. **Move channel-level transport keys**: Ensure `strategy` and `headers` are defined at the top level of the YAML file.
73-
3. **Pin Browserless when needed**: If you relied on `auto` falling through to Browserless, set `strategy: browserless` (or `--strategy browserless`).
74-
4. **Use `enclosure` for RSS media**: Do not rely on `image` becoming an RSS `<enclosure>`; select podcast/media URLs with `enclosure`.
75-
5. **Prefer `feed_result` for dual-format / cache**: Integrators that need RSS + JSON Feed from one scrape (or Marshal caching) should use `Html2rss.feed_result` and `status.to_h`.
76-
6. **Validate configurations**: Run `html2rss validate config.yml` to ensure your YAML conforms to the current schema.
87+
1. **Rename URL selectors**: Ensure article URL selectors use `url` rather than `link`.
88+
2. **Rename date selectors**: Ensure date selectors use `published_at` rather than `updated` or `pubDate`.
89+
3. **Move channel-level transport keys**: Ensure `strategy` and `headers` are defined at the top level of the YAML file.
90+
4. **Pin Browserless when needed**: If you relied on `auto` falling through to Browserless, set `strategy: browserless` (or `--strategy browserless`).
91+
5. **Use `enclosure` for RSS media**: Do not rely on `image` becoming an RSS `<enclosure>`; select podcast/media URLs with `enclosure`.
92+
6. **Prefer `feed_result` for dual-format / cache**: Integrators that need RSS + JSON Feed from one scrape (or Marshal caching) should use `Html2rss.feed_result` and `status.to_h`.
93+
7. **Validate configurations**: Run `html2rss validate config.yml` to ensure your YAML conforms to the current schema.

src/content/docs/ruby-gem/reference/auto-source.mdx

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,9 @@ To enable it, add `auto_source: {}` to your configuration:
3434
such as `window.__NEXT_DATA__`, `window.__NUXT__`, or `window.STATE`. The JSON-state scraper walks those blobs, finds arrays with
3535
`title`/`url` pairs, and converts them into feed items.
3636

37-
`auto_source` also automatically extracts and prepends kicker/teaser labels to article titles when present.
37+
`auto_source` also automatically extracts and prepends kicker/teaser labels to article titles when present, and discovers channel-level `author` and `image` (favicon, touch-icon, or OpenGraph images) from the host page.
38+
39+
You can also restrict the maximum number of discovered articles via the `limit:` keyword argument in Ruby (`Html2rss.auto_source(url, limit: 10)`) or the `--limit` CLI flag.
3840

3941
**`json_state` Limitations:** the scraper requires discoverable arrays of hashes containing clear `title` and `url` fields. Minified or
4042
obfuscated state objects, heavily encoded values, or blobs that require executing embedded functions are ignored.

src/content/docs/ruby-gem/reference/cli-reference.mdx

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ Automatically discovers items from a page and prints the generated RSS or JSONFe
2323
code={`
2424
html2rss auto https://example.com/articles ; \
2525
html2rss auto https://example.com/articles --format jsonfeed ; \
26+
html2rss auto https://example.com/articles --limit 10 ; \
2627
html2rss auto https://example.com/app --strategy browserless --max-redirects 5 --max-requests 6 ; \
2728
BOTASAURUS_SCRAPER_URL="http://localhost:4010" html2rss auto https://example.com/protected --strategy botasaurus ; \
2829
html2rss auto https://example.com/articles --items_selector ".post-card" ; \
@@ -38,6 +39,7 @@ Available options:
3839

3940
- `--strategy`: Optional request strategy (`auto`, `faraday`, `browserless`, `botasaurus`, `local_file`). Defaults to `auto`, which tries `faraday` -> `botasaurus`. Pin `browserless` explicitly when you need headless Chrome (preload/interaction).
4041
- `--format`: Output format for the auto-sourced feed (`rss` or `jsonfeed`). Defaults to `rss`.
42+
- `--limit`: Maximum number of articles to extract during discovery (defaults to `20`).
4143
- `--items_selector`: Optional CSS selector hint for item extraction.
4244
- `--max-redirects`: Maximum redirects to follow per request.
4345
- `--max-requests`: Maximum requests to allow for this feed build (defaults to `4` for auto discovery).

0 commit comments

Comments
 (0)