Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Changelog

## 9.1.0

- **CDATA accepted everywhere in the sitemap parser**: `XMLToSitemapItemStream` now treats CDATA sections as ordinary character data in every tag. Previously only the 8 tags from issue #445 (`loc`, `image:loc`, `video:title`, `video:description`, `news:name`, `news:title`, `image:caption`, `image:title`) accepted CDATA; content in any other tag was logged as unhandled. (See `docs/adr/0001-cdata-is-character-data.md`.)
- **Unified `<loc>` validation**: the sitemap parser now validates `<loc>` values with the same `validateURL` used by the sitemap index parser, adding a URL-parseability check on top of the existing length and protocol checks. A rare malformed URL that passed the old checks but fails `new URL()` is now dropped with a warning. Warn/error messages for invalid `<loc>` values changed to the `Invalid URL in sitemap: …` form.
- Internal: the parser's duplicated `text`/`cdata` SAX handlers were collapsed into one, and its inline bounds checks (priority, video duration/rating/view count, ISO dates) now delegate to shared guards in `lib/validation.ts`, with the bounds defined once in `LIMITS`.

## 9.0.1 — Security Patch

- **BB-01**: Fix XML injection via unescaped `xslUrl` in stylesheet processing instruction — special characters (`&`, `"`, `<`, `>`) in the XSL URL are now escaped before being interpolated into the `<?xml-stylesheet?>` processing instruction
Expand Down
9 changes: 9 additions & 0 deletions CONTEXT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# sitemap.js

A library and CLI for generating and parsing sitemap XML compliant with the sitemaps.org protocol.

## Language

**Sitemap URL**:
The address carried by a `<loc>` element — of a page in a sitemap, or of a sitemap file in a sitemap index. Always absolute, http or https, at most 2,048 characters, and parseable as a URL. One definition, applied identically wherever a URL is read or written.
_Avoid_: loc string, link, location
14 changes: 14 additions & 0 deletions docs/adr/0001-cdata-is-character-data.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# CDATA is ordinary character data, and a Sitemap URL has one definition

`XMLToSitemapItemStream` used to handle SAX `text` and `cdata` events in two near-identical switch statements, where the `cdata` switch covered only the 8 tags patched for issue #445 — CDATA in any other tag was rejected as "unhandled". When collapsing the duplication (July 2026) we decided that CDATA sections are semantically ordinary character data per the XML spec, so both events route through one `handleCharData` and every tag accepts CDATA; and that `<loc>` is validated at the same strength everywhere via `validateURL` (length, protocol, *and* URL-parseability), matching what the sitemap index parser already did.

## Considered Options

- **Preserve the 8-tag CDATA list** behind a `text | cdata` source flag: zero behavior change, but the special-case list would survive as permanent interface complexity, and the #445 list was an incremental patch rather than a design decision.
- **Keep the weaker inline `loc` check** (length + protocol only): also zero behavior change, but it would leave two different definitions of "valid sitemap URL" living in `lib/validation.ts` — the scatter the refactor existed to remove.

## Consequences

- Parsing is slightly more permissive (CDATA anywhere) and slightly stricter (a `<loc>` that fails `new URL()` is now dropped with a warning). Both deltas shipped in 9.1.0.
- Invalid-`loc` warnings changed shape to the wrapped `Invalid URL in sitemap: …` form used by the index parser.
- `<loc>` keeps assign-last-wins semantics for multi-chunk character data; accumulating chunks and validating at closetag was considered and deliberately deferred.
10 changes: 10 additions & 0 deletions lib/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,16 @@ export const LIMITS = {
MIN_SITEMAP_ITEM_LIMIT: 1,
MAX_SITEMAP_ITEM_LIMIT: 50000,

// Priority bounds per sitemaps.org spec
MIN_PRIORITY: 0,
MAX_PRIORITY: 1,

// Video value bounds per Google spec
MIN_VIDEO_DURATION: 0,
MAX_VIDEO_DURATION: 28800, // 8 hours
MIN_VIDEO_RATING: 0,
MAX_VIDEO_RATING: 5,

// Video field length constraints per Google spec
MAX_VIDEO_TITLE_LENGTH: 100,
MAX_VIDEO_DESCRIPTION_LENGTH: 2048,
Expand Down
234 changes: 27 additions & 207 deletions lib/sitemap-parser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,12 @@ import {
isAllowDeny,
isPriceType,
isResolution,
isValidPriority,
isValidVideoDuration,
isValidVideoRating,
isValidVideoViewCount,
isValidISODate,
validateURL,
} from './validation.js';
import { LIMITS } from './constants.js';

Expand Down Expand Up @@ -175,26 +181,21 @@ export class XMLToSitemapItemStream extends Transform {
}
});

this.saxStream.on('text', (text): void => {
// Per ADR 0001, CDATA sections are ordinary character data: both SAX
// events route through this one handler.
const handleCharData = (text: string): void => {
switch (currentTag) {
case 'mobile:mobile':
break;
case TagNames.loc:
// Validate URL
if (text.length > LIMITS.MAX_URL_LENGTH) {
this.logger(
'warn',
`URL exceeds max length of ${LIMITS.MAX_URL_LENGTH}: ${text.substring(0, 100)}...`
);
this.err(`URL exceeds max length of ${LIMITS.MAX_URL_LENGTH}`);
} else if (!LIMITS.URL_PROTOCOL_REGEX.test(text)) {
this.logger(
'warn',
`URL must start with http:// or https://: ${text}`
);
this.err(`URL must start with http:// or https://: ${text}`);
} else {
try {
validateURL(text, 'Sitemap URL');
currentItem.url = text;
} catch (error) {
const errMsg =
error instanceof Error ? error.message : String(error);
this.logger('warn', 'Invalid URL in sitemap:', errMsg);
this.err(`Invalid URL in sitemap: ${errMsg}`);
}
break;
case TagNames.changefreq:
Expand All @@ -205,12 +206,7 @@ export class XMLToSitemapItemStream extends Transform {
case TagNames.priority:
{
const priority = parseFloat(text);
if (
isNaN(priority) ||
!isFinite(priority) ||
priority < 0 ||
priority > 1
) {
if (!isValidPriority(priority)) {
this.logger(
'warn',
`Invalid priority "${text}" - must be between 0 and 1`
Expand All @@ -222,7 +218,7 @@ export class XMLToSitemapItemStream extends Transform {
}
break;
case TagNames.lastmod:
if (LIMITS.ISO_DATE_REGEX.test(text)) {
if (isValidISODate(text)) {
currentItem.lastmod = text;
} else {
this.logger(
Expand Down Expand Up @@ -255,12 +251,7 @@ export class XMLToSitemapItemStream extends Transform {
case TagNames['video:duration']:
{
const duration = parseInt(text, 10);
if (
isNaN(duration) ||
!isFinite(duration) ||
duration < 0 ||
duration > 28800
) {
if (!isValidVideoDuration(duration)) {
this.logger(
'warn',
`Invalid video duration "${text}" - must be between 0 and 28800 seconds`
Expand All @@ -286,7 +277,7 @@ export class XMLToSitemapItemStream extends Transform {
}
break;
case TagNames['video:publication_date']:
if (LIMITS.ISO_DATE_REGEX.test(text)) {
if (isValidISODate(text)) {
currentVideo.publication_date = text;
} else {
this.logger(
Expand All @@ -308,7 +299,7 @@ export class XMLToSitemapItemStream extends Transform {
case TagNames['video:view_count']:
{
const viewCount = parseInt(text, 10);
if (isNaN(viewCount) || !isFinite(viewCount) || viewCount < 0) {
if (!isValidVideoViewCount(viewCount)) {
this.logger(
'warn',
`Invalid video view_count "${text}" - must be a positive integer`
Expand All @@ -331,7 +322,7 @@ export class XMLToSitemapItemStream extends Transform {
}
break;
case TagNames['video:expiration_date']:
if (LIMITS.ISO_DATE_REGEX.test(text)) {
if (isValidISODate(text)) {
currentVideo.expiration_date = text;
} else {
this.logger(
Expand All @@ -353,12 +344,7 @@ export class XMLToSitemapItemStream extends Transform {
case TagNames['video:rating']:
{
const rating = parseFloat(text);
if (
isNaN(rating) ||
!isFinite(rating) ||
rating < 0 ||
rating > 5
) {
if (!isValidVideoRating(rating)) {
this.logger(
'warn',
`Invalid video rating "${text}" - must be between 0 and 5`
Expand Down Expand Up @@ -419,7 +405,7 @@ export class XMLToSitemapItemStream extends Transform {
if (!currentItem.news) {
currentItem.news = newsTemplate();
}
if (LIMITS.ISO_DATE_REGEX.test(text)) {
if (isValidISODate(text)) {
currentItem.news.publication_date = text;
} else {
this.logger(
Expand Down Expand Up @@ -600,176 +586,10 @@ export class XMLToSitemapItemStream extends Transform {
this.err(`unhandled text for tag: ${currentTag} '${text}'`);
break;
}
});

this.saxStream.on('cdata', (text): void => {
switch (currentTag) {
case TagNames.loc:
// Validate URL
if (text.length > LIMITS.MAX_URL_LENGTH) {
this.logger(
'warn',
`URL exceeds max length of ${LIMITS.MAX_URL_LENGTH}: ${text.substring(0, 100)}...`
);
this.err(`URL exceeds max length of ${LIMITS.MAX_URL_LENGTH}`);
} else if (!LIMITS.URL_PROTOCOL_REGEX.test(text)) {
this.logger(
'warn',
`URL must start with http:// or https://: ${text}`
);
this.err(`URL must start with http:// or https://: ${text}`);
} else {
currentItem.url = text;
}
break;
case TagNames['image:loc']:
currentImage.url = text;
break;
case TagNames['video:title']:
if (
currentVideo.title.length + text.length <=
LIMITS.MAX_VIDEO_TITLE_LENGTH
) {
currentVideo.title += text;
} else {
this.logger(
'warn',
`video title exceeds max length of ${LIMITS.MAX_VIDEO_TITLE_LENGTH}`
);

this.err(
`video title exceeds max length of ${LIMITS.MAX_VIDEO_TITLE_LENGTH}`
);
}
break;
case TagNames['video:description']:
if (
currentVideo.description.length + text.length <=
LIMITS.MAX_VIDEO_DESCRIPTION_LENGTH
) {
currentVideo.description += text;
} else {
this.logger(
'warn',
`video description exceeds max length of ${LIMITS.MAX_VIDEO_DESCRIPTION_LENGTH}`
);

this.err(
`video description exceeds max length of ${LIMITS.MAX_VIDEO_DESCRIPTION_LENGTH}`
);
}
break;
case TagNames['news:name']:
if (!currentItem.news) {
currentItem.news = newsTemplate();
}
if (
currentItem.news.publication.name.length + text.length <=
LIMITS.MAX_NEWS_NAME_LENGTH
) {
currentItem.news.publication.name += text;
} else {
this.logger(
'warn',
`news name exceeds max length of ${LIMITS.MAX_NEWS_NAME_LENGTH}`
);

this.err(
`news name exceeds max length of ${LIMITS.MAX_NEWS_NAME_LENGTH}`
);
}
break;
case TagNames['news:title']:
if (!currentItem.news) {
currentItem.news = newsTemplate();
}
if (
currentItem.news.title.length + text.length <=
LIMITS.MAX_NEWS_TITLE_LENGTH
) {
currentItem.news.title += text;
} else {
this.logger(
'warn',
`news title exceeds max length of ${LIMITS.MAX_NEWS_TITLE_LENGTH}`
);

this.err(
`news title exceeds max length of ${LIMITS.MAX_NEWS_TITLE_LENGTH}`
);
}
break;
case TagNames['image:caption']:
if (!currentImage.caption) {
currentImage.caption =
text.length <= LIMITS.MAX_IMAGE_CAPTION_LENGTH
? text
: text.substring(0, LIMITS.MAX_IMAGE_CAPTION_LENGTH);
if (text.length > LIMITS.MAX_IMAGE_CAPTION_LENGTH) {
this.logger(
'warn',
`image caption exceeds max length of ${LIMITS.MAX_IMAGE_CAPTION_LENGTH}`
);
};

this.err(
`image caption exceeds max length of ${LIMITS.MAX_IMAGE_CAPTION_LENGTH}`
);
}
} else if (
currentImage.caption.length + text.length <=
LIMITS.MAX_IMAGE_CAPTION_LENGTH
) {
currentImage.caption += text;
} else {
this.logger(
'warn',
`image caption exceeds max length of ${LIMITS.MAX_IMAGE_CAPTION_LENGTH}`
);

this.err(
`image caption exceeds max length of ${LIMITS.MAX_IMAGE_CAPTION_LENGTH}`
);
}
break;
case TagNames['image:title']:
if (!currentImage.title) {
currentImage.title =
text.length <= LIMITS.MAX_IMAGE_TITLE_LENGTH
? text
: text.substring(0, LIMITS.MAX_IMAGE_TITLE_LENGTH);
if (text.length > LIMITS.MAX_IMAGE_TITLE_LENGTH) {
this.logger(
'warn',
`image title exceeds max length of ${LIMITS.MAX_IMAGE_TITLE_LENGTH}`
);

this.err(
`image title exceeds max length of ${LIMITS.MAX_IMAGE_TITLE_LENGTH}`
);
}
} else if (
currentImage.title.length + text.length <=
LIMITS.MAX_IMAGE_TITLE_LENGTH
) {
currentImage.title += text;
} else {
this.logger(
'warn',
`image title exceeds max length of ${LIMITS.MAX_IMAGE_TITLE_LENGTH}`
);

this.err(
`image title exceeds max length of ${LIMITS.MAX_IMAGE_TITLE_LENGTH}`
);
}
break;

default:
this.logger('log', 'unhandled cdata for tag:', currentTag);
this.err(`unhandled cdata for tag: ${currentTag}`);
break;
}
});
this.saxStream.on('text', handleCharData);
this.saxStream.on('cdata', handleCharData);

this.saxStream.on('attribute', (attr): void => {
switch (currentTag) {
Expand Down
Loading
Loading