feat: classify imported documents as posts or pages

[faisalahammad]

Summary

The importer treated every static HTML document as a WordPress page. A site with a blog section, like a personal essay index, would flatten dated articles into pages with no publish date, which left no foundation for a later Query Loop on the blog index.

This PR adds generic post vs page classification to the plan materializer. A document is classified as a post when it carries a date signal that survives the Blocks Engine plan projection: a dated head meta tag, a hierarchical /YYYY/MM/ URL, or an explicit type: post in markdown frontmatter. The site entrypoint and undated documents stay pages, which matches current behavior, so existing imports are unchanged. Dated posts get the detected date written to post_date_gmt; WordPress derives the site-local post_date from it.

Fixes #513

Changes

includes/class-static-site-importer-document-type-classifier.php (new)

Single-purpose final class. classify() takes one plan page row and returns { post_type, date, signal }.

Signals in precedence order:

1. Entrypoint always stays a page.
2. A producer-declared post_type on the plan row (or metadata.post_type) that differs from the compiler default 'page' and is a registered type wins, so the Blocks Engine producer can classify without SSI detection. The compiler already fills this from markdown frontmatter, so type: post in a .md file imports as a post.
3. Any of article:published_time, article:published, pubdate, publishdate, date, dc.date.issued, dc.date, parsely-pub-date, releasedate in head meta, when strtotime parses the content, classifies as a dated post.
4. A route.path matching /(?:^|\/)\d{4}\/(?:0?[1-9]|1[0-2])(?:/|$)/ classifies as a post even without date meta.
5. Everything else stays a page.

Why: the materializer is the consumer boundary. It trusts the type and date the producer declared on the plan row and only falls back to consumer-side detection when the row is undecided ('page', the compiler default). The compiler does not yet carry <article> or JSON-LD into the plan, so head-meta and URL detection is the fallback until the producer change lands.

includes/class-static-site-importer-wordpress-site-plan-materializer.php

Before:

$post = array(
    'ID'         => $existing_post_id,
    'post_type'  => 'page',
    'post_status' => 'publish',
    ...
);

After:

$post_type = sanitize_key( (string) ( $page['post_type'] ?? 'page' ) );
if ( ! self::is_valid_post_type( $post_type ) ) {
    $post_type = 'page';
}
$post = array(
    'ID'          => $existing_post_id,
    'post_type'   => $post_type,
    'post_status' => 'publish',
    ...
);
if ( ! empty( $page['metadata']['detected_date'] ) ) {
    $post['post_date']     = (string) $page['metadata']['detected_date'];
    $post['post_date_gmt'] = (string) $page['metadata']['detected_date'];
}

• preflight_state() runs the classifier once per page and writes post_type, detected_date, and classification_signal back into the plan row, so the existing-match check, conflict check, and theme generator all read the same value.
• reconciled_post() now looks up the reconciliation identity across posts and pages instead of pages only, which keeps re-imports idempotent for documents that were previously classified as posts.

static-site-importer.php

Loads the new classifier next to the plan materializer.

tests/smoke-wordpress-site-plan-materializer.php

Added a get_post_type_object() stub and classifier scenarios covering: undated stays page, dated meta becomes a dated post, /YYYY/MM/ URL becomes a post, post-like URL without a date stays a page, a dated nested article becomes a post, explicit markdown type: post wins, and a re-import reuses post IDs with stable types.

Testing

Test 1: Classifier smoke (automated)

1. php tests/smoke-wordpress-site-plan-materializer.php
2. npm run test:site-plan-materializer
   Result: all classifier scenarios pass; npm test shows 44 passed, 0 failed. The two legacy failures (smoke-importer-block.php, smoke-inline-svg-materialization.php) now pass on the rebased branch.

Test 2: Manual import

1. Install the build zip and activate.
2. Import an artifact with index.html, blog/hello-world.html carrying <meta property="article:published_time" content="2024-03-12T10:00:00Z">, about.html, and 2024/03/essay.html.
3. Expect blog/hello-world.html and 2024/03/essay.html in the admin Posts list with their publish dates, and index.html / about.html still Pages.
4. Re-run the same import and confirm no duplicate rows are created.

WordPress 6.6+ / PHP 8.1+. No new compatibility issues: get_post_type_object and get_posts with post_type => 'any' are standard on the supported version range.

Follow-up (tracked separately)

This PR is the SSI consumer half of the classification split. The producer half, per review, is a separate Blocks Engine change:

• Extract generic article/publication evidence from source documents (<article>, <time datetime>, JSON-LD Article / BlogPosting, microdata).
• Decide the declared post_type and normalized publication timestamp in the WordPress Site Plan projection before parent_source_path / route hierarchy is built.
• Define canonical route materialization for posts on the producer side.

SSI will honor a producer-declared post_type on the plan row and bump the automattic/blocks-engine-php-transformer composer pin once that release lands.

[chubes4]

Thanks for working on this. The problem and conservative fallback are correct: explicit source intent should win, strong article/date evidence should produce posts, and undated documents should remain pages. This also addresses a real gap we reproduced on a large mixed-content import.

The main issue is where classification happens. By the time SSI's materializer receives the WordPress Site Plan, Blocks Engine has already projected every document into a page route hierarchy, generated synthetic page ancestors, and assigned parent_source_path. Changing post_type during SSI preflight leaves those page-only route and parent assumptions attached to posts.

That causes concrete WordPress behavior problems:

1. materialize_page() assigns the generated page parent ID as post_parent for posts. The parent is commonly a page, while get_page_by_path( $route, OBJECT, 'post' ) only loads post/attachment candidates when resolving ancestry, so nested post conflict detection will miss these rows.
2. get_permalink() for a post follows the site's post permalink structure, not the canonical source route carried by the plan. A source route such as /blog/hello can therefore materialize at a different URL.
3. The classifier uses date() and then writes the same value to post_date and post_date_gmt. On a non-UTC PHP timezone, 2024-03-12T10:00:00Z is normalized to local time while being labeled as GMT.
4. Updating every resolved row with array_search( ..., array_column( ... ) ) makes preflight O(n²), which is a regression for large-site imports.

The evidence contract is also incomplete relative to #513. Blocks Engine currently preserves dated <meta> rows, but does not expose usable <article>, <time datetime>, JSON-LD Article/BlogPosting, microdata, or chronological-list membership evidence. Consequently this implementation treats a generic meta name="date" as sufficient by itself, while missing many actual posts under /blog/, /essays/, or /news/.

Could you revise this through the existing ownership boundary?

1. Have Blocks Engine extract generic article/publication evidence from source documents.
2. Have its WordPress Site Plan projection decide the declared post_type and normalized publication timestamp before constructing route hierarchy and operations.
3. Define canonical route materialization for posts instead of carrying page parents into them.
4. Keep SSI focused on validating and materializing the plan's declared type/date.
5. Add WordPress-backed coverage for mixed posts/pages, nested source routes, non-UTC dates, page-to-post reclassification, and repeated/resumable imports. The current get_page_by_path() smoke mock ignores post type and hierarchy, so it cannot prove those behaviors.

This likely becomes a small Blocks Engine producer change plus an SSI consumer change. The classifier and fixtures here are useful groundwork and should be retained where they fit that split.

The branch currently conflicts with main and has no CI results, so it will need a refresh after the contract is updated.

AI assistance: OpenAI GPT-5.6 Sol via OpenCode reviewed the current SSI, Blocks Engine, and WordPress core paths and helped draft these findings. Chris Huber is responsible for the review.

[faisalahammad]

Thanks for the thorough review. I addressed the consumer-side issues and refreshed the branch, and I am tracking the producer-side change separately.

What changed in this PR:

• The branch is rebased onto current main. It was 119 commits behind and marked conflicting. It is mergeable now and CI is running.
• A dated post no longer carries the generated page ancestor as post_parent. Pages keep the plan route ancestry; every other type materializes with post_parent 0, so get_permalink follows the site post structure, not a page shaped path.
• The classifier emits UTC with gmdate and the materializer writes only post_date_gmt. wp_insert_post derives the site local post_date, so a non UTC PHP timezone no longer double shifts the value.
• Preflight no longer uses the array_search / array_column writeback. It mutates the resolved rows by reference, which removes the O(n) scan per row.
• The conflict lookup passes the classified post type to get_page_by_path, so nested posts are found by their own type. Reconciliation already queries post_type any, so re imports and page to post reclassification stay idempotent.
• The consumer trusts a producer declared post_type first. When Blocks Engine declares post_type on the plan row, SSI honors it and only runs consumer detection for undecided rows.
• The WordPress runtime smoke now covers a dated post nested under a page wrapper, proving the real get_page_by_path behavior. The standalone smoke covers page to post reclassification, non UTC timezones, and explicit markdown type post.

What I deferred to a separate Blocks Engine producer change, per your ownership boundary:

• Extracting article, time datetime, JSON-LD Article or BlogPosting, and microdata evidence from source documents.
• Deciding the declared post_type and normalized publication timestamp before the route hierarchy and parent_source_path are built.
• Defining canonical post route materialization on the producer side.
• Versioning and publishing those changes, then bumping the composer pin in SSI.

The classifier and the fixtures are kept here as the consumer side of that split. They validate and materialize the declared type and date, and fall back to detection only when the plan row is undecided.