Document internal surfaces and the PHP Cypher query path (#1045)

* Document internal surfaces and the PHP Cypher query path

Extensions have started building on two surfaces that are reachable but
internal: the rendered .ext-neowiki-view placeholders and the raw Neo4j
client. NeoWiki is pre-1.0 and cannot yet declare a stable interface,
so the extending docs gain an "Internal surfaces" section: the rendered
HTML is marked as not an integration surface, while the internal client
gets a comparison of what the documented query interfaces provide over
it (read-only enforcement, resource limits, result handling, churn
exposure) rather than a prohibition, plus a warning that direct graph
writes do not survive re-projection or rebuilds.

Since the alternative for PHP-side querying was undocumented, add a
"Running Cypher queries" subsection covering newCypherQueryService(),
backed by a new RedHerb example (SpecialRedHerbContentPageCount) with
an integration test against real Neo4j, so the documented call shape
stays working like the page's other examples.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Drop the apihighlimits mention from the query limits sentence

The tier mechanics are documented on the linked Query API page; naming
the right here duplicated that and it is default MediaWiki behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Address review precision findings on the query docs and example

Docs: name the keyword validator's CALL and SHOW rejection, state that
the row cap truncates only after the query has run (bound work with
LIMIT in the Cypher), scope the durable-write pointer to page-level
key/value metadata and state plainly that arbitrary graph structure has
no durable third-party write path, and qualify projection on save and
rebuild with the Subject-slot requirement.

Example: bind the caught exception and surface its message like the
Scribunto library does, cover the no-backend error branch via
runWithoutGraphBackend(), and rework editMainSubject to resolve the
Main Subject through getSubjectRepository()->getPageSubjects() instead
of scraping the .ext-neowiki-view placeholder the docs forbid; the docs
now cite it as the worked example of that path.

Also exclude tests/RedHerb/node_modules from phpcs: following the
RedHerb README's npm install instructions previously made make phpcs
scan thousands of dependency files and run out of memory.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Tighten internal-surfaces wording from the text-review pass

Name Neo4j in the no-backend LogicException sentence (it matches the
thrown message and stays exact once SPARQL stores can be configured
alongside), state that editMainSubject resolves the Main Subject
through the public JS API instead of the ambiguous "this way", and cut
a sentence restating the section intro's change-without-notice fact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Jeroen De Dauw <jeroendedauw@proton.me>

Author: 3 people

docs/extending/extending.md

@@ -106,7 +106,8 @@ Neo4j projection.
 ### Page Property Providers
 
 Page Property Providers contribute key/value metadata to the Page node in the graph (queryable via Cypher;
-Neo4j is currently the only graph backend). Implement `PagePropertyProvider`:
+Neo4j is currently the only graph backend). Providers run when a page carrying the NeoWiki subject slot is
+saved or rebuilt; pages without Subjects are not projected. Implement `PagePropertyProvider`:
 
 ```php
 class StaticPagePropertyProvider implements PagePropertyProvider {
@@ -198,6 +199,36 @@ you to remove pages that are already gone from your store.
 Examples: [`src/RedHerbSidebarHook.php`](https://github.com/ProfessionalWiki/NeoWiki/blob/master/tests/RedHerb/src/RedHerbSidebarHook.php)
 and [`src/Specials/SpecialRedHerbSubjectFinder.php`](https://github.com/ProfessionalWiki/NeoWiki/blob/master/tests/RedHerb/src/Specials/SpecialRedHerbSubjectFinder.php).
 
+### Running Cypher queries
+
+To run a read-only Cypher query from PHP, use `NeoWikiExtension::getInstance()->newCypherQueryService()`.
+It rejects write queries, enforces the timeout against the backend, and truncates results to the row cap;
+resolve the limits configured in [`$wgNeoWikiQueryLimits`](../api/query-api.md) with
+`Neo4jQueryLimits::forUser()`:
+
+```php
+$result = NeoWikiExtension::getInstance()->newCypherQueryService()->execute( new Neo4jQueryRequest(
+	cypher: 'MATCH (s:Subject:Person) WHERE s.`Birth year` > $minYear RETURN s.name AS name',
+	parameters: [ 'minYear' => 2000 ],
+	limits: Neo4jQueryLimits::forUser( $this->getUser() ),
+) );
+```
+
+`execute()` returns a `Neo4jQueryResult` (columns, rows, truncation flag) and throws a `QueryException`
+subclass on failure; `newCypherQueryService()` itself throws a `LogicException` on a wiki with no Neo4j
+backend configured.
+
+The `User` in `forUser()` only sizes the limits: how heavy a single query may be, not whether the user may
+query at all or how often. When running user-supplied queries, check the `neowiki-query` right and rate
+limit yourself, as the [Query API](../api/query-api.md) endpoint does.
+
+Two sharp edges: the write check is keyword-based and also rejects `CALL` and `SHOW`, even for read-only
+procedures (see the [parser function notes](../authoring/parser-functions.md)). And the row cap truncates
+the result only after the query has run in full, so bound expensive queries with `LIMIT` in the Cypher
+itself.
+
+Example: [`src/Specials/SpecialRedHerbContentPageCount.php`](https://github.com/ProfessionalWiki/NeoWiki/blob/master/tests/RedHerb/src/Specials/SpecialRedHerbContentPageCount.php).
+
 ## Frontend extension points (JS/Vue)
 
 NeoWiki's frontend is built with TypeScript and Vue. Extensions consume it as plain JavaScript and need no build step.
@@ -385,3 +416,57 @@ These extension points are designed or partially present but not yet open to ext
 - **A published TypeScript types package.** TypeScript authors get types today by pointing their `tsconfig` at
   NeoWiki's source (see "Authoring in TypeScript" and [ADR 24](../adr/024-frontend-extension-mechanism.md)). A
   published, versioned package is deferred until a consumer needs types without a NeoWiki checkout.
+
+## Internal surfaces
+
+Everything on this page is alpha, but the surfaces below are internal even by that standard: they are
+implementation details that happen to be reachable, and they can change in any release without notice.
+
+### NeoWiki's rendered HTML
+
+The `.ext-neowiki-view` placeholder elements and `data-mw-neowiki-*` attributes are the private contract
+between NeoWiki's backend and its frontend for mounting Views. They are not an integration surface: do not
+select these elements, read Subject IDs out of them, restyle their internals, or remove and replace them
+with your own rendering.
+
+To control where and how a Subject renders:
+
+- To place a Subject rendering in page content, use the
+  [`{{#view}}` parser function](../authoring/parser-functions.md), optionally with a Layout to control which
+  properties are shown.
+- To render Subjects in your own visual format, register a custom View Type
+  (see [Registering a View Type frontend](#registering-a-view-type-frontend)).
+- For fully custom UI outside the View system, fetch the data through the [REST API](../api/rest-api.md) or
+  the [public JS API](#using-neowikis-public-js-api) and render your own components, mounted as described in
+  [Mounting standalone Vue features](#mounting-standalone-vue-features). RedHerb's
+  [`editMainSubject`](https://github.com/ProfessionalWiki/NeoWiki/tree/master/tests/RedHerb/resources/editMainSubject)
+  resolves the page's Main Subject through the public JS API.
+
+### The internal Neo4j client
+
+`NeoWikiExtension::getInstance()->getNeo4jClient()` and `getReadOnlyNeo4jClient()` return the Laudis client
+NeoWiki itself uses. Neo4j access is treated as an implementation detail of NeoWiki's persistence layer
+([ADR 13](../adr/013-restrict-neo4j-access.md)). Nothing stops an extension from querying through the raw
+client, but compare what the documented query interfaces
+([`{{#cypher_raw}}`](../authoring/parser-functions.md), [`nw.query`](../authoring/lua-api.md), the
+[Query API](../api/query-api.md), and the [PHP query service](#running-cypher-queries)) provide over it:
+
+- **Read-only enforcement.** The query interfaces reject write queries (a keyword check plus `EXPLAIN`;
+  the keyword check also rejects read-only `CALL` and `SHOW`); with the raw client, a bug in calling code
+  can corrupt the graph projection, which is what ADR 13 exists to prevent.
+- **Resource limits.** The query interfaces enforce a configured timeout against the backend and cap the
+  rows returned; the raw client has neither. The row cap does not bound the work a query does, so use
+  `LIMIT` in the Cypher either way.
+- **Result handling.** The query interfaces return normalized rows and columns; the raw client returns
+  Laudis driver types that you convert yourself.
+- **Churn exposure.** The query interfaces are documented contracts (alpha, like everything on this page);
+  the client accessors are internal wiring that can change or disappear in any release.
+
+Writing to the graph directly deserves particular caution: the graph is a projection of wiki content that
+NeoWiki rewrites at will. Saving a page that carries the Subject slot re-projects that page's nodes, and
+the `RebuildGraphDatabases` maintenance script rebuilds the projection from scratch, so anything a third
+party writes into the graph can be overwritten, orphaned, or deleted at any time. For page-level key/value
+metadata there is a durable path: [Page Property Providers](#page-property-providers), which NeoWiki
+re-runs whenever a Subject-slot page is saved or rebuilt. For arbitrary nodes and relationships there is
+currently no durable third-party write path into NeoWiki's graph; a
+[Graph Database Backend](#graph-database-backends) projects durably, but into its own store.

phpcs.xml

@@ -3,6 +3,8 @@
 	<file>src/</file>
 	<file>tests/</file>
 
+	<exclude-pattern>tests/RedHerb/node_modules/</exclude-pattern>
+
 	<rule ref="./vendor/mediawiki/mediawiki-codesniffer/MediaWiki">
 		<exclude name="Generic.Files.LineLength.TooLong" />
 		<exclude name="MediaWiki.Commenting.FunctionComment" />

tests/RedHerb/README.md

@@ -24,6 +24,8 @@ extension point to the file that demonstrates it.
   [`src/RedHerbFrontendModulesHook.php`](src/RedHerbFrontendModulesHook.php).
 - **Reading NeoWiki data / authorization** — [`src/RedHerbSidebarHook.php`](src/RedHerbSidebarHook.php) and
   [`src/Specials/SpecialRedHerbSubjectFinder.php`](src/Specials/SpecialRedHerbSubjectFinder.php).
+- **Running Cypher queries (PHP query service)** —
+  [`src/Specials/SpecialRedHerbContentPageCount.php`](src/Specials/SpecialRedHerbContentPageCount.php).
 
 ### Frontend (JS/Vue)
 

tests/RedHerb/extension.json

@@ -47,6 +47,9 @@
 	"SpecialPages": {
 		"RedHerbSubjectFinder": {
 			"class": "ProfessionalWiki\\RedHerb\\Specials\\SpecialRedHerbSubjectFinder"
+		},
+		"RedHerbContentPageCount": {
+			"class": "ProfessionalWiki\\RedHerb\\Specials\\SpecialRedHerbContentPageCount"
 		}
 	},
 

tests/RedHerb/i18n/_Aliases.php

@@ -4,4 +4,5 @@
 
 $specialPageAliases['en'] = [
 	'RedHerbSubjectFinder' => [ 'RedHerbSubjectFinder' ],
+	'RedHerbContentPageCount' => [ 'RedHerbContentPageCount' ],
 ];

tests/RedHerb/i18n/en.json

@@ -6,6 +6,9 @@
 	"redherb-sidebar-create-child-company": "Create child Company",
 	"redherb-sidebar-subject-finder": "Find a subject",
 	"redherb-special-subject-finder": "RedHerb subject finder",
+	"redherb-special-content-page-count": "RedHerb content page count",
+	"redherb-content-page-count": "Content namespace pages tracked in the graph: $1",
+	"redherb-content-page-count-error": "Could not count content namespace pages tracked in the graph: $1",
 	"redherb-subject-finder-schema-label": "Schema",
 	"redherb-subject-finder-schema-placeholder": "Enter a schema name",
 	"redherb-subject-finder-pick-subject": "Pick a subject",

tests/RedHerb/i18n/qqq.json

@@ -6,6 +6,9 @@
 	"redherb-sidebar-create-child-company": "Text of the sidebar link that opens a dialog for creating a child Subject with the Company schema.",
 	"redherb-sidebar-subject-finder": "Text of the sidebar link that opens the RedHerb subject finder.",
 	"redherb-special-subject-finder": "Title shown on Special:RedHerbSubjectFinder.",
+	"redherb-special-content-page-count": "Title shown on Special:RedHerbContentPageCount.",
+	"redherb-content-page-count": "Result shown on Special:RedHerbContentPageCount, reporting how many content-namespace pages NeoWiki tracks in the graph. $1 is the count.",
+	"redherb-content-page-count-error": "Error shown on Special:RedHerbContentPageCount when the Cypher query fails. $1 is the underlying error message.",
 	"redherb-subject-finder-schema-label": "Label of the schema name input on Special:RedHerbSubjectFinder.\n{{Identical|Schema}}",
 	"redherb-subject-finder-schema-placeholder": "Placeholder text inside the schema name input on Special:RedHerbSubjectFinder.",
 	"redherb-subject-finder-pick-subject": "Label of the subject lookup on Special:RedHerbSubjectFinder.",

tests/RedHerb/resources/editMainSubject/init.js

@@ -8,7 +8,6 @@
 	const DIALOG_STATE_KEY = require( './constants.js' ).DIALOG_STATE_KEY;
 
 	const TRIGGER_SELECTOR = '.ext-redherb-edit-main-subject-trigger';
-	const MAIN_SUBJECT_SELECTOR = '.ext-neowiki-view[data-mw-neowiki-subject-id]';
 
 	const dialogState = Vue.reactive( { open: false, subjectId: null } );
 	let mounted = false;
@@ -31,21 +30,18 @@
 	}
 
 	function resolveMainSubjectId() {
-		const el = document.querySelector( MAIN_SUBJECT_SELECTOR );
-		if ( el === null ) {
-			return null;
+		const pageId = mw.config.get( 'wgArticleId' );
+		if ( !pageId ) {
+			return Promise.resolve( null );
 		}
-		return el.dataset.mwNeowikiSubjectId || null;
+		return nw.NeoWikiExtension.getInstance().getSubjectRepository().getPageSubjects( pageId )
+			.then( ( result ) => {
+				const mainSubjectId = result.pageSubjects.getMainSubjectId();
+				return mainSubjectId === null ? null : mainSubjectId.text;
+			} );
 	}
 
-	function handleClick( ev ) {
-		const trigger = ev.target.closest( TRIGGER_SELECTOR );
-		if ( trigger === null ) {
-			return;
-		}
-		ev.preventDefault();
-
-		const subjectId = resolveMainSubjectId();
+	function openDialog( subjectId ) {
 		if ( subjectId === null ) {
 			mw.notify(
 				mw.message( 'redherb-edit-main-subject-no-main' ).text(),
@@ -59,6 +55,24 @@
 		dialogState.open = true;
 	}
 
+	function handleClick( ev ) {
+		const trigger = ev.target.closest( TRIGGER_SELECTOR );
+		if ( trigger === null ) {
+			return;
+		}
+		ev.preventDefault();
+
+		resolveMainSubjectId()
+			.then( openDialog )
+			.catch( ( err ) => {
+				mw.log.error( err );
+				mw.notify(
+					err instanceof Error ? err.message : String( err ),
+					{ type: 'error' }
+				);
+			} );
+	}
+
 	queueMicrotask( () => {
 		document.body.addEventListener( 'click', handleClick );
 	} );

tests/RedHerb/src/Specials/SpecialRedHerbContentPageCount.php

@@ -0,0 +1,46 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace ProfessionalWiki\RedHerb\Specials;
+
+use Exception;
+use MediaWiki\Message\Message;
+use MediaWiki\SpecialPage\SpecialPage;
+use ProfessionalWiki\NeoWiki\GraphDatabasePlugins\Neo4j\Application\Neo4jQueryLimits;
+use ProfessionalWiki\NeoWiki\GraphDatabasePlugins\Neo4j\Application\Neo4jQueryRequest;
+use ProfessionalWiki\NeoWiki\NeoWikiExtension;
+
+class SpecialRedHerbContentPageCount extends SpecialPage {
+
+	public function __construct() {
+		parent::__construct( 'RedHerbContentPageCount' );
+	}
+
+	/**
+	 * @param ?string $subPage
+	 */
+	public function execute( $subPage ): void {
+		parent::execute( $subPage );
+
+		$out = $this->getOutput();
+
+		try {
+			$result = NeoWikiExtension::getInstance()->newCypherQueryService()->execute( new Neo4jQueryRequest(
+				cypher: 'MATCH (page:Page) WHERE page.namespaceId = $namespaceId RETURN count(page) AS pageCount',
+				parameters: [ 'namespaceId' => NS_MAIN ],
+				limits: Neo4jQueryLimits::forUser( $this->getUser() ),
+			) );
+		} catch ( Exception $e ) {
+			$out->addWikiMsg( 'redherb-content-page-count-error', $e->getMessage() );
+			return;
+		}
+
+		$out->addWikiMsg( 'redherb-content-page-count', $result->rows[0]['pageCount'] );
+	}
+
+	public function getDescription(): Message {
+		return $this->msg( 'redherb-special-content-page-count' );
+	}
+
+}

tests/phpunit/RedHerb/SpecialRedHerbContentPageCountTest.php

@@ -0,0 +1,60 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace ProfessionalWiki\NeoWiki\Tests\RedHerb;
+
+use MediaWiki\Context\DerivativeContext;
+use MediaWiki\Context\RequestContext;
+use MediaWiki\Output\OutputPage;
+use MediaWiki\Title\Title;
+use ProfessionalWiki\NeoWiki\Tests\Data\TestSubject;
+use ProfessionalWiki\NeoWiki\Tests\NeoWikiIntegrationTestCase;
+use ProfessionalWiki\RedHerb\Specials\SpecialRedHerbContentPageCount;
+
+/**
+ * @covers \ProfessionalWiki\RedHerb\Specials\SpecialRedHerbContentPageCount
+ * @group Database
+ */
+class SpecialRedHerbContentPageCountTest extends NeoWikiIntegrationTestCase {
+
+	protected function setUp(): void {
+		parent::setUp();
+		$this->setUpNeo4j();
+		$this->createSchema( TestSubject::DEFAULT_SCHEMA_ID );
+		$this->markPageTableAsUsed();
+	}
+
+	public function testCountsOnlyContentNamespacePagesTrackedInTheGraph(): void {
+		$this->createPageWithSubjects( 'RedHerb content page one', TestSubject::build( id: 'sRedHerbCP11111' ) );
+		$this->createPageWithSubjects( 'RedHerb content page two', TestSubject::build( id: 'sRedHerbCP22222' ) );
+		$this->createPageWithSubjects( 'Help:RedHerb help page', TestSubject::build( id: 'sRedHerbHP33333' ) );
+
+		$this->assertStringContainsString(
+			'Content namespace pages tracked in the graph: 2',
+			$this->executeContentPageCount()
+		);
+	}
+
+	public function testErrorOutputIncludesUnderlyingCauseWhenGraphBackendMissing(): void {
+		$html = $this->runWithoutGraphBackend(
+			fn() => $this->executeContentPageCount()
+		);
+
+		$this->assertStringContainsString( 'A configured Neo4j backend is required', $html );
+	}
+
+	private function executeContentPageCount(): string {
+		$context = new DerivativeContext( RequestContext::getMain() );
+		$context->setTitle( Title::makeTitle( NS_SPECIAL, 'RedHerbContentPageCount' ) );
+		$output = new OutputPage( $context );
+		$context->setOutput( $output );
+
+		$page = new SpecialRedHerbContentPageCount();
+		$page->setContext( $context );
+		$page->execute( null );
+
+		return $output->getHTML();
+	}
+
+}