Breaking change: advanced routing and DotJS can now be used together

Author: aral

CHANGELOG.md

@@ -4,6 +4,20 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [17.0.0] - 2021-04-25
+
+### Breaking change
+
+  - If you have an advanced routes file (routes.js) it is loaded prior to any DotJS routes. Previously, the presence of a advanced routes file meant that routes were only loaded from it and that any DotJS routes, if they existed, were ignored. This change means you can use DotJS in your sites and (instead of or), when you need to, define some of your routes using the full expressiveness of Express routes.
+
+      Although this is a breaking change in that it changes behaviour, the practical impact on existing sites should be minimal given that a project that was using advanced routing would not have had DotJS routes. The only place where this might impact you is if you forgot to delete some old DotJS routes and get surprised when they’re added to your application.
+
+### Added
+
+  - In advanced routes (in routes.js) you now have access to the Site.js class (`app.Site`) and Site.js instance (`app.site`) through the Express `app` instance.
+
+  - In the statistics view, any routes that begin with _/admin/…_ are shown as ‘Administration page’ to hide any cryptographically-secure paths that may be used as per convention.
+
 ## [16.6.0] - 2021-04-23
 
 ### Changed

README.md

@@ -1389,6 +1389,8 @@ module.exports = app => {
 
 When using the _routes.js_ file, you can use all of the features in [express](https://expressjs.com/) and [our fork of express-ws](https://github.com/aral/express-ws) (which itself wraps [ws](https://github.com/websockets/ws#usage-examples)).
 
+__As of Site.js 17.0.0,__ you can also use DotJS routes alongside your advanced routes file. The routes in the _routes.js_ file are loaded first (see [Routing precedence](#routing-precedence), below).
+
 ### Routing precedence
 
 #### Between dynamic route and static route
@@ -1420,19 +1422,19 @@ The behaviour observed under Linux at the time of writing is that _fun/index.js_
 
 #### Between the various routing methods
 
-Each of the routing conventions are mutually exclusive and applied according to the following precedence rules:
+Each of the routing conventions ­– apart from advanced _routes.js_-based routing (as of Site.js version 17.0.0) – are mutually exclusive and applied according to the following precedence rules:
 
-1. Advanced _routes.js_-based advanced routing.
+1. Advanced _routes.js_-based routing.
 
 2. DotJS with separate folders for _.https_ and _.wss_ routes routing (the _.http_ folder itself will apply precedence rules 3 and 4 internally).
 
 3. DotJS with separate folders for _.get_ and _.post_ routes in HTTPS-only routing.
 
 4. DotJS with GET-only routing.
 
-So, if Site.js finds a _routes.js_ file in the root folder of your site’s folder, it will only use the routes from that file (it will not apply file-based routing).
+If Site.js finds a _routes.js_ file in the root folder of your site’s folder, as of Site.js version 17.0.0, it will load any routes defined in that file first before looking for any file-based DotJS routes.
 
-If Site.js cannot find a _routes.js_ file, it will look to see if separate _.https_ and _.wss_ folders have been defined (the existence of just one of these is enough) and attempt to load DotJS routes from those folders. (If it finds separate _.get_ or _.post_ folders within the _.https_ folder, it will add the relevant routes from those folders; if it can’t it will load GET-only routes from the _.https_ folder and its subfolders.)
+Next Site.js, will look to see if separate _.https_ and _.wss_ folders have been defined (the existence of just one of these is enough) and attempt to load DotJS routes from those folders. (If it finds separate _.get_ or _.post_ folders within the _.https_ folder, it will add the relevant routes from those folders; if it can’t it will load GET-only routes from the _.https_ folder and its subfolders.)
 
 If separate _.https_ and _.wss_ folders do not exist, Site.js will expect all defined DotJS routes to be HTTPS and will initially look for separate _.get_ and _.post_ folders (the existence of either is enough to trigger this mode). If they exist, it will add the relevant routes from those folders and their subfolders.
 
@@ -1462,6 +1464,36 @@ const appPath = require.main.filename.replace('bin/site.js', '')
 
 The code within your JavaScript routes is executed on the server. Exercise the same caution as you would when creating any Node.js app (sanitise input, etc.)
 
+### Creating an Admin page.
+
+Given that Site.js is for single-tenant apps and sites, you can create an admin page for your site/app using the same convention that Site.js itself uses for the statistics route: by using a cryptographically secure path for it. In fact, Site.js will hide the path from your statistics view if you adhere to the convention of creating it at _/admin/cryptographically-secure-path_. Unlike the statistics URL, you will have to implement this functionality using the advanced routing feature. e.g.,
+
+```js
+const crypto = require('crypto')
+
+// Create a cryptographically-secure path for the admin route
+// and save it in a table called admin in the built-in JSDB database.
+if (db.admin === undefined) {
+  db.admin = {}
+  db.admin.route = crypto.randomBytes(16).toString('hex')
+}
+
+// Output the admin path to the logs so you know what it is.
+console.log(`   🔑️    ❨My site❩ Admin page is at /${db.admin.route}`)
+
+module.exports = app => {
+  // Add the admin route using the cryptographically-secure path.
+  app.get(`/admin/${db.admin.route}`, (request, response) => {
+    response.html(`
+    <h1>Admin page</h1>
+    <p>Welcome to the admin page.</p>
+    <hr>
+    <p><a href='https://${app.site.prettyLocation()}${app.site.stats.route}'>Site statistics.</a></p>
+  `)
+  })
+}
+```
+
 ## Wildcard routes
 
 As of version 14.5.0, if all you want to do is to customise the behaviour of your pages using client-side JavaScript based on parameters provided through the URL path, you don’t have to use dynamic routes and a `routes.js` file, you can use wildcard routes instead, which are much simpler.

index.js

@@ -332,6 +332,10 @@ class Site {
     this.stats = this.initialiseStatistics()
     this.app = express()
 
+    // Add a reference to the to Site.js instance to the app.
+    this.app.Site = Site
+    this.app.site = this
+
     // Create the HTTPS server.
     this.createServer()
   }
@@ -1446,9 +1450,11 @@ class Site {
   // Add dynamic routes, if any, if a <pathToServe>/.dynamic/ folder exists.
   // If there are errors in any of your dynamic routes, you will get 500 (server) errors.
   //
-  // Each of the routing conventions are mutually exclusive and applied according to the following precedence rules:
+  // Each of the routing conventions ­– apart from advanced _routes.js_-based routing
+  // (as of Site.js version 17.0.0) – are mutually exclusive and applied according to
+  // the following precedence rules:
   //
-  // 1. Advanced _routes.js_-based advanced routing.
+  // 1. Advanced _routes.js_-based routing.
   //
   // 2. Separate folders for _.https_ and _.wss_ routes routing (the _.http_ folder itself will apply
   // precedence rules 3 and 4 internally).
@@ -1473,10 +1479,17 @@ class Site {
       // Attempts to load HTTPS routes from the passed directory,
       // adhering to rules 3 & 4.
       const loadHttpsRoutesFrom = (httpsRoutesDirectory) => {
+
         // Attempts to load HTTPS GET routes from the passed directory.
-        const loadHttpsGetRoutesFrom = (httpsGetRoutesDirectory) => {
+        const loadHttpsGetRoutesFrom = (httpsGetRoutesDirectory, skipAdvancedRoutingFile = false) => {
           const httpsGetRoutes = getRoutes(httpsGetRoutesDirectory)
           httpsGetRoutes.forEach(route => {
+            // Skip adding the advanced routing file as an HTTPS GET route,
+            // even if it looks like one.
+            if (skipAdvancedRoutingFile && route.path === '/routes') {
+              return
+            }
+
             this.log(`   🐁    ❨site.js❩ Adding HTTPS GET route: ${route.path}`)
 
             // Ensure we are loading a fresh copy in case it has changed.
@@ -1530,7 +1543,7 @@ class Site {
         // ========================================================
         //
 
-        loadHttpsGetRoutesFrom(httpsRoutesDirectory)
+        loadHttpsGetRoutesFrom(httpsRoutesDirectory, /* skipAdvancedRoutingFile = */ true)
       }
 
       //
@@ -1544,14 +1557,13 @@ class Site {
       const advancedRoutesFile = fs.existsSync(routesJsFile) ? routesJsFile : fs.existsSync(routesCjsFile) ? routesCjsFile : undefined
 
       if (advancedRoutesFile !== undefined) {
-        this.log(`   🐁    ❨site.js❩ Found advanced routes file (${advancedRoutesFile}), will load dynamic routes from there.`)
+        this.log(`   🐁    ❨site.js❩ Found advanced routes file (${advancedRoutesFile}), adding to app.`)
         // We flag that this needs to be done here and actually require the file
         // once the server has been created so that WebSocket routes can be added also.
         this.routesJsFile = advancedRoutesFile
 
         // Add POST handling in case there are POST routes defined.
         addBodyParser()
-        return
       }
 
       //

lib/Stats.js

@@ -68,7 +68,6 @@ class Stats {
       }
 
       response.on('finish', () => {
-
         if (response.statusCode === 404) {
           this.missing.add(request.path)
         }
@@ -114,8 +113,8 @@ class Stats {
       const sortedReferrers = Object.keys(this.referrers).sort((a, b) => -(this.referrers[a] - this.referrers[b]))
 
       const none = '<li>None yet.</li>'
-      const allRequestsList = requestKeysToHtml(sortedRequestKeys).replace(this.route, 'This page') || none
-      const topThreeRequestsList = requestKeysToHtml(topThreeRequestKeys).replace(this.route, 'This page') || none
+      const allRequestsList = requestKeysToHtml(sortedRequestKeys).replace(this.route, 'This page').replace(/\/admin.*?:/, 'Administration page:') || none
+      const topThreeRequestsList = requestKeysToHtml(topThreeRequestKeys).replace(this.route, 'This page').replace(/\/admin.*?:/, 'Administration page:') || none
 
       const sortedReferrersList = referrersToHtml(sortedReferrers) || none
 

package.json

@@ -1,13 +1,14 @@
 {
   "name": "@small-tech/site.js",
-  "version": "16.6.0",
+  "version": "17.0.0",
   "description": "Small Web construction set.",
   "keywords": [
     "web server",
     "static site generator",
     "dynamic site",
     "dotJS",
     "hugo",
+    "owncast",
     "let's encrypt",
     "mkcert",
     "automatic",

test/index.js

@@ -352,11 +352,19 @@ test('[site.js] dynamic route loading from routes.js file', async t => {
 
   const routerStack = site.app._router.stack
 
-  const getRouteWithParameter = routerStack[12].route
+  const regularDotJSRoute1 = routerStack[11].route
+  t.true(regularDotJSRoute1.methods.get, 'request method should be GET')
+  t.strictEquals(regularDotJSRoute1.path, '/other-route-that-should-be-loaded', 'path should be correct and contain parameter')
+
+  const regularDotJSRoute2 = routerStack[12].route
+  t.true(regularDotJSRoute2.methods.get, 'request method should be GET')
+  t.strictEquals(regularDotJSRoute2.path, '/sub-route-that-should-be-loaded', 'path should be correct and contain parameter')
+
+  const getRouteWithParameter = routerStack[14].route
   t.true(getRouteWithParameter.methods.get, 'request method should be GET')
   t.strictEquals(getRouteWithParameter.path, '/hello/:thing', 'path should be correct and contain parameter')
 
-  const wssRoute = routerStack[13].route
+  const wssRoute = routerStack[15].route
   t.true(wssRoute.methods.get, 'request method should be GET (prior to WebSocket upgrade)')
   t.strictEquals(wssRoute.path, '/echo/.websocket', 'path should be correct and contain parameter')