admin can list users & admins; replaces user signup with request for invite (#115)

* lists users & admins, generates & validates admin invites; replaces user signup with request for invite

* Fixes bug that prevented using a proxy for HTTPS termination

* OAuth stores parameters in session, rather than passing to client & back

* Allows cross-platform passkeys, cross-origin access to storage, & user selection of grant duration

* Corrects caching headers and improves error messages when session expires

* Refactors contactURLToLink, assembleContactURL & protocol options into new protocols module w/ planned configurability

* Adds robots.txt file to discourage crawling

* Logging casts a wider net when extracting username

* Adds passkey logo everywhere passkeys are mentioned

* Logged-in user can invite themself to create another passkey

* npm audit fix

* Adds rate-limiting to defend against buggy and compromised clients

* Implements /.well-known/change-password as redirect, for password managers

Author: DougReeder

DEVELOPMENT.md

@@ -3,14 +3,17 @@
 ## Setup
 
 1. Run `git clone https://github.com/remotestorage/armadietto.git` to pull the code.
-2. Run `yarn install` or `npm install`. to install the dependencies.
+2. Run `npm install` to install the dependencies.
+3. Register for S3-compatible storage with your hosting provider, install [a self-hosted implementation](notes/S3-store-router.md), or use the public account on `play.min.io` (which is slow, and to which anyone in the world can read and write!).
 
 ## Development
 
-* Run `npm test` to run the automated tests. 
-* Run `npm run dev` to start a server on your local machine, and have it automatically restart when you update a source code file in `/lib`.
+* Run `npm test` to run the automated tests for both monolithic and modular servers (except the tests for S3 store router).
+* Set the S3 environment variables and run Mocha with `./node_modules/mocha/bin/mocha.js -u bdd-lazy-var/getter spec/armadietto/storage_spec.js` to test the S3 store router using your configured S3 server. (If the S3 environment variables aren't set, the S3 router uses the shared public account on play.min.io.) If any tests fail, add a note to [the S3 notes](notes/S3-store-router.md)
+* Run `npm run modular` to start a modular server on your local machine, and have it automatically restart when you update a source code file in `/lib`.
+* Run `npm run dev` to start a monolithic server on your local machine, and have it automatically restart when you update a source code file in `/lib`.
 
-Set the environment `DEBUG` to enable verbose logging of HTTP requests.
+Set the environment `DEBUG` to enable verbose logging of HTTP requests. For the modular server, these are the requests to the S3 server. For the monolithic server, these are the requests to Armadietto.
 
 Add automated tests for any new functionality. For bug fixes, start by writing an automated test that fails under the buggy code, but will pass when you've written a fix. Using TDD is not required, but will help you write better code.
 

README.md

@@ -24,46 +24,24 @@ This is a complete rewrite of [reStore](https://github.com/jcoglan/restore).
 
 See the `notes` directory for configuring a reverse proxy and other recipes.
 
-1. Run `armadietto -e` to see a sample configuration file.
-2. Create a configuration file at `/etc/armadietto/conf.json` (or elsewhere). See below for values and their meanings.
-3. Run `armadietto -c /etc/armadietto/conf.json`
+### Modular (new) Server
 
-To see all options, run `armadietto -h`. Set the environment `DEBUG` to log the headers of every request.
+* Streaming storage (documents don't have to fit in server memory)
+* S3-compatible storage (requires separate S3 server; AWS S3 allows documents up to 5 TB)
+* Can run multiple application servers to increase capacity to enterprise-scale
+* Bug Fix: correctly handles If-None-Match with ETag
+* Bug Fix: returns empty listing for nonexistent folder
+* Implements current spec: draft-dejong-remotestorage-22
 
-## Use as a library
+See [the modular-server-specific documentation](./modular-server.md) for usage.
 
-The following Node script will run a basic server:
+### Monolithic (old) Server
 
-```js
-process.umask(077);
-
-const Armadietto = require('armadietto');
-store   = new Armadietto.FileTree({path: 'path/to/storage'}),
-
-server  = new Armadietto({
-  store:  store,
-  http:   {host: '127.0.0.1', port: 8000}
-});
-
-server.boot();
-```
-
-The `host` option is optional and specifies the hostname the server will listen
-on. Its default value is `0.0.0.0`, meaning it will listen on all interfaces.
+* Stores user documents in server file system
+* More thoroughly tested
+* Implements older spec: draft-dejong-remotestorage-01
 
-The server does not allow users to sign up, out of the box. If you need to allow
-that, use the `allow.signup` option:
-
-```js
-var server = new Armadietto({
-  store: store,
-  http:  { host: '127.0.0.1', port: 8000 },
-  allow: { signup: true }
-});
-```
-
-If you navigate to `http://localhost:8000/` you should then see a sign-up link
-in the navigation.
+See [the monolithic-server-specific documentation](./monolithic-server.md) for usage.
 
 ## Storage security
 
@@ -159,45 +137,6 @@ setup, you can set `https.force = true` but omit `https.port`; this means
 armadietto itself will not accept encrypted connections but will apply the above
 behaviour to enforce secure connections.
 
-## Storage backends
-
-armadietto supports pluggable storage backends, and comes with a file system
-implementation out of the box (redis storage backend is on the way in
-`feature/redis` branch):
-
-* `Armadietto.FileTree` - Uses the filesystem hierarchy and stores each item in its
-  own individual file. Content and metadata are stored in separate files so the
-  content does not need base64-encoding and can be hand-edited. Must only be run
-  using a single server process.
-
-All the backends support the same set of features, including the ability to
-store arbitrary binary data with content types and modification times.
-
-They are configured as follows:
-
-```js
-// To use the file tree store:
-const store = new Armadietto.FileTree({path: 'path/to/storage'});
-
-// Then create the server with your store:
-const server = new Armadietto({
-  store:  store,
-  http:   {port: process.argv[2]}
-});
-
-server.boot();
-```
-
-## Lock file contention
-
-The data-access locking mechanism is lock-file based.
-
-You may need to tune the lock-file timeouts in your configuration:
-
-- *lock_timeout_ms* - millis to wait for lock file to be available
-- *lock_stale_after_ms* - millis to wait to deem lockfile stale
-
-To tune run the [hosted RS load test](https://overhide.github.io/armadietto/example/load.html) or follow instructions in [example/README.md](example/README.md) for local setup and subsequently run [example/load.html](example/load.html) off of `npm run serve` therein.
 
 ## Debugging an installation
 
@@ -211,8 +150,8 @@ See `DEVELOPMENT.md`
 
 (The MIT License)
 
-Copyright (c) 2012-2015 James Coglan  
-Copyright (c) 2018 remoteStorage contributors
+Copyright © 2012–2015 James Coglan
+Copyright © 2018–2024 remoteStorage contributors
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of
 this software and associated documentation files (the 'Software'), to deal in

bin/www

@@ -4,11 +4,13 @@ const http = require('http');
 const fs = require("fs");
 const path = require("path");
 const {ArgumentParser} = require("argparse");
+const { stat } = require('node:fs/promises');
 const appFactory = require('../lib/appFactory');
 const {configureLogger, getLogger} = require("../lib/logger");
-const S3Handler = require("../lib/routes/S3_store_router");
+const S3StoreRouter = require("../lib/routes/S3_store_router");
 const process = require("process");
 const https = require("https");
+const errToMessages = require("../lib/util/errToMessages");
 
 const SSL_CIPHERS = 'ECDHE-RSA-AES256-SHA384:AES256-SHA256:RC4-SHA:RC4:HIGH:!MD5:!aNULL:!EDH:!AESGCM';
 const SSL_OPTIONS = require('crypto').constants.SSL_OP_CIPHER_SERVER_PREFERENCE;
@@ -51,23 +53,23 @@ if (!hostIdentity) {
 const userNameSuffix = conf.user_name_suffix ?? '-' + hostIdentity;
 
 if (conf.http?.port) {
-  start( Object.assign({}, conf.http, process.env.PORT && {port: process.env.PORT}));
+  start( Object.assign({}, conf.http, process.env.PORT && {port: process.env.PORT})).catch(getLogger.error);
 }
 
 if (conf.https?.port) {
-  start(conf.https);
+  start(conf.https).catch(getLogger.error);
 }
 
 
-function start(network) {
-  // If the environment variables aren't set, s3handler uses a shared public account on play.min.io,
+async function start(network) {
+  // If the environment variables aren't set, s3storeRouter uses a shared public account on play.min.io,
   // to which anyone in the world can read and write!
-  // It is not entirely compatible with S3Handler.
-  const s3handler = new S3Handler({endPoint: process.env.S3_ENDPOINT,
+  // It is not entirely compatible with S3StoreRouter.
+  const s3storeRouter = new S3StoreRouter({endPoint: process.env.S3_ENDPOINT,
     accessKey: process.env.S3_ACCESS_KEY, secretKey: process.env.S3_SECRET_KEY, region: process.env.S3_REGION || 'us-east-1',
     userNameSuffix});
 
-  const app = appFactory({hostIdentity, jwtSecret, account: s3handler, storeRouter: s3handler, basePath});
+  const app = await appFactory({hostIdentity, jwtSecret, accountMgr: s3storeRouter, storeRouter: s3storeRouter, basePath});
 
   const port = normalizePort( network?.port || '8000');
   app.set('port', port);
@@ -106,7 +108,7 @@ function start(network) {
 
   server.listen(port);
   server.on('error', onError);
-  server.on('clientError', clientError);
+  server.on('clientError', clientError); // a client connection emitted an 'error' event
   server.on('listening', onListening);
 
   /** Event listener for HTTP server "error" event. */
@@ -212,10 +214,11 @@ function clientError (err, socket) {
     status = 400;
     message = 'Bad Request';
   }
-  getLogger().warning(`${socket.address().address} n/a n/a ${status} ${message} ${err.toString()}`);
+  const logNotes = errToMessages(err, new Set([message]));
+  getLogger().warning(`- - - - - ${status} - “${Array.from(logNotes).join(' ')}”`);
 
   if (err.code !== 'ECONNRESET' && socket.writable) {
     socket.end(`HTTP/1.1 ${status} ${message}\r\n\r\n`);
   }
-  socket.destroy(err);
+  socket.destroySoon();
 }

contrib/systemd/armadietto.service

@@ -11,7 +11,7 @@ Restart=always
 RestartSec=1
 User=armadietto
 Group=armadietto
-Environment=
+Environment=NODE_ENV=production
 ExecStartPre=
 ExecStart=/usr/bin/armadietto -c /etc/armadietto/conf.json
 ExecStartPost=