You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The INI settings are not in the backup, and what to do about it
Costing the "move the INI settings into the database" question turned up something
that stands on its own, so it is recorded before any of the work is done.
Configuration::XMLStore writes its <Properties> node by iterating PropertySet, and
PropertySet is the hm_settings table. So the 126 [Settings] values in hMailServer.INI
are in no backup archive at all - MTA-STS, DANE, ACME, TLS-RPT, greylisting intervals,
thread counts, rate limits, every setting this fork has added.
Back up a server, rebuild the machine, restore: domains, accounts, mail and the
database-held settings all come back, and everything else silently returns to its
default. Nothing reports it, because from the restore's point of view nothing failed.
The same-machine backend migration in docs\MigratingDatabaseBackend.md is unaffected -
the INI never moves in that procedure - but "the backup contains the settings" is not
true as stated, and a machine rebuild is a thing people do.
WHY THE ANSWER IS THE MIGRATION RATHER THAN A BACKUP PATCH
Teaching BackupExecuter to read hMailServer.INI would close the backup hole and leave
the other two open. An INI setting is invisible to remote administration - the Control
Panel's IniFeatureStore reads a file that only exists on the server, which is exactly
why the log-folder card added yesterday has to say "not readable from this machine" -
and none of the 126 appear in hmconfig.ps1's config-as-code either.
Settings that live in the database are backed up, restorable and remotely
administrable for one reason rather than three.
WHAT CAN AND CANNOT MOVE
Of 149 INI reads: 12 [Database] and 6 [Directories] cannot, because they are how the
server reaches the database and where it writes the message saying it could not.
AdministratorPassword cannot, because holding it in the database locks an operator out
of the tool they would use to diagnose a database outage. The two fault-injection
switches stay INI-only on purpose - they are deliberately unreachable over COM.
That leaves the 126 that an administrator actually wants to change.
THE DESIGN, INCLUDING THE PART THAT CHANGED WHEN THE DDL WAS READ
LoadSettings() stays INI-only and is called a second time after OpenDatabase;
ReadIniSettingString_/Integer_ prefer a database overlay for the [Settings] section
only. The bootstrap keys therefore cannot be overridden from the database they are
needed to reach, which answers the ordering question and the security question at
once, and re-running the existing loader means no per-setting registry.
A NEW TABLE, not a widening of hm_settings. settingname is nvarchar(30); five current
names are 31-40 characters (GreylistingEnabledDuringRecordExpiration is 40); and the
column is uniquely indexed on all four backends. Widening it therefore means dropping
and recreating a unique constraint on SQL CE - which commits DDL as it executes and
cannot roll back, on the backend the installer still chooses by default. CREATE TABLE
is safe everywhere and needs no index surgery. The cost is that the backup wiring
becomes about twenty explicit lines instead of coming for free, which is the better
side of that trade.
THE UPGRADE PATH, WHICH IS THE POINT FOR EXISTING INSTALLATIONS
On first start after the upgrade, any [Settings] key present in the INI with no row in
the database is copied in. Driven by "is there a row" rather than a version flag, so
it is idempotent, safe to interrupt and safe to re-run.
Nothing is deleted from the INI. Deleting an operator's configuration file during an
upgrade is how people stop trusting upgrades, and the file is also their record of
what the server used to do.
But a stale INI key whose value differs from the stored one is reported by name on
every start. Without that, somebody edits hMailServer.INI, nothing happens, and there
is no way to discover why - which is the same silent-ignore failure the three sweeps
this week spent fifty-one fixes removing. It is the part of this design that is not
optional.
Recorded rather than started: this is a multi-increment change across the schema, the
settings loader, the backup, COM and the Control Panel, and the reasoning is worth
more written down than a half-finished migration is in the tree.
Three things stand out and are worth naming rather than leaving to be inferred.
78
78
**Storage and the administration surface are the best-covered areas**, and the
@@ -677,7 +677,7 @@ the source, not from documentation.
677
677
| ✅ | Aliases, catch-all and plus-addressing | Per-domain aliases and domain aliases, a domain catch-all address, and per-domain plus-addressing with a configurable separator character. None of the address-resolution surface appears in the inventory. |
678
678
| ✅ | Automatic reconnect and statement retry | Only *connection* problems retry. `DALConnection::Execute` loops up to six times, reconnecting on attempts 2 and 4 and after any DALConnectionProblem with a 1-second pause - but it breaks out of the loop on any other result, so a statement rejected for a SQL reason is attempted exactly once. `DALRecordset::Open` hardcodes six tries and ignores `SetTryCount`. **Corrected 13 August 2026:** the reconnect branch never fired on PostgreSQL for the case it exists for. When the server closes the connection mid-statement libpq does not return a null result - it builds a PGRES_FATAL_ERROR result and marks the connection bad - so `PGConnection::CheckError`, classifying on result status alone, returned DALErrorInSQL and the statement was reported failed and dropped; only the *next* statement, which libpq refuses outright, triggered a reconnect. One statement lost per connection drop, silently. CheckError now consults `PQstatus`: a statement the server rejected leaves the connection CONNECTION_OK, one that got no answer does not. MySQL already recognised the same event through error codes 2006/2013. |
679
679
| ✅ | Background indexer thread with quick/full modes | Dedicated worker wakes every minute; a "quick" pass indexes only the newest IndexerQuickLimit (default 1000) messages, a "full" pass runs every IndexerFullMinutes (default 720) up to IndexerFullLimit (default 25000) rows. |
680
-
| ✅ | Backup and restore | BackupManager/BackupExecuter with a scheduled BackupTask, selectable components (settings, domains, messages), 7za compression, an optional messages-database-only mode (BackupMessagesDBOnly), and COM/Control Panel surfaces. A restore validates the archive and stages the message store before its first deletion (BackupRestorer::Prepare), so an archive that cannot satisfy the selected options is refused with the server untouched rather than deleting everything and finding it has nothing to put back. **Audited for honesty on 13 August 2026 - not "does it work" but "can it tell you when it did not" - because this is now documented as the database-migration mechanism and an operator's whole safety net is that a backup which reports success contains what they think it does.** It could not. `Configuration::XMLStore` called nine child stores for their side effects, discarded every answer and returned true, and `BackupExecuter` discarded that true as well: the settings half of a backup was structurally incapable of failing. `XMLLoad` beside it checks at every single step, which is what gives it away - the restore was hardened at some point and the backup was not. Both are checked now, and a settings section that cannot be written fails the backup with no archive left behind. Three more of the same shape: `SMTPConfiguration::XMLLoad` restored routes and incoming relays unchecked under a function that returned true regardless - the two collections `Configuration::XMLLoad`'s otherwise complete guard did not reach, and the two that decide where mail may go and who may relay through this server; `IMAPConfiguration` was unchecked in both directions for public folders and groups; and the public-folder `DeleteAll` in the restore path was unchecked one line below a domain `DeleteAll` that is checked, where the consequence is worse than the domains case because `XMLLoad` then puts the backup's folders in on top of whatever survived. Underneath all of them, `Collection<T,P>::XMLLoad` answered **true** when its opening `DeleteAll` failed and then loaded nothing, so a collection the restore could not clear was reported as restored - `BackupExecuter` guards the domains case from outside for exactly this reason, and the guard now lives where it covers every collection instead of the one that was noticed. |
680
+
| ✅ | Backup and restore | BackupManager/BackupExecuter with a scheduled BackupTask, selectable components (settings, domains, messages), 7za compression, an optional messages-database-only mode (BackupMessagesDBOnly), and COM/Control Panel surfaces. A restore validates the archive and stages the message store before its first deletion (BackupRestorer::Prepare), so an archive that cannot satisfy the selected options is refused with the server untouched rather than deleting everything and finding it has nothing to put back. **Audited for honesty on 13 August 2026 - not "does it work" but "can it tell you when it did not" - because this is now documented as the database-migration mechanism and an operator's whole safety net is that a backup which reports success contains what they think it does.** It could not. `Configuration::XMLStore` called nine child stores for their side effects, discarded every answer and returned true, and `BackupExecuter` discarded that true as well: the settings half of a backup was structurally incapable of failing. `XMLLoad` beside it checks at every single step, which is what gives it away - the restore was hardened at some point and the backup was not. Both are checked now, and a settings section that cannot be written fails the backup with no archive left behind. Three more of the same shape: `SMTPConfiguration::XMLLoad` restored routes and incoming relays unchecked under a function that returned true regardless - the two collections `Configuration::XMLLoad`'s otherwise complete guard did not reach, and the two that decide where mail may go and who may relay through this server; `IMAPConfiguration` was unchecked in both directions for public folders and groups; and the public-folder `DeleteAll` in the restore path was unchecked one line below a domain `DeleteAll` that is checked, where the consequence is worse than the domains case because `XMLLoad` then puts the backup's folders in on top of whatever survived. Underneath all of them, `Collection<T,P>::XMLLoad` answered **true** when its opening `DeleteAll` failed and then loaded nothing, so a collection the restore could not clear was reported as restored - `BackupExecuter` guards the domains case from outside for exactly this reason, and the guard now lives where it covers every collection instead of the one that was noticed. **A gap found on 14 August 2026 while costing the INI-settings migration, and it is a real one:** `Configuration::XMLStore` writes its `<Properties>` node by iterating `PropertySet`, which is the `hm_settings` table — so the **126 `[Settings]` values in hMailServer.INI are in no backup archive at all**. MTA-STS, DANE, ACME, TLS-RPT, greylisting intervals, thread counts, rate limits: every setting this fork has added. Back up, rebuild a machine, restore, and you get the domains, the accounts, the mail and the *old* database-held settings, while everything else silently falls back to its default. The same-machine backend migration in `docs\MigratingDatabaseBackend.md` is unaffected because the INI never moves there, but a machine rebuild is not, and "the backup contains the settings" is not true as stated. Closed by the INI-to-database migration below rather than by teaching the backup to read a file, because settings that live in the database are backed up, restorable and remotely administrable for the same single reason. |
681
681
| ✅ | Backup/restore of settings, domains and messages | Backup writes an XML document of business objects plus (optionally) the whole data directory, 7z-compressed; BackupMessagesDBOnly skips the files and keeps only the rows. Requires all message files to be inside the data folder. A restore refuses any category the archive does not contain — Collection::XMLLoad cannot tell "empty at backup time" from "absent from this archive", so restoring domains or settings from an archive without them used to empty them and report success — and refuses an archive written by a later hMailServer than the one running, which is what the index's Version attribute has been for since 2010. |
682
682
| ✅ | Backup of a live message store | The data-directory copy tolerates what a running server does underneath it: files deleted between enumeration and copy are counted and reported in the backup log, a destination file left by an earlier run is overwritten, and a file another process holds is retried within a bounded budget and then fails the backup by name. It does not use FileUtilities::CopyDirectory, whose throwing copy_file escaped the work-queue thread and terminated the service — which, with CompressDestinationFiles off, happened on the second backup an installation took. A backup is still not a snapshot: rows are read before files, so a message deleted in between is listed in the archive without its file, and the backup log says how many. |
683
683
| ✅ | Bound parameters on MSSQL/SQL CE only | ADO and SQL CE report GetSupportsCommandParameters()==true; MySQL and PostgreSQL report false, so their statements are rebuilt by literal interpolation with SQLStatement::Escape rather than server-side binding. |
@@ -798,10 +798,11 @@ the source, not from documentation.
798
798
799
799
### Administration, API and Control Panel
800
800
801
-
52 shipped · 0 underway · 8 not started · 0 deferred
801
+
52 shipped · 0 underway · 9 not started · 0 deferred
802
802
803
803
|| Capability | Detail |
804
804
|:-:|---|---|
805
+
| ⬜ | **The 126 `[Settings]` INI values move into the database** | Costed 14 August 2026. Of 149 INI reads, **126 are in `[Settings]` and should move**; 12 `[Database]`, 6 `[Directories]` and `AdministratorPassword` **cannot** — you need them to reach the database, `LogFolder` is where the "the database is down" message goes, and an administrator password held in the database locks you out of the tool you would use to diagnose a database outage. The two fault-injection switches stay INI-only deliberately. **The reason is not tidiness.** An INI setting is invisible to remote administration: `IniFeatureStore` reads a file that only exists on the server, which is why the Control Panel's own log-folder card has to say "not readable from this machine", and none of the 126 appear in `hmconfig.ps1`'s config-as-code. They are also **absent from every backup** (see the backup row above). One change closes all three. **Design.** `LoadSettings()` stays INI-only so the bootstrap still works, and is called a second time after `OpenDatabase`; `ReadIniSettingString_/Integer_` prefer a database overlay **only for the `[Settings]` section**, so the keys needed to reach the database can never be overridden from it — which is the security answer as well as the ordering one. No per-setting registry is needed because the existing loader is simply re-run. **A new table rather than widening `hm_settings`:** `settingname` is `nvarchar(30)`, five current names are 31–40 characters (`GreylistingEnabledDuringRecordExpiration` is 40), and the column is uniquely indexed on all four backends — so widening means dropping and recreating a unique constraint on **SQL CE, which commits DDL as it executes and cannot roll back**. `CREATE TABLE` is safe everywhere and needs no index surgery; the cost is that the backup wiring becomes ~20 explicit lines instead of free, which is the better trade. **Upgrade path**, which is the part that matters for existing installations: on first start, any `[Settings]` key present in the INI with no row in the database is copied in — idempotent, driven by "is there a row" rather than a version flag, so it is safe to interrupt and safe to re-run. Nothing is deleted from the INI, because deleting an operator's configuration during an upgrade is how people stop trusting upgrades. But a stale INI key that differs from the stored value is **reported by name on every start** — without that, somebody edits the INI, nothing happens, and there is no way to find out why, which is precisely the silent-ignore failure the August sweeps spent three passes removing. |
805
806
| ✅ | ACME http-01 challenge serving (/.well-known/acme-challenge/) | Serves key authorizations from the in-process AcmeChallengeStore; rejects tokens containing a slash or longer than 256 characters. Always enabled on the web-services listener (not gated behind a setting). |
806
807
| ✅ | Admin helper services | Active Directory account picker, DKIM RSA key-pair generator producing PEM + the DNS TXT p= value, password generator and strength meter, protected-secret storage, message-store consistency report parser. |
807
808
| ✅ | Advertised authentication method | Autoconfig hard-codes <authentication>password-cleartext</authentication> for every server block; there is no OAuth2 authentication element even though the server has an OAuth2 token validator… |
0 commit comments