MongoDB and Meteor upgrades by mnutt · Pull Request #3745 · sandstorm-io/sandstorm

mnutt · 2026-02-05T04:00:03Z

This has a ways to go to get ready for any sort of release, but I wanted to put it out there for discussion.

For anyone looking at the code, the interesting parts are:

feat: add MongoDB 7 migration support - if something is to go wrong in this process, it'll be here
upgrade Meteor to 2.16 with MongoDB driver 4.x compatibility - this meteor upgrade is actually quite minor and unlikely to cause many issues. There were a few things that needed shims and the bignum library didn't work and had to be replaced with native BigInt.

I wouldn't throw it straight into production before more testing. But if you would like to try it yourself:

sandstorm stop
sandstorm update sandstorm-0.tar.xz
sandstorm migrate-mongo
sandstorm start

If you later feel like rolling back:

sandstorm stop
sandstorm rollback-mongo
sandstorm update 309
sandstorm start

xet7 · 2026-02-05T05:43:02Z

Can this migrate WeKan MongoDB also to MongoDB 7 ?

ocdtrekkie · 2026-02-05T06:25:10Z

@xet7 It's likely a similar sort of script could be built for apps, but this might actually be a bad idea. MongoDB apparently has a minimum opLog file size of 990 MB now, which would make each Wekan grain enormous. (Our MongoDB 2.6 install had a 16 MB opLog file.) Sandstorm servers taking an extra gigabyte of space wouldn't be a huge blocker, but I think individual grains taking an extra gigabyte of space would be a big problem.

- Upgrade Meteor from 2.3.5 to 2.16 - Add findAndModify polyfill for MongoDB driver 4.x (replaces fongandrew:find-and-modify) - Update aggregate() calls to use cursor API (no callback) - Replace bignum package with native BigInt for IPv6 handling - Add skipLibCheck to tsconfig.json - Update meteor-testapp to Meteor 2.16

Add infrastructure to migrate Sandstorm from MongoDB 2.6 to MongoDB 7.0: - Bundle MongoDB 7.0, mongosh, and database tools alongside MongoDB 2.6 - Add version detection via /var/mongo/version file - Add 'sandstorm migrate-mongo' command to migrate database - Add 'sandstorm rollback-mongo' command to revert if issues occur - Migration exports data with mongodump, reinitializes with MongoDB 7, imports with mongorestore, and sets up replica set for oplog tailing

- Upgrade Nightwatch from 0.9 to 2.6 with native ChromeDriver support - Replace selenium-standalone with chromedriver npm package - Update nightwatch.json for WebDriver protocol and headless Chrome - Configure tests to use MongoDB 7 for fresh installs - Fix async test patterns (Meteor.logout, file uploads, download handling) - Improve loginDevAccount reliability with Meteor connection checks - Update unit tests to use async/await

- Update Chrome version from 114 to 142 for test compatibility - update lint problem count

ocdtrekkie · 2026-02-12T03:16:49Z

@troyjfarrell @orblivion Tag for review, note that run-bundle.c++ gets minimized in diff view, but it's where most of the magic is.

Michael-S · 2026-02-21T02:35:32Z

mongo-migration.log
I tried it with my install, this was the the end of the output:

** Stopping old MongoDB...
** Backing up old data directory...
** Starting MongoDB 7.0...
** Initializing replica set...
{ ok: 1 }
** Waiting for replica set election...
** Creating database user...
MongoServerError: not primary
** Migration failed! Rolling back...
** Check <sandstorm-install>/var/log/mongo-migration.log for details.
** Rollback complete. Original data restored.
*** Uncaught exception ***
sandstorm/util.c++:721: failed: expected exitCode == 0 [1 == 0]; child process failed; name = ; exitCode = 1
stack: 4096b1 639255 654394 653b1c 495cb6 4960f9

Edit: I don't think my mongo-migration.log has anything security-related, so I've attached it.
Edit 2: Now I attached it. The first time, attachment failed because I forgot to chown it.

orblivion · 2026-02-23T01:54:34Z

If you later feel like rolling back:
...
sandstorm update 309

Shouldn't this be 308?

mnutt · 2026-03-07T18:05:43Z

d00d487 adds what I think may be a bit better way to wait until mongo is ready. If that doesn't work reliably I think we should consider just adding some timed wait.

ocdtrekkie · 2026-03-07T20:48:56Z

@Michael-S Are you in a position to see if the latest commit resolves your issue?

Michael-S · 2026-03-07T21:19:55Z

@Michael-S Are you in a position to see if the latest commit resolves your issue?

I'll try it, I should be able to report back within a few days.

Michael-S · 2026-03-07T23:17:56Z

It worked without errors this time, thank you! 🎉

EDIT: I don't know how representative my timeout problem was for medium size or larger Sandstorm installs. My sandstorm server has /opt/ mounted on 7200RPM HDDs in RAID 1, with /opt/sandstorm about 230GB. I have 140 Grains. sudo find /opt/sandstorm -type f | wc -l gives 400,000 files total.

troyjfarrell

I'm really pleased that you've taken on this task. The code looks good, insofar as I'm qualified to review it.

troyjfarrell · 2026-03-26T05:49:36Z

+        KJ_SYSCALL(execl(binary.cStr(), binary.cStr(),
+            "--bind_ip", "127.0.0.1", "--port", portStr.cStr(),
+            "--dbpath", dbpath.cStr(),
+            "--replSet", "ssrs", "--oplogSize", "16",


Assuming that "ssrs" is Sandstorm Replication Set? (I don't think that it matters.)

I assume so, I was following

sandstorm/src/sandstorm/run-bundle.c++

Line 2258 in 231077a

"--replSet", "ssrs", "--oplogSize", "16",

.

troyjfarrell · 2026-03-26T06:17:58Z

Are any licensing implications since this change entails packaging MongoDB 7? Have we discussed that?

ocdtrekkie · 2026-03-26T06:35:52Z

@troyjfarrell We arguably need to update the about page to indicate the license for MongoDB is SSPL, but SSPL has no serious implications for us, as Sandstorm's use of MongoDB does not provide MongoDB as a service and therefore it effectively functions as GPL3 for us. (There is technically a purity-test fact we are no longer using OSI-approved licensed software there, but between the fact it is effectively GPL3 for us, and it puts our otherwise legacy software in a maintainable state... I am not super worried about it.)

troyjfarrell · 2026-03-26T06:54:48Z

I agree that in practical terms, it's basically GPLv3. We should disclose the SSPL license of the included binaries on the downloads page in the event that someone does not want to run it because it's not OSI approved. Because we aren't exposing MongoDB to apps or end-users, it seems that it doesn't affect the license of our code as I understand it.

Handle versions we don't know about yet, so that we don't treat them like old versions. Be more strict about not migrating or launching with versions we don't understand.

troyjfarrell

@mnutt I appreciate the refactoring. It looks good.
@ocdtrekkie Thanks for this licensing note.

ocdtrekkie reviewed Feb 5, 2026

View reviewed changes

Comment thread .github/workflows/build.yml Outdated

ocdtrekkie reviewed Feb 5, 2026

View reviewed changes

Comment thread shell/tsconfig.json Outdated

ocdtrekkie reviewed Feb 5, 2026

View reviewed changes

Comment thread src/sandstorm/run-bundle.c++ Outdated

mnutt added 5 commits February 5, 2026 17:21

ci: update Chrome to v142 and update lint count check

e6986a4

- Update Chrome version from 114 to 142 for test compatibility - update lint problem count

replace deprecated nightwatch calls

0ecacee

mnutt force-pushed the mongo-upgrade-clean branch from 411cc97 to 0ecacee Compare February 5, 2026 22:25

mnutt requested a review from ocdtrekkie February 5, 2026 22:26

ocdtrekkie approved these changes Feb 7, 2026

View reviewed changes

mnutt marked this pull request as ready for review February 14, 2026 16:18

ocdtrekkie added sandstorm-dev Issues hacking on Sandstorm install-config Installation/configuration issues ready-for-review We think this is ready for review labels Feb 14, 2026

orblivion reviewed Feb 23, 2026

View reviewed changes

Comment thread make-bundle.sh Outdated

ocdtrekkie reviewed Feb 23, 2026

View reviewed changes

Comment thread make-bundle.sh Outdated

mnutt added 3 commits February 23, 2026 22:01

use dl.sandstorm.org

9b66ba6

use real mongo7 checksum

cb440e0

better waiting for mongo to be ready

d00d487

troyjfarrell suggested changes Mar 26, 2026

View reviewed changes

mnutt and others added 8 commits March 26, 2026 10:21

better mongo version checking

1464462

Handle versions we don't know about yet, so that we don't treat them like old versions. Be more strict about not migrating or launching with versions we don't understand.

stricter log redirection

12fea35

Safer curl downloading during bundling

4fe8c11

extract sha256 helper and ensure binary mode flag

8bf52e3

KJ_SYSCALL does not return a value

e16dea7

fix curl options

6d6c9ff

ensure copied file gets original's permissions

57dce21

Update MongoDB license from AGPL to SSPL

026f0d7

troyjfarrell approved these changes Mar 27, 2026

View reviewed changes

ocdtrekkie merged commit fbe5841 into sandstorm-io:dev Mar 27, 2026
1 check passed

ocdtrekkie removed the ready-for-review We think this is ready for review label Mar 27, 2026

ocdtrekkie mentioned this pull request Jun 5, 2026

Update mongodb #3235

Closed

Uh oh!

Uh oh!

Conversation

mnutt commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

xet7 commented Feb 5, 2026

Uh oh!

ocdtrekkie commented Feb 5, 2026

Uh oh!

ocdtrekkie commented Feb 12, 2026

Uh oh!

Michael-S commented Feb 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

orblivion commented Feb 23, 2026

Uh oh!

Uh oh!

Uh oh!

mnutt commented Mar 7, 2026

Uh oh!

ocdtrekkie commented Mar 7, 2026

Uh oh!

Michael-S commented Mar 7, 2026

Uh oh!

Michael-S commented Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

troyjfarrell left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

troyjfarrell Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

mnutt Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

troyjfarrell commented Mar 26, 2026

Uh oh!

ocdtrekkie commented Mar 26, 2026

Uh oh!

troyjfarrell commented Mar 26, 2026

Uh oh!

troyjfarrell left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

mnutt commented Feb 5, 2026 •

edited

Loading

Michael-S commented Feb 21, 2026 •

edited

Loading

Michael-S commented Mar 7, 2026 •

edited

Loading