mongodb #2721
Replies: 15 comments 68 replies
-
|
@xofyarg reported an issue on the unenv repo which @pi0 fixed. It should be integrated in unenv with cloudflare/workers-sdk#6920 |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
Working on this - #3315 |
Beta Was this translation helpful? Give feedback.
-
|
it's been forever now... I can't remember when I encountered this the first time but it must be at least 2 years ago. thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Working on this - #3315 |
Beta Was this translation helpful? Give feedback.
-
thank you! this way we can use cloudflare AI on cloudflare workers and still use our mongodb atlas instance... it will save us a massive amount of time and energy. I assume that will be the same with a lot of other people once they realize the September atlas data API deprecation deadline is approaching ... anyway, thanks again! immensely appreciated! |
Beta Was this translation helpful? Give feedback.
-
|
I saw the thread on the forum which says that the only problem is that workers don't support TLS |
Beta Was this translation helpful? Give feedback.
-
|
I've started a new Mongodb instance, and I can't use it with my Cloudflare workers; I want to use a Cloudflare environment with queues, kv, etc., but I can't use it if workers don't actually work... |
Beta Was this translation helpful? Give feedback.
-
|
December 6th |
Beta Was this translation helpful? Give feedback.
-
|
Amazing, tls support for workers whereby native mongodb drivers would work! that would be such a blessing! |
Beta Was this translation helpful? Give feedback.
-
|
@anonrig this is great news! |
Beta Was this translation helpful? Give feedback.
-
|
any movement on this @anonrig |
Beta Was this translation helpful? Give feedback.
-
|
I've opened an in progress PR - #3315 |
Beta Was this translation helpful? Give feedback.
-
|
TLSSocket and connect() has landed into main. It will be available to the general public in the next couple of weeks. |
Beta Was this translation helpful? Give feedback.
-
|
How can one use mongodb on Workers, now that node:net is merged? |
Beta Was this translation helpful? Give feedback.
-
|
Great to know, thanks! |
Beta Was this translation helpful? Give feedback.
-
|
FYi, this code works on bleeding edge (available in 5-10 days)
|
Beta Was this translation helpful? Give feedback.
-
|
Perfect, should be just updating the worker library, then. Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Is there any where public these kind of releases can be tracked? |
Beta Was this translation helpful? Give feedback.
-
We'll give an update here when the changes are deployed. |
Beta Was this translation helpful? Give feedback.
-
|
As promised, here is an update: workerd now implements wrangler 3.106 which was just released contains the latest version of Meaning that you should be able to use Please test and report your experiences! 🙏 edit: You need to use the I have tested with: # wrangler.toml
compatibility_date = "2024-12-05"
compatibility_flags = ["nodejs_compat"] |
Beta Was this translation helpful? Give feedback.
-
update: everything was perfect! |
Beta Was this translation helpful? Give feedback.
-
|
@JetLua Ran your minimal repro, mine's working on a deployed worker
|
Beta Was this translation helpful? Give feedback.
-
|
Sorry, everything was perfect! Thank you. |
Beta Was this translation helpful? Give feedback.
-
|
Working for me, including with Mongoose! Mongoose just needs a quick patch to add diff --git a/package.json b/package.json
index 46468fcb6a56ecd494c5a590b2940746a29c57ba..29a1a366f9e2adbdac15ee3dc3d43648d40eb3cd 100644
--- a/package.json
+++ b/package.json
@@ -112,6 +112,15 @@
},
"main": "./index.js",
"types": "./types/index.d.ts",
+ "exports": {
+ ".": {
+ "workerd": "./index.js",
+ "node": "./index.js",
+ "browser": "./dist/browser.umd.js",
+ "types": "./types/index.d.ts",
+ "default": "./index.js"
+ }
+ },
"engines": {
"node": ">=16.20.1"
},(Otherwise, it’ll import the browser bundle. See Automattic/mongoose#14613) |
Beta Was this translation helpful? Give feedback.
-
How did you make it work? Do I need to do something or just wait for Mongoose to patch it? |
Beta Was this translation helpful? Give feedback.
-
|
/cc @alexbevi ☝️ |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for flagging @vicb. I've been following along closely and looking forward to testing this out now that the latest |
Beta Was this translation helpful? Give feedback.
-
|
The moment i finally give up and switch to D1 from mongo you release THIS. But big thank you for finally doing this!! |
Beta Was this translation helpful? Give feedback.
-
|
Same |
Beta Was this translation helpful? Give feedback.
-
|
how is your experienced moving to d1 from mongo? d1 is sql i believe. must be low latency for all over the world? |
Beta Was this translation helpful? Give feedback.
-
|
I want to raise another related issue. First off there is a workaround see: mongodb polyfill Another important point: @alexbevi, seeing as you are from Mongodb, do you have any insight about this error, have you encountered it? The Finally I'm not the only one that is getting these errors, see polyfill issue 4 |
Beta Was this translation helpful? Give feedback.
-
|
I was wrong, this bug has everything to do with the polyfill. But the good news is that I fixed it, and I am now able to use MongoDb Atlas from my Cloudflare worker in preview and production. I have made a pull-request in the polyfill's repo with changes, waiting for it to be reviewed and merged. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for staying on top of this @equalogix. I haven't had a chance to test the polyfills again (hopefully can do that soon), but FYI #3594 is currently being worked on as well (which should hopefully be the last piece of the puzzle to getting this working natively in workerd) |
Beta Was this translation helpful? Give feedback.
-
|
Amazing work guys, this will be one less reason to leave MongoDB due to the deprecation of the DATA API, hoping for documentation around this as well, or a blog post to help us newbs. |
Beta Was this translation helpful? Give feedback.
-
|
If everything's working fine in a production deployed worker, can we kindly have a blog or official documentation on it to so that googling "how to use mongodb with Cloudflare worker" shows that article and not GH discussion? 😊🙏🏻 Thanks for all your amazing work everyone. |
Beta Was this translation helpful? Give feedback.
-
|
You need to explicitly close the connection when you are done with it with |
Beta Was this translation helpful? Give feedback.
-
|
Yes was thinking the same.. I currently use lamba functions with mongdb, and functions last about 20-30 seconds, subsequent connections use the existing connection and provide super fast 10ms connectivity... pooling is out of my technical expertise, but would be great if this supports cloudflare hyperdrive which keeps connections open (currently only works with sql i believe) |
Beta Was this translation helpful? Give feedback.
-
|
@ricky11 , been testing this with a Mongo Atlas (Flex) db. const client = new MongoClient(uri, {
maxPoolSize: 1,
minPoolSize: 0,
serverSelectionTimeoutMS: 5000,
maxIdleTimeMS: 10000, // Close idle connections after 10 seconds
} as MongoClientOptions);but I would love to hear some expertise around the architecture of mongo connections and how Atlas handles that. |
Beta Was this translation helpful? Give feedback.
-
|
CF Pages returns a build error from one of my functions.. I have the flags set correctly in wrangler.toml within my functions directory. Works beautifully locally with wrangler pages dev ./dist, but when pushed to cf, pages fails to build the project. Any ideas?
|
Beta Was this translation helpful? Give feedback.
-
|
Found that this is because pages uses ⛅️ wrangler 3.101.0, now i need to see how i can modify the version in pages. |
Beta Was this translation helpful? Give feedback.
-
|
Has anyone gotten this to work with Pages Functions as oppose to native workers.. I think CF Pages Functions use a different version of wrangler which still fails. |
Beta Was this translation helpful? Give feedback.
-
|
cc @vicb |
Beta Was this translation helpful? Give feedback.
-
|
Found out that pages functions is pinned to 3.101, what we need is 3.106. I was asking the guys on discord too.
|
Beta Was this translation helpful? Give feedback.
-
|
Starting a new thread for issues with connection pooling. We’d like to re-use connections across invocations, because otherwise we add a ~1500ms cold start to every invocation. However, @gerhardcit noted that you can’t re-use connection pools (created by This isn’t that informative, because the actual issue is that a deeper exception gets caught, which causes the promises to not resolve. Firing up the debugger and breaking on caught exceptions gives us: Okay, so it seems like a non-starter to actually re-use a connection in a Worker. Instead, let’s try use a singleton Durable Object to manage the connection pool. Durable Objects shut down about a minute after their last client disconnects, which should be enough to keep a connection pool around across multiple requests. I added this to {
"durable_objects": {
"bindings": [
{
"name": "MONGO_DB_CONNECTION",
"class_name": "MongoDBConnection"
}
]
},
"migrations": [
{
"tag": "v1",
"new_sqlite_classes": ["MongoDBConnection"]
}
]
}My export { MongoDBConnection } from "./MongoDBConnection.js";
export default {
async fetch(request, { MONGO_DB_CONNECTION, MONGO_DB_URI }) {
// FWIW, you’ll probably want to do an auth check or similar here, to prevent wasting resources on fuzzing attacks
// We can just forward the request straight onto the Durable Object
const id = MONGO_DB_CONNECTION.idFromName(MONGO_DB_URI);
const connection = MONGO_DB_CONNECTION.get(id, { locationHint: "apac" });
return await connection.fetch(request);
}
};Then import { DurableObject } from "cloudflare:workers";
import mongoose from "mongoose";
export class MongoDBConnection extends DurableObject {
connection;
constructor(ctx, env) {
super(ctx, env);
this.connection = mongoose.createConnection(env.MONGO_DB_URI, {
// Decrease the selection timeout from 30s (default) to 5s, so that if something breaks, we don't wait around for it.
serverSelectionTimeoutMS: 5000,
// Time out idle connections after ~20 seconds, to avoid using up too much wall time (tweak this as needed for your use case)
maxIdleTimeMS: 20 * 1000,
});
}
async fetch(request) {
// Wait for the connection to become available
await this.connection.asPromise();
// Do work here
return new Response("OK");
}
}This code just transparently forwards all requests straight onto the DO, which can hold onto the connection pool across multiple requests and re-use it. My end-to-end timings went from ~2000ms to ~300ms after doing this! |
Beta Was this translation helpful? Give feedback.
-
|
@raininglemons fairly certain it was 6.15.0 |
Beta Was this translation helpful? Give feedback.
-
You can't pass javascript Objects/function back and forth between DO and workers. only json serializable data can be passed |
Beta Was this translation helpful? Give feedback.
-
|
Ah @Imtiajrex that's it. I've misunderstood DO. I was trying to instantiate the client, then pass it to the calling worker. So really what I need to do is actually proxy all calls to the client through the DO, then return the response after it's resolved. Hero, thanks |
Beta Was this translation helpful? Give feedback.
-
|
You can pass any structuredclone-able JS type between Workers and Durable Objects |
Beta Was this translation helpful? Give feedback.
-
|
Let me know if any of you figure out how to deal with this: 
All of my requests return quite quickly, no issues, but because the Durable Object hangs around I get 2 issues with Workers Logs. First is, as above, the wall times for all of my invocations look much longer than I’d expect them to report. And second, the logs for the Durable Objects don’t show up in Workers Logs (either real-time or stored) until the Durable Object goes idle, which can take minutes. It makes debugging annoying. Evidently, the cause of this is the long timeout on the MongoDB connection, but I don’t want to shorten that. |
Beta Was this translation helpful? Give feedback.
-
|
someone can tell me how to make a connection between cloudflare worker with mongoDB Atlas? i don't know how to get public IP of cloudflare worker to be added into mongoDB Atlas |
Beta Was this translation helpful? Give feedback.
-
|
I dont think CF will give you a fixed IP, unless I missed that part, but you will have to open to 0.0.0.0 which will allow all access. |
Beta Was this translation helpful? Give feedback.
and others
-
The
mongodbdoes not work even withnode_compat_v2on.It can be imported, but it does not properly connect to a DB. The package relies on
net.Socketandtls.TLSSocket.See Issue for more context: cloudflare/workers-sdk#6684
See Blog for more context: https://alexbevi.com/blog/2024/09/11/why-cloudflare-workers-dont-work-with-mongodb/
Beta Was this translation helpful? Give feedback.
All reactions