EasyDB Server — Specification & Requirements

1. Vision

EasyDB Server is a MongoDB wire protocol-compatible server powered by EasyDB as its storage engine.

Any standard MongoDB client — mongosh, Node.js mongodb driver, pymongo, Compass, etc. — can connect to EasyDB Server and perform operations transparently. Under the hood, the data is stored using any EasyDB adapter: SQLite, PostgreSQL, MySQL, Redis, Memory, Cloudflare D1/KV, and more.

Primary Goals

Showcase EasyDB — Demonstrate that EasyDB's adapter abstraction is powerful enough to back a real database server.
Backend flexibility — The same MongoDB-compatible interface, with 10+ swappable storage backends.
Useful on its own — Lightweight MongoDB alternative for development, testing, edge deployments, and embedded use cases.

What This Is NOT

Not a full MongoDB replacement — no sharding, no replica sets, no aggregation pipeline.
Not aiming for 100% MongoDB compatibility — we support the most-used subset.
Not a production database for high-traffic workloads (at least not in v1).

2. Architecture

┌──────────────────────────────────────────────────┐
│                MongoDB Clients                    │
│  (mongosh, drivers, Compass, ORMs like Mongoose)  │
└───────────────────┬──────────────────────────────┘
                    │ TCP :27017
                    │ MongoDB Wire Protocol (OP_MSG / OP_QUERY)
                    │ BSON serialization
                    ▼
┌──────────────────────────────────────────────────┐
│              EasyDB Server                        │
│                                                   │
│  ┌─────────────┐  ┌──────────────┐  ┌──────────┐ │
│  │  Wire       │  │   Command    │  │  Query   │ │
│  │  Protocol   │──│   Router     │──│  Engine  │ │
│  │  Layer      │  │              │  │          │ │
│  └─────────────┘  └──────────────┘  └────┬─────┘ │
│                                          │       │
│  ┌───────────────────────────────────────┘       │
│  │                                               │
│  ▼                                               │
│  ┌──────────────────────────────────────────┐    │
│  │             EasyDB Core                   │    │
│  │   .put() .get() .where() .delete() etc.   │    │
│  └─────────────────┬────────────────────────┘    │
└────────────────────┼─────────────────────────────┘
                     │
    ┌────────────────┼────────────────────┐
    │                │                    │
    ▼                ▼                    ▼
┌────────┐   ┌──────────┐   ┌──────────────────┐
│ SQLite │   │ Postgres │   │ Memory / Redis / │
│        │   │  MySQL   │   │ D1 / KV / Turso  │
└────────┘   └──────────┘   └──────────────────┘

Layers

Layer	Responsibility
Wire Protocol	TCP listener, message framing, BSON parse/serialize, OP_MSG/OP_QUERY handling
Command Router	Extract command name from BSON document, dispatch to handler
Command Handlers	Implement each MongoDB command (find, insert, update, delete, etc.)
Query Engine	Translate MongoDB query operators ($gt, $eq, $in, etc.) to EasyDB queries + JS filters
EasyDB Core	Storage abstraction — unchanged, used as a dependency

3. Supported MongoDB Commands

3.1 Connection & Handshake (Priority: Critical)

These are required for ANY client to connect.

Command	Description	Notes
`hello`	Modern handshake (MongoDB 5.0+)	Report capabilities, wire version
`isMaster` / `ismaster`	Legacy handshake	Same response as hello
`buildInfo`	Server version info	Report as compatible version
`ping`	Health check	Return `{ok: 1}`
`whatsmyuri`	Client IP address	Return client socket address
`getParameter`	Server parameters	Return sensible defaults
`getCmdLineOpts`	Server CLI options	Stub response
`getLog`	Server log	Empty/stub response
`hostInfo`	Host machine info	OS/architecture info
`serverStatus`	Server statistics	Basic uptime, connections, ops counters

3.2 CRUD Operations (Priority: Critical)

Core data operations — the main value proposition.

Command	Maps To (EasyDB)	Notes
`insert`	`store.put()` / `store.putMany()`	Auto-generate `_id` (ObjectId) if missing
`find`	`store.where()` / `store.all()` + filter	Translate query operators
`update`	`store.get()` → modify → `store.put()`	Support $set, $unset, $inc, $push, $pull, $rename
`delete`	`store.delete()`	Support single and multi
`getMore`	Continue cursor iteration	Cursor management with timeouts
`killCursors`	Release cursor resources	Cleanup
`count` / `countDocuments`	`store.count()` / query `.count()`	Use fast-path when possible
`distinct`	`store.all()` → extract unique values	JS-side aggregation
`findAndModify`	`get()` → modify → `put()` → return	Atomic per-document

3.3 Database & Collection Management (Priority: High)

Needed for show dbs, show collections, use db, etc.

Command	Description	Implementation
`listDatabases`	List all databases	List EasyDB instances
`listCollections`	List collections in a database	`db.storeNames`
`create`	Create collection	Create EasyDB store
`drop`	Drop collection	Delete store
`dropDatabase`	Drop entire database	Destroy EasyDB instance
`listIndexes`	List indexes on a collection	Report EasyDB indexes
`createIndexes`	Create indexes	EasyDB schema indexes

3.4 Query Operators (Priority: High)

Translation layer from MongoDB query syntax to EasyDB operations.

Comparison (map to EasyDB .where() when possible, .filter() otherwise):

Operator	EasyDB Mapping	Native?
`$eq`	`.where(field, value)`	Yes
`$gt`	`.where(field).gt(value)`	Yes
`$gte`	`.where(field).gte(value)`	Yes
`$lt`	`.where(field).lt(value)`	Yes
`$lte`	`.where(field).lte(value)`	Yes
`$ne`	`.filter(doc => doc[field] !== value)`	JS-side
`$in`	`.filter(doc => arr.includes(doc[field]))`	JS-side
`$nin`	`.filter(doc => !arr.includes(doc[field]))`	JS-side
`$exists`	`.filter(doc => field in doc)`	JS-side
`$regex`	`.filter(doc => regex.test(doc[field]))`	JS-side

Logical:

Operator	Implementation
`$and`	Chain filters
`$or`	Multiple queries → merge results
`$not`	Negate filter
`$nor`	Negate OR

Element / Evaluation (best-effort):

Operator	Implementation
`$type`	`typeof` check in filter
`$mod`	Modulo check in filter
`$size`	Array length check in filter

3.5 Update Operators (Priority: High)

Implemented in the command handler layer (not in EasyDB).

Operator	Description
`$set`	Set field value
`$unset`	Remove field
`$inc`	Increment numeric value
`$push`	Append to array
`$pull`	Remove from array by condition
`$addToSet`	Append to array only if not present
`$rename`	Rename field
`$min` / `$max`	Set to value if less/greater than current
`$currentDate`	Set to current date/time
`$mul`	Multiply numeric value

3.6 Projection (Priority: Medium)

Feature	Description
Field inclusion	`{name: 1, email: 1}`
Field exclusion	`{password: 0}`
`_id` suppression	`{_id: 0}`

3.7 Sort, Skip, Limit (Priority: High)

Feature	EasyDB Mapping
`sort: {field: 1}`	`.where(field).asc()` or JS sort
`sort: {field: -1}`	`.where(field).desc()` or JS sort
`skip: n`	`.skip(n)`
`limit: n`	`.limit(n)`

4. Wire Protocol Specification

4.1 Transport

TCP on configurable port (default: 27017)
TLS support optional (v2+)
Little-endian byte order

4.2 Message Format

Every message has a 16-byte header:

┌──────────────────┬──────────────────┐
│ messageLength    │ int32 (4 bytes)  │
│ requestID        │ int32 (4 bytes)  │
│ responseTo       │ int32 (4 bytes)  │
│ opCode           │ int32 (4 bytes)  │
└──────────────────┴──────────────────┘

4.3 Supported Opcodes

Opcode	Value	Direction	Usage
OP_MSG	2013	Both	All modern commands
OP_QUERY	2004	Inbound	Legacy handshake from older drivers
OP_REPLY	1	Outbound	Response to OP_QUERY

4.4 OP_MSG Structure

Header (16 bytes)
flagBits (uint32)
Section[]:
  - Kind 0 (Body): single BSON document (command)
  - Kind 1 (Document Sequence): bulk documents (optional)
[checksum] (optional uint32 CRC-32C)

4.5 BSON

Use the official bson npm package for serialization/deserialization. Key types to support:

ObjectId (auto-generated _id)
String, Number (int32, int64, double)
Boolean, Null, Date
Embedded documents, Arrays
Binary data

5. Configuration

// easydb-server.config.js
module.exports = {
  // Network
  port: 27017,
  host: '0.0.0.0',

  // Storage backend
  adapter: 'sqlite',          // memory | sqlite | postgres | mysql | redis | turso
  adapterOptions: {
    // SQLite example:
    path: './data/easydb.sqlite',
    // PostgreSQL example:
    // connectionString: 'postgresql://user:pass@localhost:5432/mydb',
  },

  // Databases
  autoCreateDatabases: true,   // Create databases on first use
  autoCreateCollections: true,  // Create collections on first insert

  // Limits
  maxConnections: 100,
  maxBsonObjectSize: 16 * 1024 * 1024,  // 16 MB
  maxMessageSize: 48 * 1000 * 1000,      // 48 MB
  maxWriteBatchSize: 100000,
  cursorTimeoutMs: 10 * 60 * 1000,       // 10 minutes

  // Logging
  logLevel: 'info',            // debug | info | warn | error
};

CLI Usage

# Start with default config (memory adapter)
npx easydb-server

# Start with SQLite
npx easydb-server --adapter sqlite --data ./mydata.db

# Start with PostgreSQL
npx easydb-server --adapter postgres --connection-string "postgresql://..."

# Start with custom port
npx easydb-server --port 27018

# Start with config file
npx easydb-server --config ./easydb-server.config.js

6. Data Mapping

6.1 MongoDB → EasyDB Concepts

MongoDB	EasyDB	Notes
Database	EasyDB instance	One `EasyDB.open()` per database
Collection	Store	Defined in schema
Document	Record/Object	Plain JS object
`_id` field	Primary key	`key: '_id'` in store config
Index	Store index	Declared in schema
ObjectId	BSON ObjectId	Auto-generated string representation

6.2 Dynamic Schema

MongoDB is schemaless — collections are created on first insert. EasyDB requires store definitions upfront. Solution:

On first insert to a new collection → create a new store with key: '_id'
Track known collections in a metadata store (_system.collections)
On server restart → read metadata and recreate schema
Indexes created via createIndexes → stored in metadata, applied to store config

6.3 ObjectId Handling

When _id is not provided on insert, generate a BSON ObjectId
Store as string representation internally (24-char hex)
Serialize back to BSON ObjectId type on output
Support queries by ObjectId or string

7. Cursor Management

MongoDB uses server-side cursors for large result sets.

Aspect	Implementation
Cursor creation	`find` returns first batch + cursorId
Iteration	`getMore` with cursorId returns next batch
Cleanup	`killCursors` or automatic timeout
Batch size	Default 101 documents (first batch), then 16MB batches
Timeout	10 minutes of inactivity
Storage	In-memory Map of cursorId → async iterator

8. Error Handling

Return standard MongoDB error documents:

{
  ok: 0,
  errmsg: "human-readable error description",
  code: <number>,           // MongoDB error code
  codeName: "ErrorCodeName"
}

Key error codes to implement:

Code	Name	When
2	BadValue	Invalid query/update
11000	DuplicateKey	Duplicate `_id` or unique index violation
13	Unauthorized	If auth is enabled (v2+)
26	NamespaceNotFound	Collection doesn't exist (for some ops)
43	CursorNotFound	Invalid/expired cursor
59	CommandNotFound	Unsupported command
9	FailedToParse	Malformed BSON or query

9. Project Structure

easydb-server/
├── src/
│   ├── index.js                  # Entry point, CLI
│   ├── server.js                 # TCP server, connection management
│   ├── wire/
│   │   ├── protocol.js           # Message framing, header parse
│   │   ├── op-msg.js             # OP_MSG parse/serialize
│   │   ├── op-query.js           # OP_QUERY parse (legacy)
│   │   └── op-reply.js           # OP_REPLY serialize (legacy)
│   ├── commands/
│   │   ├── router.js             # Command name → handler dispatch
│   │   ├── handshake.js          # hello, isMaster, buildInfo, ping, etc.
│   │   ├── crud.js               # insert, find, update, delete
│   │   ├── cursors.js            # getMore, killCursors
│   │   ├── admin.js              # listDatabases, listCollections, create, drop
│   │   └── diagnostics.js        # serverStatus, hostInfo, getLog, etc.
│   ├── query/
│   │   ├── matcher.js            # Translate MongoDB query → EasyDB operations
│   │   ├── updater.js            # Apply $set, $inc, etc.
│   │   ├── projection.js         # Field inclusion/exclusion
│   │   └── sort.js               # Multi-field sort
│   ├── storage/
│   │   ├── database-manager.js   # Manage multiple EasyDB instances (databases)
│   │   ├── schema-manager.js     # Dynamic collection/index creation
│   │   └── metadata.js           # System collections (_system.*)
│   └── utils/
│       ├── objectid.js           # ObjectId generation/parsing
│       ├── errors.js             # MongoDB error codes/formatting
│       ├── config.js             # Config loading and defaults
│       └── logger.js             # Structured logging
├── bin/
│   └── easydb-server.js          # CLI executable
├── tests/
│   ├── wire/                     # Wire protocol unit tests
│   ├── commands/                 # Command handler tests
│   ├── query/                    # Query/update operator tests
│   └── integration/              # End-to-end with mongosh/driver
├── examples/
│   ├── basic-usage.js            # Connect with Node.js driver
│   ├── mongoose-example.js       # Use with Mongoose ORM
│   └── python-example.py         # Connect with pymongo
├── SPEC.md                       # This document
├── package.json
└── README.md

10. Dependencies

Package	Purpose	Size
`bson`	BSON serialization (official MongoDB)	~120KB
`@rckflr/easydb`	Storage engine	~4.4KB
Adapter-specific (optional):
`better-sqlite3`	SQLite adapter	~8MB native
`pg`	PostgreSQL adapter	~100KB
`mysql2`	MySQL adapter	~300KB
`ioredis`	Redis adapter	~400KB

Zero runtime dependencies beyond bson and easydb. Adapter drivers are optional peer dependencies.

11. Milestones

v0.1.0 — Wire Protocol + Handshake

TCP server accepts connections
Parse OP_MSG and OP_QUERY
Handle hello/isMaster handshake
mongosh can connect (no data ops yet)

v0.2.0 — Basic CRUD

insert, find, update, delete working
Memory adapter as default backend
Basic query operators ($eq, $gt, $lt, $gte, $lte)
ObjectId generation
mongosh can do full CRUD

v0.3.0 — Query & Update Operators

Full comparison operators ($ne, $in, $nin, $exists, $regex)
Logical operators ($and, $or, $not)
Update operators ($set, $unset, $inc, $push, $pull, $addToSet)
Projection support
Sort, skip, limit

v0.4.0 — Multi-Backend

SQLite adapter integration
PostgreSQL adapter integration
CLI with adapter selection
Config file support
Dynamic database/collection creation
Schema persistence across restarts

v0.5.0 — Polish & Ecosystem

Cursor management with timeouts
Proper error codes and messages
distinct, count, findAndModify
Index management (createIndexes, listIndexes)
npm publish
Docker image
Examples (Node.js driver, Mongoose, pymongo)

12. Success Criteria

The project is successful when:

mongosh connects and runs CRUD operations without errors
Node.js mongodb driver works for basic operations
A user can swap between SQLite, PostgreSQL, and Memory backends with a single config change
The codebase demonstrates EasyDB's adapter interface handling real-world workloads
Someone can use this as a drop-in MongoDB replacement for development/testing

13. Non-Goals (Explicitly Out of Scope)

Aggregation pipeline ($group, $lookup, $unwind, $facet)
Full-text search ($text, Atlas Search)
Geospatial queries ($near, $geoWithin, 2dsphere indexes)
Transactions (multi-document ACID)
Authentication & authorization (SCRAM-SHA-256, x.509)
Replication & replica sets
Sharding
Change streams
GridFS
Capped collections
Map/Reduce
Wire compression (OP_COMPRESSED)

FilesExpand file tree

SPEC.md

Latest commit

History