Add size limits for all opcodes

[taoeffect]

Problem

Opcodes need size limits on all user-provided strings and data blobs.

See convo: #33 (comment)

Solution

1. Add size limits to all opcodes
2. Add reasonable size limit to OP_ATOMIC operation as well. I dunno, something like 1MB sounds reasonable?

[corrideat]

A naïve approach at implementing this is setting some hard-coded limits, but as we saw from real-world usage, a more refined approach is better.

Cutting to the chase, the most naïve approach is setting a limit on chel, so that attempting to post an event over a certain, configurable size errors (for example, HTTP 413). This seems like a good idea to do, regardless of anything that follows.

For a more comprehensive approach, we can also impose limits on SPMessage.deserialize / SPMessage.deserializedHEAD, which could be done globally (e.g., defining SPMessage.limit(), which could be called by Chelonia) or as arguments. This way, not only can we impose a global limit, but also impose per-opcode limits, which concern the server and clients.

Some issues with SPMessage limits

Potential for breaking a contract

Imposing a limit on SPMessage can lead to unintended consequences. For a trivial example, say that a maximum message size (whether overall or for any one opcode is immaterial) of 1MiB is used. If a message exceeding that limit should arrive, rejecting it could permanently block the contract if different clients or the server disagree on what the maximum message is, because rejecting the message and proceeding to the next one will likely not work (previousHEAD will be a mismatch)

Mitigation

Instead of locking down the contract, a 'healing' approach could be used of simply computing the hash of the oversized message and then proceeding to the next message. This won't break processing but could result in a broken state, since those messages that were skipped could have modified the state in non-trivial ways.

The most important mitigation here is probably ensuring that everyone agrees on what the limits are supposed to be.

Large contracts

Looking at current use, there are two situations that could result in large opcodes

OP_ATOMIC

When we perform key rotations, we issue a 'large' OP_ATOMIC, which mostly consists of small OP_KEY_SHARE operations, which include the new keys for each member. The size of this operation will grow linearly with the number of members that need to receive a key rotation.

This OP_ATOMIC could be broken up with caution (see okTurtles/group-income#2988 for how it can lead to subtle but serious issues), although ideally it shouldn't be.

Mitigation

As mentioned, the OP_ATOMIC could be split into smaller parts if needed, at the cost of some race conditions in rotations. A variant of this approach is sending the OP_KEY_SHARE to each contract, instead of a large OP_ATOMIC on the contract doing the rotation (this was an earlier approach, which we abandoned because using a single OP_ATOMIC is more reliable)

Alternatively, rotations could simply be skipped for very large groups.

OP_KEY_SHARE

A related issue occurs when joining a group, which is affected by past key rotations. When someone joins a group, all keys, past and present, are shared with the new member in a single OP_KEY_SHARE.

Mitigation

The good news is that this OP_KEY_SHARE can be safely split into smaller chunks, without many special considerations. Other than splitting it up, or the solution of avoiding key rotations, there's another alternative that could be especially suited for contracts such as chatrooms: a 'healing' mechanism with a sliding window.

By slightly changing how we currently do key rotations, we can implement this. First, when a key rotation happens (see OP_ATOMIC above), we also encrypt the prior key with the new key. Then, when we send OP_KEY_SHARE to a new member, we can share the N most recent keys. Finally, when 'scrolling up', if we were to find a message we can't decrypt, we can continue fetching messages until we reach the OP_KEY_SHARE that contained the old key encrypted with the more recent key we have.

Sizes

Looking at Group Income, it appears like messages are in the order of KiB, with single message exceeding about 35 KiB (OP_KEY_SHARE when joining). This suggests that, at present, a limit as low as 8 KiB for all opcodes except OP_ATOMIC and OP_KEY_SHARE would be workable.

Field sizes

The initial motivation for this issue was restricting the size of specific fields in opcodes, such as the reference field. At the time (and still now), this didn't seem very practical as a hard limit because such fields often are encrypted, which limits enforcement.

Nevertheless, we can (and probably should) implement schema validation on opcodes, just noting that messages that are rejected after decryption should not result in a different state than if they couldn't have been decrypted.

[corrideat]

This is an alternative, AI-assisted view of the previous comment

Proposal: Message Size Limiting for SPMessage and Chelonia

Executive Summary

This proposal addresses the need for implementing message size limits in the system to prevent potential issues with oversized messages while maintaining contract integrity. The approach balances immediate protective measures with comprehensive long-term solutions, including special handling for large operations like key rotations and group joins.

Key Challenge: Implementing size limits without breaking contract processing when clients/servers disagree on limits or when legitimate large operations occur.

---

Actionable Items

Phase 1: Immediate Protective Measures (Short-term)

1.1 Implement Chelonia Size Limit

• Action: Add configurable hard limit on chel serve to reject oversized event posts (default 256 KiB)
• Implementation: Return HTTP 413 (Payload Too Large) for messages exceeding threshold
• Rationale: Simple, immediate protection regardless of downstream solutions

---

Phase 2: SPMessage Limit Infrastructure (Medium-term)

2.1 Implement Global Limit Method

• Action: Create SPMessage.limit() method callable by Chelonia, or argument to SPMessage.deserialize() / SPMessage.deserializeHEAD()
• Scope: Define global maximum message size
• Critical Requirement: All clients and servers must agree on size limits

2.2 Implement 'Healing' Approach (optional)

• Action: Instead of rejecting oversized messages, compute their hash and proceed to next message
• Rationale: Prevents contract lockup from previousHEAD mismatches, if not all parties agree on sizes
• Trade-off: May result in broken state, but preserves contract processing

---

Phase 3: Large Contract Mitigations

3.1 OP_ATOMIC Handling for Key Rotations

Current Issue: Key rotations produce large OP_ATOMIC operations that scale linearly with group size.

Options (choose one):

• Option A (Recommended): Keep single OP_ATOMIC, set appropriately high limit

  • Pro: Most reliable, avoids race conditions
  • Con: May hit size limits for very large groups

• Option B: Split into smaller OP_ATOMIC operations with careful ordering

  • Pro: Stays under size limits
  • Con: Introduces race conditions (see issue #2988)

• Option C: Send individual OP_KEY_SHARE to each contract

  • Pro: Distributes message size
  • Con: Less reliable than single atomic operation (reason this was abandoned)

• Option D: Skip rotations for very large groups

  • Pro: Simple
  • Con: Security implications

Action Required: Evaluate and document maximum supported group size for each option

3.2 OP_KEY_SHARE Chunking for Group Joins

Current Issue: Joining members receive all historical keys in single OP_KEY_SHARE, growing with rotation count.

Solution (Safe to implement):

• Action: Split OP_KEY_SHARE into smaller chunks when sending to new members
• Priority: Medium
• Complexity: Low (can be done safely without special considerations)

3.3 Sliding Window "Healing" Mechanism (Advanced)

For chatroom-style contracts:

• Action: Implement alternative key sharing approach:

  1. During rotation: Encrypt prior key with new key
  2. On join: Share only N most recent keys in OP_KEY_SHARE
  3. On scroll-up: Fetch additional messages to decrypt older keys from encrypted key chain

• Priority: Low (optimization)

• Benefit: Dramatically reduces join message size for long-lived chatrooms

---

Phase 4: Schema Validation (Long-term)

4.1 Implement Field-Level Validation

• Action: Add schema validation for individual opcode fields (e.g., reference field size)
• Limitation: Hard to enforce on encrypted fields
• Priority: Medium

4.2 Ensure Rejection Consistency

• Action: Verify that messages rejected after decryption produce same state as if they couldn't be decrypted
• Rationale: Prevents state divergence between clients
• Priority: High