INTERFACES

Kivaloo daemon interfaces
-------------------------

Kivaloo components communicate via sockets (either UNIX domain sockets or TCP
sockets) using a request-response wire protocol supporting out-of-order
responses.  The protocol uses CRCs to provide some error-detection, but has no
encryption, authentication, or authorization mechanisms; if required, these
should be handled externally.

If a badly-formed request is sent to the server, it will drop the connection.

Wire protocol
-------------

Connections between components carry packets of the form:
[8 byte request ID]
[4 byte record length]
[4 byte CRC of first 12 bytes]
[record]
[(4 byte CRC of record) xor (4 byte CRC of first 12 bytes)]

The 8 byte request ID is arbitrary; the header and data depend on the type of
request or response.  Responses can be sent back out-of-order; the requestor
is responsible for ensuring that the request IDs are unique in order to be
able to match up responses with requests.

The CRC function is the 32-bit CRC computed using the Castagnoli polynomial
0x11EDC6F41 such that 1[data][CRC], with the bit order of [CRC] and [data]
reversed, is a multiple of the polynomial.  Note that most 32-bit CRC code
represents the CRC as a 32-bit integer; when using such routines one must
ensure that the value is serialized with the right endianness (most often
little, but this will depend on the library).  The leading 1 bit ensures that
0{32} is not the CRC of 0*, and the xoring of header and record CRCs makes it
possible to detect certain other error types (e.g., if a packet has the
header from one packet and the record from a different packet).

Request records start with a 4 byte request type number.  All integers are
big-endian unless specified otherwise.

Block store interface
---------------------

PARAMS:	Request type = 0x00000000

	Request:
	[4 byte request type]

	Response:
	[4 byte block size]
	[8 byte next block #]

PARAMS2:Request type = 0x00000004

	Request:
	[4 byte request type]

	Response:
	[4 byte block size]
	[8 byte next block #]
	[8 byte last block #]

GET:	Request type = 0x00000001

	Request:
	[4 byte request type]
	[8 byte block number]

	Response if block exists:
	[4 byte status code = 0]
	[1 block of data]

	Response if block does not exist:
	[4 byte status code = 1]

APPEND:	Request type = 0x00000002

	Request:
	[4 byte request type]
	[4 byte number of blocks]
	[8 byte starting block #]
	[1 or more blocks of data]

	Response if the starting block # is correct:
	[4 byte status code = 0]
	[8 byte next block #]

	Response if the starting block # is incorrect:
	[4 byte status code = 1]

FREE:	Request type = 0x00000003

	Request:
	[4 byte request type]
	[8 byte first block # to keep]

	Response:
	[4-byte status code = 0]

Key-value data store interface
------------------------------

PARAMS:	Request type = 0x00000100

	Request:
	[4 byte request type]

	Response:
	[4 byte maximum key length]
	[4 byte maximum value length]

SET:	Request type = 0x00000110

	Request:
	[4 byte request type]
	[1 byte key length][X byte key]
	[1 byte value length][X byte value]

	Response (value set):
	[4 byte status code = 0]

CAS:	Request type = 0x00000111

	Request:
	[4 byte request type]
	[1 byte key length][X byte key]
	[1 byte oval length][X byte oval]
	[1 byte value length][X byte value]

	Response (value set):
	[4 byte status code = 0]

	Response (value not set):
	[4 byte status code = 1]

ADD:	Request type = 0x00000112

	Request:
	[4 byte request type]
	[1 byte key length][X byte key]
	[1 byte value length][X byte value]

	Response (value set):
	[4 byte status code = 0]

	Response (value not set):
	[4 byte status code = 1]

MODIFY:	Request type = 0x00000113

	Request:
	[4 byte request type]
	[1 byte key length][X byte key]
	[1 byte value length][X byte value]

	Response (value set):
	[4 byte status code = 0]

	Response (value not set):
	[4 byte status code = 1]

DELETE:	Request type = 0x00000120

	Request:
	[4 byte request type]
	[1 byte key length][X byte key]

	Response:
	[4 byte status code = 0]

CAD:	Request type = 0x00000121

	Request:
	[4 byte request type]
	[1 byte key length][X byte key]
	[1 byte oval length][X byte oval]

	Response (value deleted):
	[4 byte status code = 0]

	Response (value not deleted):
	[4 byte status code = 1]

GET:	Request type = 0x00000130

	Request:
	[4 byte request type]
	[1 byte key length][X byte key]

	Response (value present):
	[4 byte status code = 0]
	[1 byte value length][X byte value]

	Response (value not present):
	[4 byte status code = 1]

RANGE:	Request type = 0x00000131

	Request:
	[4 byte request type]
	[4 byte maximum total size of returned key-value pairs]
	[1 byte start length][X byte start]
	[1 byte end length][X byte end]

	Response:
	[4 byte status code = 0]
	[4 byte number of returned key-value pairs]
	[1 byte next length][X byte next]
	[1 byte key length][X byte key][1 byte value length][X byte value]
	...
	[1 byte key length][X byte key][1 byte value length][X byte value]

S3 interface
------------

In all S3 messages, "HTTP status" indicates an integer from 100 to 599 (as per
RFC 7231), or 0 to indicate a generic failure at the HTTP layer.

PUT:	Request type = 0x00010000

	Request:
	[4 byte request type]
	[1 byte bucket name length][X byte bucket name]
	[1 byte object name length][X byte object name]
	[4 byte object length][X byte object data]

	Response:
	[4 byte HTTP status]

GET:	Request type = 0x00010010

	Request:
	[4 byte request type]
	[1 byte bucket name length][X byte bucket name]
	[1 byte object name length][X byte object name]
	[4 byte maxlen]

	Response:
	[4 byte HTTP status]
	[4 byte object length][X byte object data]

RANGE:	Request type = 0x00010011

	Request:
	[4 byte request type]
	[1 byte bucket name length][X byte bucket name]
	[1 byte object name length][X byte object name]
	[4 byte range offset]
	[4 byte range length]

	Response:
	[4 byte HTTP status]
	[4 byte object length][X byte object data]

HEAD:	Request type = 0x00010020

	Request:
	[4 byte request type]
	[1 byte bucket name length][X byte bucket name]
	[1 byte object name length][X byte object name]

	Response:
	[4 byte HTTP status]
	[4 byte object length]

DELETE:	Request type = 0x00010030

	Request:
	[4 byte request type]
	[1 byte bucket name length][X byte bucket name]
	[1 byte object name length][X byte object name]

	Response:
	[4 byte HTTP status]

DynamoDB-KV interface
---------------------

PUT:	Request type = 0x00010100

	Request:
	[4 byte request type]
	[1 byte key length][X byte key]
	[4 byte value length][X byte value]

	Response:
	[4 byte status (0 = success, 1 = failure)]

ICAS:	Request type = 0x00010101

	Request:
	[4 byte request type]
	[1 byte key length][X byte key]
	[4 byte val1 length][X byte val1]
	[4 byte val2 length][X byte val2]

	Response:
	[4 byte status (0 = success, 1 = failure, 2 = precondition failed)]

CREATE:	Request type = 0x00010102

	Request:
	[4 byte request type]
	[1 byte key length][X byte key]
	[4 byte value length][X byte value]

	Response:
	[4 byte status (0 = success, 1 = failure, 2 = precondition failed)]

GET:	Request type = 0x00010110

	Request:
	[4 byte request type]
	[1 byte key length][X byte key]

	Response (success):
	[4 byte status = 0]
	[4 byte value length][X byte value]

	Response (otherwise):
	[4 byte status = 1 (failure) or 2 (no such key/value pair)]

GETC:	Request type = 0x00010111

	Request:
	[4 byte request type]
	[1 byte key length][X byte key]

	Response (success):
	[4 byte status = 0]
	[4 byte value length][X byte value]

	Response (otherwise):
	[4 byte status = 1 (failure) or 2 (no such key/value pair)]

DELETE:	Request type = 0x00010120

	Request:
	[4 byte request type]
	[1 byte key length][X byte key]

	Response:
	[4 byte status (0 = success, 1 = failure)]