Provide CoAP endpoint in Ditto's gateway, providing the Ditto HTTP API via CoAP

[thjaeckle]

Currently, Ditto provides a very advanced (e.g. supporting partial requests/updates, "PATCH" updates, server sent events for notifications) HTTP API to be consumed by either frontends, mobile applications or even backends.

Ditto currently does not directly provide an API for devices to interact with.
That was envisioned to be encapsulated via a separate "Device Connectivity Layer" (like an MQTT broker or an Eclipse Hono or a Apache Kafka).
That way, a managed Ditto "connection" would be responsible for connecting to this extra device connectivity layer and translating from/to the specific protocol (e.g. MQTT, AMQP, Kafka).

This has the downside that a (productive) setup with Ditto always requires an additional broker or infrastructure for connecting devices.

CoAP

CoAP, as an equivalent to HTTP for constrained devices, could be a perfect addition to Ditto in order to enable an IoT backend without the need for an additional broker.
As a CoAP API can be implemented basically the same as an HTTP API, Ditto's gateway service (currently responsible for providing the HTTP API, the WebSocket API and the SSE API) would be the perfect location for additionally providing CoAP endpoints.

Devices could then, using CoAP, directly interact with their "twins" (things), e.g. retrieving a property, updating a property, or even retrieving the "desired" state, etc.

CoAP Endpoints

The most straight forward way of defining the CoAP endpoints to support is to basically support all of the existing "/api/2/things" HTTP endpoints also as CoAP endpoints:

• Things: https://www.eclipse.org/ditto/http-api-doc.html#/Things
• Features: https://www.eclipse.org/ditto/http-api-doc.html#/Features

Authentication

CoAP supports different authentication mechanisms, as far as I know:

• PSK (pre shared keys) based
• Client Certificate based

I currently do not have more knowledge - I assume that Ditto could configure a Certificate Authority (CA) for e.g. authenticating client certificates. Or that PSKs could be configured as well via configuration or a file mount.

Feedback on that would be appreciated :)

Authorization

Once a device via CoAP was authenticated (see above), a "subjectId" would be determined from the authenticated device, e.g.:

coap-psk:<the-authenticated-device-id>

Using this subject in a Ditto Policy, a device can get authorized to e.g. read/write its twin data or to send/receive messages.

Example policy:

{
  "policyId": "namespace:the-policy",
  "entries": {
    "DEVICE": {
      "subjects": {
        "coap-psk:my-device-4711": {
          "type": "CoAP PSK based authentication"
        }
      },
      "resources": {
        "thing:/": {
          "revoke": [],
          "grant": [
            "READ",
            "WRITE"
          ]
        },
        "message:/": {
          "revoke": [],
          "grant": [
            "READ",
            "WRITE"
          ]
        }
      }
    },
    "DEFAULT": {
      "subjects": {
        "nginx:ditto": {
          "type": "Authenticated user via nginx htpasswd"
        }
      },
      "resources": {
        "policy:/": {
          "revoke": [],
          "grant": [
            "READ",
            "WRITE"
          ]
        },
        "thing:/": {
          "revoke": [],
          "grant": [
            "READ",
            "WRITE"
          ]
        },
        "message:/": {
          "revoke": [],
          "grant": [
            "READ",
            "WRITE"
          ]
        }
      }
    }
  }
}

Implementation

The implementation would rely on our sister project, Eclipse Californium.
I see 2 options how to do the implemtation:

• either implementing a CoAP to HTTP proxy (fully supporting all existing HTTP APIs)
  • an example can be found in Californium: https://github.com/eclipse-californium/californium/tree/main/californium-proxy2
  • downside: an additional "hop" (but only from "localhost" to "localhost", so should not be that bad)
  • upside: if mapped once correctly (e.g. "PATCH", "GET", "UPDATE", "DELETE" and also server sent events), all available HTTP APIs should automatically work via CoAP as well
• or re-implementing the existing HTTP endpoints via californium, all explicitly
  • downside: much effort, could divert from the HTTP API
  • upside: no additional HTTP request required
• maybe there is a third option which could have the benefits from both variants above - without the need to do another (internal) HTTP call
  • that has to be investigated

Testing

Is there an equivalent to e.g. cURL (or httpie / Postman / etc.) for doing CoAP requests to test during development?
For unit testing I assume that we also can use Californium as the client?

Deployment

I also have no idea what CoAP (UDP) endpoints means to a (cloud) deployment, e.g. regarding loadbalancers.

Maybe someone has more input on that, what to consider, etc.?

Please provide feedback / input

To anyone who is interested in this topic, directly providing CoAP endpoints in Ditto, please comment and provide feedback.

May I invite @boaks, the project lead of Eclipse Californium and the CoAP expert, to join the discussion? :)
Especially regarding the authentication of devices and what Ditto would have to support to configure credentials I do currently not know ..

[boaks]

A lot of questions :-).

Let's start:

Authentication:

Californium itself uses "callbacks" to fetch the credentials. The callbacks may be used in a synchronous or asynchronous manner. Californium comes also with some demo-implementations. I don't know, how ditto keeps/manages the credentials, but maybe it's possible to implement the Californium interfaces using the current ditto credentials implementation.

PSK:
similar to "username/password", but the password is not exchanged and must be available on both sides (server and client) in clear.

AdvancedPskStore

certificate based:
Californium supports x509 and RFC7250 (Raw public key).

To authenticate Californium uses CertificateProvider

To verify the authentication of the other peer NewAdvancedCertificateVerifier

(Just to mention: the curios names are the result of a development over several periods. Maybe with Californium 4.0 we polish the names. It will then be some "typing work", but I guess it would make it easier afterwards. For now, no schedule for 4.0 exists).

All the interfaces are in java packages and comes there with demonstration implementations. Though I consider it valuable to use Californium on its own, I added a MultiPskFileStore and I will see, what I can provide in the future. Anyway, for upstream projects it's mostly preferable to implement these interfaces using the own infrastructure.

[boaks]

Authorization:

Each Message comes with it's EndpointContext. Depending on incoming or outgoing messages, the context is the destination or source context. For incoming request it will be the sourcecontext, and that keeps the Principal, if DTLS (or TLS) is used. If not encrypted communication is to be used, the id/principal must be sent e.g. as URI query parameter or part of the URI path.

[boaks]

Implementation:

I guess, that's also a question, if the common functionality could be extracted in a reusable module and each connector then mainly implements it's own mapping layer. I guess, a first PoC directly using Californium will provide then the information to decide, what's the best approach.

[boaks]

Testing:

I use several clients. The most clients are very simple and I'm used to adapted or extend them for special tests.

UI Client

CLI Client

JMeter Client Plugin

Benchmark Client

[boaks]

Deployment:

At least today, many clouds provide UDP support. Also the main tools, docker and k8s comes with UDP support.

Californium offers some help here, see
Wiki

cf-extplugtest-server

One common pitfall is the use of NATs (maybe even not intended and more unware done by some part of the infrastructure). For UDP that quite often comes with short timeouts. For "single endpoints", using RFC 9146 (DTLS 1.2 CID) is a great solution here. For clusters it's more complicated. I've used

built-in dtls cid cluster support
cf-cluster

My impression of the past years was, that it was frequently requested, but close to no deployment reached even the limit of a single connector.

[thjaeckle]

Tanks, Achim @boaks for the many inputs 👍

I guess, a first PoC directly using Californium will provide then the information to decide, what's the best approach.

Yes, that would be my next step .. :)

Regarding

I don't know, how ditto keeps/manages the credentials, but maybe it's possible to implement the Californium interfaces using the current ditto credentials implementation.

Ditto currently does not keep/manage credentials - for HTTP it relies on OpenId Connect providers to authenticate (and then "just" uses the JWT issued by those providers to apply authorization).
Or it relies on a reverse proxy (e.g. a "nginx") to manage credentials.

I would probably want to avoid adding a separate persistence for credentials and go into the direction of providing them also as kind of a ".htpasswd" file format (like "nginx" also does) which can be provided via volume mount.

[boaks]

I would probably want to avoid adding a separate persistence for credentials and go into the direction of providing them also as kind of a ".htpasswd" file format (like "nginx" also does) which can be provided via volume mount.

Yes. Therefore I implemented the MultiPskFileStore. It offers automatic reloading from files, but you may also provide a InputStream. The secrets maybe encrypted, if wanted.

[thjaeckle]

I managed to add a first rough implementation in #1588
Still a lot of TODOs open, but conceptually this works really really well ..

[boaks]

Happy to read that.

Do you need any help for tests with coap-clients? Or you OK with it?