RFC: gluon-config-api

[lemoer]

This is my suggestion to support a REST API for the config-mode.

It basically works like this:

• A single JSON object called "config" is exposed via http.
• Packages can add config definitions in /lib/gluon/config-api/parts/*.lua.
  • Such a file contains a json-schema for their config options,
  • a method to get the config,
  • and a method set the config.
• The complete config definition is merged from all snippets in /lib/gluon/config-api/parts/*.lua.
• The json-schema verification is based on libucl.

Here is an example usage. To get the config, use the following command:

$> curl http://localhost:6080/cgi-bin/api/v1/config -X GET
{
   "wizard": {
     "contact": "Leo",
     "location": {
       "lon": 9.7192900000000009,
       "lat": 52.386696000000001,
       "share_location": false
     }
   }
 }

To get the complete schema, use:

#> curl http://localhost:6080/cgi-bin/api/v1/config -X OPTIONS
{
   "schema": {
     "type": "object",
     "additionalProperties": false,
     "properties": {
       "wizard": {
         "properties": {
           "contact": {
             "type": "string"
           },
           "location": {
             "type": "object",
             "required": [
               "lat",
               "lon",
               "share_location"
             ],
             "properties": {
               "lon": {
                 "type": "number"
               },
               "lat": {
                 "type": "number"
               },
               "share_location": {
                 "type": "boolean"
               }
             }
           }
         },
         "additionalProperties": false,
         "type": "object"
       }
     },
     "required": [
       "wizard"
     ]
   },
   "allowed_methods": [
     "GET",
     "POST",
     "OPTIONS"
   ]
 }

Change contact from Leo to Hans:

#> curl http://localhost:6080/cgi-bin/api/v1/config -X POST --data @- <<EOI
{
   "wizard": {
     "contact": "Hans",
     "location": {
       "lon": 9.7192900000000009,
       "lat": 52.386696000000001,
       "share_location": false
     }
   }
 }
EOI
{
   "status": 200,
   "error": "Accepted"
 }

See the result:

#> curl http://localhost:6080/cgi-bin/api/v1/config -X GET
{
   "wizard": {
     "contact": "Hans",
     "location": {
       "lon": 9.7192900000000009,
       "lat": 52.386696000000001,
       "share_location": false
     }
   }
 }

Feel free to comment this approach.

---

(This is based on the ideas in #2238)

[lemoer]

This has not yet received a lot of attention. Can I provide additional information? Is there anything missing?

[neocturne]

Some unsorted thoughts:

• I like the JSON-based data structure
• Having the ability to download configuration as a whole is probably generally a good thing for performance. While I'd love to have some GraphQL-like filtering capabilities, the increased processing time would likely more than outweigh the lower transferred data volume
• I'm not a fan of JSON schemas, but I guess I can live with it if it works well for our use case.
  • What's the binary size of libucl?
  • How are config schema definitions combined?
  • See also https://gitlab.com/neoraider/ece/-/wikis/design/schema, which was the simplified/modified JSON schema approach of my GSoC project a few years ago - in particular it was designed to allow merging of modular schemas. Use a more maintained library like libucl is definitely preferable to my old code though.
• We should not abuse the OPTIONS method, there should be a separate schema URL you can GET
• Is deleting values possible, or do we expect that no values are optional and must be set to "null" when they don't have a value?
• Should modifying array values be possible in an atomic way, or do you need to GET + POST the whole array? Should a way to detect or prevent concurrent changes be provided?

I have not looked at the implementation yet.

[lemoer]

Some unstructured answers...

What's the binary size of libucl?

IIRC, libucl is ~51kb. I decided to use it for my RFC, since it worked more or less without much effort.

How are config schema definitions combined?

I used (more or less) the merging algorithm that you developed for ece.

See also https://gitlab.com/neoraider/ece/-/wikis/design/schema, which was the simplified/modified JSON schema approach of my GSoC project a few years ago ...

Yes. I am aware of that code and looked into it. As it uses blobs instead of json, I decided not to use it as of now. Also I think, some parts were even unfinished.

[lemoer]

• Is deleting values possible, or do we expect that no values are optional and must be set to "null" when they don't have a value?
  Should modifying array values be possible in an atomic way, or do you need to GET + POST the whole array?
• Should a way to detect or prevent concurrent changes be provided?

In the current version of this RFC, there is no patching of the old config based on the POST request. The new config is actually directly the body of the POST request. So there is no merging of the old config with the new config.

This means that if you want to change just a single value, you have to retrieve the entire config and post it again.

I know this is unperformant and ugly (especially for larger applications), but I think it could be useful for this API:

• It's easy to understand for users of the API.
• The code stays simple. (The old state is irrelevant. No merge between old and new state necessary).

[lemoer]

Btw. an example of how a config could look like can be found here: Link

[lemoer]

@neoraider This is lying around for over a year without any feedback. Is there any progress?

[AiyionPrime]

takeaways

• frontend and backend should be able to verify against the same schema
• should still be possible to have config objects depending on installed packages (like batman-adv, or community specific packages)
• have a look at gluon-web-model for current implementation of data attributes and logic
  • https://github.com/freifunk-gluon/gluon/blob/master/package/gluon-web-model/files/lib/gluon/web/view/model/section.html
  • js in frontend evaluates that
  • types and dependencies of fields should be exposed as well
• way to go:
  • go step by step.
  • gluon-web-model already provides information about dependencies/type/....
  • expose this as API.
  • try to build on this.
• javascript code should reference fields from backend.
  • based on the information about dependencies/type/..., the frontend code could render a field.
• Most of the model definition could be reused.
  • The biggest problem is, that fields are not well defined by their identifiers.
  • One of the first steps would be to replace the enumeration of forms and sections.

@lemoer will also have a look at OpenAPI / swagger.

[AiyionPrime]

found it

[AiyionPrime]

spdx

[AiyionPrime]

spdx

[AiyionPrime]

spdx no further license comments

[AiyionPrime]

https://github.com/vstakhov/libucl/releases/tag/0.8.2