Metadata on recipe container?

[josephguillaume]

At the moment, when loading a recipe container there is no easy way of:

1. knowing it contains recipes
2. knowing the format used for the recipes
3. enforcing or validating this format

I propose that Umai could add a triple to the container metadata:

<> rdf:type ex:RecipeContainer.

This would directly address (1), and allow (2) and (3) to be inferred.

Here are some alternatives:

1. Using a shape tree assignment: in my opinion this would increase the barrier to interoperability due to its complexity - using a class is substantially simpler. The two could easily co-exist in future, with the shape tree being optional.

2. Using existing type registrations: this is already more complex as it requires following the path ?container^solid:instance container!solid:forClass and checking that this is equal to schema:Recipe. In addition, it assumes that preferences have been loaded. This then means that it does not yet resolve the issue that when dereferencing the container, one doesn't know it contains recipes.

<#Recipes>
a solid:TypeRegistration; 
solid:forClass schema:Recipe;
solid:instanceContainer <https://example.com/private/cookbook/>.

Given the nature of Solid, it is of course possible for a user to use any of these solutions manually now already without Umai support, but Umai is the frontrunner and nicest product in this space, so it would be good to have Umai on board with any strategy.

[NoelDeMartin]

Hey, thanks for opening an issue!

I think the problem you're raising is related with solid/specification#227 (comment). Ideally, this is something that should be solved on the POD side. But you're right that at the moment, the only way to know what a container contains is actually reading the documents themselves :(.

Right now, I am solving this with the type index. I'm already doing your 2nd alternative proposal, did you have any problem with that? I am registering the container in a private type index, though. Maybe that's why you didn't find it. I'm not sure if we should be registering recipe containers publicly by default :/.

About your main proposal, I'm not convinced about adding that type of metadata to the container, because it's the first time I hear anything like that so very few apps would know that this thing even exists. The type index, for all the flaws it has, is the only real way we have for solving this problem at the moment. So many apps are already using it.

Finally, I really like what SAI is doing with Shape Trees, but it is my understanding that it requires server-side support to work. So until it's part of the official spec and it's implemented in all POD providers, I don't intend on using it because I want to make my apps compatible with any POD that is spec-compliant. But I'm not sure if what you meant by your first alternative proposal is related to that, or it can be implemented in the client. Did I misunderstanding something?

[josephguillaume]

Hey, thanks for opening an issue!

Thanks for the discussion!

I think the problem you're raising is related with solid/specification#227 (comment). Ideally, this is something that should be solved on the POD side. But you're right that at the moment, the only way to know what a container contains is actually reading the documents themselves :(.

This is related (thanks for pointing it out!), but what I am proposing is possible right now by PATCHing the .meta resource of the container (which is linked in the header)
I already use this mechanism in other contexts, e.g. to add labels for files so that they are searchable by PodOS

Right now, I am solving this with the type index. I'm already doing your 2nd alternative proposal, did you have any problem with that? I am registering the container in a private type index, though. Maybe that's why you didn't find it. I'm not sure if we should be registering recipe containers publicly by default :/.

The problem is 1) the user needs to know to and load preferences to get this information, 2) this information is only available to the owner of the data, 3) it is more complex than using a single type statement 🤣

About your main proposal, I'm not convinced about adding that type of metadata to the container, because it's the first time I hear anything like that so very few apps would know that this thing even exists. The type index, for all the flaws it has, is the only real way we have for solving this problem at the moment. So many apps are already using it.

This is pretty similar to how each object is given a type within each document. The main difference is that the container already has a (generic) type, and we'd just be adding a bit more specific information.
In the long run, I expect that e.g. a ex:CookBook class might emerge, which might not be based on a container structure and which allows providing additional metadata about the cookbook as a whole. For the time being, simply a subclass of container would be useful.

Finally, I really like what SAI is doing with Shape Trees, but it is my understanding that it requires server-side support to work. So until it's part of the official spec and it's implemented in all POD providers, I don't intend on using it because I want to make my apps compatible with any POD that is spec-compliant. But I'm not sure if what you meant by your first alternative proposal is related to that, or it can be implemented in the client. Did I misunderstanding something?

My proposal is already spec and server compliant.
My understanding is that shape trees only require server side support to enforce validation/consistency, e.g.
https://github.com/CommunitySolidServer/shape-validator-component The rest should be doable client side but I find it a bit daunting so keen for others to go first 😁

Edit: what I mean is that shape trees can be used client side without relying on the server providing headers
https://shapetrees.org/TR/specification/#shapetree-support-from-proxy-or-client-side-library

[NoelDeMartin]

The problem is 1) the user needs to know to and load preferences to get this information, 2) this information is only available to the owner of the data, 3) it is more complex than using a single type statement 🤣

1. App developers should know about this because the type index is already the de-facto solution for this problem (finding things in a POD). Or at least that's my opinion, given that the spec is still a draft 😅.
2. This could be solved by using the public type index, I made a conscious choice to use the private one. I could make this configurable, but I'm solving "public recipes" in a different way.
3. I agree, but it shouldn't be a problem to read multiple documents in Solid; it's actually the point of Solid that things are interlinked and not everything is in a single place (not even a single POD). Furthermore, this type of thing could be encapsulated in a library so app developers shouldn't have to think about that.

In the long run, I expect that e.g. a ex:CookBook class might emerge, which might not be based on a container structure and which allows providing additional metadata about the cookbook as a whole. For the time being, simply a subclass of container would be useful.

On that note, the way I solve "public recipes" is that I have a schema:ItemList registered in the public type index. There are two reasons why I'm using a list and not a container. First, this means that recipes can become public/private without changing containers (thus maintaining the same url). And second, I can have the same recipe in multiple lists. At the moment, the app only does a single public list; but in the future I may implement custom lists and the app already supports it.

But I am aware that it's a problem, because schema:ItemList does not convey that the list has recipes :/. The problem is that schema.org doesn't have a class for recipe lists, so I decided to use this one and search for recipes inside of all registered schema:ItemList. But I'm aware it's not ideal.

My understanding is that shape trees only require server side support to enforce validation/consistency, e.g.
https://github.com/CommunitySolidServer/shape-validator-component The rest should be doable client side but I find it a bit daunting so keen for others to go first 😁

I see, that is interesting but in my case I don't want to spend too much time working on enforcing this because it hasn't been a real problem that I've seen. And if I did, I would enforce that in the client using Soukai model definitions.

So until the server doesn't do anything with shapes, I don't think I'll be using them.

Having said all that, I understand what you mean and I don't think there is any harm in having additional information :). So I'm ok with adding the following things:

• a new <> a ex:Cookbook in the container .meta.
• a new </cookbook/public> a ex:RecipesList declaration in the public type index for recipe lists (in addition to the existing one).

The problem, then, is what ontology to use 😅. schema.org doesn't have any terms for this, do you know of any ontology that would be appropriate for this? In case we don't, I may create a new /recipes namespace in my vocab, but I'd prefer using an existing term instead.

[josephguillaume]

Thanks for your willingness to add this additional information!

In terms of ontology, I think it would make sense to publish versioned SHACL shapes for Umai and as part of that to include a definition of these two new classes.
In this case, the classes imply certain shapes and it therefore makes sense to mint your own uri rather than reusing another with a likely different structure (if it even exists)
These shapes could then be used in shape trees in future, as well as allowing other apps to test their compatibility with Umai.
The only reason I suggest SHACL rather than shex is because it is expressed in RDF.
In the long run, versioning the SHACL shapes means that you/others could also express schema upgrade rules in RDF too (I have Cambria lenses in mind as inspiration here).
In the shapes, I would also include a definition of/reference to the metadata/CRDT shape, which would allow the entire document to be validated.

At risk of going off topic, I do want to also underline what looks like some differences in our perspectives:

• While app developers should know about the type index, I am interested in user-developers who would interact with data in a similar way to mavo.io : https://dl.acm.org/doi/10.1145/2984511.2984551 From my point of view the user will know about object types and traversing a graph using properties, but type indices would already be an advanced topic
• With this in mind, I believe encapsulation should be syntactic sugar and not a requirement. If the user cannot easily understand how their data is connected, then they do not own it. While it may be appropriate in this case, in general I am against solutions in which data is too complex to be easily understood and is in effect deliberately hidden behind encapsulation.
• In general, client side validation is a difficult problem because the user is permitted/encouraged to edit their own data in arbitrary ways. The advantage of defining validation shapes in the data rather than in the app is that the user then owns it - it is their choice to abide by the app's shape.
• In general my attitude to minting new URIs is that it's important to keep the cost of switching low, which in turn keeps the cost of creating new URIs low. Otherwise, every time a user-developer wants to express a new relationship or concept they end up in decision paralysis in what predicate/class to use. I think translating data at read time is important for this. Mavo supports renaming https://mavo.io/docs/migration/#mv-alias
• A public type index is by definition public, whereas I want any user with a read permission to know what they are looking at when they open the container

I hadn't tried public sharing before as it doesn't fit my use case. Thanks for explaining its design. The idea of recipe lists does sound like it works for the use case where a user might share some of their recipes within a private but shared list. In that case, I would not be defining the shared list in the public type index, and I might not even register it there, e.g. I would share a direct link instead. And as you say, I would want to specify that it's a recipe list, not just an item list.
It would make sense for the shape definition/ontology document to specify that ex:RecipeList rdfs:subClassOf schema:ItemList
Unfortunately I get an error message with no permissions changed or public type index created so haven't been able to see the public sharing in practice.

[NoelDeMartin]

In terms of ontology, I think it would make sense to publish versioned SHACL shapes for Umai and as part of that to include a definition of these two new classes.

I think it would be fine as way of documentation, but otherwise I don't see much use for defining SHACL shapes at the moment. I agree it may be useful in the future, but I'm not sure about it now.

I've been thinking about how better to document the data used by the app for a long time, but I haven't come up with a good solution yet. Initially I created a kind of Entity-Relationship diagram for Soukai models: Data schema for Media Kraken. And I guess SHACL shapes could accomplish the same, but I'm not sure they are more accessible for people who are not super familiar with RDF.

In the long run, versioning the SHACL shapes means that you/others could also express schema upgrade rules in RDF too (I have Cambria lenses in mind as inspiration here).

Yeah, this is also a problem I've had in mind for a while but I haven't solved yet. The idea of Cambria is awesome, and I'd love to see something like that for RDF. For now, I've just implemented manual migration of data in my apps, but that's not ideal.

I do want to also underline what looks like some differences in our perspectives

Yes, it's true that in my case I don't care much about "user-developers". I care about people using my app who may not know anything about RDF or even Solid.

For user-developers, I assume they already understand RDF and how Solid works, so they should be able to understand the data just by looking at it. Type indexes are a mechanism for interoperability between apps, I'm not using them for people to understand their own data.

If the user cannot easily understand how their data is connected, then they do not own it.

100% agree with that :). I always say that data that cannot be reused is like not having the data at all. But for that same reason, for most people using apps who are not developers, it doesn't matter how well "documented" or interlinked it is. If it doesn't work out of the box with other apps, it'll be worthless to them. So for me the important thing is that apps are interoperable.

I wrote this blog post a while ago, not sure if you've seen it: Interoperable Serendipity.

The idea of recipe lists does sound like it works for the use case where a user might share some of their recipes within a private but shared list. [...] I would share a direct link instead.

Yes, right now this is what I call "unlisted recipes" in Umai. When you share a recipe, you can make it private, public, or unlisted. The only difference between "public" and "unlisted" is that it's not included in that list of public recipes, but otherwise the permissions are the same.

Unfortunately I get an error message with no permissions changed or public type index created so haven't been able to see the public sharing in practice.

Do you mean that you got an error trying to create public recipes? Can you give me more details? (maybe open a separate issue if that's the case).

[josephguillaume]

Just so I can find it again, adding here a link to comment about a RecipeManager as the "main thing" solid/type-indexes#25 (comment)