enhance : Search for the name of the property item in "Search for artifacts" API.

[Yuutakasan]

When multiple people are developing or using many APIs, it becomes difficult to find out when and where the API was defined just by searching the name. If I want to search for APIs that handle similar data, I can use the property name, label, description, and other items to search.

The search for artifacts api does not have a function to search for a property name, so if I want to search for a property name, I need to open all the APIs and check them.

Search for artifacts API Document

Please add the ability to search by item property name.

Register a jsonshema that contains the property "post-office-box”.

{
  "$id": "https://example.com/address.schema.json",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "description": "An address similar to http://microformats.org/wiki/h-card",
  "type": "object",
  "properties": {
    "post-office-box": {
      "type": "string"
    },
    "extended-address": {
      "type": "string"
    },
    "street-address": {
      "type": "string"
    },
    "locality": {
      "type": "string"
    },
    "region": {
      "type": "string"
    },
    "postal-code": {
      "type": "string"
    },
    "country-name": {
      "type": "string"
    }
  },
  "required": [
    "locality",
    "region",
    "country-name"
  ],
  "dependencies": {
    "post-office-box": [
      "street-address"
    ],
    "extended-address": [
      "street-address"
    ]
  }
}

Searching for post-office-box does not return any results.

http://localhost:8080/api/search/artifacts?limit=10&order=asc&over=everything&search=post-office-box
api response
{"artifacts":[],"count":0}

[EricWittmann]

@Apicurio/developers what do you guys think about content based searching? :)

[carlesarnal]

I think this will make the search even slower for the streams storage although, IMO, it can be very useful. Maybe it can be used as a good opportunity to improve the search with proper indexes etc?

[famarting]

content based search? that will be very slow in every storage I think

I think it could be better to extract somehow the jsonschema properties to ArtifactMetaData properties, and then finish up the search api to search by properties (so we will be forced to decide how the search by properties will work)

[jsenko]

I think we considered using Lucene before, but decided against it? I think there are basically two options:

• Use some third-party tool or library like Lucene
• Fabian's idea - only search on derived metadata (do preprocessing to extract properties or other data that are interesting for searching)

[EricWittmann]

I think the concept of content-based searching is extremely useful. Certainly we can't do it by actually searching the content on-demand. I agree with the previous comments - we either need to extract the content as Fabian suggests, or we need to use an indexing engine like Lucene/Solr or Elasticsearch.

I would strongly prefer the latter approach I think, although we are already extracting meta-data from the content - so including properties in that extraction process is likely an easier/quick solution, and there is precedent for doing that.

[jsenko]

The downside of that could be occasionally being asked to add more types of extracted data. If we are OK with that I think this would be a good approach.
I guess we have to decide where the line is between extracting useful metadata for searching and extracting everything (essentially option 1).

[famarting]

two things:

About using lucene/solr ... via this approach we will have to extract metadata from artifacts anyway, because not all artifact types are represented as json and then cannot be automatically indexed by, for example, ElasticSearch...

so because we will have to extract metadata anyway and we already have in place some concepts in ArtifactMetaData (lables, properties, ...) useful for "content-based searching". I think it makes more sense to just use it and avoid adding yet another dependency on infrastructure.

[EricWittmann]

Overall I think I agree, but I can imagine more advanced content-based search use-cases where extracting and storing as tags or properties would not work. Basically any use-case that requires structured queries within the content. Perhaps we simply never do this (I'm fine with that).

[Yuutakasan]

I am glad that you are discussing this in this way.
I am planning to use the Apicurio registry when I am designing APIs using apicuio studio, to investigate the APIs and objects I have already defined.

So I registered an issue here in the past.
#752.

I'm wondering if it's possible to use apicurio registry to manage the registration of a data structure, for example, to keep the item name and pattern validation of an email address in a structured way, inherit it to manage the user object, and inherit the user object to manage the registrant object. I'm thinking about this.

In that case, I would like to be able to search by various search conditions, and as a first requirement, I would like to be able to search by property name, and if I want, I can use only the artifacts registered in the apicurio-registry. I'd like to be able to search by title or property name of the artifact.

[EricWittmann]

I'm wondering if this use case would be better solved by implementing an external integration via the registry's eventing system. Specifically, a custom event listener could update an external index (for example an Elasticsearch server) whenever a change is made to the registry (add, update, delete an artifact). The registry can fire events when these things happen, and a custom integration could then update the ES server.

That might be a more reasonable thing to consider, since I don't expect the core Registry application to have structured content search anytime soon (if ever).

[Yuutakasan]

Yes. It's great.

[Yuutakasan]

@EricWittmann
When I register this json-schema to the apicurio regisrty 2.0.0.Final server and run the following search, I get the result that all APIs are searched. Is there a difference in usage?

My expectation was that the following data would be searched.
$.properties.postal_code

http://127.0.0.1:8080/apis/registry/v2/search/artifacts?limit=10&order=asc&orderby=name&properties=properties:postal_code

Registered json-schema

{
    "$schema": "http://json-schema.org/draft-07/schema",
    "$id": "https://api.yodobashi.com/schema/address.json",
    "type": "object",
    "title": "address",
    "description": "住所データ型",
    "default": {},
    "examples": [
        {
            "postal_code": "101-0028",
            "prefecture": "東京都",
            "city": "千代田区",
            "address": "神田花岡町1-1"
        }
    ],
    "properties": {
        "postal_code": {
            "$id": "#/properties/postal_code",
            "type": "string",
            "title": "郵便番号",
            "description": "郵便番号データ型",
            "pattern": "^[0-9]{3}-[0-9]{4}$",
            "minLength": 8,
            "maxLength": 8,
            "examples": [
                "101-0028"
            ]
        },
        "prefecture": {
            "$id": "#/properties/prefecture",
            "type": "string",
            "title": "都道府県",
            "description": "都道府県データ型",
            "enum": [
                "東京都",
                "神奈川県",
                "千葉県"
            ],
            "examples": [
                "東京都"
            ]
        },
        "city": {
            "$id": "#/properties/city",
            "type": "string",
            "title": "市区町村",
            "description": "市区町村データ型",
            "enum": [
                "千代田区",
                "中央区",
                "豊島区"
            ],
            "examples": [
                "千代田区"
            ]
        },
        "address": {
            "$id": "#/properties/address",
            "type": "string",
            "title": "その他住所",
            "description": "その他住所データ型",
            "examples": [
                "神田花岡町1-1"
            ]
        }
    }
}