ElasticsearchProviderQueryMapping

Overview

pygeoapi's Elasticsearch provider provides the capability to offer up ES indexes via OGC API - Features and OGC API - Records. This wiki page explains how OGC API queries are translated into their equivalent ES queries.

Technical Details

• code is at https://github.com/geopython/pygeoapi/blob/master/pygeoapi/provider/elasticsearch_.py
• support is for ES 7+
• the basic setup for ES to pygeoapi integration is having an ES index of GeoJSON documents, and explicitly setting the geometry object as a ES geo_shape type in the index mappings
• a custom provider can be written to work with alternative ES setups, which would then serialize results as GeoJSON for pygeoapi upstream callers
• a useful litmus test is running ogrinfo directly against a given ES index. If the ES index is compatible with GDAL/OGR, it will be compatible with pygeoapi's ES provider
• a given ES index requires the document _id to be consistent with an equivalent property (can be named anything) in the underlying GeoJSON document's properties object (this allows pygeoapi to do id queries with consistency
• for OGC API - Records, we exclude certain internal fields which are not displayed to the user
• limit and startindex parameters from the OGC API endpoint are not passed in the ES request payload, but as parameters to the Python Elasticsearch API

Requests

items endpoint with no query parameters

URL: http://localhost:5000/collections/records/items?f=json

ES request:

{
    "track_total_hits": true,
    "query": {
        "bool": {
            "filter": []
        }
    },
    "_source": {
        "excludes": [
            "properties._raw_metadata"
        ]
    }
}

items endpoint with bbox parameter

URL: http://localhost:5000/collections/records/items?f=json&bbox=-142,42,-52,84

ES request:

{
    "track_total_hits": true,
    "query": {
        "bool": {
            "filter": [
                {
                    "geo_shape": {
                        "geometry": {
                            "shape": {
                                "type": "envelope",
                                "coordinates": [
                                    [
                                        -142.0,
                                        84.0
                                    ],
                                    [
                                        -52.0,
                                        42.0
                                    ]
                                ]
                            },
                            "relation": "intersects"
                        }
                    }
                }
            ]
        }
    },
    "_source": {
        "excludes": [
            "properties._raw_metadata"
        ]
    }
}

items endpoint with bbox and property parameters

URL: http://localhost:5000/collections/records/items?f=json&bbox=-142,42,-52,84&title=foo

ES request:

{
    "track_total_hits": true,
    "query": {
        "bool": {
            "filter": [
                {
                    "geo_shape": {
                        "geometry": {
                            "shape": {
                                "type": "envelope",
                                "coordinates": [
                                    [
                                        -142.0,
                                        84.0
                                    ],
                                    [
                                        -52.0,
                                        42.0
                                    ]
                                ]
                            },
                            "relation": "intersects"
                        }
                    }
                },
                {
                    "match": {
                        "properties.title": "foo"
                    }
                }
            ]
        }
    },
    "_source": {
        "excludes": [
            "properties._raw_metadata"
        ]
    }
}

items endpoint with bbox and q parameters

URL: http://localhost:5000/collections/records/items?f=json&bbox=-142,42,-52,84&q=foo

ES request:

{
    "track_total_hits": true,
    "query": {
        "bool": {
            "filter": [
                {
                    "geo_shape": {
                        "geometry": {
                            "shape": {
                                "type": "envelope",
                                "coordinates": [
                                    [
                                        -142.0,
                                        84.0
                                    ],
                                    [
                                        -52.0,
                                        42.0
                                    ]
                                ]
                            },
                            "relation": "intersects"
                        }
                    }
                }
            ],
            "must": {
                "query_string": {
                    "query": "foo"
                }
            }
        }
    },
    "_source": {
        "excludes": [
            "properties._raw_metadata"
        ]
    }
}

items endpoint with q and datetime parameters

URL: http://localhost:5000/collections/records/items?f=json&q=metar&datetime=2020-09-11T00:00:00Z/2020-09-12T00:00:00Z

ES request:

{
    "track_total_hits": true,
    "query": {
        "bool": {
            "filter": [
                [
                    {
                        "range": {
                            "properties.datetime": {
                                "gte": "2020-09-11T00:00:00Z",
                                "lte": "2020-09-12T00:00:00Z"
                            }
                        }
                    }
                ]
            ],
            "must": {
                "query_string": {
                    "query": "metar"
                }
            }
        }
    },
    "_source": {
        "excludes": [
            "properties._raw_metadata"
        ]
    }
}

items endpoint with bbox, datetime, and q parameters

URL: http://localhost:5000/collections/records/items?f=json&bbox=-152,42,-52,84&q=metar&datetime=2020-09-11T00:00:00Z/2020-09-12T00:00:00Z

ES request:

{
    "track_total_hits": true,
    "query": {
        "bool": {
            "filter": [
                {
                    "geo_shape": {
                        "geometry": {
                            "shape": {
                                "type": "envelope",
                                "coordinates": [
                                    [
                                        -152.0,
                                        84.0
                                    ],
                                    [
                                        -52.0,
                                        42.0
                                    ]
                                ]
                            },
                            "relation": "intersects"
                        }
                    }
                },
                [
                    {
                        "range": {
                            "properties.datetime": {
                                "gte": "2020-09-11T00:00:00Z",
                                "lte": "2020-09-12T00:00:00Z"
                            }
                        }
                    }
                ]
            ],
            "must": {
                "query_string": {
                    "query": "metar"
                }
            }
        }
    },
    "_source": {
        "excludes": [
            "properties._raw_metadata"
        ]
    }
}