dft also provides an HTTP server that exposes a simple REST interface to DataFusion.
dft serve-httpThe current endpoints provided are:
/sql => Make POST requests with body { sql: string, flightsql?: bool }
/catalog => View the catalog for the database, optionally accepts a flightsql query param
/table/{CATALOG}/{SCHEMA}/{TABLE} => Fetch records from the provided table, Optionally accepts a flightsql query param
Require basic or bearer authentication to make requests.
[flightsql_server.auth]
bearer_token = "MyToken"
basic_auth.username = "User"
basic_auth.password = "Pass"Prometheus metrics are automatically published.
[flightsql_server]
server_metrics_port = "0.0.0.0:9000"Something useful that comes from having an HTTP server is that we can leverage an extensive ecosystem of the HTTP load generation tools to benchmark our performance.
To get started with benchmarking you will first want to install the necessary tools.
# Installs `oha` which is a CLI for HTTP benchmarking
just install-toolsThen you will want to start the dft HTTP server in one terminal window.
just serve-httpFinally, you can run a simple benchmark to make sure all the flows are working.
# Benchmarks running `SELECT 1`
just bench-http-basicYou can add your own benchmark files in the bench/url_files directory to cater the benchmarks to your needs. An example file is provided that you can use as reference.
just bench-http-custom example_custom_bench.txt