Timeplus Schema Linter

[mfreeman451]

Carver Automation/ServiceRadar

Would like to have a linter that would check my schemas before deployment, would speed up development/testing time quite a bit.

I could contribute this myself, would just need a little guidance on how to find documentation around everything needed to put the rules/expert system together.

.m

Use case

Testing/Developing timeplus schemas

Describe the solution you'd like

A standalone CLI tool that can be used to validate Timeplus schemas

Describe alternatives you've considered

N/A

Additional context

No response

[chenziliang]

May you give some examples of the linter ? How do you like to use it ?

[mfreeman451]

timeplus-linter migration.sql and it would tell me if I have any syntax issues, like if I was using argMax instead of arg_max. If I threw in a bad schema into timeplus right now, it would reject it/throw an error/give me a hint as to what I did wrong.

[chenziliang]

Thanks @mfreeman451.

proton already has sqlanalyzer REST endpoint which may be helpful for this use case.

curl http://127.0.0.1:3218/proton/v1/sqlanalyzer -X POST -d '{"query": "select window_start as _tp_time, count() from tumble(test, 5s) group by window_start emit timeout 6s;"}'

If we like to have a sub command of proton, we can follow existing pattern like this.

k@t+ proton % ./proton-v3.0.2-Darwin-arm64                                                                                                                                                                                               
Use one of the following commands:
proton client [args] 
proton benchmark [args] 
proton server [args] 
proton format [args] 
proton git-import [args] 
proton nlog [args] 
proton meta [args] 
proton python [args] 
proton klog-benchmark [args] 
proton install [args] 
proton start [args] 
proton stop [args] 
proton status [args] 
proton restart [args] 
proton hash-binary [args] 

So the linter usage may be something like

proton linter --user username --password password --query "...."

or

proton linter --user username --password password  --sqlfile my.sql 

You can check the sqlanalyzer code to see how it works and can borrow the same idea in the linter

https://github.com/timeplus-io/proton/blob/develop/src/Server/RestRouterHandlers/SQLAnalyzerRestRouterHandler.cpp#L284-L324

Ref:

https://github.com/timeplus-io/proton/wiki/Contributing
https://github.com/timeplus-io/proton/wiki/Proton-Code-Process-for-Engineers
https://github.com/timeplus-io/proton/wiki/Stream-Smoke-Test
https://github.com/timeplus-io/proton/wiki/Writing-Stream-Smoke-Test

[mfreeman451]

brilliant, thank you

[yokofly]

The entry point is in the programs folder: https://github.com/timeplus-io/proton/blob/develop/programs/CMakeLists.txt.
For testing, I think editing the stateless suites [tests/queries_ported/0_stateless/] and related bash script is sufficient

Smoke cases is much more complex.
https://github.com/timeplus-io/proton/blob/develop/tests/cluster/smoke/README.md