Merge pull request #556 from KxSystems/v.1.11.0-dev-to-main

V.1.11.0 dev to main

Author: Philip-Carneiro-KX

CHANGELOG.md

@@ -2,6 +2,26 @@
 
 All notable changes to the **kdb VS Code extension** are documented in this file.
 
+# v1.11.0
+
+### Enhancements
+
+- Added UDA tab to the Datasource Files
+- Added the **Reset Scratchpad** option for insights connections version 1.13 or higher
+- Allow users to create custom authentication logic
+- Improved language server performance and resource usage
+- Support Query Environment Enabled for Insights connections.
+
+### Fixes
+
+- Fixed kdb results view theme issue
+- Fixed case insensitivity for connection labels
+
+### Internal Improvements
+
+- Improved ESlint rules for the codebase
+- Merged PyKX kdb+ structuredText changes
+
 # v1.10.2
 
 ### Fixes

README.md

@@ -62,6 +62,8 @@ Once the **kdb VS Code extension** is installed **KX** appears in the Activity B
 - [Workbooks](#workbooks)
 - [Query History](#query-history)
 
+**Note** Customized authentication has been implemented for the kdb VS Code extension, allowing you to add custom logic when authenticating with kdb. Refer to [customized authentication](https://github.com/KxSystems/kx-vscode-auth) for details on how to set this up.
+
 ### Installing q
 
 After you install **kdb VS Code extension**, if q is not already installed the extension provides a seamless integration with q, by displaying a notification with an option to download, register and install [kdb Insights Personal Edition](https://kx.com/kdb-insights-personal-edition-license-download/). For details on the other versions available see [here](#versions-available).
@@ -79,8 +81,6 @@ After you install **kdb VS Code extension**, if q is not already installed the e
 
 Once registered you will receive an email with you license details. The base64 encoded license string can be found in the welcome email received after registration, under the download link for the license file.
 
-![welcomeemaillicense](https://github.com/KxSystems/kx-vscode/blob/main/img/weclomeemail.jpg?raw=true)
-
 With your license details to hand, you can link this to VS Code by either choosing **Paste license string** or **Select license file** from your PC. The latter method is recommended for new users.
 
 ![findlicense](https://github.com/KxSystems/kx-vscode/blob/main/img/pastelicense.jpg?raw=true)
@@ -104,7 +104,7 @@ There are commercial and non-commercial editions available. We recommend you sta
 | Edition                                                                                         | write q | run q queries | explore results | shared kdb process with kdb Insights |
 | ----------------------------------------------------------------------------------------------- | ------- | ------------- | --------------- | ------------------------------------ |
 | [kdb+ Personal Edition](https://kx.com/kdb-personal-edition-download/)                          | yes     | yes           | yes             | no                                   |
-| [kdb Insights Personal Edition](https://kx.com/kdb-insights-personal-edition-license-download/) | yes     | yes           | yes             | no                                   |
+| [kdb Insights SDK Personal Edition](https://kx.com/kdb-insights-sdk-personal-edition-download/) | yes     | yes           | yes             | no                                   |
 | **kdb Insights Enterprise**                                                                     | yes     | yes           | yes             | yes                                  |
 
 > **kdb Insights Enterprise** requires a commercial license. Please contact licadmin@kx.com for further information.
@@ -218,7 +218,7 @@ If you close the extension, the connection also closes.
 
 #### Meta
 
-When connected **Insights** connections can be expanded to show the details returned by the [getMeta API](https://code.kx.com/draft/insights/api/database/query/get-meta.html) call, which provides information on the database schemas and all the analytics available.
+When connected **Insights** connections can be expanded to show the details returned by the [getMeta API](https://code.kx.com/insights/api/database/query/get-meta.html) call, which provides information on the database schemas and all the analytics available.
 
 ![Insights Meta Tree](https://github.com/KxSystems/kx-vscode/blob/main/img/insights-meta-tree.png?raw=true)
 
@@ -490,7 +490,92 @@ To create a data source and run it against a specific connection:
 
 In addition to [API queries](https://code.kx.com/insights/api/database/query/get-data.html), if the query environment is enabled on the deployed instance of **kdb Insights Enterprise**, qSQL and SQL queries can be used within a data source with the appropriate parameterization. If qSQL or SQL is required and issues occur trying to run these queries contact a kdb Insights Enterprise administrator for assistance.
 
-### Populate scratchpad
+### API queries
+
+The getData API provides a method of querying a table using a defined set of parameters. These parameters can be configured through the getData form available in the VSCode extension.
+
+Refer to the [`getData` API](https://code.kx.com/insights/api/database/query/get-data.html) documentation for more information and a full list of available parameters.
+
+### QSQL queries
+
+The `.com_kx_edi.qsql` API is a QSQL query builder that assembles QSQL queries based on a q expression. It is a developer tool that allows running freeform q code against a specific database tier. 
+
+This function runs an QSQL query.
+
+```
+.com_kx_edi.qsql[args]
+```
+
+**Note**: Along with the query itself, you must also specify the target database and tier.
+
+Refer to the [QSQL documentation](https://code.kx.com/insights/api/database/query/qsql.html) for more details.
+
+**Warning!** Starting with kdb Insights Enterprise version 1.13, QSQL queries and populating QSQL only work if the Query Environment (QE) is enabled. Ensure you have enabled QEs to use QSQL; they are disabled by default in kdb VS Code. Refer to [Query Environments](https://code.kx.com/insights/enterprise/configuration/base.html#query-environments) for more details.
+
+### SQL queries
+
+The `.com_kx_edi.sql` SQL API allows running freeform SQL queries. Each query is distributed across all available databases. The results are then aggregated and returned as a single dataset.
+
+This function runs an SQL query.
+
+```
+.com_kx_edi.sql[query]
+```
+
+Refer to the [SQL documentation](https://code.kx.com/insights/api/database/query/sql.html) for more details.
+
+### UDA queries
+
+User-Defined Analytics (UDAs), also known as custom APIs, are essential for developers to leverage the capabilities of kdb when using Insights Enterprise. These UDAs are deployed to Insights through the Data Access Processes (DAPs) and Aggregators (Aggs).
+
+UDAs can be called directly within the VSCode extension through the UDA tab in a data source. This provides a form-based approach to populating the UDA parameters. UDAs can also be called within the Insights Enterprise web interface using Pipelines, Queries, and Views.
+
+![Query UDAs in VSCode extension](https://github.com/KxSystems/kx-vscode/blob/main/img/udas-query.png?raw=true)
+
+When interacting with UDAs, parameter fields are shown for configuration. Note the following:
+
+- Required fields are marked with an asterisk (*).
+
+- Optional fields are not required but may be displayed in your results, even if they are empty.
+
+You can add new parameters by clicking **Add Parameter**. Both optional and distinguished parameters can be added as needed.
+
+![Add parameters to call UDAs](https://github.com/KxSystems/kx-vscode/blob/img/udas-add-parameters.png?raw=true)
+
+**Important!** A UDA cannot be queried if one or more parameters are invalid.
+
+If you attempt to run a UDA with invalid parameters, an error occurs and a pop-up message appears to alert you to the issue.
+
+![Error showing invalid parameters for UDA](https://github.com/KxSystems/kx-vscode/blob/img/udas-invalid-parameter.png?raw=true)
+
+![Pop-up message UDAs include invalid parameter](https://github.com/KxSystems/kx-vscode/blob/img/udas-error-pop-up.png?raw=true)
+
+In some cases, you can successfully query UDAs without any parameters, as seen in the screenshot below.
+
+![UDAs with no parameters](https://github.com/KxSystems/kx-vscode/blob/img/udas-no-parameters.png?raw=true)
+
+However, you can still modify the parameter list to add parameters by clicking **Add parameter** or deleting parameters using the recycle bin icon.
+
+![Delete UDA parameters](https://github.com/KxSystems/kx-vscode/blob/img/udas-delete-parameters.png?raw=true)
+
+For more information on User-Defined Analytics, refer to the [UDAs documentation](https://code.kx.com/insights/api/database/uda/uda-overview-introduction.html).
+
+### Run and populate scratchpad
+
+Running and populating scratchpad are two actions used to execute q code, allowing you to run queries and perform operations on your kdb Insights Enterprise database but they differ in how and where the output is stored and accessed.
+
+The [QSQL API](#qsql-queries) is designed to run queries against a specific Insights database and tier. However, sometimes you might need to run more flexible or freeform queries against a wider set of data. For this purpose each Insights Enterprise user is assigned a 'scratchpad' q process. This scratchpad is separate from the dedicated query processes, allowing you to run freeform code without impacting the overall system.
+
+You can populate variables in your scratchpad with the results from your queries, and then perform freeform manipulations on those variables.
+
+After you configure a data source in the VSCode extension you have two options for executing it:
+
+- **Run** executes your query directly against the relevant API (getData, QSQL, SQL, or UDA) and displays the results in the extension
+- **Populate Scratchpad** executes your query through the scratchpad, which passes it to the relevant API, assigns the results to a variable of your choosing in your scratchpad process and then displays them in the VSCode extension
+
+For more details on populating scratchpad, refer to the [populate scratchpad](#populate-scratchpad) section below.
+
+#### Populate scratchpad
 
 You can use a data source to populate a scratchpad process running in a **kdb Insights Enterprise** instance with a dataset. This allows you to then execute q or python code against the data stored in that variable in the scratchpad. This facilitates the generation of complex APIs and pipelines within VS Code and kdb Insights Enterprise.
 
@@ -508,6 +593,26 @@ To do this:
 
 1. Use a [Workbook](#workbooks) to execute q or Python code against the data in your scratchpad using the variable you provided.
 
+### Reset scratchpad
+
+The **Reset Scratchpad** option in kdb VS Code allows you to delete all the data from a connected instance and restart your development from scratch.
+
+There are several ways to reset the scratchpad:
+
+1. Right click on the appropriate connection on the left-hand side and select **Reset Scratchpad**. The connection can be either active or idle but it must be connected.
+
+   ![Reset Scratchpad with right click on connected instance](https://github.com/KxSystems/kx-vscode/blob/main/img/reset-scratchpad-right-click.png?raw=true)
+
+   This displays a dialog box at the bottom right corner asking you to confirm your action.
+
+   ![Dialog to confirm reset scratchpad](https://github.com/KxSystems/kx-vscode/blob/main/img/reset-scratchpad-dialog.png?raw=true)
+
+2. Using the **Command Palette** in VSCode, start typing **Reset Scratchpad** and click on the option as it shows. This action resets the scratchpad for the active connection.
+
+   ![Reset scratchpad using the Command Palette](https://github.com/KxSystems/kx-vscode/blob/main/img/reset-scratchpad-command-palette.png?raw=true)
+
+3. Use the [MacOS or Windows shortcuts](#shortcuts) in the q file or in the workbook. Note that running the shortcuts in the q file resets the scratchpad for the active connection, while running them in the workbook resets the scratchpad for the connection chosen in the workbook.
+
 ## Workbooks
 
 Workbooks provide a convenient way to prototype and execute q and python code against a q process and using the variables [populated into the scratchpad](#populate-scratchpad) of a **kdb Insights Enterprise** deployment by data sources.
@@ -755,6 +860,7 @@ The following setting will change double click behaviour to select the whole ide
 | Ctrl + Shift + D   | Execute entire file               |
 | Ctrl + Shift + R   | Run q file in new q instance      |
 | Ctrl + Shift + Y   | Toggle paramater cache for lambda |
+| Ctrl + Shift + Delete   | Reset scratchpad |
 
 ### For MacOS
 
@@ -768,3 +874,4 @@ The following setting will change double click behaviour to select the whole ide
 | ⌘ + Shift + D   | Execute entire file               |
 | ⌘ + Shift + R   | Run q file in new q instance      |
 | ⌘ + Shift + Y   | Toggle paramater cache for lambda |
+| ⌘ + Shift + Delete   | Reset scratchpad |

eslint.config.cjs

@@ -0,0 +1,77 @@
+const js = require("@eslint/js");
+const tseslint = require("typescript-eslint");
+const headerPlugin = require("eslint-plugin-header");
+const licenseHeaderPlugin = require("eslint-plugin-license-header");
+const unusedImportsPlugin = require("eslint-plugin-unused-imports");
+const importPlugin = require("eslint-plugin-import");
+
+const currentYear = new Date().getFullYear();
+
+module.exports = [
+  {
+    ignores: ["**/*.d.ts", "**/*.js", "src/ipc/**"],
+  },
+  js.configs.recommended,
+  ...tseslint.configs.recommended,
+  {
+    languageOptions: {
+      parser: tseslint.parser,
+      parserOptions: {
+        ecmaVersion: 2018,
+        sourceType: "module",
+      },
+    },
+    plugins: {
+      header: headerPlugin,
+      import: importPlugin,
+      "license-header": licenseHeaderPlugin,
+      "unused-imports": unusedImportsPlugin,
+    },
+    rules: {
+      "@typescript-eslint/no-use-before-define": "off",
+      "@typescript-eslint/explicit-function-return-type": "off",
+      "@typescript-eslint/no-non-null-assertion": "off",
+      "@typescript-eslint/explicit-module-boundary-types": "off",
+      "@typescript-eslint/no-explicit-any": "off",
+      "license-header/header": [
+        "error",
+        [
+          "/*",
+          ` * Copyright (c) 1998-${currentYear} Kx Systems Inc.`,
+          " *",
+          ' * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the',
+          " * License. You may obtain a copy of the License at",
+          " *",
+          " * http://www.apache.org/licenses/LICENSE-2.0",
+          " *",
+          " * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an",
+          ' * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the',
+          " * specific language governing permissions and limitations under the License.",
+          " */",
+        ],
+      ],
+      "import/order": [
+        "error",
+        {
+          groups: [
+            ["builtin", "external"],
+            ["internal", "parent", "sibling", "index"],
+          ],
+          "newlines-between": "always",
+          alphabetize: { order: "asc", caseInsensitive: true },
+        },
+      ],
+      "@typescript-eslint/no-unused-vars": "off",
+      "unused-imports/no-unused-imports": "error",
+      "unused-imports/no-unused-vars": [
+        "warn",
+        {
+          vars: "all",
+          varsIgnorePattern: "^_",
+          args: "after-used",
+          argsIgnorePattern: "^_",
+        },
+      ],
+    },
+  },
+];