version: 0.8.3

Author: ohsayan

versioned_docs/version-0.8.3/a.installation.md

@@ -0,0 +1,101 @@
+---
+id: installation
+title: Installation
+---
+
+Getting started with Skytable involves choosing a mode of installation, downloading any required files and then starting up the database. You can choose to either use:
+
+- [**Native binaries (recommended)**](#native-binaries): This is what is generally recommended for the best performance. You will need to download a bundle and then start the server binary; no expert knowledge required
+- [**Using a Debian package (recommended)**](#debian-package): If you're deploying on Ubuntu or any other Debian based Linux distribution, then consider using this method. Configuration files, users and passwords are autogenerated.
+- [**A Docker image**](#docker-image): We generally recommend using a Docker image for experimenting with Skytable on your local system during development and you want to keep your local system *clean*. If you want to use a Docker image for deployment, you're always free to do so!
+  > **Note:** You might experience slightly degraded performance from the storage engine due to Docker engine's abstractions.
+
+:::tip
+All client tools (such as `skysh` and `sky-bench`) *can* use the `SKYDB_PASSWORD` variable for authentication. If you're using Skytable in a testing environment and frequently need to use `skysh`, you may consider setting this variable to your password to avoid having to pass the `--password` argument every time.
+
+However, we strongly recommend **not** using it outside testing environments.
+:::
+
+## Native binaries
+
+To use native binaries you need to download a bundle which is simply a ZIP file with all the necessary binaries that you'll ever need to develop on and deploy Skytable.
+
+1. **First download the latest bundle** for your platform. You can find [download links on the releases page](https://github.com/skytable/skytable/releases/v0.8.3).
+2. **Unzip the ZIP file**. You'll find the following binaries in the extracted archive:
+   - `skyd`: This is the database server binary which when started runs as a daemon, serving requests
+   - `skysh`: This is the Skytable shell and it provides a very helpful interactive REPL database client
+   - `sky-bench`: This is the benchmarking tool that you can use to load test Skytable
+3. **Start up the server**. You need to choose a `root` password for the `root` account which will have complete control over the database.
+
+    ```bash
+    ./skyd --auth-root-password=<your root password>
+    ```
+
+    **Replace with your own secure password!**
+
+    Explanation:
+    - `--auth-root-password`: sets the root password
+
+The server starts up at `localhost:2003` and is ready to run queries.
+
+:::info
+Your operating system might sometimes not let you run binaries directly. On Unix based systems, you'll need to run: `chmod +x skyd skysh sky-bench`.
+
+And on Windows systems you might need to right-click on the binaries and click on "unblock"
+:::
+
+## Debian package
+
+Find the correct `*.deb` file [from the releases page](https://github.com/skytable/skytable/releases). Now simply run:
+
+```sh
+sudo dpkg -i <file name>.deb
+```
+
+The package will:
+
+- **Generate a root password:** Watch the terminal output!
+- **Create a `systemd` unit**: So you can start and stop the process using `systemd` like `systemd start skyd`
+- **Generate a configuration**: Your configuration is stored in `/var/lib/skytable/config.yaml`. Go ahead and modify it if you need to!
+
+## Docker image
+
+:::info You must have docker set up!
+
+- Use [this great guide from Docker](https://docs.docker.com/engine/install/) to install and get started
+- To be able to run `docker run` and related commands, you may need administrative privileges
+:::
+
+### Simple setup
+
+1. **Download the bundle**: To be able to run queries you need to download the bundle as described above
+2. **Start the container**:
+
+    ```shell
+    docker run -d --name skydb -p 2003:2003 skytable/skytable:v0.8.3
+    ```
+
+:::tip
+The password for the Skytable instance on the Docker container is auto-generated. Run `docker logs -f skydb` and you'll see a log
+message with the generated password.
+:::
+
+### With persistence
+
+1. **Download the bundle**: To be able to run queries you need to download the bundle as described above
+2. **Create the data directory**: To ensure that our database is persistent and all our data doesn't vanish as soon as the container is terminated, we'll map the data directory to an actual directory on our local system.
+    > **Note:** Create a folder called `skytable` in a convenient location. We recommend having a directory in `$HOME/docker-containers` where you can store the Skytable container's data and any other containers that you might use. It's a great way to keep things organized.
+3. **Create your configuration**: [Download this template file](https://raw.githubusercontent.com/skytable/skytable/v0.8.3/examples/config-files/template.yaml) and place it into the directory you created. Update the password with your `root` password of choice.
+4. **Start the container**:
+
+     ```shell
+     docker run -d --name skydb \
+        -v $HOME/docker-containers/skytable:/var/lib/skytable \
+        -p 2003:2003 \
+        skytable/skytable:v0.8.3
+     ```
+
+     Explanation:
+     - This starts a container with name `skydb`
+     - It maps the folder (as discussed earlier) `$HOME/docker-containers/skytable` from your local file system to `/var/skytable` (in the container's file system)
+     - Maps port `2003` on the host to the containers port `2003` so that you can use the command-line client `skysh` without having to inspect the container's IP address

versioned_docs/version-0.8.3/b.using-repl.md

@@ -0,0 +1,97 @@
+---
+id: using-the-repl
+title: Using the REPL
+---
+
+Once you've set up Skytable [following our guide](installation), you can now get started using the REPL. Note that you must have downloaded the bundle. If not, go back to the installation guide and do it now.
+
+## Starting the repl
+
+Run this to start the command-line REPL:
+
+```sh
+./skysh
+```
+
+The REPL will then ask you for a password which you need to type in. This is the root password that you would have set during your installation, so if you don't remember it go ahead and check what you set during the installation step.
+
+:::tip
+Keep the root password safe! It's the admin access to your database and without it you won't be able to create or drop or alter
+or ... do anything administrative. But if you do happen to lose it, there's an easy way to reset the password (this however requires you to stop the database, which is also the case with many other databases, and that's primarily for security).
+
+You can read more in the [configuration](system/configuration) page.
+:::
+
+## Using the REPL
+
+You will now see a welcome message and the REPL will prompt you to run something. Now is a good time to run `sysctl report status` which should just print out `(Okay)` in a cyan shade:
+
+```sh
+> sysctl report status
+(Okay)
+```
+
+You can also run queries like: `inspect global` to see available global system information.
+
+:::info Quick notes
+
+- The REPL stores all command history in a file located at `$HOME/.sky_history` (here `$HOME` represents your home directory)
+- If you wish to change where the REPL stores the history file, set the `SKYSH_HISTORY_FILE` environment variable to your preferred path
+- The REPL will automatically parameterize queries. Don't worry about what this means; you'll learn about it ahead.
+- The REPL applies custom formatting to `DDL` queries. For example, even though  `inspect global` returns a JSON as a string,
+the REPL formats it and outputs it without quotes, to improve readability
+- To connect using different settings (the REPL will attempt to authenticate as `root` by default), see the options using `skysh --help`
+:::
+
+## First steps
+
+Skytable's data model is discussed in depth on [this page](architecture#data-model), but let us understand some basics. If you've used a SQL
+database, you would be used to the idea of a `database` — just like this, Skytable has `space`s. A `space` is a collection
+of `models` (which are like SQL's `table`s with slightly different functionality) and other containers.
+
+### Create a space
+
+Let us create a `space`:
+
+```sql
+CREATE SPACE myspace
+```
+
+### Create a model
+
+Let us create a `model`. We want to store something that resembles the structure:
+
+```json
+{"username": "string username", "password": "string password", "notes": []}
+```
+
+To do this, we create the following Skytable model:
+
+```sql
+CREATE MODEL myspace.mymodel(username: string, password: string, notes: list { type: string })
+```
+
+### Add, update and remove some data
+
+- **Insert some data**: 
+  ```sql
+    INSERT INTO myspace.mymodel('sayan', 'password123', [])
+  ```
+- **Update some data**:
+    ```sql
+    UPDATE myspace.mymodel SET notes += "mynewnote" WHERE username = 'sayan'
+    ```
+- **Select some data**:
+    ```sql
+    SELECT notes, password FROM myspace.mymodel WHERE username = 'sayan'
+    ```
+
+## Use in your own apps
+
+While we would recommend you to learn BlueQL and more about Skytable's architecture by using the REPL, you can always start using Skytable
+in your own apps. [Find a client driver for your language or framework here](libraries).
+
+Once you've found the driver for your framework, you can come back here and follow on with the guide. Be sure to check the driver's
+documentation to see how the client driver should be used.
+
+Good luck!

versioned_docs/version-0.8.3/blueql/1.overview.md

@@ -0,0 +1,185 @@
+---
+title: Overview
+id: overview
+---
+
+In this document we explore some of the meta parts of BlueQL. If you want to look at how you can use BlueQL, consider looking at the sections that follow.
+
+Design principles:
+- **Simplicity and clarity**: The language shouldn't be overwhelming to understand
+- **Security with mandatory parameterization**: We want to reduce the surface of injection attacks. For this reason, [parameterization is mandatory](#parameters).
+
+Just like SQL, BlueQL has three categories of commands/queries inside it:
+
+- **DDL**: Data definition language is used to define, modify and/or remove DDL objects such as `space`s and `model`s
+- **DCL**: Data control language is used to control the access to data, and perform other administrative tasks
+- **DML**: Data manipulation language is used to manipulate data
+
+Jump to [differences from SQL](#differences-from-sql).
+
+:::info
+This text is *not* a detailed, formal guide. It's meant for developers and users who want to work with 
+Skytable.  If you need a more formal specification, like a grammar definition, please ask us, and we'll create 
+it. We haven't published it yet because no one has requested it.
+:::
+
+## Identifiers
+Can begin with any ASCII alphabet or an underscore (`_`) and then have any number of alphanumeric characters and/or underscores.
+
+## Keywords
+
+Keywords are identifiers with special meanings and hence can't be used as identifiers in other places. Here's a full-list of 
+keywords:
+
+```ts
+[
+    "sysctl", "create", "alter", "drop", "use", "inspect", "describe", "insert", "select", "update",
+    "delete", "exists", "table", "model", "space", "index", "type", "function", "rename", "add",
+    "remove", "transform", "set", "return", "order", "sort", "group", "limit", "asc", "desc", "all",
+    "by", "with", "on", "from", "into", "as", "to", "in", "of", "and", "or", "not", "if", "else",
+    "where", "when", "allow", "auto", "default", "null", "transaction", "batch", "lock", "read",
+    "write", "begin", "end", "key", "value", "primary", "truncate"
+]
+```
+
+## Data types
+
+### Boolean
+A boolean value, either `true` or `false`
+
+### Unsigned integers
+
+- `uint8`: unsigned 8-bit integer
+- `uint16`: unsigned 16-bit integer
+- `uint32`: unsigned 32-bit integer
+- `uint64`: unsigned 64-bit integer
+
+### Signed integers
+
+- `sint8`: signed 8-bit integer
+- `sint16`: signed 16-bit integer
+- `sint32`: signed 32-bit integer
+- `sint64`: signed 64-bit integer
+
+### Floating point values
+
+- `float32`: a single-precision float
+- `float64`: a double-precision float
+
+### Simple collections
+
+- `binary`: a binary blob represented as a sequence of `uint8` values
+- `string`: an UTF-8 string
+
+### Complex collections
+
+- `list`: a list of any of the data types, including nested lists
+  - A list is represented as: `[]` with values inbetween. For example, a `list { type:string }` would be represented as:
+    ```sql
+    ["sayan", "loves", "dogs"]
+    ```
+  - **Lists cannot contain null values**
+  - **List can be nested**: You can have heavily nested lists like: `[[[]], [["another one"]]]`
+  - **List can only have one base type**: This means that if you have a list like `[[[string]]]` each element must either be the same nested list, or an empty list
+
+:::info Note
+New data types are frequently added, so treat this list as non-exhaustive.
+:::
+
+## Literals
+
+- Null literal: `null`
+- Numeric literals:
+  - Unsigned: `1234`
+  - Signed: `[-]1234`
+- Floating point literals: `[-]1234.5678`
+- String literals: `"hello"`
+- Binary literals: \`binary\`
+- List literals: `[..]`
+- Dictionaries: `{<ident>: <literal>}`
+
+:::warning Literals are not available everywhere
+It is very important for you to know that literals are not allowed everywhere. The only literals allowed everywhere are:
+- Lists
+- Dictionaries
+
+Read below to understand why.
+:::
+
+## Parameters
+
+All literals apart from dictionaries and lists must be used as parameters. **BlueQL only allows literals as parameters**. For example, using the Rust client, if you were to run this:
+
+```sql
+insert into myspace.mymodel('sayan', 'pass123', ['myfirstnote'])
+```
+
+You are required to parameterize the query like this:
+
+```rust
+use skytable::query;
+
+let query = query!("insert into myspace.mymode(?, ?, [?])", "sayan", "pass123", "myfirstnote")
+```
+
+If you try to run it without any parameters (don't forget that `skysh` automatically parameterizes for convenience) the query
+won't even compile.
+
+:::tip
+Just so you know, parameterizing involves passing a separate parameter list, with each parameter encoded. You wouldn't need to worry about this because the client driver does all of that for you!
+
+**The question is why? The answer is security.** SQL-injection vulernabilties have already costed companies a lot, so we don't
+want to inherit that from SQL.
+
+**Also, parameterization is exclusively possible for literals**. This means that whenever you're accepting data from an untrusted
+source, it becomes a parameter. If you try to not use parameters, the query will not even compile.
+
+On a final note, BlueQL doesn't support comments of any form also for security reasons.
+:::
+
+## Expressions
+
+- `+=`: add RHS to LHS
+  - Can be used outside arithmetic contexts
+  - Add a char to a field `mystring`: `mystring += ",world"`
+  - Add a list to a nested list field: `mylist += ["item in nested list"]`
+- `-=`: subtract RHS from LHS
+- `/=`: divide LHS by RHS
+- `*=`: multiply LHS by RHS
+
+## DDL
+
+Queries include:
+- Spaces:
+  - `CREATE SPACE myspace [WITH { property: value, ... }]`
+  - `ALTER SPACE myspace [WITH { property: updated_value, ... }]`
+  - `DROP SPACE [allow not empty] myspace`
+- Models:
+  - `CREATE MODEL myspace.mymodel([primary] [null] field: field_type, ...) [WITH { property: value, ... }]`
+  - `ALTER MODEL myspace.mymodel (ADD ... | UPDATE ... | REMOVE ...)`
+  - `DROP MODEL [allow not empty] myspace.mymodel`
+- `INSPECT GLOBAL`: inspects global state, shows a list of spaces and users
+- `INSPECT SPACE <space>`: inspects a single space, shows a list of models and other things
+- `INSPECT MODEL <model>`: inspects a single model, shows information about stored data and other things
+
+## DML
+
+- `INSERT INTO myspace.mymodel(...)`
+- `SELECT col1, ... FROM myspace.mymodel WHERE ...`
+- `UPDATE myspace.mymodel SET counter += 1 WHERE ...`
+- `DELETE FROM myspace.mymode WHERE ...`
+
+## DCL
+
+Queries include:
+- `SYSCTL REPORT STATUS`: returns the status of the system. (Not a control query per se)
+- `SYSCTL CREATE USER "username" WITH { password: ... }`: create a new user
+- `SYSCTL DROP USER "username"`: removes the user in question
+
+## Differences from SQL
+
+- No literals (see above)
+- Mandatory parameterization (see above)
+- No semicolons!
+- Only one statement per query. For multiple statements batches must be used
+- DML queries are point queries (hence must contain a `WHERE` clause)