REFERENCE.adoc

= Turicum Programming Language Reference

== Introduction

=== What is Turicum?

Turicum is a modern programming language designed for expressiveness, safety, and concurrency.
It combines functional and object-oriented paradigms with a clean and coherent syntax.
Its preprocessing capabilities and dynamic nature make it a uniquely powerful tool for programming.

=== Turicum Is

* **Functional**
+
Functions, closures, and macros are first-class citizens in Turicum.
Any of these can be used as values.
Every command evaluates to a value, so any command can appear within an expression, and every expression is also a command.

* **Object-Oriented**
+
Programs can define classes, methods, multiple inheritance, and objects that inherit properties from their class.

* **Dynamic**
+
All definitions are created at runtime and can be modified at runtime.
For example, a class or function is created when its definition is executed, and it exists within the defining context.
Methods are just fields containing closures or function values, and they can be added to any class or object—even after creation.

* **Flexible**
+
The language's syntax is extensible.
You can write a preprocessor to hook into the processing chain between lexical and syntax analysis.
The preprocessor can be written in Turicum itself, and it works on the lexical token list.

* **Multithreaded**
+
The interpreter is built on Java 21 and supports Virtual Threads using native language constructs.

* **Typed and Scoped**
+
Variables can be optionally typed, and assignments are dynamically type-checked at runtime.

Turicum functions both as a scripting language and as an embeddable engine.
It can run as a standalone command-line tool or be integrated into Java or Kotlin applications as a library.

Turicum is implemented in Java and is designed to be approachable for JVM developers while introducing powerful new abstractions.

The name “Turicum” is the ancient Roman name for the city of Zürich, Switzerland, where the language is being developed.

=== Implementations

The interpreter is currently available in three different applications:

* command line version

* as a Maven extension

* as a Java library.

== Command Line

The command line version is available as an installation kit for Windows, Linux, and macOS.
The kits are available on GitHub, and they include everything you need to run Turicum programs.

To get on-screen help, type

  turi -h

after the installation.

== Maven Extension

The Maven extension is automatically downloaded and started if you have an `extensions.xml` file in your project's `.mvn` directory.

The content of this XML file should follow the format required by Maven documentation.
The Turicum project configuration is

[source]
----
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<extensions>
    <extension>
        <groupId>ch.turic</groupId>
        <artifactId>turicum-maven-extension</artifactId>
        <version>1.4.2</version>
    </extension>
</extensions>

----

== Java library






If you want to embed the application into a Java or Kotlin application, use the following dependency:

[source]
----
<dependency>
   <groupId>ch.turic</groupId>
   <artifactId>turicum-parent</artifactId>
   <version>1.4.2</version>
</dependency>
----

NOTE: The above version is probably a version before release.

Better, use the Maven extension of Turicum and specify the project using `pom.turi`.
For an example, see the `cli/pom.turi` project definition.
It is the command line version of Turicum that depends on the core library.

An embedder controls how much a Turicum program is allowed to run through the `LocalContext` constructor: a step limit bounds total execution, and an optional bounded grace allowance lets `finally`/`with` cleanup code run for a little while after that limit (or an external abort) fires, without being able to run forever — see <<cleanup-after-halt>>.
Richer embedding and sandboxing APIs, including limiting a program's access to the JDK and system resources, are under active development.

=== Design Goals and Philosophy

Turicum was designed with the following principles:

* A **concise syntax** that remains readable and expressive.
* A **functional-first** execution model with support for **side effects** when necessary.
* **Multithreading and pipeline constructs** are built directly into the language.
* **Safe scoping and closures**, including *reclosable* closures, which can adapt to new context bindings.
Closure reclosing is a unique feature of the language.
* **Macro-based metaprogramming**, allowing deferred and explicit evaluation of arguments.
* A flexible **parameter system**, supporting:
- `normal` parameters,
- positional-only,
- named-only,
- rest parameters (`[x]`),
- meta parameters (`+{x}+`),
- and trailing callable parameters (`^x`).

The language is intended for **experienced developers**, especially those working with the JVM, who require a powerful scripting or DSL tool that feels natural but scales to complex architectural needs.

=== Lexical and Syntax Conventions

Turicum syntax is clean, indentation-insensitive, and largely expression-oriented.
It does not aim to reinvent syntax, so it will feel familiar to programmers coming from Java, C, and—occasionally—Python.
At the same time, it avoids syntactically superfluous elements, such as mandatory parentheses `()` around `if` conditions, or loop header parts.

The core syntactic conventions are as follows:

* Semicolons (`;`) can be used to separate commands but are generally optional.
It is recommended to use semicolons to avoid ambiguity.
* Code blocks are enclosed in curly braces `+{}+` and evaluate to the result of the last executed command.
* Single-line comments begin with `//` and continue to the end of the line.
* Multiline comments start with `+/*+` and end with `*/`, like in Java, C, Go etc.
Unlike Java, nested multiline comments are supported.
* Identifiers follow conventional naming rules and are case-sensitive.
* Any string enclosed between backticks (`pass:[`]`) is treated as an identifier (similar to Kotlin).
* All keywords are lowercase.
* Strings enclosed in double quotes (`pass:["]`) must terminate on the same line.
Multiline strings are enclosed in triple double quotes (`"""`).
* String literals preceded with a `$` character are interpolated.

More details about expressions, control flow, and declarations can be found in the chapters that follow.

== Basic Concepts

A Turicum program is a list of commands, executed from start to end.

.source of snippet `first_sample`

[source]
----
print "Hello,";
println " World"
for i=1 ; i < 3 ; i = i+1 : println(i)

----

.output of snippet `first_sample`
[source]
----
Hello, World
1
2

----



A command is usually terminated by a semicolon (`;`).
A semicolon can sometimes be replaced by a newline at the end of a command.
Some commands explicitly require the use of `;`, like the `for` loop.

Commands can be grouped into blocks, enclosed in `+{+` and `+}+`.
Everything in Turicum has a value.
Even commands result in a value.
For example, an `if` statement returns the value of the executed command:

* the branch after the `if` condition,
* the `else` block if the condition is `false` and an `else` is present, or
* `none` if there was no `else` block.

.source of snippet `if_value`

[source]
----
mut z : num = {if 232 % 2 == 0 : 1 else: 2 }
println(z)

----

.output of snippet `if_value`
[source]
----
1

----



Command blocks can appear within expressions.
The value of a command block is the result of the last executed expression.

.source of snippet `block_value`

[source]
----
mut z = {
    mut k = "Hi";
    k + "cory"
    }
println(z)

----

.output of snippet `block_value`
[source]
----
Hicory

----



Expressions can contain:

* Literals:
** strings
** numbers
** JSON values
* Variables
* Operators (with precedences fixed)
* Function calls
* Closure calls
* Macro calls
* Method calls
* Async expressions
* block commands enclosed between `+{+` and `+}+` or `(` and `)`.

Symbolic constants are the following:

* `true`
 has the constant boolean __true__ value.
* `false`
 has the constant boolean __false__ value.

* `none`
 is the undefined value.
 The Java representation of the undefined value is `null`.

* `inf`
 is the infinite numeric value.

* `fini`
 is the special value that, when returned from a cell command, prevents the cell from updating
 but it also signals that this cell is stopped, should not be evaluated any further

* `non_mutat`
 is the special value that, when returned from a cell command, prevents the cell from updating
 its value and halts propagation to dependent cells in a flow.

* `nan`
 is the special IEEE-754 "not a number" floating-point value.



They are pinned (immutable, final) global variables.

==== Shebang

The interpreter ignores the first line of the program if it begins with `#!`.
This applies not only to the command line version but to all embeddings.
This helps use the same source code across any embedding on any operating system.

==== Whitespace and Line Breaks

Whitespace—including spaces, tabs, and line breaks—is generally insignificant, except where indentation aids readability.
Multiple blank lines are allowed between tokens and commands.
Line breaks alone do not terminate a command.

Whitespace terminates tokens, so `1 2` are two tokens, and `12` is one.

==== Identifiers

Identifiers are names used for variables, functions, classes, and other user-defined symbols.

* Identifiers must begin with a Unicode letter or an underscore (`_`), and can contain letters, digits, and underscores.

* Identifiers are case-sensitive.

Some Unicode characters like `∞` (infinity) and `∅` (none) are automatically translated into meaningful identifiers (`inf`, `none`).

===== Non-standard Identifiers

* You can explicitly define an identifier using backticks: `+`non-standard-id`+`.
This allows you to use any string as an identifier—even reserved words, symbols, or operators.
The primary use of this feature is to override operators, such as `+`==`+`.

[NOTE]
====
Many languages allow operator overloading through specially named methods.
For example, in Python, you can override the `+` operator by defining a method named `pass:[__plus__]`.
Turicum does not require such special naming conventions.
You can define a method with the name `pass:[`+`]`.
====

==== Reserved Words

Turicum defines a set of reserved keywords that cannot be used as identifiers—unless the name is enclosed in backticks, like `pass:[`if`]`.

The reserved keywords are:


`as`
`async`
`await`
`break`
`catch`
`class`
`continue`
`die`
`done`
`each`
`else`
`elseif`
`finally`
`flow`
`fn`
`for`
`global`
`if`
`in`
`let`
`list`
`loop`
`mut`
`or`
`otherwise`
`pin`
`print`
`println`
`return`
`sync`
`try`
`until`
`veil`
`when`
`while`
`with`
`yield`

== Expressions and Operators

=== Literals

Turicum supports a rich set of literals, including numeric values, strings, JSON-like constants, and lists.
All literals are expressions that evaluate to immutable values.

==== Numeric Literals

Numeric literals can be either integers or floating-point numbers.

* Integers are written as sequences of digits, e.g., `42`, `0`, `123456`.
* or in hexadecimal with `0x` prefix

.source of snippet `hexNumber`

[source]
----
mut z= 0xFfE3 ;
println z

----

.output of snippet `hexNumber`
[source]
----
65507

----



* Floating-point numbers contain a decimal point or exponent part, e.g., `3.14`, `2e10`, `1.0e-3`.

There is no special suffix for numeric types.
Type promotion is automatic and based on the context of usage.
The internal implementation of numerical types uses `Long` and `Double`.

There is no hexadecimal floating point literal like in Java.
Turicum does not implement a feature just because it is fancy.

==== String Literals

Strings in Turicum are enclosed in double quotes `pass:["]`, e.g., ``"hello world"``.

Backslash escapes are supported:

* `\n` — newline
* `\t` — tab
* `\"` — double quote
* `\\` — backslash

Example:

.source of snippet `string`

[source]
----
print "this is a simple string"

----

.output of snippet `string`
[source]
----
this is a simple string
----



==== Multiline Strings

Multiline string literals are enclosed in triple quotes `pass:["""]`.

They preserve line breaks and indentation exactly as written.
Escape sequences are still processed.

Example:
.source of snippet `mlstring`

[source]
----
println """
Hello,
  This is a multiline string.
Goodbye.
"""

----

.output of snippet `mlstring`
[source]
----

Hello,
  This is a multiline string.
Goodbye.


----



==== Interpolated Strings

A string literal starting with `$"` or `$"""` is interpolated every time it is used.
Interpolation means that the parts of the string enclosed between `${}` and `$()` are executed as command blocks and their result takes their place in the string.
You can also omit the `(` and `)` characters if the expression is a single variable.

.source of snippet `istring1`

[source]
----
let a = 3;
println $"a is $(a)"
println $"a is also $a"

----

.output of snippet `istring1`
[source]
----
a is 3
a is also 3

----



The blocks are executed in the same context as the string.
The `${}` being a wrapping block opens a new local context.
On the other hand, `$()` does not open a new local context.
That way, any declaration inside the `$()` will happen in the surrounding context.

.source of snippet `istring2`

[source]
----
println $"define a, but local ${let a = 3}"
println $"define b, not local $(let b = 4)"
println "is a defined? ", if is_defined(a) : $"yes, it is ${a}" else: "no"
println "is b defined? ", if is_defined(b) : $"yes, it is ${b}" else: "no"

----

.output of snippet `istring2`
[source]
----
define a, but local 3
define b, not local 4
is a defined? no
is b defined? yes, it is 4

----



You can use any expression or Turicum command inside interpolated strings, but the following rules apply:

* Within the interpolated sections:
** From `${` to `+}+`
** From `$(` to `)`

* The number of opening and closing brackets must match:
** The number of `+{+` must equal the number of `+}+`
** The number of `(` must equal the number of `)`

This restriction is usually simple and should not affect your code in most cases.

However, in more complex scenarios—especially when you try to include string literals inside the interpolated parts—it can confuse the parser.
That’s why the rule exists.

If you find this rule limiting, you are likely overcomplicating your usage of interpolation.

Prefer simpler expressions and commands within interpolated strings.

Here is an example of what not to do:

.source of snippet `istring3`

[source]
----
println $"${ $\"${6*6}\"}"

----

.output of snippet `istring3`
[source]
----
36

----



It's best to use simple, clear expressions and straightforward commands that avoid nested structures or embedded string literals.

[NOTE]
====
To implement string interpolation, we selected `${}` because it is widely recognized and familiar from other languages such as JavaScript, Kotlin, and Bash.
It clearly delimits the interpolated expression and is easy to parse when balanced.
By using `${}`, Turicum allows embedding full expressions, including blocks, directly into strings.

Instead of making all strings implicitly interpolated when `${` appears, which would introduce ambiguity and reduce readability, Turicum requires an explicit prefix.
The character `$` was chosen as the prefix for interpolated strings (e.g. `$"..."`) because:

- It is widely associated with interpolation (e.g., C#, PowerShell).
- It is not allowed as part of an identifier in Turicum, so there is no syntactic ambiguity.
- It is visually lightweight and avoids keyword clashes that might arise from using prefixes like `s`, `f`, or `t`.

Turicum extends the interpolation mechanism with two syntactic forms, each with distinct semantics:

- `+${expression}+` evaluates the given expression in a *new temporary context*.
This is useful when a local computation is needed that should not affect the surrounding environment.
- `+$(expression)+` evaluates the expression in the *surrounding context*, allowing modifications (e.g., variable definition using `let` or `mut`) to persist beyond the interpolated expression.

This dual syntax provides clarity and control over variable scope and side effects inside interpolated strings, supporting both functional and imperative styles of expression.
====

==== JSON Constants

Turicum supports JSON-like literals directly in the language syntax.
These can be written using JavaScript-style object and array literals:

.source of snippet `json1`

[source]
----
mut project = {
      name : "turicum-parent",
      packaging : "pom",
      groupId : "ch.turic",
      artifactId : "turicum-parent",
      version : "1.0.0-SNAPSHOT",
      description : "Turicum programming language",
      modules : [ "core", "maven", "cli" ]
      }
println project.name
println project.packaging

----

.output of snippet `json1`
[source]
----
turicum-parent
pom

----



JSON keys can be identifiers or strings.
The values can be any Turicum expression.
The value of a JSON expression is a class-less Turicum object.
The values are evaluated when the literal is evaluated.

A JSON object is recognized when the parser sees a `{` followed by a string or identifier and then a `:`.

==== Lazy JSON

To defer the evaluation of a JSON literal until it is accessed, Turicum provides _lazy JSON_ literals.

These are written with a `+&+` prefix before the JSON object:

.source of snippet `lazy_json1`

[source]
----
fn get_name(){
    println "Getting name";
    "turicum-parent"
}
mut project = &{
      name : get_name(),
      packaging : "pom",
      groupId : "ch.turic",
      artifactId : "turicum-parent",
      version : "1.0.0-SNAPSHOT",
      description : "Turicum programming language",
      modules : [ "core", "maven", "cli" ]
      }
println project.name
println project.packaging
println project.name

----

.output of snippet `lazy_json1`
[source]
----
Getting name
turicum-parent
pom
Getting name
turicum-parent

----



Lazy JSON is not evaluated at definition time.
Each field is computed lazily, on demand, using the current execution context wrapping the context where the JSON was defined.
This behavior is similar to that of closures.

In the example above `get_name()` is only invoked if and when the field `name` is accessed.
However, it is called each time the field is accessed.
Lazy JSON objects are pinned, meaning it is not possible to change any of their fields.

==== List Literals

Lists can be defined using square brackets `+[]+`.

List elements are expressions and are separated by commas:

.source of snippet `ListLiteral1`

[source]
----
mut computed = {fn(a,b) = a + b}(1,2);
println [1, 2, 3, 4]
println ["apple", "banana", "cherry"]
println [1 + 2, computed]

----

.output of snippet `ListLiteral1`
[source]
----
[1, 2, 3, 4]
[apple, banana, cherry]
[3, 3]

----



Lists are mutable by default and can be indexed using the `+[]+` operator.
They support iteration in `+for each+` loops.

Empty lists can be written simply as `+[]+`.

Lists can contain mixed types, but you are encouraged to use homogeneous lists for clarity.

List literals also support flattening other lists using the `..` prefix.
For example:

.source of snippet `ListLiteral2`

[source]
----
mut a = [1, 2, 3, 4]
mut b = ["apple", ..a, "banana", "cherry"]
println b

----

.output of snippet `ListLiteral2`
[source]
----
[apple, 1, 2, 3, 4, banana, cherry]

----



This is also the way to concatenate lists:

.source of snippet `ListLiteral3`

[source]
----
mut a = [1, 2, 3, 4]
mut b = [5, 6, 7]
println([..a, ..b])

----

.output of snippet `ListLiteral3`
[source]
----
[1, 2, 3, 4, 5, 6, 7]

----



The `pass:[+]`, `-` and `*` operators are also defined for lists.

.source of snippet `ListLiteral4`

[source]
----
mut a = [1, 2, 3, 4]
mut b = [1, 3]
println a,"+",b,"=", a+b
println a,"-",b,"=", a-b
println a,"*", b,"=", a*b
println a,"+\"sixteen\"=", a+"sixteen"
println a,"-",2,"=", a-2

----

.output of snippet `ListLiteral4`
[source]
----
[1, 2, 3, 4]+[1, 3]=[1, 2, 3, 4, 1, 3]
[1, 2, 3, 4]-[1, 3]=[2, 4]
[1, 2, 3, 4]*[1, 3]=[[1, 1], [1, 3], [2, 1], [2, 3], [3, 1], [3, 3], [4, 1], [4, 3]]
[1, 2, 3, 4]+"sixteen"=[1, 2, 3, 4, sixteen]
[1, 2, 3, 4]-2=[1, 3, 4]

----



The `pass:[+]` will concatenate the two lists, or append a single element to the list.
If you want to append a list to a list instead of concatenating, you have to enclose it in another list.

.source of snippet `ListLiteral5`

[source]
----
mut a = [1, 2, 3, 4]
mut b = [1, 3]
println a+[b]

----

.output of snippet `ListLiteral5`
[source]
----
[1, 2, 3, 4, [1, 3]]

----



The `-` will remove the elements from the list.

The operator `*` will create the Descartes product of the two lists.

If you want to remove a certain element or part of the list, you can compose a new list from the necessary parts:

.source of snippet `ListLiteral6`

[source]
----
mut a = [1, 2, 3, 4]
println [a[0],..a[2..inf]]

----

.output of snippet `ListLiteral6`
[source]
----
[1, 3, 4]

----



List literals also support filters and mappers.
For example:

.source of snippet `ListLiteral7`

[source]
----
mut a = [1, 2, 3, 4 ? {|x| x % 2 == 0 } ]
println a

----

.output of snippet `ListLiteral7`
[source]
----
[2, 4]

----



will select only the even elements from the list.
The expression or closure following the `?` works as a filter.

You can also apply mappers following a `->`, for example:

.source of snippet `ListLiteral8`

[source]
----
mut a = [1, 2, 3, 4 ? {|x| x % 2 == 0 } -> {|z| return z/2 }]
println a

----

.output of snippet `ListLiteral8`
[source]
----
[1, 2]

----



selects only the even elements and divides each with two.
You can have multiple mappers and filters following each other mixed.

The mappers and the filters are usually closures and have one argument.
As a little syntactic sugar the variable `it` is defined when evaluating them.
It can save up using an argument:

.source of snippet `ListLiteral9`

[source]
----
mut a = [1, 2, 3, 4 ? {|| it % 2 == 0 } -> {|| return it/2 }]
println a

----

.output of snippet `ListLiteral9`
[source]
----
[1, 2]

----



To ease more the syntax, you can use ordinary expressions instead of closures:

.source of snippet `ListLiteral10`

[source]
----
mut a = [1, 2, 3, 4 ? it % 2 == 0  -> it/2 ]
println a

----

.output of snippet `ListLiteral10`
[source]
----
[1, 2]

----



This makes writing filters and mappers very easy.

.Some deep technical detail
[NOTE]
====
The closure definition `+{|x|x*2}+` is technically an expression.
Saying that the mappers and filters can be an expression OR a closure is a bit sloppy, because closures are also results of an expression.
What really happens is that the interpreter evaluates the expression, and if the result is a closure, then it will apply it.
In other cases it just uses the result of the expression.
In the case of filters, the value has to be something that can be interpreted as a boolean value.
In the case of mappers, the value of the expression is used as is.

How does the interpreter know that you want the closure to be evaluated and not use the closure itself as the new value?
Closures are just objects and can be in a list.

The answer is that it does not.
If it sees a closure as the result of the expression, it will evaluate it.

If you want to map a list element to a closure, then you have to write a closure that returns that closure you want to be the list element.
====

The last element of a list literal following the optional filters and mappers is an optional object or class attachment.

When a list is attached to a class or object, the fields and methods of the attached class or object can be invoked on the list.
The keyword `with` can attach the resulting list to an object or to a class.

When a method is invoked, the special variables `this` and `cls` are set the same way as if the call were performed on the original object or class.
In addition to that, the special field `it` will point to the list.

If you create a slice from a list that has an object or class attached, the slice will inherit this attachment.
Other operators, like `+`, `-`, `*` and so on, which are defined for lists will inherit the attachment of the left-hand side operand.

.source of snippet `objectAttachment`

[source]
----
class B {
    fn a {
        if is_defined(this) :
            println $"hello ${this}, ${cls}  ${it}";
        else :
            println $"hello ${none}, ${cls}  ${it}";
    }
}
let b = B()
let z = [ 1,2,3 ]
let k = [ ..z with b];
let h = [ ..z with B];
b.Kukk = 2;
k.a();
println k.Kukk
println "Now with h"
h.a()
println h.Kukk

let w = k[1..3]
w.a()

----

.output of snippet `objectAttachment`
[source]
----
hello {this: {...}, Kukk: 2, cls: class B}, class B  [1, 2, 3]
2
Now with h
hello none, class B  [1, 2, 3]
none
hello {this: {...}, Kukk: 2, cls: class B}, class B  [2, 3]

----



Using this possibility, you can handle lists as objects, while at the same time you can also access the individual elements.

==== Operators and Symbols

The lexer recognizes both symbolic and word-based operators in expressions.
Operators are tokenized with priority for the **longest matching operator**.

Recognized symbols are:


`---`, `pass:[+++]`, `--`, `++`, `pass:[->]`, `pass:[<-]`, `(`, `)`, `,`, `.(`, `.`, `?.`,
`&{`, `{`, `}`, `[`, `]`, `;`, `:`, `|`, `?`, `@`, `^`, `pass:[##]`, `pass:[#]`, `**`, `'`
 

=== Field Access

Objects have fields that the code can set and read.
They are accessed

* using the `.` operator followed by the identifier of the field, or
* using the field identifier, as a string between `[`, and `]`.

NOTE: You can also use a keyword as a field name or method name, either enclosed between `pass:[`]` or without.
Turicum allows you to use a keyword like `as` without the backticks around it `pass:[`as`]` when they stand right after `.` or `?.`.

.source of snippet `field_access`

[source]
----
let a = {}
a.k = 55
a["z"] = a["k"]
die when a.z != 55
println "it is ok"

----

.output of snippet `field_access`
[source]
----
it is ok

----



Creating new fields in an object requires less care than creating new variables.
You do not need to use the `let` or `mut` keyword to create a new field.
As soon as you assign a value to a field, it will be added to the object.

By default, fields are mutable, but you can pin them.
You cannot write `pin a.b`, however.
It would be a syntax error.
`pin` works on identifiers and not on expressions.

On the other hand, you can use the `pin` command inside the class functions, where the fields are a kind of local variables, and you can also use the `with` command.

The following example demonstrates how the code can pin a field and leave another mutable:

.source of snippet `pinned_field`

[source]
----
class Car {
    fn init(power:num=125, color:str="red"){
        pin power
    }
    fn repaint(color:str){
        this.color = color
    }
}
let my_porsch = Car(power=467)
my_porsch.repaint("yellow")
try {
    with my_porsch {
        power = power + 100;
    }
}catch e : println "You cannot do home tuning on a porsch";
print $"The color of my porsch is ${my_porsch.color}"

----

.output of snippet `pinned_field`
[source]
----
You cannot do home tuning on a porsch
The color of my porsch is yellow
----



The following example gives you an example to pin a field of a classless object.

.source of snippet `pinned_field_with`

[source]
----
let a = {}
a.b = 13
with a : pin b
try{ a.b = 14 } catch e: println e
println a.b

----

.output of snippet `pinned_field_with`
[source]
----
pinned variable cannot be altered 'b'
13

----



The automatic creation of the fields is transitive.

.source of snippet `field_transitive_creation`

[source]
----
let a = {}
a.b.c.d.e.f = 1
println a

----

.output of snippet `field_transitive_creation`
[source]
----
{b: {c: {d: {e: {f: 1}}}}}

----



You do not need to create `b`, `c`, `d`, and `e` one by one.
They will automatically become an `+{}+` classless object.

Accessing a field on the right-hand side of an assignment, or in any expression, is more restrictive.
You can still access a nonexistent field, and the value will be `none`, but you cannot access a field on `none`.
To overcome this, you can use the `or` operator as in the example, or you can use the `?.` field access.

.source of snippet `field_none_access`

[source]
----
let a  = {}
println $"is a.b defined? ${with a: is_defined(b)}"
try: let q = a.b.c
catch e: println e
// q is still undefined we can assign
let p = (a.b or {}).c
// the same, but shorter
let q = a.b?.c
println $"q is ${q}.. eventually, but at least exists..."
println $"a was not changed ${a}"

----

.output of snippet `field_none_access`
[source]
----
is a.b defined? false
Cannot access the field 'c' because the object it is used on is undefined.
q is none.. eventually, but at least exists...
a was not changed {}

----



In the example above we cannot directly access `a.b.c`, but `a.b?.c` works.
It is also important that accessing the non-existent `a.b` this way will *not* insert a new classless object into `a`.
It is just a lenient way of accessing fields.

On the left side of the `?.` can be any expression resulting in an object or `none`, even a `["xxx"]` style field access.
On the other hand, there is no `?.` equivalent for accessing a field through `["xxx"]`.
In that case the only remedy is the `or` operator:

.source of snippet `field_none_indexed_access`

[source]
----
let a  = {}
let q = (a["b"] or {}) ["c"]
println $"q is ${q}"

----

.output of snippet `field_none_indexed_access`
[source]
----
q is none

----



[#_defined_operators]
=== Defined Operators

Operators have precedence and are evaluated from left to right.
You can use `(` and `)` to change the evaluation order.
The operators are the following in order of growing precedence:

* `or` results in the left side expression if it is defined, and the right side if the left side value is `none` (undefined).
The operator does a shortcut evaluation.
The right side is only evaluated if the left side is `none`.

.source of snippet `oror`

[source]
----
println none or 3

----

.output of snippet `oror`
[source]
----
3

----



* `||` logical OR operation, shortcut evaluated.
The example below demonstrates the shortcut evaluation, as it works and does not throw an exception:

.source of snippet `logical_or`

[source]
----
if (7&2) == 2 || 5/0 == inf {
    println "dandy"
}else{
    println "something is wrong"
}

----

.output of snippet `logical_or`
[source]
----
dandy

----



* `&&` logical AND operation, shortcut evaluated.
The example below demonstrates the shortcut evaluation, as it works and does not throw an exception:

.source of snippet `and`

[source]
----
if(7 & 2) == 3 && 5 / 0 == inf {
    println "something is wrong"
    } else{
    println "dandy"
}

----

.output of snippet `and`
[source]
----
dandy

----

 Note that you cannot divide by zero, the result is an error and not `inf`.

* `..` range operator. `start..end` creates a range to index a list.
The first index is `start`, the last is `end-1`.
Ranges can be used to index lists as values and also in left values to replace a part of a list.
When used as a unary prefix operator, it must be inside a list literal and in front of a list to flatten the list content into the literal.
+
It is also used to flatten objects or lists to function parameter passing.

* `|` bitwise OR operation.
+
The operator works the conventional way on integers.
+
The operator is also defined for lists.
The result is the union of the two lists with the unique elements.
That way you can also create a "set" from a list `|`-ing it with an empty list, as in the example below:
+
The operator is further defined for objects.
The result is a merged object that includes all fields from both operands.
If a field exists in both objects and is not itself an object, the value from the second object is used.
If the field values in both objects are objects, the result is their recursive merge using the `|` operation.
If the field values in both objects are lists, the result is the concatenation of those lists.

.source of snippet `bor1`

[source]
----
println 1 | 2
println( [1,1,2,3,3,2,1] | [])
println ( { a:1, b: 2, c: { h: 1, z:2} w: ["k","h","dr."] }
            | { a:11, b: 22, c: { k:7} , w : [1,2,3]} )
try:
    3.0 | 3.14
catch e: println e

----

.output of snippet `bor1`
[source]
----
3
[1, 2, 3]
{a: 11, b: 22, c: {h: 1, z: 2, k: 7}, w: [k, h, dr., 1, 2, 3]}
Cannot calculate '3.0' | '3.14'

----



You cannot `|` floating point numbers.

* `^` bitwise XOR operation for integer values and power in the case of double values.
It calculates the symmetric difference of lists (the concatenation of the two lists without the elements that appear in both).

.source of snippet `xor1`

[source]
----
println 12 ^ 2
println ?(12.0 ^ 2) or "does not work for floating points"
println( [1,2,3] ^ [3,4,5] )

----

.output of snippet `xor1`
[source]
----
14
does not work for floating points
[1, 2, 4, 5]

----



* `&` bitwise AND operation.
+
This operator behaves conventionally for integers.
+
It is also defined for lists, where it returns the intersection of the two lists, containing only unique elements.
+
The `+&+` operator is not defined for floating-point numbers.

.source of snippet `band1`

[source]
----
println 7 &  2
println [1,1,2,3,3,2,1] & [2, 3, 7]
try:
    3.0 & 3.14
catch e: println e

----

.output of snippet `band1`
[source]
----
2
[2, 3]
Cannot calculate '3.0' & '3.14'

----



* `!==`, `===`, `==`, `!=`, `in` are __not-the-same__, __the-same__, __equal__, __not-equal__, and __part-of__ operators.
The `in` operator works on a list to check if an element is in that list, or on strings to check the existence of a substring.

.source of snippet `equal1`

[source]
----
mut k = { a:1 , b:2 }
mut q = { a:1 , b:2 }
mut w = "ohmy"
println 1 == 1, " one is one"
println 1 == 2, " one is not two"
println 1 === 1, " one is the same as one"
println 1 === 2, " one is not the same as two"
println k == q, " k equals q, but"
println k === q, " k is not the same as q"
println w == "ohmy", " w is 'ohmy'"
println w === "ohmy", " w is not that 'ohmy'"

----

.output of snippet `equal1`
[source]
----
true one is one
false one is not two
true one is the same as one
false one is not the same as two
true k equals q, but
false k is not the same as q
true w is 'ohmy'
false w is not that 'ohmy'

----



The operator `===` tests that the two objects are the same.
Numerical values are compared by value, but all other objects are tested for being the same, even strings.

The operator `!==` tests that the two objects are not the same.
This operator is the negated version of `===`.

The `in` operation works for strings, checking if a string is part of another, as well as for lists.

.source of snippet `inop`

[source]
----
println "raka" in "avraka dabra", " there is 'raka' in 'avraka dabra'"
println "raka" in "evreke debre", " there is no raka in 'evreke debre'"
println 3 in [ 1, 2, 3], " 3 is in [1,2,3]"
println 3 in [ 1, 2, 4], " 3 is not in [1,2,4]"
// checks member not sublist
println [ 1, 2] in [ 1, 2, 3], " [1, 2] is not in [ 1, 2, 3]"
println [ 1, 2] in [ 1,[ 1, 2], 3], " [1, 2] is in [ 1, [1, 2], 3]"

----

.output of snippet `inop`
[source]
----
true there is 'raka' in 'avraka dabra'
false there is no raka in 'evreke debre'
true 3 is in [1,2,3]
false 3 is not in [1,2,4]
false [1, 2] is not in [ 1, 2, 3]
true [1, 2] is in [ 1, [1, 2], 3]

----



* `<`, `+<=+`, `>`, `>=` usual size comparison operators.

.source of snippet `compare`

[source]
----
println 1 < 2
println 3 <= 3
println 4 > 3
println 4 >= 4

----

.output of snippet `compare`
[source]
----
true
true
true
true

----



* `<<`, `>>`, `>>>` left shift, right shift, and signed right shift operators.

The `>>` works also on strings and lists.
It merges the characters/elements of the two operands:

.source of snippet `shr1`

[source]
----
println 8>>2
println "artany" >> "...... ..."
println([1,2,3] >> [-1,-2,-3])

----

.output of snippet `shr1`
[source]
----
2
a.r.t.a.n.y. ...
[1, -1, 2, -2, 3, -3]

----



* `+`, `-` binary and unary plus and minus.
+
The unary `+` operator does not perform any checks; it operates on all objects and returns the object itself.
+
** The binary `+` operator
+
*** concatenates strings (second argument will be converted to string if not string already),
*** concatenates unevaluated (quoted, as `'$"${a}" +'$"${a}"`) strings
*** concatenates unevaluated (quoted, as `'$"${a}" +"a"`) strings with evaluated string
*** concatenates quoted blocks and quotes list expressions,
*** concatenates lists
*** merges objects, and
*** eventually, works on numerical values.
+
** The binary `-` works
+
*** on strings, removing all occurrences of the second argument from the first,
*** on lists, removing the elements of the second argument from the first, and
*** eventually, it also works on numerical values.

* `pass:[*]`, `/`, `%` binary multiply, divide, and remainder operators.
The `pass:[*]` and `%` are also used in the case of strings and lists.
A string multiplied by a number results in the string concatenated to itself that many times.
The `%` operator is also used for formatting.
The left-hand side is the format string, and the right-hand side is a single element or list of elements.

In the case of lists `pass:[*]` will result in the Descartes product of the two lists.
You cannot `pass:[*]` a list and something else, like a number.

`%` for lists creates the symmetric difference, thus the elements of the two lists which are presented mutually only in one of them.
The following example shows the use of these operators and also the Turicum list literal filtered equivalent for the symmetric difference calculation.

.source of snippet `mod1`

[source]
----
println "2*3=%s" % [2*3]
println "2/3=%s" % [2/3]
println "2%%3=%s" % [2%3]
println "aa " * 6
println( [1,2] * [6])
println( [1,2,3] % [3,4,5] )
println( [1,2,3 ? it in [3,4,5]] )

----

.output of snippet `mod1`
[source]
----
2*3=6
2/3=0.6666666666666666
2%3=2
aa aa aa aa aa aa 
[[1, 6], [2, 6]]
[3]
[3]

----



.functions argument default values
[NOTE]
====
The bitwise OR `|` is not allowed in the expression specifying a default value for a function or closure argument.
You can, however, freely enclose the expression or a part of it between `(` and `)` and use the operator freely.
====

* `pass:[**]` and `pass:[##]` are the power operator and method joining.
The expression `a**b` is `a` to the power of `b`.
Many languages use `^` to denote this operator; however, in Turicum, that character is for bitwise XOR.
+
`pass:[##]` creates a new function from two functions by chaining them.
In the following example `f##g` is a function that calls `f` and then `g`.

.source of snippet `func_concat1`

[source]
----
fn f (a,b,c) {
    return a + b + c
}
fn g(x):num{
    return x+1
}
let z =  (f ## g)(1,1,1)
println z

----

.output of snippet `func_concat1`
[source]
----
4

----



The right-hand side of the `pass:[##]` operator has to be a macro, closure, or function that can be invoked with one single argument.
It will be the result of calling the left-hand side function.
In the example above calling `f##g(1,1,1)` is the same as `g(f(1,1,1))`.

=== Pre-, and Post- Increment, and Decrement

Turicum implements the pre- and post-increment and decrement operators as assignments.
`pass:[a++]` or `pass:[a--]` are technically assignments that return the value of the variable before the modification.
Similarly `pass:[++a]` and `pass:[--a]` are assignments returning the value after the modification.

The expression analyzer recognizes and allows you to use pre-increment and pre-decrement operators inside an expression.
It is easy for the parser -- and similarly for the programmer -- to recognize that a `++` or `--` introduces a pre-increment/pre-decrement assignment.

On the other hand, a `++` or `--` after a complex left value may be hard to read.
For this reason the language permits the post-increment and post-decrement operators inside an expression only if it is applied on a simple variable left value.
If you want to apply a post-increment or a post-decrement operator on a field value or a list element, then you have to enclose it between `(` and `)`.

.source of snippet `incremento`

[source]
----
mut k = 3;
let z = k++; // valid it is only a variable
let w = ++k; // valid, it is only a variable

let x = [1,2,3]
x[0]++; // valid, it is a statement
// let h = x[0]++; it would not compile
let h = (x[0]++); // valid, there is a command between ( and )
// let t = ++x[0]; it would not compile
let t = (++x[0]); // valid again, enclosed

----

.output of snippet `incremento`
[source]
----

----



Although the increment and decrement operators are implemented as commands, they can be overloaded for objects.
If a class defines the method `pass:[`++`]` and/or `pass:[`--`]` it will be invoked when the program uses the pre- or post-increment or decrement operator.

The method should work on the object `this`, must have no argument, and should return the modified object.
It may be the same object or it may create a new object.
The examples show four implementations, two for increment and two for decrement.
One implementation in each pair treats the object immutable and returns a new object.
The other mutates the object and returns the modified object.

The example functionality is incrementing, decrementing a field, but the actual use can be anything.

.source of snippet `incrementable1`

[source]
----
// stochastic
class Incrementable {
    fn init(counter:num=0);

    fn `++`{
         Incrementable(counter+1)
    }

    fn p(name:str) {
        println name,": ",counter;
    }
}

mut h = Incrementable();
h.p("h original");
println "h id=%X" % id_hash(h);
let k = h++;
h.p("h after post increment");
k.p("k");
println "h id=%X" % id_hash(h);
println "k id=%X" % id_hash(k);
let z = ++h;
h.p("h after pre increment");
z.p("z")
println "h id=%X" % id_hash(h);
println "z id=%X" % id_hash(z);

----

.output of snippet `incrementable1`
[source]
----
h original: 0
h id=6DF4AF5
h after post increment: 1
k: 0
h id=26E8FF8C
k id=6DF4AF5
h after pre increment: 2
z: 2
h id=5AA76AD2
z id=5AA76AD2

----



You can track the different object instances using the hex output from the `id_hash` function.

.source of snippet `incrementable2`

[source]
----
// stochastic
class Incrementable {
    fn init(counter:num=0);

    fn `++`{
         counter++
         this
    }

    fn p(name:str) {
        println name,": ",counter;
    }
}

mut h = Incrementable();
h.p("h original");
println "h id=%X" % id_hash(h);
let k = h++;
h.p("h after post increment");
k.p("k");
println "h id=%X" % id_hash(h);
println "k id=%X" % id_hash(k);
let z = ++h;
h.p("h after pre increment");
z.p("z")
println "h id=%X" % id_hash(h);
println "z id=%X" % id_hash(z);

----

.output of snippet `incrementable2`
[source]
----
h original: 0
h id=251C8145
h after post increment: 1
k: 1
h id=251C8145
k id=251C8145
h after pre increment: 2
z: 2
h id=251C8145
z id=251C8145

----



In this example, the identity of the object remains the same; all the identity hex codes are the same.
There is only one object created.
It also means that there is not much difference between pre-, and post-increment.

The next two examples implement the same functionality for the `--` operator.

.source of snippet `decrementable1`

[source]
----
// stochastic
class Decrementable {
    fn init(counter:num=0);

    fn `--`{
         Decrementable(counter-1)
    }

    fn p(name:str) {
        println name,": ",counter;
    }
}

mut h = Decrementable();
h.p("h original");
println "h id=%X" % id_hash(h);
let k = h--;
h.p("h after post decrement");
k.p("k");
println "h id=%X" % id_hash(h);
println "k id=%X" % id_hash(k);
let z = --h;
h.p("h after pre decrement");
z.p("z")
println "h id=%X" % id_hash(h);
println "z id=%X" % id_hash(z);

----

.output of snippet `decrementable1`
[source]
----
h original: 0
h id=30A7653E
h after post decrement: -1
k: 0
h id=7CFF3F1D
k id=30A7653E
h after pre decrement: -2
z: -2
h id=6401188A
z id=6401188A

----


.source of snippet `decrementable2`

[source]
----
// stochastic
class Decrementable {
    fn init(counter:num=0);

    fn `--`{
         counter--
         this
    }

    fn p(name:str) {
        println name,": ",counter;
    }
}

mut h = Decrementable();
h.p("h original");
println "h id=%X" % id_hash(h);
let k = h--;
h.p("h after post decrement");
k.p("k");
println "h id=%X" % id_hash(h);
println "k id=%X" % id_hash(k);
let z = --h;
h.p("h after pre decrement");
z.p("z")
println "h id=%X" % id_hash(h);
println "z id=%X" % id_hash(z);

----

.output of snippet `decrementable2`
[source]
----
h original: 0
h id=510689AF
h after post decrement: -1
k: -1
h id=510689AF
k id=510689AF
h after pre decrement: -2
z: -2
h id=510689AF
z id=510689AF

----



See also the next section about operator overloading.

=== Operator Overloading

Operators work for numerical types, string types, as usual, and they can also work on any object that defines them.
The name of the method implementing the operator should be the operator itself.
There is no special name mapping, like Python's dunder methods (e.g., `pass:[__add__]`).
Instead, you can use it, for example, `pass:[`+`]` as an operator identifier.

The operator methods must have one single positional argument, usually named `other`.
The functionality should implement the operation and return the result of "`this` operation `other`".
When an operation is used as a prefix unary operator, the value of the other will be `none`, and `this` will hold the value on the right of the operator.

Only `+`, `-`, `~`, `!`, `..`, `?`, and `pass:[']` can be used as unary operators.

As mentioned above, `+`, `-`, and `..` are binary as well as unary operators.

* The unary operator `!` is logical negation.
* The unary operator `~` is bitwise negation.
* The unary operator `?` returns the original value without any modification or `none` if the calculation of the argument resulted in an error.
  The original value can also be `none`, that way you cannot distinguish from the value if it is a genuine `none` or the result of an error.
* The unary `pass:[']` quotes the right-hand side, and has the same effect as the `quote` or `thunk` built-in functions.
  When you `'{}` you get an empty block.
  On the other hand, `thunk({})` will be an empty **object**, resulting in `{}` when executed.
  If you execute `('{}).execute` you will execute an empty **block**.
  The result in this case is `none`.

.source of snippet `bnot`

[source]
----
println $"negating all bits in 3 ${~3}"
println $"!true is ${!true}"
println $"+13 is just ${+13}"
println $"-13 is just ${-(13)}"

----

.output of snippet `bnot`
[source]
----
negating all bits in 3 -4
!true is false
+13 is just 13
-13 is just -13

----



NOTE: The character `?` can have a different meaning based on the context.
It can be part of conditional field access, when used as `a?.b` returning `none` when `a` is `none` or has no field `b`.
It can also signal the start of a filter in a list literal.

Operators can be overridden in the classes.
If you have a method named the same name as an operator, it will be invoked to perform the calculation.

.source of snippet `objectAddition1`

[source]
----
class LN {
    fn init( a ){

    }
    fn `+` ( other ){
            LN( this.a * other.a )
    }
}
println (LN(2) + LN(4)).a

----

.output of snippet `objectAddition1`
[source]
----
8

----



In the example above the class is redefining the `+` operator, and it returns an object that has the product of the fields `a`.
Thus `LN(2) + LN(4)` is `LN(8)`.

You can override any of the predefined operators.

When overriding the operator `==` the result has to be something that can be converted to boolean.

.source of snippet `myEqual`

[source]
----
class A {
  fn `==`(other) : bool {
    this.a == other.a
  }
}

mut x = A()
mut y = A()
mut z = { a: 1, b: "x" }

x.a = 1
x.b = "x"

y.a = 1
y.b = "y"

println "x == y is %s" % (x == y)
println "y == z is %s" % (y == z)

----

.output of snippet `myEqual`
[source]
----
x == y is true
y == z is true

----



Class `A` redefines the `==` operator to compare only the field `a`.
If there is a difference in other fields, the instances of the class are still the same.

The default implementation compares all the fields in the two objects, and it also checks that the two objects are instances of the same class.

In the example, all the objects are equal.

If we modify the equality check to consider the classes, we get a different result:

.source of snippet `myEqual2`

[source]
----
class A {
  fn `==`(other) : bool {
    return false when cls != other.cls
    a == other.a
  }
}

mut y = A()
mut z = { a: 1, b: "x" }

y.a = 1
y.b = "y"

println "y == z is %s" % (y == z)

----

.output of snippet `myEqual2`
[source]
----
y == z is false

----



=== Using parentheses



The different operators have their precedences as in most of the programming languages.

The operator's table is defined in the source file `BinaryExpressionAnalyzer.java`.

|===
|Precedence |Operators

|0
|or

|1
|`&#124;&#124;`

|2
|`&&`

|3
|`..`

|4
|`&#124;`

|5
|`^`

|6
|`&`

|7
|`!==`, `===`, `==`, `!=`, `in`

|8
| `<`, `<=`, `>`, `>=`

|9
| `+`, `-`

| 10
| `*`, `/`, `%`, `**`, `##`

| 11
| `.`, `?.`
|===

The higher precedence operators are executed first, and the lower precedence operators are executed on the results.
The order of operator execution can be altered using parentheses.
This is a widespread practice in most programming languages.

The usual practice is that you can have a whole expression between `()` characters.
Turicum is more liberal and offers significantly more freedom to developers.
You can have a whole code block between the `()` characters, similarly to `+{}+`.

The significant difference between the two is that the code between `()` is executed in the same context as the expression, while the code block between `+{}+` opens a new context.

.source of snippet `pscope`

[source]
----
println "2*3=",2*(mut s=3)
println "s is defined and it is ",s
println "2*3=",2*{mut k=3}
try : mut h = 2*k
catch e : println e;

----

.output of snippet `pscope`
[source]
----
2*3=6
s is defined and it is 3
2*3=6
Variable 'k' is undefined.

----



Because every command is a value and every expression is a command, you can use `()` to group commands to be executed in the same context as the surrounding code.

== Variables and Context

Variables can be global or local.
They are defined in scopes.
Scopes are wrapped around each other.
The top-level context is the global context.

Scopes are created at runtime, like everything in Turicum.

=== Block scopes

Code blocks open a new context, and they wrap the surrounding context.

.source of snippet `scope1`

[source]
----
mut z = "cory"
z = { // z is global
    println "just a print"
    mut k = "Hi" // k is local
         + z // z is accessible
    // the value of the block is the last command executed
    // 'mut' assignments result the value assigned
    }
println(z)
try:
    println(k)
catch err: println err

----

.output of snippet `scope1`
[source]
----
just a print
Hicory
Variable 'k' is undefined.

----



The variable `k` is local inside the code block, and it has no value outside.
The variable `z` is part of the wrapped context, and in this case it is global.
The block executes in its own context, and this context wraps the global context.

You can nest scopes.

You can use a variable name defined in a wrapped (outer) context essentially hiding the outer variable, but only if you do not use the outer variable in the inner context.
You cannot refer to both in a context.
When you use a variable defined in a wrapped (outer) context, you cannot define it after that as local using `let` or `mut`.

.source of snippet `local_redefined_error`

[source]
----
mut z = 5
try{
    z = {
        println(z);
        mut z = 3
        }
}catch ex {
    println(ex)
}

----

.output of snippet `local_redefined_error`
[source]
----
5
Variable cannot be local, it is already used as non-local 'z'

----



=== Class, object and function context

Class declarations and functions do not wrap the surrounding context.
In the following example, modifying the global variable `glb` is not possible without declaration.
The class and the function do not wrap the surrounding, global context.

.source of snippet `function_scope`

[source]
----
mut glb = "global variable";
class A {
  fn z(){
    println("You can read glb, which is a " + glb)
    try{
       // you cannot change it without declaring that it is global
       // you could write 'let/mut glb', then it is a local variable, or
       // 'global glb', then it is a global variable
       glb = 5; // this will create an error
    }catch(e1){
        println("exception 1:",e1)
    }
    try{
       let h = glb // after the try it is a new scope
       // you touched it as global, you cannot declare it as local anymore
       mut glb = 5;
    }catch(e2){
        println("exception 2:",e2)
    }
    global glb;
    glb = "modified"
    return "You can access glb here it is " + glb ;
  }
}
mut obj = A()
println(obj.z())

----

.output of snippet `function_scope`
[source]
----
You can read glb, which is a global variable
exception 1:Variable 'glb' was used as global, but is not declared, cannot be changed.
exception 2:Variable cannot be local, it is already used as non-local 'glb'
You can access glb here it is modified

----



The methods -- functions defined inside a class -- wrap the context of the class.
Whenever a method is called its own context is created and the "local variables" `cls` and `this` are created.
Unless the method is a constructor (named `init`), these variables are pinned and cannot be changed.

Code inside a class, outside any method is executed when the class is defined.

.source of snippet `class_definition`

[source]
----
class X {
  mut s = "hu"
}
println("class var ", X.s)
mut x = X();
println("object var ", x.s)

----

.output of snippet `class_definition`
[source]
----
class var hu
object var hu

----



The code above defines a class named `X`.
This class has a class context variable `s`.
It can be accessed as a field of the class itself `X.s`.

Variable `x` is initialized to hold an instance of this class.
Using `x.s` will reference the same variable, because objects wrap the context of the class.

The following example details these different access scenarios:

.source of snippet `object_scope`

[source]
----
class X {
  mut s = "class field"
  cls.h = "another class field"
  fn constructor {
    mut g = "object field"
  }
  fn t(){
    // access object field explicitly through 'this'
    println("this.g=",this.g)
    // the object wraps the class scope and not the other way around
    // there is no 'g' defined in the class
    try: println("cls.g=",cls.g) catch err1: println err1
    // 'g' is in the wrapped scope of the object scope
    // this is same as 'this.g'
    try: println("g=",g) catch err2: println err2
    // 'this.h' is the same as 'cls.h' because of wrapping
    println("this.h=",this.h)
    println("cls.h=",cls.h)
    println("h=",h)
    // the same is true for the other field
    // does not matter how it was defined
    println("this.s=",this.s)
    println("cls.s=",cls.s)
    println("s=",s)
  }
}
mut x = X();
x.t()

----

.output of snippet `object_scope`
[source]
----
this.g=none
cls.g=none
g=Variable 'g' is undefined.
this.h=another class field
cls.h=another class field
h=another class field
this.s=class field
cls.s=class field
s=class field

----



=== Closure scopes

Closures are blocks with arguments.
The general syntax is:

[source]
----
  {|arguments declaration| commands}
----

Note that executing a closure and calling a closure are two different things.
When the closure command definition is executed, it returns the closure.
When a closure is called, the arguments are evaluated in the actual context, and the closure is called in its context.
The context of the closure does not wrap the context of the caller.
It does wrap the context of the definition, where we created the closure.

Technically functions are closures that do not wrap their definition contexts.

In the following example, the closure can access and even modify the variable `whom` inside a block.

.source of snippet `closure_example`

[source]
----
mut c = {
    mut whom = "World!";
    {|greeting|
        println(greeting,whom);
        whom = "Universe!"
    }
}
c("Hello, ")
c("Hi, ")

----

.output of snippet `closure_example`
[source]
----
Hello, World!
Hi, Universe!

----



The variable `whom` lives only inside the block.
The closure wraps this context and keeps the variable alive as long as the closure exists, and it also modifies the content of the variable.

[NOTE]
====
The closure arguments follow the same syntax as the function arguments.
There is however, a small difference.

When a closure argument list defines types and alternatives for the types, how does the parser know whether the `|` is separating different types or closing the closure argument list?
For example, in the next example

  {|a:str| "*" + a + "*" }

`a:str|` could continue with alternative types, like `a:str|any`.
The syntax analyzer looks ahead a few tokens, and in the case of closures it treats the `|` as a type alternatives separator if

* the next token after it is an identifier and after that another `|`, a `=`, or `,` follows, or
* the next token is `(`.

This analysis is heuristic and can cause issues.
To avoid those:

* Do not start the first command of the closure with `(` if the closure's last argument has type.

====

=== Macro scopes

Macro scopes are the same as closure scopes.
Macros are closures that get their arguments unevaluated.
You can create a macro using the built-in global function `macro` with a closure or function as an argument.
Code inside macros can call the global function `evaluate` on the arguments.

This way, macros can decide which arguments to evaluate and in what order, or how many times.
The following simple macro reimplements the `if` command's functionality using it:

.source of snippet `macro_example`

[source]
----
mut c = macro(
    fn(condition,then,`else`){
        if evaluate(condition) : evaluate(then) else: evaluate(`else`)
    });

print c(true,"yes, it is true",{ println("this will never be printed") })

----

.output of snippet `macro_example`
[source]
----
yes, it is true
----



When a macro calls the function `evaluate` the evaluation happens in the caller context.
This ensures the same context and variable visibility as calling a function or closure.
The caller does not need to know whether what it calls is implemented as a function, closure, or macro.

=== Variable Declaration

On first assignment, the code must use the keyword `let` or `mut` to define the variable.
Until that happens, you cannot use a variable.

Variables defined using the keyword `let` are immutable.
Variables defined using the keyword `mut` are mutable.
Mutable variables can be pinned, making them immutable at any time using the `pin` command.
Immutable variables remain immutable while they exist.

You cannot read an undefined variable.
Reading an undefined variable will throw an exception.

NOTE: There is a subtle difference between an undefined variable and a defined variable having the value `none`.
An undefined variable was never defined.
An already defined variable can have the value of `none`, and it can also have a type declaration.
Also, you can assign values to object fields without the `let` keyword, even when the field was not "defined".

The syntax of a `let` and `mut` command is the following:

[source]
----
'let' | 'mut'  identifier [ ':' type ] [ = expression ]
----

or

[source]
----
'let' | 'mut'  mapping = expression
----

In the first version, you assign a value to an identifier and optionally define the type.
If the type is defined, only values that match the type can be assigned to the variable.
Type is

* one of the predefined types, or

* the name of a class,

* or a java object type.

You can specify multiple types separated using `|` character.

If you do not specify an expression, then the value will be `none`.
In this case, the type has to include `any` or `none` as an alternative.
A variable of a type that does not have `none` or `any` in the allowed types cannot be `none`.

The `: type` and the `= expression` parts are optional.
If the type is not defined, then the variable can get any value.
If the expression is missing, the value will be `none`, and the variable is defined.
There can be multiple `identifier: type = expression` after the `let` keyword, comma-separated.
There are predefined type names.

* `bool` boolean type
* `str` string
* `bin` byte array
* `num` any numeric type, integer or float
* `int` any integer type
* `float` float type
* `any` the variable can hold any value
* `obj` the variable can hold any object without restriction on the class of that object
* `lst` the variable has to be a list
* `que` the variable has to be a queue
* `task` the variable has to be an asynchronous task
* `mtx` the variable has to be a mutex
* `atm` the variable has to be an atomic cell
* `err` the variable has to be an exception (error) object
* `cls` the variable has to be a class
* `fn` the variable value has to be a function or closure
* `macro` the variable value has to be a macro
* `none` the variable can hold the value `none`
* `some` the variable can hold any value, except `none`


If the name starts with `java.`, then the variable can hold a Java object.
For example, `java.java.math.BigInteger` can hold any `java.math.BigInteger` object.

In other cases, the type can be the name of a class defined in the program.
The class has to be defined by the time the variable is defined.

If the name of the type is specified between `(` and `)` then the expression between the parentheses is evaluated and the string value is used as a type name.

[NOTE]
====
Note that a variable can hold `none` only if

* there is no type defined,

* `none` is among the allowed types, or

* `any` is among the allowed types.

For example, `let a:num;` will fail, because it allows only numerical values, but missing the initial value will try to set it to `none`.
====

You can define multiple types for a variable, separating the types with `|`.
For example, `let a:num|str|myClass` makes `a` eligible to store a numeric value, a string, or an instance of `MyClass`.

The following example declares a variable that can hold a string or a numeric value.

.source of snippet `let_with_types_example`

[source]
----
mut c : str |num = 3;
c = "" + c + " == " + c
println(c)

----

.output of snippet `let_with_types_example`
[source]
----
3 == 3

----



The format

[source]
----
'let' | 'mut'  mapping = expression
----

uses the expression as a list or an object and assigns part of it to the variables following the structure of the mapping.
The following is a simple example when the mapping copies the values from an object to variables.

.source of snippet `flat_let_object1`

[source]
----
let z = { a: "apple", b:"bier", c : "count" }
let {a, b, c} = z
println a
println b
println c

----

.output of snippet `flat_let_object1`
[source]
----
apple
bier
count

----



It is not an error if there are fields in the object that are not read during the assignment, and also if some of the variables do not have matching fields.

.source of snippet `flat_let_object2`

[source]
----
let z = { a: "apple", b:"bier", c : "count" , d: "not used" }
let {a, b, c, f} = z
println a
println b
println c
println f

----

.output of snippet `flat_let_object2`
[source]
----
apple
bier
count
none

----



You can also map a list to variables, like in the following example:

.source of snippet `flat_let_list1`

[source]
----
mut [a, b, c] = [1,2,3]
println a
println b
println c

----

.output of snippet `flat_let_list1`
[source]
----
1
2
3

----



It is an error if there are not enough or too many elements in the list on the right-hand side.

.source of snippet `flat_let_list2`

[source]
----
try:
    let [a, b, c, f] = [1,2,3]
catch err: println err

----

.output of snippet `flat_let_list2`
[source]
----
let assignment right-hand side has too few values

----



.source of snippet `flat_let_list3`

[source]
----
try:
    mut [a, b] = [1,2,3]
catch err: println err

----

.output of snippet `flat_let_list3`
[source]
----
mut assignment right-hand side has too many values

----



If you have too many elements, you can cut the list:

.source of snippet `flat_let_list4`

[source]
----
mut [a, b] = [1,2,3,4,5,6,7,8][0..2]
println a
println b

----

.output of snippet `flat_let_list4`
[source]
----
1
2

----



If you have too few you can extend the list:

.source of snippet `flat_let_list5`

[source]
----
fn adjust(_x, @to){
    mut x = _x;
    if len(x) < to :
        while len(x) < to :
            x = [..x, none]
    else :
        x = x[0..len(x)]
    x
}
mut [a, b, c, d] = adjust(to=4,[1,2])
println a
println b
println c
println d

----

.output of snippet `flat_let_list5`
[source]
----
1
2
none
none

----



or even

.source of snippet `flat_let_list6`

[source]
----
fn adjust(@to,[_x]){
    mut x = _x;
    if len(x) < to :
        while len(x) < to :
            x = [..x, none]
    else :
        x = x[0..len(x)]
    x
}
mut [a, b, c, d] = adjust(to=4, 1,2)
println a
println b
println c
println d

----

.output of snippet `flat_let_list6`
[source]
----
1
2
none
none

----



Mapping can be more complex than just listing variables for an object field or elements of a list.
Any mapping element can also be a mapping.
The following example maps the fields of an object to variables.
The field `k` is an array, and it is mapped to variables in a submapping.
Submapping can go any level deep, in this case, the submapping list has a fourth element, which is an object also submapped to the variables `f` and `g`.

.source of snippet `let_map1`

[source]
----
let { a , b , k -> [c,d,e , {f , k -> g }]} = {a:1,b:2, k: [3,4,5,{f:6, k: 7}]};
println a
println b
println c
println d
println e
println f
println g

----

.output of snippet `let_map1`
[source]
----
1
2
3
4
5
6
7

----



The name of the field is the same as the name of the variable it is mapped to, unless you specify the field name using the `->` symbol.
In the example above, the variable `g` gets the value of the field `k`.
It is specified as `k -> g`, read: the field `k` goes into the variable `g`.

You cannot use the default field name mapping when you map to a submapping or when you map to a left value other than a naked variable.
In these cases, there is no reliable field name to depend on without explicit declaration.

In the mapping, you can use a left value to get the value if the declaration is `mut`.

.source of snippet `let_map2`

[source]
----
let b = []
mut { a:num , b -> b[0] } = {a:1,b:2};
println a
println b

----

.output of snippet `let_map2`
[source]
----
1
[2]

----



In the example above, the field `b` is mapped to the first element of the array `b`.
You cannot use left values if the mapping is `let`.
You will get a compilation error:

[source]
----
ch.turic.BadSyntax: you cannot map to left value in let
at let_map3.turi:2:20
  1. let b = []
  2. let { a:num , b -> b[0] } = {a:1,b:2};
   --------------------^
----

`let` is always declarative, declaring new variables.
It is an error to reference a variable in the `let` mapping or submapping, which already exists, even if it is mutable.

.source of snippet `let_map3`

[source]
----
mut b = 5
try:
  let { a , b  } = {a:1,b:1};
catch e: println e

----

.output of snippet `let_map3`
[source]
----
Variable 'b' is already defined.

----



You can also use the `let` keyword in front of any submapping, making that part immutable and declarative.

.source of snippet `let_map4`

[source]
----
let b = []
mut { a:num , let b -> [ x, y, z ] } = {a:1,b:[1,2,3]};
a = 5 // a is mutable
try: x = 77 catch e: none // x is immutable will not change
println a
println x
println y
println z

----

.output of snippet `let_map4`
[source]
----
5
1
2
3

----



You can use the `let` keyword in front of individual mappings to make them immutable and declarative.

.source of snippet `let_map5`

[source]
----
mut { a:num , let b } = {a:1,b:1};
a = 5 // a is mutable
try: b = 5 catch e: none // b is immutable will not change
println a
println b

----

.output of snippet `let_map5`
[source]
----
5
1

----



Although `mut` is not strictly declarative, you can declare new variables in a `mut` mapping as in the example above.
However, if the variable you declare with a type already exists, it is an error:

.source of snippet `let_map6`

[source]
----
mut a:num = 3
try:
  mut { a:num , let b } = {a:1,b:1};
catch e: println e

----

.output of snippet `let_map6`
[source]
----
Variable 'a' is already defined.

----



You cannot make a submapping mutable inside a declarative `let` mapping or submapping.
In other words, you cannot use `mut` on a submapping.

Although there is no limit on the nesting depth of mapping and destructuring, it is usually not good practice to create excessively deep structures.


==== Updating a Variable

Mutable variables can receive a new value at any time.
The assignment that updates a variable does not have the `let` or `mut` keyword and does not define a type.
The syntax is

[source]
----
LEFT_VALUE ASSIGNMENT_OPERATOR EXPRESSION
----

which simply looks like `a = 5*3` in most cases.
The left value can be

* a variable identifier, (`my_variable`)
* object field access, (`object.field`)
* list element access, (`list[index]`).

The `ASSIGNMENT_OPERATOR` can be

`=`,`+=`,`-=`,`*=`,`/=`,`%=`,`&=`,`|=`,`^=`,`pass:[**]=`,`or=`,`&&=`,`||=`,`pass:[<<=]`,`>>=`,`>>>=`,

The `=` is the usual assignment.
It calculates the expression on the right side and replaces the current value of the variable with the new value.
The other assignments do three steps:

. Calculate the current value of the left value.
. Execute the operator that stands before the `=` sign applying it on the left value and the expression.
. Update the left value with the new value.

[NOTE]
====
The operations are responsible for evaluating the operands, and they may decide not to.
For example, the `or` operation evaluates the right-hand side only if the left-hand side is `none`.
They work on commands and not already calculated values.

To accommodate this, the evaluation of the left value creates a temporary command.
This temporary command returns the current value of the left value when evaluated.
====

.source of snippet `assign_calculate`

[source]
----
mut z= 3
println "z=", z
z += 1 ;
println "z +=1 :", z
z -= 1 ;
println "z -=1 :", z
z *= 4 ;
println "z *=4 :", z
z /= 2 ;
println "z /=2 :", z
z %= 5 ;
println "z %=5 :", z
z |= 128 ;
println "z |=128 :", z
z &= 7 ;
println "z &=0x7F :", z
z ^= 0xFF ;
println "z ^=0xFF :", z
z **= 2 ;
println "z **=2 :", z
z or= inf;
println "z or=inf :", z
z <<= 2
println "z <<=2 :", z
z >>= 2
println "z >>=2 :", z
z >>>= 2
println "z >>>=2 :", z
z >>>= 2
println "z >>>=2 :", z
z >>>= 2
println "z >>>=2 :", z
z >>>= 2
println "z >>>=2 :", z
z >>>= 2
println "z >>>=2 :", z
z >>>= 2
println "z >>>=2 :", z
z >>>= 2
println "z >>>=2 :", z
z >>>= 2
println "z >>>=2 :", z

mut b=true
println "b=", b
b &&= false;
println "b &&=false :", b
b ||= true;
println "b ||=true :", b

----

.output of snippet `assign_calculate`
[source]
----
z=3
z +=1 :4
z -=1 :3
z *=4 :12
z /=2 :6
z %=5 :1
z |=128 :129
z &=0x7F :1
z ^=0xFF :254
z **=2 :64516
z or=inf :64516
z <<=2 :258064
z >>=2 :64516
z >>>=2 :16129
z >>>=2 :4032
z >>>=2 :1008
z >>>=2 :252
z >>>=2 :63
z >>>=2 :15
z >>>=2 :3
z >>>=2 :0
b=true
b &&=false :false
b ||=true :true

----



==== Restrictions while updating

While an assignment updates a variable, the variable is pinned.
The expression can contain method calls, command blocks that may try to update the variable.
They will fail.

.source of snippet `update_forbiden`

[source]
----
mut z = 13
try {
    z = {z = 12; 14}
}catch e : println e
print z," not changed"

----

.output of snippet `update_forbiden`
[source]
----
Variable 'z' is pinned.
13 not changed
----



The same is true for object fields.

.source of snippet `update_forbiden3`

[source]
----
mut z = { z: 13 };
try {
    z.z = {z.z = 12; 14}
}catch e : println e
print z.z, " not changed"

----

.output of snippet `update_forbiden3`
[source]
----
pinned variable cannot be altered 'z'
13 not changed
----



In the case of array elements, pinning is not possible for the individual elements.
In that case, the assignment recognizes if the left value was changed during the evaluation of the expression.

.source of snippet `update_forbiden2`

[source]
----
mut z = [13]
try {
    z[0] = {z[0] = 12; 14}
}catch e : println e
print z[0], " already changed"

----

.output of snippet `update_forbiden2`
[source]
----
Assigned value changed while calculating new value z[0]
12 already changed
----



The error message is different, and the error happens later, after the value was changed.
This is also the case when a field of an object is accessed via indexing:

.source of snippet `update_forbiden4`

[source]
----
mut z = { z: 13 };
try {
    z["z"] = {z.z = 12; 14}
}catch e : println e
print z.z, " already changed"

----

.output of snippet `update_forbiden4`
[source]
----
Assigned value changed while calculating new value z[z]
12 already changed
----



You are also not allowed to change a string variable while you assign a new string to a part of it:

.source of snippet `update_forbiden5`

[source]
----
mut z = "apple juice";
try: z[1..5] = {z = "worm"; "lma"}
catch e: println e
print z

----

.output of snippet `update_forbiden5`
[source]
----
Variable 'z' is pinned.
apple juice
----



Doing so is essentially the same as assigning a new value to the variable altogether:

.source of snippet `update_forbiden6`

[source]
----
mut z = "apple juice";
try:
    z = z[0..1] + {z = "worm"; "lma"} + z [5..inf];
catch e: println e
print z

----

.output of snippet `update_forbiden6`
[source]
----
Variable 'z' is pinned.
apple juice
----



The reason for that is that strings in the underlying Java ecosystem are immutable.
Turicum is tightly integrated with Java and does not use wrapper classes around `String`, `Long`, or `Double` values.
When you change a part of a string, Turicum creates a new string and then updates the underlying variable to hold this new string.

You are also not allowed to change a variable holding a list while an element of that list is changed.

.source of snippet `update_forbiden8`

[source]
----
mut z = []
try: z[3] = { z = [];  "K"}
catch e: println e
println z

----

.output of snippet `update_forbiden8`
[source]
----
Variable 'z' is pinned.
[]

----



This is not true, however, when the list is stored in a list element:

.source of snippet `update_forbiden7`

[source]
----
mut z = []
z[1] = []
z[1][3] = { z[1] = [];  "K"}
println z

----

.output of snippet `update_forbiden7`
[source]
----
[none, []]

----



A named variable can be pinned, but a list element cannot.

==== Updating with `mut`

When you use the

  mut { variable_list } = expression

or

  mut [ variable_list ] = expression

command you can use an already defined variable in the list.
In this case, the previously defined variable will be updated.
A typical example for the use is a loop:

.source of snippet `string_partition`

[source]
----
mut s:str = "abraka; muhaha dabra; mutabor; kalifa";
while s.is_not_empty() {
  mut [start:str, sep:str, s ] = s.partition("; ");
  println start
}

----

.output of snippet `string_partition`
[source]
----
abraka
muhaha dabra
mutabor
kalifa

----



Without this possibility you would need an extra variable in the list:

.source of snippet `string_partition_alt`

[source]
----
mut s:str = "abraka; muhaha dabra; mutabor; kalifa";
while s.is_not_empty() {
  mut [start:str, sep:str, s_new ] = s.partition("; ");
  s = s_new;
  println start;
}

----

.output of snippet `string_partition_alt`
[source]
----
abraka
muhaha dabra
mutabor
kalifa

----



There are some restrictions on this kind of variable update:

* the variable must be mutable,
* only `mut []` and `mut {}` are allowed,
* `mut variable = expression` results in error if the variable is already defined,
* the variable in the list must not have a type declaration at the location of the update.

==== Pinned Variables

Pinning a variable makes it illegal to change it in the context.
The syntax of the `pin` command is

[source]
----
  pin list
----

The list contains comma-separated elements, which can be

* a variable name, like `turic`,

* a variable name enclosed between `+{+` and `+}+`, like `+{object}+`, or

* a variable name enclosed between `[` and `]`, like `[list]`.

A variable in the list will make the variable in the current context pinned.
If the variable name is decorated with `+{}+` then the object, which is the value of the variable gets pinned.
Note that this is changing the mutability of the object itself and not the variable.

.source of snippet `pin_object`

[source]
----
let a = {a:1, b:2};
pin {a};

try: a.k = 13 catch e: println e


----

.output of snippet `pin_object`
[source]
----
You cannot change a pinned object

----



If the variable name is decorated with `[]` then the list, which is the value of the variable gets pinned.
Note that this is changing the mutability of the list itself and not the variable.

.source of snippet `pin_list`

[source]
----
let a = [1,2,3,4];
pin [a];

try: a[0] = 13 catch e: println e

----

.output of snippet `pin_list`
[source]
----
Cannot change a pinned list.

----



It is an error to use the decoration on a variable that is not an object or list.

.source of snippet `pin_wrong`

[source]
----
let a = [1,2,3,4];
try: pin {a}; catch e1: println e1
let b = {a:1, b:2, c:3}
try: pin [b]; catch e2: println e2

----

.output of snippet `pin_wrong`
[source]
----
value of 'a' is not an object to be pinned
value of 'b' is not a list to be pinned

----



The following example demonstrates the different pinning possibilities:

.source of snippet `pinning`

[source]
----
mut turicum = 13
mut `list` = [1,2,3]
mut object = { x:1, y:2};
pin turicum, [`list`], {object}

try {
  turicum = 14;
}catch e1: println("could not change the variable")

try {
  `list`[1] = 0;
}catch e2: println("could not change the `list`")
`list` = [ 0, ..`list`, 4]
println("variable 'list' still can be changed: ", `list`)

try {
  object.x = 3;
}catch e3: println("could not change the object")
object = { x:1, y:3 }
println("variable 'object' still can be changed: ", object)


----

.output of snippet `pinning`
[source]
----
could not change the variable
could not change the `list`
variable 'list' still can be changed: [0, 1, 2, 3, 4]
could not change the object
variable 'object' still can be changed: {x: 1, y: 3}

----



== `print`, `println` commands

`print` and `println` are implemented as commands.
They both accept a list of expressions separated by commas.
`print` prints out the values from left to right.
`println` does the same, but also appends a new line character at the end.

By default, the output goes to the standard output using the Java `System.out` static field.

The output of `print` and `println` can also be redirected.
If the variable `+print_target+` is defined and is an object or closure the output of the statements will be redirected.

When the `+print_target+` is an object the method `+write+` is used to send the strings and
`+flush+` is called to close every print statement.
`+write+` has to be defined, `+flush+` is optional, but when defined it has to be a method.

When the `+print_target+` is a closure, it is used to send the strings, and flushing is ignored.

The following example shows how to use this feature:

.source of snippet `print_redirect`

[source]
----
class Printer {
    fn init(m:str="");
    fn write(s:str){
        if !len(m) == 0 && !m.ends_with("\n"):
            m = m + " "
        m = m + s;
    }
    fn flush {
        println "flushing is essential"
    }
    fn entry {
        this
    }
    fn exit(e){
    }

    fn get {
        m
    }
}


let printer = Printer();

with printer as print_target {
    println "Once upon a midnight dreary, ","Edgar"
    println "God in his wisdom made the fly, ","Ogden"
    }
println "Are you mental mixing poems?"
println printer.get()

{
mut print_target = {|message:str|
                        let print_target = none;
                        if message == "\n" :
                            println ;
                        print "Said the raven:", message;
                        }
println "never more, to the fly";
}

----

.output of snippet `print_redirect`
[source]
----
flushing is essential
flushing is essential
Are you mental mixing poems?
Once upon a midnight dreary, Edgar
God in his wisdom made the fly, Ogden

Said the raven:never more, to the fly
Said the raven:

----



The value of the command is the string sent to the output.

== Control Flow

=== Conditional Execution (`if` / `elseif` / `else`)

Command `if` is very conventional and similar to any other programming language.
The syntax of the command is:

[source]
----
'if' EXPRESSION ( ':' COMMAND | BLOCK ) [ 'else' ( [':'] COMMAND | BLOCK )] ;
----

If the expression evaluates to `true`, the first command or block is executed.
If the expression evaluates to `false`, the second command or block following the `else` keyword is executed.
The `else` keyword and its command or block are optional.

The block is a series of commands enclosed between `+{+` and `+}+`.
If there is only a single command after the condition or the keyword `else` it can be preceded with a `:`, or enclosed in a block.
The `:` following the `else` keyword is optional.
Its use can emphasize the structure and may help Python lovers.

The conditional expression can be enclosed in parentheses but does not need to be.

The result of the command is the value of the command or block executed.
If the condition is `false` and there is no "else" block or command, then the result is `none`.

.source of snippet `ifCommand`

[source]
----
mut a= 13;
if a%2 == 0: println("even") else: println("odd")
a = {if a%2 == 0: "even" else: "odd"}
print("again ", a)

----

.output of snippet `ifCommand`
[source]
----
odd
again odd
----



The `if` command does not open a new context.
The condition, the commands following it, and the optional one after the `else` keyword are evaluated and executed in the context of the `if` statement.

If, however, the "then" or "else" commands are block commands, they execute in a new context that wraps the actual context.

=== Conditional expression

`if` statements can be used inside expressions.
When you use `{}` or `()` parentheses, you can use any statement in an expression, but `if` statements do not need parentheses.
An `if` statement can be placed on "top-level" in an expression and can occupy a place that is for a whole expression inside another one.

.source of snippet `if_expression_1`

[source]
----
print 5 + if 5 > 4 : 1 else 2

----

.output of snippet `if_expression_1`
[source]
----
6
----



The `if` statement in this example serves as the second argument for the `+` operator.

The keyword can also be used following the syntax

    expression_2 if condition else expression_2

This structure resembles how Python uses conditional expressions.
The following example gives this variant for the previous calculation:

.source of snippet `if_expression_2`

[source]
----
print 5 + 1 if 5 > 4 else 2

----

.output of snippet `if_expression_2`
[source]
----
6
----



Note, however, that you may need parentheses when using this form:

.source of snippet `if_expression_3`

[source]
----
println 5 + 1 if 5 < 4 else 2
println (5 + 1) if 5 < 4 else 2
println 5 + (1 if 5 < 4 else 2)

----

.output of snippet `if_expression_3`
[source]
----
7
2
7

----



The `if` in the expression stands on the highest level in precedence.
Since the `if` "operator" and its precedence is not so obvious as the usual numerical operators, it is recommended to use parentheses for readability.

=== Loops

There are three types of loops in Turicum:

* `loop` / `while`

* `for`

* `for each` loop.

Loops, like any command, have a result value.
The value of a loop is either the last executed command or a list of the last executed command for each iteration.
If the loop is skipped, the value is either `none` or an empty list.

.source of snippet `emptyLoopResult`

[source]
----
println { for i=0 ; i < 0 ; i=i+1 : "anything, not executed, will print none" }
println { for i=0 ; i < 0 ; i=i+1 list: "anything, not executed, will print []" }

----

.output of snippet `emptyLoopResult`
[source]
----
none
[]

----



==== `loop` / `while`

The `while` looping construct is a complex structure.
It can contain up to five command blocks and two conditional expressions executed at different points.
The full syntax of the command is the following:


[source]
----
  WHILE ::= [ 'loop'  ( BLOCK | ':' COMMAND ) ]
  'while' [EXPRESSION] ['list']( BLOCK | ':' COMMAND )
  [ 'until' EXPRESSION ]
  [ 'done' ( BLOCK | ':' COMMAND ) ]
  [ 'otherwise' ( BLOCK | ':' COMMAND ) ]
  [ 'finally' ( BLOCK | ':' COMMAND ) ]
  <p>

----


image::pictures/while_loop.png[]

It

* executes the block following the `loop`, then

* iterates
  ** 'while' the condition after the `while` is true, or
  ** 'until' the condition after the `until` becomes true, or
  ** until a `break` or `return` command is executed in the loop.

If the `while` condition was false already at the start and the iteration DID NOT start, the block after the `otherwise` is executed.

If the iteration started and finished without executing a `break` or a `return` command, the block after the `done` is executed.

Finally, the block following the `finally` keyword is executed.
This block is executed even if a `break` or `return` command was executed in the loop.
The special, pinned variable `it` will hold the loop's value in these blocks.
This variable is not defined when the block `otherwise` is executed.

The loop is executed in two contexts.
The outer context wraps the surrounding context.
The inner context wraps the outer.
The loop body is executed in the inner context.
A new inner context is created for each execution of the loop.
The `while` condition and the other blocks are executed in the outer context.
The `until` condition is executed in the inner context.

.source of snippet `whileLoop`

[source]
----
loop: mut i = 1;
while i < 5 {
  print i, " ";
  i++;
} until i % 3 == 0
done { println "loop is done" }
otherwise { println "Loop did not even start" }
finally { println "We are done here" }
try: print i catch err: println err

----

.output of snippet `whileLoop`
[source]
----
1 2 loop is done
We are done here
Variable 'i' is undefined.

----



The value of the loop is the result of the last executed command in the loop body.

.source of snippet `while_result`

[source]
----
println {
    mut k = 0
    while k < 10 : k++
}

----

.output of snippet `while_result`
[source]
----
9

----



The value from the `done`, and `finally`  blocks do not affect the loop's value.

.source of snippet `while_result_with_done`

[source]
----
println {
    mut k = 0
    while k < 10 : k++ done: 666;
}

----

.output of snippet `while_result_with_done`
[source]
----
666

----



.source of snippet `while_result_with_finally`

[source]
----
println {
    mut k = 0
    while k < 10 : k++ finally: 666;
}

----

.output of snippet `while_result_with_finally`
[source]
----
9

----



The value of the loop is the `otherwise` block's value if that was executed.

.source of snippet `while_result_with_otherwise`

[source]
----
println {
    mut k = 0
    while false : k++ done: k otherwise: 42;
}

----

.output of snippet `while_result_with_otherwise`
[source]
----
42

----



The loop can also be used to produce a list.
To get the list as a result, you have to write the `list` keyword in front of the loop body.
The list will contain the results of each body loop execution.

.source of snippet `while_list`

[source]
----
println { loop: mut k = 0 while k < 5 list: k++ }

----

.output of snippet `while_list`
[source]
----
[0, 1, 2, 3, 4]

----



For list resulting loops, when the `done` and/or `finally` blocks are executed, the special variable `it` will contain the resulting list.
The variable as well as the list itself are pinned, so you cannot add or remove elements from the list or change the elements.

.source of snippet `while_with_done_list1`

[source]
----
let k = {
    loop: mut i = 1
    while i <= 10 list{
      i++;
      } done {
          // cannot do that the variable is pinned
          mut e;
          try: it = [3,2,1] catch e1: println e1
          // cannot do that the list itself is pinned
          try: it[0] = 100 catch e2: println e2
      }
    }
print k;

----

.output of snippet `while_with_done_list1`
[source]
----
Variable 'it' is pinned.
Cannot change a pinned list.
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
----



.a note on the execution context of loops
[NOTE]
====
When a loop executes, two distinct contexts are used: an *outer context* and an *inner context*.

* The *outer context* is created before the loop begins and is discarded after the loop ends.
* The *inner context* is created freshly for each execution of the loop body and is discarded immediately after that execution completes.

The outer context *wraps* the surrounding scopes, and the inner context *wraps* the outer context.
This means that both contexts have access to all variables defined in the surrounding code.
Additionally, the inner context can read and modify variables from the outer context and the scopes it wraps.
However, *new variables defined in the inner context do not propagate into the outer context*.


* The `loop` block, as well as
* the *head condition* following the `+while+` keyword are evaluated in the outer context.

 * The *loop body* and the *exit condition* following the `+until+` keyword are evaluated in the inner context.

This design ensures that each iteration of the loop starts in a clean environment.
Variables defined inside a previous iteration are *not available* in the next iteration, unless explicitly defined in the outer context.
This approach avoids unintended side effects and provides a clear and consistent model for variable visibility and lifecycle within loops.
====

The `loop` part clearly separates the loop from the surrounding code.
Variables declared here are used only in the loop and are shared across iterations.

.source of snippet `while_with_loop`

[source]
----
loop: mut i = 1
while i <= 10 list : i++ finally: println $"The last i after the loop is $i"
try: println "i does not exist outside of the loop" catch e: println e

----

.output of snippet `while_with_loop`
[source]
----
The last i after the loop is 11
i does not exist outside of the loop

----



The following example demonstrates that `otherwise` is not executed when the loop is executed at least once.

.source of snippet `while_until_not_otherwise`

[source]
----
mut i=1
while i <= 10 {
  println $"i is $i"
  i++
  } until i > 10
 otherwise: die "i is not less than 10"

----

.output of snippet `while_until_not_otherwise`
[source]
----
i is 1
i is 2
i is 3
i is 4
i is 5
i is 6
i is 7
i is 8
i is 9
i is 10

----



The next example shows that the `otherwise` is executed when the loop head condition is false from the very start.

.source of snippet `while_otherwise`

[source]
----
mut i = 20
while i <= 10 {
  die "i is 20, must not execute the loop."
  } otherwise: println "we are other, we are wise"

----

.output of snippet `while_otherwise`
[source]
----
we are other, we are wise

----



The behavior is the same even with an `until` exit condition, which is never evaluated in this case.

.source of snippet `while_until_otherwise`

[source]
----
mut i = 20
while i <= 10 {
  die "i is 20, must not execute the loop."
  } until i > 10
  otherwise: println "we are other, we are wise"

----

.output of snippet `while_until_otherwise`
[source]
----
we are other, we are wise

----



The next example demonstrates that the `done` block is not executed if the loop is broken.

.source of snippet `while_until_with_done_broken`

[source]
----
mut i = 1
while i <= 10 {
  println $"i is $i"
  break {
    println "we broke the loop"
    55
    } when i > 5
  i++;
  } until i > 10
  done {
      die "we broke the loop, we are not done"
  }

----

.output of snippet `while_until_with_done_broken`
[source]
----
i is 1
i is 2
i is 3
i is 4
i is 5
i is 6
we broke the loop

----



.a note on Python loop `else:` constructs in Turicum
[NOTE]
====
Python has an `else:` construct that can follow a loop.
In Python, it is executed when the loop completes without being broken.
Turicum does not need to provide this extra block for the loops.

In Turicum, if you break a loop and want to execute some extra code, you can do it in the expression of the `break`.
The break command can provide an expression, and every command, or command block, is an expression in Turicum.
As you can see in the example above, this feature can be used for the same purpose as the `else:` branch for loops in Python.
====

The following two examples show the case when the `done` block is executed after the normal termination of the loop.

.source of snippet `while_with_done`

[source]
----
mut i = 1
while i <= 10 {
  println $"i is $i"
  break {
    println "we broke the loop"
    55
    } when i > 10
  i++;
  } done {
      println "we are done nice and neat, not broken"
  }

----

.output of snippet `while_with_done`
[source]
----
i is 1
i is 2
i is 3
i is 4
i is 5
i is 6
i is 7
i is 8
i is 9
i is 10
we are done nice and neat, not broken

----



It is also a normal termination of the loop when the loop is terminated by the `until` condition:

.source of snippet `while_until_with_done`

[source]
----
mut i = 1
while i <= 10 {
  println $"i is $i"
  break {
    println "we broke the loop"
    55
    } when i > 10
  i++;
  } until i > 5
  done {
      println "we are done nice and neat, not broken"
  }

----

.output of snippet `while_until_with_done`
[source]
----
i is 1
i is 2
i is 3
i is 4
i is 5
we are done nice and neat, not broken

----




==== `for`

The `for` loop is very much similar to the conventional `for` loop we have since it was invented in the language C.
There are some syntactical and some execution differences though.

The syntactic difference is that the start, condition, and step parts do not need to be enclosed between `(` and `)`.
The start, condition, and end parts are separated by `;` and they are commands.
The `;` in this place is not optional.

.source of snippet `for1`

[source]
----
for i = 1 ; i < 5 ; i=i+1 {
  print(i," ")
}until i%3 == 0 ;

----

.output of snippet `for1`
[source]
----
1 2 3 
----



The semantic difference is that you cannot alter the loop variable, or variables if there are multiple inside the loop.

.source of snippet `for5`

[source]
----
mut i;
for i = 1 ; i < 3 ; i=i+1 {
    try: i++
    catch e: println "not possible to change loop variable in the loop it remained ",i
}


----

.output of snippet `for5`
[source]
----
not possible to change loop variable in the loop it remained 1
not possible to change loop variable in the loop it remained 2

----



The `for` loop also has an optional `until` part the same way as `while`.
The next example shows the same loop with the optional parentheses but without the tail condition.

.source of snippet `for2`

[source]
----
for( i = 1 ; i < 5 ; i=i+1 ){
  print(i," ")
}

----

.output of snippet `for2`
[source]
----
1 2 3 4 
----



If the start command is an assignment, it is interpreted as a local assignment.

.source of snippet `for3`

[source]
----
// same as 'for( mut i = 1 ;' ...
for( i = 1 ; i < 5 ; i=i+1 ){
  print(i," ")
}until i%3 == 0 ;
try:
    println("i at end=",i);
catch err { println(); println err ; }

----

.output of snippet `for3`
[source]
----
1 2 3 i at end=
Variable 'i' is undefined.

----



The variable `i` after the loop is still undefined.
To use a variable from the enclosing wrapped context, you can enclose the assignment into a block.

.source of snippet `for4`

[source]
----
mut i;
for {i = 1} ; i < 5 ; i=i+1 :  print(i," "); until i%3 == 0;
println("i at end=",i);

----

.output of snippet `for4`
[source]
----
1 2 3 i at end=3

----



In this last example we also use a single command as the body of the loop.
In this case the command is preceded by a `:` character.

The value of the loop is the result of the last executed command.

.source of snippet `for_result`

[source]
----
println {
    mut k = 0
    for ; k < 10 ; : k++
}

----

.output of snippet `for_result`
[source]
----
9

----



The loop can also be used to produce a list.
The list will contain the results of each body loop execution.

.source of snippet `for_list`

[source]
----
println {for i=0; i < 10 ; i++ list: i}

----

.output of snippet `for_list`
[source]
----
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

----



To get the list as a result, you have to write the `list` keyword in front of the loop body.

[NOTE]
====
When a loop executes, two distinct contexts are used: an *outer context* and an *inner context*.

* The *outer context* is created before the loop begins and is discarded after the loop ends.
* The *inner context* is created freshly for each execution of the loop body and is discarded immediately after that execution completes.

The outer context *wraps* the surrounding scopes, and the inner context *wraps* the outer context.
This means that both contexts have access to all variables defined in the surrounding code.
Additionally, the inner context can read and modify variables from the outer context and the scopes it wraps.
However, *new variables defined in the inner context do not propagate into the outer context*.


* The *start*, *condition*, and *step* expressions in the `+for+` loop header are evaluated in the outer context.

 * The *loop body* and the *exit condition* following the `+until+` keyword are evaluated in the inner context.

This design ensures that each iteration of the loop starts in a clean environment.
Variables defined inside a previous iteration are *not available* in the next iteration, unless explicitly defined in the outer context.
This approach avoids unintended side effects and provides a clear and consistent model for variable visibility and lifecycle within loops.
====

==== `for each`

The command `for each` iterates through a list

.source of snippet `foreach1`

[source]
----
for each i in [1,2,3,4]: print(i," ");

----

.output of snippet `foreach1`
[source]
----
1 2 3 4 
----



The `for each` loop can also have a tail condition.

.source of snippet `foreach2`

[source]
----
for each i ∈ [1,2,3,4]: print(i," "); until i%3 == 0

----

.output of snippet `foreach2`
[source]
----
1 2 3 
----



You can also define an index variable using the `with` keyword.

.source of snippet `foreach3`

[source]
----
for each z with i in ["apple", "banana", "orange", "peach"] {
    println "%s. %s" % [i,z]
}

----

.output of snippet `foreach3`
[source]
----
0. apple
1. banana
2. orange
3. peach

----



The index starts from zero and increments by one with each iteration.
The index variable is local to the loop.
Without this feature, you would need to declare a variable at a higher context level and then use it in the loop.
That would violate the locality principle: do not rely on anything outside a context that can change, and even less modify anything outside the context, which in this case is a loop.

It is also possible to deconstruct multiple values in the for-each loop.

.source of snippet `foreach4`

[source]
----
for each [fruit, origin ] in [ ["apple","Ukraine"], ["banana","Brasil"], ["orange","Spain"], ["peach","Hungary"]] {
println "Fruit %s comes from %s" % [fruit, origin]
}

----

.output of snippet `foreach4`
[source]
----
Fruit apple comes from Ukraine
Fruit banana comes from Brasil
Fruit orange comes from Spain
Fruit peach comes from Hungary

----



If you specify multiple loop variables between `[` and `]`, then the loop values will be deconstructed.
Deconstructing objects is not supported.

[NOTE]
====
Other languages allow multiple variables in for-each loops without extra `[` and `]` characters.
In Turicum, we decided to require the brackets, because it can distinguish between

[source]
----
 for each x in [[1],[2],[3],[4]] : ...
----

and

[source]
----
 for each [x] in [[1],[2],[3],[4]] : ...
----

The value of `x` is a list for each iteration in the first case and a number in the second case.

It also makes the code easier to read.
====


If you do not want to use the `with` part feature of the for-each loop, you can use the `enumerate()` built-in function.
This function (the idea) was taken from Python.

.source of snippet `foreach5`

[source]
----
for each [ j, z ] with i in enumerate(["apple", "banana", "orange", "peach"]) {
println "%s.%s. %s" % [i, j, z]
}

----

.output of snippet `foreach5`
[source]
----
0.0. apple
1.1. banana
2.2. orange
3.3. peach

----



As you can see, the variables `j` and `i` get the same value in each iteration.

When used in an expression position, a `for each` loop returns the result of the last executed loop body,
or a list if the `list` keyword was used.

.source of snippet `for_each_result`

[source]
----
println {
    for each t with i in [1,2,3] : t + 10*i
}

----

.output of snippet `for_each_result`
[source]
----
23

----



The loop can also return a list: to do so, use the `list` keyword before the loop body.
The result will include the result of each iteration.

.source of snippet `for_each_list`

[source]
----
println {
    for each z with i in [1,2,3] list { 10*i + z }
    }

----

.output of snippet `for_each_list`
[source]
----
[1, 12, 23]

----



To get the list as a result, you have to write the `list` keyword in front of the loop body.

[NOTE]
====
When a loop executes, two distinct contexts are used: an *outer context* and an *inner context*.

* The *outer context* is created before the loop begins and is discarded after the loop ends.
* The *inner context* is created freshly for each execution of the loop body and is discarded immediately after that execution completes.

The outer context *wraps* the surrounding scopes, and the inner context *wraps* the outer context.
This means that both contexts have access to all variables defined in the surrounding code.
Additionally, the inner context can read and modify variables from the outer context and the scopes it wraps.
However, *new variables defined in the inner context do not propagate into the outer context*.


* The *list* is evaluated only once in the outer context.

* The loop variable and the optional loop counter are part of the inner context, but they are pinned.
Pinned variables cannot be modified.

 * The *loop body* and the *exit condition* following the `+until+` keyword are evaluated in the inner context.

This design ensures that each iteration of the loop starts in a clean environment.
Variables defined inside a previous iteration are *not available* in the next iteration, unless explicitly defined in the outer context.
This approach avoids unintended side effects and provides a clear and consistent model for variable visibility and lifecycle within loops.
====

==== `break`

The command can break the execution of the loop and can also specify the value of the loop.

==== `continue`

Continues the execution of the loop, starting the body again without executing the rest of the commands in the current iteration.

=== Returning, Breaking, Continuing, and Yielding

These commands belong to one family.
Each of them can pass a value, and each can have a condition following the expression.
The syntax is

[source]
----
break [expression] [when condition]
return [expression] [when condition]
yield expression [when condition]
----

The expression and the ``when condition`` parts are optional, except for the `yield`.
`yield` must have the expression.
When the conditional part is missing, the command is executed as if it were `when true`.

NOTE: Earlier versions permitted the use of the `if` keyword, but it became ambiguous when the `pass:[x if condition else y]` expression syntax was introduced.

==== `break`

The command `break` stops the execution of a loop.
If it is executed, the value of the loop will be the expression that follows the command.
If it is missing then the value `none` will be the result.

There is a slight difference between not having an expression or having an expression returning `none`.
When the `break` command is used in a list-producing loop and there is an expression following the `break` keyword, the value of this expression will be appended to the list as the last element.
When there is no expression, then the execution of the loop still terminates, but no expression will be appended to the result list.

The following samples demonstrate these cases.

.source of snippet `break1`

[source]
----
println {
    for i=0 ; i < 10 ; i++ {
      println ".. ",i;
      break when i == 2
    }
}

----

.output of snippet `break1`
[source]
----
.. 0
.. 1
.. 2
none

----



.source of snippet `break2`

[source]
----
println {
    for i=0 ; i < 10 ; i++ {
      println ".. ",i;
      break -2 when i == 2
    }
}

----

.output of snippet `break2`
[source]
----
.. 0
.. 1
.. 2
-2

----



.source of snippet `break3`

[source]
----
println {
    for i=0 ; i < 10 ; i++ list{
      break when i == 2
      i;
    }
}

----

.output of snippet `break3`
[source]
----
[0, 1]

----



.source of snippet `break4`

[source]
----
println {
    for i=0 ; i < 10 ; i++ list{
      break -2 when i == 2
      i;
    }
}

----

.output of snippet `break4`
[source]
----
[0, 1, -2]

----



.source of snippet `break5`

[source]
----
println {
    for i=0 ; i < 10 ; i++ list{
      break none when i == 2
      i;
    }
}

----

.output of snippet `break5`
[source]
----
[0, 1, none]

----



The last example shows the difference between using the break statement following an `if` condition.
In the first case, the statement executes within the loop, and the `break` statement aborts the loop's execution.

In the second case, the `break` is inside a block, and it aborts the execution of the block itself; the loop is not disturbed.

.source of snippet `break6`

[source]
----
println {
    for i=0 ; i < 4 ; i++ {
      if i == 2 : break;
      println i;
    }
}
println "however"
println {
    for i=0 ; i < 4 ; i++ {
      if i == 2 { break };
      println i;
    }
}

----

.output of snippet `break6`
[source]
----
0
1
none
however
0
1
2
3
3


----



When the expression following the `break` keyword starts with the `if` keyword, it must be enclosed in `()` parentheses.

.source of snippet `break7`

[source]
----
println {
    for i=0 ; i < 4 ; i++ {
      break if i == 2 : 3 else : 4 when i == 2;
      println i;
    }
}

----

.output of snippet `break7`
[source]
----
0
1
3

----



==== `continue`

The continue command will "break" the current loop execution and start the loop over for the next iteration.
In the case of a for loop, the step command will be executed, and the exit condition will also be evaluated.

The `continue` command can have an expression, but it is only allowed if the loop is list-producing.
In this case, the value of the expression will be appended to the resulting list.
If the value of the expression is `none`, then `none` as a value will be appended to the list.
If there is no expression after the `continue` keyword, then the resulting list will remain untouched for the actual execution.

.source of snippet `mixed_continue`

[source]
----
println {for each z in [1, 2, 3, 4, 5, 10, 11, 12] list {
            if z % 5 == 0 : continue;
            if z % 3 == 0 : continue z + 0.1;
            z
          }
        }

----

.output of snippet `mixed_continue`
[source]
----
[1, 2, 3.1, 4, 11, 12.1]

----



As you can see, the resulting list adds 0.1 to each element that is a multiple of three, but the elements that are multiples of five are skipped.

You cannot use `continue` with a value in a loop that does not produce a list:

.source of snippet `bad_continue`

[source]
----
try:let k={ for each z in[1, 2, 3] {
        continue z / 2 when z % 2 == 0 ;
        continue z;
    }
} catch e:println e

----

.output of snippet `bad_continue`
[source]
----
You cannot 'continue' with a value in a non-list resulting loop.

----



==== `return`

The command `return` stops the execution of a function, method, closure, or macro.
If it is executed, the value of the function, method, closure, or macro will be the expression that follows the command.

If there is no expression following the `return` keyword then `none` will be returned.

.source of snippet `return1`

[source]
----
fn a(x) {
    if x == 2 {
        return 2
    }
    return
}
println a(1)
println a(2)
println a(3)

----

.output of snippet `return1`
[source]
----
none
2
none

----



==== `yield`

The command yield produces a value, and the execution of the code continues.
It can only be used when the code was started so that another thread is expecting the yielded value.
The typical example is when you execute a command with `async`.

.source of snippet `yield1`

[source]
----
fn gen_1_to_10 {
    for i=1 ; i <= 10 ; i++:
        yield i;
}

println([ .. (async gen_1_to_10()) ])

----

.output of snippet `yield1`
[source]
----
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

----



The return value can be used in a `for each` loop, or you can iterate through it the same way you would with any iterator.

.source of snippet `yield2`

[source]
----
fn gen_1_to_10 {
    for i=1 ; i <= 10 ; i++:
        yield i;
}

mut st = async gen_1_to_10();
while st.has_next() :
    println(st.next());

----

.output of snippet `yield2`
[source]
----
1
2
3
4
5
6
7
8
9
10

----



=== `with`

The `with` command executes a block in the context of one or more objects, and it also functions as a resource manager similar to Python.

[source]
----
with object list { block }
----

or

[source]
----
with object list: command
----

The `object list` is a comma-separated list of expressions, each with an optional `as identifier` part.
When there is an `as identifier` the functionality is similar to that of Python.

When there is an `as ID` after the expression, then the expression is treated as a resource manager.
The method `entry()` is invoked on the resource manager, and the returned value is assigned to the given `ID`.
The returned value is usually a resource object.

For example, handling a file, the resource manager should return an object you can use to write into a file.

The identifier is used in the block following the `with` command referencing the handle returned by `entry`.

If `entry` returns `none`, then the resource manager object itself is used as a resource.
This can be used in simplified cases when the resource manager and the resource object are the same.

When the execution of the block is finished the method `exit` will be invoked on the resource manager.
This method will be invoked even if there was any error during the execution of the block.

.source of snippet `withAs`

[source]
----
class ResourceManager {
    fn init {
        // each resource manager object can manage one resource object
        let res = Resource();
    }

    fn entry {
        println $"entry to ResourceManager ${this}"
        res
    }

    fn exit( e: err|none = none) {
        println $"exit fom ResourceManager ${this} with \"${e}\" as exception"
        if e != none : // we handle the exception printing it
            println e
        res.close() // close the resource
        return true; // suppress exceptions
    }
}

class Resource {
    fn init {
        println "Creating resource"
    }
    fn printf(s,[rest]){
        println "printf was called"
        println s % rest
    }
    fn close(){
        println $"closing resource"
    }
}

with ResourceManager() as r {
    println $"r resource is ${r}"
    r.printf("executing with resource %s %s",1,2)
    die "mad dog"
}
println "did not die"

----

.output of snippet `withAs`
[source]
----
Creating resource
entry to ResourceManager {res: {this: {...}, cls: class Resource}, this: {...}, cls: class ResourceManager}
r resource is {this: {...}, cls: class Resource}
printf was called
executing with resource 1 2
exit fom ResourceManager {res: {this: {...}, cls: class Resource}, this: {...}, cls: class ResourceManager} with "mad dog" as exception
mad dog
closing resource
did not die

----



The argument to the method `exit` is the exception that was raised while processing the block, or `none` if there was not any.
If the return value of the method `exit` is `true` (or anything that can be converted to `true`), then the exception is treated as caught and handled.
If there is any exception during the closing, then it will be thrown.
Exceptions during resource closing do not get suppressed.

If there is more than one resource in a `with` statement, their `entry` methods are called from left to right and their `exit` methods from right to left.
The last "opened" resource is closed first.

[NOTE]
====
Handling the resources this way is similar to how Python handles the resources with some notable differences:

* There are no underscore characters in the name of the method `entry` and `exit`.
* `exit` cannot suppress exceptions on its own
* `entry` can return `none`.
It is the same as returning `this`.
* `exit` gets only one argument, the exception that holds all the information.
In Python, you get the stack trace and the exception type in a separate argument.
* You _MUST_ provide an `as` alias for resources, otherwise the expression is not treated as a resource (see below).
====

If the expression following the `with` statement is not followed by an `as` and an alias, it is treated as an ordinary object.
The fields of the object get mapped to the actual context.
It means that the fields of the objects can be used as variables without prefixing them with any object reference.

.source of snippet `simpleWith`

[source]
----
mut x = {a:1,b:2};
mut y = {a:4,c:5};
with x,y {
    println a
    println b
    println c
}

----

.output of snippet `simpleWith`
[source]
----
4
2
5

----



Technically, the command executes the block in a context that references the fields of the object and wraps the surrounding context.
When more than one object is listed, then multiple wrapping scopes are created from left to right.
It also means that the objects to the right shadow the objects to the left.
In the example both `x` and `y` have field `a` and since `y` is later in the list the field of `y` is used.

You can also modify or even create and pin fields:

.source of snippet `modifyWith`

[source]
----
mut x = {a:1,b:2};
mut y = {a:4,c:5};
with x, y {
    a = 0
    b = 0
    c = 0
    mut k = 55;
    pin b
}
try: x.b = 55 catch e: println e
println x
println y

----

.output of snippet `modifyWith`
[source]
----
pinned variable cannot be altered 'b'
{a: 1, b: 0}
{a: 0, c: 0, k: 55}

----



Note that the field `x.a` was not modified, because it is shadowed by `y.a`.
You can only create new fields in the rightmost object.
You can pin a field in any of the objects unless they are shadowed.

The value of the command is the value of the block executed.

.source of snippet `valueWith`

[source]
----
mut x = {a:1,b:2};
mut y = {a:4,c:5};
println {with x,y : a * b + c}

----

.output of snippet `valueWith`
[source]
----
13

----



It is important to note that the `with` command can also be used with classes.
The `with` command can handle classes similar to objects and also as resource managers.
Using a class as a resource manager instead of an object is not recommended.
Remember: each resource manager can manage a single resource.

When you use a class in the `with` command instead of an object the class fields and methods will be invoked.

.source of snippet `withClass`

[source]
----
class Resource {
    fn k(){
        println "haha"
    }
}

with Resource {
    k()
}

----

.output of snippet `withClass`
[source]
----
haha

----



=== Mutual Exclusion: `mutex` and `sync`


The function `mutex()` creates a mutual exclusion lock, a mutex.
The `sync` command executes a block or a command while holding a mutex:

[source]
----
sync expression { block }
----

or

[source]
----
sync expression : command
----

The expression must evaluate to a mutex.
Only one thread at a time can hold a mutex; a thread executing a `sync` command on a mutex that another thread holds waits until the holder releases it.
Use it to protect invariants that span several variables or objects, for example, moving an amount between two accounts so that no other thread can see the intermediate state.

The mutex is released when the body finishes.
The release is guaranteed: it also happens when the body throws an exception, and even when the executing thread is aborted because of a time or step limit.
The exception raised in the body, if there is any, propagates; `sync` never suppresses it.
The value of the `sync` command is the value of the body.

The mutex is reentrant: a `sync` command nested inside another `sync` on the same mutex executes without waiting.

The methods `lock()`, `unlock()`, `try_lock()`, `try_lock(seconds)`, `is_locked()`, and `is_held()` are also available on the mutex object for advanced use.
When you call these methods directly, the program is responsible for releasing the mutex it acquired.
The `sync` command is the recommended form because it guarantees the release.

The type of a mutex is `mtx`, as in

[source]
----
let l : mtx = mutex()
----

[NOTE]
====
A mutex is not a resource manager: it cannot be used with an `as` alias in a `with` command.
Use the `sync` command.

A mutex protects only the accesses that go through it.
If any code path reads or writes the shared data without holding the mutex, the protection is gone.
When a single shared value is enough, prefer an atomic value, where unsynchronized access is not possible at all.
====



.source of snippet `sync_mutex`

[source]
----
// The mutex() built-in creates a reentrant mutual exclusion lock, and the 'sync' command
// executes its body while holding it. The release is guaranteed: it happens on normal
// completion, on exception, and even when the executing thread is aborted.
let l : mtx = mutex()

sync l {
    println "inside sync"
}

// the value of the sync command is the value of its body
let r = { sync l : 42 }
println "r = " + r

// nested sync on the same mutex is allowed, the lock is reentrant
sync l : sync l : println "reentrant ok"

// the lock methods are available for advanced use
println "try_lock: " + l.try_lock()
println "is_held: " + l.is_held()
println "is_locked: " + l.is_locked()
l.unlock()
println "after unlock is_locked: " + l.is_locked()

// sync releases the mutex even when the body dies
try {
    sync l : die "boom"
} catch e {
    println "caught: " + e.message
}
println "released after die: " + (l.is_locked() == false)

----

.output of snippet `sync_mutex`
[source]
----
inside sync
r = 42
reentrant ok
try_lock: true
is_held: true
is_locked: true
after unlock is_locked: false
caught: boom
released after die: true

----



=== Atomic Values

The function `atomic()` creates an atomic cell holding a single value.
The value is reachable only through the synchronized operations of the cell; there is no way to access it without synchronization.
Any value can be stored, not only numbers.

The operations of the cell are:

* `get()` returns the current value.
It does not use a lock; it is safe and fast to call from any thread.
* `set(v)` replaces the value.
* `update(f)` atomically replaces the value with `f(value)` and returns the new value.
The function runs exactly once, while the cell is locked.
Do not perform blocking operations and do not use other atomic cells or mutexes inside it.
* `cas(expected, new)` sets the value to `new` if the current value equals `expected` and returns `true` if it did.
* `incr()`, `decr()`, and `add(n)` are numeric conveniences; they atomically modify the value and return the new value.
They report an error when the stored value or the argument is not a number.

Lists and objects stored in the cell are pinned: the cell holds and hands out immutable snapshots.
To change the value, build a new one and `set()` or `update()` it in.
This way, a value read from the cell can never be modified behind the cell's back.

.source of snippet `atomic_cell`

[source]
----
// The atomic() built-in creates a cell holding a single value. All mutating operations are
// synchronized on the cell, reads are lock-free, so threads can share the cell safely.
let a : atm = atomic(0)

println "incr: " + a.incr()
println "add: " + a.add(9)
println "decr: " + a.decr()
println "get: " + a.get()

a.set(100)
println "cas hit: " + a.cas(100, 5)
println "cas miss: " + a.cas(100, 7)
println "get: " + a.get()

// update atomically replaces the value with f(value) and returns the new value
println "update: " + a.update({|x| x * 2})

// any value can be stored; lists and objects become pinned (immutable) snapshots
let boxed = atomic([1, 2])
println "boxed: " + boxed.get()
let peeked = boxed.get()
try {
    peeked[0] = 9
} catch e {
    println "stored snapshot is pinned"
}
println a

----

.output of snippet `atomic_cell`
[source]
----
incr: 1
add: 10
decr: 9
get: 9
cas hit: true
cas miss: false
get: 5
update: 10
boxed: [1, 2]
stored snapshot is pinned
atomic[10]

----



The type of an atomic cell is `atm`, as in

[source]
----
let counter : atm = atomic(0)
----

Tasks started with `async` receive frozen copies of the surrounding variables; they cannot assign the variables of the parent.
An atomic cell is shared by reference: the frozen copy in the task refers to the same cell, so a task can `incr()` a counter or publish a result that the parent reads with `get()` without any further ceremony.

=== Asynchronous execution

You can execute any command asynchronously using the `async` command.
You simply have to write `async` in front of the command.
The command will start the execution in a separate thread (Java 21 virtual thread) and the starting thread will go on executing.

The return value of the `async` command is a `task`.

To wait for an asynchronously started command to finish, use the `await` command.
It returns the value produced by the task.

In addition to returning a value, a task can also receive and send objects.
The `yield` command inside the task will either receive an object or send an object.

.source of snippet `async`

[source]
----
mut t: task=async{
    for i= 0 ; i < 10 ; i ++ {
        yield i
    }
    55
}

    while t.has_next() :
print t.next(), " "
println()
println( await t )


----

.output of snippet `async`
[source]
----
0 1 2 3 4 5 6 7 8 9 
55

----



A task object implements the following methods:

* `is_done` is `true` when the task has finished.

* `is_cancelled` is `true` when the task was cancelled

* `is_timeout` is `true` when the task was stopped due to timeout

* `is_err` is `true` when the task ended in an error

* `get_err` returns the error

* `get` returns the result of the task.
It is the same as the result of the `await` command.

* `close` closes the channel to the task.
After calling this method, you cannot send anything to the task and the task will see that the channel is closed.

* `send` will send an object to the task

* `has_next` will be `true` if there are objects sent by the task

* `next` gets the next value from the task

The command `async` has four parameters:

* `in` sets the size of the queue for objects to be sent to the task.
The default value is unlimited.
If the queue is limited and full, sending an object will block until a place is freed.

* `out` sets the size of the queue for the task to yield values back to the main thread.
If the queue is full, the task will wait.
This can be used to limit the memory and CPU usage of infinite generators.

[NOTE]
====

Say you want to create a generator that returns the prime numbers.
If you do not limit the channel size, the generator will run ahead and fill the memory with prime numbers.
By limiting the channel size, you can balance the calculations.
The generator will stop when it has generated enough numbers and resume when some of the results are consumed.

.source of snippet `channel_limit`

[source]
----
// stochastic
/**
  * a very simple prime number filter
  */
fn is_prime(x:num):bool {
    return true when x == 2;
    return false when x % 2 == 0;
    for n = 3 ; n*n <= x ; n = n + 2 :
        return false when x % n == 0
    true
}
/**
  * Create a generator with limited que size.
  * Then read from it and print.
  */
fn execute(name:str, que_length:num){
    mut prime = async[in=que_length] {
        yield 2;
        mut i = 3;
        while {
            if is_prime(i) {
                yield i
                println name, " generated ", i
            }
            i = i + 2; // only the odd numbers
        }
    }

    mut count = 0
    while prime.has_next(){
        println " prime read ",prime.next()
        count++
        }until count > 3 // limit the printout to the first three primes
    sleep 1 // will give time for the generator to run ahead uselessly
    println "stopping"
    prime.stop()
    // the error is cancellation, since we stopped the task
    try: await prime catch e : println "exception ", e
}
// only 5 for demonstration, in real life it can be millions eating memory
// five now stands for "unlimited size"
execute("Thread 1",5)
// limited channel size will stop the running ahead
execute("Thread 2",2)

----

.output of snippet `channel_limit`
[source]
----
 prime read 2
Thread 1 generated 3
 prime read Thread 1 generated 5
3
Thread 1 generated 7
 prime read 5
 prime read Thread 1 generated 11
Thread 1 generated 13
7
Thread 1 generated 17
Thread 1 generated 19
Thread 1 generated 23
stopping
exception Task stopped.
Thread 2 generated 3
 prime read 2
 prime read 3
Thread 2 generated 5
Thread 2 generated 7
 prime read 5
 prime read 7
Thread 2 generated 11
Thread 2 generated 13
stopping
exception Task stopped.

----



In the output you can see that the calculation went ahead uselessly in the first case.
In the second case the calculation was limited.
====

* `steps` limits the number of elementary steps in the execution.
The task will stop after this many steps.
When a task exceeds the step limit the task `is_done()` and `is_err()` methods will return `true`.
The error message will indicate the step limit that was reached.
See <<cleanup-after-halt>> for what happens to `finally`/`with` cleanup code when the step limit is reached (or the task is aborted from outside).

.source of snippet `async_step_limit`

[source]
----
let z=async[ steps= 100] {
    loop{
        mut i= 0
        } while{
        print $"${i++} "
    }
}

// sleep 100ms, should be enough to timeout
sleep 0.1

println "\nResults are true, true and false"
println $"  z is done ${z.is_done()}"
println $"  z is err ${z.is_err()}"
println $"  z is timeout ${z.is_timeout()}, it is step limit, not time"
println $"  ${z.get_err()}"

----

.output of snippet `async_step_limit`
[source]
----
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 
Results are true, true and false
  z is done true
  z is err true
  z is timeout false, it is step limit, not time
  Step limit 100 reached

----



* `time` limits the execution time of the task; the unit is seconds.
The task will stop after the specified time.
When a task times out, the underlying thread stops and the task's `is_done()`, `is_err()`, and `is_timeout()` methods will return `true`.

These parameters can be specified after the `async` keyword between `[` and `]` characters.
When the command to be executed asynchronously starts with a `[` character, you either have to place `[]` in front of it,
or enclose the command between `(` and `)` or `+{+` and `+}+` to help the syntax analysis.

.source of snippet `async_timeout`

[source]
----
let z=async[ time= 1 / 20] {
    while{
        //burn some CPU doing nothing
    }
}

// sleep 100ms, should be enough to timeout
sleep 0.1

println "All three results should be true"
println $"  z is done ${z.is_done()}"
println $"  z is err ${z.is_err()}"
println $"  z is timeout ${z.is_timeout()}"

----

.output of snippet `async_timeout`
[source]
----
All three results should be true
  z is done true
  z is err true
  z is timeout true

----



The command can also execute a list in parallel.
When the expression to execute asynchronously is a list literal, the command will execute the individual elements of the list in parallel.
In that case the result is a list of tasks instead of a task that results in a list.

[NOTE]
====
Do not enclose the list literal between `(` and `)` when using `async`.

* `[1, 2, 3]` is a list literal.
`async` will evaluate it as a parallel list.

* `([1, 2, 3])`, and `{[1, 2, 3]}` are block commands with a list literal in them.
`async` will evaluate the block as a single command.

The only way to create an async list is to provide the possibly empty options as in the next example.
====

The following example creates a list of `[1,2,3]` in the conventional way and also in parallel.
It prints out the elapsed time using the built-in `nano_time` function.

.source of snippet `asyn_list`

[source]
----
// stochastic
fn sleeper( x ){
    sleep 0.1
    x
}
{
    mut start=nano_time()
    mut serial=[ sleeper(1), sleeper(2), sleeper(3) ]
    mut end=nano_time()
    println serial
    println( ( end - start ) / 1000000, "ms")
}
{
    mut start=nano_time()
    mut serial=[ .. (async[][ sleeper(1), sleeper(2), sleeper(3) ])  -> it.get() ]
    mut end=nano_time()
    println serial
    println( ( end - start ) / 1000000, "ms")
}

----

.output of snippet `asyn_list`
[source]
----
[1, 2, 3]
315.751416ms
[1, 2, 3]
103.638042ms

----



As you can see, in the first case the execution time is a bit above 300ms, which is three times the 0.1s wait in the function.
In the second case the functions wait the same time in parallel, and the execution time is just above the 100ms single wait time.

[[flow-command]]
== `flow` Command

The `flow` command defines a *reactive execution block* in which named expressions (called *cells*) are evaluated in a concurrent and data-driven fashion.
It is particularly useful for implementing iterative or mutually dependent computations where values must be recalculated until a condition is satisfied or a fixed number of changes have occurred.

Cells are defined as variable-expression bindings, and their dependencies are automatically detected by analyzing identifier usage.
When a cell value changes, all other cells that depend on it are automatically rescheduled for evaluation in a new thread (using Java virtual threads).

The `flow` block terminates under one of the following conditions:

- The `until` condition evaluates to `true`.
It is repeatedly evaluated after each task in the context of the `flow` command.
- The maximum number of task executions (`limit`) is reached.
- The allowed wall-clock execution time (`timeout`) is exceeded.
- The flow reaches a stable state: there are no more tasks to run.

In these cases the already scheduled tasks are still executed before the flow stops.

If a `yield` expression is specified, its result is returned as the result of the `flow` command.
It is evaluated once after all scheduled tasks are finished.
The evaluation happens in the context of the `flow` command.
If there is no `yield` in the `flow` command, `null` is returned.

The evaluation starts with the cells that have no dependencies.
These are evaluated in asynchronous threads.
Scheduling and executing dependent tasks begins once all start cells have finished and the flow has reached its initial state.

If there are cells that have dependencies (and therefore are not start cells), but whose transitive closure of dependencies does not include any start cells, it is an error.
This is detected at compilation time and results in a syntax error.

=== Syntax

The basic syntax of the `flow` command is:

[source]
----
flow [<identifier>] [until <condition>] [limit <maxTasks>] [timeout <seconds>] {
  <identifier> <- <expression>
  ...
  yield <expression>
}
----

- The identifier following the `flow` keyword is optional.
It is used solely for diagnostic purposes.
It does not enter any symbol table and does not hide any object of the same name in the context hierarchy.
It is used only in error messages and is available for command introspection.
This identifier cannot be `limit` or `timeout`, because those signal the start of the limit and timeout expressions respectively.

- `until <condition>`::
Optional.
A boolean expression that terminates the flow once it evaluates to true.
- `limit <maxTasks>`::
Optional.
A numeric expression that sets the maximum number of dependent cell executions allowed.
It is evaluated once, when the flow starts.
- `timeout <seconds>`::
Optional.
A numeric expression (can be fractional) specifying the maximum wall-clock time allowed for execution, in seconds.
It is evaluated once, when the flow starts.

[NOTE]
====
`until`, `limit` and `timeout` are all optional and can be in any order.
`limit` and `timeout` are not keywords.
The syntax analyzer cannot tell whether they start the limit or timeout definition or you want to name the flow "limit" or "timeout".
You cannot name the flow either of these.
====

The values of the clauses are parsed as expressions, the same way as the conditions of the `if` and `while` commands.
A bare identifier is a valid condition, and no `;` is needed between the clauses or before the body:

[source]
----
flow until finished limit 100 timeout 1.5 {
   ...
}
----

You can still write a `;` after each clause if you find it more readable.
Commands are not expressions in Turicum; if the condition needs one, enclose it in `{` and `}` as a block expression.

Each

[source]
----
<identifier> <- <expression>
----

is a __cell__.
During the execution the expression, which can even be a whole block, just like any expression in Turicum, is evaluated asynchronously, and when it is ready, the resulting value updates the flow local variable on the left side of the `pass:[<-]`.

The expression is the cell expression.
The identifier is the __cell state variable__.

When the syntax analysis is done, the compiler performs several semantic checks.
It checks that there are __start cell__s, that can be calculated independent of any other.

No two __start cell__s should update the same __cell state variable__.

Following this analysis, the compiler checks that each cell is reachable from the __start cell__s.
All cells must be reachable.

NOTE: Unreachable cells only can happen if a cell depends on itself or there is a circular dependency.
Without such a dependency the cell would be a start cell.

The last check is that all dependency cycles contain at least one __start cell__.
If there is a dependency cycle that does not contain a __start cell__, that cycle would use a __cell state variable__ before it is initialized.

If any of the checks fail, the compiler reports a syntax error and the command is not created.

=== Semantics

The `flow` block contains a series of *cells*.
Each cell defines a named variable and an expression that computes its value.
These variables can reference each other, and the runtime will detect and track these dependencies.

When a __cell state variable__ is updated, (i.e., its computed value differs from the previous value in the local context), all cells that depend on it are automatically rescheduled.
This process continues until one of the termination conditions is met.

A __cell state variable__ is updated if

* it was undefined and the new value is not `non_mutat`, and not `fini`, OR
* it was defined and the value of the evaluation does not `equals` the current value.

When the value of the __cell block__ evaluation is `non_mutat` the __cell state variable__ is not updated.

When the value of the __cell block__ evaluation is `fini` the __cell state variable__ is not updated and the cell will not be scheduled anymore.
If there are already running instances of the cell, they will finish and possibly update the __cell state variable__ assuming they were started after the cell block that resulted the `fini` value.

When a result is `fini`, the __cell state variable__ counter is updated even though the variable value is not; thus any block that was started earlier and not yet finished will not update the variable.

When the result is `non_mutat`, nothing is updated.

NOTE: `fini` and `non_mutat` are predefined pinned global constant objects.

Cell evaluation is performed in isolated thread-local contexts derived from the flow's original context.
This context contains copies of the flow context variables, which are read-only.

The system avoids infinite re-evaluation loops by tracking value stability and respecting the `limit` and `timeout` constraints.

The execution of the code blocks is done asynchronously.
There is no guarantee for any order.
When a cell variable that a block depends on changes (because the block uses it in some calculation), the block is scheduled for execution and started as soon as the JVM and the operating system allow.
Each cell block is executed in a new virtual thread.

It can happen that the same block executes in parallel in multiple instances.
This could lead to instability in the flow.
An earlier-started instance can run longer than a later one.
Also, different blocks can update the same __cell state variable__.
To avoid this the `flow` command only updates a __cell state variable__ to the value resulting from a cell block execution only if the __cell state variable__ was not updated in the meantime.
If the __cell state variable__ was already updated, the result is ignored.
It is implemented using __cell state variable__ version counters.

When the command is created, the dependencies between the __cell state variable__s are analyzed.
Those cells that do not depend on any other cell are __start cell__s.
They will be executed first.

It is important that other cell calculations start only after the __start cell__s have been calculated.
If you have multiple __start cell__s and a cell block depends on more than one, then they will both be defined when the dependent cell calculation starts.

==== Cells are Local

The __cell state variable__s are local to the `flow` command.
A flow does not care what variables are defined around it: a cell may freely have the same name as a variable of the surrounding scope or a global variable.
The cell is used inside the flow; the outer variable is neither read nor modified.
In particular, before a cell has computed its first value, the `until` condition and the other cells never fall back to a same-named outer variable.
Variables of the surrounding scope that do not collide with any cell name can be used freely in the cell expressions and in the `until` condition.

.source of snippet `flow_cell_locality`

[source]
----
// Cells are local to the flow. The exit condition may freely use variables of the
// surrounding scope as long as their names do not collide with cell names.
// The single self-chained counter makes the result deterministic: the exit condition is
// re-evaluated before every cell value save, so no save can happen past the first true.
let threshold = 5

let counted = {
    flow until counter >= threshold {
        counter <- 0;
        counter <- counter + 1;
        yield counter
    }
}
println "counted: " + counted

// During warm-up the exit condition may reference a cell that has not computed yet:
// 'b' is still undefined at the first evaluation of the condition, which is tolerated.
let doubled = {
    flow until b >= 2 {
        a <- 1;
        b <- a * 2;
        yield b
    }
}
println "doubled: " + doubled

// A flow does not care what variables are defined around it: a cell may shadow a
// variable of the surrounding scope. The flow uses its own local cell; the outer
// variable is neither read (not even before the cell computes) nor modified.
let value = 100
let shadowing = {
    flow until value >= 3 {
        value <- 0;
        value <- value + 1;
        yield value
    }
}
println "shadowing flow: " + shadowing
println "outer value: " + value

----

.output of snippet `flow_cell_locality`
[source]
----
counted: 5
doubled: 2
shadowing flow: 3
outer value: 100

----



==== Undefined Variables in the Exit Condition

The `until` condition is evaluated repeatedly, and during the start-up of the flow it may reference cells that have not computed their first value yet.
When the evaluation of the condition fails because a *cell* is undefined, the condition counts as `false` and the flow keeps running; the cell will get its value and the condition will be evaluated again.
When the evaluation fails because a name is undefined that is *not* a cell — for example, a typo — the flow stops and reports the error.
Any other error in the evaluation of the condition also stops the flow and reports the error.

[NOTE]
====
- The order of `until`, `limit`, and `timeout` is flexible in the syntax.
- All expressions in the `flow` block are commands and follow standard language rules.
- Only direct assignments (`x <- expression`) are allowed; complex control flow must be expressed inside expression blocks.
====

=== Use Cases

The `flow` command is useful for:

- Converging numeric approximations (e.g., computing square roots)
- Reactive data propagation
- Declarative fixed-point computations
- Defining cyclic dependencies with convergence guarantees

=== Examples

This example demonstrates a classical use of the `flow` command for computing square roots using the Babylonian method.
It shows how reactive cells can update each other iteratively until convergence is reached, and how the `until` clause controls flow termination based on a numeric precision condition.

.source of snippet `example_flow_squareroot`

[source]
----
// stochastic
mut _sqrt = {
    flow until epsilon < 0.0001 {
        a <- 13;
        a <- {
            mut newA = (a + 13 / a) / 2;
            println("a =", newA);
            newA;
        }
        epsilon <- abs(a - (a + 13 / a) / 2);
        yield a
    }
}

println("sqrt(13) ≈ ", _sqrt);
println("sqrt(13) ≈ ", sqrt(13), " calculated by java Math");
println("square ≈ ", _sqrt * _sqrt);

----

.output of snippet `example_flow_squareroot`
[source]
----
a =7
a =4.428571428571429
a =3.6820276497695854
a =3.6063454894655185
a =3.6055513629176015
a =3.6055512754639905
a =3.6055512754639896
a =3.6055512754639896
sqrt(13) ≈ 3.6055512754639896
sqrt(13) ≈ 3.605551275463989 calculated by java Math
square ≈ 13.000000000000002

----



This example illustrates how the `flow` command can be bounded by wall-clock time using the `timeout` clause.
It simulates a long-running or slow computation and demonstrates how flow execution is safely terminated after the specified time limit.

.source of snippet `example_flow_timeout`

[source]
----
// stochastic
try{
    flow timeout 0.01 { // 10 milliseconds
        counter <- 0 ;
        counter <- {
            mut n=counter + 1 ;
            println("counter =", n );
            sleep(0.005); // 5 milliseconds per step
            n;
        }
        yield counter
    }
    } catch e {
    println e
}

----

.output of snippet `example_flow_timeout`
[source]
----
counter =1
counter =2
Flow '#unnamed' timed out after 10 ms

----



This example highlights the use of the `limit` clause to constrain the number of task executions within a `flow`.
It shows how the system halts with a clear error message once the execution limit is reached, making it useful for preventing runaway or cyclic computations.

.source of snippet `example_flow_limit`

[source]
----
mut result = {try {
               flow limit 5 {
                   x <- 1;
                   x <- {
                       mut newX = x + 1;
                       println("x =", newX);
                       newX;
                   }
                   yield x
               }
           } catch e: println e.message}

           println("Final x = ", result);

----

.output of snippet `example_flow_limit`
[source]
----
x =2
x =3
x =4
x =5
x =6
x =7
Task limit has been reached in flow '#unnamed' command after 5 tasks.
Final x = Task limit has been reached in flow '#unnamed' command after 5 tasks.


----



== Functions

Functions play a similar role as in other languages.
They can be top-level functions and methods attached to classes or to objects.

=== Function Declarations

A function declaration is a statement that defines the function.
The function becomes usable after the function definition is executed.
The syntax of the function definition is

[source]
----
fn function_name (parameters) : types {
   body
}
----

or

[source]
----
fn function_name (parameters) -> types {
   body
}
----

`fn` is a keyword, and it is never optional.

.source of snippet `fun1`

[source]
----
fn inc(x) {
  x+1
}
println str(inc).before("@")
println(inc(1))

----

.output of snippet `fun1`
[source]
----
ch.turic.commands.Closure
2

----



`function_name` is the name of the function.
It can be missing, typically when the result, the function itself is assigned to a variable or passed directly as an argument.

.source of snippet `fun3`

[source]
----
fn inc x {
  x+1
}
println str(inc).before("@")
println(inc(1))

----

.output of snippet `fun3`
[source]
----
ch.turic.commands.Closure
2

----



`(parameters)` are the parameters, including default values and parameter types and parameter-passing requirements.
The `(` and `)` is optional when the function has a name.
When it does not have a name, `(` and `)` must be present even if there are zero arguments.

.source of snippet `fun2`

[source]
----
mut inc = fn (x) {
  x+1
}
println str(inc).before("@")
println(inc(1))

----

.output of snippet `fun2`
[source]
----
ch.turic.commands.Closure
2

----



The body of the function can be a block of statements or a single expression.
The expression is specified following a `=` character and the parentheses are a must in this case also.
This is to avoid the unreadable `fn = 2*3` to be declared as a function declaration but `pass:[`fn`]=2*3` an assignment.
`fn = 2*3` is a syntax error.

[NOTE]
====
Be careful when using the `fn x()=...` format.
The following code does not do what you may expect:

.source of snippet `fun10`

[source]
----
fn p()=len "original K";
println str(p()).left("ch.turic.builtins.functions.Len".length())

----

.output of snippet `fun10`
[source]
----
ch.turic.builtins.functions.Len

----



You may expect that it will calculate the length of the string and return that value.
In reality, the expression is the function object `len` itself.
The following string is an expression that by itself stands on its own as a command.
The string constant is just a statement with no effect.

If you enclose the whole expression in a pair of parentheses you get the desired result:

.source of snippet `fun11`

[source]
----
fn p()=(len "original K");
println p()

----

.output of snippet `fun11`
[source]
----
10

----



or

.source of snippet `fun12`

[source]
----
fn p()=len("original K");
println p()

----

.output of snippet `fun12`
[source]
----
10

----



Use `=` to define functions only for straightforward expression functions, that are true expressions.
====

The parameter list is comma-separated.
Each element defines a formal parameter.

Types are optional and follow the same syntax as in variable declarations, but function parameters are always immutable.
You cannot use `mut` or `let` in the parameter list.
Attempting to do so will result in a syntax error.

[NOTE]
====
Function parameters are implicitly immutable.
You cannot use `mut` or `let` in a function or closure parameter list.

The only exception is the `init` method: its parameters become fields of the object being initialized, and therefore they are mutable.
====

The individual types can be identifiers or `(` and `)` enclosed expressions.
The types are separated by `|`.
The type of the returned value is checked against the declared types upon returning from a function.

.source of snippet `fun7`

[source]
----
fn concatenate(a:str, b:str):str = a + b
println concatenate("ab","ba")

----

.output of snippet `fun7`
[source]
----
abba

----



.source of snippet `fun8`

[source]
----
fn concatenate(a:str, b:str):str = 13
try:
    println concatenate("ab","ba")
catch err: println err

----

.output of snippet `fun8`
[source]
----
Cannot return from 'concatenate' the value '13' as it does not fit any of the accepted type of the macro (str)

----



The types are evaluated when the function is defined.

.source of snippet `fun9`

[source]
----
mut myString="num"
fn concatenate(a:str, b:str):(myString) = 13
myString = "str"
try:
    println concatenate("ab","ba")
catch err: println err

----

.output of snippet `fun9`
[source]
----
13

----



There is no effect changing the value of the variable `myString` to `str` when the function was already defined.

==== Parameter Passing

When a function is invoked, the actual parameters can be passed to the function in two different ways between `(` and `)`:

* by position, and
* by name.

Some parameters can be passed in both ways, while others can only be passed positionally, and some can only be passed by name.
This is declared in the formal parameter list.
Each parameter declaration starts with the identifier of the argument.
This is the identifier the function body uses for the variable as well as the identifier the caller uses if the parameter is passed by name.

Parameters preceded with '!' can only be passed positionally.

Parameters preceded with `@` can only be passed by name.

.source of snippet `fun4`

[source]
----
fn inc !x, @how_much {
  x+how_much
}
println str(inc).before("@")
println(inc(1,how_much=5))

----

.output of snippet `fun4`
[source]
----
ch.turic.commands.Closure
6

----



Here `x` is a positional parameter.
Passing a value as `x=1` would be an error.
Parameter `how_much` must be passed by name.

Parameters can also have type declarations and default values.
Passing a value that does not fit the type declaration is an error.
Not passing a value for a parameter that does not have a default value is also an error.

.source of snippet `fun5`

[source]
----
fn inc !x, @how_much=1 {
  x+how_much
}
println(inc(1,how_much=5))
println(inc(1))

----

.output of snippet `fun5`
[source]
----
6
2

----



Parameter types and the default value assignments have the same syntax as in the case of variable assignment.
There is one notable difference: default value expressions can use the binary or `|` operator only if it is between parentheses.

==== Positional and Named parameters order

You are allowed to use positional and named parameters intermixed.
There can be a named parameter and then a position following it.
It can be beneficial in some special situations, but the general recommendation is not to overuse this feature.
List the positional parameters first, named following, and optional parameters (see soon) the last.

The actual algorithm of parameter passing is the following:

* The actual arguments are evaluated from left to right.
* Named passings are assigned to the argument with the matching name.
* Positional arguments are assigned to the first non-named only argument.
That is the first not-yet-assigned positional or positional/named argument.

.source of snippet `fun6`

[source]
----
fn wrong !x, h, !k {
  // just don't do
}
try: wrong(1,3, h= 2)
catch e: println("There is an error",e)

----

.output of snippet `fun6`
[source]
----
There is an errorParameter 'h' is already defined

----



[#_extra_parameters]
==== Extra parameters

In some cases, you want to allow the callers to pass extra arguments and for your function body to use them.
To do that, there are three extra arguments you can use in your function definition:

* `[rest]`,

* `+{meta}+`, and

* `^closure`.

You can name them any way you want, but for the sake of simplicity we will use the names above.
The syntax distinguishing them from the other parameters is the use of the extra `[]`, `+{}+`, and `^` characters.
If they are present, they have to be the last formal parameters in the order as listed above.
Any of them can be missing, but if two present then `[rest]` should be before `+{meta}+` and `^closure`, and `+{meta}+` has to precede `^closure`.

When the function code starts

* `rest` will be a list containing the value of the positional arguments that were not yet assigned to any parameter.
* `meta` will be an object with the named parameters not assigned as field names and the values holding the values.
* `closure` will hold the last parameter of the function call that usually is a closure.

The following example shows how you can use the rest-parameter:

.source of snippet `restParameters`

[source]
----
mut f = fn (a,b,[rest]) {
  println("a=",a)
  println("b=",b)
  for each r in rest: print("%s " % r)
}

f(1,2,3,4,5,6)

----

.output of snippet `restParameters`
[source]
----
a=1
b=2
3 4 5 6 
----



The following example demonstrates the use of the meta-parameter:

.source of snippet `metaParameters`

[source]
----
mut f = fn (a,b,{meta}) {
  println("a=",a)
  println("b=",b)
  for each r in keys(meta): print("%s=%s\n" % [r , meta[r]])
}

f(1,b=2,k=3,x=4,juice=5,six=6)

----

.output of snippet `metaParameters`
[source]
----
a=1
b=2
six=6
x=4
k=3
juice=5

----



[NOTE]
====
When you

* have a `!name` declared positional only parameter, and
* you also have `+{meta}+` in the parameter list, and
* and you call the function providing a `name=expression` named argument,

it will not be assigned to the positional only parameter, and it is also not an error.
It will be a field in the meta-parameter object created.

The reason for this is that the naming of the positional only parameters is not part of the function signature.
They are the internal business of the function itself, and as such must not have any effect how you call the function.

On the other hand, no named parameter name, or named/positional parameter name will get into the meta-parameter.

Also, you cannot use a parameter name in an argument list more than once.
====

You can also combine the two:

.source of snippet `metaRestParameters`

[source]
----
mut f = fn (a,b,[rest],{meta}) {
  println("a=",a);
  println("b=",b);
  for each r in keys(meta): print("%s=%s\n" % [r , meta[r]]);
  println("rest:");
  for each r in rest: print("%s " % r);
}

f(1,b=2,k=3,x=4,juice=5,7,six=6);

----

.output of snippet `metaRestParameters`
[source]
----
a=1
b=2
six=6
x=4
k=3
juice=5
rest:
7 
----



The last argument, preceded by the `^` character is special.
When you call a function and the last argument is a closure, then you can place that closure outside of the `(` and `)` of the arguments, but ONLY if the closure is defined right there.

For example:

.source of snippet `closureArgument`

[source]
----
mut f = fn (a,b,^closure) {
  println("a=",a);
  println("b=",b);
  println(closure(a,b))
}

f(2,3){|x,y| x+y}
f(2,3){|x,y| x*y}

----

.output of snippet `closureArgument`
[source]
----
a=2
b=3
5
a=2
b=3
6

----



The function gets the closure adding the two numbers and then prints the numbers and the result.

The example:

.wrong example
[source]
----
let f = fn (a,b,^closure) {
  println("a=",a);
  println("b=",b);
  println(closure(a,b))
}
let closure = {|x,y| x+y}

f(2,3)closure
----

does not even compile, since the mandatory `;` or new line is missing after the function call.
Also, it would be unreadable to recognize `closure` as an argument.

NOTE: The actual implemented syntax checks that the next two tokens following the `)` are `+{+`and `|`.

This call can aid in creating domain-specific languages.

==== Spreading lists and objects as arguments

When calling a function, you can precede a parameter with the `..` operator.
This will spread the content of the argument across the parameters.

If the value of the parameter is an object, the fields of the object will be used as named parameters.
The field names will be used as argument names and the values as values.

.source of snippet `spread`

[source]
----
fn a(@a,@b,@c,@d){
    println("%s %s %s %s " % [a,b,c,d])
}
fn b(!a,!b,!c,!d){
    println("%s %s %s %s " % [a,b,c,d])
}
fn c(a,b,c,d){
    println("%s %s %s %s " % [a,b,c,d])
}

a(..{a:1,b:2,c:3,d:4});
b(..[1,2,3,4]);
c(..[1,2],..{d:4},3);

----

.output of snippet `spread`
[source]
----
1 2 3 4 
1 2 3 4 
1 2 3 4 

----



If the value is a list, then the values will be used as positional parameters.

This is useful when you want to pass on parameters you got in the `rest` or `meta` parameters to some other function.

[NOTE]
====
Technically, you can spread any value resulting from an expression that internally implements the `HasFields`, or ``Iterable<?>`` interface.

Currently, the following classes implement the `HasField` interface:

* `Lex` one token in the lexical analysis.
It is used in preprocessors written in Turicum.

* `LexList` the list of tokens.
Just like the class `Lex`, it is used in preprocessors.

* `Closure` is the implementation class for functions and closures.
It has one read-only field, `name`.

* `Macro` is the implementation class for macros.
It has one read-only field, `name`.

* `JavaObject` is the class that implements the handling of any Java object, when all else fails.
It implements reflection to access the declared fields of the class on the enveloped object.

* `LazyObject` is the lazy object that has fields which are evaluated every time they are accessed.

* `LngClass` Turicum class object.
It handles the class level fields.

* `LngException` is any exception caught by a `catch` in the program.
It has a constant set of read-only fields.

* `LngList` Turicum lists.
It has one read-only field `length`.
This class also implements Iterable and is handled as such when used in an argument list spread.

* `LngObject` is a Turicum object.

* `MapObject` any Java `Map<?,?>` object.
The implementation will handle the keys as strings.

The following classes implement the `Iterable` interface in Turicum but do not implement `HasField`, and thus are spread as positional parameters:

* `IndexedString` envelops a string during commands that modify a String.
This is the technical implementation that internally replaces an immutable Java String with a mutable wrapper.

* `JavaArray` makes it possible to index any Java array.

* `LngList` is the list object in Turicum.
Although this class also implements the `HasFields` interface this is handled as an exception and passed as positional arguments when spread.

* `AsyncStreamHandler` is the class handling asynchronous operations and yields.
It makes it possible to use the result of an `async` expression as arguments.
+
.source of snippet `spreadStream`

[source]
----
fn c(a,b,c,d){
    println("%s %s %s %s " % [a,b,c,d])
}
fn numbers {
  for i = 1 ; i <= 4 ; i++ : yield i
}

c(..(async numbers()))

----

.output of snippet `spreadStream`
[source]
----
1 2 3 4 

----


====

=== Modifying Arguments

Function arguments in Turicum are immutable.

When a function is called, its arguments are *pinned* — they are bound to their initial values and cannot be reassigned within the function body.

The only exception is the `init` method, which acts as the object initializer.
In this case, the arguments correspond to the fields of the object being constructed.
Assignment is allowed during initialization and also later because these fields persist beyond the execution of `init`.
If they were pinned like regular arguments, the object's fields would become permanently immutable, which is not desired.

== Joker Method

It is possible to define a special method in a class or attached to an object, named `pass:[.]` (a single dot), which is automatically invoked when the code tries to call a method not defined in the object or class hierarchy.

.source of snippet `dot_method1`

[source]
----
class B {
  fn init(a,b,c);

  fn `.` {
    println a,b,c
  }
}
B(1,2,3).`we can write here anything`()

----

.output of snippet `dot_method1`
[source]
----
123

----



It is a special tool for meta-programming.
In this special method, you can access the name of the method used to invoke the joker method using the variable named  `pass:[.]`.

.source of snippet `dot_method2`

[source]
----
class B {
  fn `.` {
    // print the name of the method, `.`
    // `.` is a local variable holding the name of the metod
    // this.`.` is still the method itself (not the name)
    println `.`
  }
}
B().`we can write here anything`()

----

.output of snippet `dot_method2`
[source]
----
we can write here anything

----



Note that in this case, this variable is a local variable inside the method execution.
You can still access the method itself via the `this` reference:

.source of snippet `dot_method3`

[source]
----
class B {
  fn `.` {
    global str;
    println str(this.`.`).before("@")
  }
}
B().`we can write here anything`()

----

.output of snippet `dot_method3`
[source]
----
ch.turic.commands.Closure

----



Also note that global methods you want to use, like `str` in the example, have to be declared as `global`.
Without that the call will revert to the joker method itself.

== Currying

Currying is a functional programming technique named after the logician Haskell Curry, where a function with multiple arguments is transformed into a sequence of functions, each taking a single argument.
This allows partial application of arguments, creating reusable and composable functions.

In Turicum, you can curry a function by using `pass:[.(...)]` instead of the regular function call syntax `pass:[(...)]`.
The `.` before the parentheses indicates that you are not calling the function but creating a curried version of it with some arguments pre-bound.

.source of snippet `curry1`

[source]
----
let add = fn(x, y) { x + y }
let add5 = add.(5)
print(add5(3))

----

.output of snippet `curry1`
[source]
----
8
----



The curried function remembers the arguments passed during the currying process.
If the function being curried is a method, the resulting function also retains the object it was bound to:

.source of snippet `curry2`

[source]
----
class AddAndOffset {
    fn init(offset=0);
    fn add(a:num,b:num) = a + b + offset
}

let offset5 = AddAndOffset(5)
let add8 = offset5.add.(3)
println add8(2)

----

.output of snippet `curry2`
[source]
----
10

----



This mechanism enables concise and expressive partial application, supporting functional composition and deferred execution.

Currying can follow the same parameter order as ordinary invocation.
If there are named parameters, you can curry them:

.source of snippet `curry_named`

[source]
----
fn add(a:num,@b:num) = a + 2 * b

 let add6 = add.(b=3)
 let times2_add3 = add.(3)
 println add6(2)
 println times2_add3(b=1)

----

.output of snippet `curry_named`
[source]
----
8
5

----



You can also curry functions and methods if they have `[rest]` and/or `{meta}` parameters:

.source of snippet `curry_rest`

[source]
----
fn echo(a,!b,@c,[rest],{meta}){
    println $"a=${a}"
    println $"b=${b}"
    println $"c=${c}"
    println $"rest=${rest}"
    println $"meta=${meta}"
    "who am I kidding?"
}
let ho = echo.(c="cica","alamaba","bourbenk","rist",rust="the best programming language ever", joke=true)
println ho("yayy")

----

.output of snippet `curry_rest`
[source]
----
a=alamaba
b=bourbenk
c=cica
rest=[rist, yayy]
meta={rust: the best programming language ever, joke: true}
who am I kidding?

----



You get the same result when you curry a closure

.source of snippet `curry_closure`

[source]
----
let echo= {|a,!b,@c,[rest],{meta}|
    println $"a=${a}"
    println $"b=${b}"
    println $"c=${c}"
    println $"rest=${rest}"
    println $"meta=${meta}"
    "who am I kidding?"
}
let ho = echo.(c="cica","alamaba","bourbenk","rist",rust="the best programming language ever", joke=true)
println ho("yayy")

----

.output of snippet `curry_closure`
[source]
----
a=alamaba
b=bourbenk
c=cica
rest=[rist, yayy]
meta={rust: the best programming language ever, joke: true}
who am I kidding?

----



It is not only functions and closures that you can curry, but also macros.

.source of snippet `curry_macro`

[source]
----
fn add(a,b) = evaluate(a) + evaluate(b)
let addM = macro(add)
println addM.(3)(4)

----

.output of snippet `curry_macro`
[source]
----
7

----



You can mix currying and macroing a closure or function:

.source of snippet `curry_macro_mix`

[source]
----
fn add(a,b,c) = a + evaluate(b) + evaluate(c)
println macro(add.(1)).(2)(3)

----

.output of snippet `curry_macro_mix`
[source]
----
6

----



The result is a special something that is half closure, half macro.
The argument `a` is evaluated when passed to the currying.
After that, the new curried closure is converted to a macro.
The following parameter `b` in the next curry call is passed without evaluation, and so is the parameter `c` in the actual call.
That way, the argument `a` can be used as it is, a value.
The arguments `b` and `c` need evaluation.

A special feature in Turicum, that you can uncurry a curried function.

.source of snippet `uncurry1`

[source]
----
fn add(a,b)=a + b
let add5 = add.(5)
println add5(5)
println is_curried(add)
println is_curried(add5)
let unc = uncurry(add5)
println unc(6,6)

----

.output of snippet `uncurry1`
[source]
----
10
false
true
12

----



The built-in function converts a curried function, closure, or macro back to its uncurried form.
Uncurry is not a step-by-step transformation.
It does not matter how you got to the curried closure, in one or multiple curry steps:

.source of snippet `uncurry2`

[source]
----
fn add(a,b)=a + b
let be10 = add.(5).(5)
println be10()
let unc = uncurry(be10)
println unc(6,6)

----

.output of snippet `uncurry2`
[source]
----
10
12

----



If you want to do partial uncurrying, you can specify how many arguments you want to uncurry:

.source of snippet `uncurry3`

[source]
----
fn add(a,b)=a + b
let be10 = add.(5).(5)
println "be10() ", be10()
println "uncurry(be10)(6,7) ", uncurry(be10)(6,7)
println "uncurry(be10,0)() ", uncurry(be10,0)()
println "uncurry(be10,1)(6) ", uncurry(be10,1)(6)
println "uncurry(be10,2)(6,6) ", uncurry(be10,2)(6,6)
println "uncurry(be10,curried_arity(be10))(6,6) ", uncurry(be10,curried_arity(be10))(6,6)

----

.output of snippet `uncurry3`
[source]
----
be10() 10
uncurry(be10)(6,7) 13
uncurry(be10,0)() 10
uncurry(be10,1)(6) 11
uncurry(be10,2)(6,6) 12
uncurry(be10,curried_arity(be10))(6,6) 12

----



The built-in function `curried_arity()` returns the number of curried arguments.

There is a slight difference between using `uncurry()` without a second argument and using it with the curried arity specified.
In the first case, the object attachment is also uncurried.
In the second case, the result is still curried; it only has zero arguments, and the curried object association is still there.

.source of snippet `uncurry4`

[source]
----
class CurriedAway {
    fn init(spice);
    fn a(x,y) = x + y + spice
}

let z = CurriedAway(1)
let fun = z.a.(1,1)
println fun()
println uncurry(fun,2)(2,1)
try: uncurry(fun)(1,3) catch err: println err

----

.output of snippet `uncurry4`
[source]
----
3
4
Variable 'spice' is undefined.

----



When `fun` is uncurried in the last call, it gets disconnected from the object and is called as a standalone closure, without the surrounding object and class.
That is the reason why the error message occurs.

Even if the curried closure is not tied to an object, it technically remains curried if the `uncurry()` second, arity parameter is present:

.source of snippet `uncurry5`

[source]
----
fn a(x,y){}
println is_curried(uncurry(a.(1,2)))
println is_curried(uncurry(a.(1,2),2))

----

.output of snippet `uncurry5`
[source]
----
false
true

----



An uncurried and disconnected closure can be surrounded and reclosed, however:

.source of snippet `uncurry6`

[source]
----
class CurriedAway {
    fn init(spice);
    fn a(x,y) = x + y + spice
}

let z = CurriedAway(1)
let fun = z.a.(1,1)
println fun()
println uncurry(fun,2)(2,1)
{
let spice = 3;
println reclose(uncurry(fun))(3,1);
}

----

.output of snippet `uncurry6`
[source]
----
3
4
7

----



== Classes and Objects

=== Class declaration

Classes can be defined using the `class` keyword.
A class is defined when the class definition is executed.

.source of snippet `class1`

[source]
----
class A {
    fn init {
        mut a = "a";
    }
    mut c = "c"
}
mut a1 = A()
mut a2 = A()
a1.a = "1"
a2.a = "2"
A.c = "d"
println a1.a, a2.a, a1.c, a2.c


----

.output of snippet `class1`
[source]
----
12dd

----



Classes can contain methods and can define class variables.
In the example above the class contains the method `init`, which is a special one, and declares one variable, `c`.

The method `init` is the initializer method.
It is invoked when the class is instantiated.
To create an object, you should use the name of the class like a method with parameters.
The parameters will be passed to the `init` method.

When executing a method, there are two special variables:

* `this` is the current object.

* `cls` is the class of the current object.

If you use a variable without the `this.` prefix, it will be resolved automatically, but you can always use the `this.` prefix explicitly.
That way the following example is the same as the previous:

.source of snippet `class2`

[source]
----
class A {
    fn init {
        this.a = "a";
    }
    cls.c = "c"
}
mut a1 = A()
mut a2 = A()
a1.a = "1"
a2.a = "2"
A.c = "d"
println a1.a, a2.a, a1.c, a2.c


----

.output of snippet `class2`
[source]
----
12dd

----



When using variables in a class first the object variables are consulted.
If there is no object variable, then the class variables are used.
Classes and the methods inside the classes do not wrap any surrounding context.

The methods defined in a class belong to the class.
You can define methods attached to the objects.

.source of snippet `class3`

[source]
----
class A {
    fn init {
        this.a = "a";
    }
    fn p {
        println "I am class level"
    }
}
mut a1 = A()
a1.p()
a1.p = fn (){
  println "I am a1"
}
mut a2 = A()
a2.p = fn (){
  println "I am a2"
}

a1.p()
a2.p()


----

.output of snippet `class3`
[source]
----
I am class level
I am a1
I am a2

----



The variables `cls` and `this` are pinned and cannot be modified in methods.

.source of snippet `class4`

[source]
----
class A {
    fn p {
        this = none
    }
}
try :
    A().p()
catch ex :
    println ex

----

.output of snippet `class4`
[source]
----
Variable 'this' is pinned.

----



and

.source of snippet `class5`

[source]
----
class A {
    fn p {
        cls = none
    }
}
try :
    A().p()
catch ex :
    println ex

----

.output of snippet `class5`
[source]
----
Variable 'cls' is pinned.

----



The only exception is the `init` method.

.source of snippet `class6`

[source]
----
class A {
    fn init {
      this = B()
    }
}

class B {

}

println A().cls


----

.output of snippet `class6`
[source]
----
class B

----



The example above alters the `this` variable, and it alters what the construction of the class returns.
That way an initializer can function as a factory method.

You cannot change the value of the `cls` even in the initializer:

.source of snippet `class7`

[source]
----
class A {
    fn init {
      cls = B
    }
}

class B {

}
try:
    A()
catch ex:
    println ex


----

.output of snippet `class7`
[source]
----
Variable 'cls' is pinned.

----



Classes can be nested and defined inside other classes.
When calling the constructor of the class to create a new instance, the variable `this` would be ambiguous.

For `a.Z()` it usually points to `a`, but if `Z` is a class and we called the constructor, then `this` should be the new instance.
In this situation, a special variable called `that` will point to `a`.

.source of snippet `class8`

[source]
----
class A {
    class B {
        fn init {
            println "this ", this
            println "cls ", cls
            println "that ", that
        }
    }
}

mut a = A()
mut b = a.B()

----

.output of snippet `class8`
[source]
----
this {that: {this: {...}, cls: class A}, this: {...}, cls: class B}
cls class B
that {this: {...}, cls: class A}

----



=== Class inheritance

Classes support inheritance in Turicum.
When you declare a class, the parent class or classes are specified by the name after a `:`.

.source of snippet `inherit1`

[source]
----
class P {
    fn a()=1
    mut v = 2
}
class C : P {
    fn b()=3
    mut z = 4
}
mut o = C();
println o.a(),o.v,o.b(),o.z;

----

.output of snippet `inherit1`
[source]
----
1234

----



You can use the methods or fields declared in the class, as well as any other methods and fields defined in the parent classes.
You can inherit from multiple parent classes:

.source of snippet `inherit2`

[source]
----
class P1 {
    fn a()=1
    mut v = 2
}
class P2 {
    fn aa()=3
    mut vv = 4
}
class C : P1,P2 {
    fn b()=5
    mut z = 6
}
mut o = C();
println o.a(), o.v, o.aa(), o.vv, o.b(), o.z;

----

.output of snippet `inherit2`
[source]
----
123456

----



Inheritance is ordered.
It is important when there are multiple implementations of the same method name or the same field.

.source of snippet `inherit3`

[source]
----
class P1 {
    fn a()=1
    mut v = 2
}
class P2 {
    fn a()=3
    mut v = 4
}
class C : P1,P2 {
    fn b()=5
    mut z = 6
}
mut o = C();
println o.a(), o.v, o.b(), o.z;

----

.output of snippet `inherit3`
[source]
----
1256

----



In the example `P1` is inherited/extended first in the list, so it prevails over `P2` when there is a name collision.

Initializing a class having parent classes is not trivial.
Since there can be, and in the example there are multiple parents, the initialization does not automatically invoke the initialization of the parent classes.

.source of snippet `inherit4`

[source]
----
class P1 {
    fn init {
        println "init in P1, cls='%s' this='%s'" % [cls, this]
    }
}
class P2 {
    fn init {
        println "init in P2, cls='%s' this='%s'" % [cls, this]
    }
}
class C : P1,P2 {
    fn init {
        println "init in C, cls='%s' this='%s'" % [cls, this]
    }
}
mut o = C();

----

.output of snippet `inherit4`
[source]
----
init in C, cls='class C' this='{this: {...}, cls: class C}'

----



You can invoke them directly:

.source of snippet `inherit5`

[source]
----
class P1 {
    fn init {
        println "init in P1, cls='%s' this='%s'" % [cls, this]
    }
}
class P2 {
    fn init {
        println "init in P2, cls='%s' this='%s'" % [cls, this]
    }
}
class C : P1,P2 {
    fn init {
        println "init in C, cls='%s' this='%s'" % [cls, this]
        P1.init()
        P2.init()
    }
}
mut o = C();

----

.output of snippet `inherit5`
[source]
----
init in C, cls='class C' this='{this: {...}, cls: class C}'
init in P1, cls='class P1' this='{this: {...}, cls: class C}'
init in P2, cls='class P2' this='{this: {...}, cls: class C}'

----



If a class does not implement `init` but some of the parent classes do, then it will be invoked.

.source of snippet `inherit6`

[source]
----
class P {
    fn init {
        println "init in P, cls='%s' this='%s'" % [cls, this]
    }
}
class C : P {
}
mut o = C();

----

.output of snippet `inherit6`
[source]
----
init in P, cls='class C' this='{this: {...}, cls: class C}'

----



Note the difference in the value of the field `cls`.
When the parent class `init` is invoked directly from the child `init` the `cls` points to the parent.
When it is inherited, the `init` is executed in the context of the child class; it is simply inherited, hence the value of the field `cls` points to the child class.

When a class has parent class or classes, the class methods and fields are inherited, but there is only one single `this` object.
There is a single set of object fields.
Object fields do not shadow each other, because there is only one field for a given name.

.source of snippet `inherit7`

[source]
----
class P {
    fn init(a,b);
}
class C : P {
    fn init(b,c){
        P.init(3,4)
    }
}

let object = C(1,2);
print object

----

.output of snippet `inherit7`
[source]
----
{a: 3, b: 4, c: 2, this: {...}, cls: class C}
----



In the example `b` is `4`, even though it is initialized to be `1` when calling the initializer of `C` but it is overwritten when the `P` initializer is invoked.

There is no restriction on which other class initializers an initializer can invoke.
Your code could theoretically call the initializer of a class that the current class does not extend.

The object fields are created when the initializers are invoked.
The type declaration of a parent class may not be the same for a field as in the child class.
In this case the type definition of the field will be the union of the types.

.source of snippet `inherit8`

[source]
----
class Parent {
    fn init(b:str){
        }
}
class Child : Parent {
    fn init(b:num){
        Parent.init("mutabor")
        this
    }
}

let k = Child(42);
try: k.b = none; // will fail as it is num and str, which
catch e: println e; // we should see in the error

----

.output of snippet `inherit8`
[source]
----
Cannot set variable 'b' to value 'none' because it does not fit any of the declared types of the variable (num|str)

----



The field `b` is declared to be a number in the class `Child` but it also inherits the type `str` through the invocation of the initializer of `Parent`.

=== Veiled Fields


The `veil` command makes fields inaccessible from the outside:

[source]
----
class Account {
    fn init(opening) {
        mut balance = opening
        veil balance                    // instance field, veiled per object
    }
    fn deposit(d) { balance = balance + d }
    fn helper() { 1 }
    veil helper                         // class level method
}
----

A veiled name cannot be read or written through the field access of the object or the class
(`a.balance`, `a["balance"]`, calling `a.helper()`), the access reports an error.
Inside the methods of the object, the name is used as a variable the usual way; the veil does not
restrict the lexical name resolution.
Methods of a subclass also see the veiled names of the parent.

The veil is a guardrail, not a security boundary.
The `with` command and macros see through it deliberately.
For example, a method comparing two instances can read the veiled field of the other instance:

[source]
----
fn equals(other) { v == { with other : v } }
----

The names must be defined before they are veiled.
The `keys()` function does not list veiled names; `keys_all()` lists all of them.
Do not veil `init`, `entry`, `exit`, `to_string`, or operator methods: they are invoked through
the field access, and veiling them makes the object unusable for the corresponding purpose.



.source of snippet `veil_fields`

[source]
----
// The 'veil' command makes fields and methods inaccessible from the outside. Inside the
// methods of the object the veiled names are used as variables the usual way.
class Account {
    fn init(opening) {
        mut balance = opening
        veil balance
    }
    fn deposit(d) { balance = balance + d }
    fn total() { balance }
    fn helper() { "internal" }
    fn front() { helper() }
    veil helper
}

let a = Account(100)
a.deposit(50)
println "total: " + a.total()

// veiled fields and methods are not accessible from the outside
try {
    a.balance
} catch e {
    println "caught: " + e.message
}
try {
    a.helper()
} catch e {
    println "caught: " + e.message
}
// but methods of the object can use them
println "front: " + a.front()

// keys() does not list veiled names, keys_all() does
mut in_keys = false
for each k in keys(a) : if k == "balance" : in_keys = true
mut in_keys_all = false
for each k in keys_all(a) : if k == "balance" : in_keys_all = true
println "balance in keys: " + in_keys
println "balance in keys_all: " + in_keys_all

// the veil is a guardrail: 'with' sees through it deliberately, which is how a method
// can compare its veiled field with the one of another instance
class Box {
    fn init(v0) {
        mut v = v0
        veil v
    }
    fn same(other) { v == { with other : v } }
}
println "same: " + Box(42).same(Box(42))
println "same: " + Box(42).same(Box(43))

----

.output of snippet `veil_fields`
[source]
----
total: 150
caught: Field 'balance' is veiled.
caught: Field 'helper' is veiled.
front: internal
balance in keys: false
balance in keys_all: true
same: true
same: false

----



== Error Handling

Error handling is similar to Java.
If an error happens, an exception is thrown.
An exception can be caught using `catch` pairing with a `try`.

An exception is an object.
You can declare a variable to hold an error.

.source of snippet `try3`

[source]
----
mut error:err|none
try{
        z = 55 // undefined variable
}catch x : error = x

println error


----

.output of snippet `try3`
[source]
----
Variable 'z' is undefined.

----



The predefined type for an exception is `err`.

An exception has

* `stack_trace`

* `message`

* `cause`, and

* `suppressed` fields.

`stack_trace` is a list (`lst`), each element is an object with the fields:

* `file`
* `line`
* `column`
* `source`

.source of snippet `try4`

[source]
----
mut error: err | none
try{
    z= 55// undefined variable
} catch err: error=err;

println """message "%s" """ % [ error.message ]
println """cause "%s" """ % [ error.cause ]
println """suppressed %s" """ % [ error.suppressed ]
for each st in error.stack_trace {
    println("%s:%s:%s" % [ st.file, st.line, st.column ] )
    println "source: ", st.source
}



----

.output of snippet `try4`
[source]
----
message "Variable 'z' is undefined." 
cause "null" 
suppressed []" 
try4.turi:3:4
source:     z= 55// undefined variable
try4.turi:2:3
source: try{
try4.turi:2:0
source: try{
try4.turi:1:0
source: mut error: err | none

----



=== Try-Catch

Try is very similar to Java `try`-`catch`-`finally`.
The command `try` tries to execute a block.
If an exception happens, it executes the block following the keyword `catch`.
The syntax of the command is

[source]
----
try ( [':'] COMMAND | BLOCK ) [catch [exception_variable] ([':'] COMMAND | BLOCK)] [finally ([':'] COMMAND | BLOCK)]
----

The command is executed in the same context as the whole try-catch command.
If, however, a block between `+{+` and `+}+` is used, it will create its own context.
The use of the `:` is optional and is also discouraged as clutter.
It is provided to let it have the same structure as the conditional and loop commands.

If an exception variable is specified, it will be defined when the exception is caught.
This is equivalent to defining a variable using the `let` command.
It will retain the exception's value during execution of the catch block.

The exception variable can be defined as `catch(e)` with parentheses to make it more familiar to Java developers.
Using the extra parentheses is not idiomatic in Turicum.

If an exception happens, the execution of the try command stops, the exception variable will get the exception object, and the catch block starts to execute.
The exception variable is not available during the execution of the try command.

The exception variable is pinned and cannot be modified.

The value of the try-catch command is

* the value of the try command if there was no exception, and
* the value of the catch command if there was an exception.

The command after the `finally` keyword does not affect the result of the command.
It is an error to execute a `return`, `break` or `continue` inside the `finally` block.

[[cleanup-after-halt]]
==== Cleanup After a Halt (Step Limit / Abort)

Two things can stop a Turicum program in a way that ordinary Turicum code cannot catch, suppress, or delay: the *step limit* being reached (see the `steps` option of `async`, above, and the embedder-configured interpreter-wide limit), and an *abort* requested from outside the program (for example, a supervising thread cancelling a task).
Both are enforced at the language boundary, deliberately bypassing `try`/`catch` entirely — a program running under a step limit must not be able to catch the limit and keep going, since that would defeat the whole point of using it as a sandboxing tool.

By default, this means a `finally` block (or a `with` block's `exit()` method) does not get a chance to do anything when one of these events happens: its very first statement hits the same limit and stops immediately, before it can, for example, close a file handle or release a lock.
The one exception is `mutex`/`sync`, whose release is implemented directly in the host language below the interpreter loop, and is therefore unaffected by any of this.

An embedder can opt in to a bounded *cleanup grace* period: a small, fixed number of extra steps that a `finally`/`exit()` block is allowed to spend releasing resources after a halt, before the halt becomes final.
This is configured on the Java side, not from Turicum source, since it is a property of how the interpreter is embedded rather than of the language itself:

[source,java]
----
// stepLimit:  the normal execution budget
// graceSteps: extra steps a finally/exit block may spend cleaning up after a halt fires
LocalContext ctx = new LocalContext(stepLimit, graceSteps);
----

With `graceSteps` left at its default of `0`, behavior is unchanged: cleanup blocks never run after a halt.
Configuring it grants a strictly bounded window, with three guarantees that keep it from reopening the hole the step limit exists to close:

* The grace budget is spent once, across the *whole* unwind — not once per `try`/`finally` or `with` block.
If a `finally` is nested inside another `finally`, the two share the same allowance; the outer one does not get a fresh window just because the inner one already used its share.

* Once the budget runs out, the halt becomes final and unconditional: no further Turicum code runs at all, including any other cleanup block still waiting its turn.
There is no way to be granted a second grace period.

* A `finally` or `exit()` cannot suppress a halt.
`exit()` returning `true` normally tells `with` to swallow an exception; during a halt this has no effect at all — the halt propagates regardless of what the cleanup code returns.

The effective ceiling on a program's execution is therefore `stepLimit + graceSteps`, fixed and known in advance — the same shape as the SIGTERM-then-SIGKILL pattern used by process supervisors: the halt fires, a bounded interval is granted for a clean shutdown, and if that is not enough, termination happens unconditionally.

== Decorators, Metaprogramming

Decorators are an inherent part of Turicum.
To make them similar to Python decorators and Java annotations, they can have a `@` in front of the decorator function call.
However, in the case of closures, you do not even need that.
Every function call that has a closure as the last parameter can act as a decorator.

.source of snippet `closureDecorator1`

[source]
----
fn closure_decorator(^closure){
    {|| println("We start"); closure(); println("We finish")}
  }

  mut z = closure_decorator(){|| println("We are closure") }

  z()

----

.output of snippet `closureDecorator1`
[source]
----
We start
We are closure
We finish

----



If you add a `@` then the result is the same:

.source of snippet `closureDecorator2`

[source]
----
fn closure_decorator(^closure){
    {|| println("We start@"); closure(); println("We finish@")}
  }

  mut z = @closure_decorator(){|| println("We are closure") }

  z()

----

.output of snippet `closureDecorator2`
[source]
----
We start@
We are closure
We finish@

----



The `@` helps you emphasize that the call is a decorator call that modifies the closure following it.
The use of `@` also notifies the syntax analyzer that the call is a decorator call, and it extends the recognition of the following `^closure` parameter.
When the `@` precedes the function call, a `fn` or `class` are also recognized as `^closure` parameter, and you do not need to enclose them between `+{+` and `+}+`.

.source of snippet `fnDecorator1`

[source]
----
fn fun_decorator(^closure){
    {|| println("We start the fun"); closure(); println("The fun is over")}
  }

mut z =
@fun_decorator()
fn q(){
  println("We function");
}
  z()

----

.output of snippet `fnDecorator1`
[source]
----
We start the fun
We function
The fun is over

----



Without the `@` the program is still valid.
However, as soon as we try to invoke `z()` the program fails.
It tries to invoke the closure returned by `fun_decorator()`, which tries to call `closure()` which now has the value `none`.
This happens because the syntax analysis does not use the function definition as a parameter.

The code can, however, be fixed not only by using `@` but also by passing the function definition as a regular argument between `(` and `)`:

.source of snippet `fnDecorator2`

[source]
----
fn fun_decorator(^closure){
    {|| println("We start the fun"); closure(); println("The fun is over")}
  }

mut z =
fun_decorator(
fn q(){
  println("We function");
})
  z()

----

.output of snippet `fnDecorator2`
[source]
----
We start the fun
We function
The fun is over

----



The application of the `@` is therefore only a syntactic sugar.
The following example shows that a decorator cannot only return the decorated function, class, or closure.
They can also modify an existing definition.

.source of snippet `fnDecorator3`

[source]
----
mut fun_decorator={|^closure|
        set_force(closure.name,
            {|| println("We start the fun"); closure(); println("The fun is over")}
           )
    }

@fun_decorator()
fn q(){
  println("We function");
}

q()

----

.output of snippet `fnDecorator3`
[source]
----
We start the fun
We function
The fun is over

----



Technically the decorator does not modify the function `q`.
It creates a new function and redefines the variable named `q` in the wrapped context.

Note that if `fun_decorator` were a function declared with `fn`, it would not work.
A function does not wrap the surrounding context and thus cannot redefine anything in it.
To redefine an already existing variable, it needs to be a closure.

You usually just assign a value to the variable updating its value.
In this case we have to use the built-in `set` function, because we have the name of the variable as a string.

In the following example we moved the function definition to a local context.
The decorator in this case simply defines the global variable of the same name as we used locally for the function.
The value is the closure or function itself, whatever we decorate.

.source of snippet `fnDecorator4`

[source]
----
{
    fn globalize(^closure){
            set_global(closure.name,closure)
        }

    @globalize()
    fn q(){
      println("We function");
    }
}
q()

----

.output of snippet `fnDecorator4`
[source]
----
We function

----



Since we use `set_global` the decorator can be a function, it does not need access to the enclosing context.

=== Macro and Reclose

The built-in functions `macro` and `reclose` are special closure conversion functions.

The function `macro()` accepts a closure as an argument and returns the closure converted to a macro.
The major difference between macros and closures is that the arguments of a closure are evaluated before the call.
A macro gets its arguments unevaluated.

=== Special variables

There are a few predefined variables, constants, and some special variables.

* `true`
 has the constant boolean __true__ value.
* `false`
 has the constant boolean __false__ value.

* `none`
 is the undefined value.
 The Java representation of the undefined value is `null`.

* `inf`
 is the infinite numeric value.

* `fini`
 is the special value that, when returned from a cell command, prevents the cell from updating,
 but it also signals that this cell is stopped, should not be evaluated any further

* `non_mutat`
 is the special value that, when returned from a cell command, prevents the cell from updating
 its value and halts propagation to dependent cells in a flow.

* `nan`
 is the special IEEE-754 * `not a number* ` floating-point value.



The special variables are not constants.
They are defined dynamically to represent the current computing environment.
These are the following:

* `this`

* `cls`

* `that` will point to the object on which a method was called when the method is a class constructor.
Class constructors are invoked using the name of the class.
When a class name is used as a field in an object either directly injected to the object or as a class field as in the next example, then the constructor can access the target object of the call through `that`.
+
.source of snippet `that`

[source]
----
class OuterClass {
    mut name = "outer class";
    fn init(){
      mut name = "outer object"
    }
    class InnerClass {
        mut name = "inner class";
        fn init() {
            mut name = "inner object"
            println("this name %s " % this.name)
            println("this name %s " % cls.name)
            println("this name %s " % that.name)
            println("this name %s " % that.cls.name)
            }
        }
    }
mut outer = OuterClass()
mut inner = outer.InnerClass()

----

+ .output of snippet `that`
[source]
----
this name inner object 
this name inner class 
this name outer object 
this name outer class 

----


+
Usually the target object is called `this`, but in the case of the constructor `this` points to the current object to be initialized.
What is more, the constructor, always named `init()`, is allowed to modify the variable `this`.
In this environment `this` cannot be used for both the object to be initialized and the target object.
Hence, the special variable `that` was introduced.
+
You will hardly ever need it.

* `me` it always points to the currently executing function or closure.
You can use it to make recursive calls.
This is necessary because a function defined in a local context does not capture the surrounding context.
For more, read the next section.

=== Function scopes and `me`

Functions are usually defined in the global context or as methods inside classes.
In both cases there is a wrapped context that contains the name of the function defined.
It is either the global context or the class context.

The following example is very straightforward.
It defines the function `fib()`, that calls itself recursively.

.source of snippet `fibonacci1`

[source]
----
fn fib(x) {
    if x == 1 || x == 0 : 1
    else: fib(x - 1) + fib(x - 2)
}
println(fib(10))

----

.output of snippet `fibonacci1`
[source]
----
89

----



If we create the same program using a class memoizing the results:

.source of snippet `fibonacci2`

[source]
----
class FibCalculator {
    mut cache = [];
    fn fib(x) {
        if x == 1 || x == 0 : 1
        else{
            return cache[x] when cache[x] != none;
            cache[x] = fib(x - 1) + fib(x - 2);
        }
    }
}
println(FibCalculator.fib(10))

----

.output of snippet `fibonacci2`
[source]
----
89

----



we get the same result and there is no problem with that.
In this case `fib()` is a class level method, and when we call it Turicum automatically calls it as a method.

The difficulty comes when you define the function in a local context.

.source of snippet `fibonacci3`

[source]
----
try
{
    fn fib(x) {
        if x == 1 || x == 0 : 1
        else: fib(x - 1) + fib(x - 2)
    }
    println(fib(10))
}catch ex : println(ex)

----

.output of snippet `fibonacci3`
[source]
----
Variable 'fib' is undefined.

----



The function does not see itself.
The function `fib` in this case is defined in the local context of the block, and functions do not wrap the surrounding context.
One solution can be to convert the function to a closure:

.source of snippet `fibonacci4`

[source]
----
{
    mut fib={|x|
        if x == 1 || x == 0 : 1
        else: fib(x - 1) + fib(x - 2)
    }
    println(fib(10))
}

----

.output of snippet `fibonacci4`
[source]
----
89

----



Closures wrap the surrounding context, therefore, the body of the closure knows the value of `fib`, which is the closure itself.
If you do not want to wrap the whole context, you can do a little trick using functions.

.source of snippet `fibonacci5`

[source]
----
{
    fn fib(x,fib=fib) {
        if x == 1 || x == 0 : 1
        else: fib(x - 1) + fib(x - 2)
    }
    println(fib(10))
}

----

.output of snippet `fibonacci5`
[source]
----
89

----



The function `fib` has an extra optional parameter.
The left side of the `fib=fib` is the name of the parameter, which will become a variable in the context of the function.
The right side is the default value, which is evaluated in the caller context before the call.

One can like this approach, for explicitly declaring that `fib` needs access to `fib` hence a recursive call.
Others can find it cryptic, and it is not really clear without knowing the details how contexts and default values for variable declarations work in Turicum.

An alternative possibility is to use the special variable `me`.

.source of snippet `fibonacci6`

[source]
----
{
    fn fib(x) {
        if x == 1 || x == 0 : 1
        else: me(x - 1) + me(x - 2)
    }
    println(fib(10))
}

----

.output of snippet `fibonacci6`
[source]
----
89

----



This variable is reserved for this purpose, and it gets the function itself as its value before a function is called.
It can also be appealing that the recursive call is explicit whatever the name of the function is, not needing renaming when the function is renamed.

As an extra, here is a version of the fibonacci calculation using a closure with a retained local context as a state instead of a class:

.source of snippet `fibonacci7`

[source]
----
mut fib = {
    mut cache = [];
    {|x|
        if x == 1 || x == 0 : 1
        else{
            return cache[x] when cache[x] != none;
            cache[x] = me(x - 1) + me(x - 2);
        }
    }
}
println(fib(10))

----

.output of snippet `fibonacci7`
[source]
----
89

----



Finally, here is a modified fibonacci calculation that can replace the start value and uses an object cache instead of class level cache.

.source of snippet `fibonacci8`

[source]
----
class FibCalculator {
    fn init(start=1) {
        mut cache = [];
    }
    fn fib(x) {
        if x == 1 || x == 0 : start
        else{
            return cache[x] when cache[x] != none;
            cache[x] = fib(x - 1) + fib(x - 2);
        }
    }
}
println(FibCalculator().fib(10))
println(FibCalculator(2).fib(10))

----

.output of snippet `fibonacci8`
[source]
----
89
178

----



== String Functions

Turicum defines many methods to handle strings.
These methods can be called on the string objects as methods.




            * `to_string` 
 returns the string itself. This method exists so that you can use it like on any other objects.
            * `length` 
 a convenience method that makes it possible to use it like in Java.
 The Turicum way is to use `len()` built-in function, but this method also works on strings.
 
+
.source of snippet `string_length`
[source]
----
print len("abraka dabra") == "abraka debra".length()

----
+
.output of snippet `string_length`
[source]
----
true
----


            * `after` 
 returns the part of the string that follows the argument string.
 If the argument string is not found, then it returns an empty string.
 
+
.source of snippet `string_after`
[source]
----
println "abraka badraka".after("ra")

----
+
.output of snippet `string_after`
[source]
----
ka badraka

----


            * `after_last` 
 returns the part of the string that follows the last occurrence of the argument string.
 If the argument string is not found, then it returns an empty string.
 
+
.source of snippet `string_after_last`
[source]
----
println "abraka badraka".after_last("ra")

----
+
.output of snippet `string_after_last`
[source]
----
ka

----


            * `before` 
 returns the part of the string that is before the argument string.
 If the argument string is not in the string, then it returns an empty string.
 
+
.source of snippet `string_before`
[source]
----
println "abraka badraka".before("ra")

----
+
.output of snippet `string_before`
[source]
----
ab

----


            * `before_last` 
 returns the part of the string that is before the argument string.
 If the argument string is not in the string, then it returns an empty string.
 
+
.source of snippet `string_before_last`
[source]
----
println "abraka badraka".before_last("ra")

----
+
.output of snippet `string_before_last`
[source]
----
abraka bad

----


            * `pad_left` 
 add characters on the left of the string to get the desired length.
 You can specify only the number of characters to pad with spaces.
 Alternatively, you can also specify a string and the desired total length of the result.
 The string is supposed to be a single character, but it can be any length.
 If the necessary number of padding characters is not the multiple of string length, the resulting string may be shorter than desired.
 
+
.source of snippet `string_pad_left`
[source]
----
println "avalanche".pad_left(15)
println "avalanche".pad_left(15,".")

----
+
.output of snippet `string_pad_left`
[source]
----
      avalanche
......avalanche

----


            * `pad_right` 
 add character to the right of the string to get the desired length.
 This method works the same way as `pad_left`, but it appends the characters.
 
+
.source of snippet `string_pad_right`
[source]
----
println "avalanche".pad_right(15),"<<"
println "avalanche".pad_right(15,"."),"<<"

----
+
.output of snippet `string_pad_right`
[source]
----
avalanche      <<
avalanche......<<

----


            * `between` 
 returns the part of the string that is between two specific strings.
 if one of the parameter strings is not found in the string, the result is an empty string.
 
+
.source of snippet `string_between`
[source]
----
println "alma ata".between("al","ta")
// is the same as
println "alma ata".after("al").before("al","ta")


----
+
.output of snippet `string_between`
[source]
----
ma a


----


            * `lines` 
 returns a list containing the lines of the string.
 
+
.source of snippet `string_lines`
[source]
----
println jsonify("alma\na\nta".lines())

----
+
.output of snippet `string_lines`
[source]
----
["alma","a","ta"]

----


            * `words` 
 returns a list containing the words of the string.
 
+
.source of snippet `string_words`
[source]
----
println jsonify("alma ata".words())

----
+
.output of snippet `string_words`
[source]
----
["alma","ata"]

----


            * `turi_lex` 
 returns a list containing the tokens of the string matching the tokenization of the language (Turicum).
 
+
.source of snippet `string_turi_lex`
[source]
----
println "for each i in range(1,55) : println i".turi_lex()

----
+
.output of snippet `string_turi_lex`
[source]
----
[Lex{type=RESERVED, text='for', atLineStart=false}, Lex{type=RESERVED, text='each', atLineStart=false}, Lex{type=IDENTIFIER, text='i', atLineStart=false}, Lex{type=RESERVED, text='in', atLineStart=false}, Lex{type=IDENTIFIER, text='range', atLineStart=false}, Lex{type=RESERVED, text='(', atLineStart=false}, Lex{type=INTEGER, text='1', atLineStart=false}, Lex{type=RESERVED, text=',', atLineStart=false}, Lex{type=INTEGER, text='55', atLineStart=false}, Lex{type=RESERVED, text=')', atLineStart=false}, Lex{type=RESERVED, text=':', atLineStart=false}, Lex{type=RESERVED, text='println', atLineStart=false}, Lex{type=IDENTIFIER, text='i', atLineStart=false}]

----


            * `execute` 
 executes the string as Turicum code.
 
+
.source of snippet `string_execute`
[source]
----
sys_import turi.range;

mut j = 3;
println "for each i in range(1,3) { println i+j; j ++ }".execute()
println j;

----
+
.output of snippet `string_execute`
[source]
----
4
6
4
5

----


            * `compile` 
Compile a string and return the compiled code as a program.
You can use the returned object the same way as you would use a macro argument, or the return value of `thunk`.
The block is wrapping the commands.
It means that the commands are executed in a local context when you call `unthunk` and not in the context where the `compile` was used.
If the code you compile is a code block, you can use the function `unwrap` to unwrap it.
+
A straightforward example of the use is:

+
.source of snippet `compile1`
[source]
----
mut z = "println 1+1".compile()
z.execute()

----
+
.output of snippet `compile1`
[source]
----
2

----


+
The following example demonstrates that the block is wrapping.
The variable `x` not defined after the execution of the block.

+
.source of snippet `compile2`
[source]
----
// the program contains one command, which is a block
mut z = "{let x = 1}".compile()[0] // get the block
z.execute()
print is_defined(x)

----
+
.output of snippet `compile2`
[source]
----
false
----


+
In the following example, we `unwrap` the compilation result.
Executing the block will define the variable `x`.

+
.source of snippet `compile3`
[source]
----
mut z = "{let x = 1}".compile()[0]
unwrap(z).execute()
print is_defined(x)," and it is ", x

----
+
.output of snippet `compile3`
[source]
----
true and it is 1
----


            * `url_encode` 
 encode the string for URL
 
+
.source of snippet `string_url_encode`
[source]
----
println "63/2%3 =55".url_encode()

----
+
.output of snippet `string_url_encode`
[source]
----
63%2F2%253+%3D55

----


            * `url_decode` 
 decode the string from URL-encoded form
 
+
.source of snippet `string_url_decode`
[source]
----
println "63%2F2%253+%3D55".url_decode()

----
+
.output of snippet `string_url_decode`
[source]
----
63/2%3 =55

----


            * `md5` 
 calculate the md5 hash of the string
 
+
.source of snippet `string_md5`
[source]
----
println "borra sor meggyotor".md5()

----
+
.output of snippet `string_md5`
[source]
----
1a356faf32bae1cabee0198ecc072edd

----


            * `sha_1` 
 calculate the sha-1 hash of the string
 
+
.source of snippet `string_sha_1`
[source]
----
println "borra sor meggyotor".sha_1()

----
+
.output of snippet `string_sha_1`
[source]
----
b6c3368549b65c29ee4d90d0e972c49422c8d0ee

----


            * `sha_256` 
 calculate the sha-256 hash of the string
 
+
.source of snippet `string_sha_256`
[source]
----
println "borra sor meggyotor".sha_256()

----
+
.output of snippet `string_sha_256`
[source]
----
4b1b6e48f60bc62bd7c876d264e784e752fc1ea2b540fec897b1ba4a2f8f9d19

----


            * `sha_512` 
 calculate the sha-512 hash of the string
 
+
.source of snippet `string_sha_512`
[source]
----
println "borra sor meggyotor".sha_512()

----
+
.output of snippet `string_sha_512`
[source]
----
9ae2d8cdfa063385b3c630a2065e8d9562305ef31c41e91b8d3671fdaa9384f718c7722bf0e0b0e40c8aea0caaa547d49a3fe3da8818f3c17b376710a7319149

----


            * `digest` 
 calculate the hash of the string. The algorithm name has to be provided as an argument.
 
+
.source of snippet `string_digest`
[source]
----
println "borra sor meggyotor".digest("sha-512")

----
+
.output of snippet `string_digest`
[source]
----
9ae2d8cdfa063385b3c630a2065e8d9562305ef31c41e91b8d3671fdaa9384f718c7722bf0e0b0e40c8aea0caaa547d49a3fe3da8818f3c17b376710a7319149

----


            * `base64` 
 base64 encode the string
 
+
.source of snippet `string_base64`
[source]
----
println "borra sor meggyotor".base64()

----
+
.output of snippet `string_base64`
[source]
----
Ym9ycmEgc29yIG1lZ2d5b3Rvcg==

----


            * `from_base64` 
 create a bin (byte array) from a base64 encoded string
 
+
.source of snippet `string_from_base64`
[source]
----
println "Ym9ycmEgc29yIG1lZ2d5b3Rvcg==".from_base64()

----
+
.output of snippet `string_from_base64`
[source]
----
bin"626f72726120736f72206d656767796f746f72"

----


            * `from_hex` 
 create a bin (byte array) from a hexadecimal string.
 An optional `0x` prefix is accepted, and the number of digits has to be even.
 
+
.source of snippet `string_from_hex`
[source]
----
println "48656c6c6f".from_hex()
println "0x48656C6C6F".from_hex().text()

----
+
.output of snippet `string_from_hex`
[source]
----
bin"48656c6c6f"
Hello

----


            * `from_base64_str` 
 create a string from a base64 encoded string
 
+
.source of snippet `string_from_base64_str`
[source]
----
println "Ym9ycmEgc29yIG1lZ2d5b3Rvcg==".from_base64_str()

----
+
.output of snippet `string_from_base64_str`
[source]
----
borra sor meggyotor

----


            * `contains` 
 returns `true` if the string contains the argument.
 
+
.source of snippet `string_contains`
[source]
----
println "Ym9ymcAgc29yIG1lZ2d5b3Rvcg==".contains("ymcA")
println "Ym9ymcAgc29yIG1lZ2d5b3Rvcg==".contains("bora nora")

----
+
.output of snippet `string_contains`
[source]
----
true
false

----


            * `contains_regex` 
 returns `true` if the string contains some substring that matches the argument as a regular expression.
 
+
.source of snippet `string_contains_regex`
[source]
----
println "Ym9ymcAgc29yIG1lZ2d5b3Rvcg==".contains_regex("IG.*cg")
println "Ym9ymcAgc29yIG1lZ2d5b3Rvcg==".contains_regex("YmZ.*?nora")

----
+
.output of snippet `string_contains_regex`
[source]
----
true
false

----


            * `matches` 
 returns `true` if the string matches the argument as a regular expression.
 
+
.source of snippet `string_matches`
[source]
----
println "Ym9ymcAgc29yIG1lZ2d5b3Rvcg==".matches("Ym.*cg..")
println "Ym9ymcAgc29yIG1lZ2d5b3Rvcg==".matches("ym.*cA")

----
+
.output of snippet `string_matches`
[source]
----
true
false

----


            * `matches_glob` 
 returns `true` if the string matches the argument as a glob string (`pass:[*]` matching any substring except `/`,
 `**` macthes any substring, and `?` matching any single character).
 
+
.source of snippet `string_matches_glob`
[source]
----
println "my_precious_text_file.txt".matches_glob("*.txt")
println "my_precious_text_file.txt".matches_glob("*.turx")

----
+
.output of snippet `string_matches_glob`
[source]
----
true
false

----


            * `is_blank` 
 returns `true` if the string is blank
 
+
.source of snippet `string_is_blank`
[source]
----
println "\n\t  ".is_blank()
println "\n\t  wupps".is_blank()

----
+
.output of snippet `string_is_blank`
[source]
----
true
false

----


            * `is_not_blank` 
 returns `false` if the string is blank
 
+
.source of snippet `string_is_not_blank`
[source]
----
println "\n\t  ".is_not_blank()
println "\n\t  wupps".is_not_blank()

----
+
.output of snippet `string_is_not_blank`
[source]
----
false
true

----


            * `is_empty` 
 returns `true` if the string is empty
 
+
.source of snippet `string_is_empty`
[source]
----
println "".is_empty()
println "\n\t  ".is_empty()

----
+
.output of snippet `string_is_empty`
[source]
----
true
false

----


            * `is_not_empty` 
 returns `false` if the string is empty
 
+
.source of snippet `string_is_not_empty`
[source]
----
println "".is_not_empty()
println "\n\t  ".is_not_empty()

----
+
.output of snippet `string_is_not_empty`
[source]
----
false
true

----


            * `is_numeric` 
 returns `true` if the string holds a decimal numeric value, either integer or floating point formatted.
 
+
.source of snippet `string_is_numeric`
[source]
----
println "653".is_numeric()
println "6E23".is_numeric()
println "3.1425923".is_numeric()
println "10.13.55.2".is_numeric()

----
+
.output of snippet `string_is_numeric`
[source]
----
true
true
true
false

----


            * `is_not_numeric` 
 returns `false` if the string holds a decimal numeric value, either integer or floating point formatted.
 
+
.source of snippet `string_is_not_numeric`
[source]
----
println "653".is_not_numeric()
println "6E23".is_not_numeric()
println "3.1425923".is_not_numeric()
println "10.13.55.2".is_not_numeric()

----
+
.output of snippet `string_is_not_numeric`
[source]
----
false
false
false
true

----


            * `is_digit` 
 returns `true` if the string contains only digits
 
+
.source of snippet `string_is_digit`
[source]
----
println "653".is_digit()
println "6E23".is_digit()
println "3.1425923".is_digit()
println "10.13.55.2".is_digit()

----
+
.output of snippet `string_is_digit`
[source]
----
true
false
false
false

----


            * `is_not_digit` 
 returns `false` if the string contains only digits
 
+
.source of snippet `string_is_not_digit`
[source]
----
println "653".is_not_digit()
println "6E23".is_not_digit()
println "3.1425923".is_not_digit()
println "10.13.55.2".is_not_digit()

----
+
.output of snippet `string_is_not_digit`
[source]
----
false
true
true
true

----


            * `is_alpha` 
 returns `true` if the string contains only alpha characters
 
+
.source of snippet `string_is_alpha`
[source]
----
println "ABC".is_alpha()
println "ABC DEF".is_alpha()
println "ABCDEF13".is_alpha()

----
+
.output of snippet `string_is_alpha`
[source]
----
true
false
false

----


            * `is_not_alpha` 
 returns `false` if the string contains only alpha characters
 
+
.source of snippet `string_is_not_alpha`
[source]
----
println "ABC".is_not_alpha()
println "ABC DEF".is_not_alpha()
println "ABCDEF13".is_not_alpha()

----
+
.output of snippet `string_is_not_alpha`
[source]
----
false
true
true

----


            * `is_alphanumeric` 
 returns `true` if the string contains only alphanumeric characters
 
+
.source of snippet `string_is_alphanumeric`
[source]
----
println "ABC".is_alphanumeric()
println "ABC DEF".is_alphanumeric()
println "ABCDEF13".is_alphanumeric()

----
+
.output of snippet `string_is_alphanumeric`
[source]
----
true
false
true

----


            * `is_not_alphanumeric` 
 returns `false` if the string contains only alphanumeric characters
 
+
.source of snippet `string_is_not_alphanumeric`
[source]
----
println "ABC".is_not_alphanumeric()
println "ABC DEF".is_not_alphanumeric()
println "ABCDEF13".is_not_alphanumeric()

----
+
.output of snippet `string_is_not_alphanumeric`
[source]
----
false
true
false

----


            * `is_hex` 
 returns `true` if the string contains only hexadecimal characters
 
+
.source of snippet `string_is_hex`
[source]
----
println "ABC".is_hex()
println "ABC DEF".is_hex()
println "ABCDEF13".is_hex()

----
+
.output of snippet `string_is_hex`
[source]
----
true
false
true

----


            * `is_not_hex` 
 returns `false` if the string contains only hexadecimal characters
 
+
.source of snippet `string_is_not_hex`
[source]
----
println "ABC".is_not_hex()
println "ABC DEF".is_not_hex()
println "ABCDEF13".is_not_hex()

----
+
.output of snippet `string_is_not_hex`
[source]
----
false
true
false

----


            * `int` 
 returns the integer value of the string
 
+
.source of snippet `string_int`
[source]
----
println "632".int()

----
+
.output of snippet `string_int`
[source]
----
632

----


            * `bool` 
 returns the boolean value of the string.
 The method works the same as the automatic conversion, and thus there is no need to use it
 when the string is in an environment that expects a boolean value.
 
+
.source of snippet `string_bool`
[source]
----
fn p(s:str){
    println s, " is ", s.bool()
}
p "true"
p "no"
p "" // empty string
p "nominative"
p "false"
p "0"
p "1"
p "1.0"
p "0.0"

----
+
.output of snippet `string_bool`
[source]
----
true is true
no is false
 is false
nominative is false
false is false
0 is false
1 is false
1.0 is false
0.0 is false

----


            * `big_int` 
 returns the BigInteger value of the string.
 
+
.source of snippet `string_bigi`
[source]
----
print "11111111111111111111111111111111111111111111111111111111111".big_int() +
      "11111111111111111111111111111111111111111111111111111111111".big_int()

----
+
.output of snippet `string_bigi`
[source]
----
22222222222222222222222222222222222222222222222222222222222
----


            * `big_decimal` 
 returns the BigDecimal value of the string.
 
+
.source of snippet `string_bigd`
[source]
----
print "0.11111111111111111111111111111111111111111111111111111111111".big_decimal() +
      "0.11111111111111111111111111111111111111111111111111111111111".big_decimal()

----
+
.output of snippet `string_bigd`
[source]
----
0.22222222222222222222222222222222222222222222222222222222222
----


            * `path` 
 returns the Java path object created from the string.
 
+
.source of snippet `string_path`
[source]
----
println type("C:\\Users\\user\\Documents\\my_file.txt".path())

----
+
.output of snippet `string_path`
[source]
----
java.sun.nio.fs.UnixPath

----


            * `uri` 
 returns the URI created from the string.
 
+
.source of snippet `string_uri`
[source]
----
println type("http://cnn.com".uri())

----
+
.output of snippet `string_uri`
[source]
----
java.java.net.URI

----


            * `url` 
 returns the URL created from the string.
 
+
.source of snippet `string_url`
[source]
----
println type("http://cnn.com".url())

----
+
.output of snippet `string_url`
[source]
----
java.java.net.URL

----


            * `float` 
 returns the floating number contained in the string
 
+
.source of snippet `string_float`
[source]
----
println "3.1415929".float()

----
+
.output of snippet `string_float`
[source]
----
3.1415929

----


            * `number` 
 returns the int number contained in the string encoded with the argument radix
 
+
.source of snippet `string_number`
[source]
----
println "61".number(7)

----
+
.output of snippet `string_number`
[source]
----
43

----


            * `hex` 
 returns the number contained in the string as a hexadecimal number.
 It is not the same as calling `"0AF6".number(16)` because this method respects the leading `0x` if there is any.
 
+
.source of snippet `string_hex`
[source]
----
println "2F".hex()
println "0x2F".hex()

----
+
.output of snippet `string_hex`
[source]
----
47
47

----


            * `substring` 
returns a substring of the string.
 There has to be at least one (`a`), and there can be at most two (`a`,`b`) integer arguments.
 The substring starts at the character indexed `a` and finishes before the character `b`.
 If there is no `b`, the substring lasts till the end of the string.
 The implementation honors overindexing and returns characters only to the end of the string or an empty string.
 
+
.source of snippet `string_substring`
[source]
----
println "alma ata".substring(5)
println "alma ata".substring(5,53)
println "alma ata".substring(5,7)
println "alma ata".substring(-3,7)

----
+
.output of snippet `string_substring`
[source]
----
ata
ata
at
at

----


            * `remove_prefix` 
 removes the argument from the start of the string if that is a prefix of the string.
 If the string does not start with the prefix, it just returns the original string.
 
+
.source of snippet `string_remove_prefix`
[source]
----
println "alma ata".remove_prefix("alma")
println "alma ata".remove_prefix("apple")

----
+
.output of snippet `string_remove_prefix`
[source]
----
 ata
alma ata

----


            * `remove_postfix` 
 removes the argument from the end of the string if that is a postfix of the string.
 If the string does not end with the postfix, it just returns the original string.
 
+
.source of snippet `string_remove_postfix`
[source]
----
println "alma ata".remove_postfix("ata")
println "alma ata".remove_postfix("alma")

----
+
.output of snippet `string_remove_postfix`
[source]
----
alma 
alma ata

----


            * `count_substring` 
 counts the occurrences of the argument string.
 Note that when an occurrence is found, the other occurrences are sought for after that occurrence.
 It means that, for example, `("a"*6).count_substring("aa")` is 3 and not 5.
 
+
.source of snippet `string_count_substring`
[source]
----
println "almal atal".count_substring("al")
println ("a"*6).count_substring("aa")

----
+
.output of snippet `string_count_substring`
[source]
----
3
3

----


            * `count_substring_overlap` 
 counts all the occurrences of the argument string, even if they overlap
 It means that, for example, `("a"*6).count_substring("aa")` is 5.
 
+
.source of snippet `string_count_substring_overlap`
[source]
----
println "almal atal".count_substring_overlap("al")
println ("a"*6).count_substring_overlap("aa")

----
+
.output of snippet `string_count_substring_overlap`
[source]
----
3
5

----


            * `left` 
 return the left of the string.
 This method uses the argument as a number and returns the string that contains at most that number of characters.
 Essentially, the `string.left(n)` is the `n` leftmost characters if `n` is smaller than the number of characters in the string.
 If `n` is equal to, or larger than the number of characters in the string, then the whole string is the result.
 
+
.source of snippet `string_left`
[source]
----
println "alma ata".left(4)

----
+
.output of snippet `string_left`
[source]
----
alma

----


            * `right` 
 return the right of the string.
 This method uses the argument as a number and returns the string that contains at most that number of characters.
 Essentially, the `string.right(n)` is the `n` rightmost characters if `n` is smaller than the number of characters in the string.
 If `n` is equal to, or larger than the number of characters in the string, then the whole string is the result.
 
+
.source of snippet `string_right`
[source]
----
println "alma ata".right(3)

----
+
.output of snippet `string_right`
[source]
----
ata

----


            * `replace_all` 
 replaces the occurrences of the first argument interpreted as regular expression with the second argument interpreted as string.
 
+
.source of snippet `string_replace_all`
[source]
----
println "alma ata".replace_all("a(.*?)a","b$1b")

----
+
.output of snippet `string_replace_all`
[source]
----
blmb btb

----


            * `quote` 
 quotes the special characters in the string.
 The result is a string that can be used in a string literal to represent the original string.
 
+
.source of snippet `string_quote`
[source]
----
let s= """z\t \\ a\n"a"""
let q = s.quote()
println $"${q} len q=${len(q)} len s=${len(s)}"

----
+
.output of snippet `string_quote`
[source]
----
z\t \\ a\n\"a len q=13 len s=9

----


            * `reverse` 
 returns the string that contains the same characters as the original string, but in reverse order.
 
+
.source of snippet `string_reverse`
[source]
----
println "alma ata".reverse()

----
+
.output of snippet `string_reverse`
[source]
----
ata amla

----


            * `chop` 
 returns string without the last character
 
+
.source of snippet `string_chop`
[source]
----
mut s = "alma ata\n";
println s;
s = s.chop()
println s;
s = s.chop()
println s;
s = s.chop()
println s;

----
+
.output of snippet `string_chop`
[source]
----
alma ata

alma ata
alma at
alma a

----


            * `chomp` 
 returns the string without the last new-line character.
 If the last character is not a new line character, the original string is returned.
 
+
.source of snippet `string_chomp`
[source]
----
mut s = "alma ata\n";
println s;
s = s.chomp()
println s;
s = s.chomp()
println s;
s = s.chomp()
println s;

----
+
.output of snippet `string_chomp`
[source]
----
alma ata

alma ata
alma ata
alma ata

----


            * `times` 
 repeat the string as specified by the argument.
 This is same as multiplying a string by an integer number.
 
+
.source of snippet `string_times`
[source]
----
println "alma ".times(5)
// same as
println "alma "*5

----
+
.output of snippet `string_times`
[source]
----
alma alma alma alma alma 
alma alma alma alma alma 

----


            * `lower_case` 
 return the string lower-cased character.
 
+
.source of snippet `string_lower_case`
[source]
----
println "aLMa aTTa".lower_case()

----
+
.output of snippet `string_lower_case`
[source]
----
alma atta

----


            * `upper_case` 
 return the string upper-cased characters.
 
+
.source of snippet `string_upper_case`
[source]
----
println "aLMa aTTa".upper_case()

----
+
.output of snippet `string_upper_case`
[source]
----
ALMA ATTA

----


            * `swap_case` 
 swaps the case of each letter.
 
+
.source of snippet `string_swap_case`
[source]
----
println "aLMa aTTa".swap_case()

----
+
.output of snippet `string_swap_case`
[source]
----
AlmA AttA

----


            * `capitalize` 
 return the string with the upper-cased first character.
 
+
.source of snippet `string_capitalize`
[source]
----
println "aLMa aTTa".capitalize()

----
+
.output of snippet `string_capitalize`
[source]
----
ALMa aTTa

----


            * `title` 
 return the string with the upper-cased first character of each word.
 A word is a group of alpha characters that are at the start of the string or follow a white space.
 
+
.source of snippet `string_title`
[source]
----
println "aLMa aTTa".title()

----
+
.output of snippet `string_title`
[source]
----
Alma Atta

----


            * `casefold` 
 return the string with cases folded to the lower case.
 This transformation is aggressive and is to be used for case-insensitive comparison.
 For most of the characters, it is the same as `lower_case`, but it also converts many Unicode characters, like `ß` to `ss`.
 
+
.source of snippet `string_casefold`
[source]
----
println "Straße".casefold()

----
+
.output of snippet `string_casefold`
[source]
----
strasse

----


            * `trim` 
 return the string with spaces removed from the start and the end.
 
+
.source of snippet `string_trim`
[source]
----
let s=" \u202Fspace elol hatul "
println s
println s.trim()

----
+
.output of snippet `string_trim`
[source]
----
  space elol hatul 
 space elol hatul

----


            * `strip` 
 return the string with spaces removed from the start and the end.
 While `trim()` removes only characters with Unicode code points ≤ U+0020 (i.e., ASCII control characters and basic whitespace like:
 space ' ', tab '\t', newline '\n', carriage return '\r', etc.) `strip()` removes all Unicode whitespace, using Character.isWhitespace(int), which includes:

 * ASCII spaces
 * Unicode spaces like:
 * U+00A0 (non-breaking space)
 * U+2003 (em space)
 * U+202F (narrow no-break space)
 * U+3000 (ideographic space)
 
+
.source of snippet `string_strip`
[source]
----
let s="\u202F space elol hatul "
println s
println s.strip()

----
+
.output of snippet `string_strip`
[source]
----
  space elol hatul 
  space elol hatul

----


            * `strip_leading` 
 return the string with spaces removed from the start.
 
+
.source of snippet `string_strip_leading`
[source]
----
let s="\u202F space elol hatul "
println s
println s.strip_leading()

----
+
.output of snippet `string_strip_leading`
[source]
----
  space elol hatul 
  space elol hatul 

----


            * `strip_trailing` 
 return the string with spaces removed from the end.
 
+
.source of snippet `string_strip_trailing`
[source]
----
let s="\u202F space elol hatul "
println s
println s.strip_trailing()

----
+
.output of snippet `string_strip_trailing`
[source]
----
  space elol hatul 
  space elol hatul

----


            * `strip_indent` 
 return the string with some spaces removed from the start of each line, in the case of a multi-line string.
 It removes so many spaces that the line that starts at the leftmost position will have no leading spaces.
 This function also removes the spaces from the end of the lines.
 
+
.source of snippet `string_strip_indent`
[source]
----
let s="""
    bruhaha
      marha
  korte
  valaka mortunaktraburnaki                  """
println s.strip_indent()

----
+
.output of snippet `string_strip_indent`
[source]
----

  bruhaha
    marha
korte
valaka mortunaktraburnaki

----


            * `starts_with` 
 return `true` if the string starts with the specified string.
 
+
.source of snippet `string_starts_with`
[source]
----
println "alma ata".starts_with("alma")
println "alma ata".starts_with("ata")

----
+
.output of snippet `string_starts_with`
[source]
----
true
false

----


            * `ends_with` 
 return `true` if the string ends with the specified string.
 
+
.source of snippet `string_ends_with`
[source]
----
println "alma ata".ends_with("alma")
println "alma ata".ends_with("ata")

----
+
.output of snippet `string_ends_with`
[source]
----
false
true

----


            * `partition` 
 Splits a string into three parts and returns a three-element list.
 If the string has the format `xxxZZZyyy` where `ZZZ` is the partitioning element, then
 the list will be `["xxx", "ZZZ", "yyy"]`.
 If `ZZZ` is not found in the string, then the first element of the list will contain the whole string, and the second and last elements will be empty strings.
 
+
.source of snippet `string_partition`
[source]
----
mut s:str = "abraka; muhaha dabra; mutabor; kalifa";
while s.is_not_empty() {
  mut [start:str, sep:str, s ] = s.partition("; ");
  println start
}

----
+
.output of snippet `string_partition`
[source]
----
abraka
muhaha dabra
mutabor
kalifa

----


            * `partition_regex` 
 Splits a string into three parts and returns a three-element list.
 If the string has the format `xxxZZZyyy` where `ZZZ` is a substring that matches the partitioning regular expression, then
 the list will be `["xxx", "ZZZ", "yyy"]`.
 If `ZZZ` is not found in the string, then the first element of the list will contain the whole string, and the second and last elements will be empty strings.

 It is an error to specify a regular expression that matches a zero-length string.
 
+
.source of snippet `string_partition_regex`
[source]
----
mut s = "abraka, muhaha dabra, mutabor; kalifa";
while s.is_not_empty() {
  mut [start, sep, s ] = s.partition_regex("[ ,;]+")
  println start
}

----
+
.output of snippet `string_partition_regex`
[source]
----
abraka
muhaha
dabra
mutabor
kalifa

----


            * `split` 
 split the string into a list using the argument as a regular expression.
 If there is no argument, the function uses `"\n"`, making it behave the same as `lines()`.
 Optionally, there is a second argument that can limit how many pieces the string is split.
 If it is `0` then there is no limit, but the trailing empty strings are discarded.
 If the limit is -1, the trailing empty elements are kept.
 
+
.source of snippet `string_split`
[source]
----
println jsonify("abraka, muhaha dabra, mutabor; kalifa".split("[ ,;]+",3));

----
+
.output of snippet `string_split`
[source]
----
["abraka","muhaha","dabra, mutabor; kalifa"]

----


            * `msplit` 
 split the string into a list of lists of... using the characters of the argument.
 There is no default splitter character(s) in the case of msplit.
 Optionally, there are second, third, and so on arguments that can limit how many pieces the strings are split.
 If it is `0` then there is no limit, but the trailing empty strings are discarded.
 If the limit is -1, the trailing empty elements are kept.
 
+
.source of snippet `string_msplit`
[source]
----
println jsonify("gropeId:artichokId:varsio,mersio,mubakka|abraka:debarbara,mucika::hoppana|mutabor.kalofa:nohabor,kalifa:abrbor,kalifa".msplit("|:,"));

----
+
.output of snippet `string_msplit`
[source]
----
[[["gropeId"],["artichokId"],["varsio","mersio","mubakka"]],[["abraka"],["debarbara","mucika"],[""],["hoppana"]],[["mutabor.kalofa"],["nohabor","kalifa"],["abrbor","kalifa"]]]

----


            * `bytes` 
 return a bin (byte array) that contains the bytes of the string using the given character encoding.
 The character coding is UTF-8 by default.
 In other cases, it has to be defined by name as a string.
 
+
.source of snippet `string_bytes`
[source]
----
println "alma ata".bytes()
println "alma ata".bytes("Cp037") // EBCDIC encoding

----
+
.output of snippet `string_bytes`
[source]
----
bin"616c6d6120617461"
bin"819394814081a381"

----


            * `chars` 
 return a list that contains the characters of the string, each as a string.
 
+
.source of snippet `string_chars`
[source]
----
println jsonify("Hello, World!".chars())

----
+
.output of snippet `string_chars`
[source]
----
["H","e","l","l","o",","," ","W","o","r","l","d","!"]

----


            * `join` 
 joins the elements of the argument list using the string as a separator/joiner character.
 
+
.source of snippet `string_join`
[source]
----
println ":".join([".","/usr/bin","usr/local/bin"])

----
+
.output of snippet `string_join`
[source]
----
.:/usr/bin:usr/local/bin

----


            * `char_at` 
 get the character at the given position as a single character string.
 
+
.source of snippet `string_char_at`
[source]
----
for i=0 ; i < 8 ; i++ :
    print "alma ata".char_at(i)

----
+
.output of snippet `string_char_at`
[source]
----
alma ata
----


            * `safe_char_at` 
 get the character at the given position or an empty string if the index is out of range.
 The return value is a single character string or an empty string.
 
+
.source of snippet `string_safe_char_at`
[source]
----
for i=-3 ; i < 18 ; i++ :
    print "alma ata".safe_char_at(i);

----
+
.output of snippet `string_safe_char_at`
[source]
----
alma ata
----


            * `ord` 
 return the Unicode code point of the first character of the string.
 
+
.source of snippet `string_ord`
[source]
----
println "a".ord()

----
+
.output of snippet `string_ord`
[source]
----
97

----


            * `index_of` 
 return the first position of the argument string, or -1 if the argument string cannot be found in the original string.
 
+
.source of snippet `string_index_of`
[source]
----
println "alma ata".index_of("lma")
println "alma ata".index_of("amla")

----
+
.output of snippet `string_index_of`
[source]
----
1
-1

----


            * `last_index_of` 
 return the last position of the argument string, or -1 if the argument string cannot be found in the original string.
 
+
.source of snippet `string_last_index_of`
[source]
----
println "alma ata".last_index_of("a")
println "alma ata".last_index_of("amla")

----
+
.output of snippet `string_last_index_of`
[source]
----
7
-1

----





.source of snippet `string_functions`

[source]
----
println " 1. ","alma kadarka".after("ka")
println " 2. ","alma kadarka".before("ka")
println " 3. ","alma kadarka".between("ma","da")
println " 4. ","aa ".times(6)
println " 5. "," aa ".trim()
println " 6. ","aa ".starts_with("a")
println " 7. ","aa".ends_with("a")
println " 8. ","AA".lower_case()
println " 9. ","aa".upper_case()
println "10. ","a,b,c,d".split(",")
println "11. ","abcd".bytes()
println "12. ","abc".char_at(2), " is c"
println "13. ","alma kadarka".index_of("ka")
println "14. ","alma kadarka".last_index_of("ka")
println "15. ","alma kadarka".replace_all("a|d","e")
println "16. ","\n\t\r".quote()
println "17. ","alma kadarka".reverse()
println "18. ","alma kadarka".chop().chop()
println "19. ","alma kadarka\n".chomp()
println "20. ","alma kadarka".chomp()
println "21. ","alma kadarka".contains("ma ka")
println "22. ","alma kadarka".left(4)
println "23. ","alma kadarka".right(4)
println "24. ","alma kadarka".count_substring("a")
println "25. ","kadarka".pad_left(12)
println "26. ","kadarka".pad_left(12,".")
println "27. ","kadarka".pad_right(12) + "<"
println "28. ","kadarka".pad_right(12,".")
println "29. ","   ".is_blank()
println "30. ","   ".is_empty()
println "31. ","+6.2E23".is_numeric()
println "32. ","+6.2E23".is_digit()
println "33. ","6223".is_digit()
println "34. ","Euler Number".is_numeric()
println "35. ","EulerNumber".is_alpha()
println "36. ","EulerNumberisnot314".is_alphanumeric()
println "37. ","2AF".is_hex()
println "38. ","0x2AF".is_hex()
println "39. ","2AF".hex()
println "40. ","0x2AF".hex()
println "41. ","0x2AF".base64()
println "42. ","MHgyQUY=".from_base64()
println "43. ","MHgyQUY=".from_base64_str()
println "44. ","alma van a fa = alarr://".url_encode()
println "45. ","alma+van+a+fa+%3D+alarr%3A%2F%2F".url_decode()
println "46. ","line 1\nline2\n\nline4".lines()
println "47. ","What do you read, my lord? Words, words, words.".words()
println "48. ","alma kadarka".remove_prefix("alma")
println "49. ","alma kadarka".remove_postfix("kadarka")
println "50. ","alma kadarka".remove_prefix("apple")
println "51. ","alma kadarka".remove_postfix("kik")
println "52. ",">" + ("alma".safe_char_at(666)) + "<"
println "53. ",",".join(["abrak","a","dabra","lonak","zabla"])
println "54. ","aaa".md5()
println "55. ","aaa".sha_1()
println "56. ","aaa".sha_256()
println "57. ","aaa".sha_512()
println "60. ","aaa".digest("MD5")
println "61. ",61.hex()
println "62. ","mut z = if a == 3 :  3 else: 5;".turi_lex()
println "63. ","mut M= \"\"; for i=0 ; i < 8 ; i++ : M = M + i; M".execute()
println "64. ",M
println "65. ","65".int() , "is 65... literally"
println "65. ","3.14".float(), " is 3.14"
println "66. ","0765".number(8), " is ", 8*8*7+8*6+5

----

.output of snippet `string_functions`
[source]
----
 1. darka
 2. alma 
 3.  ka
 4. aa aa aa aa aa aa 
 5. aa
 6. true
 7. true
 8. aa
 9. AA
10. [a, b, c, d]
11. bin"61626364"
12. c is c
13. 5
14. 10
15. elme keeerke
16. \n\t\r
17. akradak amla
18. alma kadar
19. alma kadarka
20. alma kadarka
21. true
22. alma
23. arka
24. 5
25.      kadarka
26. .....kadarka
27. kadarka     <
28. kadarka.....
29. true
30. false
31. true
32. false
33. true
34. false
35. true
36. true
37. true
38. true
39. 687
40. 687
41. MHgyQUY=
42. bin"3078324146"
43. 0x2AF
44. alma+van+a+fa+%3D+alarr%3A%2F%2F
45. alma van a fa = alarr://
46. [line 1, line2, , line4]
47. [What, do, you, read, my, lord, Words, words, words, ]
48.  kadarka
49. alma 
50. alma kadarka
51. alma kadarka
52. ><
53. abrak,a,dabra,lonak,zabla
54. 47bce5c74f589f4867dbd57e9ca9f808
55. 7e240de74fb1ed08fa08d38063f6a6a91462a815
56. 9834876dcfb05cb167a5c24953eba58c4ac89b1adf57f28f2f9d09af107ee8f0
57. d6f644b19812e97b5d871658d6d3400ecd4787faeb9b8990c1e7608288664be77257104a58d033bcf1a0e0945ff06468ebe53e2dff36e248424c7273117dac09
60. 47bce5c74f589f4867dbd57e9ca9f808
61. 0x3D
62. [Lex{type=RESERVED, text='mut', atLineStart=false}, Lex{type=IDENTIFIER, text='z', atLineStart=false}, Lex{type=RESERVED, text='=', atLineStart=false}, Lex{type=RESERVED, text='if', atLineStart=false}, Lex{type=IDENTIFIER, text='a', atLineStart=false}, Lex{type=RESERVED, text='==', atLineStart=false}, Lex{type=INTEGER, text='3', atLineStart=false}, Lex{type=RESERVED, text=':', atLineStart=false}, Lex{type=INTEGER, text='3', atLineStart=false}, Lex{type=RESERVED, text='else', atLineStart=false}, Lex{type=RESERVED, text=':', atLineStart=false}, Lex{type=INTEGER, text='5', atLineStart=false}, Lex{type=RESERVED, text=';', atLineStart=false}]
63. 01234567
64. 01234567
65. 65is 65... literally
65. 3.14 is 3.14
66. 501 is 501

----



== Bin Values and Functions

A `bin` value is a byte array.
It is the type Turicum uses for binary data: encoded text, hashes, base64 and hex decoded content, and binary file content.
You can create a `bin` value with the built-in function `bin()` (see the built-in functions), or from a string with the methods `bytes()`, `from_hex()`, and `from_base64()`.

The bytes of a `bin` are presented unsigned on the language surface.
Reading a byte — indexing, iteration, `to_list()` — gives an integer in the range `0..255`.
Writing a byte accepts an integer in the range `-128..255` and stores the low 8 bits; any other value is an error.

A `bin` is mutable and has a fixed length.
Assigning to a single index, as in `b[0] = 255`, modifies the byte array in place, and the change is visible through every variable that refers to the same `bin`.
Use the method `copy()` when you need an independent value.
Operations that change the length — concatenation with `+`, repetition with `*`, slicing — always create a new `bin`.
Assigning a `bin` to an index range, as in `b[1..3] = bin([9])`, may also change the length; in that case the variable is updated to hold a new byte array, and other variables that referred to the original `bin` keep the unchanged original.

You can index and slice a `bin` the same way as a string: `b[1]` is the byte at position one, `b[1..3]` is a new `bin` with the bytes of the range, and negative range ends count from the end of the value.
You can iterate over the bytes with `for each`.
The function `len()` gives the number of bytes.

The operators work on `bin` values the following way:

* `+` concatenates two `bin` values into a new one; mixing a `bin` with any other type on the right side of `+` is an error, and has to be converted explicitly.
* `*` with an integer repeats the content, like for strings.
* `==` and `!=` compare the content, byte by byte.
* `<`, `\<=`, `>`, `>=` order `bin` values lexicographically by the unsigned byte values.
* `===` is `true` only when the two operands are the same byte array.
* `in` tests containment: a byte value, as in `2 in b`, or a contiguous subsequence, as in `bin([1,2]) in b`.

A `bin` cannot be used as a boolean, and `jsonify()` refuses it; convert explicitly with `base64()`, `hex()`, or `to_list()`.

When a `bin` is converted to a string — printing, interpolation, `str()` — the result is `+bin"..."+` with the content in hexadecimal, for example `+bin"48656c6c6f"+`.
The hexadecimal content is the same string the method `hex()` returns and the string method `from_hex()` accepts.

The methods callable on `bin` values are the following:

            * `to_string` 
 returns the textual representation of the bin, `bin"..."` with the content in hexadecimal.
            * `length` 
 a convenience method that returns the number of bytes.
 The Turicum way is to use the `len()` built-in function, but this method also works on bin values.
            * `text` 
 decode the bytes to a string.
 The character encoding is UTF-8 by default; a different one can be named as an argument.
            * `to_list` 
 return a list that contains the byte values as integers in the range 0..255.
            * `base64` 
 encode the bytes with Base64 and return the string.
            * `base64_url` 
 encode the bytes with the URL and filename safe Base64 alphabet and return the string.
            * `hex` 
 return the content as a lowercase hexadecimal string, two digits per byte, no separator.
            * `copy` 
 return a new bin with the same content.
 Use it to get an independent value, since bin values are mutable and assignment does not copy.
            * `md5` 
 calculate the md5 hash of the bytes. The result is a bin; use `hex()` on it for display.
            * `sha_1` 
 calculate the sha-1 hash of the bytes. The result is a bin.
            * `sha_256` 
 calculate the sha-256 hash of the bytes. The result is a bin.
            * `sha_512` 
 calculate the sha-512 hash of the bytes. The result is a bin.
            * `digest` 
 calculate the hash of the bytes. The algorithm name has to be provided as an argument. The result is a bin.
            * `index_of` 
 return the first position of the argument, or -1 when it is not found.
 The argument is a byte value (an integer) or a bin subsequence.
 The optional second argument is the position where the search starts.
            * `last_index_of` 
 return the last position of the argument, or -1 when it is not found.
 The argument is a byte value (an integer) or a bin subsequence.
 The optional second argument is the highest position where the occurrence may start.
            * `contains` 
 return `true` if the bin contains the argument, which is a byte value (an integer) or a bin subsequence.
 The same test is available as the `in` operator.
            * `starts_with` 
 return `true` if the bin starts with the argument bin.
            * `ends_with` 
 return `true` if the bin ends with the argument bin.
            * `count` 
 count the non-overlapping occurrences of the argument, which is a byte value (an integer) or a bin subsequence.
            * `is_empty` 
 return `true` if the bin has no bytes.
            * `is_not_empty` 
 return `false` if the bin has no bytes.
            * `left` 
 return a new bin with at most the argument number of leading bytes.
 If the argument is equal to, or larger than the length, the whole content is returned (as a new bin).
            * `right` 
 return a new bin with at most the argument number of trailing bytes.
 If the argument is equal to, or larger than the length, the whole content is returned (as a new bin).
            * `reverse` 
 return a new bin that contains the bytes in reverse order.
            * `xor` 
 return a new bin that is the byte-wise XOR of this bin and the argument bin.
 The two bins must have the same length.
            * `fill` 
 set a range of the bin to the given byte value, in place, and return the bin itself.
 Without further arguments the whole bin is filled.
 The optional second and third arguments give the start (inclusive) and the end (exclusive) of the range.
            * `set` 
 copy the whole argument bin into this bin starting at the given position, in place,
 and return the bin itself. The copied content must fit.
            * `u8_at` 
 read the byte at the given position as an unsigned integer, 0..255.
            * `i8_at` 
 read the byte at the given position as a signed integer, -128..127.
            * `u16_at` 
 read two bytes at the given position as an unsigned integer.
 The optional second argument is the byte order: `"be"` (default), `"le"`,
 or a permutation of the digits 1..2 where 1 is the least significant byte.
            * `i16_at` 
 read two bytes at the given position as a signed integer.
            * `u32_at` 
 read four bytes at the given position as an unsigned integer.
 The optional second argument is the byte order: `"be"` (default), `"le"`,
 or a permutation of the digits 1..4, for example `"3412"` for the PDP-11 layout.
            * `i32_at` 
 read four bytes at the given position as a signed integer.
            * `u64_at` 
 read eight bytes at the given position as an unsigned integer.
 A value above the signed 64-bit range is an error, because Turicum integers are signed 64-bit values.
            * `i64_at` 
 read eight bytes at the given position as a signed integer.
            * `set_u8_at` 
 write one byte at the given position, in place. The value must be in the range 0..255.
            * `set_i8_at` 
 write one byte at the given position, in place. The value must be in the range -128..127.
            * `set_u16_at` 
 write a value as two bytes at the given position, in place.
 The optional third argument is the byte order, as for `u16_at`.
            * `set_i16_at` 
 write a signed value as two bytes at the given position, in place.
            * `set_u32_at` 
 write a value as four bytes at the given position, in place.
            * `set_i32_at` 
 write a signed value as four bytes at the given position, in place.
            * `set_u64_at` 
 write a non-negative value as eight bytes at the given position, in place.
            * `set_i64_at` 
 write a signed value as eight bytes at the given position, in place.


The following example demonstrates the use of the `bin` methods and operators:

.source of snippet `bin_functions`

[source]
----
println " 1. ", bin([72, 101, 108, 108, 111])
println " 2. ", "Hello".bytes()
println " 3. ", "Hello".bytes().text()
println " 4. ", "Hello".bytes().hex()
println " 5. ", "48656c6c6f".from_hex()
println " 6. ", "Hello".bytes().base64()
println " 7. ", "SGVsbG8=".from_base64()
println " 8. ", "Hello".bytes().to_list()
println " 9. ", len("Hello".bytes())
println "10. ", "Hello".bytes()[1]
println "11. ", "Hello".bytes()[1..4]
println "12. ", bin([1, 2]) + bin([3])
println "13. ", bin([7]) * 3
println "14. ", "Hello".bytes().md5().hex()
println "15. ", bin([1, 2, 3, 2, 1]).index_of(2)
println "16. ", bin([1, 2, 3, 2, 1]).last_index_of(2)
println "17. ", bin([1, 2, 3]).contains(bin([2, 3]))
println "18. ", bin([1, 2, 3]).starts_with(bin([1]))
println "19. ", bin([1, 2, 3]).ends_with(bin([3]))
println "20. ", bin([1, 1, 1, 1]).count(bin([1, 1]))
println "21. ", bin([1, 2, 3]).left(2)
println "22. ", bin([1, 2, 3]).right(2)
println "23. ", bin([1, 2, 3]).reverse()
println "24. ", bin([255, 0]).xor(bin([15, 15]))
println "25. ", bin(4).fill(7)
println "26. ", bin(4).set(1, bin([9, 9]))
mut b = bin(8)
b.set_u32_at(0, 305419896)
println "27. ", b
println "28. ", b.u32_at(0)
println "29. ", b.u16_at(0, "le")
println "30. ", b.u32_at(0, "3412")
println "31. ", bin([128]).u8_at(0), " ", bin([128]).i8_at(0)
mut m = bin([1, 2, 3])
m[0] = 255
println "32. ", m
m[1..2] = bin([8, 8, 8])
println "33. ", m
println "34. ", 2 in bin([1, 2, 3])
for each x in bin([10, 20]) : println "35. ", x

----

.output of snippet `bin_functions`
[source]
----
 1. bin"48656c6c6f"
 2. bin"48656c6c6f"
 3. Hello
 4. 48656c6c6f
 5. bin"48656c6c6f"
 6. SGVsbG8=
 7. bin"48656c6c6f"
 8. [72, 101, 108, 108, 111]
 9. 5
10. 101
11. bin"656c6c"
12. bin"010203"
13. bin"070707"
14. 8b1a9953c4611296a827abf8c47804d7
15. 1
16. 3
17. true
18. true
19. true
20. 2
21. bin"0102"
22. bin"0203"
23. bin"030201"
24. bin"f00f"
25. bin"07070707"
26. bin"00090900"
27. bin"1234567800000000"
28. 305419896
29. 13330
30. 873625686
31. 128 -128
32. bin"ff0203"
33. bin"ff08080803"
34. true
35. 10
35. 20

----



== Built-in Functions

Built-in functions are functions provided by the interpreter, which extend the functionality of the run-time.

=== Mathematical Functions

Turicum implements many mathematical functions.
All mathematical functions from Java's Math class are available in Turicum.
In addition to that, there are some Turicum-specific ones also.

The functions implemented are the following:

 * `random()` Returns a random number between 0 and 1.
 * `java_long(x)` Converts the argument to a Java long.
 Technically, this is the same function as `int` because Turicum stores integer numbers as longs.
 The use of this function is recommended over `int` when the argument is used to pass to a Java method that expects a long argument.
 * `int(x)` Converts the argument to an integer.
 * `java_int(x)` Converts the argument to a Java Integer.
 Use this function when the argument is used to pass to a Java method that expects an int or Integer argument.
 * `java_short(x)` Converts the argument to a Java Short.
 Use this function when the argument is used to pass to a Java method that expects a short or Short argument.
 * `java_char(x)` Converts the argument to a Java Character.
 Use this function when the argument is used to pass to a Java method that expects a char or Character argument.
 Note that some of the numerical values interpreted as code points may not be valid characters.
 Multiple Java characters may represent them.
 In such cases, an error is thrown.
 * `java_byte(x)` Converts the argument to a Java Byte.
 Use this function when the argument is used to pass to a Java method that expects a byte or Byte argument.
 * `float(x)` Converts the argument to a float.
 The actual value is represented as a double.
 Turicum stores floating numbers internally as doubles.
 * `java_double(x)` Converts the argument to a Java double.
 Use this function when the argument is used to pass to a Java method that expects a double or Double argument.
 Turicum stores floating numbers internally as doubles.
 So technically this function is the same as `float`.
 * `java_float(x)` Converts the argument to a Java float.
 Use this function when the argument is used to pass to a Java method that expects a float or Float argument.
 * `num(x)` Converts the argument to a number.
 The result will either be a float or an int (Java double or long).
 * `scalb(d,s)` returns `d` times `2` to the power of `s`.
 *`sin` returns the trigonometric sine of an angle.
 * `cos(x)` returns the trigonometric cosine of an angle.
 * `acos(x)` returns the trigonometric arc cosine of a value.
 * `asin(x)` returns the trigonometric arc sine of a value.
 * `atan(x)` returns the trigonometric arc tangent of a value.
 * `cbrt(x)` returns the cube root of a value.
 * `ceil(x)` returns the smallest integer value that is greater than or equal to the argument and is equal to a mathematical integer.
 * `abs(x)` returns the absolute value of a number.
 * `exp(x)` returns the value `e` raised to the power of a value.
 * `floor(x)` returns the largest integer value that is less than or equal to the argument and is equal to a mathematical integer.
 * `log(x)` returns the natural logarithm (base `e`) of a value.
 * `log10(x)` returns the base-10 logarithm of a value.
 * `sqrt(x)` returns the positive square root of a value.
 * `tan(x)` returns the trigonometric tangent of an angle.
 * `tanh(x)` returns the hyperbolic tangent of a value.
 * `sinh(x)` returns the hyperbolic sine of a value.
 * `cosh(x)` returns the hyperbolic cosine of a value.
 * `get_exponent(x)` returns the exponent used in the representation of a number.
 * `round(x)` rounds a number to the nearest integer.
 * `signum(x)` returns the sign of a number.
 * `next_down(x)` returns the next representable floating-point value after a given number towards negative infinity.
 * `ulp(x)` returns the size of an ulp of the argument.
 An ulp, unit in the last place, of a double value is the positive distance between this floating-point value and the double value next larger in magnitude.
 * `rint(x)` returns the floating-point value that is closest in value to the argument and is equal to a mathematical integer.
 * `to_degrees(x)` converts an angle measured in radians to an approximately equivalent angle measured in degrees.
 * `to_radians(x)` converts an angle measured in degrees to an approximately equivalent angle measured in radians.
 * `pow(x,y)` returns the first argument raised to the power of the second argument.
 * `atan2(x,y)` returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
 * `ieee_remainder(x,y)` returns the remainder operation on floating-point values according to IEEE 754.
 * `copy_sign(x,y)` returns a value with the magnitude of the first argument and the sign of the second argument.
 * `hypot(x,y)` returns the hypotenuse of a right triangle with sides of length a and b.
 * `next_after(x,y)` returns the next representable floating-point value after a given number towards positive infinity.
 * `next_up(x)` returns the next representable floating-point value after a given number towards positive infinity.


.source of snippet `math_functions`

[source]
----
println "Testing math functions:"
println "sin(1) = ", sin(1)
println "cos(1) = ", cos(1)
println "acos(0.5) = ", acos(0.5)
println "asin(0.5) = ", asin(0.5)
println "atan(1) = ", atan(1)
println "atan2(1,1) = ", atan2(1,1)
println "cbrt(27) = ", cbrt(27)
println "ceil(3.2) = ", ceil(3.2)
println "abs(-3) = ", abs(-3)
println "exp(1) = ", exp(1)
println "floor(3.8) = ", floor(3.8)
println "log(2.718281828459045) = ", log(2.718281828459045)
println "log10(100) = ", log10(100)
println "sqrt(16) = ", sqrt(16)
println "tan(1) = ", tan(1)
println "tanh(1) = ", tanh(1)
println "sinh(1) = ", sinh(1)
println "cosh(1) = ", cosh(1)
println "to_degrees(3.14159) = ", to_degrees(3.14159)
println "to_radians(180) = ", to_radians(180)
println "pow(3,3) =", pow(3,3)
println "copy_sign(3.5,-2.1) = ", copy_sign(3.5,-2.1)
println "get_exponent(123.456) = ", get_exponent(123.456)
println "hypot(3,4) = ", hypot(3,4)
println "ieee_remainder(5,2) = ", ieee_remainder(5,2)
println "next_after(1.0,2.0) = ", next_after(1.0,2.0)
println "next_down(1.0) = ", next_down(1.0)
println "next_up(1.0) = ", next_up(1.0)
println "rint(3.7) = ", rint(3.7)
println "round(3.7) = ", round(3.7)
println "signum(-3.7) = ", signum(-3.7)
println "ulp(1.0) = ", ulp(1.0)
println "scalb(2.0,3) = ", scalb(2.0,3)

----

.output of snippet `math_functions`
[source]
----
Testing math functions:
sin(1) = 0.8414709848078965
cos(1) = 0.5403023058681398
acos(0.5) = 1.0471975511965979
asin(0.5) = 0.5235987755982989
atan(1) = 0.7853981633974483
atan2(1,1) = 0.7853981633974483
cbrt(27) = 3.0
ceil(3.2) = 4.0
abs(-3) = 3.0
exp(1) = 2.7182818284590455
floor(3.8) = 3.0
log(2.718281828459045) = 1.0
log10(100) = 2.0
sqrt(16) = 4.0
tan(1) = 1.5574077246549023
tanh(1) = 0.7615941559557649
sinh(1) = 1.1752011936438014
cosh(1) = 1.543080634815244
to_degrees(3.14159) = 179.9998479605043
to_radians(180) = 3.141592653589793
pow(3,3) =27.0
copy_sign(3.5,-2.1) = -3.5
get_exponent(123.456) = 6
hypot(3,4) = 5.0
ieee_remainder(5,2) = 1.0
next_after(1.0,2.0) = 1.0000000000000002
next_down(1.0) = 0.9999999999999999
next_up(1.0) = 1.0000000000000002
rint(3.7) = 4.0
round(3.7) = 4
signum(-3.7) = -1.0
ulp(1.0) = 2.220446049250313E-16
scalb(2.0,3) = 16.0

----




=== `add_java_classes`

Adds a jar file or location to the Turi environment.

The argument of the function is a string that is the name of a JAR file.
The function searches the `APPIA` directories for the JAR file and loads it into the Turi environment.
The argument should specify the full name, including the `.jar` extension.
If the name contains a partial path, it will be tried for each of the `APPIA` directories.

After the JAR file is loaded, the application can use the classes defined in the JAR file.
It means that `java_class` can reference the classes defined in the JAR file, and all the defined functions, macros, and class extensions will be available for the application.
You can also `sys_import` turicum files from the JAR, and you can call `java_resources()` to access the resources defined in the JAR.

=== `all_parents`

Returns a list of classes that are the parent classes of the argument with a transitive closure.
It means that all classes, including the parents of the parents and so on, are recursively included in the list.

.source of snippet `all_parents`

[source]
----
class A { }
class B: A { }
mut K=class: B { }
println all_parents( K )
println( [ .. all_parents( K ) -> type( it() ) ] )

----

.output of snippet `all_parents`
[source]
----
[class B, class A]
[B, A]

----



The classes are ordered in inheritance order.


==== `as_object`

This function converts a command into an object.
This function supports reflective programming and is likely used together with the `thunk`, `unthunk`, inside macros.

.source of snippet `as_object`

[source]
----
let z=thunk( { let k= 5 } )
println jsonify_beauty( as_object( z ) )

----

.output of snippet `as_object`
[source]
----
{
    "java$canonicalName": "ch.turic.commands.BlockCommand",
    "wrap": true,
    "commands": [
                    {
                        "assignments": [
                                           {
                                               "identifier": "k",
                                               "types": [ ],
                                               "expression": {
                                                                 "java$canonicalName": "ch.turic.commands.IntegerConstant",
                                                                 "value": 5
                                                             },
                                               "java$canonicalName": "ch.turic.analyzer.AssignmentList.Assignment"
                                           }
                                       ],
                        "java$canonicalName": "ch.turic.commands.LetAssignment",
                        "mut": false
                    }
                ]
}

----



The field `java$canonicalName` contains the canonical name of the class that implements the command.
The other fields are the fields of the command implementation.
In the example above, the command is a block command.
It is a wrapping command (it sees the variables defined before it outside of the block) and contains an array of commands.
This array has one element, which is an assignment (`LetAssignment`).
One assignment command may contain multiple assignments; hence, the field `assignments` is named plural and is an array.
In this case, it has one element.
It has an identifier, `k`, a types array, and an expression, which is an integer constant with the value `5`.
The assignment uses the `let` keyword; therefore, the `mut` field is set to `false`.


=== `arity`

The built-in function `arity` returns the number of parameters of the argument function.
You pass a function, macro, or closure to this function, and the return value is the number of arguments.
The arguments `[rest]`, `{meta}`, and `^closure` do ot count into the arity.


.source of snippet `arity`

[source]
----
fn a( x, z, h ){ }
fn b( x, z,{ h } ){ }
println arity( a )
println arity( b )

----

.output of snippet `arity`
[source]
----
3
2

----



==== `block_list`

The function `block_list` will return a list containing the commands in a command block.
The following example shows you how it can be used together with `thunk` and `unthunk`.

.source of snippet `block_list`

[source]
----
mut commands = block_list(thunk({k = 13; suss = "huss"; mut x = 3; println "hello"}))
println "before execution"
mut i = 1;
for each command in commands {
    mut suss;
    println "%s. before is_defined(x) = %s" % [i,is_defined(x)]
    if i == 1 {// k is not defined, no assignable
        try: unthunk(command) catch e: println "ERROR: %s" % [e];
    } else {
        // 2 can assign value to 'suss'
        // 3 can define 'x'
        // 4 prints 'hello'
        unthunk(command)
        println "%s. after is_defined(x) = %s" % [i,is_defined(x)]
    }
    i++
}

----

.output of snippet `block_list`
[source]
----
before execution
1. before is_defined(x) = false
ERROR: Variable 'k' is undefined.
2. before is_defined(x) = false
2. after is_defined(x) = false
3. before is_defined(x) = false
3. after is_defined(x) = true
4. before is_defined(x) = false
hello
4. after is_defined(x) = false

----



The counter `i` is used to display which command we execute in the loop.
It is also used to execute the first command in a `try-catch` block.
`k` is not defined; therefore, unthunking this command will not work.

The second command updates a declared variable.
It does not need to be in a `try-catch` block.

The third command defines the variable `x`.
In this execution, the variable `x` after the ``unthunk``ing is defined.

`x` becomes a normal local variable in the loop, which is defined for the execution of the loop core only.
When the next execution starts the variable `x` is undefined again.


=== `chr`

Converts a number interpreted as a Unicode code point to a one-character string.


=== `bin`

Creates a `bin` value, a byte array.
The result depends on the type of the argument:

* without argument, the result is an empty `bin`,

* an integer argument creates a `bin` of that many zero bytes,

* a list argument converts the elements; each element has to be an integer in the range -128..255 and is stored as an unsigned byte,

* a string argument is encoded to bytes using UTF-8, or the character set named by the optional second argument,

* a `bin` argument is copied to a new, independent `bin`.

.source of snippet `bin`

[source]
----
println bin()
println bin(3)
println bin([65, 66, 67])
println bin("túró")
println bin("túró", "ISO-8859-2")
println bin(bin([1, 2, 3]))

----

.output of snippet `bin`
[source]
----
bin""
bin"000000"
bin"414243"
bin"74c3ba72c3b3"
bin"74fa72f3"
bin"010203"

----




=== `curried_arity`

The built-in function `curried_arity()` returns the number of curried arguments.

When you curry a function, then the arity of the resulting function decreases by the number of curried arguments.
The function `arity()` returns the full arity of the function.

.source of snippet `curried_arity`

[source]
----
fn add(a,b)=a + b
let add5 = add.(5)
fn test(s){ println s," = ", s.execute() }
test "add5(5)"
test "curried_arity(add5)"
test "arity(add5)"
test "curried_arity(add)"
test "arity(add)"

----

.output of snippet `curried_arity`
[source]
----
add5(5) = 10
curried_arity(add5) = 1
arity(add5) = 2
curried_arity(add) = 0
arity(add) = 2

----



The function `add()` in the example adds two numbers.
The arity of the function is two, as it has two arguments.
The function `add5()` curries one of the arguments.
The arity of this new function is still two in Turicum, but one argument is now curried.
The number of the curried arguments is returned by `curried_arity()`.


=== `delete`

`delete` is a macro to delete a field from an object.
The macro should have one or two arguments.

When it has two arguments, the first argument is an object and the second argument is a field name.
That way, you can delete the field named `pass:[b]` of object `pass:[a]` as

[source]
----
    delete a,b
----

You can also use the form:

[source]
----
    delete a.b
----

In this case, there is only one argument: an expression resulting from accessing an object field.
The field of the object is deleted.

Although you can access a field using the `a["b"]` format, this format is not usable in the delete command.
Instead, you can use the two-argument version with a string as the second argument.

[source]
----
    delete a, "b"
----
It is an error if there is no such field in the object or if the first argument is not an object.

.source of snippet `delete`

[source]
----
let a = { a: 1 , b: 2, c: ["kedavra"], d:"avraka" , h: none}
a.a = none // this does not delete the field
println "'a' set to none ", jsonify(a)
delete a, a // delete the field a
println "deleted field 'a' ", jsonify(a)
delete a.h
println "deleted field 'h' via field access", jsonify(a)
delete {13; 55; a }, b // the object is calculated
delete a, "c" // string name for the field
println "deleted field 'c' using a string ", jsonify(a)
delete a, chr("c".ord()+1) // string calculated name
println "deleted field 'd' using calculated string ", a
try delete a, d catch e1 println e1  // was already deleted
a.c = []
try delete a.c, h catch e2 println e2

----

.output of snippet `delete`
[source]
----
'a' set to none {"a":null,"b":2,"c":["kedavra"],"d":"avraka","h":null}
deleted field 'a' {"b":2,"c":["kedavra"],"d":"avraka","h":null}
deleted field 'h' via field access{"b":2,"c":["kedavra"],"d":"avraka"}
deleted field 'c' using a string {"d":"avraka"}
deleted field 'd' using calculated string {}
Variable 'd' is not defined in the local context, you cannot unlet it.
'[]' is not a valid object

----




=== `enumerate()`

The enumerate function is included, although Turicum already provides this capability through the `with` keyword in loops.
Its presence is mainly a courtesy to those who might otherwise feel deprived after growing accustomed to it in Python.

As you can see, the elements of the list are enumerated, and the index is stored in the first element of the pair.
Note that this function works on anything that you can use in a `for each` loop.

The result of the `enumerate()` function is an iterable object, which creates the next element only when it is requested.
It means that you can use it to loop through possibly extremely large iterables without consuming excessive memory.

.source of snippet `foreach5`

[source]
----
for each [ j, z ] with i in enumerate(["apple", "banana", "orange", "peach"]) {
println "%s.%s. %s" % [i, j, z]
}

----

.output of snippet `foreach5`
[source]
----
0.0. apple
1.1. banana
2.2. orange
3.3. peach

----



In the example above, the elements of the `enumerate()` are stored in `j` and `z`.
The variable `j` holds the index of the element, and `z` holds the element itself.
The example demonstrates that this is the same as the index value, stored in the variable `i` provided through the `with` keyword.



=== `env`

This function returns the value of the environment variable.

Without argument, it returns an object containing all environment variables as fields.

.source of snippet `env1`

[source]
----
println env().JAVA_HOME

----

.output of snippet `env1`
[source]
----
/Users/verhasp/.sdkman/candidates/java/current

----



With a given string argument without `*` or `?` wild card characters, the return value is the value of the environment variable.

.source of snippet `env2`

[source]
----
println env("JAVA_HOME")

----

.output of snippet `env2`
[source]
----
/Users/verhasp/.sdkman/candidates/java/current

----



If the argument contains wild card characters, the result will be an object containing all environment variables as fields that match the argument.

.source of snippet `env3`

[source]
----
println env("JAVA_*")

----

.output of snippet `env3`
[source]
----
{JAVA_HOME: /Users/verhasp/.sdkman/candidates/java/current}

----




=== `evaluate`

Evaluate a command object in the caller environment.
It is not to evaluate a string.
If you want to evaluate a string containing Turicum code, you have to use `compile()` or `execute()` string methods.

Usually, macro implementations use this function.
Macros get their arguments unevaluated and can use this function to evaluate them.
If you try to evaluate anything other than a macro argument, you likely will get an error.

.source of snippet `evaluate1`

[source]
----
mut twice = macro(fn (arg){ evaluate(arg); evaluate(arg);});
twice( {println("Hello")} )

----

.output of snippet `evaluate1`
[source]
----
Hello
Hello

----



The specific requirement to use this function is that the argument is a command object and that there is a caller environment.
If you pass a `thunk`-ed command to a function (non-macro) you can also use this function to evaluate.

The following example uses all the possible tools that can appear in a macro and a function.
The actual functionality is basic: it just invokes the closure or function provided as the first argument with the rest of the arguments passed on.
In real life, you could just call the function itself without making it through a macro, but in this case, we focus on doing that and how.

The unnamed function, which is the argument of the macro, has a mandatory, position-only argument and all the three extra arguments.
The first argument will be the function or closure we will call.
The function first called `evaluate` to get the function or closure.
Since this function is never called as a function, only after it is converted to be a macro, the argument is never a function, but a function definition.
Evaluating it will result in the function.

When we call this function, adding `(..rest,..meta,..closure)` after it, we spread the extra parameters.
In this example, the three argument values for these parameters are

* `+{}+` empty object,
* `[]` empty list, and
* `none`

None of them adds extra parameters when spread.

.source of snippet `evaluate2`

[source]
----
macro(fn (!arg,[rest],{meta},^closure)
            { evaluate(arg)(..rest,..meta,..closure)}
     )(fn () {println("Hello")})

----

.output of snippet `evaluate2`
[source]
----
Hello

----



To be honest, since the evaluation of the argument happens only once and without condition, this functionality does not really need a macro.
It could be shortened as

.source of snippet `evaluate3`

[source]
----
(fn (!arg,[rest],{meta},^closure)
            { arg(..rest,..meta,..closure)})
     (fn () {println("Hello")})

----

.output of snippet `evaluate3`
[source]
----
Hello

----



Note, however, that the code starts with a `(`.
That will make the function definition be part of an expression and process the following `(fn () {println("Hello")})` as an argument.
Without that we would have two unnamed function definitions, one after the other.

[NOTE]
====
The first design required enclosing a function definition like the one above to be enclosed between `+{+` and `+}+` to be part of an expression.
In most of the cases, like assigning a function to a variable would require superfluous `+{+` and `+}+` braces.
The philosophy of Turicum is to be dense and use extra characters where it helps readability or avoids syntax ambiguity.

Omitting the requirement around a `fn` (or for that matter a `class`) definition inside an expression make the language leaner, but this does not make a function or class definition to be an expression itself.
====


=== `export`, `export_all`

These are two functions to be used in files imported.
The macro `export` will export the variables listed as arguments.
That way, these variables will be copied into the context of the importing code.

You can specify identifiers and/or expressions resulting in strings as arguments.

NOTE: Technically, `export` is a macro, recognizing variables instead of their values as arguments.
If you have a variable that contains the name of the symbol you want to export use `export (variable)` to force the evaluation
instead of `export variable`, exporting the variable itself.

`export_all()` will export all the variables from the importing context.
Note that in the case of `export_all()`, you cannot omit the `(` and `)`.


=== `glob`

`glob` is a built-in function to list files.

   fn glob(pattern: str,@path: str|none=none,@recursive: bool=false)

The function has one mandatory and two optional parameters.

* `patter` is a mandatory string parameter.
  It has to define the file name pattern.
  The list matches the usual "glob" patterns, which means

** `pass:[*]` matches zero or more characters, except `/`,
** `pass:[**]` matches zero or more any characters, and
** `pass:[?]` matches exactly one any character.

* `path` optional, named, string parameter.
If not specified, the current working directory will be used.

* `recursive` optional, named, boolean parameter.
Default value is `false`.

The following example lists the first three class files in the test target directory:

.source of snippet `glob1`

[source]
----
sys_import "turi.io"

println jsonify_beauty(glob("**/test-classes/**/*.class", recursive=true)[0..3])

----

.output of snippet `glob1`
[source]
----
[
    "target/test-classes/ch/turic/TestJavaDocSnippets.class",
    "target/test-classes/ch/turic/TestJavaFields.class",
    "target/test-classes/ch/turic/TestTuriParameters$AnnotatedBuiltin.class"
]

----



The second example lists the files in the current working directory only.
It is also an example that shows how to import only one function from the import library,
which otherwise exports multiple functions and classes.

.source of snippet `glob2`

[source]
----
println jsonify_beauty(glob("*"))

----

.output of snippet `glob2`
[source]
----
[
    ".DS_Store",
    "target/",
    "pom.xml",
    "pom.turi",
    ".env",
    "src/"
]

----



The directories in the result list have a trailing `/` at the end of their names (it is guaranteed).

In a sandbox with configured file roots, `glob` is confined by the file read/read-write
roots (not by the import root): a relative pattern is globbed under every root and the
results are the union over the roots; an absolute pattern must point under a root; a result
that a symbolic link would carry outside the roots is silently omitted.


=== `id_hash`

The function `id_hash()` returns the identity hash of an object or a list.
For other arguments, it returns `none`.


=== `import`

Import allows you to get definitions from external files into your code.

When you have an extensive application, you can separate some of the function definitions or classes into separate files.
These separate files can be referenced, and their definitions can be loaded into the currently running interpreter.

The function `import` searches the file, looking at each directory listed in the environment variable `APPIA`.

[NOTE]
====
The application

* first looks at the variable named `APPIA` (either global or local),
* then it tries the Java system property named `APPIA`,
* then the environment variable, and finally
* it tries to load the search path from the local directory `.env` file.

Handling the `.env` file is implemented in the code; there is no need to use external helper tools.
====

[NOTE]
====
JavaScript, Python, and some other scripting languages differentiate between static and dynamic importing.
Due to its fully dynamic nature, all imports are dynamic in Turicum.
====

`APPIA` is a string that lists the directories where the imported files may be.
The individual directories are separated using `|` characters.

[NOTE]
====
The character `|` was selected because it works on Windows as well as on Linux and on other Unix-based operating systems.
The drawback is that the shell may interpret it, but the value of `APPIA` is rarely defined on the command line.
====

When the import function encounters a request to import the file `a.b.c.d`, it will examine the subdirectory `a/b/c` for each `APPIA` directory.
The search is performed from left to right, as defined by the directories in `APPIA`.
The first location where the  `d.turi` file is found "wins" and the file is loaded.

Technically, the function

* loads a Turicum source file,
* executes using a separate interpreter, and
* returns the global context of the interpreter as a class-less object.

If you have function or class definitions in the imported file, they will be fields in the returned object.
You can assign the returned object to a variable or use it immediately, selecting specific fields as needed.
If you want a specific class or function in your namespace, you can `let` assign it to a variable.

The following is a file that is imported by the sample after it.

.file `import_this.turi`
[source]
----
fn imported_fun(){
    "fluffy dog"
}

----

.source of snippet `import`

[source]
----
global APPIA = ["./src/test/resources/"];

let fun = import("import_this").imported_fun

print fun()


----

.output of snippet `import`
[source]
----
fluffy dog
----



There are two functions related to the macro `import`:

* `export()`, and
* `export_all()`.

When these are used in an imported file, the variables from the imported file are copied automatically to the importing context.
With `export()` you can list the symbols that you want to be exported.
The function `export_all()` will export all the global symbols from the imported file, except the predefined ones.
Those are defined already in the importing context, and it would be dangerous to redefine things like `true`, `false`, and built-in functions using an import; hence, it is not done.

Specifying extra arguments after the imported file name can define the symbols you want to import.
These arguments can be identifiers and/or expressions resulting in a string.
If there is such a list in the import function, the `export()` or `export_all()` in the imported file is ignored.

.source of snippet `limited_import`

[source]
----
sys_import "turi.maven" , "lic*"

println license.apache2_0.name
println "The next printout is the error message, because github() is exported but we do not import it."
try : println github("verhas", "turicum"); catch e: println e

----

.output of snippet `limited_import`
[source]
----
The Apache License, Version 2.0
The next printout is the error message, because github() is exported but we do not import it.
Variable 'github' is undefined.

----



Also, when you specify the name of the imported program, you can specify it as a string or as a series of identifiers with dots.



=== `is_curried`

The function `is_curried()` returns `true` if the argument is in a curried function.

.source of snippet `is_curried`

[source]
----
fn add(a,b)=a + b
let add5 = add.(5)
let unc = uncurry(add5)
println is_curried(add)
println is_curried(add5)
println is_curried(unc)

----

.output of snippet `is_curried`
[source]
----
false
true
false

----





=== `is_defined`

This macro accepts an identifier and returns `true` if the identifier is defined in the current context.

.source of snippet `is_defined`

[source]
----
if is_defined(z) : println "defined" else: println "not defined";
mut z = 1
if is_defined(z) : println "defined" else: println "not defined";
try:
    println {if is_defined(z+2) : "defined" else: "not defined"}
catch: println "no way"

----

.output of snippet `is_defined`
[source]
----
not defined
defined
no way

----




=== `is_obj`

This is a function that returns `true` if the argument is an object.

.source of snippet `is_obj`

[source]
----
fn test(s){println jsonify(s), "=" , "is object" if is_obj(s) else "is not object"}

test "hello"
test 1
test {}
test {a:1}
test( [3,2,1] )
test none

----

.output of snippet `is_obj`
[source]
----
"hello"=is not object
1=is not object
{}=is object
{"a":1}=is object
[3,2,1]=is not object
null=is not object

----




=== `is_type`

The return value is true if the object is an instance of the class or is an instance of a child class of the named class.

This function accepts two arguments:

* and object, and
* a string naming a class or the class itself.


.source of snippet `is_type`

[source]
----
class A {}
class B : A {}
mut K = class : B {}
mut z = K()
println is_type(z,"A"),""" "A" yes, it is the root class"""
println is_type(z,A)," A yes, when referencing the class itself"
println is_type(z,"H"),""" "H" no, there is not even defined"""
try: is_type(z,H) catch e: println $"H is not defined: ${e}"
println is_type(z,"K"),""" "K" no, it is not the name of the klass, only a variable"""
println is_type(z,K)," K yes, it is the class itself"
println is_type(z,"#undefined")," #ndefined yes, it is also has a class type that is unnamed"

----

.output of snippet `is_type`
[source]
----
true "A" yes, it is the root class
true A yes, when referencing the class itself
false "H" no, there is not even defined
H is not defined: Variable 'H' is undefined.
false "K" no, it is not the name of the klass, only a variable
true K yes, it is the class itself
true #ndefined yes, it is also has a class type that is unnamed

----




=== `java_call`

This function calls a Java method.

The first argument is either

* the object on which we want to call a method, or

* the name of the class to call a static method on.

The second argument is the name of the method.
The rest of the arguments are passed to the Java method.
The return value is the value returned by the Java method.

This method works with a fixed number of argument functions as well as with vararg methods.
Note that you can call the Java methods on Java objects or on Java class objects directly, like they were Turicum methods.
Use this method only when the name of the method is known only at run-time.

.source of snippet `java_call1`

[source]
----
let sb = java_object("java.lang.StringBuilder", "xBruhahha");
java_call sb,"deleteCharAt",java_int(0)
println sb

----

.output of snippet `java_call1`
[source]
----
Bruhahha

----



The example above performs the same call as in the `java_object` example, but using a string to specify the name of the Java method.


=== `java_import`

Import the classes from a Java package.

The function returns a map of classes from the specified package.
The keys are the class names, and the values are the Java classes themselves.

Listing classes from the classpath is not always possible.
The possibility depends on the actual class loaders.
For example, importing standard JDK packages will not work.

The following example imports the Turicum exception classes:

.source of snippet `java_import`

[source]
----
println jsonify_beauty(java_import("ch.turic.exceptions"))

----

.output of snippet `java_import`
[source]
----
{
    "ExecutionAborted": JavaClass[klass=class ch.turic.exceptions.ExecutionAborted],
    "ch.turic.exceptions.StringNotTerminated": JavaClass[klass=class ch.turic.exceptions.StringNotTerminated],
    "ch.turic.exceptions.StepLimitReached": JavaClass[klass=class ch.turic.exceptions.StepLimitReached],
    "InterpreterHalt": JavaClass[klass=class ch.turic.exceptions.InterpreterHalt],
    "ch.turic.exceptions.UndefinedVariable": JavaClass[klass=class ch.turic.exceptions.UndefinedVariable],
    "BadSyntax": JavaClass[klass=class ch.turic.exceptions.BadSyntax],
    "ch.turic.exceptions.ExecutionException": JavaClass[klass=class ch.turic.exceptions.ExecutionException],
    "ch.turic.exceptions.UnexpectedCharacter": JavaClass[klass=class ch.turic.exceptions.UnexpectedCharacter],
    "ch.turic.exceptions.InterpreterHalt": JavaClass[klass=class ch.turic.exceptions.InterpreterHalt],
    "UnexpectedCharacter": JavaClass[klass=class ch.turic.exceptions.UnexpectedCharacter],
    "StepLimitReached": JavaClass[klass=class ch.turic.exceptions.StepLimitReached],
    "StringNotTerminated": JavaClass[klass=class ch.turic.exceptions.StringNotTerminated],
    "ch.turic.exceptions.BadSyntax": JavaClass[klass=class ch.turic.exceptions.BadSyntax],
    "UndefinedVariable": JavaClass[klass=class ch.turic.exceptions.UndefinedVariable],
    "ExecutionException": JavaClass[klass=class ch.turic.exceptions.ExecutionException],
    "ch.turic.exceptions.ExecutionAborted": JavaClass[klass=class ch.turic.exceptions.ExecutionAborted]
}

----



As you can see, the classes are present in the object with their simple names as well as with their FQN (canonical) name.
This redundancy is intended to avoid ambiguity that can occur when a package is deeply nested, and two different classes have the same simple name.
In that case, only one of them will be included with the simple name, but both are present with the FQN.



=== `java_callback`

This function creates a Java callback.
A java_callback can be passed to Java methods as an argument where it expects a `java.util.function.*` type.

To do the conversion, you have to supply a function, closure, or something callable to the function, and then apply
`.as("...")` on the result to get the actual type. For example:

.source of snippet `java_callback1`

[source]
----
// note you can use the keyword 'as' without the backticks
println java_callback(fn(x) = x*2).as("IntFunction").apply(java_int(2))

----

.output of snippet `java_callback1`
[source]
----
4

----



Although this example is not the typical use of the function, it demonstrates the conversion.

The parameter of the method `as()` is the simple name of the Java class to convert to.
The implemented conversions are listed below.

* `BiConsumer`
* `BiFunction`
* `BinaryOperator`
* `BiPredicate`
* `BooleanSupplier`
* `Consumer`
* `DoubleBinaryOperator`
* `DoubleConsumer`
* `DoubleFunction`
* `DoublePredicate`
* `DoubleSupplier`
* `DoubleToIntFunction`
* `DoubleToLongFunction`
* `DoubleUnaryOperator`
* `Function`
* `IntBinaryOperator`
* `IntConsumer`
* `IntFunction`
* `IntPredicate`
* `IntSupplier`
* `IntToDoubleFunction`
* `IntToLongFunction`
* `IntUnaryOperator`
* `LongBinaryOperator`
* `LongConsumer`
* `LongFunction`
* `LongPredicate`
* `LongSupplier`
* `LongToDoubleFunction`
* `LongToIntFunction`
* `LongUnaryOperator`
* `ObjDoubleConsumer`
* `ObjIntConsumer`
* `ObjLongConsumer`
* `Predicate`
* `Runnable`
* `Supplier`
* `ToDoubleBiFunction`
* `ToDoubleFunction`
* `ToIntBiFunction`
* `ToIntFunction`
* `ToLongBiFunction`
* `ToLongFunction`
* `UnaryOperator`



=== `java_class`

Create a Turicum object that is a Java class itself.
This object can be used as a constructor to create instances of the class and to invoke static methods.

.source of snippet `java_class1`

[source]
----
let m = java_class("java.lang.Math");
println m.abs(-5)
println m.absExact(int(-5))

----

.output of snippet `java_class1`
[source]
----
5.0
5

----



You can also use these objects to create a new instance of the Java class.
These objects are callable, so you can call them to get a new instance of the underlying Java class:

.source of snippet `java_class2`

[source]
----
let BigDecimal = java_class("java.math.BigDecimal")
let h = BigDecimal("10.50")
println $"h is ${h} of ${type(h)}"

----

.output of snippet `java_class2`
[source]
----
h is 10.50 of num

----



The constructor can also be vararg:

.source of snippet `java_class3`

[source]
----
let Locale = java_class("java.util.Locale")
// Locale constructor is vararg
let l1 = Locale("en", "US");
let l2 = Locale("en", "US", "POSIX");
println "%s %s" % [l1, l2]

----

.output of snippet `java_class3`
[source]
----
en_US en_US_POSIX

----




=== `java_object`

This function can be used to create new instances of any Java class.
The first argument to the function is the binary name of the class.
The rest of the arguments should be the constructor arguments.

NOTE: Since Turicum stores numbers as `Double` and `Long`, strings as `Strings`, and boolean values as `Boolean` without any wrapping class.
Calling this function is straightforward when the constructor arguments are one of these primitive values.
If the parameter type is something else, then you have to use `java_object()` to create that object.
If some of the parameters are `Integer` or `int`, you can use the built-in function `java_int()` that converts a numeric value to an `Integer` Java object.
Similarly, if the argument is a `Float` or `float`, you can use the built-in function `float()` that converts a numeric value to a `Float` Java object.

.source of snippet `java_object1`

[source]
----
let bd1 = java_object("java.math.BigDecimal", "10.50");
let bd2 = java_object("java.math.BigDecimal", "3.25");
let result = bd1.add(bd2)
println result

----

.output of snippet `java_object1`
[source]
----
13.75

----



As you can see in the example above, you can invoke the Java methods on the object directly.

.source of snippet `java_object2`

[source]
----
let sb = java_object("java.lang.StringBuilder", "xBruhahha");
try: sb.deleteCharAt(0) catch e: println e
sb.deleteCharAt(java_int(0))
println sb

----

.output of snippet `java_object2`
[source]
----
Cannot find method 'deleteCharAt' on object 'xBruhahha' with arguments class java.lang.Long:0
Bruhahha

----



The example shows that you may need to convert the argument calling `java_int()` when invoking methods that do not work with a `Long` value.


=== `java_resources`

Load the named Java resource or resources and return the strings in a list.

The function takes a string argument and loads the corresponding Java resources.
Java resources are usually in the `resources` directory in the Java project source,
and their content gets into the JAR file together with the binary class files.

There can be multiple resources in the different JAR files and in directories with the same name.
A typical example is the `META-INF/services/...` files that list the classes implementing a service interface.

This function loads all those files and returns all the strings in a list.
If there is only one file for a given name, then the list will only have one element.

.source of snippet `java_resources`

[source]
----
let appia_handlers = java_resources("turi/appia.turi")
println $"There is ${len(appia_handlers)} appia handler sys import file."
println $"It contains ${len(appia_handlers[0])} characters."

----

.output of snippet `java_resources`
[source]
----
There is 1 appia handler sys import file.
It contains 756 characters.

----




=== `java_type`

Returns the canonical name of the Java class of the argument.

.source of snippet `java_type`

[source]
----
println java_type(42), " ", type(42)
println java_type(java_int(42)), " ", type(java_int(42))
println java_type(thunk({let z=2}))," ",type(thunk({let z=2}))
println java_type(java_class("java.math.BigInteger"))," ",type(java_class("java.math.BigInteger"))
println java_type(java_class("java.math.BigInteger")("42"))," ",type(java_class("java.math.BigInteger")("42"))

----

.output of snippet `java_type`
[source]
----
java.lang.Long int
java.lang.Integer num
ch.turic.commands.BlockCommand java.ch.turic.commands.BlockCommand
ch.turic.memory.JavaClass fn
java.math.BigInteger num

----



=== `jsonify`, `jsonify_beauty`

`jsonify` will accept one argument and convert it to a string that is the JSON representation of the argument.
Note that this is not exactly the same as the string produced by `to_sting()` on an object.

* The resulting string will enclose the keys and string value between `pass:["]`.
* All the special characters in the strings will be escaped.
* `none` values are represented as `null`.

This encoding is for machine consumption, without formatting, and without any unnecessary spaces.

If you need a human-readable version of the JSON, you can use the `jsonify_beauty`.
This function uses three parameters:

* the object to jsonify,
* the tab size (how many characters indented structures should be moved to the right),
* desired maximum margin.

.source of snippet `jsonify`

[source]
----
let z = {
  a: 1, b:? [1,2,3]
  pi : 3.1415926,
  location : {
    latidude : 16.990635373109665,
    longidude: 17.935398980291257,
    altidude: [2000, 1000,  -3 , false, "karma" ]
    the_dude : "Peter Verhas",
  }
};

println $"object to_string=${z}";
println;
println $"object jsonify=${jsonify(z)}";
println;
println $"object beauty=${jsonify_beauty(z,2,60)}";


----

.output of snippet `jsonify`
[source]
----
object to_string={a: 1, b: [1, 2, 3], pi: 3.1415926, location: {the_dude: Peter Verhas, longidude: 17.935398980291257, latidude: 16.990635373109665, altidude: [2000, 1000, -3, false, karma]}}

object jsonify={"a":1,"b":[1,2,3],"pi":3.1415926,"location":{"the_dude":"Peter Verhas","longidude":17.935398980291257,"latidude":16.990635373109665,"altidude":[2000,1000,-3,false,"karma"]}}

object beauty={
  "a": 1,
  "b": [ 1, 2, 3],
  "pi": 3.1415926,
  "location": {
                "the_dude": "Peter Verhas",
                "longidude": 17.935398980291257,
                "latidude": 16.990635373109665,
                "altidude": [
                              2000,
                              1000,
                              -3,
                              false,
                              "karma"
                            ]
              }
}

----



=== `keys`

The function `keys()` returns a list containing the keys (strings) of the argument.

* If it is a class, then the keys are the class-level fields, including those that have closure value, hence are class methods.

* If it is an object, then the list contains the object fields.

* If it is a macro, a closure, or a function, then it will return the parameter names.

.source of snippet `keys1`

[source]
----
class A {
    mut z:num = 1
    let k:str = "two"
    fn zibabwa(a,b,@shia,[ta]){}
}
let k = A()
let uu = {|f,y,z| none}

println keys(A), " of A"
println keys(A.zibabwa), " of A.zibabwa"
println keys(k), " of k"
println keys(uu), " of uu"

let BigDecimal = java_class("java.math.BigDecimal")
println keys(BigDecimal)
println keys(BigDecimal("0"))

----

.output of snippet `keys1`
[source]
----
[zibabwa, z, cls, k] of A
[a, b, shia, ta] of A.zibabwa
[this, cls] of k
[f, y, z] of uu
[ZERO, ROUND_DOWN, ROUND_UNNECESSARY, ROUND_CEILING, ROUND_HALF_DOWN, ROUND_HALF_UP, ONE, ROUND_UP, ROUND_FLOOR, ROUND_HALF_EVEN, TEN, TWO]
[]

----



Note that the return value contains the special fields, like `cls`, `this` and so on.

Veiled names (see the `veil` command) are not listed.
Use `keys_all()` to list every key, including the veiled ones.


=== `keys_all`

The function `keys_all()` is the same as `keys()`, but it also lists the veiled names of an
object or a class. It is an introspection tool; accessing the listed veiled fields through the
field access still reports an error.


=== `len`

This function returns the length of the argument.

The argument can be:

* string, the value is the length of the string.

* list, the value is the number of elements in the list.

* Java array, the value is the array length.

* Java collection, the value is the size of the collection.

.source of snippet `len`

[source]
----
println len("abc")
println len([1,2,3])


----

.output of snippet `len`
[source]
----
3
3

----




=== `macro`

This function converts a closure or a function into a macro.

.source of snippet `macro_example`

[source]
----
mut c = macro(
    fn(condition,then,`else`){
        if evaluate(condition) : evaluate(then) else: evaluate(`else`)
    });

print c(true,"yes, it is true",{ println("this will never be printed") })

----

.output of snippet `macro_example`
[source]
----
yes, it is true
----



=== `min`, `max`

These functions return the minimum and the maximum value of their arguments.
If there is only one argument and that argument is a list, then the functions return the extreme element of the list.

The elements can be anything comparable, such as numbers or strings.
If the arguments are objects, they must have the `pass:[`<`]` method defined.

.source of snippet `min_max`

[source]
----
println max(3,5,8,7,4,11,-1)
println max([3,5,8,7,4,11,-1])
println min(3,5,8,7,4,11,-1)
println min([3,5,8,7,4,11,-1])

println max( "giraffe", "zibra", "elephant" )
println min( "affe", "zibra", "elephant" )

class Complex {
    fn init( real:float|num, imaginary:float|num ){
        pin real
        pin imaginary
    }
    fn `<`(other:Complex):bool = real*real + imaginary*imaginary < (with other: real*real + imaginary*imaginary);
    fn to_string()= $"${real}+i${imaginary}"
}

println min( Complex(1,1), Complex(3,2), Complex(0.5,1.7))
println max( Complex(1,1), Complex(3,2), Complex(0.5,1.7))

try: max( Complex(1,1), "Donald")
catch e: println e

----

.output of snippet `min_max`
[source]
----
11
11
-1
-1
zibra
affe
1+i1
3+i2
Cannot set variable 'other' to value 'Donald' because it does not fit the declared type Complex

----



=== `nano_time`

The `nano_time` function in Turicum returns the current value of the most precise available system timer, expressed in nanoseconds.
It is a direct wrapper around Java’s `+System.nanoTime()+` method.
It is typically used for measuring elapsed time with high resolution.

Unlike timestamps, the returned value has no absolute meaning but is strictly monotonic, making it ideal for performance profiling and timing operations.

.source of snippet `nano_time`

[source]
----
// stochastic
println nano_time()

----

.output of snippet `nano_time`
[source]
----
391905376482875

----



=== `pack`

This function creates an object with the actual local context as the object context.
It does not copy the context.
If the local context changes after the object is created, the object updates accordingly.
The following example shows how a function can return its local context, essentially the set of local variables, as an object.

.source of snippet `pack_1`

[source]
----
fn a {
    let h = 10
    let i = 20
    let j = 30
    let k = 40
    let l = 50
    let m = 60
    let n = 70
    let o = 80
    let p = 90
    let q = 100
    let r = pack()
    let s = 110
    return r
}
let h = a()
println $" $(h.h) $(h.i) $(h.j) $(h.k) $(h.l) $(h.m) $(h.n) $(h.o) $(h.p) $(h.q) $(h.s)"

----

.output of snippet `pack_1`
[source]
----
 10 20 30 40 50 60 70 80 90 100 110

----



The following example returns two different objects that share the same context.
It means that their fields are not only identical, but they are the same.
Changing a field in one will change the field in the other, even though the two objects are not the same, but their fields are.

.source of snippet `pack_2`

[source]
----
fn a {
    mut h = 10
    return [pack(),pack()]
}
let [x,y] = a()
println $"x.h = $(x.h)"
println $"y.h = $(y.h)"
x.h = 100
println "AFTER x.h = 100"
println $"x.h = $(x.h)"
println $"y.h = $(y.h)"
println $"are they the same? $(x === y)"

// field w is defined anew, not shared, inserted into x and y separately
x.w = "x.w"
y.w = "y.w"
println $"x.w = $(x.w)"
println $"y.w = $(y.w)"


----

.output of snippet `pack_2`
[source]
----
x.h = 10
y.h = 10
AFTER x.h = 100
x.h = 100
y.h = 100
are they the same? false
x.w = y.w
y.w = y.w

----



The following example demonstrates that context wrapping persists even after the context has been packed.
Through the returned object, you can access the closure's wrapped context, in this case, the variable `b`,
even though this is not a field in the object.

.source of snippet `pack_3`

[source]
----
let b=55
let a = {||
    let h = 10
    return pack()
}
let h = a()
println $"$(h.h) $(h.b)"
println keys(h)

----

.output of snippet `pack_3`
[source]
----
10 55
[me, h, .]

----



=== `parents`

Returns a list of the classes that are the direct parent classes of the argument.

.source of snippet `parents`

[source]
----
class A {}
class B : A {}
mut K = class : B {}
println([ ..parents(K) -> type(it()) ])

----

.output of snippet `parents`
[source]
----
[B]

----



Note that although `B` is a child class of `A`, the variable `K` contains an anonymous class that has only `B` as a parent.
The function `parent()` lists only `B` as a direct parent, and the list does not contain `A`.
If you need all the parents, you have to use the function `all_parents()`.


=== `que`

Create a Queue to communicate between threads.

This function will create a FIFO queue that different code fragments can write to and read from.
These fragments will probably run in different threads.

.source of snippet `que1`

[source]
----
let q = que(3) // create a queue that can hold 3 messages
println q.send("apple"), " apple is sent" // return value true means the message was sent
println q.send("birne"), " birne ist geschickt"
println q.try_send("peach"), " means ok"

// q.send("queue is full") // this would wait infinitely
// or until some other thread reads something from the queue and there is space to send

println q.try_send("fail"), " means no" // false means sending failed, the queue is full

// just print out the fruits we read from the queue
println q.receive()
println q.receive()
println q.receive()

// println q.receive() // this would wait infinitely, there is nothing in the queue
// or until some other thread puts something into the queue

// try_receive() returns none when there is nothing in the queue
println q.try_receive()," return none, there is nothing in the queue"

// it also returns none when there is a none in the queue
q.send(none)
println q.receive(), " was received... how do we know?"
// we send again a none
q.send(none);
// check that there is something in the queue
if q.has_next() : print "has_next() is true: it has value and it is "
// we can be sure the 'none' was coming as a message from the queue
println q.try_receive()

----

.output of snippet `que1`
[source]
----
true apple is sent
true birne ist geschickt
true means ok
false means no
apple
birne
peach
none return none, there is nothing in the queue
none was received... how do we know?
has_next() is true: it has value and it is none

----



Here is a complex example using this function:

.source of snippet `que`

[source]
----
// stochastic
/**
 * A printing function that will print the objects received through the que `q` till it gets closed.
 */
fn printer(q,@name){
    mut n = 0 // counts the number of not-ready que attempts
    while {
        mut s = q.try_receive(); // try to receive one or get none if there is nothing in the queue
        return { println "%s is done" % name } when q.is_closed(); // return when the que is closed
        if s == none {
            println "%s is not ready %s" % [name,n]
            n++ // count the attempts
            sleep 0.003 // sleep 3 mili
        } else {
            n = 0 // reset the attempts, we go an object to print
            println "%s is ready %s" % [name,s];
        }
    }
}

// create ONE queue that will be consumed by two
mut q : que = que(3)
// create two asynchronous task. they start immediately
mut task1 : task = async printer(q,name="task1")
mut task2 : task = async printer(q,name="task2")
// send the numbers 1 to 10 to the queue, whoever is lucky can get it
for i=1 ; i < 10 ; i++ {
    q.send(i);
    sleep 0.01
    }
println "closing the channel"
q.close();
println "channel is closed"
println "1 is done %s" % task1.is_done()
println "2 is done %s" % task2.is_done()

// get only the tasks into the list which are not done yet
mut task_list = [ task1, task2 ? !it.is_done() ]

println "we start to wait the first time"
await task_list

println "task1 done %s" % task1.is_done()
println "task2 done %s" % task2.is_done()
println "We start to wait the second time"
// have to use [] empty options, no timeout because a list literal stands after it
await[] [ task1, task2 ? !it.is_done() ]

println "task1 done %s" % task1.is_done()
println "task2 done %s" % task2.is_done()

none

----

.output of snippet `que`
[source]
----
task2 is ready 1
task1 is not ready 0
task2 is not ready 0
task2 is not ready 1task1 is not ready 1

task2 is not ready 2
task1 is not ready 2
task1 is not ready 3
task2 is not ready 3
task1 is ready 2
task1 is not ready 0
task2 is not ready 4
task1 is not ready 1
task2 is not ready 5
task2 is not ready 6task1 is not ready 2

task2 is ready 3
task1 is not ready 3
task2 is not ready 0
task2 is not ready 1
task1 is not ready 4
task1 is not ready 5
task2 is not ready 2
task2 is not ready 3task1 is ready 4

task1 is not ready 0
task2 is not ready 4
task1 is not ready 1
task2 is not ready 5
task1 is not ready 2
task2 is not ready 6
task1 is not ready 3
task1 is not ready 4
task2 is ready 5
task2 is not ready 0
task2 is not ready 1task1 is not ready 5

task2 is ready 6
task2 is not ready 0
task1 is not ready 6
task2 is not ready 1
task1 is not ready 7
task2 is not ready 2
task1 is not ready 8
task1 is not ready 9task2 is not ready 3

task2 is ready 7
task2 is not ready 0
task1 is not ready 10
task1 is not ready 11
task2 is not ready 1
task1 is not ready 12
task2 is not ready 2
task2 is not ready 3
task1 is ready 8
task1 is not ready 0
task2 is not ready 4
task1 is not ready 1
task2 is not ready 5
task1 is not ready 2
task2 is ready 9
task1 is not ready 3
task2 is not ready 0
task2 is not ready 1
task1 is not ready 4
task1 is not ready 5task2 is not ready 2

task1 is not ready 6
task2 is not ready 3
closing the channel
channel is closed
1 is done false
2 is done false
we start to wait the first time
task1 is done
task2 is done
task1 done true
task2 done true
We start to wait the second time
task1 done true
task2 done true

----



=== `quote`

The macro `quote` is an alias for the macro `thunk`.
The name is a complimentary addition to honour the functionality from LISP.


=== `reclose`

This function will "open" and reclose a closure in the current environment.

.source of snippet `reclose`

[source]
----
fn my_fun(@close_it) {
  mut s: str = "in function";
  return reclose(close_it)
}

mut s:str ="outer";
// this closure has a reference to 's' that contains "outer"
// it prints the value of the enclosed variable and then
// changes to a new value passed as argument
let closure = {|x| println s; s = x};
// after reclosing the new closure stored in 'reclosure' refers to the
//variable 's' in the 'my_fun' function
let reclosure = my_fun(close_it=closure);
// another reclosure will reference another 's'
// local variables are allocated dynamically
let rereclosure = my_fun(close_it=closure);

// prints the original "outer" and sets the variable to "closure 1"
closure("closure 1")
// prints "in function" and sets the 's' inside the function
reclosure("reclosure 1")
// prints "in function" and sets the 's' inside the second function
rereclosure("rereclosure 1")
println "s = %s" % s // prints "closure 1"

// we repeat the call sequence
closure("closure 2") // output is "closure 1"
reclosure("reclosure 2") // and "reclosure 1"
rereclosure("rereclosure 2") // and "rereclosure 1"
println "s = %s" % s // the new value is "closure 2"

----

.output of snippet `reclose`
[source]
----
outer
in function
in function
s = closure 1
closure 1
reclosure 1
rereclosure 1
s = closure 2

----



The function opens and "recloses" the closure in the context of the function call.
It will see and alter the `s` in that context of the re-closing, while the original closure alters and sees the global `s`.


=== `rng`

Returns a number range.

[source]
----
rng(start, end, step)
----
or

[source]
----
rng(start, end)
----

It is a read-only list containing all numbers from the start (inclusive) to the end (exclusive) with the given step.
It does not create an actual list, but instead provides access to the numbers in the range by index.
You can also iterate through the elements.

The following example demonstrates that you can have extensive ranges.

.source of snippet `rng`

[source]
----
let z = rng(0,9223372036854775807)
mut i = 0
for each k in z {
  print k, " "
  i++
} until i > 5

----

.output of snippet `rng`
[source]
----
0 1 2 3 4 5 
----



If you want a physical list with the elements, you can create it using the `..` spread operator:

.source of snippet `rngThousand`

[source]
----
mut z = rng(0,1000)
mut array = [ ..z ] // this creates an array of 1000 elements
println len(array)
println [..array[0 .. 5], "...", ..array[-5 .. inf]]

----

.output of snippet `rngThousand`
[source]
----
1000
[0, 1, 2, 3, 4, ..., 995, 996, 997, 998, 999]

----



In this case, `array` will be a list containing the elements we print out.
Avoid using this with too large numbers unless you want to test for out-of-memory errors.

You can index a number range, and you can also use a range to index another range:

.source of snippet `rngRange`

[source]
----
// a range does not store the numbers only where to start, end, and the step
mut z = rng(100,9223372036854775807,100)
// a range indexed is still only a range with a minimal memory footprint
mut range = z[20..2000000]
// the values come from the sub-range
print( [..range[0 .. 5], "...", ..range[-5 .. len(range)]])

----

.output of snippet `rngRange`
[source]
----
[2100, 2200, 2300, 2400, 2500, ..., 199999600, 199999700, 199999800, 199999900, 200000000]
----



In this case, you index the range without spreading it into a list.
We only spread the new range for printing.
The `range` object does not allocate nearly two million elements.
It is simply another number range.
At the end, the code allocates a list of ten elements for the printout.

The third parameter `step` is optional.
If not specified, it will be `+1`, `0`, or `-1` depending on the start and the end value.

If `start` is larger than `end`, the step value must be negative.
The default value is `-1` in this case.

If `start` is smaller than `end`, the step value must be positive.
The default value is `+1` in this case.

When the `start` and `end` values are the same, the range will

* be empty if the step is specified and is not zero (positive or negative), and
* contain the one single `start`/`end` value if the step is zero.

The default value for `step` is `0` in this case.


If specified, it must not be zero, and it has to be positive or negative based on the relations between the start and end value.
Essentially, stepping has to approach the end value.

.source of snippet `rngErrs`

[source]
----
try:
    rng(1,10,-1)
catch e1:
    println e1.message
try:
    rng(10,1,+1)
catch e2:
    println e2.message
try:
    rng(10,11,0)
catch e3:
    println e3.message

println "Special zero element range can have zero as step %s" % [rng(10,10,0)]
println "but it is just the same as %s" % [rng(10,10)]


----

.output of snippet `rngErrs`
[source]
----
rng(1,10,-1) would never end
rng(10,1,1) would never end
rng(10,11,0) has zero step size
Special zero element range can have zero as step [10]
but it is just the same as [10]

----



It is recommended not to use the `rng()` function directly.
You can import the system module `range` that defines a more flexible name:

.source of snippet `range`

[source]
----
// import the Turicum range function to get named parameters for range
sys_import turi.range

// start from zero
// also note ranges print wisely, when they are long
println range(end=10)
// all elements are printed when not more than five
println range(end=5)
println range(start=100, 200, 50)
println range(end=200, 100, 50)
// range object has one field: 'length'
println "length=",range(end=200, 100, 50).length
// you can also index a range object, it will not allocate memory
println "r[1]=",range(end=200, 100, 50)[1]
println range(100,100)
println range(100,100,step=0)

----

.output of snippet `range`
[source]
----
[0, 1, 2,..., 8, 9]
[0, 1, 2, 3, 4]
[100, 150]
[100, 150]
length=2
r[1]=150
[100]
[100]

----




=== Regular Expressions

Regular expression handling is implemented with two primitive functions `pass:[_rx()]` and `pass:[_rx_match()]`.
They start with the `pass:[_]` character to signal that they are not to be used directly.
Instead, the code has to import the system file `re`:

.source of snippet `re`

[source]
----
sys_import "turi.re"

fn print_match(m){
    for i=0 ; i < len(m.group) ; i++ {
        mut {index, start, end } = m.group[i];
        println "%s. '%s'.substring(%s,%s)='%s'" % [index,s,start, end,s[start..end]];
    }
}

mut s = "abrakadabra";
mut rx =Re("a(b)ra(ka)(dabra)")
mut m = rx.match(s);
println m
print_match(m);

s = "xxx"+s+"yyy";
m = rx.match(s);
if m.group == none :
    println "does not match because of xxx and yyy"
println m, " is an empty object"
m = rx.find(s);
if m.group != none :
    println "matches because we find and not match"
print_match(m);

----

.output of snippet `re`
[source]
----
{start: 0, name: {}, end: 11, group: [{start: 1, index: 1, end: 2}, {start: 4, index: 2, end: 6}, {start: 6, index: 3, end: 11}]}
1. 'abrakadabra'.substring(1,2)='b'
2. 'abrakadabra'.substring(4,6)='ka'
3. 'abrakadabra'.substring(6,11)='dabra'
does not match because of xxx and yyy
{} is an empty object
matches because we find and not match
1. 'xxxabrakadabrayyy'.substring(4,5)='b'
2. 'xxxabrakadabrayyy'.substring(7,9)='ka'
3. 'xxxabrakadabrayyy'.substring(9,14)='dabra'

----



You can create a regular expression object using `Re(pattern)` or `Re(pattern, flags)`.
It has two methods:

* `match()` to match the whole string passed as argument, and

* `find()` to find a matching substring in the string.

If there is no match, the return value is an empty object.
If there is a match, then the return value is an object containing the matching groups.

Even if there are no matching groups, the object is not empty as it will have an empty list field named `group`.

You can also use named arguments that will get into the field `name`.
In the example above that field was empty.
In the following example we have named arguments:

.source of snippet `re2`

[source]
----
sys_import "turi.re"

mut m = Re("a(?<firstb>b)ra(?<twoletter>ka)(dabra)").match("abrakadabra");
println "the ks is '%s'" % [m.name.twoletter]

----

.output of snippet `re2`
[source]
----
the ks is '{start: 6, index: 3, end: 11}'

----



Each named argument contains the start, end, and the index of the named group.

The optional `flags` argument is a string. Each character enables a Java regular-expression
option:

* `i` enables case-insensitive matching,
* `m` enables multiline matching,
* `s` enables dot-all mode,
* `u` enables Unicode-aware case folding,
* `x` enables comments and whitespace in the pattern,
* `c` enables canonical equivalence,
* `d` enables Unix-lines mode,
* `l` treats the pattern as a literal string, and
* `U` enables the Unicode character class mode.

=== `set_caller`, `set_caller_force`

This function sets the variable in the caller context.

This is particularly useful when implementing a decorator that redefines a function or class,
especially when the decorator operates in a different context than the decorated function.

Calling this function from inside another function can redefine a variable in the context of the calling function or its closure.

The `*_force` version overwrites the value even when the variable is pinned.

.source of snippet `fnDecorator5`

[source]
----
fn fun_decorator(^closure){
        set_caller_force(closure.name,
            {|| println("We start the fun"); closure(); println("The fun is over")}
           )
    }

{
    @fun_decorator()
    fn q(){
      println("We function");
    }

    q()
}

----

.output of snippet `fnDecorator5`
[source]
----
We start the fun
We function
The fun is over

----



Note that the example uses the `_force` version of the function.
Named functions assign the function to a symbol, such as `q` in this example, and then pin these symbols.
It is an error to define a function in a context where it was already defined.

=== `set`, `set_force`

Set the value of a variable in the local context.
This function is handy for meta-programming when the variable's name is available as a string.
An example is in the sample:

.source of snippet `fnDecorator3`

[source]
----
mut fun_decorator={|^closure|
        set_force(closure.name,
            {|| println("We start the fun"); closure(); println("The fun is over")}
           )
    }

@fun_decorator()
fn q(){
  println("We function");
}

q()

----

.output of snippet `fnDecorator3`
[source]
----
We start the fun
We function
The fun is over

----



It uses this function to decorate a function and then set the variable of the same name, overwriting the definition with a new, decorated closure.

Note that the example that way works only when the decorating closure is defined in the same context as the decorated function or in a subcontext.
This way, the closure sees the function symbol `q` and it can redefine it.
A more robust approach to creating decorators involves calling `set_caller_force`.

The `*_force` version overwrites the value even when the variable is pinned.



=== `set_global`

It is similar to `set()`, but it sets the value of a global variable.
Note that there is no `*_force` version of this function.
Pinning is always local, and this method works directly on the global variables.



=== `signature`

The built-in function `signature` returns an object that describes the signature of the argument function.
You pass a function, macro, or closure to this function, and the return value is the signature object.
The object is classless and has the following fields:

* `name` is the name of the function, closure, macro or `none` if it has no name
* `return_type` is a list of return type strings
* `parameters` is a list of parameters. Each element is an object with the following fields:
** `identifier` is the name of the parameter
** `named` is `true` if the parameter can be specified by name, `false` otherwise
** `positional` is `true` if the parameter can be specified by position, `false` otherwise
* `types` is a list of types. Each element is a string or a command object
* `default` is the default value expression command object or `none`.
* `rest` is the name of the `[rest]` parameter or `none`
* `meta` is the name of the `{meta}` parameter or `none`
* `closure` is the name of the `^closure` parameter or `none`


.source of snippet `signatura1`

[source]
----
fn f (a:num=53,[reliquum],{nominati},^clausura):any {
    return a + b + c
}

println jsonify_beauty(signature(f))

----

.output of snippet `signatura1`
[source]
----
{
    "return_type": [ "any"],
    "rest": "reliquum",
    "meta": "nominati",
    "name": "f",
    "closure": "clausura",
    "parameters": [
                      {
                          "identifier": "a",
                          "default": 53,
                          "types": [ "num"],
                          "named": true,
                          "positional": true
                      }
                  ]
}

----




=== `sleep`

Sleep pauses the execution.

`sleep(N)` will sleep approximately `N` seconds.
`N` is an integer or a floating point value.
The sleep time does not need to be whole seconds, but the precision is not finer than 0.001 seconds.

The return value is the actual number of seconds spent with millisecond precision.
It is the same as the argument or a bit more.

.source of snippet `sleep`

[source]
----
// stochastic
println sleep(0.001) // sleep one milli

----

.output of snippet `sleep`
[source]
----
0.002

----



This function is implemented by calling the Java JDK method `Thread.sleep()`.
If the specified time is negative, the function does not call the JDK sleep method.
In all other cases, a `Thread.sleep()` call is invoked, with all multi-thread synchronization effects.


=== `source_directory`

This function returns the directory as a string of the source file or the compiled file that is currently being executed.
It may return `none` if the environment executes a script that lacks a specified file.

It is NOT the script name.
It is not the current working directory.
This is the directory where the Turicum program is located.


=== `str`

Converts the argument to a string.

Similar to the `to_string()` method, but it works safely on any value, and in some situations, it may be more readable.
This function converts the value `none` to `pass:["none"]`.

=== `sys_import`

This command can be used to import Turicum system files.

The Turicum run-time and other libraries supporting a Turicum program can deliver Turicum source code files in the Java resource path.
Using this macro, you can import those files as if they were in a directory listed in the APPIA path.

The Turicum run-time itself provides such files

* to handle Maven POM files when Turicum is used for Maven projects,
* support debugging Turicum programs,
* io support,
* measurement units,

and some others.

You can use `sys_import` in the same way as `import`,
either specifying only the name of the import resource,
or specifying the name and the list of imported symbols.

Use `.` as a separator when importing from a package.
For example, the Turicum run-time in the Turicum JAR file is in the directory `classes/turi`.
If you want to import the resource from the file `appia.turi` you can do it as

    sys_import turi.appia

In this case, `turi` at the start is the directory name inside the JAR file under `classes`, and `appia` is the file name.
The extension `.turi` is appended automatically in the same way as for the `import` command.


=== `thunk`

This macro simply returns it's argument.
Because this is a macro, it means that the argument is NOT evaluated.

The name is the past tense o


=== `time`

Returns the current time in milliseconds,
the difference, measured in milliseconds, between the current time and midnight, January 1, 1970 UTC.

Note that while the unit of time of the return value is a millisecond,
the granularity of the value depends on the underlying operating system and may be larger.
For example, many operating systems measure time in units of tens of milliseconds.


=== `try_yield`

Try to get an object from the task's que.
It returns `none` if currently there is no receivable object, the object is `none` or the que is closed.


=== `http_client`


=== `atomic`

Creates an atomic cell holding a single value. The value can only be accessed through the
synchronized operations of the cell, so concurrent threads can share it safely:

[source]
----
let counter = atomic(0)          // any value can be stored, not only numbers
counter.incr()                   // atomic increment, returns the new value
counter.add(10)                  // atomic addition
counter.get()                    // lock-free read
counter.set(0)                   // replace the value
counter.update({|x| x * 2})      // atomically replace the value with f(value)
counter.cas(0, 42)               // set to 42 if the value equals 0; returns true/false
----

The argument of `atomic()` is the initial value; called without an argument the cell starts
holding `none`. Lists and objects stored in the cell are pinned: the cell holds and hands out
immutable snapshots, to change the value build a new one and `set()` or `update()` it in.

The function given to `update` runs exactly once, while holding the cell's internal lock. Do not
perform blocking operations or use other atomic cells inside it.


=== `mutex`

Creates a new mutual exclusion lock to be used with the `sync` command:

[source]
----
let l = mutex()
sync l {
    // only one thread at a time executes here
}
----

The returned object also provides the methods `lock()`, `unlock()`, `try_lock()`,
`try_lock(seconds)`, `is_locked()` and `is_held()` for advanced use. When these methods are used
directly, the program is responsible for releasing the mutex it acquired; the `sync` command is
the recommended form because it guarantees the release even on errors and thread abortion.


=== `http_server`



=== `type`

This method returns the type of the argument as a string.

.source of snippet `type1`

[source]
----
mut s = "13"; mut i = 13; mut d = 3.14;

mut err
try: z = 6.55
catch ex: err=ex
class A {}

println "\"13\" ", type("13")
println "13 ", type(13)
println "3.14 ", type(3.14)
println "err ",type(err)
println "A() ",type(A())
println "[1,2,3] ",type([1,2,3])
println "A ",type(A)
println "fn()=3 ",type(fn()=3)
println "{|| 3} ",type({|| 3})
println "que(1) ",type(que(1))
println "async ",type(async 1)
println "macro ",type(macro(fn()=3))
println "none ",type(none)
println "java object ", type(java_object("java.lang.Object"))


----

.output of snippet `type1`
[source]
----
"13" str
13 int
3.14 float
err err
A() A
[1,2,3] lst
A cls
fn()=3 fn
{|| 3} fn
que(1) que
async task
macro macro
none none
java object java.java.lang.Object

----



When the argument is a Java object, then the type is `java.` + the canonical name of the class.


=== `uncurry`

Uncurries a curried function or closure.
The return value is the original function, closure, or a less curried version of that.

The first argument of the function is the curried function or closure.
If there is no second argument, then all curried parameters will be removed.

If there is a numerical second argument, then that many arguments will be uncurried as specified.

=== `unlet`

Unlet is a macro that accepts one identifier and deletes it from the local context.
It is an error if the variable is not defined in the local context or is defined but not in the local context.

This macro was designed for a special purpose: to remove some of the `init` arguments from the object.
By default, the arguments to the init method become fields of the object, and they are also mutable.
This differs significantly from the parameters of other functions.
Some of the arguments you may not want to become fields.

A typical example is when you want to "clone" another object inheriting the field,
  but you do not want the object to be a field by itself in the initialized new object.

For example:

.source of snippet `unlet`

[source]
----
class Z {
    fn init(obj){
        for each t in keys(obj){
            this[t] = obj[t]
        }
        // arguments to `init` are automatically fields
        // we do not need this extra field here
        unlet obj;
    }
}
mut z = Z({
    a: {
        B : "abraka"
        },
    c : ""
    });
println z

----

.output of snippet `unlet`
[source]
----
{a: {B: abraka}, c: , this: {...}, cls: class Z}

----



Without the call to `unlet`, the class would also have the variable `obj` as a field.





=== `unthunk`

Execute a command object in the current context.
The command object is probably the result of a `thunk()` call.
Note that `quote()` is an alias for `thunk()`.
Using the unary `'` operator is similar to `thunk()/quote()` but not exactly the same.
For more information, see the documentation of the `'` operator.

.source of snippet `thunk_unthunk`

[source]
----
// thunk is a macro, gets the block before it evaluates, and it does not evaluate it
mut z = thunk({mut x = 3; println "hello"})
println "before eval"
// unthunking will evaluate the block, as a block
unthunk(z)
println "after eval"
// 'x' was defined as a mutable variable inside the block, so here it is undefined
println is_defined(x) // will print false
// this will evaluate the block unwrapped
unthunk(unwrap(z))
// the block was unwrapped, so 'x' is defined in this context
println is_defined(x) // will print true and
println x             // then the value

----

.output of snippet `thunk_unthunk`
[source]
----
before eval
hello
after eval
false
hello
true
3

----




=== `unwrap`

Unwrap a block command.

When a block command (commands between `{` and `}`) is executed it opens a new context for the execution.
The new context wraps the surrounding context.
Any symbol defining and altering command, like `let`, `mut`, `global`, `pin` are executed in this new wrapping context.
It also means, that these changes are local for the command block.
Variables pinned are not pinned outside, variables defined in it are not defined outside (or they have their wrapped old definition).

The function `unwrap` returns a block of commands that is unwrapped.

.source of snippet `unwrap1`

[source]
----
let a = 1; // defined 'a' outside

// wrapping block
{
    let a = 12; // redefining 'a'  for the local use
    let b = 77; // defining 'b' valid only  inside
}

println "AFTER THE WRAPPING BLOCK"
println $"a is $a"
println $"b is not defined is_defined(b) is ${is_defined(b)}"

unwrap(thunk({
    try: let a = 12; // redefining 'a' will raise error
    catch e: println "redefinition is not possible non-wrapped"
    let b = 77; // defining 'b' valid outside when unwrapped
})).execute()

println "AFTER THE UNWRAPPING BLOCK"
println $"a is $a"
println $"b is not defined is_defined(b) is ${is_defined(b)}"
println $"b is $b"

// you can get the same result using an expression with a non-wrapping part
!(
    try: let a = 12; // redefining 'a'  will raise error
    catch e1: println "redefinition is not possible non-wrapped"
    let k = 78; // defining 'k' valid also outside
)

println "AFTER THE UNWRAPPING EXPRESSION"
println $"a is $a"
println $"k is $k"


----

.output of snippet `unwrap1`
[source]
----
AFTER THE WRAPPING BLOCK
a is 1
b is not defined is_defined(b) is false
redefinition is not possible non-wrapped
AFTER THE UNWRAPPING BLOCK
a is 1
b is not defined is_defined(b) is true
b is 77
redefinition is not possible non-wrapped
AFTER THE UNWRAPPING EXPRESSION
a is 1
k is 78

----





=== `xml_format`

Format an XML document.

The argument of the function must be a string representing the XML document to be formatted.
The function returns a formatted XML string with proper indentation and structure.

The indentation is set to 4 spaces.
The character encoding is set to UTF-8.

If the argument is not a valid XML document, the function will raise an error.

.source of snippet `xml_format`

[source]
----
println xml_format("""<a>b<marnacle xmlns="hupsi"></marnacle> </a>""")

try xml_format("<a>b<") catch e: println e

----

.output of snippet `xml_format`
[source]
----
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<a>
    b
    <marnacle xmlns="hupsi"/>
</a>

There was an XML exception formatting XML

----




=== `yield()`, `try_yield()`, `yield_is_closed()`

When a task is started asynchronously, there is always a channel between the parent thread and the child thread.
You can create multiple channels (`que` objects), allowing different threads to read and write them, which creates a complex mesh of messaging.
However, one channel is always created when a task is started asynchronously, and this channel is tied to the task.

The parent task starting thread can send objects into this channel using the `send()` method on the task object, and the asynchronous thread can get the objects sent by the parent thread by using `yield()`.

`yield()` will wait infinitely for an object and return one when one arrives.
If the queue from the parent thread is already closed, then `yield()` will throw an exception.

NOTE: `yield` is an operator and not a function, and it is used in other contexts with slightly different semantics also.

`try_yield()` is more lenient.
It does not wait.
It returns `none` if there is no message or the channel is closed.

`yield_is_closed()` can be used to test whether the queue is open from the parent to the child thread.

.source of snippet `try_yield`

[source]
----
// stochastic
fn printer(){
    mut n = 0
    while {
        mut s = try_yield();
        return { println "we are done" } when yield_is_closed();
        if s == none {
            println "not ready %s" % n
            n++
            sleep 0.003
        } else {
            n = 0
            println "received %s" % s;
        }
    }
}

mut task : task = async printer()
for i=1 ; i < 4 ; i++ {
 println "sending ",i
 task.send(i);
 sleep 0.009
 }
println "closing the channel"
task.close();
println "channel is closed"
println "is done %s" % task.is_done()
await task
println "is done %s" % task.is_done()

none

----

.output of snippet `try_yield`
[source]
----
sending 1
not ready 0
received 1
not ready 0
not ready 1
sending 2
received 2
not ready 0
not ready 1
not ready 2
not ready 3
sending 3
received 3
not ready 0
not ready 1
not ready 2
closing the channel
channel is closed
is done false
we are done
is done true

----



Note that the output of this sample is not deterministic.
The sending happens __approximately__ every 9ms, and the reading is retried __approximately__ every 3ms.



=== `file_read`

Reads the whole content of a file.

   fn file_read(path: str, @binary: bool = false, @charset: str = "UTF-8") -> str | bin

* `path` is the file to read.
In a sandbox with configured file roots, a relative path is resolved against the roots in
declaration order, taking the first existing candidate; the resolved file must lie under one
of the roots.

* `binary` selects binary mode: the result is a `bin` with the exact bytes of the file.
In text mode (the default) the result is a `str` decoded using `charset`.

* `charset` is the character set used to decode the file in text mode.

The function requires the `FILE_READ` capability.


=== `file_lines`

Reads a text file as a list of lines.

   fn file_lines(path: str, @charset: str = "UTF-8") -> lst

The result is a `lst` of `str`, one element per line, without the line terminators. Path
resolution and sandbox confinement are the same as for `file_read`.

The function requires the `FILE_READ` capability.


=== `file_write`

Writes the whole content of a file.

   fn file_write(path: str, content,
                 @append: bool = false,
                 @overwrite: bool = true,
                 @mkdirs: bool = false,
                 @binary: bool = false,
                 @charset: str = "UTF-8") -> none

* `path` is the file to write.
In a sandbox, a relative path resolves against the first declared read-write root, and the
resolved file must lie under a read-write root.

* `content` is the data to write.
In text mode anything is accepted and its string form is encoded using `charset`; in binary
mode the content must be a `bin`, written verbatim (convert a string explicitly, e.g. with
`s.bytes()`).

* `append` appends to the file instead of replacing its content.

* `overwrite` set to `false` makes writing an *existing* file an error; the file is created
atomically, so there is no race between the existence check and the creation.
It is ignored when `append` is set.

* `mkdirs` creates the missing parent directories of the target.

* `binary` selects binary mode.

The function requires the `FILE_WRITE` capability; when the target file does not exist — or
`mkdirs` has to create parent directories — it additionally requires `FILE_CREATE` at
runtime.


=== `file_exists`, `is_file`, `is_dir`

File-system tests.

   fn file_exists(path: str) -> bool
   fn is_file(path: str)     -> bool
   fn is_dir(path: str)      -> bool

`file_exists` returns whether the path names an existing entry; `is_file` whether it is a
regular file; `is_dir` whether it is a directory. Symbolic links are followed. Path
resolution and sandbox confinement are the same as for `file_read`; a relative path that
exists under none of the configured roots simply yields `false`.

The functions require the `FILE_READ` capability.


=== `file_stat`

Returns the metadata of a file-system entry as an object.

   fn file_stat(path: str) -> obj

The result object has the fields

* `size` — the size in bytes,
* `modified`, `created` — the modification and creation times as epoch milliseconds,
* `is_file`, `is_dir`, `is_symlink` — the kind of the entry (`is_symlink` refers to the path
itself; the other fields follow the link),
* `readable`, `writable` — whether the hosting process may read/write the entry.

The function requires the `FILE_READ` capability.


=== `mkdir`

Creates a directory.

   fn mkdir(path: str, @recurse: bool = true) -> none

With `recurse` (the default), missing parent directories are created as well, and an already
existing directory is not an error. With `recurse=false`, exactly one directory is created;
a missing parent or an existing target is an error. In a sandbox, the path must lie under a
read-write root.

The function requires the `FILE_CREATE` capability.


=== `file_delete`

Deletes a file or a directory.

   fn file_delete(path: str, @force: bool = false, @must_exist: bool = false) -> bool

The function returns whether something was deleted. Deleting a non-empty directory without
`force` is an error; with `force=true` the directory tree is deleted recursively —
the most destructive primitive of the file family, guarded by the `FILE_DELETE` capability
and, in a sandbox, by the read-write root confinement. With `must_exist=true`, a missing
path is an error instead of a `false` result.

The function requires the `FILE_DELETE` capability.


=== `file_copy`

Copies a file.

   fn file_copy(src: str, dst: str, @overwrite: bool = false) -> none

Copies the file `src` to `dst`. When `dst` exists, it is replaced only with
`overwrite=true`; without it, an existing target is an error. In a sandbox, `src` must lie
under a file root (read-only or read-write) and `dst` under a read-write root.

The function requires the `FILE_READ` capability; at runtime it additionally requires
`FILE_WRITE` when the target exists and `FILE_CREATE` when it does not.


=== `file_move`

Moves (renames) a file.

   fn file_move(src: str, dst: str, @overwrite: bool = false) -> none

Moves the file `src` to `dst`. When `dst` exists, it is replaced only with `overwrite=true`.
Moving always removes the source, so in a sandbox both `src` and `dst` must lie under a
read-write root.

The function requires the `FILE_DELETE` capability (it always removes the source); at
runtime it additionally requires `FILE_WRITE` when the target exists and `FILE_CREATE` when
it does not.


=== `tmp_file`, `tmp_dir`

Creates a temporary file or directory in the session's scratch area.

   fn tmp_file(@prefix: str|none = none, @suffix: str|none = none) -> str
   fn tmp_dir(@prefix: str|none = none) -> str

Both functions return the absolute path of the freshly created entry as a string. The
scratch area is a per-session directory created on first use; it acts as an additional
read-write file root, so every file built-in can operate on the entries under it — and it is
deleted, with everything in it, when the session ends.

The functions require the `FILE_TEMP` capability. Granting `FILE_TEMP` implies the whole
file capability family (`FILE_READ`, `FILE_WRITE`, `FILE_CREATE`, `FILE_DELETE`); there is
no use case for temp files that can be created but not read and written.


=== `file_reader`

Opens a file for sequential reading and returns a read handle.

   fn file_reader(path: str, @binary: bool = false, @charset: str = "UTF-8") -> handle

The handle has the methods

* `read_line()` — the next line without its terminator, `none` at the end of the file
(text mode only);
* `read(n)` — up to `n` characters as a `str` (in binary mode: up to `n` bytes as a `bin`),
`none` at the end of the file;
* `read_all()` — the rest of the stream as one `str` (binary mode: `bin`);
* `close()` — closes the handle; idempotent;
* `entry()` / `exit(e)` — the `with` protocol, so the handle can be used in a
`with file_reader(...) as f { ... }` command, which closes it on every exit path.

Every operation on a closed handle is an error (except `close`). A handle that is never
closed is force-closed when the session ends. Path resolution and sandbox confinement are
the same as for `file_read`.

The function requires the `FILE_READ` capability.


=== `file_writer`

Opens a file for sequential writing and returns a write handle.

   fn file_writer(path: str,
                  @append: bool = false, @overwrite: bool = true,
                  @mkdirs: bool = false,
                  @binary: bool = false, @charset: str = "UTF-8") -> handle

The options have the same meaning as for `file_write`. The handle has the methods

* `write(x)` — writes the string form of `x` encoded with `charset` (in binary mode `x`
must be a `bin`, written verbatim);
* `write_line(x)` — `write(x)` followed by a newline (text mode only);
* `flush()` — flushes the buffered output;
* `close()` — flushes and closes; idempotent;
* `entry()` / `exit(e)` — the `with` protocol:

   with file_writer("report.txt") as out {
       out.write_line("header")
       for each line in file_lines("data.txt") {
           out.write_line(line)
       }
   } // closed here, also on exception

Every operation on a closed handle is an error (except `close`). A handle that is never
closed is force-closed when the session ends.

The function requires the `FILE_WRITE` capability; creating a missing target file — or
parent directories with `mkdirs` — additionally requires `FILE_CREATE` at runtime.


=== `file_random_reader`

Opens a file for random-access reading and returns a read handle.

   fn file_random_reader(path: str) -> handle

Random access is inherently byte-oriented: positions and sizes are byte offsets, there is no
text mode, and reads return `bin` values. The handle has the methods

* `position()` — the current position;
* `seek(pos)` — sets the position; a position past the end of the file is legal, reads
there report the end of the file;
* `size()` — the current file size in bytes;
* `read(n)` — up to `n` bytes as a `bin` from the current position, advancing it; `none` at
the end of the file;
* `read_at(pos, n)` — absolute-positioned read; does *not* move the position;
* `close()` — idempotent;
* `entry()` / `exit(e)` — the `with` protocol.

A handle that is never closed is force-closed when the session ends. Path resolution and
sandbox confinement are the same as for `file_read`.

The function requires the `FILE_READ` capability.


=== `file_random_editor`

Opens a file for random-access reading and in-place updating.

   fn file_random_editor(path: str, @create: bool = false, @truncate: bool = false) -> handle

* `create` creates the file when it is missing (this needs the `FILE_CREATE` capability at
runtime); without it, a missing file is an error.

* `truncate` empties the file when it is opened.

The handle has every method of the `file_random_reader` handle — the editor can also read;
write implies read — plus

* `write(x)` — writes the `bin` `x` at the current position, advancing it; writing past the
end of the file extends it;
* `write_at(pos, x)` — absolute-positioned write; does *not* move the position;
* `truncate(size)` — cuts the file to `size` bytes (the position clamps to the new end);
* `flush()` — forces pending writes to the file.

Writes accept `bin` values only; convert a string explicitly, e.g. with `s.bytes()`. In a
sandbox, the path must lie under a read-write root.

The function requires the `FILE_WRITE` capability.


=== `file_map_reader`

Maps a file — or a region of it — into memory for reading.

   fn file_map_reader(path: str, @offset: int = 0, @length: int|none = none) -> handle

`length=none` maps from `offset` to the end of the file. The handle has the methods

* `length()` — the length of the mapped region in bytes;
* `get(pos, n)` — `n` bytes at offset `pos` within the mapping, as a `bin`;
* `close()` — invalidates the handle;
* `entry()` / `exit(e)` — the `with` protocol.

CAUTION: On Java 21 there is no deterministic unmap: `close()` invalidates the handle, but
the mapped pages are released only when the buffer is garbage collected. On Windows, the
file cannot be deleted or truncated while it is mapped. The total length of all live
mappings of a session is capped by the sandbox policy's `maxMappedBytes` — the untrusted
default is `0`, so an untrusted script can map files only when the host opts in.

The function requires the `FILE_READ` capability.


=== `file_map_editor`

Maps a file — or a region of it — into memory for reading and writing.

   fn file_map_editor(path: str, @offset: int = 0, @length: int|none = none) -> handle

`length=none` maps from `offset` to the end of the file; an explicit `length` may extend the
file to `offset+length` bytes. The handle has every method of the `file_map_reader` handle,
plus

* `put(pos, x)` — writes the `bin` `x` at offset `pos` within the mapping; it cannot grow
the mapping;
* `flush()` — pushes the changes to the file.

`exit` flushes and closes, so a `with file_map_editor(...) as m { ... }` block persists its
changes on every exit path. Writes accept `bin` values only. The Java 21 unmap caveat and
the `maxMappedBytes` cap of `file_map_reader` apply here as well. In a sandbox, the path
must lie under a read-write root.

The function requires the `FILE_WRITE` capability.







=== Setting values in Contexts

Currently, there are three 'set' functions to update the value of a variable when the name of the variable is available in a string.

* `set`, `set_force`
* `set_global`
* `set_caller`, `set_caller_force`

The use of these functions should be considered only for special meta-programming cases like writing decorators.







=== Elephant functions: `thunk`, `unthunk`, `unwrap`, `block_list`

==== `thunk`

The `thunk` macro returns its argument unevaluated, preserving it as a command or expression.
You can also use the macro `quote`, which is an alias and does the same as `thunk`, or use the `'` (single apostrophe) operator.
This deferred computation can later be executed using the function `unthunk` or the `execute` method defined on the resulting command object.

`unthunk` and `execute` evaluate the stored command in the **context where `unthunk` is called**, not the context where the thunk was created.

.difference from `evaluate`
[NOTE]
====
Although `evaluate` provides similar functionality, it differs in two key ways:

- `evaluate` can only be used **inside a macro**, or in a function or closure.

- It evaluates the argument in the **caller’s context**, i.e., the context of the macro's caller.
====

Note also that the argument to `thunk`, `quote`, or the operator `pass:[']` can only be an expression and not an arbitrary command.
That is a restriction imposed by Turicum syntax.
However, you can enclose any command in a block, and then you can use the block as an expression or part of an expression.

==== `unthunk`

When you `unthunk` a block it will be evaluated in a local context freshly created for the block execution.
Although this context wraps the surrounding context, locally defined variables will not be visible after the execution of the block.
To overcome this limitation and execute the commands like they were literally written where the `unthunk` call is made you can `unwrap` a block.

.block wrapping explanation
[NOTE]
====
A block can be wrapping and non-wrapping.
Most of the blocks in Turicum are wrapping.
When you execute a wrapping block, the execution starts in a new context.
This context wraps the surrounding context, thus the code can access and modify the variables defined there, but any locally defined variable will only be available in the block and will disappear when the block is finished and the context is not used anymore.

A non-wrapping block executes in the surrounding context.
Any variable defined in the block will also be available outside.
An example of a non-wrapping block is an object initializer.
The body of the `init` method is executed in the surrounding context.
This is the object context in this case.
Any local variable defined in the `init` method becomes a field.

`unwrap` takes a block command as its argument, typically the result of a `thunk` call, and returns a non-wrapping copy of the block.
====

The following example shows how these functions work.

.source of snippet `thunk_unthunk`

[source]
----
// thunk is a macro, gets the block before it evaluates, and it does not evaluate it
mut z = thunk({mut x = 3; println "hello"})
println "before eval"
// unthunking will evaluate the block, as a block
unthunk(z)
println "after eval"
// 'x' was defined as a mutable variable inside the block, so here it is undefined
println is_defined(x) // will print false
// this will evaluate the block unwrapped
unthunk(unwrap(z))
// the block was unwrapped, so 'x' is defined in this context
println is_defined(x) // will print true and
println x             // then the value

----

.output of snippet `thunk_unthunk`
[source]
----
before eval
hello
after eval
false
hello
true
3

----



== Preprocessing

The syntax analyzer works on the token list.
When it encounters a `#` symbol, it starts preprocessing for the remainder of the token list.

The steps of the preprocessing are the following:

. First of all, it parses the command following the `#` symbol.
It has to be a macro, a function, or a closure.
This is the preprocessor command.

. After that, it removes all tokens from the processed token list.
This includes the tokens just parsed after the `#` character.

. The parser creates an interpreter context to execute the preprocessor command.
It is the preprocessor context.
This context is created and reused by the analyzer if multiple preprocessors are executed one after the other.

. The analyzer executes the commands just parsed using the preprocessor context.
This will create global variables, functions, and classes.

. It executes the command, passing the list of tokens as the only argument.
That is why the command has to be a macro, a function, or a closure.
The command should read the token list and result in a list.
This list may contain the original tokens, newly created tokens, and strings.

. It removes all the tokens from the input and replaces them with the token list returned by the preprocessor.

. If there are non-token-list elements, it parses them and converts each into a list of tokens.

The analyzer uses this new token list to analyze the program code.
The commands created up to this point are erased from the program being built.
They served only the purpose of the preprocessor.

While the remaining code is analyzed, the interpreter will use the already existing preprocessor context in subsequent preprocessing steps.

The following example shows preprocessing that adds lines defining

* a global variable `M`,

* defines a function, and

* sets the value of the variable `M` to `"hello hello"`.

.source of snippet `preprocessorSample`

[source]
----
fn hi(){ // this will be deleted, it only exists in the preprocessor context
  "hello "
}
// the next line starts the preprocessor
# {|lexes|
["""
    // this is a string that will lead the new lexical token list
    mut M:str = ""
    fn helvetic_hi(){
      "Gruetzi "
    }
    fn hi() { "hullo" }
    M = M + "hello ";
    M = M + (hi() or "none ");
""",
// and just mut's have the rest of the tokens
..lexes]
} // end of the preprocessor command

// this is preprocessed
M = M + helvetic_hi() + "ant hunter"
print M

----

.output of snippet `preprocessorSample`
[source]
----
hello hulloGruetzi ant hunter
----



The example above is quite contrived.
Preprocessors are usually defined in imported files and invoked using a simple function call.

== Language Grammar (Appendix)

=== Full Annotated EBNF

The following is the non-standard EBNF definition of the programming language called Turicum.

* Strings between ' characters are literal keywords.
* Syntax between `[` and `]` are optional
* Syntax between `+{+` and `+}+` are repeated zero or more times
* Syntax between `(` and `)` is a group with alternatives
* Alternative syntaxes are separated using the `|` character.
* Non-terminal symbols are all capital letters.
* EBNF non-terminals are followed by `::=` and then the definition closed by `;`

* Non-literal terminal symbols are:

** IDENTIFIER
** number
** string

A program is a series of commands.
Commands can be separated by `;` character, and it is recommended to use it, but it is not required.
A newline can replace the use of `;`.
The recognition of the command end upon new-line is heuristic.
The safe way is to use `;` to close commands.


[source]
----
     PROGRAM ::= { COMMAND [;] } ;

----


A program is an ordered list of commands optionally separated by `;`.

Commands have values.
It is the __result__ of the command.


[source]
----
  COMMAND ::=  KEYWORD_COMMAND |
  ASSIGNMENT |
  FUNCTION_CALL |
  EXPRESSION |
  ; //empty command

----



[source]
----
      KEYWORD_COMMAND ::=  IF |
      LOOP |
      WITH |
      FLOW |
      SYNC |
      FN |
      CLASS |
      PIN |
      GLOBAL |
      BREAK |
      CONTINUE |
      DIE |
      RETURN |
      YIELD |
      PRINT |
      PRINTLN |
      TRY |
      MUT |
      LET
      ;

----



   LOOP ::= ( FOR | FOR_EACH ) [ 'until' EXPRESSION];


[source]
----
  WHILE ::= [ 'loop'  ( BLOCK | ':' COMMAND ) ]
  'while' [EXPRESSION] ['list']( BLOCK | ':' COMMAND )
  [ 'until' EXPRESSION ]
  [ 'done' ( BLOCK | ':' COMMAND ) ]
  [ 'otherwise' ( BLOCK | ':' COMMAND ) ]
  [ 'finally' ( BLOCK | ':' COMMAND ) ]
  <p>

----



[source]
----
  FOR ::= 'for' COMMAND ';' [EXPRESSION] ';' COMMAND ['list']( BLOCK | ':' COMMAND ) |
             'for' '(' COMMAND [EXPRESSION] COMMAND ')' ['list'](  BLOCK | ':' COMMAND )

----



[source]
----
  // the conventional '(' and ')' around the expressions giving the iterable is optional
  FOR_EACH ::= 'for each' identifier [with identifier]
  'in' EXPRESSION ['list']( BLOCK | ':' COMMAND ) |
  'for each' '(' identifier ['with' identifier]
  'in' EXPRESSION ')' ['list'](  BLOCK | ':' COMMAND )

----


If the condition expression is missing after 'for' or 'while', it is `true`.
The expression following the 'until' keyword is not optional, but the whole `until` `EXPRESSION` part is.
The value of a loop is the last __executed__ command, or a list if the `list` keyword was specified.
The elements of the list are the last __executed__ commands for each iteration.

The value of an `if` command is the value of the first or the second block, whichever is executed.
If the condition expression is false and there is no second block, then the value is `none`.


[source]
----
  IF ::= 'if' EXPRESSION ( ':' COMMAND | BLOCK ) [ 'else' ( [':'] COMMAND | BLOCK )] ;

----


The result of a CLASS declaration is the declared class itself.


[source]
----
  CLASS ::= 'class' [IDENTIFIER] [INHERITANCE] BODY
 
  INHERITANCE ::= ':' PARENT_LIST
 
  PARENT_LIST ::= IDENTIFIER (',' IDENTIFIER)
 
  BODY ::= '{' COMMAND '}'
 

----


The class object is also assigned to the variable denoted by the `IDENTIFIER` in the current context.

The result of a function declaration is the function itself.


[source]
----
  FUNCTION ::= 'fn' (NAMED_FUNCTION | ANONYMOUS_FUNCTION)
 
  NAMED_FUNCTION ::= IDENTIFIER FUNCTION_BODY
                    | IDENTIFIER '(' PARAM_LIST ')' [RETURN_TYPE] FUNCTION_BODY
 
  ANONYMOUS_FUNCTION ::= '(' PARAMETER_LIST ')' [RETURN_TYPE] FUNCTION_BODY
 
  RETURN_TYPE ::=':'IDENTIFIER (','IDENTIFIER)
 
  FUNCTION_BODY ::= BLOCK_BODY
                   |'='EXPRESSION
                   |';'
 
  BLOCK_BODY ::= '{' COMMAND '}'
 

----


[source]
----
  PARAMETER_LIST ::= PARAMETER (',' PARAMETER)
 
  PARAMETER ::= NORMAL_PARAMETER | SPECIAL_PARAMETER
 
  NORMAL_PARAMETER ::= [PARAMETER_TYPE] IDENTIFIER [TYPE_DECLARATION] [DEFAULT_VALUE]
 
  PARAMETER_TYPE ::= '!'  // positional only
                  | '@'  // named only
 
  SPECIAL_PARAMETER ::= REST_PARAMETER | META_PARAMETER | CLOSURE_PARAMETER
 
  REST_PARAMETER ::= '[' IDENTIFIER ']'
 
  META_PARAMETER ::= '{' IDENTIFIER '}'
 
  CLOSURE_PARAMETER ::= '^' IDENTIFIER
 
  TYPE_DECLARATION ::= ':' TYPE_LIST
 
  TYPE_LIST ::= IDENTIFIER (',' IDENTIFIER)
 
  DEFAULT_VALUE ::= '=' EXPRESSION
 
  // Special parameters must appear in this order, if present
  PARAMETERS_ORDER ::= NORMAL_PARAMETER [REST_PARAMETER] [META_PARAMETER] [CLOSURE_PARAMETER]

----


For functions with an expression body following a '=', the parameter list is mandatory.
The parameter list may be empty, having only '(' and ')'.

For functions with a block body, both the function name and parameter list are optional, but you must have at least one of them.

Variables declared as local will be local to the context, even if the same name is used in some enclosing context or as a global variable.
The value of the local declaration is the value of the last assignment in the command


[source]
----
  LET ::= SINGLE_LET | MULTI_LET
  <p>
  SINGLE_LET ::= ('let' | 'mut') ASSIGNMENT_LIST
  <p>
  MULTI_LET ::= ('let' | 'mut') DESTRUCTURING '=' EXPRESSION
  <p>
  DESTRUCTURING ::= '[' MAP_LIST ']'
  | '{' FIELD_MAP_LIST '}'
  <p>
  MAP_LIST ::= ['let'] MAP ( ',' ['let'] MAP )
  <p>
  FIELD_MAP_LIST ::== ['let'] FIELD_MAP (',' ['let'] FIELD_MAP)
  <p>
  MAP ::_= IDENTIFIER [':' TYPE_DECLARATIONS ] | LEFT_VALUE | DESTRUCTURING
  <p>
  FIELD_MAP ::= [ IDENTIFIER '->' ] MAP
  <p>
  TYPE_DECLARATIONS ::= TYPE_DECLARATION ( '|' TYPE_DECLARATION )
  <p>
  TYPE_DECLARATION ::= IDENTIFIER | '(' EXPRESION ')'
  <p>
  EXPRESSION ::= // Any valid Turicum expression

----


When there are braces or brackets around the identifier list the `EXPRESSION` has to be an object (for braces) or a list (for brackets).
The fields or the elements of the object or list are assigned to the individual variables.

Variables declared global will refer to the global variable.
The value of the global declaration is the value of the last assignment in the command.

 GLOBAL ::= 'global' IDENTIFIER_ASSIGNMENT_LIST;

Variables pinned cannot be changed in the local context.
Pinning has no effect on outer scopes.
You can also pin an object.
In that case use `+{variable}+` instead of the name.

 PIN ::= 'pin' IDENTIFIER_ASSIGNMENT_LIST;

 IDENTIFIER_ASSIGNMENT_LIST ::= [identifier [: TYPE_DEFINITION ] ['=' expression]]
                                    { ',' identifier [: TYPE_DEFINITION ] [ '=' expression] };

 TYPED_IDENTIFIER_LIST ::= identifier [: TYPE_DEFINITION ]
                                    { ',' identifier [: TYPE_DEFINITION ] };

 TYPE_DEFINITION ::= TYPE_NAME { '|' TYPE_NAME }

 TYPE_NAME ::= IDENTIFIER

It is valid to have no identifiers in an identifier list, i.e., an empty identifier list.

 IDENTIFIER_LIST ::= [ IDENTIFIER { ',' IDENTIFIER } ];

Using '=' updates a variable but never creates a new one.
It first has to be created using `let`.

 ASSIGNMENT ::= LEFT_VALUE '=' EXPRESSION ;
 LEFT_VALUE ::= IDENTIFIER { '.' IDENTIFIER | '[' EXPRESSION ']'};
 EXPRESSION ::= BINARY_EXPRESSION[0];

Define precedence levels using indexed rules

 N=10;
 BINARY_EXPRESSION[i <N ] ::= BINARY_EXPRESSION[i+1] { BINARY_OPERATOR[i] BINARY_EXPRESSION[i+1] };
 BINARY_EXPRESSION[i == N] ::= UNARY_EXPRESSION  ;// Highest precedence level

Unary operators and primary expressions

 UNARY_EXPRESSION ::= PREFIX_UNARY_OPERATOR UNARY_EXPRESSION | PRIMARY_EXPRESSION;

 PRIMARY_EXPRESSION ::= number | string
 | FN
 | CLASS
 | IDENTIFIER
 | '(' COMMAND_LIST ')'
 | FUNCTION_CALL
 | FIELD_ACCESS
 | METHOD_CALL
 | BLOCK
 | ARRAY_ACCESS
 ;

Function call: name(params)

 FUNCTION_CALL: := IDENTIFIER '(' [ EXPRESSION { ',' EXPRESSION } ] ')' ;

Field access: obj.field

 FIELD_ACCESS: := PRIMARY_EXPRESSION ('.'|'?.') (IDENTIFIER | KEYWORD) ;

Method call: obj.method(params)

 METHOD_CALL: := PRIMARY_EXPRESSION ('.'|'?.') (IDENTIFIER | KEYWORD) '(' [ EXPRESSION { ',' EXPRESSION } ] ')' ;

Block returning an expression value.
The value of the block is always the last executed expression.

 BLOCK: := '{' COMMAND_LIST '}' ;

 COMMAND_LIST: := { COMMAND [';' | '\n'] }

 BREAK: := 'break' CONDITIONAL_VALUE ;
 RETURN: := 'return' CONDITIONAL_VALUE ;
 YIELD: := 'yield' CONDITIONAL_VALUE ;
 CONDITIONAL_VALUE = EXPRESSION [ ('if' | 'when') EXPRESSION ];

Array element access: array[index]

 ARRAY_ACCESS: := PRIMARY_EXPRESSION '[' EXPRESSION ']' ;

Unary operators

 PREFIX_UNARY_OPERATOR ::= '+' | '-' | '!' ;

Binary operators grouped by precedence (higher index = higher precedence)

 BINARY_OPERATOR[0] ::= '||' ; Lowest precedence
 BINARY_OPERATOR[1] ::= '&&' ;
 BINARY_OPERATOR[2] ::= '|' ;
 BINARY_OPERATOR[3] ::= '^' ;
 BINARY_OPERATOR[4] ::= '&' ;
 BINARY_OPERATOR[5] ::= '==' | '!=' ;
 BINARY_OPERATOR[6] ::= '<' | '<=' | '>' | '>=' ;
 BINARY_OPERATOR[7] ::= '<<' | '>>' ;
 BINARY_OPERATOR[8] ::= '+' | '-' ;
 BINARY_OPERATOR[9] ::= '*' | '/' | '%' ; Highest precedence for binary ops