llms-full.txt

# Josh Language

Josh is a domain-specific language (DSL) designed for ecological modeling, with a particular focus on vegetation and agent-based simulations. The language enables the description of multi-occupancy patch-based ecological simulations where multiple species occupying grid cells can be modeled through individual behaviors with optional state changes.

## Purpose

Josh supports the nexus between science, policy, and software engineering by executing vegetation-focused ecological simulations to support management decisions. The language prioritizes readability and interpretability for a broad audience including ecologists, policy makers, and researchers who may not have extensive software engineering backgrounds.

## Execution Environments

Josh runs across multiple execution environments without requiring code changes:

1. **Browser via WebAssembly**: Complete simulations run directly in web browsers using WebAssembly compilation via TeaVM, requiring no local installation
2. **Local JVM**: High-performance execution on local machines using Java Virtual Machine for intensive computations and parallelization
3. **Distributed via JoshCloud**: Community infrastructure for large-scale distributed processing across multiple machines with API key access

## Geospatial Data Preprocessing

Josh supports preprocessing of external geospatial data files into an optimized binary format called `.jshd` (Josh Data) files. The preprocessing workflow converts data from formats like GeoTIFF and NetCDF into a format optimized for simulation execution, handling coordinate system transformations, temporal alignment, and spatial resampling.

## Command Line Interface (CLI)

Josh provides a comprehensive command line interface through the `joshsim.jar` file via the `JoshSimCommander` class. The CLI supports various operations for validation, execution, preprocessing, and management of Josh simulations.

### Basic Usage

```bash
java -jar joshsim-fat.jar [COMMAND] [OPTIONS]
```

### Available Commands

#### `run` - Execute Simulations
Runs a specified simulation from a Josh script file with support for replicates and various output formats.

```bash
java -jar joshsim-fat.jar run <script.josh> <SimulationName> [OPTIONS]
```

**Parameters:**
- `<script.josh>`: Path to Josh simulation file
- `<SimulationName>`: Name of simulation to execute

**Options:**
- `--crs <CRS>`: Coordinate Reference System (optional)
- `--replicates <N>`: Number of replicates to run (default: 1)
- `--use-float-64`: Use double precision instead of BigDecimal for speed vs precision tradeoff
- `--data <path>`: Path to directory containing external data files
- `--output-format <format>`: Output format (csv, netcdf, geotiff)
- `-o, --output <path>`: Output file path
- `--parallel`: Enable parallel patch processing
- `--verbose`: Enable verbose output
- `--upload-source`: Upload source .josh file to MinIO after completion (requires MinIO configuration)
- `--upload-config`: Upload configuration .jshc files to MinIO after completion (requires MinIO configuration)
- `--upload-data`: Upload data .jshd files to MinIO after completion (requires MinIO configuration)

**Examples:**
```bash
# Basic simulation run
java -jar joshsim-fat.jar run simulation.josh Main -o results.csv

# Run with multiple replicates
java -jar joshsim-fat.jar run simulation.josh Main --replicates 10 -o results.csv

# Run with external data
java -jar joshsim-fat.jar run simulation.josh Main --data ./data -o results.netcdf
```

#### `preprocess` - Data Preprocessing
Converts external geospatial data files into optimized binary `.jshd` format for efficient simulation loading.

```bash
java -jar joshsim-fat.jar preprocess <script.josh> <SimulationName> <dataFile> <variable> <units> <output.jshd> [OPTIONS]
```

**Parameters:**
- `<script.josh>`: Path to Josh simulation file
- `<SimulationName>`: Name of simulation for preprocessing
- `<dataFile>`: Path to input data file (NetCDF, GeoTIFF, etc.)
- `<variable>`: Variable name or band number to extract
- `<units>`: Units of the data for simulation use
- `<output.jshd>`: Path for output preprocessed file

**Options:**
- `--amend`: Amend existing JSHD file rather than overwriting
- `--crs <CRS>`: Coordinate Reference System for reading input file
- `--x-coord <name>`: Name of X coordinate dimension
- `--y-coord <name>`: Name of Y coordinate dimension
- `--time-coord <name>`: Name of time dimension
- `--verbose`: Enable verbose output

**Examples:**
```bash
# Basic preprocessing
java -jar joshsim-fat.jar preprocess simulation.josh Main temperature.nc temp K temperature.jshd

# Preprocessing with custom coordinates
java -jar joshsim-fat.jar preprocess simulation.josh Main precip.nc rainfall "mm/year" precipitation.jshd --x-coord longitude --y-coord latitude
```

#### `validate` - Syntax Validation
Validates Josh script syntax and reports parsing errors without executing simulations.

```bash
java -jar joshsim-fat.jar validate <script.josh>
```

**Parameters:**
- `<script.josh>`: Path to Josh script file to validate

**Options:**
- `--verbose`: Enable verbose validation output
- `--upload-source`: Upload source .josh file to MinIO after validation (requires MinIO configuration)

#### `discoverConfig` - Configuration Discovery
Analyzes Josh scripts to discover all configuration variables used, helping generate appropriate `.jshc` files.

```bash
java -jar joshsim-fat.jar discoverConfig <script.josh>
```

**Parameters:**
- `<script.josh>`: Path to Josh script file to analyze

#### `inspectJshd` - Data Inspection
Inspects values in preprocessed JSHD files at specific coordinates and timesteps for debugging and validation.

```bash
java -jar joshsim-fat.jar inspectJshd <file.jshd> <variable> <timestep> <x> <y>
```

**Parameters:**
- `<file.jshd>`: Path to JSHD file to inspect
- `<variable>`: Variable name to examine
- `<timestep>`: Time step to inspect
- `<x>`: X coordinate (grid space)
- `<y>`: Y coordinate (grid space)

#### `server` - HTTP Server Mode
Starts Josh as an HTTP server for web-based simulation execution and API access.

```bash
java -jar joshsim-fat.jar server [OPTIONS]
```

**Options:**
- `--port <port>`: Server port (default: 8080)
- `--host <host>`: Server host (default: localhost)

#### `runRemote` - Cloud Execution
Executes simulations on Josh Cloud infrastructure for distributed processing.

```bash
java -jar joshsim-fat.jar runRemote <script.josh> <SimulationName> [OPTIONS]
```

**Parameters:**
- `<script.josh>`: Path to Josh simulation file
- `<SimulationName>`: Name of simulation to execute

**Options:**
- `--api-key <key>`: Josh Cloud API key
- `--replicates <N>`: Number of replicates
- `--endpoint <url>`: Custom Josh Cloud endpoint

### Global Options

All commands support these global options:
- `--help`: Show command help
- `--version`: Display version information
- `--verbose`: Enable verbose output
- `--quiet`: Suppress non-essential output

### Integration with Build Systems

The CLI can be integrated with build systems and scripts:

```bash
# Validate all Josh files in a directory
find . -name "*.josh" -exec java -jar joshsim-fat.jar validate {} \;

# Preprocessing pipeline
java -jar joshsim-fat.jar preprocess simulation.josh Main temp.nc temperature K temp.jshd
java -jar joshsim-fat.jar preprocess simulation.josh Main precip.nc precipitation "mm/year" precip.jshd
java -jar joshsim-fat.jar run simulation.josh Main --data . -o results.csv
```

Preprocessing command structure:
```
java -jar joshsim.jar preprocess simulation.josh SimulationName data.nc variable units output.jshd
```

## Configuration Files (.jshc)

Josh supports external configuration files using the `.jshc` format for parameterizing simulations without modifying Josh code. This enables users to modify simulation parameters without changing Josh source code.

### Format Specification
- One variable per line: `variableName = value units`
- Comments using `#` character (all text after # ignored until newline)
- Empty lines allowed and ignored
- Whitespace flexible around equals sign (zero or more spaces/tabs)
- Variable names follow Josh identifier pattern: `[A-Za-z][A-Za-z0-9]*`
- Values must be valid EngineValue format (number + optional units)

### Usage in Josh Code
- Access via `config configName.variableName` expressions
- Example: `config example.testVar1` references `testVar1` from `example.jshc`
- Files loaded from working directory or provided with HTTP request
- Values are constant across all timesteps and spatial locations
- Configuration variables can be discovered using `discoverConfig` command

### Example Configuration File (example.jshc)
```
# Group 1 - Tree parameters
testVar1 = 5 meters
testVar2 = 10m

# Group 2 - Environment parameters  
testVar3 = 15 km
```

### Commands
- `discoverConfig script.josh` - Lists all configuration variables used in a Josh script
- Variables discovered can be used to generate appropriate .jshc files

## Language Structure

Josh uses a stanza-based approach similar to SmallTalk/HyperTalk languages. Each stanza defines an entity (object) using `start` and `end` blocks. The language is imperative with procedural elements, prioritizing behavioral description over computational specification.

### Entity Types

- **simulation**: Defines simulation parameters, grid specifications, time steps, and export configurations
- **patch**: Represents spatial cells containing organisms and environmental conditions
- **organism**: Individual agents with attributes and behaviors that change over time
- **disturbance**: Events that affect organisms or patches (e.g., fire, drought)
- **management**: Intentional interventions and management actions
- **external**: References to external data sources and resources
- **unit**: Custom unit definitions with conversions and aliases

### Stanza Structure

Each entity stanza follows this pattern:
```
start <entity_type> <identifier>
  <attribute>.<event> = <expression>
  <attribute>.<event>:<conditional> = <expression>
end <entity_type>
```

### Events

Event handlers are run once per variable and may be made conditional based on values on the entity. When a variable is referenced in an expression, its value will be resolved if it is not yet evaluated, automatically creating a computational graph such that the ordering of event handlers does not matter.

- **init**: Entity construction event, executed once when entity is created
- **start**: Start of simulation step event, executed at beginning of each timestep
- **step**: Mid-step event, executed during main simulation timestep
- **end**: End of step or termination event, executed at end of timestep or when entity is removed

## Language Features

### Comments
Single-line comments use `#` syntax:
```
# This is a comment
```

### Data Types

#### Scalar Values
Numeric values with optional units:
```
5 meters
10.5 years
42 count
33.7 degrees latitude
```

#### Strings
String literals enclosed in double quotes:
```
"Default"
"file:///tmp/output.csv"
```

#### Boolean Values
```
true
false
```

#### Distributions
Statistical distributions for stochastic modeling:
```
uniform from 0 meters to 1 meters
normal with mean of 10 years std of 2 years
```

### Unit System

Units are strongly typed and support conversions:

#### Built-in Units
- **Percentage**: `%`, `percent`
- **Count**: `count`, `counts`
- **Degrees**: `degrees`, `degree` (for latitude/longitude)
- **Meters**: `meter`, `meters`, `m`
- **Kilometers**: `kilometer`, `kilometers`, `km`
- **Boolean**: `bool`
- **String**: `string`

#### Custom Units
```
start unit year
  alias years
  alias yr
  alias yrs
end unit
```

#### Unit Conversions
```
start unit inch
  meter = 0.0254 meters
end unit
```

### Identifiers and Attributes

Identifiers follow the pattern `[A-Za-z][A-Za-z0-9]*` and can be chained with dot notation:
```
organism.attribute
patch.ForeverTree.age
current.height
```

### Keywords and Context

#### Temporal Keywords
- **current**: Current state of entity
- **prior**: State from previous timestep
- **meta**: Simulation metadata and context
  - `meta.stepCount`: Current simulation timestep (0-based)
  - `meta.year`: Current simulation year

#### Spatial Keywords
- **here**: Current grid cell context
- **within**: Spatial query for nearby entities

### Operators

#### Arithmetic Operators
- `+`: Addition
- `-`: Subtraction
- `*`: Multiplication
- `/`: Division
- `^`: Exponentiation
- `%`: Modulo

#### Comparison Operators
- `==`: Equality
- `!=`: Inequality
- `>`: Greater than
- `>=`: Greater than or equal
- `<`: Less than
- `<=`: Less than or equal

#### Logical Operators
- `and`: Logical AND
- `or`: Logical OR
- `xor`: Logical XOR

#### String Operator
- `|`: String concatenation

### Expressions

#### Function Calls
```
function_name(arguments)
mean(ForeverTree.age)
sample(uniform from 0 to 10)
```

#### Built-in Functions
- **mean()**: Calculate mean of collection
- **std()**: Calculate standard deviation
- **count()**: Count elements in collection
- **sample()**: Sample from distribution or collection
- **create**: Create new entities
  ```
  create ForeverTree                    # Create single entity
  create 10 count of ForeverTree        # Create multiple entities
  ```
- **force**: Type conversion and casting
  ```
  force myValue as string               # Force conversion to string type
  force prior.temperature as degrees    # Force unit conversion
  ```

#### Collection Operations
```
ForeverTree[ForeverTree.age > 5 years]  # Filtering
all  # All entities of specified type
```

### Control Flow

#### Conditional Modifiers
Event handlers can use conditional modifiers:
```
attribute.event:if(condition) = expression
attribute.event:elif(condition) = expression
attribute.event:else = expression
```

#### Conditional Expressions
Ternary-style conditional expressions:
```
value if condition else alternative
```

#### Full Body Conditionals
Multi-statement conditional blocks:
```
if condition {
  const temp = prior.value
  return temp + 1
} elif other_condition {
  return prior.value
} else {
  return 0
}
```

### Full Bodies

Multi-statement blocks enclosed in curly braces `{}` supporting:
- `const` variable declarations
- `return` statements
- Conditional blocks
- Complex logic

Example full body:
```
age.step = {
  const currentAge = prior.age
  const increment = 1 year
  return currentAge + increment
}
```

Example lambda (single expression):
```
height.step = prior.height + sample uniform from 0 meters to 1 meters
```

### Operations

#### Sampling
```
sample uniform from low to high
sample count from collection
sample count from collection with replacement
sample count from collection without replacement
```

#### Limiting
```
limit operand to [min,]      # Minimum bound
limit operand to [,max]      # Maximum bound
limit operand to [min,max]   # Both bounds
```

#### Mapping
```
map operand from [fromlow,fromhigh] to [tolow,tohigh]              # Linear mapping (default)
map operand from [fromlow,fromhigh] to [tolow,tohigh] linear       # Linear mapping
map operand from [fromlow,fromhigh] to [tolow,tohigh] sigmoid      # Sigmoid mapping
map operand from [fromlow,fromhigh] to [tolow,tohigh] quadratic    # Quadratic mapping
map operand from [fromlow,fromhigh] to [tolow,tohigh] method(arg)  # Method with argument
```

Available mapping methods:
- **linear**: Linear interpolation between ranges. Does not take optional arguments.
- **sigmoid**: S-curve mapping for smooth transitions. Takes optional argument. True: increasing. False: decreasing.
- **quadratic**: Quadratic curve mapping with controllable concavity. Takes optional argument. True: the center of the domain maps to range maximum, endpoints to minimum. False: the center maps to range minimum, endpoints to maximum.

### Spatial Queries

```
identifier within distance radial at prior
```

### Imports and Configuration

*Note: Imports and configuration are reserved for future use and are not currently implemented.*

#### Imports
```
import "filename.josh"
```

#### Configuration
```
config expression as identifier
```

### Assertions

Validation statements for testing and debugging:
```
assert.attribute.event = boolean_expression
assert.attribute.event:if(condition) = boolean_expression
```

Assertions expect boolean values and are evaluated during event handler execution.

### Exports

Define simulation outputs:
```
export.variable.event = expression
exportFiles.type = "output_path"
```

### State Management

Entities can define custom states. Event handlers in a state are only evaluated if the state attribute on an entity is set to that given state.

```
start state "StateName"
  attribute.event = expression
end state
```

### Reserved Words

The following keywords are reserved and should not be used for user-defined entities or variables:
`as`, `const`, `disturbance`, `elif`, `else`, `end`, `if`, `management`, `limit`, `map`, `return`, `start`, `state`, `step`, `within`

### Program Structure

A complete Josh program consists of:
1. Optional configuration statements
2. Optional import statements  
3. Simulation definition
4. Other entity stanzas (patch, organism, etc.)
5. Unit definitions

Example minimal program:
```
start simulation Main
  grid.size = 1000 m
  grid.low = 33.7 degrees latitude, -115.4 degrees longitude
  grid.high = 34.0 degrees latitude, -116.4 degrees longitude
  steps.low = 0 count
  steps.high = 10 count
end simulation

start patch Default
  ForeverTree.init = create 10 count of ForeverTree
end patch

start organism ForeverTree
  age.init = 0 year
  age.step = prior.age + 1 year
end organism

start unit year
  alias years
end unit
```

This language specification covers all features defined in the Josh grammar and provides comprehensive documentation for developing ecological simulations using the Josh domain-specific language.