Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/workflows/system_lexa.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Lexa

# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches: [ main ]
pull_request:
branches: [ main ]

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

#From https://docs.github.com/en/actions/guides/publishing-docker-images
jobs:
bench-system-lexa:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write

steps:
- name: Checkout repository
uses: actions/checkout@v2

- uses: satackey/action-docker-layer-caching@v0.0.11
# Ignore the failure of a step and avoid terminating the job.
continue-on-error: true

- name: Add write permission to directories
run: |
find . -type d -exec chmod 777 {} \;

- name: Test Lexa system
run: |
make test_lexa

12 changes: 12 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,18 @@ test_koka: system_koka
docker run -v $(shell pwd):/source $(DOCKERHUB):koka \
make -C /source/benchmarks/koka test

# Lexa
system_lexa:
docker build -t $(DOCKERHUB):lexa systems/lexa

bench_lexa: system_lexa
docker run -it --init -v $(shell pwd):/source $(DOCKERHUB):lexa \
make -C /source/benchmarks/lexa

test_lexa: system_lexa
docker run -v $(shell pwd):/source $(DOCKERHUB):lexa \
make -C /source/benchmarks/lexa test

# libmpeff
system_libmpeff: system_base
docker build -t $(DOCKERHUB):libmpeff systems/libmpeff
Expand Down
29 changes: 15 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,12 @@ contains the results of running the OCaml benchmarks.
## System availability

| System | Availability |
|--------|--------------|
|-------|--------------|
| [Eff](https://github.com/matijapretnar/eff) | [![Eff](https://github.com/effect-handlers/effect-handlers-bench/actions/workflows/system_eff.yml/badge.svg)](https://github.com/effect-handlers/effect-handlers-bench/actions/workflows/system_eff.yml) |
| [Effekt](https://github.com/effekt-lang/effekt) | [![Effekt](https://github.com/effect-handlers/effect-handlers-bench/actions/workflows/system_effekt.yml/badge.svg)](https://github.com/effect-handlers/effect-handlers-bench/actions/workflows/system_effekt.yml) |
| [Handlers in Action](https://github.com/slindley/effect-handlers) | [![Handlers in Action](https://github.com/effect-handlers/effect-handlers-bench/actions/workflows/system_hia.yml/badge.svg)](https://github.com/effect-handlers/effect-handlers-bench/actions/workflows/system_hia.yml) |
| [Koka](https://github.com/koka-lang/koka) | [![Koka](https://github.com/effect-handlers/effect-handlers-bench/actions/workflows/system_koka.yml/badge.svg)](https://github.com/effect-handlers/effect-handlers-bench/actions/workflows/system_koka.yml) |
| [Lexa](https://github.com/lexa-lang/lexa) | [![Lexa](https://github.com/effect-handlers/effect-handlers-bench/actions/workflows/system_lexa.yml/badge.svg)](https://github.com/effect-handlers/effect-handlers-bench/actions/workflows/system_lexa.yml) |
| [libhandler](https://github.com/koka-lang/libhandler) | [![libhandler](https://github.com/effect-handlers/effect-handlers-bench/actions/workflows/system_libhandler.yml/badge.svg)](https://github.com/effect-handlers/effect-handlers-bench/actions/workflows/system_libhandler.yml) |
| [libmpeff](https://github.com/koka-lang/libmprompt) | [![libmpeff](https://github.com/effect-handlers/effect-handlers-bench/actions/workflows/system_libmpeff.yml/badge.svg)](https://github.com/effect-handlers/effect-handlers-bench/actions/workflows/system_libmpeff.yml) |
| [libseff](https://github.com/effect-handlers/libseff.git) | [![libseff](https://github.com/effect-handlers/effect-handlers-bench/actions/workflows/system_libseff.yml/badge.svg)](https://github.com/effect-handlers/effect-handlers-bench/actions/workflows/system_libseff.yml) |
Expand All @@ -37,19 +38,19 @@ contains the results of running the OCaml benchmarks.

## Benchmark availability

| | Eff | Effekt | Handlers in Action | Koka | OCaml | Libseff | Libmpeff |
| :---------------------- | :----------------: | :----------------: | :----------------: | :----------------: | :----------------: | :----------------: | :----------------: |
| **Countdown** | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| **Fibonacci Recursive** | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| **Product Early** | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| **Iterator** | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| **Nqueens** | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_minus_sign: | :x: |
| **Generator** | :heavy_check_mark: | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| **Tree explore** | :heavy_check_mark: | :heavy_check_mark: | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_minus_sign: | :x: |
| **Triples** | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_minus_sign: | :x: |
| **Parsing Dollars** | :heavy_check_mark: | :heavy_check_mark: | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| **Resume Nontail** | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| **Handler Sieve** | :heavy_check_mark: | :x: | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| | Eff | Effekt | Handlers in Action | Koka | OCaml | Lexa | Libseff | Libmpeff |
| :---------------------- | :----------------: | :----------------: | :----------------: | :----------------: | :----------------: | :----------------: | :----------------: | :----------------: |
| **Countdown** | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| **Fibonacci Recursive** | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| **Product Early** | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| **Iterator** | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| **Nqueens** | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_minus_sign: | :x: |
| **Generator** | :heavy_check_mark: | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| **Tree explore** | :heavy_check_mark: | :heavy_check_mark: | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_minus_sign: | :x: |
| **Triples** | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_minus_sign: | :x: |
| **Parsing Dollars** | :heavy_check_mark: | :heavy_check_mark: | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| **Resume Nontail** | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| **Handler Sieve** | :heavy_check_mark: | :x: | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |

Legend:

Expand Down
48 changes: 48 additions & 0 deletions benchmarks/lexa/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
bench: build
hyperfine --export-csv results.csv \
'countdown/main 200000000' \
'fibonacci_recursive/main 42' \
'product_early/main 100000' \
'iterator/main 40000000' \
'nqueens/main 12' \
'generator/main 25' \
'tree_explore/main 16' \
'triples/main 300' \
'parsing_dollars/main 20000' \
'resume_nontail/main 20000' \
'handler_sieve/main 60000'

test: build
cd countdown ; ./main 5 > actual ; echo 0 > expected ; diff expected actual
cd fibonacci_recursive ; ./main 5 > actual ; echo 5 > expected ; diff expected actual
cd product_early ; ./main 5 > actual ; echo 0 > expected ; diff expected actual
cd iterator ; ./main 5 > actual ; echo 15 > expected ; diff expected actual
cd nqueens ; ./main 5 > actual ; echo 10 > expected ; diff expected actual
cd generator ; ./main 5 > actual ; echo 57 > expected ; diff expected actual
cd tree_explore ; ./main 5 > actual ; echo 946 > expected ; diff expected actual
cd triples ; ./main 10 > actual ; echo 779312 > expected ; diff expected actual
cd parsing_dollars ; ./main 10 > actual ; echo 55 > expected ; diff expected actual
cd resume_nontail ; ./main 5 > actual ; echo 37 > expected ; diff expected actual
cd handler_sieve ; ./main 10 > actual ; echo 17 > expected ; diff expected actual

build:
cd countdown ; lexa main.lx -o main
cd fibonacci_recursive ; lexa main.lx -o main
cd product_early ; lexa main.lx -o main
cd iterator ; lexa main.lx -o main
cd nqueens ; lexa main.lx -o main
cd generator ; lexa main.lx -o main
cd tree_explore ; lexa main.lx -o main
cd triples ; lexa main.lx -o main
cd parsing_dollars ; lexa main.lx -o main
cd resume_nontail ; lexa main.lx -o main
cd handler_sieve ; lexa main.lx -o main

clean:
-rm */clue_table.txt
-rm */main.c
-rm */offset_functions.h
-rm */main
-rm results.csv
-rm */expected
-rm */actual
35 changes: 35 additions & 0 deletions benchmarks/lexa/countdown/main.lx
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
effect State {
get: () -> int
set: (int) -> unit
}

def countdown [; state_stub: State] (): int {
val i = raise state_stub.get();
if i == 0 then
i
else (
raise state_stub.set(i-1);
countdown:[; state_stub]()
)
}

def run(n: int): int {
val s = newref {n};
handle <> {
countdown:[; state_stub]()
} with state_stub: State {
def get() {
s[0]
}

def set(i) {
s[0] := i;
0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Why does this return 0 instead of the unit value?

}
}
}

def main(): int {
~printInt(run(~readInt()));
0
}
12 changes: 12 additions & 0 deletions benchmarks/lexa/fibonacci_recursive/main.lx
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
def fib(n: int): int {
if n == 0 then 0 else
if n == 1 then 1 else
fib(n - 1) + fib(n - 2)
}

def main(): int {
val arg = ~readInt();
val res = fib(arg);
~printInt(res);
0
}
58 changes: 58 additions & 0 deletions benchmarks/lexa/generator/main.lx
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
effect Yield {
yield: (int) -> unit
}

type tree =
| Leaf
| Node of int * tree * tree

type generator =
| Empty
| Thunk of int * (cont <> unit -> generator)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change: The other implementations appear to store functions here instead of continuations. I think to maintain consistency the same should be done here.


def make(n: int): tree {
if n == 0 then
Leaf
else
val t = make(n - 1);
Node(n, t, t)
}

def iterate [; yield_stub: Yield] (t: tree): int {
match t with
| Leaf -> { 0 }
| Node (value, left, right) -> {
iterate: [; yield_stub] (left);
raise yield_stub.yield(value);
iterate: [; yield_stub] (right)
}
}

def generate(f: <> [; yield_stub: Yield] () -> int): generator {
handle <> {
f:[; yield_stub]();
Empty()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Does Lexa have return clauses? It would be nice to maintain consistency if it does.

} with yield_stub: Yield {
hdl_1 yield(x, k) {
Thunk(x, k)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above comment about storing functions versus continuations.

}
}
}

def sum(a: int, g: generator): int {
match g with
| Empty -> { a }
| Thunk (v, f) -> {
sum(v + a, resume_final f (()))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above comment about storing functions versus continuations.

}
}

def run(n: int): int {
val f = fun <> [; yield_stub: Yield] (): int { iterate:[; yield_stub](make(n)) };
sum(0, generate(f))
}

def main(): int {
~printInt(run(~readInt()));
0
}
39 changes: 39 additions & 0 deletions benchmarks/lexa/handler_sieve/main.lx
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
effect Prime {
prime: (int) -> bool
}

def primes [; prime_stub: Prime] (i: int, n: int, a: int): int {
if i < n then
if raise prime_stub.prime(i) then
handle <prime_stub> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personal question: What does the <...> syntax mean?

primes: [;new_prime_stub](i + 1, n, a + i)
} with new_prime_stub: Prime {
def prime(e) {
if (e % i) == 0 then
false
else
raise prime_stub.prime(e)
}
}
else
primes: [; prime_stub](i + 1, n, a)
else
a
}

def run(n: int): int {
handle <> {
primes: [;prime_true_stub](2, n, 0)
} with prime_true_stub: Prime {
def prime(e) {
true
}
}
}

def main(): int {
val arg1 = ~readInt();
val arg2 = run(arg1);
~printInt(arg2);
0
}
30 changes: 30 additions & 0 deletions benchmarks/lexa/iterator/main.lx
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
effect Emit {
emit: (int) -> unit
}

def range [; emit_stub: Emit] (l: int, u: int): int {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change: Return type should be unit here to match other implementations.

if l > u then
0
else (
raise emit_stub.emit(l);
range:[; emit_stub](l + 1, u)
)
}

def run(n: int): int {
val s = newref {0};
handle <> {
range:[; emit_stub](0, n)
} with emit_stub: Emit {
def emit(e) {
s[0] := s[0] + e;
0
}
};
s[0]
}

def main(): int {
~printInt(run(~readInt()));
0
}
56 changes: 56 additions & 0 deletions benchmarks/lexa/nqueens/main.lx
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
effect Search {
pick: (int) -> int
fail: () -> int
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically the return type of fail is prescribed to be void/empty. Does Lexa have this?

}

def safe(queen: int, diag: int, xs: node_t::[int]): bool {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: I assume that node_t is a linked list?

val is_empty = ~listIsEmpty::[int](xs);
if is_empty then
true
else
val q = ~listHead::[int](xs);
val qs = ~listTail::[int](xs);
if queen != q && queen != q + diag && queen != q - diag then
safe(queen, diag + 1, qs)
else
false
}

def place [; search_stub: Search] (size: int, column: int): node_t::[int] {
if column == 0 then
~listEnd::[int]()
else
val rest = place:[; search_stub](size, column - 1);
val next = raise search_stub.pick(size);
if safe(next, 1, rest) then
~listNode::[int](next, rest)
else
(raise search_stub.fail();
~listEnd::[int]())
}

def run(n: int): int {
handle <> {
place:[; search_stub](n, n);
1
} with search_stub: Search {
exc fail() { 0 }
hdl_s pick(size, k) {
loop(1, 0, size, k)
}
}
}

def loop(i: int, a: int, size: int, k: cont <> int -> int): int {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be refactored to be a local function defined in the body of the pick handling clause?

if i == size then
a + resume_final k i
else
loop(i + 1, a + resume k i, size, k)
}

def main(): int {
val n = ~readInt();
val run_res = run(n);
~printInt(run_res);
0
}
Loading
Loading