Skip to content

Commit 6387763

Browse files
Copilotnilp0inter
Copilot
and
nilp0inter
authored
Initialize montest project scaffold (#1)
* Initial plan * Initialize montest project scaffold Co-authored-by: nilp0inter <1224006+nilp0inter@users.noreply.github.com> * Address review feedback: tox matrix, NOTICE file, copyright, CI steps Co-authored-by: nilp0inter <1224006+nilp0inter@users.noreply.github.com> * Add Python 3.12–3.14 to flake.nix dev environment Co-authored-by: nilp0inter <1224006+nilp0inter@users.noreply.github.com> * Update CI: use tox for tests and matrix for Python version verification Co-authored-by: nilp0inter <1224006+nilp0inter@users.noreply.github.com> * Add smoke test for __version__ to fix pytest exit code 5 (no tests collected) Co-authored-by: nilp0inter <1224006+nilp0inter@users.noreply.github.com> * Add go-task to flake, create Taskfile.yml, update CI and shellHook Co-authored-by: nilp0inter <1224006+nilp0inter@users.noreply.github.com> * docs: update README with license and dev setup sections Co-authored-by: nilp0inter <1224006+nilp0inter@users.noreply.github.com> * Add CI badge to README Add CI badge to README for build status visibility. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: nilp0inter <1224006+nilp0inter@users.noreply.github.com> Co-authored-by: Roberto Abdelkader Martínez Pérez <robertomartinezp@gmail.com>
1 parent 9aeda60 commit 6387763

9 files changed

Lines changed: 309 additions & 1 deletion

File tree

‎.github/workflows/ci.yml‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: ["main"]
6+
pull_request:
7+
branches: ["main"]
8+
9+
jobs:
10+
lint:
11+
runs-on: ubuntu-latest
12+
permissions:
13+
contents: read
14+
15+
steps:
16+
- name: Checkout code
17+
uses: actions/checkout@v4
18+
19+
- name: Install Nix
20+
uses: cachix/install-nix-action@v27
21+
with:
22+
nix_path: nixpkgs=channel:nixos-unstable
23+
24+
- name: Sync dependencies
25+
run: nix develop --command task sync
26+
27+
- name: Check linter
28+
run: nix develop --command task lint
29+
30+
- name: Check types
31+
run: nix develop --command task typecheck
32+
33+
test:
34+
runs-on: ubuntu-latest
35+
permissions:
36+
contents: read
37+
strategy:
38+
matrix:
39+
include:
40+
- python: "3.11"
41+
tox-env: "py311"
42+
- python: "3.12"
43+
tox-env: "py312"
44+
- python: "3.13"
45+
tox-env: "py313"
46+
- python: "3.14"
47+
tox-env: "py314"
48+
49+
steps:
50+
- name: Checkout code
51+
uses: actions/checkout@v4
52+
53+
- name: Install Nix
54+
uses: cachix/install-nix-action@v27
55+
with:
56+
nix_path: nixpkgs=channel:nixos-unstable
57+
58+
- name: Verify dev environment
59+
run: nix develop --command bash -c "uv --version && python${{ matrix.python }} --version"
60+
61+
- name: Sync dependencies
62+
run: nix develop --command task sync
63+
64+
- name: Run tests
65+
run: nix develop --command task test TOX_ENV=${{ matrix.tox-env }}

‎NOTICE‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
BBVA - montest
2+
Copyright 2026 Banco Bilbao Vizcaya Argentaria, S.A.
3+
4+
This product includes software developed at
5+
BBVA (https://www.bbva.com/)

‎README.md‎

Lines changed: 46 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,47 @@
11
# montest
2-
A stochastic BDD framework for Python designed to test non-deterministic systems (e.g., LLMs) by evaluating statistical distributions rather than binary outcomes
2+
3+
[![CI](https://github.com/BBVA/montest/actions/workflows/ci.yml/badge.svg)](https://github.com/BBVA/montest/actions/workflows/ci.yml)
4+
5+
A stochastic BDD framework for Python designed to test non-deterministic systems (e.g., LLMs) by evaluating statistical distributions rather than binary outcomes.
6+
7+
## License
8+
9+
Copyright 2026 Banco Bilbao Vizcaya Argentaria, S.A.
10+
11+
Licensed under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0).
12+
See the [NOTICE](NOTICE) file for additional attribution information.
13+
14+
## Development Setup
15+
16+
### Prerequisites (without Nix)
17+
18+
You will need the following tools installed on your system:
19+
20+
- **Python 3.11, 3.12, 3.13, and 3.14** — all four versions must be discoverable on `$PATH` as `python3.11`, `python3.12`, `python3.13`, and `python3.14`
21+
- **[uv](https://docs.astral.sh/uv/)** — fast Python package and project manager
22+
- **[Task](https://taskfile.dev/)** (`go-task`) — task runner used to execute all CI steps
23+
24+
Once the tools are available, install the project dependencies:
25+
26+
```bash
27+
task sync
28+
```
29+
30+
Then you can run the individual workflow steps:
31+
32+
```bash
33+
task lint # ruff linter
34+
task typecheck # mypy type checker
35+
task test # run tests with the default tox env (py311)
36+
task test TOX_ENV=py313 # run tests against a specific Python version
37+
```
38+
39+
### Prerequisites (with Nix)
40+
41+
If you have [Nix](https://nixos.org/) with [flakes](https://nixos.wiki/wiki/Flakes) enabled, all required tools are provided automatically by the dev shell — no manual installation needed:
42+
43+
```bash
44+
nix develop
45+
```
46+
47+
This drops you into a shell that has `uv`, `task`, and Python 3.11–3.14 all ready to use. The same `task` commands above apply once you are inside the dev shell.

‎Taskfile.yml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
version: "3"
2+
3+
tasks:
4+
sync:
5+
desc: "Sync project dependencies"
6+
cmds:
7+
- uv sync --extra dev
8+
9+
lint:
10+
desc: "Run ruff linter"
11+
cmds:
12+
- uv run ruff check src tests
13+
14+
typecheck:
15+
desc: "Run mypy type checker"
16+
cmds:
17+
- uv run mypy src
18+
19+
test:
20+
desc: "Run tests via tox (set TOX_ENV to py311/py312/py313/py314)"
21+
vars:
22+
TOX_ENV: '{{.TOX_ENV | default "py311"}}'
23+
cmds:
24+
- uv run tox -e {{.TOX_ENV}}

‎flake.nix‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
{
2+
description = "montest development environment";
3+
4+
inputs = {
5+
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
6+
flake-utils.url = "github:numtide/flake-utils";
7+
};
8+
9+
outputs = { self, nixpkgs, flake-utils }:
10+
flake-utils.lib.eachSystem [
11+
"aarch64-linux"
12+
"x86_64-linux"
13+
"aarch64-darwin"
14+
"x86_64-darwin"
15+
] (system:
16+
let
17+
pkgs = nixpkgs.legacyPackages.${system};
18+
in {
19+
devShells.default = pkgs.mkShell {
20+
buildInputs = [
21+
pkgs.uv
22+
pkgs.go-task
23+
pkgs.python311
24+
pkgs.python312
25+
pkgs.python313
26+
pkgs.python314
27+
];
28+
29+
shellHook = ''
30+
echo ""
31+
echo " 🎭 ✨ W E L C O M E T O M O N T E S T ✨ 🎭"
32+
echo ""
33+
echo " 🐍 Python versions available:"
34+
echo " python3.11 — $(python3.11 --version)"
35+
echo " python3.12 — $(python3.12 --version)"
36+
echo " python3.13 — $(python3.13 --version)"
37+
echo " python3.14 — $(python3.14 --version)"
38+
echo ""
39+
echo " 🚀 Available tasks (run: task <name>)"
40+
echo " task sync — sync project dependencies"
41+
echo " task lint — run ruff linter"
42+
echo " task typecheck — run mypy type checker"
43+
echo " task test — run tests (e.g. task test TOX_ENV=py313)"
44+
echo ""
45+
echo " 💃 Happy hacking! 🕺"
46+
echo ""
47+
'';
48+
};
49+
});
50+
}

‎pyproject.toml‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
[build-system]
2+
requires = ["hatchling"]
3+
build-backend = "hatchling.build"
4+
5+
[project]
6+
name = "montest"
7+
version = "0.1.0"
8+
description = "A stochastic BDD testing framework for non-deterministic systems."
9+
authors = [{ email = "robertomartinezp@gmail.com" }]
10+
license = { text = "Apache-2.0" }
11+
readme = "README.md"
12+
requires-python = ">=3.11"
13+
classifiers = [
14+
"License :: OSI Approved :: Apache Software License",
15+
"Programming Language :: Python :: 3.11",
16+
"Programming Language :: Python :: 3.12",
17+
"Programming Language :: Python :: 3.13",
18+
"Programming Language :: Python :: 3.14",
19+
]
20+
dependencies = [
21+
"gherkin-official>=29.0.0",
22+
]
23+
24+
[project.optional-dependencies]
25+
dev = [
26+
"pytest>=8.0.0",
27+
"pytest-cov>=5.0.0",
28+
"hypothesis>=6.0.0",
29+
"tox>=4.0.0",
30+
"ruff>=0.4.0",
31+
"mypy>=1.10.0",
32+
]
33+
34+
[tool.hatch.build.targets.wheel]
35+
packages = ["src/montest"]
36+
37+
[tool.ruff]
38+
line-length = 88
39+
target-version = "py311"
40+
41+
[tool.ruff.lint]
42+
select = [
43+
"E", # pycodestyle errors
44+
"W", # pycodestyle warnings
45+
"F", # pyflakes
46+
"I", # isort
47+
"B", # flake8-bugbear
48+
"UP", # pyupgrade
49+
]
50+
51+
[tool.pytest.ini_options]
52+
testpaths = ["tests"]
53+
addopts = "--cov=montest --cov-report=term-missing"
54+
55+
[tool.mypy]
56+
strict = true
57+
mypy_path = "src"
58+
59+
[tool.coverage.run]
60+
source = ["src/montest"]
61+
62+
[tool.tox]
63+
legacy_tox_ini = """
64+
[tox]
65+
envlist = py311, py312, py313, py314
66+
isolated_build = True
67+
68+
[testenv]
69+
extras = dev
70+
commands =
71+
pytest {posargs}
72+
"""

‎src/montest/__init__.py‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Copyright 2026 Banco Bilbao Vizcaya Argentaria, S.A.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
__version__ = "0.1.0"

‎tests/__init__.py‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright 2026 Banco Bilbao Vizcaya Argentaria, S.A.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎tests/test_version.py‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Copyright 2026 Banco Bilbao Vizcaya Argentaria, S.A.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
from montest import __version__
16+
17+
18+
def test_version() -> None:
19+
assert __version__ == "0.1.0"

0 commit comments

Comments
 (0)