Improve python tooling

[dangotbanned]

Description

In a similar vein to (#1087) I've run into a few other issues.
The general theme for fixes are:

• Configuration that makes existing tooling more useful
• Ensuring the (python) test suites are green¹ by default

The end result? More things that just work™ and we get helpful feedback for things that don't/may not

Issues

I may split some of these out into other issues if they start looking big or a theme emerges.
Most can be individual PRs, just wanting to see them organized for now:

Pytest

• test(python): Enable strict xfail for pytest #1091
  • strict_xfail = true (Related)
• test(python): Avoid requiring optional dependencies #1092
  • importorskip for tests with optional dependencies
• filterwarnings = ["error"] to convert unhandled warnings into errors, and either
  • Fixing the source, so that they don't surface to users during correct usage
  • Expecting the warnings in tests, so that they don't produce noise for contributors
• test(python): Avoid using relative paths #1094
  • Avoid working-directory-sensitive filepaths (tests don't need to depend on this)

Ruff

Not sure if this was intentional, but I will mention it here:
most of what ruff can do is disabled.

Is that so bad?

I'd consider that a risky combination with LLM usage, or at least more expensive to fix than having Ruff do it 😄

I'm not looking to resolve the above here, but these should be quick, uncontroversial wins:

• Ruff-specific rules (RUF) like
  • RUF10* rules which fix/flag unused/invalid suppression comments
• unsorted-imports (Related)
  • required-imports = ["from __future__ import annotations"] combines with this suggestion to avoid circular dependencies, where the symbols are not used at runtime

Important

Ruff's new default rule set will fix most of this - but will be alarming in CI

Typing

• ty
  • ty documents a Recommended configuration when Coming from mypy or pyright
    • By default it does not flag issues that will show up in other type checkers
    • The recommendation includes config for Ruff too, which is not yet enabled
  • ty is still in beta, and for people like me that means it is missing features that (for now) make slower alternatives a better option
• pyright
  • chore(typing): Add pyright and fix errors #1095
  • Fix __all__ definitions (see (chore(typing): Add pyright and fix errors #1095 (comment)))

Misc

• ci: ignore .python-version #1102
  • Currently 3.12 and the final beta for 3.15 is out
  • Feature releases occur annually

Footnotes

1. Or at least yellow with expectations ↩

[domoritz]

Thank you. All of this makes sense. Whatever we do, let's make it consistent across the few python packages we have (vgplot, widget, server) when possible.

[dangotbanned]

Ruff

Not sure if this was intentional, but I will mention it here:
most of what ruff can do is disabled.

This will become more visible with (https://github.com/astral-sh/ruff/releases/tag/0.16.0) as:

Ruff now enables a much larger set of rules by default (413, up from 59)

It looks like that grenade will pop in around 7 days when uv.lock bumps.

I've been using this ruff.toml locally for anything new I touch in vgplot-python.
Sharing it here because it has ignores on existing code, that will help unbreak CI 🙂¹

Show ruff.toml

Adapted from narwhals-dev/narwhals/pyproject.toml#L242-L355

# packages/vgplot/vgplot-python/ruff.toml
fix = true

[format]
skip-magic-trailing-comma = true

[lint]
isort.split-on-trailing-comma = false

extend-select     = ["ALL"]
ignore            = [
  "A001",                                             # builtin-variable-shadowing
  "A004",                                             # builtin-import-shadowing
  "ARG002",                                           # unused-method-argument
  "ANN401",                                           # any-type
  "COM812",                                           # missing-trailing-comma
  "D100",                                             # undocumented-public-module
  "D101",                                             # undocumented-public-class
  "D102",                                             # undocumented-public-method
  "D103",                                             # undocumented-public-function
  "D104",                                             # undocumented-public-package
  "D105",                                             # undocumented-magic-method
  "D107",                                             # undocumented-public-init
  "D203",                                             # incorrect-blank-line-before-class
  "D213",                                             # multi-line-summary-second-line
  "DTZ001",                                           # call-datetime-without-tzinfo
  "E501",                                             # line-too-long
  "FBT003",                                           # enforce at definition site when it is a flag, not a value
  "FIX",                                              # flake8-fixme
  "PLC0415",                                          # `import` should be at the top-level of a file
  "PLR0913",                                          # too-many-arguments
  "PLR2004",                                          # magic-value-comparison
  "PLW1641",                                          # Object does not implement `__hash__` method
  "SLF001",                                           # private-member-access
  "TD003",                                            # missing-todo-link
  "TD004",                                            # missing-todo-colon
  "N801",                                             # capwords naming
  "N806",                                             # lowercase variable
  "PLC0105",
  "S608",                                             # sql injection
]
extend-safe-fixes = ["W293", "FA", "TC", "UP", "EM"]

[lint.per-file-ignores]
"test/*"              = [
  "D",
  "ANN",
  "B018",
  "S101",
  "C901",
  "PLR0912",                                          # too-many-branches
  "PLR0916",                                          # too-many-boolean-expressions
  "S102",                                             # use of exec
  "TRY003",
  "EM101",
  "EM102",
]
"vgplot/_generated/*" = ["ALL"]
"vgplot/__init__.py"  = ["I001", "RUF100", "RUF005"]
"vgplot/encodings.py" = ["I001"]
"vgplot/plot.py"      = [
  "I001",
  "ANN201",
  "ANN204",
  "PLR0911",
  "TRY003",
  "EM101",
  "EM102",
  "PLW2901",
  "A002",
  "FBT001",
  "FBT002"
]
"vgplot/spec.py"      = [
  "I001",
  "C901",
  "PLR0912",
  "TRY003",
  "EM101",
  "EM102",
  "D205",
  "D212",
  "ANN003",
  "ANN001",
  "ANN201",
  "ANN202",
  "N803"                                              # plotDefaults (want to fix)
]

Footnotes

1. I can (and want to) address the rule-breaking later ↩