ENH: read TOML files for configurations

[TTsangSC]

Synopsis

This PR adds the capability to configure kernprof and line_profiler via TOML files like pyproject.toml.

Namespaces

• [tool.line_profiler.kernprof]:
  Defaults for command-line options of kernprof
• [tool.line_profiler.cli]:
  Defaults for command-line options of python -m line_profiler
• [tool.line_profiler.setup]:
  Defaults for the line_profiler.profile.setup_config table
• [tool.line_profiler.write]:
  Defaults for the line_profiler.profile.write_config table, except for the output_prefix key, which maps instead to line_profiler.profile.output_prefix
• [tool.line_profiler.show]:
  Defaults for the line_profiler.profile.show_config table, except for the subtable [tool.line_profiler.show.column_widths] (see below)
• [tool.line_profiler.show.column_widths]:
  Default column widths for line_profiler.LineProfiler.print_stats(), line_profiler.line_profiler.show_text(), etc.

Specification and discovery

The TOML config file in question can be specified by

• (Updated 18 May) Passing the new --config flag (or -c for python -m line_profiler) to the kernprof and python -m line_profiler CLI apps;
• Passing the new config argument to line_profiler.explicit_profiler.GlobalProfiler.__init__(), line_profiler.line_profiler.LineProfiler.print_stats(), and line_profiler.line_profiler.show_text() and .show_func(); or
• Setting the LINE_PROFILER_RC environment variable.

Failing that, the config file is looked up on the file system starting from the current directory. The filenames pyproject.toml and line_profiler.toml (the latter taking priority) are checked, and if any of them is valid TOML it is chosen to load configs from. Otherwise, we check in the parent directory, and so on, until we reach the file-system/drive root.

The looked up file (if any) is then merged with the default configs (line_profiler/rc/line_profiler.toml) to form the final configuration table, which is guaranteed to contain the same subtables and keys as the default one.

Motivation

In #323's discussion (comments 1, 2, 3), it was suggested that TOML config files can be used to:

• Avoid common repetition patterns like kernprof -l -p <script> <script>, and
• Make hard-coded tables like line_profiler.explicit_profiler.GlobalProfiler.show_config configurable.

It was also suggested that all config options used by this package be placed under the tool.line_profiler namespace.

Code changes

• line_profiler/rc:
  New subpackage for containing config files (currently only line_profiler.toml)
• line_profiler/cli_tools.py:
  New module for common functions shared between kernprof and python -m line_profiler
• line_profiler/toml_config.py:
  New module for finding and loading configs from TOML files
  • get_config(): load the configs from an explicitly-supplied TOML file, the file discovered from the environment, or the default file
  • get_default_config(): load the configs from the default TOML file (line_profiler/line_profiler.toml)
• kernprof.py, line_profiler/line_profiler.py:
  • Moved common utilities used by the CLI apps out to line_profiler.cli_tools
  • (Updated 18 May) Added new option --config (and -c only for line_profiler/line_profiler.py since kernprof -c is taken (ENH: auto-profile stdin or literal snippets #338)) for specifying a TOML config file
  • Added new option --no-config for disabling loading of TOML configs (that isn't the default file)
  • For each boolean option --some-flag (action='store_true' or 'store_false'):
    • The default value is None and the action replaced with 'store_const' or 'store', so that:
      • Boolean options can default to true (based on the configs), and
      • We can distinguish between when the option is explicitly set to false and when it is not supplied
    • The long and short flags are shunted into separate actions, so that:
      • Short-flag actions remain simple 'store_const's, which allow them to be concatenated
      • Long-flag actions can optionally take a single argument, which is parsed into a boolean (allowing invocations like --some-flag=false --other-flag, which resolves into Namespace(some_flag=False, other_flag=True))
    • A complementary --no-some-flag is added
  • Added the loaded-from-config defaults to the option help texts
  • Updated the --help output
  • Most options except special ones (-V/--version; -c/--config/--no-config; -m) now have corresponding TOML entries (in [tool.line_profiler.kernprof] and [tool.line_profiler.cli]) and load their defaults therefrom
• kernprof.py:
  • Updated __doc__
  • Grouped CLI options with argument groups
  • Revised the help text of --prof-imports to better reflect its function
  • Updated the const (value stored when the flag is passed without an argument) of the -i/--output-interval option, since the old value (0) is un-intuitively equivalent to disabling it
  • Removed redundant instantiation of RepeatedTimer
  • Wrapped some long lines
• line_profiler/line_profiler.py:
  • Added the argument config to LineProfiler.print_stats(), show_func(), and show_text() so that the output column widths can be customized
    • The minimal widths are pinned to the default values, so that the output columns never become too narrow
  • Replaced hard-coded table of column widths in show_func() with the config table [tool.line_profiler.show.column_widths]
• line_profiler/explicit_profiler.py:
  • Added the argument config to GlobalProfiler.__init__() so that configs can be explicitly specified
  • Replaced hard-coded values for GlobalProfiler.setup_config, .write_config, .show_config, and .output_prefix with the config tables [tool.line_profiler.setup], [.write], and [.show]

Packaging changes

• line_profiler/rc/line_profiler.toml:
  New global-default config file
• requirements/runtime.txt:
  Updated so that tomli is installed for Python versions (< 3.11) without tomllib in the stdlib
• setup.py, MANIFEST.in:
  Updated so that line_profiler/rc/line_proiler.toml is included in both wheels and source distributions

Test suite changes

• tests/test_toml_config.py:
  New tests for line_profiler/toml_config.py
  • test_environment_isolation():
    Test that tests in this test module are reasonably isolated from its environment (env variables and file system)
  • test_default_config_deep_copy():
    Test that each call to line_profiler.toml_config.get_default_config() returns a fresh and separate set of configs
  • test_table_normalization():
    Test that config files with missing and/or extra keys are normalized into the same layout as the default file
  • test_malformed_table():
    Test that malformed config files (e.g. one which sets [tool.line_profiler.show.column_widths] to an array, instead of the expected table result in a ValueError
  • test_config_lookup_hierarchy():
    Test that the choice of config file follows the resolution scheme outlined in Specification and discovery
• tests/test_explicit_profile.py:
  • Refactored existing tests to use tempfile.TemporaryDirectory to ensure the removal of the temporary directories
  • test_explicit_profiler_with_customized_config():
    New test for the configuration of line_profiler.explicit_profiler.GlobalProfiler (i.e. @line_profiler.profile) via TOML config files
• tests/test_autoprofile.py:
  • Refactored existing tests to use tempfile.TemporaryDirectory to ensure the removal of the temporary directories
  • test_autoprofile_with_customized_config():
    New test for the configuration of kernprof, python -m line_profiler, and line_profiler.line_profiler.show_text() via TOML config files
  • test_autoprofile_with_no_config():
    New test for disabling user-config loading in kernprof and python -m line_profiler via the --no-config flag
• tests/test_cli.py:
  • test_boolean_argument_help_text():
    New test for the augmented help texts generated by line_profiler.cli_utils.add_argument() for boolean options
  • test_boolean_argument_parsing():
    New test for the parsing of boolean options generated by line_profiler.cli_utils.add_argument()

Caveats

This PR further couples kernprof to line_profiler. But it's probably fine, given that:

• The ship has sailed as of FIX: kernprof erroring out in non--l, non--b mode in Python 3.12+ #326, which made kernprof.contextualProfiler directly dependent on line_profiler.profiler_mixin.ByCountProfilerMixin.
• The rationale for isolating the parts where kernprof is dependent on line_profiler and making the former (somewhat) usable without the latter was to avoid necessitating the configuration of C(-ython) compilation. But since thw whole package can now just be pip install-ed with no additional configuration, it's a mostly solved problem.

Acknowledgement

The idea for the PR originated with @Erotemic.

[codecov]

Codecov Report

Attention: Patch coverage is 83.47107% with 40 lines in your changes missing coverage. Please review.

Project coverage is 68.46%. Comparing base (0ab4471) to head (3e12ee8).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
line_profiler/cli_utils.py	74.44%	16 Missing and 7 partials ⚠️
line_profiler/toml_config.py	86.53%	11 Missing and 3 partials ⚠️
line_profiler/line_profiler.py	93.10%	1 Missing and 1 partial ⚠️
line_profiler/explicit_profiler.py	94.73%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #335      +/-   ##
==========================================
+ Coverage   64.84%   68.46%   +3.62%     
==========================================
  Files          13       15       +2     
  Lines        1041     1243     +202     
  Branches      228      277      +49     
==========================================
+ Hits          675      851     +176     
- Misses        306      326      +20     
- Partials       60       66       +6     

Files with missing lines	Coverage Δ
line_profiler/explicit_profiler.py	93.42% <94.73%> (+4.39%)	⬆️
line_profiler/line_profiler.py	74.66% <93.10%> (+2.67%)	⬆️
line_profiler/toml_config.py	86.53% <86.53%> (ø)
line_profiler/cli_utils.py	74.44% <74.44%> (ø)

... and 3 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b34b801...3e12ee8. Read the comment docs.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Erotemic]

Getting a start on this review. There are a lot of changes, so I'm going to need to do a few passes over it before we merge, but at a high level it looks very good.

In my first pass, I see a few things:

• I think we should use line_profiler.toml instead of line_profiler_rc.toml. It seems like it is more standard to just use .toml (e.g. cibuildwheel, black).

• Should tomli be an optional requirement in Python <3.11? I'm on the fence about this, on one hand it's such a small package, on the other, I do like keeping all dependencies optional for packages like this.

• Can we defer the call to get_default_config() in the top-level? I prefer to avoid executing any code at module import time.

• For the --config option, we should add the ability to set it to None or null to indicate that all external configs should be ignored.

• For resource files I like having a .rc submodule to store them and logic to access them. So I think we should make a line_profiler.rc submodule here. Here is an example of a place where I've done this before.

• FYI: we can keep the changes to the doctests where >>> are replaced with ... , but a main motivation of xdoctest is to allow the more flexible syntax.

I'm also curious, what was the LLM usage on this PR like? I'm wondering if the minor changes came from a LLM editor, or if you were explicitly making the style consistent.

[TTsangSC]

line_profiler.toml

having a .rc submodule to store them

Good point, will do. So basically instead of line_profiler/line_profiler_rc.toml we do line_profiler/rc/line_profiler.toml?

tomli

Yeah this could use some deliberation... on the one hand, it's very convenient to just have to specify all the config in a central TOML file (including our defaults) and read from it, but on the other it does introduce a dependency. Maybe just for the sake of keeping the requirement list short, it is worth it to have a bit of duplication and put the default configs also in the Python code... it's a tradeoff that we have to decide on.

(One concern though is that that may cause the behaviors between platforms to go out of sync, if a future contributor (could easily be us in a month) only updated the code-side configs and forgot to do so to the rc-side configs. But since our dev machines all have newer Python versions I guess we can put a dev/ script for syncing the two, or maybe include a test to automatically check for equivalence.)

get_default_config()

I guess you mean when we call that to get the line_profiler/line_profiler.py::minimum_column_widths? I see what you mean, guess that it can be isolated into a @functools.lru_cache-ed function.

indicate that all external configs should be ignored

That's a brilliant idea! Yep, we're currently missing the option to ignore config files, and that'd be helpful.

xdoctest

making the style consistent

I see your point. In hindsight I was perhaps overzealous in making those changes... the intent was to make them backwards compatible with vanilla doctest, but then again the examples probably wouldn't have been fully backwards compatible anyway, given the differences in how xdoctest and doctest function (e.g. the parsing and verification of expected outputs). Sorry if that made for unduly big diffs. (Same for the reflowing of lines, I just find long lines a bit hard to work with...)

LLM

Never really used one – I'm staunchly a vim + terminal guy, and perhaps too much so for my own good. But my colleagues seem to be having a blast using Copilot as a productivity tool, and maybe it's worth looking into someday...

[Erotemic]

Yes: line_profiler/rc/line_profiler.toml specifies the default toml config. The user can choose to overwrite / merge it with their local config based on the priority rules you outlined for line_profiler.toml and pyproject.toml.

On tomli: I'm fine with keeping it as is. It's so small and standard, and modern Python versions don't need it.

On @functools.lru_cache-ed function: That works for me, and is how I've handled it in the past. I try to make imports happen as soon as possible. Nothing is worse than a bottleneck in response time when you startup python (e.g. torch, pytorch-lightning, etc...).

The option to ignore configs is critical for testing and debugging.

xdoctest: it's fine to leave it. The more compatible the better I suppose. All doctest code should be forward compatible with xdoctest. Using xdoctest as a runner isn't supposed to be "better", it just makes it faster for me to write - although I suppose it would be possible to write a "normalizer" in xdoctest so it can factor itself out if needed.

LLM: I'm gvim+terminal as well (gvim users exist), but I have been using the free LLMs like DeepSeek, ChatGPT, and a few ones I can run locally. They save a ton of time. It would be interesting to see if you gave it a default config and asked it to generate documentation (maybe provide a diff of the other code for context, so it can figure out what they do), and see how that compares to the docs you wrote.

First thing I did for this is throw the diff at an LLM and ask it for places to look into first. It was also used for finding things (like checking if you did implement the null config case, or if it just wasn't documented). All my usage is copy+paste, but I've also been considering a switch to vscode to get copilot. If you have a GPU, open-webui in a docker container is a nice way to use them. I think nvim might have good integrations, but vim/gvim do seem to be lacking.

[Erotemic]

Partial review while I had a spare cycle.

[Erotemic]

Suggested change

	Run and profile a python script or module.Boolean options can be negated by passing the corresponding flag (e.g. `--no-view` for `--view`).
	Run and profile a python script or module.

The extra docs "Boolean options can be negated by passing the corresponding flag (e.g. --no-view for --view" should probably be less prominent in some notes section if we want it at all.

What do you think about instead of using the --flag --no-flag convention, using --flag always results in setting the option to True, but if you want to explicitly set it you use a key/value pair --flag=True or --flag=False.

Another one of my design soapboxes is that you shouldn't have to delete a flag to disable it, you should always be able to set state via key/value pairs on the command line. However, I didn't add that feature here as my primary goal with this package is maintenance and facilitating development from other devs like yourself, as I spend quite a bit of effort developing other packages.

One of those packages is scriptconfig. I don't want to add a dependency on it here, but we could borrow some code from it. Namely: BooleanFlagOrKeyValAction, which allows for flexible specification of boolean variables, e.g.:

        --flag        > {'flag': True}
        --flag=1      > {'flag': True}
        --flag True   > {'flag': True}
        --flag True   > {'flag': True}
        --flag False  > {'flag': False}
        --flag 0      > {'flag': False}
        --no-flag     > {'flag': False}
        --no-flag=0   > {'flag': True}
        --no-flag=1   > {'flag': False}

[TTsangSC]

Agreed, big fan of this style of boolean flags (I think I did the same for pytest-autoprofile); will implement.

[TTsangSC]

Just realized that there will be unintended consequences: short flags can no longer be concatenated since they are no longer simple boolean flags. But that can be circumvented by creating separate actions for the short and long flags I guess.

[Erotemic]

I think we can just remove one of these. I noticed this too, and I think it was a mistake.

[Erotemic]

I've also tackled this in scriptconfig, although the code for doing so was complicated by internal changes to argparse in python/cpython@c02b7ae

Code is here:
https://gitlab.kitware.com/utils/scriptconfig/-/blob/main/scriptconfig/argparse_ext.py?ref_type=heads#L423

Might not be worth the extra complexity, but I figured I would point it out.