ENH: auto-profile stdin or literal snippets

[TTsangSC]

This PR is inspired by #323, which added the -m option to kernprof so that one can profile the execution of modules and packages. Much like how kernprof <script> parallels python <script>, and kernprof -m <module> python -m <module>, there may be use-cases that call for similar parallels for:

• python -c <code> (running literal code), and
• ... | python - or python - <<<"..." (running code from stdin).

Hence this PR which adds to kernprof:

• Support for the special option -c (like how -m is implemented), which terminates the parsing of options and runs the following argument as literal Python code, and
• Support for the special script name -, which prompts kernprof to read from stdin.

It is implemented by a minor refactoring, which writes the received code to a temporary file and profiles that instead.

2 tests are also added:

• test_autoprofile_from_stdin(): test for ... | kernprof -.
• test_autoprofile_from_inlined_script(): test for kernprof -c <code>.

EDIT 6 May: added documentations on the -m, -c, and - invocations for kernprof.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 64.84%. Comparing base (771d0b7) to head (96047f4).
Report is 10 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #338      +/-   ##
==========================================
+ Coverage   63.78%   64.84%   +1.05%     
==========================================
  Files          13       13              
  Lines        1041     1041              
  Branches      228      228              
==========================================
+ Hits          664      675      +11     
+ Misses        316      306      -10     
+ Partials       61       60       -1     

see 1 file with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e35b6cf...96047f4. Read the comment docs.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Erotemic]

This PR looks easy to review. Can you rebase to resolve conflicts?

[TTsangSC]

Done! Will do the same for the other PRs.

[Erotemic]

As usual you've done great work and I only have minor comments.

[Erotemic]

What is the purpose of parsing and unparsing? Should we output a warning if the exception case is hit?

[TTsangSC]

It's basically just for putting each statement onto its own line, so that we can show and profile the code more cleanly. (The caveat is that the piped-in code is lost after running the kernprof process anyway, so it can't be python -m line_profiler-ed and only benefits kernprof --view).

Either way I agree, we should probably put a warning in case if the round-tripping goes south.

[Erotemic]

Hmm, might it be unexpected if the user is trying to profile a specific statement and its broken up in unexpected ways? OTOH, it might be convenient to have those statements automatically broken up instead of needing to do it yourself after you realize all of the time is on a single line, and you don't know which part is causing it (something that happens a lot when I profile code normally).

But given that the input to -c can be multi-line, it might be safer to opt for the simpler case first and profile exactly what the user gave. Adding an example to the docs that demonstrates how to use this and what expected results would look like might help to decide on this point (and is probably something we want anyway).

[TTsangSC]

... fair enough, will do.

As for the docs thing, I realized that we kinda don't have docs for the -m flag either (outside of the CHANGELOG and kernprof --help. Should we fix that too in this PR or a separate one?

[Erotemic]

I tend to prioritize docs as-needed. This PR is small, so if you want to add docs for that you can do it in this PR.

For new manually written documentation, there are two ways it can be done:

1. Put it in a docstring and have the autodocs handle it.
2. Manually write an rst file and put it in docs/source/manual (maybe in the examples subfolder) and then link to it in docs/source/index.rst

For this case, because the feature isn't tied to a module it probably makes more sense to chose option 2, but there is an argument for putting it in kernprof's docstring (I do like coupling docs and code, but at some point it does get excessive).

[TTsangSC]

Unfortunately docs/source/manual/examples seems to be broken – though we have

.. toctree::
    :maxdepth: 8
    :caption: Package Layout

    auto/line_profiler
    auto/line_profiler.autoprofile
    auto/line_profiler.explicit_profiler
    auto/kernprof

    manual/examples/index

the examples aren't included in the generated doc. Will also try to fix, but I'll have to first figure sphinx out.

[Erotemic]

As long as you write the docs in there, it's good enough. If it's broken, that might be on me to fix.

You should be able to simply run make html in the docs folder to get everything to build, and then you can check locally. I would not expect the latest URL on readthedocs to include anything from a PR.

[TTsangSC]

The build worked alright on local but the outputs are rather different from the readthedocs version (see attachments)... haven't made all that many changes yet, so IDK what is happening here.

[Erotemic]

It might be that the RTD integration hasn't updated itself in a while. In fact, the hash shows that it isn't: d6a2ef4 is from 2023-12-05, which is confusing because that doesn't even correspond cleanly to a release. It is a few commits after v4.1.2, so something didn't work right for the 4.1.3 or 4.2.0 release.

I don't even see the project in my RTD dashboard, so something weird is happening. I'll look into it.

EDIT: It looks like RTD updated the webhooks on their end, so probably none of my stuff is getting updated. I think I was able to fix it, and the latest now shows a more recent version of the docs.

[TTsangSC]

Good to hear! I've added docs for the aforementioned changes in eb451de and e0a5e7d.

[Erotemic]

We should probably call textwwrap.dedent on the code. This feature will be part of CPython in 3.14: python/cpython#103998

[TTsangSC]

Good catch, thanks. This reminds me, I should probably get 3.14 and play around with it.

[TTsangSC]

Thanks for the review! Just implemented all the suggestions.

[Erotemic]

In addition to the bashisms I specifically mention, I'm on the fence about using the PS1/PS2 prompt strings to separate code from output. On the one hand, it's standard and I see it everywhere. On the other hand, I hate it; It prevents the user from just copy/pasting the code into the terminal and trying it out themselves.

This is definitely one of my personal soapboxes, and its very much an opinion. But at the same time, I know I have other documentation which actively avoids it, so it would be inconsistent to have both in the same code base.

At the same time, the my alternative has issues. I usually do things like:

Run this:

echo "hello world"

and the output is

hello world

which isn't nearly as clean looking as:

$ echo "hello world"
hello world

So on one hand, the style I hate looks better in docs, but on the other hand it hinders user interaction, and requires precise mouse selection that my old hands scream about. I'd be interested in any brainstorming on potentially better ways to do this.

[Erotemic]

Suggested change

	$ code="import sys; "
	$ code+="from fib import _run_fib, fib_no_cache as fib; "
	$ code+="for n in sys.argv[1:]: print(f'fib({n})', '=', fib(int(n)))"
	$ kernprof --prof-mod fib._run_fib --line-by-line --view -c "${code}" 10 20
	$ kernprof --prof-mod fib._run_fib --line-by-line --view -c "
	import sys
	from fib import _run_fib, fib_no_cache as fib
	for n in sys.argv[1:]:
	print(f'fib({n})', '=', fib(int(n)))
	" 10 20

I strongly recommend just using newlines in your invocation of -c, rather than setting up a convoluted semicolon separated set of statements in a bash variable. (This is the reason dedent is important for -c arguments, although previously you could do something like:

python -c "if 1:
    for i in range(2):
        print('hello multiline world')
"

Furthermore, this example does not work. In your version I get:

  File "/tmp/tmprybp7qo7/kernprof-command.py", line 1
    import sys; from fib import _run_fib, fib_no_cache as fib; for n in sys.argv[1:]: print(f'fib({n})', '=', fib(int(n)))
                                                               ^^^
SyntaxError: invalid syntax

And in my version I get:

  File "/tmp/tmphmw5r33x/kernprof-command.py", line 3, in <module>
ModuleNotFoundError: No module named 'fib'

Is that one of the issues addressed by a different PR?

[TTsangSC]

Yep I did mess up with this example; will fix as suggested.

It's probably not related to any PRs though, since your ModuleNotFoundError is just a result of not having ${PWD} in ${PYTHONPATH} (which as stated in the intro of this page is assumed by the examples). Will add a PYTHONPATH="${PYTHONPATH}:${PWD}" to make it more compatible EDIT: I meant portable.

[Erotemic]

Suggested change

	$ read -d '' -r code <<-'!'
	> from fib import fib
	>
	> def my_func(n=50):
	> result = fib(n)
	> print(n, '->', result)
	>
	> my_func()
	> !
	$ kernprof -lv -c "${code}"
	$ kernprof -lv -c "
	from fib import fib
	def my_func(n=50):
	result = fib(n)
	print(n, '->', result)
	my_func()
	"

Would read better without bashisms.

[Erotemic]

Suggested change

	$ kernprof --prof-mod fib._run_fib --line-by-line --view - 10 20 <<-'!'
	> import sys
	> from fib import _run_fib, fib_no_cache as fib
	> for n in sys.argv[1:]:
	> print(f"fib({n})", "=", fib(int(n)))
	> !
	$ kernprof --prof-mod fib._run_fib --line-by-line --view - 10 20 <<-'!'
	$ echo '
	import sys
	from fib import _run_fib, fib_no_cache as fib
	for n in sys.argv[1:]:
	print(f"fib({n})", "=", fib(int(n)))
	' | kernprof --prof-mod fib._run_fib --line-by-line --view - 10 20

Debatable which one is better here.

[TTsangSC]

Regarding what you said about separating Bash/shell inputs from outputs, I wonder if interlacing normal (input) and collapsible (output) code blocks helps in delineating the two and keeping thing relatively compact?

Thanks for the comments, will take a closer look after dinner.

[Erotemic]

The collapsible output looks like a nice way to handle it.