'print' statement ignores 'sys.stdout'

[alanjds]

google#290 opened by @alanjds on 22 Apr 2017

Replacing sys.stdout have no effect over print statement.

This test script outputs AssertionError: 0 chars printed, instead of 3, after printing 'foo' in the stdout.

import StringIO
import sys

sio = StringIO()
stdout = sys.stdout
sys.stdout = sio

print 'foo'

sys.stdout = stdout

chars = sio.tell()
assert chars == 3, '%s chars printed, instead of 3' % chars

[alanjds]

Comment by trotterdylan
Saturday Apr 22, 2017 at 22:41 GMT

---

Effectively, the sys module does this:

from __go__.grumpy import Stdout

stdout = Stdout

The Print() function in the Go runtime writes directly to grumpy.Stdout. It does not get it from the sys module. So things work fine until you re-bind sys.stdout, as you've found.

There are a couple ways to fix this:

1. Write the sys module in Go and have the grumpy package import it at package init time so that the Print() function can access the sys module and write to the stdout attribute instead of a global Go variable. This is close to how CPython works. However, I think it would require adding some special casing into the import system, which could get ugly.

2. Make the sys module an instance of some class where stdout is a property whose setter updates the value of Stdout in the grumpy runtime package. Because there's no way to assign to Go package variables from Python at the moment, this would require adding a SetStdout() function to the grumpy runtime package, which feels a little hacky.

In short, there's no great way to fix it at the moment, however, I think some avenues will open up once Grumpy is self-hosted.

[alanjds]

Comment by alanjds
Sunday Apr 23, 2017 at 03:57 GMT

---

What about implementing the print statement as a pure-python function, and hook the golang Print function to the python one?

I will need to reimplement the print stuff very soon anyway...

[alanjds]

Comment by trotterdylan
Sunday Apr 23, 2017 at 16:18 GMT

---

If, for example, you define print() in __builtins__.py then you have the problem of accessing the __builtins__ Go package from within the grumpy runtime package, because they each would depend on each other.

[alanjds]

Comment by alanjds
Sunday Apr 23, 2017 at 18:57 GMT

---

I am thinking about defining some _print_statement.py and hooking Print()
to use it.

Ideally, it should import only sys.stdout

I did not yet took a look there to see what the Print() imports

Il 23 apr 2017 1:18 PM, "Dylan Trotter" notifications@github.com ha
scritto:

If, for example, you define print() in builtins.py then you have the
problem of accessing the builtins Go package from within the grumpy
runtime package, because they each would depend on each other.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
google#290 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAJf5ywht3Zc_SoCZu71nbK2xoVoBm7Dks5ry3m_gaJpZM4NFOm8
.

[alanjds]

Comment by trotterdylan
Sunday Apr 23, 2017 at 19:13 GMT

---

I think the fundamental issue you're doing to run into is circular
dependencies between the grumpy runtime and the packages generated from the
Python module you write.
On Sun, Apr 23, 2017 at 11:57 AM Alan Justino da Silva <
notifications@github.com> wrote:

I am thinking about defining some _print_statement.py and hooking Print()
to use it.

Ideally, it should import only sys.stdout

I did not yet took a look there to see what the Print() imports

Il 23 apr 2017 1:18 PM, "Dylan Trotter" notifications@github.com ha
scritto:

If, for example, you define print() in builtins.py then you have the
problem of accessing the builtins Go package from within the grumpy
runtime package, because they each would depend on each other.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
google#290 (comment),
or mute
the thread
<
https://github.com/notifications/unsubscribe-auth/AAJf5ywht3Zc_SoCZu71nbK2xoVoBm7Dks5ry3m_gaJpZM4NFOm8

.

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
google#290 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAHLexHc0dg-_mbK7ycf_BujVrHN6Psdks5ry58EgaJpZM4NFOm8
.

[alanjds]

Comment by alanjds
Sunday Apr 23, 2017 at 22:20 GMT

---

I just found the Print() function, and it should not rely on 'file' arg being a File anyway, otherwise "print" will never accept a duck-typed file-like, in my opinion.

For now, my plan is to factor it out from builtins, even if it is always loaded

Will draw some sketches on dead-trees to help with the circular dependencies. Maybe lazy-importing sys.Stdout, with a local cache to not penalize the most used case. I like it not, however.

[alanjds]

Comment by trotterdylan
Sunday Apr 23, 2017 at 22:22 GMT

---

+1 to taking any object instead of a file. Looking forward to seeing what
you come up with.
On Sun, Apr 23, 2017 at 3:20 PM Alan Justino da Silva <
notifications@github.com> wrote:

I just found the Print() function
https://github.com/google/grumpy/blob/91ffc9fd18ee51cce79d2930e6850e284c3cd721/runtime/core.go#L1261,
and it should not rely on 'file' arg being a File anyway, otherwise "print"
will never accept a duck-typed file-like, in my opinion.

For now, my plan is to factor it out from builtins, even if it is always
loaded

Will draw some sketches on dead-trees to help with the circular
dependencies. Maybe lazy-importing sys.Stdout, with a local cache to not
penalize the most used case. I like it not, however.

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
google#290 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAHLe2DGzZa32iGJWegwGVKaV1agVe62ks5ry86YgaJpZM4NFOm8
.

[alanjds]

Comment by cclauss
Monday Apr 24, 2017 at 08:03 GMT

---

Are there a test cases for Python scripts that start with from future import print_function? That import makes the printer statement, the >> and the trailing comma all invalid. It also enables the end function parameter.

[alanjds]

Comment by alanjds
Monday Apr 24, 2017 at 14:11 GMT

---

@cclauss Yeah, but is a flag already handled by the compiler. I remember to saw some ifs in the compiler code about this. Print from "print statement" and from "print function" are different code paths, per file compiled, IIUC.

[alanjds]

Comment by alanjds
Monday Apr 24, 2017 at 14:29 GMT

---

@trotterdylan: Thinking in Print() check for 'sys' in SysModules and, if found, use 'sys.stdout'. If not found, fallback to Stdout as the 'sys' was not imported and sys.stdout is sure to be Stdout anyway.

[alanjds]

Comment by trotterdylan
Tuesday Apr 25, 2017 at 04:42 GMT

---

Although that solution should work (if sys was not imported, then there's no way for someone to override it from Python), it's not very satisfying IMO. It's pretty easy to imagine a scenario where this logic would diverge from CPython (probably involving messing around with sys.modules). Although these scenarios are far fetched, I'm more worried that this implementation divergence from CPython will cause other, more subtle problems. I think the right approach is to make the sys module "built in" in a similar way to CPython.

A variation on option 1. above is to do exactly what CPython does and define the sys module within the runtime itself: https://github.com/python/cpython/blob/2.7/Python/sysmodule.c

This would still require some changes to the import system to special case sys, but it's probably the best option we have available in the short term.

[alanjds]

Comment by corona10
Tuesday Apr 25, 2017 at 07:04 GMT

---

@trotterdylan I definitely agree with your idea. If we need modules of python which are implemented by C-level API. At that case, We should implement it by Go also.

[alanjds]

Comment by alanjds
Tuesday Apr 25, 2017 at 15:08 GMT

---

I do not know if I like this path.

I remember of news of some conference (could not find the reference, sorry) where people of different Python "runtimes" including CPython agreed that the best would be to write the stdlib in pure-python if possible, falling back to C-API only for serious speed needs or low-level access needs.

In the long term, do you expect to be easier to maintain a pure-python sys or a Go sys?
For now, the pure-python code have 73 LOC. The CPython one have 1801 LOC.
And I do not see a huge speed improvement on this path.

Another point is that, as we are borrowing PyPy and Ouroboros code, they could take it back and share the maintenance burden with us, but only if our modules are useful to him.

For now, my view of the thing is to implement whatever possible in pure-python (will be transpiled to Go anyway...), not worrying with implementation divergence from CPython. The result could be simpler and easier to maintain. And more useful to other pals too (PyPy for exemple).

[alanjds]

Comment by alanjds
Tuesday Apr 25, 2017 at 15:12 GMT

---

Be clear that it is my philosophical perspective, more aligned with PyPy "view". But a view like "lets transpile CPython code to Go, whatever it is a good code or not" will sure work too, and will easily emulate the goods and bads there. Including maintenance ones.

[alanjds]

Comment by alanjds
Tuesday Apr 25, 2017 at 15:19 GMT

---

Please confirm your spirits. I can wait or can give a shot, even if to change it later. just confirm.