Skip to content

Conversation

@francis-shuoch
Copy link

I write a python decorator to silence prints' output into stdout. Meanwhile sendMsgToParent() function doesn't get affected, so the communication to Java Layer is not affected, too. Use this wrapper on any custom python scripts containing prints without worrying those prints would mess up the tuple emits. I tested with storm-starter's WordCountTopology.

stdout. Meanwhile sendMsgToParent() function doesn't get affected, so the
communication to Java Layer is not affected, too. Use this wrapper on
any custom python scripts containing prints without worrying those
prints would mess up the tuple emits. I tested with storm-starter's WordCountTopology.
@xumingming
Copy link
Collaborator

So developers need to call this nullify_stdout method manually?

@francis-shuoch
Copy link
Author

Yes, as lbustelo pointed out in issue #351, custom python function containing prints can use this decorator to wrapper it, so user defined prints won't mess up the tuple emits from the bolts. Like this:

@nullify_stdout
def custom_fun:
    print "this won't come into python process's stdout to mess up emits."

sorry I didn't add an example like this in my PR.

@francis-shuoch
Copy link
Author

Another way is to assign sys.stdout = NullDevice() globally in the module, while sendMsgToParent() use original_stdout so it can write to stdout normally. But this way I don't know when to reassign sys.stdout back. Any thoughts here? I would like to follow it.

@xumingming
Copy link
Collaborator

Would it be better if storm.py handles all the silence prints details, so it will be transparent to the developers?

@francis-shuoch
Copy link
Author

Then I think assigning sys.stdout = NullDevice() globally in the module while only sendMsgToParent() using original_stdout will suffice. What do you think?

@xumingming
Copy link
Collaborator

About reassigning sys.stdout back, one way I can think of is wrapping all calls to self.process(tuple), self.initialize(stormconf, context) etc with your nullify_stdout.

@francis-shuoch
Copy link
Author

I gave it some fresh thoughts today, now I think, since all prints will be read by storm's JVM-component, we must forbid any prints outside of sendMsgToParent() function. So, there is no need to reassign sys.stdout back. Does this make sense to you?

Or, alternatively, we can reassign sys.stdout back at the end of run() function, since this should be the last call of custom python script (see splitsentence.py example for reference).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants