Streaming json instead of loading whole object to memory.

[noway]

Not always stdin give you all data instantly. So instead of waiting EOF, would be better to process json instantly.
Some implementations of json readers with stream support:
https://github.com/dominictarr/JSONStream
https://pykler.github.io/yajl-py/
https://github.com/rtyler/py-yajl/

[amarao]

Thanks, I'll look to it.

[amarao]

I've checked JSONStream. May be I doing something wrong, but it does not process generator input until it done. Means it can't parse incoming stream. Failed attemt here: https://github.com/amarao/json4shell/tree/jsonstream/tools

[noway]

sys.stdout.flush() was the case. Also, appears that sys.stdin is not iterable in python2.
I fixed it and it's working now.
JSONSream is really nice choice, by the way. It doesn't require compilation, and it's written in pure python. Though it can be compiled as native python module (through cython), python version works fine. It can be even included directly in git repo! So there would be no need to do some extra deploy work while installing json4shell.

[amarao]

Thanks!

I'll recheck it again.

On 06/17/2014 04:49 PM, Way, No wrote:

|sys.stdout.flush()| was the case. Also, appears that sys.stdin is not
iterable in python2.
I fixed it and it's working now.
JSONSream is really nice choice, by the way. It doesn't require
compilation, and it's written in pure python. Though it can be
compiled as native python module (through cython), python version
works fine. It can be even included directly in git repo! So there
would be no need to do some extra deploy work while installing json4shell.

—
Reply to this email directly or view it on GitHub
#3 (comment).