Implement `times` builtin #441

timetoplatypus · 2019-07-20T02:43:18Z

I'm working on a pull request to implement this feature. Things have changed a lot since the last time I worked on OSH, so I'm encountering errors I'm not familiar with, etc.

Planning to use the following module in the implementation:
https://docs.python.org/2.7/library/resource.html

andychu · 2019-07-20T03:25:55Z

OK great! What are the errors? You can post them here or on #oil-dev on Zulip.

resource.c appears here, so we have the resource module:

https://www.oilshell.org/release/0.7.pre1/metrics.wwz/line-counts/nativedeps.txt

But I would still accept a PR if we didn't. "Make it work" and "make it small" are separate steps, etc.

timetoplatypus · 2019-07-20T04:48:09Z

$> build/dev.sh all
removed '_devbuild/gen/find_asdl.py'
removed '_devbuild/gen/find.marshal'
removed '_devbuild/gen/find_nt.py'
removed '_devbuild/gen/grammar.marshal'
removed '_devbuild/gen/grammar_nt.py'
removed '_devbuild/gen/id.h'
removed '_devbuild/gen/id_kind_asdl.py'
removed '_devbuild/gen/id_kind_asdl.pyc'
removed '_devbuild/gen/__init__.py'
removed '_devbuild/gen/__init__.pyc'
removed '_devbuild/gen/osh_help.py'
removed '_devbuild/gen/osh-lex.re2c.h'
removed '_devbuild/gen/osh-types.h'
removed '_devbuild/gen/runtime_asdl.py'
removed '_devbuild/gen/syntax_asdl.py'
removed '_devbuild/gen/syntax_asdl.pyc'
removed '_devbuild/gen/typed_arith_asdl.py'
removed '_devbuild/gen/typed_demo_asdl.py'
removed '_devbuild/gen/types_asdl.py'
removed '_devbuild/gen/types_asdl.pyc'
Wrote _devbuild/osh-quick-ref/ and _devbuild/gen/osh_help.py
Wrote _release/VERSION/doc/osh-quick-ref.html
renamed '_tmp/types_asdl.py' -> '_devbuild/gen/types_asdl.py'
Wrote _devbuild/gen/types_asdl.py
renamed '_tmp/syntax_asdl.py' -> '_devbuild/gen/syntax_asdl.py'
Wrote _devbuild/gen/syntax_asdl.py
renamed '_tmp/runtime_asdl.py' -> '_devbuild/gen/runtime_asdl.py'
Wrote _devbuild/gen/runtime_asdl.py
renamed '_tmp/find_asdl.py' -> '_devbuild/gen/find_asdl.py'
Wrote _devbuild/gen/find_asdl.py
480 _devbuild/gen/typed_demo_asdl.py
554 _devbuild/gen/typed_arith_asdl.py
Compiled oil_lang/grammar.pgen2 -> _devbuild/gen/grammar.marshal and _devbuild/gen/grammar_nt.py
Compiled tools/find/find.pgen2 -> _devbuild/gen/find.marshal and _devbuild/gen/find_nt.py
running build
running build_ext
building 'libc' extension
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-Ra3Yar/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -UNDEBUG -I/usr/include/python2.7 -c native/libc.c -o build/temp.linux-x86_64-2.7/native/libc.o
creating _devbuild/py-ext/x86_64
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-Ra3Yar/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-Ra3Yar/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC build/temp.linux-x86_64-2.7/native/libc.o -o _devbuild/py-ext/x86_64/libc.so
'libc.so' -> '_devbuild/py-ext/x86_64/libc.so'
libc.so: symbolic link to _devbuild/py-ext/x86_64/libc.so
.....real	0.100
user	0.200
sys	0.300
.......
----------------------------------------------------------------------
Ran 12 tests in 0.001s

OK
running build
running build_ext
building 'line_input' extension
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-Ra3Yar/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -UNDEBUG -I/usr/include/python2.7 -c native/line_input.c -o build/temp.linux-x86_64-2.7/native/line_input.o
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-Ra3Yar/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-Ra3Yar/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC build/temp.linux-x86_64-2.7/native/line_input.o -lreadline -o _devbuild/py-ext/x86_64/line_input.so
'line_input.so' -> '_devbuild/py-ext/x86_64/line_input.so'
line_input.so: symbolic link to _devbuild/py-ext/x86_64/line_input.so
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK
running build
running build_ext
building 'posix_' extension
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-Ra3Yar/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Ibuild/oil-defs -I/usr/include/python2.7 -c native/posixmodule.c -o build/temp.linux-x86_64-2.7/native/posixmodule.o
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-Ra3Yar/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-Ra3Yar/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC build/temp.linux-x86_64-2.7/native/posixmodule.o -o _devbuild/py-ext/x86_64/posix_.so
'posix_.so' -> '_devbuild/py-ext/x86_64/posix_.so'
posix_.so: symbolic link to _devbuild/py-ext/x86_64/posix_.so
........
----------------------------------------------------------------------
Ran 8 tests in 0.001s

OK
-- Generating AST, IDs, and lexer in _devbuild/gen
build/codegen.sh: line 54: _deps/re2c-1.0.3/re2c: No such file or directory

After cloning the repository (no modifications made to the codebase), I try to build everything. I get a No such file or directory error at the very end.

That's one error. When I ignore that and try to build with the following modifications, I get some different errors.

In builtin.py, I uncomment the special builtin field

#"times": builtin_e.TIMES,  # no implemented
becomes
"times": builtin_e.TIMES,  # no implemented

and register the new command with a simple class

TIMES_SPEC = _Register('times')
class Times(object):
  def __init__(self, mem):
    self.mem = mem

  def __call__(self, arg_vec):
    print("Hello, world!")

Compiling and executing with the modifications gives me this:

$> build/dev.sh all; bin/osh
removed '_devbuild/gen/find_asdl.py'
removed '_devbuild/gen/find.marshal'
removed '_devbuild/gen/find_nt.py'
removed '_devbuild/gen/grammar.marshal'
removed '_devbuild/gen/grammar_nt.py'
removed '_devbuild/gen/grammar_nt.pyc'
removed '_devbuild/gen/id.h'
removed '_devbuild/gen/id_kind_asdl.py'
removed '_devbuild/gen/id_kind_asdl.pyc'
removed '_devbuild/gen/__init__.py'
removed '_devbuild/gen/__init__.pyc'
removed '_devbuild/gen/osh_help.py'
removed '_devbuild/gen/osh_help.pyc'
removed '_devbuild/gen/osh-lex.re2c.h'
removed '_devbuild/gen/osh-types.h'
removed '_devbuild/gen/runtime_asdl.py'
removed '_devbuild/gen/runtime_asdl.pyc'
removed '_devbuild/gen/syntax_asdl.py'
removed '_devbuild/gen/syntax_asdl.pyc'
removed '_devbuild/gen/typed_arith_asdl.py'
removed '_devbuild/gen/typed_demo_asdl.py'
removed '_devbuild/gen/types_asdl.py'
removed '_devbuild/gen/types_asdl.pyc'
Wrote _devbuild/osh-quick-ref/ and _devbuild/gen/osh_help.py
Wrote _release/VERSION/doc/osh-quick-ref.html
renamed '_tmp/types_asdl.py' -> '_devbuild/gen/types_asdl.py'
Wrote _devbuild/gen/types_asdl.py
renamed '_tmp/syntax_asdl.py' -> '_devbuild/gen/syntax_asdl.py'
Wrote _devbuild/gen/syntax_asdl.py
renamed '_tmp/runtime_asdl.py' -> '_devbuild/gen/runtime_asdl.py'
Wrote _devbuild/gen/runtime_asdl.py
renamed '_tmp/find_asdl.py' -> '_devbuild/gen/find_asdl.py'
Wrote _devbuild/gen/find_asdl.py
480 _devbuild/gen/typed_demo_asdl.py
554 _devbuild/gen/typed_arith_asdl.py
Compiled oil_lang/grammar.pgen2 -> _devbuild/gen/grammar.marshal and _devbuild/gen/grammar_nt.py
Compiled tools/find/find.pgen2 -> _devbuild/gen/find.marshal and _devbuild/gen/find_nt.py
running build
running build_ext
'libc.so' -> '_devbuild/py-ext/x86_64/libc.so'
libc.so: symbolic link to _devbuild/py-ext/x86_64/libc.so
.....real	0.100
user	0.200
sys	0.300
.......
----------------------------------------------------------------------
Ran 12 tests in 0.002s

OK
running build
running build_ext
building 'line_input' extension
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-Ra3Yar/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -UNDEBUG -I/usr/include/python2.7 -c native/line_input.c -o build/temp.linux-x86_64-2.7/native/line_input.o
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-Ra3Yar/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-Ra3Yar/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC build/temp.linux-x86_64-2.7/native/line_input.o -lreadline -o _devbuild/py-ext/x86_64/line_input.so
'line_input.so' -> '_devbuild/py-ext/x86_64/line_input.so'
line_input.so: symbolic link to _devbuild/py-ext/x86_64/line_input.so
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK
running build
running build_ext
building 'posix_' extension
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-Ra3Yar/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Ibuild/oil-defs -I/usr/include/python2.7 -c native/posixmodule.c -o build/temp.linux-x86_64-2.7/native/posixmodule.o
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-Ra3Yar/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-Ra3Yar/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC build/temp.linux-x86_64-2.7/native/posixmodule.o -o _devbuild/py-ext/x86_64/posix_.so
'posix_.so' -> '_devbuild/py-ext/x86_64/posix_.so'
posix_.so: symbolic link to _devbuild/py-ext/x86_64/posix_.so
........
----------------------------------------------------------------------
Ran 8 tests in 0.001s

OK
-- Generating AST, IDs, and lexer in _devbuild/gen
build/codegen.sh: line 54: _deps/re2c-1.0.3/re2c: No such file or directory
Traceback (most recent call last):
  File "/home/timetoplatypus/Documents/other/oil/bin/oil.py", line 75, in <module>
    from osh import builtin
  File "/home/timetoplatypus/Documents/other/oil/osh/builtin.py", line 68, in <module>
    "times": builtin_e.TIMES,  # no implemented
AttributeError: type object 'builtin_e' has no attribute 'TIMES'

timetoplatypus · 2019-07-20T04:48:52Z

I'm sure there are plenty of things I'm missing, I just don't know what all to change. (i.e. where to also make sure that mem gets passed to the method when invoked)

andychu · 2019-07-20T05:44:41Z

Try build/dev.sh minimal. That skips generating the lexer, which doesn't require re2c. https://github.com/oilshell/oil/wiki/Contributing (or you can build and install re2c as described there)
TIMES probably needs to be defined in runtime.asdl. https://github.com/oilshell/oil/wiki/Using-Zephyr-ASDL
The builtins are instantiated in bin/oil.py

timetoplatypus · 2019-07-20T12:29:31Z

All of that worked, so I'm able to execute the command and see the "Hello, world!". I'll submit a pull request when it's ready for review. Thanks.

andychu · 2019-07-27T06:29:10Z

@timetoplatypus While running the smoosh test suite on #331 , I noticed that several cases rely on times. So it would be great to get this in to enable more testing -- let me know if you're blocked on anything!

I will accept basically anything that passes some spec tests...

And I have tried to keep this page up to date, but let me know if you have problems:

https://github.com/oilshell/oil/wiki/Spec-Tests

andychu · 2020-01-15T07:07:11Z

@timetoplatypus I'm still interested in this if you have time for it :) If not let me know and I'll label it as up for grabs. Thanks for taking a look.

timetoplatypus · 2020-01-15T13:55:07Z

Sure, I'll take another look at this today and let you know

timetoplatypus · 2020-01-15T13:55:26Z

(accidentally closed the issue, ignore that)

andychu · 2020-01-22T08:13:19Z

OK this is merged except as mentioned the floating point issue will make it fail in the release build. Not sure if we should use translation or port it ahead of time. I guess it depends on when we get to that file

andychu · 2020-03-07T17:52:39Z

Fixed: https://www.oilshell.org/release/0.8.pre2/

andychu added the osh-language label Jan 15, 2020

timetoplatypus closed this as completed Jan 15, 2020

timetoplatypus reopened this Jan 15, 2020

timetoplatypus mentioned this issue Jan 15, 2020

Implements times builtin #580

Merged

andychu added the pending-release label Jan 22, 2020

andychu closed this as completed Mar 7, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement `times` builtin #441

Implement `times` builtin #441

timetoplatypus commented Jul 20, 2019

andychu commented Jul 20, 2019

timetoplatypus commented Jul 20, 2019

timetoplatypus commented Jul 20, 2019 •

edited

Loading

andychu commented Jul 20, 2019 •

edited

Loading

timetoplatypus commented Jul 20, 2019

andychu commented Jul 27, 2019 •

edited

Loading

andychu commented Jan 15, 2020

timetoplatypus commented Jan 15, 2020

timetoplatypus commented Jan 15, 2020

andychu commented Jan 22, 2020

andychu commented Mar 7, 2020

Implement times builtin #441

Implement times builtin #441

Comments

timetoplatypus commented Jul 20, 2019

andychu commented Jul 20, 2019

timetoplatypus commented Jul 20, 2019

timetoplatypus commented Jul 20, 2019 • edited Loading

andychu commented Jul 20, 2019 • edited Loading

timetoplatypus commented Jul 20, 2019

andychu commented Jul 27, 2019 • edited Loading

andychu commented Jan 15, 2020

timetoplatypus commented Jan 15, 2020

timetoplatypus commented Jan 15, 2020

andychu commented Jan 22, 2020

andychu commented Mar 7, 2020

Implement `times` builtin #441

Implement `times` builtin #441

timetoplatypus commented Jul 20, 2019 •

edited

Loading

andychu commented Jul 20, 2019 •

edited

Loading

andychu commented Jul 27, 2019 •

edited

Loading