Iterative Statement Extension - Question 1 and 2 Electric Boogaloo

[RyanBurnside]

I've been tinkering a bit with the iterative statement. I think I've got the hang of generators they're a very cool way to abstract iterating over a process or container of unknown origin.

The IRM gives a recursive example - which is fine. I was able to bend it to make a generator for Fibonacci sequences and also a neat iterator over a Bezier curve.

I was able to do an array stepper this way just fine, though I fear it conses more than necessary.
Usually I'd do a LET over LAMBDA and use a lexical closure with mutation but I'm not sure if that'd be dangerous. As it stands I believe I either need to push the bindings into optional parameters or keep recalculating them from scratch each recursion.

Maybe I can (DECLARE (LOCALVARS ORIGIN START LASTINDEX)) ?

(LAMBDA (ARRAY START)
       (* Create an array traversal generator. START may be left out or may specify the starting 
          element.)
       (LET* ((ORIGIN (ARRAYORIG ARRAY))
              (START (OR START ORIGIN))
              (LASTINDEX (IDIFFERENCE (ARRAYSIZE ARRAY)
                                (if (ZEROP ORIGIN)
                                    then 1 else 0))))
             (if (ILEQ START LASTINDEX)
                 then
                 (PRODUCE (ELT ARRAY START))
                 (ARRAYGEN ARRAY (ADD1 START)))))

I called mine ARRAYGEN after the pattern given in the IRM.

I can then do:
(FOR N OUTOF (ARRAYGEN FOO) <do whatever with N> ) ETC.

I currently have been poking at the IS extension section of the IRM and feel there isn't quite enough in that section for me to grok how I might do an "ACROSS" like functionality. Perhaps it's just beyond the intent, and if so that's OK.

So step II: New Clause Electric Boogaloo...

[krotname]

Looking at the IRM and the current source, I would not use (DECLARE (LOCALVARS ORIGIN START LASTINDEX)) as the main fix here.

DECLARE/declare: inside an iterative statement is inserted into the translated loop, and the manual describes it as a way to declare variables bound in that iterative statement, e.g. for X ... declare: (LOCALVARS X) .... So it is useful for temps introduced by an i.s. expansion, but it does not by itself make ORIGIN, START, or LASTINDEX into private persistent generator state across OUTOF calls.

For this array case, the simpler non-generator form is probably the best baseline:

(for I from (ARRAYORIG A)
       to (IPLUS (ARRAYSIZE A) (ARRAYORIG A) -1)
   collect (ELT A I))

That pattern is also used in the tree today, for example in lispusers/SOLITAIRE, where array indexes run from (ARRAYORIG A) through (IPLUS (ARRAYSIZE A) (ARRAYORIG A) -1) and the element is read with ELT.

If you want an ACROSS-style clause, I would model it after the existing user-defined i.s. operators in sources/LLCHAR: expand with I.S.OPR into binded temporaries, add declare (LOCALVARS ...) for those temps, compute start/end in first, assign I.V. in eachtime, and increment in repeatwhile. That avoids the recursive generator/coroutine path entirely.

OUTOF itself is defined in sources/COROUTINE: it binds a generated handle to (GENERATOR BODY), calls (GENERATE GENVAR) each iteration, and exits when the result is the generator handle. So the recursive PRODUCE style is expected for real generators, but for array traversal I would use either the plain index loop above or a small I.S.OPR expansion before reaching for a generator.

[RyanBurnside]

Thanks for the response @krotname I've been meaning to get back to this thread.
I think for now the more verbose forms are OK.

I did look at the source and it became a struggle programming session to try and divine if it was possible to do the ACROSS extension. Maybe you can demo what you were thinking?

At any rate, I think It's OK for the most part writing out that pattern.

• Ryan

[krotname]

Sure. The shape I had in mind was not a generator, but a small I.S.OPR that expands ACROSS into the same indexed array loop you are already writing by hand.

Source-level sketch, following the pattern in sources/LLCHAR (instring, inatom, etc.):

(I.S.OPR 'across NIL
  '[SUBST (GETDUMMYVAR)
          '$$ARRAY
          '(bind $$ARRAY _ BODY $$INDEX $$LAST
             declare (LOCALVARS $$ARRAY $$INDEX $$LAST)
             first (SETQ $$INDEX (ARRAYORIG $$ARRAY))
                   (SETQ $$LAST (IPLUS (ARRAYSIZE $$ARRAY)
                                        (ARRAYORIG $$ARRAY)
                                        -1))
             eachtime (AND (IGREATERP $$INDEX $$LAST)
                           (GO $$OUT))
                      (SETQ I.V. (ELT $$ARRAY $$INDEX))
             repeatwhile (PROGN (SETQ $$INDEX (ADD1 $$INDEX))
                                T))]
  T)

Conceptually this would let:

(for X across A collect X)

mean the same thing as:

(for I from (ARRAYORIG A)
       to (IPLUS (ARRAYSIZE A) (ARRAYORIG A) -1)
   collect (ELT A I))

The important part is where the state lives. $$ARRAY, $$INDEX, and $$LAST are loop temporaries introduced by the i.s. expansion, so declare (LOCALVARS ...) belongs there. first initializes the traversal once, eachtime produces the current element into I.V., and repeatwhile advances the index.

I have not run this in a Medley image, so treat it as a sketch rather than a drop-in patch. If I were turning it into a real extension, I would start with only 1-D arrays, add it to the file package's I.S.OPRS list, and test the edge cases: non-zero ARRAYORIG, empty arrays, and mutation of the array while the loop is running.