Support for input in freestyle format

[j4james]

I'm a big fan of the freestyle format, but unfortunately it doesn't support input yet. I don't know if you already had something in mind for this, but I was thinking one possible syntax could be <=, <==, and <=== which nicely mirrors the output prefixes. So an example test case (in Befunge) could look something like this:

~~~...@

<= ABC
=> 67 66 65 

It turns out this wasn't as difficult to add as I had imagined it would be, but here's my patch FWIW.
freestyle-input-patch.txt

I made a brief attempt to add some test cases as well, but I got half way through adding them to the test-freestyle-format file before I realised it would need a different functionality, and I figured I should probably just leave that to you to sort out. :)

[cpressey]

There are things I've been meaning to say regarding this, which I might also need to mention in one of the issues on the Befunge-93 repo, for context.

When I added the "freestyle" format to Falderal, one of my imagined use cases was, indeed, writing tests for Befunge. I figured the | at the beginning of each line would be more confusing than helpful in that context (or, really, any context where the sources look like line noise).

Since having that thought, I've been really slow to act on it, due to various other considerations. Most of them are minor, but they add up. I'll just list them.

One is that Markdown seems to have evolved since the early time to become less WYSIWYG rather than more. (Just view the source of a random README on GitHub and chances are you'll see what I mean.) In particular, it seems people like the three-backticks syntax for including preformatted blocks, probably (I mean almost certainly) because it lets them specify the syntax highlighting. But for reading a Markdown file as a text file, it definitely feels like a step backwards to me. (I guess reading them as text files is a minor use case for many users - something you have to do out of necessity when editing it only, like wiki markup.)

Another is that, instead of explaining that every line starts with | because of the test suite and that's not actually part of the program don't type it in, you have to explain that every example ends with ===> because of the test suite and that's not actually part of the program don't type it in. That's probably a better situation to be in, but it only reduces the problem, it doesn't actually solve it (especially for a language where the sources look like line noise).

Another is that (and this is the part that should really be noted in the Befunge-93 repo) Befunge-93 hails from an era before automated testing was a standard thing. Honestly I'm not sure how we survived back then, but apparently we did, and it feels somewhat odd to modernize a piece of software from out of that era by giving it automated tests. Or, at least it does for Befunge. It's very similar to the situation described in #4 - appending a file extension automatically feels like a very crufty thing, but fixing it would feel like "poor curation" somehow, like unfairly dragging it out of its bygone idiom.

Yet another, which interacts strongly with the previous point, is that I've found that if you put Falderal tests in a specification, they can be taken too seriously. Often, a spec doesn't completely specify behaviour, and often, that's intentional. An implementation, otoh, necessarily completes that behaviour somehow, and Falderal necessarily expects that concrete behaviour. So, often, failing some tests doesn't necessarily mean an implementation is non-compliant, and vice-versa, passing all the tests isn't a guarantee that the implementation is totally conformant. But people can fail to see that, and they can fail to see that the behaviour and the implementation are intended to be independent from each other in that sense. Befunge, in particular, has never been a strongly specified language, and I fear that adding a document that declares "this behaviour is the expected behaviour" too loudly will not help the language retain its charm.

All that said, I'm still not against the idea of writing a Falderal document that demonstrates how to program in Befunge. It just has to be presented in a good way.

[j4james]

Just want to chip in my 2c on the subject of Befunge tests. I can perhaps understand you not wanting the nail down many of the edge cases as expected behaviour for a compliant implementation, but that's exactly what I'm looking to do with my test suite. I write a lot of Befunge programs, and in my experience, knowledge of implementation bugs and edge cases is absolutely essential if I want to have any hope of producing portable code.

It's not necessarily a matter of what is right or wrong. It's about knowing which instructions I need to be wary of using because they aren't implemented consistently. What ranges I should limit myself to in order to get the widest support. What are the common bugs that interpreters are likely to have and how can I work around them. Being able to document these sorts of details along side the test cases is what make Falderal so appealing to me.

And approaching things from another angle, as a interpreter author I definitely want to know if my implementation is deviating from the reference implementation. The number one priority for me is writing an interpreter that is actually capable of executing the Befunge programs that others have written. And I believe the best was to achieve that is by following the reference implementation as closely as possible.

Bottom line: I don't find the poor compatibility of Befunge interpreters particularly charming - it's actually what I dislike most about the language. But having a comprehensive test suite can at least help deal with the problem.

[cpressey]

Fair enough. I'd be surprised if you haven't already heard of it, but your 2nd paragraph in particular reminded me of Mycology, which is a test program for Befunge-98, and whose topmost-leftmost 80x25 sub-block can be extracted and used to test Befunge-93, and which takes the approach of "what does this interpreter do/not do."

(The existence of Mycology is probably one other reason I haven't been in a rush to write a Falderal document for Befunge. Besides being an already-existing test suite, it is also an incredibly impressive piece of Befunge programming -- a thing which is, in my opinion, more in the spirit of Befunge than a readable, sensible test document would be.)

At any rate. Thank you for the contributions to Falderal 0.12 -- they've made it a more useful format.

If you have a Falderal document (or anything else) that you'd like to contribute to the Befunge-93 distribution, feel free to open a PR for it, but since tomorrow it will be December and since I'm going to try to get 2.25 released before 2019, anything you'd like to see in 2.25 would have to be sooner rather than later. And along those lines, it's unlikely that I'll be writing any significant amount of new documentation myself, for 2.25.

[j4james]

Yeah I'm aware of Mycology, and it's an impressive bit of code, but it just doesn't cover enough of the edge cases to be of much use to me. And when you cram all the tests into one bit of code like that, an early failure can prevent you seeing the results of later tests (e.g. many interpreters won't even load the code, or will fail after printing just two digits, because of the oversized source). With Falderal I can write a whole load of little tests, and each will be run separately without having any effect on those that follow.

Unfortunately I'm a long way from having anything ready for public consumption yet. I had grand plans for all the Befunge-related projects I was going to release for the Jubilee, but it's beginning to look like none of them will be finished this year. Although now that I think about it, the one goal I did achieve was getting Befunge back into the top 100 languages on Rosetta Code. Not sure that means much, but that was my little contribution to the Jubilee. :)

[cpressey]

one goal I did achieve was getting Befunge back into the top 100 languages on Rosetta Code

That's impressive, considering Befunge would have a rather high handicap, if Rosetta Code was golf, if you see what I mean.

Your research and constructive criticism was also a great help in "gardening" the examples for the 2.25 release of Befunge-93, so don't discount that either.

I too would've liked to do more for the anniversary year, but esolangs and deadlines have never mixed well...