Merge pull request erlang#10230 from bjorng/bjorn/compiler/documentation/OTP-19784

Update compiler and erlc documentation

Author: bjorng

erts/doc/references/erlc_cmd.md

@@ -166,7 +166,12 @@ The following compilers are supported:
 
   Supported options: same as for `.erl`.
 
-- **`.core`** - Erlang core source code. It generates a `.beam` file.
+- **`.core`** - Core Erlang source code. It generates a `.beam` file.
+
+  Supported options: same as for `.erl`.
+
+- **`.abstr`** - Erlang abstract format as produced by `to_abstr`.
+  It generates a `.beam` file.
 
   Supported options: same as for `.erl`.
 

lib/compiler/src/compile.erl

@@ -154,6 +154,49 @@ other Erlang code.
 See `m:erl_id_trans` for an example and an explanation of the function
 `parse_transform_info/0`.
 
+## Recommendations for Language Implementors
+
+Except for Erlang, there are quite a few other languges that can run
+on the Erlang runtime system. Generally, such compilers produce a file
+or binary that is fed into the Erlang compiler. There are four ways
+to do that using documented functionality:
+
+* **Erlang source code (`.erl` file)**: The file is compiled
+  using `compile:file/2` or `erlc <FILE>.erl`. This is the most
+  straightforward and portable way. The main disadvantage is that
+  it is hard or impossible to map each line of Erlang code back to the
+  corresponding line in the original source file.
+
+* **[The abstract format](`e:erts:absform.html`)**: This format
+  supports mapping every Erlang source line back to its corresponding
+  line in the original source file. To compile from the abstract
+  format, do one following:
+
+    * Call `compile:forms/2` to directly pass in the term
+      representation of the [abstract format](`e:erts:absform.html`).
+
+    * Write the term representation to a file with the extension
+      `.abstr` and compile that file using `compile:file(<File>,
+      [from_abstr])` or `erlc <File>.abstr`.
+
+* **Core Erlang**: While there is a specification for Core Erlang,
+  certain details are left to the implementation via the `primop`
+  expression. Primops can be added, deleted, or changed in any major
+  release without notice. Note that by generating Core Erlang
+  directly, it is possible to construct code that the Core-to-BEAM
+  backend has never encountered before, and there are no guarantees
+  that the final BEAM code will be safe.
+
+* **BEAM assembly code**: Strongly discouraged, as it is very hard to
+  get right and requires continuous maintenance. New instructions are
+  typically introduced in each major release (and sometimes old ones
+  are removed). In particular, note that BEAM code that does not
+  follow the correct conventions can cause the runtime system to crash
+  (segfault).
+
+Our recommendation is to use either the abstract format or Erlang
+source code.
+
 ## See Also
 
 `m:epp`, `m:erl_expand_records`, `m:erl_id_trans`, `m:erl_lint`, `m:beam_lib`
@@ -476,19 +519,31 @@ Available options:
   `Module:parse_transform/2` to be applied to the parsed code before the code is
   checked for errors.
 
-- **`from_abstr`** - The input file is expected to contain Erlang terms
-  representing forms in abstract format (default file suffix ".abstr"). Note
-  that the format of such terms can change between releases.
+- **`to_abstr`** - Dumps the terms representing the [abstract
+  format](`e:erts:absform.html`), after preprocessing and parse
+  transforms, in the file `<File>.abstr`. No object file is produced.
 
-  See also the `no_lint` option.
+- **`to_exp`** - Dumps the terms representing the [abstract
+  format](`e:erts:absform.html`), after all source code
+  transformations have been performed, to the file `<File>.exp`. No
+  object file is produced.
 
-- **`from_asm`** - The input file is expected to be assembler code (default file
-  suffix ".S"). Notice that the format of assembler files is not documented, and
+- **`from_abstr`** - The input file is expected to contain Erlang
+  terms representing forms in the [abstract
+  format](`e:erts:absform.html`) as generated by the `to_abstr` option
+  (default file suffix ".abstr"). Note that the format of such terms
   can change between releases.
 
-- **`from_core`** - The input file is expected to be core code (default file
-  suffix ".core"). Notice that the format of core files is not documented, and
-  can change between releases.
+  See also the `no_lint` option.
+
+- **`from_asm`** - The input file is expected to be BEAM assembly code
+  as generated by the `'S'` option (default file suffix ".S"). Note
+  that the format of assembler files is not documented, and can change
+  between releases.
+
+- **`from_core`** - The input file is expected to be Core Erlang code
+  (default file suffix ".core"). Note that the exact format of Core Erlang
+  files is not documented, and can change between releases.
 
 - **`no_spawn_compiler_process`** - By default, all code is compiled in a
   separate process which is terminated at the end of compilation. However, some