Rewrite to include Decl in the standard AST

[elegios]

This PR implements #826 by adding a TmDecl term to Expr. Most of the changes are fairly direct translations to the new style (e.g., TmLet {...} to TmDecl {decl = DeclLet {...}, ...}), but I have also made some refactorings making use of the new structure to remove code and make things simpler.

As mentioned in the issue, the idea is to use TmDecl for everything where we have a "side-effect" followed by a single expression. For example, TmMatch does not fit this criteria (two options on how to continue), but TmLet (name-binding as a side-effect) and TmUtest (running a test) do.

• ast-builder.mc now makes Decls when using, e.g., let_. The separate mlang/ast-builder.mc no longer has, e.g., decl_let_. Changed types:
  • bind_ : Expr -> Expr -> Expr becomes bind_ : Decl -> Expr -> Expr and can no longer implicitly lose AST nodes
  • bindall_ : [Expr] -> Expr becomes bindall_ : [Decl] -> Expr -> Expr.
• I chose to have smap_Expr_Expr for TmDecl use smap_Decl_Expr to make it behave as it used to, to minimize the number of hard to find semantic mismatches. We might want to change this later, or we might not, I'm not sure at this point.

I also accidentally snuck in an additional feature for generate-pprint.mc, the ability to register custom pprint functions for named (non-alias) types, similar to what's already available for generate-json-serializers.mc.

Performance

This change increases the number of nodes in most ASTs, thus we expect there to be some impact on performance, both for runtime and memory. The runtime can perhaps be somewhat mitigated by simpler code that needs fewer checks, but the added pointer chasing and potential for cache misses and garbage collection seem likely to be greater.

Here are summary statistics over 10 runs for each step in the bootstrapping process, first before this PR, then after.

Step 1:

N	min	max	sum	mean	sd	sderr	 (before)
10	142.181	153.928	1497.55	149.755	3.18581	1.00744	
N	min	max	sum	mean	sd	sderr	 (after)
10	148.469	159.917	1539.17	153.917	3.87211	1.22447	

Step 2:

N	min	max	sum	mean	sd	sderr	 (before)
10	39.003	40.861	399.094	39.9094	0.645259	0.204049	
N	min	max	sum	mean	sd	sderr	 (after)
10	39.139	41.715	404.034	40.4034	0.826885	0.261484	

Step 3:

N	min	max	sum	mean	sd	sderr	 (before)
10	40.684	43.378	421.615	42.1615	0.841775	0.266193	
N	min	max	sum	mean	sd	sderr	 (after)
10	40.524	43.56	422.524	42.2524	0.854545	0.270231	

Data underlying the above

 2) [153.928s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [153.573s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [151.896s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [150.879s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [150.405s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [149.167s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [149.151s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [148.525s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [147.846s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [142.181s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
N	min	max	sum	mean	sd	sderr	
10	142.181	153.928	1497.55	149.755	3.18581	1.00744	

 1) [40.861s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [40.842s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [40.592s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [40.038s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [39.892s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [39.793s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [39.524s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [39.515s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [39.034s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [39.003s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
N	min	max	sum	mean	sd	sderr	
10	39.003	40.861	399.094	39.9094	0.645259	0.204049	

 0) [43.378s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [43.108s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [42.960s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [42.432s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [42.214s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [42.208s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [41.975s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [41.768s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [40.888s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [40.684s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
N	min	max	sum	mean	sd	sderr	
10	40.684	43.378	421.615	42.1615	0.841775	0.266193	

...and here's after:

 2) [159.917s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [159.486s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [156.838s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [156.257s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [153.694s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [152.899s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [151.850s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [150.682s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [149.079s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
 2) [148.469s] [build] src: ./boot eval main/mi-lite.mc -- 0 main/mi-lite.mc ../build/src/mi-lite        
N	min	max	sum	mean	sd	sderr	
10	148.469	159.917	1539.17	153.917	3.87211	1.22447	

 1) [41.715s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [41.528s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [41.132s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [40.945s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [40.300s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [39.975s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [39.843s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [39.800s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [39.657s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
 1) [39.139s] [build] src: ./mi-lite 1 main/mi.mc ../build/src/mi1                                       
N	min	max	sum	mean	sd	sderr	
10	39.139	41.715	404.034	40.4034	0.826885	0.261484	

 0) [43.560s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [43.280s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [42.844s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [42.566s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [42.174s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [42.138s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [42.125s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [42.078s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [41.235s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
 0) [40.524s] [build] src: ./mi1 compile main/mi.mc --output ../build/src/mi                             
N	min	max	sum	mean	sd	sderr	
10	40.524	43.56	422.524	42.2524	0.854545	0.270231	

In summary, we do indeed see a slowdown, most notably in the first step and then decreasing for later steps. The last step in particular is a slowdown of 0.09s on average, out of ~40s total, which seems quite acceptable.