-
Notifications
You must be signed in to change notification settings - Fork 196
585 lines (552 loc) · 18.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
# typed: true
# DO NOT EDIT MANUALLY
# This is an autogenerated file for types exported from the `ast` gem.
# Please instead update this file by running `bin/tapioca gem ast`.
# {AST} is a library for manipulating abstract syntax trees.
#
# It embraces immutability; each AST node is inherently frozen at
# creation, and updating a child node requires recreating that node
# and its every parent, recursively.
# This is a design choice. It does create some pressure on
# garbage collector, but completely eliminates all concurrency
# and aliasing problems.
#
# See also {AST::Node}, {AST::Processor::Mixin} and {AST::Sexp} for
# additional recommendations and design patterns.
#
# source://ast//lib/ast.rb#13
module AST; end
# Node is an immutable class, instances of which represent abstract
# syntax tree nodes. It combines semantic information (i.e. anything
# that affects the algorithmic properties of a program) with
# meta-information (line numbers or compiler intermediates).
#
# Notes on inheritance
# ====================
#
# The distinction between semantics and metadata is important. Complete
# semantic information should be contained within just the {#type} and
# {#children} of a Node instance; in other words, if an AST was to be
# stripped of all meta-information, it should remain a valid AST which
# could be successfully processed to yield a result with the same
# algorithmic properties.
#
# Thus, Node should never be inherited in order to define methods which
# affect or return semantic information, such as getters for `class_name`,
# `superclass` and `body` in the case of a hypothetical `ClassNode`. The
# correct solution is to use a generic Node with a {#type} of `:class`
# and three children. See also {Processor} for tips on working with such
# ASTs.
#
# On the other hand, Node can and should be inherited to define
# application-specific metadata (see also {#initialize}) or customize the
# printing format. It is expected that an application would have one or two
# such classes and use them across the entire codebase.
#
# The rationale for this pattern is extensibility and maintainability.
# Unlike static ones, dynamic languages do not require the presence of a
# predefined, rigid structure, nor does it improve dispatch efficiency,
# and while such a structure can certainly be defined, it does not add
# any value but incurs a maintaining cost.
# For example, extending the AST even with a transformation-local
# temporary node type requires making globally visible changes to
# the codebase.
#
# source://ast//lib/ast/node.rb#40
class AST::Node
# Constructs a new instance of Node.
#
# The arguments `type` and `children` are converted with `to_sym` and
# `to_a` respectively. Additionally, the result of converting `children`
# is frozen. While mutating the arguments is generally considered harmful,
# the most common case is to pass an array literal to the constructor. If
# your code does not expect the argument to be frozen, use `#dup`.
#
# The `properties` hash is passed to {#assign_properties}.
#
# @return [Node] a new instance of Node
#
# source://ast//lib/ast/node.rb#72
def initialize(type, children = T.unsafe(nil), properties = T.unsafe(nil)); end
# Concatenates `array` with `children` and returns the resulting node.
#
# @return [AST::Node]
#
# source://ast//lib/ast/node.rb#168
def +(array); end
# Appends `element` to `children` and returns the resulting node.
#
# @return [AST::Node]
#
# source://ast//lib/ast/node.rb#177
def <<(element); end
# Compares `self` to `other`, possibly converting with `to_ast`. Only
# `type` and `children` are compared; metadata is deliberately ignored.
#
# @return [Boolean]
#
# source://ast//lib/ast/node.rb#153
def ==(other); end
# Appends `element` to `children` and returns the resulting node.
#
# @return [AST::Node]
#
# source://ast//lib/ast/node.rb#177
def append(element); end
# Returns the children of this node.
# The returned value is frozen.
# The to_a alias is useful for decomposing nodes concisely.
# For example:
#
# node = s(:gasgn, :$foo, s(:integer, 1))
# var_name, value = *node
# p var_name # => :$foo
# p value # => (integer 1)
#
# @return [Array]
#
# source://ast//lib/ast/node.rb#56
def children; end
# Nodes are already frozen, so there is no harm in returning the
# current node as opposed to initializing from scratch and freezing
# another one.
#
# @return self
#
# source://ast//lib/ast/node.rb#115
def clone; end
# Concatenates `array` with `children` and returns the resulting node.
#
# @return [AST::Node]
#
# source://ast//lib/ast/node.rb#168
def concat(array); end
# Enables matching for Node, where type is the first element
# and the children are remaining items.
#
# @return [Array]
#
# source://ast//lib/ast/node.rb#253
def deconstruct; end
# Nodes are already frozen, so there is no harm in returning the
# current node as opposed to initializing from scratch and freezing
# another one.
#
# @return self
#
# source://ast//lib/ast/node.rb#115
def dup; end
# Test if other object is equal to
#
# @param other [Object]
# @return [Boolean]
#
# source://ast//lib/ast/node.rb#85
def eql?(other); end
# Returns the precomputed hash value for this node
#
# @return [Integer]
#
# source://ast//lib/ast/node.rb#61
def hash; end
# Converts `self` to a s-expression ruby string.
# The code return will recreate the node, using the sexp module s()
#
# @param indent [Integer] Base indentation level.
# @return [String]
#
# source://ast//lib/ast/node.rb#211
def inspect(indent = T.unsafe(nil)); end
# Returns the children of this node.
# The returned value is frozen.
# The to_a alias is useful for decomposing nodes concisely.
# For example:
#
# node = s(:gasgn, :$foo, s(:integer, 1))
# var_name, value = *node
# p var_name # => :$foo
# p value # => (integer 1)
#
# @return [Array]
#
# source://ast//lib/ast/node.rb#56
def to_a; end
# @return [AST::Node] self
#
# source://ast//lib/ast/node.rb#229
def to_ast; end
# Converts `self` to a pretty-printed s-expression.
#
# @param indent [Integer] Base indentation level.
# @return [String]
#
# source://ast//lib/ast/node.rb#187
def to_s(indent = T.unsafe(nil)); end
# Converts `self` to a pretty-printed s-expression.
#
# @param indent [Integer] Base indentation level.
# @return [String]
#
# source://ast//lib/ast/node.rb#187
def to_sexp(indent = T.unsafe(nil)); end
# Converts `self` to an Array where the first element is the type as a Symbol,
# and subsequent elements are the same representation of its children.
#
# @return [Array<Symbol, [...Array]>]
#
# source://ast//lib/ast/node.rb#237
def to_sexp_array; end
# Returns the type of this node.
#
# @return [Symbol]
#
# source://ast//lib/ast/node.rb#43
def type; end
# Returns a new instance of Node where non-nil arguments replace the
# corresponding fields of `self`.
#
# For example, `Node.new(:foo, [ 1, 2 ]).updated(:bar)` would yield
# `(bar 1 2)`, and `Node.new(:foo, [ 1, 2 ]).updated(nil, [])` would
# yield `(foo)`.
#
# If the resulting node would be identical to `self`, does nothing.
#
# @param type [Symbol, nil]
# @param children [Array, nil]
# @param properties [Hash, nil]
# @return [AST::Node]
#
# source://ast//lib/ast/node.rb#133
def updated(type = T.unsafe(nil), children = T.unsafe(nil), properties = T.unsafe(nil)); end
protected
# By default, each entry in the `properties` hash is assigned to
# an instance variable in this instance of Node. A subclass should define
# attribute readers for such variables. The values passed in the hash
# are not frozen or whitelisted; such behavior can also be implemented
# by subclassing Node and overriding this method.
#
# @return [nil]
#
# source://ast//lib/ast/node.rb#98
def assign_properties(properties); end
# Returns `@type` with all underscores replaced by dashes. This allows
# to write symbol literals without quotes in Ruby sources and yet have
# nicely looking s-expressions.
#
# @return [String]
#
# source://ast//lib/ast/node.rb#264
def fancy_type; end
private
def original_dup; end
end
# This class includes {AST::Processor::Mixin}; however, it is
# deprecated, since the module defines all of the behaviors that
# the processor includes. Any new libraries should use
# {AST::Processor::Mixin} instead of subclassing this.
#
# @deprecated Use {AST::Processor::Mixin} instead.
#
# source://ast//lib/ast/processor.rb#8
class AST::Processor
include ::AST::Processor::Mixin
end
# The processor module is a module which helps transforming one
# AST into another. In a nutshell, the {#process} method accepts
# a {Node} and dispatches it to a handler corresponding to its
# type, and returns a (possibly) updated variant of the node.
#
# The processor module has a set of associated design patterns.
# They are best explained with a concrete example. Let's define a
# simple arithmetic language and an AST format for it:
#
# Terminals (AST nodes which do not have other AST nodes inside):
#
# * `(integer <int-literal>)`,
#
# Nonterminals (AST nodes with other nodes as children):
#
# * `(add <node> <node>)`,
# * `(multiply <node> <node>)`,
# * `(divide <node> <node>)`,
# * `(negate <node>)`,
# * `(store <node> <string-literal>)`: stores value of `<node>`
# into a variable named `<string-literal>`,
# * `(load <string-literal>)`: loads value of a variable named
# `<string-literal>`,
# * `(each <node> ...)`: computes each of the `<node>`s and
# prints the result.
#
# All AST nodes have the same Ruby class, and therefore they don't
# know how to traverse themselves. (A solution which dynamically
# checks the type of children is possible, but is slow and
# error-prone.) So, a class including the module which knows how
# to traverse the entire tree should be defined. Such classes
# have a handler for each nonterminal node which recursively
# processes children nodes:
#
# require 'ast'
#
# class ArithmeticsProcessor
# include AST::Processor::Mixin
# # This method traverses any binary operators such as (add)
# # or (multiply).
# def process_binary_op(node)
# # Children aren't decomposed automatically; it is
# # suggested to use Ruby multiple assignment expansion,
# # as it is very convenient here.
# left_expr, right_expr = *node
#
# # AST::Node#updated won't change node type if nil is
# # passed as a first argument, which allows to reuse the
# # same handler for multiple node types using `alias'
# # (below).
# node.updated(nil, [
# process(left_expr),
# process(right_expr)
# ])
# end
# alias_method :on_add, :process_binary_op
# alias_method :on_multiply, :process_binary_op
# alias_method :on_divide, :process_binary_op
#
# def on_negate(node)
# # It is also possible to use #process_all for more
# # compact code if every child is a Node.
# node.updated(nil, process_all(node))
# end
#
# def on_store(node)
# expr, variable_name = *node
#
# # Note that variable_name is not a Node and thus isn't
# # passed to #process.
# node.updated(nil, [
# process(expr),
# variable_name
# ])
# end
#
# # (load) is effectively a terminal node, and so it does
# # not need an explicit handler, as the following is the
# # default behavior. Essentially, for any nodes that don't
# # have a defined handler, the node remains unchanged.
# def on_load(node)
# nil
# end
#
# def on_each(node)
# node.updated(nil, process_all(node))
# end
# end
#
# Let's test our ArithmeticsProcessor:
#
# include AST::Sexp
# expr = s(:add, s(:integer, 2), s(:integer, 2))
#
# p ArithmeticsProcessor.new.process(expr) == expr # => true
#
# As expected, it does not change anything at all. This isn't
# actually very useful, so let's now define a Calculator, which
# will compute the expression values:
#
# # This Processor folds nonterminal nodes and returns an
# # (integer) terminal node.
# class ArithmeticsCalculator < ArithmeticsProcessor
# def compute_op(node)
# # First, node children are processed and then unpacked
# # to local variables.
# nodes = process_all(node)
#
# if nodes.all? { |node| node.type == :integer }
# # If each of those nodes represents a literal, we can
# # fold this node!
# values = nodes.map { |node| node.children.first }
# AST::Node.new(:integer, [
# yield(values)
# ])
# else
# # Otherwise, we can just leave the current node in the
# # tree and only update it with processed children
# # nodes, which can be partially folded.
# node.updated(nil, nodes)
# end
# end
#
# def on_add(node)
# compute_op(node) { |left, right| left + right }
# end
#
# def on_multiply(node)
# compute_op(node) { |left, right| left * right }
# end
# end
#
# Let's check:
#
# p ArithmeticsCalculator.new.process(expr) # => (integer 4)
#
# Excellent, the calculator works! Now, a careful reader could
# notice that the ArithmeticsCalculator does not know how to
# divide numbers. What if we pass an expression with division to
# it?
#
# expr_with_division = \
# s(:add,
# s(:integer, 1),
# s(:divide,
# s(:add, s(:integer, 8), s(:integer, 4)),
# s(:integer, 3))) # 1 + (8 + 4) / 3
#
# folded_expr_with_division = ArithmeticsCalculator.new.process(expr_with_division)
# p folded_expr_with_division
# # => (add
# # (integer 1)
# # (divide
# # (integer 12)
# # (integer 3)))
#
# As you can see, the expression was folded _partially_: the inner
# `(add)` node which could be computed was folded to
# `(integer 12)`, the `(divide)` node is left as-is because there
# is no computing handler for it, and the root `(add)` node was
# also left as it is because some of its children were not
# literals.
#
# Note that this partial folding is only possible because the
# _data_ format, i.e. the format in which the computed values of
# the nodes are represented, is the same as the AST itself.
#
# Let's extend our ArithmeticsCalculator class further.
#
# class ArithmeticsCalculator
# def on_divide(node)
# compute_op(node) { |left, right| left / right }
# end
#
# def on_negate(node)
# # Note how #compute_op works regardless of the operator
# # arity.
# compute_op(node) { |value| -value }
# end
# end
#
# Now, let's apply our renewed ArithmeticsCalculator to a partial
# result of previous evaluation:
#
# p ArithmeticsCalculator.new.process(expr_with_division) # => (integer 5)
#
# Five! Excellent. This is also pretty much how CRuby 1.8 executed
# its programs.
#
# Now, let's do some automated bug searching. Division by zero is
# an error, right? So if we could detect that someone has divided
# by zero before the program is even run, that could save some
# debugging time.
#
# class DivisionByZeroVerifier < ArithmeticsProcessor
# class VerificationFailure < Exception; end
#
# def on_divide(node)
# # You need to process the children to handle nested divisions
# # such as:
# # (divide
# # (integer 1)
# # (divide (integer 1) (integer 0))
# left, right = process_all(node)
#
# if right.type == :integer &&
# right.children.first == 0
# raise VerificationFailure, "Ouch! This code divides by zero."
# end
# end
#
# def divides_by_zero?(ast)
# process(ast)
# false
# rescue VerificationFailure
# true
# end
# end
#
# nice_expr = \
# s(:divide,
# s(:add, s(:integer, 10), s(:integer, 2)),
# s(:integer, 4))
#
# p DivisionByZeroVerifier.new.divides_by_zero?(nice_expr)
# # => false. Good.
#
# bad_expr = \
# s(:add, s(:integer, 10),
# s(:divide, s(:integer, 1), s(:integer, 0)))
#
# p DivisionByZeroVerifier.new.divides_by_zero?(bad_expr)
# # => true. WHOOPS. DO NOT RUN THIS.
#
# Of course, this won't detect more complex cases... unless you
# use some partial evaluation before! The possibilites are
# endless. Have fun.
#
# source://ast//lib/ast/processor/mixin.rb#240
module AST::Processor::Mixin
# Default handler. Does nothing.
#
# @param node [AST::Node]
# @return [AST::Node, nil]
#
# source://ast//lib/ast/processor/mixin.rb#284
def handler_missing(node); end
# Dispatches `node`. If a node has type `:foo`, then a handler
# named `on_foo` is invoked with one argument, the `node`; if
# there isn't such a handler, {#handler_missing} is invoked
# with the same argument.
#
# If the handler returns `nil`, `node` is returned; otherwise,
# the return value of the handler is passed along.
#
# @param node [AST::Node, nil]
# @return [AST::Node, nil]
#
# source://ast//lib/ast/processor/mixin.rb#251
def process(node); end
# {#process}es each node from `nodes` and returns an array of
# results.
#
# @param nodes [Array<AST::Node>]
# @return [Array<AST::Node>]
#
# source://ast//lib/ast/processor/mixin.rb#274
def process_all(nodes); end
end
# This simple module is very useful in the cases where one needs
# to define deeply nested ASTs from Ruby code, for example, in
# tests. It should be used like this:
#
# describe YourLanguage do
# include ::AST::Sexp
#
# it "should correctly parse expressions" do
# YourLanguage.parse("1 + 2 * 3").should ==
# s(:add,
# s(:integer, 1),
# s(:multiply,
# s(:integer, 2),
# s(:integer, 3)))
# end
# end
#
# This way the amount of boilerplate code is greatly reduced.
#
# source://ast//lib/ast/sexp.rb#20
module AST::Sexp
# Creates a {Node} with type `type` and children `children`.
# Note that the resulting node is of the type AST::Node and not a
# subclass.
# This would not pose a problem with comparisons, as {Node#==}
# ignores metadata.
#
# source://ast//lib/ast/sexp.rb#26
def s(type, *children); end
end