sync with the recently upstreamed parser tests

jbellerb · jbellerb · commit 7b107bacc71e · 2025-04-12T23:41:36.000-04:00
diff --git a/README.md b/README.md
@@ -4,12 +4,12 @@
 
 ## Scope
 
-This parser is designed for syntax highlighting and does not strictly follow the Dhall language semantics. My goal is for all valid Dhall expressions to parse to something sensible. An invalid expression parsing is fine, except when it is obviously wrong. To help with this, the output of the parser is checked with hand-written [syntax trees](test/corpus/) for every Dhall [successful parse test](https://github.com/dhall-lang/dhall-lang/tree/master/tests/parser/success) (current with dhall-lang commit [4fecb49](https://github.com/dhall-lang/dhall-lang/commit/4fecb49a83caac4477d1e413fa6e4ee722b43b62)) except:
+This parser is designed for syntax highlighting and does not strictly follow the Dhall language semantics. My goal is for all valid Dhall expressions to parse to something sensible. An invalid expression parsing is fine, except when it is obviously wrong. To help with this, the output of the parser is checked with hand-written [syntax trees](test/corpus/) for every Dhall [successful parse test](https://github.com/dhall-lang/dhall-lang/tree/master/tests/parser/success) (current with dhall-lang commit [6d4c9d6](https://github.com/dhall-lang/dhall-lang/commit/6d4c9d6b8812ed3df1f45aabeca3deaf0615ba7b)) except:
 
 - unit/ShebangA.dhall, unit/ShebangNixA.dhall: Handled by the text editor.
 - unit/TrailingLineCommentWithoutNewlineA.dhall: Unclear how to test with tree-sitter.
 
-These tests also serve as a reference for what syntax trees various Dhall expressions produce. A shell script ([tests.sh](tests.sh)) is used to check that all tests are included. There may be extra test cases besides the Dhall ones. Ideally, those would eventually get upstreamed.
+Additionally, these tests serve as a reference for what syntax trees various Dhall expressions produce. A shell script ([tests.sh](tests.sh)) is provided to check that all tests, excluding a few intentionally non-compliant ones, are included.
 
 ## References
 
diff --git a/flake.lock b/flake.lock
diff --git a/flake.nix b/flake.nix
@@ -105,15 +105,16 @@
             dhall = fetchFromGitHub {
               owner = "dhall-lang";
               repo = "dhall-lang";
-              rev = "4fecb49a83caac4477d1e413fa6e4ee722b43b62";
-              hash = "sha256-VjOBG5kYY89njGpOxIpLFRK2mRP1UWr6QDOHtb7VMzE=";
+              rev = "6d4c9d6b8812ed3df1f45aabeca3deaf0615ba7b";
+              hash = "sha256-5cWQPqnTb59Z8Mov/t/ycaQ94jE+w2mol+0pbdRVbEI=";
             };
 
             missing = [
-              "builtinsA.dhall" # modified
               "unit/ShebangA.dhall"
               "unit/ShebangNixA.dhall"
               "unit/TrailingLineCommentWithoutNewlineA.dhall"
+              "Unterminated block comment"
+              "de Bruijn indices don't apply to builtins"
             ];
 
           in stdenvNoCC.mkDerivation {
@@ -122,7 +123,7 @@
 
             buildPhase = ''
               ${./tests.sh} -c ${test/corpus}\
-                -i ${lib.strings.concatStringsSep "," missing}\
+                -i "${lib.strings.concatStringsSep "," missing}"\
                 ${dhall}/tests/parser/success/
             '';
             installPhase = "touch $out";
diff --git a/test/corpus/builtins.txt b/test/corpus/builtins.txt
@@ -2,48 +2,45 @@
 All builtins (builtinsA.dhall*)
 ==================
 
-[ Natural/fold
-, Natural/build
-, Natural/isZero
-, Natural/even
-, Natural/odd
-, Natural/toInteger
-, Natural/show
-, Integer/toDouble
-, Integer/show
-, Integer/negate
-, Integer/clamp
-, Natural/subtract
+[ Date/show
 , Double/show
+, Integer/clamp
+, Integer/negate
+, Integer/show
+, Integer/toDouble
 , List/build
 , List/fold
-, List/length
 , List/head
-, List/last
 , List/indexed
+, List/last
+, List/length
 , List/reverse
-, Text/show
+, Natural/build
+, Natural/even
+, Natural/fold
+, Natural/isZero
+, Natural/odd
+, Natural/show
+, Natural/subtract
+, Natural/toInteger
 , Text/replace
-, Date/show
+, Text/show
 , Time/show
 , TimeZone/show
 , Bool
-, True
+, Bytes
+, Date
+, Double
 , False
-, Optional
-, None
-, Natural
 , Integer
-, Double
+, List
+, Natural
+, None
+, Optional
 , Text
-, Bytes
-, Date
 , Time
 , TimeZone
-, List
-, Type
-, Kind
-, Sort
+, True
 ]
 
 ---
@@ -76,13 +73,10 @@ All builtins (builtinsA.dhall*)
     (expression (primitive_expression (identifier (builtin))))
     (expression (primitive_expression (identifier (builtin))))
     (expression (primitive_expression (identifier (builtin))))
-    (expression (primitive_expression (boolean_literal)))
-    (expression (primitive_expression (boolean_literal)))
-    (expression (primitive_expression (identifier (builtin))))
-    (expression (primitive_expression (identifier (builtin))))
     (expression (primitive_expression (identifier (builtin))))
     (expression (primitive_expression (identifier (builtin))))
     (expression (primitive_expression (identifier (builtin))))
+    (expression (primitive_expression (boolean_literal)))
     (expression (primitive_expression (identifier (builtin))))
     (expression (primitive_expression (identifier (builtin))))
     (expression (primitive_expression (identifier (builtin))))
@@ -91,7 +85,7 @@ All builtins (builtinsA.dhall*)
     (expression (primitive_expression (identifier (builtin))))
     (expression (primitive_expression (identifier (builtin))))
     (expression (primitive_expression (identifier (builtin))))
-    (expression (primitive_expression (identifier (builtin)))))))
+    (expression (primitive_expression (boolean_literal))))))
 
 ==================
 Bool (unit/BoolA.dhall)
diff --git a/test/corpus/comments.txt b/test/corpus/comments.txt
@@ -26,20 +26,6 @@ Line comment ending in CLRF (lineCommentCRLFA.dhall)
   (line_comment (line_comment_prefix) (line_comment_content))
   (primitive_expression (numeric_literal (natural_literal))))
 
-==================
-Trailing line comments terminate without newline (unit/TrailingLineCommentWithoutNewlineA.dhall)
-==================
-
--- This test verifies that a trailing line comment does not require a newline
-1--
-
----
-
-(expression
-  (line_comment (line_comment_prefix) (line_comment_content))
-  (primitive_expression (numeric_literal (natural_literal)))
-  (line_comment (line_comment_prefix) (line_comment_content)))
-
 ==================
 Block comment (blockCommentA.dhall)
 ==================
@@ -75,12 +61,13 @@ Nested block comments (nestedBlockCommentA.dhall)
   (primitive_expression (numeric_literal (natural_literal))))
 
 ==================
-Mixed comments
+Mixed comments (mixedBlockLineCommentA.dhall)
 ==================
 
 {- foo -}
 -- bar
 {- --baz -}
+-- {- qux -}
 
 1
 
@@ -96,6 +83,7 @@ Mixed comments
     (block_comment_open)
     (block_comment_content)
     (block_comment_close))
+  (line_comment (line_comment_prefix) (line_comment_content))
   (primitive_expression (numeric_literal (natural_literal))))
 
 ==================
diff --git a/test/corpus/functions.txt b/test/corpus/functions.txt
@@ -242,7 +242,7 @@ toMap { foo= 1, bar= 4, baz= 9 }
           (numeric_literal (natural_literal)))))))))
 
 ==================
-showConstructor statement
+showConstructor statement (unit/ShowConstructorA.dhall)
 ==================
 
 showConstructor <A>.A
@@ -257,7 +257,7 @@ showConstructor <A>.A
       (selector (selector_dot) (label)))))
 
 ==================
-showConstructor statement with a value
+showConstructor statement with a value (unit/ShowConstructorWithValueA.dhall)
 ==================
 
 showConstructor (<A : Bool>.A False)
diff --git a/test/corpus/identifiers.txt b/test/corpus/identifiers.txt
@@ -145,7 +145,7 @@ missingFoo
   (primitive_expression (identifier (label))))
 
 ==================
-de Bruijn indices
+de Bruijn indices (unit/DeBruijnIndexA.dhall)
 ==================
 
 x == x@0
diff --git a/test/corpus/lambda.txt b/test/corpus/lambda.txt
@@ -113,7 +113,7 @@ forall(x: T) -> x
     expression: (expression (primitive_expression (identifier (label))))))
 
 ==================
-Unicode forall expression
+Unicode forall expression (unit/ForallUnicodeA.dhall)
 ==================
 
 ∀(x : T) -> y
diff --git a/test/corpus/lists.txt b/test/corpus/lists.txt
@@ -31,7 +31,7 @@ Annotated non-empty list (unit/ListLitNonEmptyAnnotatedA.dhall)
     (primitive_expression (identifier (label))))))
 
 ==================
-Multi-line lists
+Multi-line lists (unit/ListWithNewlineA.dhall)
 ==================
 
 [ [
@@ -68,7 +68,7 @@ Multi-line lists
           (numeric_literal (natural_literal))))))))))
 
 ==================
-Comments in lists
+Comments in lists (listWithCommentA.dhall)
 ==================
 
 [
diff --git a/test/corpus/numbers.txt b/test/corpus/numbers.txt
@@ -192,7 +192,7 @@ NaN
   (primitive_expression (numeric_literal (double_literal))))
 
 ==================
-Integer-valued double literal (unit/DoubleLitSecretelyIntA.dhall)
+Integer-valued double literal (unit/DoubleLitSecretlyIntA.dhall)
 ==================
 
 {-
diff --git a/test/corpus/operators.txt b/test/corpus/operators.txt
@@ -203,7 +203,7 @@ w * x * y * z
     (primitive_expression (identifier (label)))))
 
 ==================
-Combine record types operator (unit/operators/RecursiveRecordMergeA.dhall)
+Combine record types operator (unit/operators/RecursiveRecordTypeMergeA.dhall)
 ==================
 
 x //\\ y
@@ -217,7 +217,7 @@ x //\\ y
     (primitive_expression (identifier (label)))))
 
 ==================
-Combine record types operator associativity (unit/operators/RecursiveRecordMergeAssocA.dhall)
+Combine record types operator associativity (unit/operators/RecursiveRecordTypeMergeAssocA.dhall)
 ==================
 
 w //\\ x //\\ y //\\ z
@@ -271,7 +271,7 @@ w ⩓ x //\\ y ⩓ z
     (primitive_expression (identifier (label)))))
 
 ==================
-Combine records operator (unit/operators/RecursiveRecordTypeMergeA.dhall)
+Combine records operator (unit/operators/RecursiveRecordMergeA.dhall)
 ==================
 
 x /\ y
@@ -285,7 +285,7 @@ x /\ y
     (primitive_expression (identifier (label)))))
 
 ==================
-Combine records operator associativity (unit/operators/RecursiveRecordTypeMergeAssocA.dhall)
+Combine records operator associativity (unit/operators/RecursiveRecordMergeAssocA.dhall)
 ==================
 
 w /\ x /\ y /\ z
@@ -421,7 +421,7 @@ x === y
     (primitive_expression (identifier (label)))))
 
 ==================
-Equivalence operator associativity
+Equivalence operator associativity (unit/operators/EquivalenceAssocA.dhall)
 ==================
 
 w === x === y === z
diff --git a/test/corpus/records.txt b/test/corpus/records.txt
@@ -405,7 +405,7 @@ Empty record literal with a trailing comma (unit/RecordLitEmptyTrailingCommaA.dh
   (primitive_expression (record_literal (empty_record_literal))))
 
 ==================
-Empty record literal with a trailing and leading comma
+Empty record literal with a trailing and leading comma (unit/RecordLitEmptyBothCommasA.dhall)
 ==================
 
 { , =, }
@@ -416,7 +416,7 @@ Empty record literal with a trailing and leading comma
   (primitive_expression (record_literal (empty_record_literal))))
 
 ==================
-Empty record type
+Empty record type (unit/RecordTypeEmptyA.dhall)
 ==================
 
 {}
diff --git a/test/corpus/selectors.txt b/test/corpus/selectors.txt
@@ -6,19 +6,6 @@ r.x
 
 ---
 
-(expression
-  (primitive_expression
-    (identifier (label))
-    (selector (selector_dot) (label))))
-
-==================
-Record field selection (unit/RecordFieldAccessA.dhall)
-==================
-
-r.x
-
----
-
 (expression
   (primitive_expression
     (identifier (label))
@@ -71,19 +58,7 @@ in  Prelude.List.map
       selector: (selector (selector_dot) (label) (selector_dot) (label))))))
 
 ==================
-Field projection (unit/RecordProjectFieldsA.dhall)
-==================
-
-r.{ x, y, z }
-
----
-
-(expression (primitive_expression
-  (identifier (label))
-  (selector (selector_dot) (label_set (label) (label) (label)))))
-
-==================
-Field projection without spaces (unit/ProjectionA.dhall)
+Field projection (unit/ProjectionA.dhall)
 ==================
 
 r.{x, y, z}
diff --git a/test/corpus/strings.txt b/test/corpus/strings.txt
diff --git a/tests.sh b/tests.sh
diff --git a/tree-sitter.json b/tree-sitter.json
