Parse pallene types file

[igrvlhlb]

This pull request introduces support for Pallene type declaration files (.d.pln), allowing the compiler to parse and verify/typecheck type files in addition to regular Pallene source files. The changes include updates to the parser, typechecker, driver, CLI, and utility functions to handle the new file format, as well as new tests to verify correct parsing and type checking of type declaration files.

Type Declaration File Support

• Added a new AST union TypeFile in src/pallene/ast.lua to represent type declaration files and their components (Typealias, Record, Decl).
• Implemented parsing logic for .d.pln files in src/pallene/parser.lua, including the TypeDeclarationFile parser method and extension-based dispatch in the main parse function. [1] [2]
• Extended the typechecker to handle type declaration file ASTs, adding the check_type_file method in src/pallene/typechecker.lua. [1] [2]

Compiler and CLI Integration (Extra)

• Added a new CLI flag --emit-types to src/pallene/pallenec.lua, enabling users to generate .d.pln files, and integrated it into the main command dispatch. [1] [2]

Utilities and Testing

• Introduced utility functions in src/pallene/util.lua to correctly split and recognize multi-part extensions like .d.pln, supporting robust file name handling.
• Added tests for parsing and type checking type declaration files in spec/parser_spec.lua.

[hugomg]

If we encounter an invalid token, the while loop will exit and this function will return as if there was not a problem.

1. I think the while should be "while not end-of-file"
2. Please create test cases for the "expected a type declaration" errors

[igrvlhlb]

I think the while should be "while not end-of-file"

Should we still throw an error if we don't find at least one declaration? If so, we can go with a repeat-until

[hugomg]

We should allow empty files, in case the module exports nothing. (Also test that...)

[igrvlhlb]

I tried to write the error test in a assert_program_error() fashion, but I couldn't get it to work. So I wrote it another way (src/spec/parser_spec.lua:979-990)

[hugomg]

This kind of if-else-if doesn't feel right. The type (ast.Program) should be the same in all branches.

I suspect that you actually want to have separate type checking functions instead of a catch-all typechecker.check

[igrvlhlb]

The same goes to Parser.parse()? Or is it ok for that case?

If I understand it correctly, you are suggesting that we have a new function in the typechecker and the responsibility to choose which one to call must be in driver.compile_internal(). Right?

[hugomg]

Yes.

[hugomg]

I wonder if it would be simpler to not split the string. We can could the string itself to see if it ends in ".pln" or ".d.pln".

[igrvlhlb]

Right. Also now I see that logic is not necessary, as "." is not an allowed character in file names.

So what I plan to do is this:

• get rid of that new function util.split_pallene_ext() (and split_all_ext)
• keep using util.split_ext with the following modification

 function util.split_ext(file_name)
-    local name, ext = string.match(file_name, "(.*)%.(.*)")
+    local name, ext = string.match(file_name, "(.-)%.(.*)")
     return name, ext
 end

That way string.match() will match name only until the first '.', and ext will be the rest of the string. Is that ok?

[hugomg]

Acho que precisamos de funções separadas para o compile_internal e o parser.parse.

Pense em qual seria o tipo de parser.parse. Hoje é uma função que as vezes retorna um ast.Program e as vezes retorna um ast.TypeFile. Isso até dá pra escrever em Lua, que é uma linguagem dinâmica, mas não funcionaria se quisessemos converter esse código para Pallene no futuro.

Um problema semelhante acontece na compile_internal, que hoje está com um if que verifica a extensão e retorna tipos de dados diferentes dependendo do valor da extensão.