Enhance test automation to include compilation and check for compilation errors. #1448
Description
#1443 highlights an issue that seems to have been with LSP4IJ that was fortunately resolved. In order to catch an issue like this with our tests we would need to enhance them to include compilation (for example by starting dev mode) and then checking if there are any errors with compiling the application.