The GradleAnnotationGenerator project is the data generation part of an interactive build and dependency visualization and analytics utility for Gradle-builds, based on ElectricInsight. This extends or complements the functionality provided by the existing Gradle --profile and HTML dependency report features - enabling quick visual overview of build performance as well as ability for an engineer to really zoom in and dig deep in order to interactively understand structural behaviour and dependency relationships, allowing for troubleshooting and performance optimization at millisecond resolution.
Below is a sample output from this project - a screenshot of the parallel Gradle-build of Gradle itself where the Y-axis represents concurrent threads, the X-axis represents time and each box represents the workload (sections of a task or a test) as individual jobs. Jobs are categorized by type which is visualized by the colour-coding. More details in the example section below:
ElectricInsight is a powerful tool to visually depict the structure of a software build, down to the file level - empowering software and build engineers to easily pinpoint performance problems and conflicts in a build. The default usage of ElectricInsight is as an add-on to ElectricAccelerator, mining the information produced by ElectricAccelerator to provide an easy-to-understand, graphical representation of the build structure for performance analysis. It provides detailed information and reports on each parallel worker of the build infrastructure - for at-a-glance visualization, analytics and diagnostics. It can also predict and model how build times would be impacted by adding additional build infrastructure, to help guide hardware investment decisions.
- ElectricInsight, which comes bundled with the freely available ElectricAccelerator Developer Edition.
- A gradle build running in your environment, or any of the annotations from this repository
- Make sure you meet the prerequisites
- Retrieve the Gradle init-script from this project
- Add the --init-scipt/-I flag to load the initialization script when running your gradle build:
<command-to-run-your-regular-gradle-build> --init-script <path-to-GradleAnnotationGenerator-init-script> <gradle-tasks> - Open gradle.anno (the generated Gradle annotation file) with ElectricInsight:
einsight gradle.anno - From here you can interactively explore the ElectricInsight reporting and analysis engine for your gradle build, see some example findings and use-cases below.
If you lack a Gradle build environment but want to give this a try, go ahead and download some pre-existing annotation files here.
While working on this project I have used four gradle builds as my testbed and benchmark - gradle, hibernate-orm, spring-framework and griffon. Interestingly, they all exhibit different behaviours and constraints with different workload dominating the total time by type - test execution (gradle), code analysis (hibernate-orm), Javadoc generation (spring-framework) and Java compiles (griffon).
Below are further details and screenshots from my explorations. The following command-line were used for building all projects:
./gradlew --parallel --init-script <path-to-GradleAnnotationGenerator-init-script> clean build
The gradle build is interesting because the workload is heavily dominated by testing, which is forked and hence triggers a large amount of parallelization. Interestingly though, build time is not bounded by the test workload and we can easily spot the :docs:dslHtml as being the likely long pole. Using the Longest Serial Chain report we can also confirm that being the case, informing us it would be worthwhile to try and move the :docs project up earlier in the execution graph.
Legend: "Library Link"=>"JUnit/TestNG", "Noop"=>"Non-classified", "Miscellaneous"=>"Gradle Overhead", "Java Compile", "Compile"=>"Codenarc/Checkstyle/Findbugs", "Code gen"=>"Javadoc"
ElectricInsight can visually highlight all dependent and waiting jobs from any given job, and allow for interactive traversal through the dependency/execution graph. Red emphasis in below screenshot indicates all waiting jobs from the "Graph Populated" job.
The Longest Serial Chain report is using dependency and runtime information to determine and visualize the chain of jobs included in the longest serial chain. Note that in this current implementation where there are no available dependency information between Gradle TestDescriptor objects, all test suites and test cases within the suites are treated as non-dependent, hence parallelizable.
The ElectricSimulator lets you visualize how the build time would change given a varying amount of build infrastructure.
During my testing I have built the Gradle 1.9-rc-2 codebase which was failing in my environment due to some gcc test problems. Clicking on the binoculars with a red cross gives a quick listing of all the failed tests:
The first half of the hibernate-orm build is dominated by parallel FindBugs workload, while the later half flattens out into a string of serial test workload.
Legend: "Library Link"=>"JUnit/TestNG", "Noop"=>"Non-classified", "Miscellaneous"=>"Gradle Overhead", "Java Compile", "Compile"=>"Codenarc/Checkstyle/Findbugs", "Code gen"=>"Javadoc"
The spring-framework build is dominated by parallel javadocs workload, where one of those jobs happens to fail in my environment.
Legend: "Library Link"=>"JUnit/TestNG", "Noop"=>"Non-classified", "Miscellaneous"=>"Gradle Overhead", "Java Compile", "Compile"=>"Codenarc/Checkstyle/Findbugs", "Code gen"=>"Javadoc"
Let's use ElectricInsight to find out more about the failing javadoc-job. Clicking on the binoculars with a red cross gives a quick listing of all the failed tests, then clicking on that top job brings up a view with further details where I can see e.g. start/end-time, thread and also the exception that triggered the failure.
Moving on to the Annotation tab gives even more details such as e.g. the job output and also all the read/written files.
The griffon build is dominated by Java/Groovy compile time and codenarc analysis workload. It's a short build but still has quite a bit of serialization, so it seems could still be further shortened by more aggressive parallelization.
Legend: "Library Link"=>"JUnit/TestNG", "Noop"=>"Non-classified", "Miscellaneous"=>"Gradle Overhead", "Java Compile", "Compile"=>"Codenarc/Checkstyle/Findbugs", "Code gen"=>"Javadoc"
I think this project may be pretty useful as-is (it was certainly useful for me as a project to learn more about Gradle! :-)), but there are obviously a number of interesting extensions/improvements that could be considered fur the future. Off the top of my head, here are some:
-
Thread Allocation
Rather than relying on a custom algorithm for determining which thread a particular job is running on (which turned out to be fairly complex, and by no means is perfect), explore extending the Gradle Task/Test API's with information about which worker/thread is currently in use. This would yield a true representation of how the workload is scheduled and allocated on the build infrastructure.
-
Capturing stdout/stderr per job
It seems Gradle has no current ability to capture stdout/stderr per task/test, at least not that I've been able to figure out. If you register an outputlistener of some kind, all output at the specified loglevel will be captured regardless of origin. It would obviously be very beneficial to have capability to separate and identify source of the output.
-
Live monitoring
ElectricInsight supports live monitoring of builds. Rather than write data to file it would not be too complex to write it to a socket connected to ElectricInsight. This would allow one to look at this project as a real time build monitor as well as a post-build analysis utility.
-
Custom job categorization
It would be a nice feature to have ability to customize ElectricInsight job categorization.
-
Improve flexibility of usage, e.g. by allowing for custom generated annotation-files and for varying amount of annotation detail.
Please take note this proejct is not commercially supported by Electric Cloud. Any issues or problems are best discussed at http://ask.electric-cloud.com or by contacting me directly on email/twitter. Contributions are very welcome - please let me know if you have an interest to help.
Lastly, I'm very interested in hearing your feedback! Email: drosen@electric-cloud.com Twitter: @adr0sen