General questions around generating a cpg for joern

[warsang]

Hi!

I'm trying to create a cpg to later analyze with joern. One thing I've noticed with joern is that it struggles with strongly templated C++ code and macros. I'm thinking I'll likely get a much better CPG if I generate it from the llvm IR outputed by a clang compiler on a Cpp project. It seems like this repo can help generate that cpg?

Unfortunately, I'm not a very good Java/Kotlin dev and therefore, wasn't super sure what the steps I'd need to take to achieve this be.

My current understanding is that I'd have to write my own cli in Kotlin and add the following gradle/maven imports:

    implementation("de.fraunhofer.aisec", "cpg-core", cpgVersion)
    implementation("de.fraunhofer.aisec", "cpg-langugage-cxx", cpgVersion)

or maybe import cpg-langugage-llvm instead? Will generating the cpg from llvm ir provide a more complete CPG since template expansion should have been handled by clang when generating the corresponding llvm IR?

Once I have my CLI, I can point it to my project's folder containing either the cpp or llvm IR files ; that will generate a CPG at runtime; Is there an easy way I can export the cpg to a cpg.bin.zip so I can use it with joern later?

I'm not hard set on using joern if I can run the same queries directly on the cpg outputted by this lib instead using neo4j or codyze. I mostly wanted to benefit from the existing joern documentation and existing queries I could use to build my own.

Thanks in advance!

[KuechA]

Hi @warsang ,

first of all, sorry for the late reply, we haven't checked the discussions page for a while.

I'm afraid you're currently mixing this CPG implementation and Joern which are not interoperable and provide distinct features. In particular, you cannot run Joern queries on this CPG neither am I aware of an export to cpg.bin.zip (there's an export to json but we cannot import it yet, so it doesn't fit to your need, there's ongoing development on this). Let me try to help you nevertheless with your matter since I believe it may be worth looking at our implementation nevertheless :)

Frist, if you have a directory with code and want to run some queries on the code I see two options: 1) using the project codyze (which is part of this directory) or 2) writing an own analyzer (similar to codyze).

For 1), I'd refer to the documentation https://fraunhofer-aisec.github.io/cpg/GettingStarted/codyze/ and add that you basically can specify your queries in kotlin scripts with the ending query.kts and run codyze scan. For an example on the directory structure, please check https://github.com/Fraunhofer-AISEC/cpg/tree/main/codyze-compliance/src/integrationTest/resources/demo-app.

For 2) you have to create a new project with the gradle imports you already mentioned above. You can probably inspire yourself with the implementation of the codyze and neo4j modules. The main steps in the code are described here: https://fraunhofer-aisec.github.io/cpg/GettingStarted/library/ and include configuring the translation and providing the path to the code directory and finally building the CPG. On this representation, you can run queries (the same ones you would write for codyze). This approach is probably more suitable for debugging and manually interacting with the graph as compared to using codyze.

For writing the queries, please check the following two documentation sites: https://fraunhofer-aisec.github.io/cpg/GettingStarted/shortcuts/ and https://fraunhofer-aisec.github.io/cpg/GettingStarted/query/ which should provide you with a starting point.

You have an interesting question if the LLVM-IR output is easier to analyze compared to cpp-code with templates. My guess would be that it may be a bit better for this but there may be other downsides since the support for analyses on LLVM-IR may be currently limited (should mainly require extending some implementation details with moderate effort). I'm curious about your experience!

[warsang]

Thanks for the input! Super helpful stuff!

I'm afraid you're currently mixing this CPG implementation and Joern which are not interoperable and provide distinct features. In particular, you cannot run Joern queries on this CPG neither am I aware of an export to cpg.bin.zip (there's an export to json but we cannot import it yet, so it doesn't fit to your need, there's ongoing development on this). Let me try to help you nevertheless with your matter since I believe it may be worth looking at our implementation nevertheless :)

Joern has a way to export to cpg.bin.zip but I'm guessing their spec is fairly different from the one in this project.

there's an export to json

Is that in joern or this cpg library? That may be an option? CPG to json and then json to Joern CPG? Assuming there's no data loss when exporting to json, I'd have to compare both cpg specs to better understand how they differ and see if it wouldn't be too much work to convert from one to the other?

I'll try and get the codyze approach you described to run on cpp and see how that ends up going. Thanks for all the info!

[KuechA]

Yes, the spec is quite different, so also exporting the json of this CPG and generating the Joern CPG from this input is probably not really a good idea and is likely to be quite a substantial amount of work. So, instead of these workarounds, I'd just give our tooling a try. If you need support writing queries or custom passes for your use-cases, let us know! :)