NEW @W-21365643@ - Optimize SFGE Path Expansion Rule Performance

[nikhil-mittal-165]

…mance

• Increase default java_thread_count from 4 to 8 for faster parallel execution
• Reduce default java_thread_timeout from 15 min (900000ms) to 30 sec (30000ms)
• Update Java EnvUtil.java defaults to match TypeScript config.ts defaults
• Both values remain configurable via CLI --sfge-thread-count and --sfge-thread-timeout flags or via engines.sfge section in code-analyzer.yaml

[nikhil-mittal-165]

evidences

timeout =1min
threads = 8
execution time= 12 mins

when
timeout =1min
threads = 2
execution time= 42 mins

when
timeout = 3 min
threads = 8
execution time = 25 mins

[nikhil-mittal-165]

timeout 30 seconds and thread count = 8
execution time is 7.5 minutes
violations : 378

timeout 1 seconds and thread count = 16
execution time is 4 minutes
violations : 35

[namrata111f]

In the PR description we say we are keeping this as 3 mins but are changing this to 30 seconds here?

[nikhil-mittal-165]

updated description

[namrata111f]

1- From what I understand this java_thread_count increase will cause more cpu consumption on the user side. Will this have any memory consumption impact on the user end? Also I am not sure if every user hardware will allow for 8 threads to be spawned, in that case to which number we will fallback to?
2- What is the use of changing java_thread_timeout? Is that to ensure no thread is stuck for more than that time? And if a thread is terminated in between how will it's work be continued later?

[nikhil-mittal-165]

1. the memory consumption will be not very significant as for running sfge already requires significant heap memory
   and 1 thread stack will have 1 MB which should be managable
   JVM can spawn 8 threads in a single core machine as well with some context switching, so this behaviour should not break anything

2. by chabging java_thread_timeout we interrupt threads which go on in exponential path expansion for a single entry point and free up the thread to be used by some other entry point. thread terminated will not affect other timeouts this is the current behaviour as well for 15 min timeout

[namrata111f]

which go on in exponential path expansion for a single entry point and free up the thread to be used by some other entry point

So we are doing this to stop at around 93% of path expansion from single entry point as you had explained? Also I am assuming we would be just missing out on 6-7% of the violation in the remaining path and it's not like the final end vertex needs to be mandatory reached.

[nikhil-mittal-165]

yes 93% is the total overall violations caught, once the thread gets interrupted we discard that entry point altogether and do not check violations, this is the current timeout behaviour which remains unchanged

[namrata111f]

Ack, just one recommendation to test with some other large code bases as well to ensure this 93% theory in 30s of thread timeout hold good there as well.