Is there a method to exclude classes by name?

[cubecull]

My current target uses several third party libraries which I'm not interested in, but the tooling seems to be spending most of it's time crunching that data.

I'm seeing thousands of log lines like:

reasonNOTMergeClasses_Q(0x5d6800, 0xb82b58, 'xercesc_2_6::SAXParser').
Concluding factNOTMergeClasses(0x5d6800, 0xb82b58).
reasonNOTMergeClasses_Q(0x5d6800, 0xb82a50, 'xercesc_2_6::AbstractDOMParser').
Concluding factNOTMergeClasses(0x5d6800, 0xb82a50).
reasonNOTMergeClasses_Q(0x5d6800, 0xb8290c, 'DPVS::Object').
Concluding factNOTMergeClasses(0x5d6800, 0xb8290c).
reasonNOTMergeClasses_Q(0x5d6800, 0xb82b58, 'xercesc_2_6::SAXParser').
Concluding factNOTMergeClasses(0x5d6800, 0xb82b58).
reasonNOTMergeClasses_Q(0x5d6800, 0xb82a50, 'xercesc_2_6::AbstractDOMParser').

Is there any way to exclude xerces* et al. from being analysed at all stages?

[sei-eschwartz]

There are a few options. I think the easiest one would be to filter the symbolClass facts for xercesc_2_6 from your facts file.

The other option would be to extract a list of all the addresses that are part of that class (probably by parsing the facts file), and then pass them as arguments to --exclude when re-creating the facts file.

Just to verify, you aren't seeing the same line repeated thousands of times are you?

[cubecull]

Just to verify, you aren't seeing the same line repeated thousands of times are you?

Difficult to say, there is a lot of output, but it does seem to be moving forward:

There are 240,850 known facts.
reasoningLoop: pre-reason sanityChecks
Constraint checks succeeded, proceeding to reason forward!
reasoningLoop: reasonForardAsManyTimesAsPossible
reasonForwardAsManyTimesAsPossible
Starting reasonForward.
Processing trigger fact... findint(0x4ca340, 0x5dd8f0)
Processing trigger fact... factClassSizeGTE(0x5dd8f0, 0x20)
reasonForwardAsManyTimesAsPossible complete.
reasoningLoop: post-reason sanityChecks
Constraint checks succeeded, guess accepted!
reasoningLoop: guess
Starting guess. There are currently 11,249 guesses.

facts + guesses count do increase over time.

I'll see what I can learn from the facts file and what I may be able to filter in there.

I think the easiest one would be to filter the symbolClass facts for xercesc_2_6 from your facts file.

May be worth noting that xerces and one other library seem to make up every single symbolClass fact? Is that normal?

Is there any documentation I can read on the --exclude option?

[sei-eschwartz]

May be worth noting that xerces and one other library seem to make up every single symbolClass fact? Is that normal?

It's not abnormal.

Is there any documentation I can read on the --exclude option?

https://github.com/cmu-sei/pharos/blob/master/share/doc/pharos_options.pod

Apparently it's --exclude-func, not --exclude

[cubecull]

Ah those CLI args were what I was hunting for, thanks!

So I've tried removing all the symbolClass facts, and now I'm getting the following error. Not sure if this is because I've removed all those facts, or if I would have got this error anyway once an analysis with those facts in place got further along.

Fail-Retracting guessedNOTConstructor(0x5e6b00)...
Fail-Retracting factNOTConstructor(0x5e6b00)...
tryBinarySearch completely failed on [0x5e6b00] and will now backtrack to fix an upstream problem.
guess: We have back-tracked to the call of tryBinarySearch(tryConstructor, tryNOTConstructor, [0x5e6b00, 0x483f70, 0x403b30])
Refusing to backtrack into reasoningLoop to fix an upstream problem because backtrackForUpstream/0 is not set.
This likely indicates that there is a problem with the OO rules.
Please report this failure to the Pharos developers!
 [228] prolog_stack:get_prolog_backtrace(100,[frame(228,clause(<clause>(0x55df5d041480),6),_3385522)|_3385510],[goal_term_depth(100)]) at /usr/local/lib/swipl/library/prolog_stack.pl:137
 [227] throw_with_backtrace(error(system_error(upstreamProblem))) at /usr/local/share/pharos/prolog/oorules/util.pl:185
  [26] solve_internal at /usr/local/share/pharos/prolog/oorules/setup.pl:681
  [25] catch(user:solve_internal,_3385746,user:((_3385814=error(resource_error(private_table_space),_3385828)->complain_table_space(ooscript);_3385878=error(resource_error(stack),_3385892)->complain_stack_size(ooscript);true),throw(_3385924))) at /usr/local/lib/swipl/boot/init.pl:562
  [24] solve(ooscript) at /usr/local/share/pharos/prolog/oorules/setup.pl:617
  [23] psolve_no_halt('<garbage_collected>') at /usr/local/share/pharos/prolog/oorules/report.pl:23
  [22] catch(user:psolve_no_halt(stream(<stream>(0x55df5d06d870))),_3386098,user:(print_message(error,_3386164),(globalHalt->halt(1);true))) at /usr/local/lib/swipl/boot/init.pl:562
  [21] catch_with_backtrace('<garbage_collected>','<garbage_collected>','<garbage_collected>') at /usr/local/lib/swipl/boot/init.pl:629
  [20] run_with_backtrace('<garbage_collected>') at /usr/local/bin/ooprolog:177
  [19] <meta call>
  [18] with_output_to(<stream>(0x55df5d1d91f0),run_with_backtrace(psolve_no_halt(stream(<stream>(0x55df5d06d870))))) <foreign>
  [17] setup_call_catcher_cleanup(user:(var('ltrclnt-results-nosemantics-rose.pl')->open_null_stream(<stream>(0x55df5d1d91f0));open('ltrclnt-results-nosemantics-rose.pl',write,<stream>(0x55df5d1d91f0))),user:with_output_to(<stream>(0x55df5d1d91f0),run_with_backtrace(psolve_no_halt(stream(<stream>(0x55df5d06d870))))),_3386516,user:close(<stream>(0x55df5d1d91f0))) at /usr/local/lib/swipl/boot/init.pl:663
  [15] setup_call_catcher_cleanup(user:open('ltrclnt-facts-nosemantics-rose.pl',read,<stream>(0x55df5d06d870)),user:setup_call_cleanup((var('ltrclnt-results-nosemantics-rose.pl')->open_null_stream(<stream>(0x55df5d1d91f0));open('ltrclnt-results-nosemantics-rose.pl',write,<stream>(0x55df5d1d91f0))),with_output_to(<stream>(0x55df5d1d91f0),run_with_backtrace(psolve_no_halt(stream(<stream>(0x55df5d06d870))))),close(<stream>(0x55df5d1d91f0))),_3386726,user:close(<stream>(0x55df5d06d870))) at /usr/local/lib/swipl/boot/init.pl:663
  [12] run([script('/usr/local/bin/ooprolog'),json(_3387004),ground(_3387024),rtti(true),guess(true),config(_3387084),stacklimit(200000000000),tablespace(200000000000),oorulespath(_3387144),halt(true),load_only(false),help(_3387204),facts('ltrclnt-facts-nosemantics-rose.pl'),results('ltrclnt-results-nosemantics-rose.pl'),loglevel(6)]) at /usr/local/bin/ooprolog:235
   [9] catch(user:main(['/usr/local/bin/ooprolog','--facts','ltrclnt-facts-nosemantics-rose.pl','--results','ltrclnt-results-nosemantics-rose.pl','--log-level=6']),_3387328,user:(print_message(error,_3387458),halt(1))) at /usr/local/lib/swipl/boot/init.pl:562
   [7] catch(user:main,_3387532,'$toplevel':true) at /usr/local/lib/swipl/boot/init.pl:562
   [6] catch_with_backtrace('<garbage_collected>','<garbage_collected>','<garbage_collected>') at /usr/local/lib/swipl/boot/init.pl:

[sei-eschwartz]

It's hard to say. If you want to post your facts file and/or executable, I can take a closer look.

[cubecull]

Would be good to try and track down why the crash occurs. The files are chonkers (which probably doesn't help); binary is 11mb, serialized file is 530mb and the facts file is 25mb. Is there an email address I can send to? Appreciate your time.

[sei-eschwartz]

Wow, that is quite large. If you can send the exe and facts file to eschwartz@cert.org, that would be great.

[cubecull]

Files sent, thanks

[sei-eschwartz]

I have downloaded the files, thanks. Can you also provide the command line you used to produce the facts file?

[cubecull]

I've lost the original args due to restarting the container, but I was following the guide for analysing larger files. I think this is what I came up with:

ooanalyzer --serialize=ltrclnt-nosemantics-rose.ser --maximum-memory 64000 --no-semantics --prolog-facts=ltrclnt-facts-nosemantics-rose.pl --threads=30 --per-function-timeout=10000000000000000 ltrclnt.exe

can't remember exactly the value for --per-function-timeout, I think I just spammed zeros.

The serialised file was made with something along these lines:

partition --serialize=ltrclnt.ser --maximum-memory=64000 --no-semantics --partitioner=rose ltrclnt.exe

Then as we discussed I went through and manually removed every symbolClass fact entry from the facts file.

[sei-eschwartz]

That should be close enough

[sei-eschwartz]

Using the facts file that you provided, I end up with:

Guessing factConstructor(0x5e6b00).
reasonNOTConstructor_G(0x5e6b00, 0xb82b8c).
Contradictory information about constructor: factConstructor(0xb82b8c) but reasonNOTConstructor(0xb82b8c)
Guessing factNOTConstructor(0x5e6b00).
reasonNOTConstructor_F(0xb82b8c, 0x5e6b00).
Contradictory information about constructor: factConstructor(0xb82b8c) but reasonNOTConstructor(0xb82b8c)

.idata:00B82B8C ; protected: __thiscall xercesc_2_6::InputSource::InputSource(class xercesc_2_6::MemoryManager * const)
.idata:00B82B8C extrn ??0InputSource@xercesc_2_6@@iae@QAVMemoryManager@1@@z:dword

So 0xb82b8c is obviously a constructor.

From reasonNOTConstructor_G:

    % Since we don't have visibility into VFTable writes from imported constructors and
    % destructors this rule does not apply to imported methods.
    not(symbolClass(Method, _, _, _)),

Oops. So apparently my suggestion to comment out the symbolClass was not so good.

[sei-eschwartz]

Another thing you could try is adding fail, at this line to disable the NOTMergeClasses_Q rule. You would want to use the original facts file with the symbolClass facts