Wrong return type on unsafe use

[mwhicks1]

The following program (run with -alltypes) rewrites callee's return type to _Ptr, but it would seem like it should be _Array_Ptr:

int *callee(int *a) {  // BECOMES _Ptr<int> callee(int *a : itype(_Array_ptr<int>)) {
    int *z = a;  // BECOMES _Array_ptr<int> z =  a;
    z += 2;
    return z; 
} 
void caller() { 
    int *y = malloc(sizeof(int));
    callee(y);
} 

The malloc is creating a pointer to a single thing; the constraint on &&malloc_4 is PTR. But then the constraint forces PTR --> .... --> ARR on the blue edges, which is not satisfiable. As such, somehow y_3 is changed to WILD but callee:a_0 and z_2 themselves are kept as ARR.

I'm not sure why this happens, but here's a theory. See lines 250-282 in Constraints.cpp (16fd8d1) and also 353-370, which check for failed constraints and change solutions. What might be happening is that the change to a WILD for &&malloc_4 flows on the red edges up through y_3 and then stops. But the blue edge from PTR keeps going all the way to <<callee:$ret_1.

[sroy4899]

For an analogous situation, consider the following file: (in the regression tests, these include tests of the form b<number>_...caller..c).

struct np *callee(struct p x, struct p y) {
	//CHECK_NOALL: struct np *sus(struct p x, struct p y) : itype(_Ptr<struct np>) {
	//CHECK_ALL: struct np * sus(struct p x, struct p y) {
  struct np *z = malloc(sizeof(struct np));
	//CHECK_NOALL: _Ptr<struct np> z =  malloc<struct np>(sizeof(struct np));
	//CHECK_ALL:   struct np *z = malloc<struct np>(sizeof(struct np));
  z->x = 1;
  z->x = 2;
  return z;
}
struct np *caller_1() {
	//CHECK_NOALL: _Ptr<struct np> foo(void) {
	//CHECK_ALL: struct np *foo() {
  struct p x, y;
  x.x = 1;
  x.y = 2;
  y.x = 3;
  y.y = 4;
  struct np *z = callee(x, y);
	//CHECK_NOALL: _Ptr<struct np> z =  sus(x, y);
	//CHECK_ALL:   struct np *z = sus(x, y);
  return z;
}
struct np *caller_2() {
	//CHECK: struct np *bar() {
  struct p x, y;
  x.x = 1;
  x.y = 2;
  y.x = 3;
  y.y = 4;
  struct np *z = callee(x, y);
	//CHECK: struct np *z = sus(x, y);
  z += 2;
  return z;
}

Note that caller_1() uses callee() safely: call the function and return it. caller_2(), on the other hand, calls callee() and then adds 2 to the result. With alltypes, adding 2 is an indication to the tool that z is an _Array_ptr. With alltypes on, what I think is happening is that the tool sees the discrepancy of returning a normal ptr vs using it as an array in bar() and makes it WILD. Without alltypes, the z+=2 is marked WILD and the tool concludes that the function sus returns a _Ptr for all intents and purposes.

In the original example, we internally treat the argument to callee as an array pointer inside the function even though what we passed in was a pointer. In this example, we internally treat the return value in callee as a normal pointer, but use it as an array pointer in caller_2. I think both of these issues have the same root cause.

[john-h-kastner]

edit: that's without all types, see below

[mwhicks1]

Looks like it works now!

[john-h-kastner]

[mwhicks1]

Aha, it's still there. So the blue edges go from PTR to ARR.