Skip to content

Potential unintended shape inconsistency with permute. #183

@ppaleja

Description

@ppaleja

When working on #68, I came across this curiosity when edge testing different autograd calculations. Consider this code:

let a = cx.tensor((2, 1)).set([[1.0], [2.0]]);
let a: GraphTensor = a.expand((2, 2));

Since a is now a view with shape (2, 2), I should be able to permute with

let a = a.permute((1, 0));

or something similar.

But this code fails. I can only permute, by knowing the stride length (which I could only know if I was keeping track of shape tracker, i.e. seems like bad encapsulation)

let a: GraphTensor = a.permute((0, 2, 1));

Not only this, but

// 4. Sum to a scalar loss
let loss = a.sum((0, 1)); // Sum over logical axes 0 and 1

// 5. Autograd
let grads = cx.compile(Autograd::new(orig_a_id, loss), ());
cx.keep_tensors(&grads);
cx.execute();

Will cause an OOB error.

I couldn't find any equivalent pytorch methodology, so I don't know what the convention would be for this (if this behavior is intended), but it seems like a vulnerability to me.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions