Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow batch_size > 1 in quantized RoIAlign #3655

Open
NicolasHug opened this issue Apr 9, 2021 · 0 comments
Open

Allow batch_size > 1 in quantized RoIAlign #3655

NicolasHug opened this issue Apr 9, 2021 · 0 comments

Comments

@NicolasHug
Copy link
Member

#3624 was recently merged and we identified a potential issue: #3624 (comment)

In short, the rois tensor contains indices in the first column, but depending on the quantization, some indices cannot be properly represented. For example uneven numbers can't be represented if the tensor was quantized with qscale = 2.

To prevent any potential bug, we currently force the batch size to be 1 and hard-code the index to 0:

TORCH_CHECK(
input.size(0) == 1,
"Only one image per batch is allowed in roi_align when quantized tensors are passed.");

// FIXME: change this when batches of size > 1 are allowed
const int roi_batch_ind = 0;

We should try to allow more than one element per batch. A potential solution would involve using per-channel quantized tensors for the roi tensor, where the first column containing the indices would be quantized in a different way from the rest of the columns.

In roi_align python op:

  • if a tensor with 5 columns is passed, raise an error if it's not per-channel: there's a high changes the indices are wrong and it's too risky. If the tensor is per-chanel, pass it through: we can assume that the user knows what they're doing and that the indices are properly represented. As a good sanity check, we can still check that the batch size is within the range of the quantized type of the first column.
  • if a list of tensors is passed, convert that list of tensors into a per-channel quantized tensor with 5 columns.

The convert_boxes_to_roi_format utils should be modified. To ensure consistency throughout the library, it should also be used in MultiScaleRoIAlign.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants