if the scale_size is true,
the depth is resized with image data and not depth data.
Following is the code snippet:
if self.scale_size:
image = cv2.resize(image, (self.scale_size[0], self.scale_size[1]))
depth = cv2.resize(image, (self.scale_size[0], self.scale_size[1]))