What is a Tensor in Deep Learning? | Maxime Labonne

[utterances-bot]

What is a Tensor in Deep Learning? | Maxime Labonne

A definition in simple terms with examples

https://mlabonne.github.io/blog/what-is-a-tensor/

[hideousparadigm]

Hy, thanks for the intro. I had a play with this and it is a slighty unfair test as the default dtype in tensor.rand() is float32, wheras numpy uses numpy.float64.
I copied the tensor data to numpy arrays and repeated;
array1 = tensor1.cpu().detach().numpy()
array2 = tensor2.cpu().detach().numpy()
and the function clocked at exactly half the time. My i5-4670 took 360 ms (instead of 706ms using float64 values), versus my RTX3060 (4.89 ms) == 74x speed for the fair comparison (that is a 10 year old cpu though:)

Also, using tensors on the CPU, instead of GPU, makeas a more fair test;
device = torch.device("cpu")
And I got 156ms, which makes tensor 32x faster than equivalent numpy. This just suggests to me that numpy arrays carry a real overhead. Surely there is a way to operate on arrays in python much better than this without using tensors??

Anyway, it was fun watching nvtop as my new gpu was maxed out for a few seconds.
Thanks!

[mlabonne]

Hi @hideousparadigm, thanks for your insightful comment, I didn't think about checking that.

I think the float32 vs. float64 is particularly relevant. Storing the tensors on a GPU might not be fair, but this is their main feature and it would be a shame not to use it. However, it's good to know that tensors are still faster than numpy arrays.

Thanks a lot, and I hope you'll upgrade your poor CPU soon! :)