[low priority] Inspect dependencies

There is a lot of deps in go.mod which adds a lot of transitive dependencies to projects who import gorgonia/cu (like mine)

Try to inspect for what are these deps needed, and possibly eliminate them to shorten download time.

I would like to take a look at this.